# Presentation: 051946

## Improving Peer  Review 

**Improving Peer  Review**** **
***CSR Initiatives***
***toni scarpa                                                            ******scarpat@csr.nih.gov***
***	******	******	******	******	******	******301-435-1109***

**PRAC  **
**Nov. 3, 2008**

**National Institutes of Health****U.S. Department of Health and Human**** ****Services**

## New Data and the Drivers for Change

CSR’s Efforts to Enhance Peer Review

The NIH Director’s Peer Review Initiatives

**New Data and the Drivers for Change**
**CSR’s Efforts to Enhance Peer Review**
**The NIH Director’s Peer Review Initiatives**

**Improving Peer  Review**

## New Data and the Drivers for Change

## CSR Peer Review: 2008

**77,000 applications received**
**56,000 applications reviewed **
**16,000 reviewers**
**240 Scientific Review Officers**
**1,800 review meetings**

## 1st Driver: The NIH Budget 

**$13.7**

**$15.6**

**$17.8**

**$20.5**

**$23.3**

**$27.1**

**$28.0**

**$28.6**

**$28.6**

**$29.1**

**$29.5**

**$29.5**

- $0

- $5

- $10

- $15

- $20

- $25

- $30

- 1998

- 1999

- 2000

- 2001

- 2002

- 2003

- 2004

- 2005

- 2006

- 2007

- 2008

- 2009

**1st Driver:**** The NIH Budget **

- Doubling

## 2nd Driver: Number of Applications Submitted

**2****nd**** Driver: ****Number of Applications Submitted**

**Historical Growth**

## SBIR  Applications

**SBIR  Applications**

- 7000

- 6000

- 5000

- 4000

- 3000

- 2000

- 1000

- 0

- 1999

- 2000

- 2001

- 2002

- 2003

- 2004

- 2005

- 2006

- 2007

**INST**
**CSR**

## R01 Applications Reviewed at CSR vs. Other ICs

## R21 Applications Reviewed at CSR vs. Other ICs

##     3rd Driver: Reviewer’s Load

**    ****3****rd**** Driver: ****Reviewer’s Load**

- Applications Per Reviewer

**October Council Rounds**

## 4th Driver: CSR Budget

**$ Millions **

## 4th Driver: CSR Budget

**$ Millions **

## Annual Savings in Reviewers’ Expenses Budget

**Non-refundable tickets with one possible change**
**$15 million**
**3,000 fewer reviewers**
**$3 million**
**15% reviews using electronic platforms**
**$5 million **
**One meeting a year on the West Coast**
**$1.8 million **

## Next Savings?

**Replace shipping of CDs with online downloading**
**Cost of 18,000 CDs                      $750,000**
**Cost of shipping****	****	****$200,000**
**Handling****	****	****	****  $50,000**
**5% more reviews using electronic platform**
**SREA budget****	****	****          $1,500,000**
**CSR budget****	****	****            -$100,000**
**More meetings in Baltimore**
**Savings/100 meetings****	****	****$500,000**
**Review of fellowship applications**

## CSR’s Efforts to Enhance Peer Review

## Reorganizing CSR’s Review Groups

Recruiting CSR Staff

Shortening the Review Cycle

Holding Out-of-Town Meetings

Recruiting the Best Reviewers

Performing a Second Cycle of  IRG Reviews

Revising Study Section Guidelines

Training Chairs

Managing Conflicts of Interest

Advancing Additional Review Platforms and Processes

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town Meetings**
**Recruiting the Best Reviewers**
**Performing a Second Cycle of  IRG Reviews**
**Revising Study Section Guidelines**
**Training Chairs**
**Managing Conflicts of Interest**
**Advancing Additional Review Platforms and Processes**

**CSR’s Efforts to Enhance Peer Review**

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**

## 1.  CSR Organization of Peer Review 

**Divisions**

**Scientific Review Officers**

**Integrated Review Groups**

## 1. CSR Reorganization

**Rene**
**   ****Etcheberrigaray ****  **

**Don**
**Schneider**

## Update on CSR Realignments	Don Schneider

***Update on CSR Realignments***
***	******Don Schneider***

**1.  CSR Organization of Peer Review **

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**

## 2. Recruiting CSR Staff

**3 Division Directors**
**6 Integrated Review Chiefs**
**20 Scientific Review Officers**

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**

## 3. Shortening the Review Cycle    The Goal

**To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past.**

## RO1 A1 Resubmission Within 4 Months of Original Application

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town-Meetings**

## 4. Holding Out-of-Town Meetings

**To facilitate participation from colleagues on the West Coast, one meeting out of three each year will be held in the metro areas of:**
**Los Angeles**
**San Francisco**
**Seattle**
**Chicago**
**Benefits**
**Great satisfaction**
**Significant savings**
**Ranking of hotels**

## 5. Near-Term Solutions for Recruiting and Retaining the Best Reviewers

**	**
**Move a meeting a year to the West Coast**
**Develop a national registry of volunteer reviewers**
- Searchable database with 3,500 reviewers
**Provide tangible rewards for reviewers**
- No submission deadlines for chartered members
-  of study sections (effective February 2008).
- 1,574 chartered members used flexible deadlines
-  during the last 6 months
**Provide flexible time for  reviewers**
- Choice of 3 times/year for 4 years or
- 2 times/year for 6 years

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Out-of-Town Meetings**
**Recruiting the Best Reviewers**
**Performing a Second Cycle of IRG Reviews**

## Slide 30

| 6.  Second Cycle of IRG Reviews |  |
| --- | --- |
| Scheduled 2008-5 IRGs | Scheduled 2009-12 IRGs (Continued) |
| Brain Disorders and Clinical Neuroscience (BDCN) and Molecular, Cellular and Developmental Neuroscience (MDCN) | Bioengineering Sciences and Technologies (BST) andSurgical Sciences, Biomedical Imaging and Bioengineering (SBIB) |
| Biology of Development and Aging (BDA), 
Emerging Technologies and Training in Neuroscience (ETTN) andIntegrative, Functional and Cognitive Neuroscience (IFCN) | AIDS and Related Research (AARR) and
Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) |
| Scheduled 2009-12 IRGs | Scheduled 2010-8 IRGs |
| Biobehavioral and Behavioral Processes (BBB) and
Risk, Prevention and Health Behavior (RPHB) | Vascular and Hematology (VH) and
Cardiovascular and Respiratory Sciences (CVR) |
| Healthcare Delivery and Methodologies (HDM) and
Epidemiology and Population Sciences (EPS) | Musculoskeletal, Oral and Skin Sciences (MOSS) and 
Digestive, Kidney and Urological Systems (DKUS) |
| Immunology (IMM) and 
Infectious Diseases and Microbiology (IDM) | Genes, Genomes and Genetics (GGG) and
Biological, Chemical and Macromolecular Biophysics (BCMB) |
| Oncology 1—Basic and Translational (OBT) and
Oncology 2—Translational Clinical (OTC) | Interdisciplinary Molecular Sciences and Training (IMST) and 
Cell Biology (CB) |

## 6.  Performing a Second Cycle of IRG Reviews

**All pertinent study section chairs were invited**
**Advice on study section and IRG**
**Revision of study sections description/outline**
**Training of the chair**

## 6.  Performing a Second Cycle of IRG Reviews

***Second Cycle of IRG Reviews ***
***	******Don Schneider***

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town Meetings**
**Recruiting the Best Reviewers**
**Performing a Second Cycle of IRG Reviews**
**Revising Study Section Guidelines**

## 7. Revising Study Section Outlines

**Cellular Signaling and Regulatory Systems**
**[Roster]**
**The Cellular Signaling and Regulatory Systems (CSRS) study section reviews applications that focus on the initiation and execution of programs that control cellular homeostasis and physiology.  A distinguishing characteristic of these applications is an emphasis on signaling networks and the coordination of processes related to cell proliferation, survival, and growth.**
**Cell cycle regulation, mitosis, meiosis, checkpoint controls and regulation by ubiquitination**
**Proteolytic mechanisms associated with cell cycle, senescence and death**
**Programmed cell death and apoptosis, particularly their regulation in the context of stress, growth, and transformation.  **
**Proliferation and growth control by the nucleus; signaling pathways regulating transcription**
**Integrative cell physiology, e.g., stress, clocks, cellular modeling; cell differentiation and transformation**
**Basic studies of cytokine signaling**
**Application of state-of-the-art technologies such as imaging and computational modeling of cellular signaling networks**
**Study sections with most closely related areas of similar science listed in rank order are:**
**Molecular and Integrative Signal Transduction**
**Intercellular Interactions**
**Membrane Biology and Protein Processing**
**Molecular Genetics A**
**Molecular Genetics B**

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town Meetings**
**Recruiting the Best Reviewers**
**Performing a Second Cycle of IRG Reviews**
**Revising Study Section Guidelines**
**Training Chairs**

## 8. Training Study Section Chairs

**Scope and Schedule**** **
**120 new chairs every year, more if “permanent” SEPs chairs are included**
**5 meetings in January 2009 for chairs recently appointed**
**5 meetings in July 2009 for chairs appointed next June**
**Training Program**
**Share data **
**Explain the new changes and the significance**
**Share the best practice (ours and theirs)**
**Answer questions and address concerns**
**Make Chairs more effective stakeholders**

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town Meetings**
**Recruiting the Best Reviewers**
**Performing a Second Cycle of IRG Reviews**
**Revised Study Section Guidelines**
**Training of Chairs**
**Managing Conflicts of Interest**

## 9. Managing Conflicts of Interest

***Report of Conflict of Interest Committee***
***Noni Byrnes***

## CSR’s Efforts to Enhance Peer Review

**Reorganizing CSR’s Review Groups**
**Recruiting CSR Staff**
**Shortening the Review Cycle**
**Holding Out-of-Town Meetings**
**Recruiting Reviewers**
**Performing a Second Cycle of IRG Review**
**Revising Study Section Guidelines**
**Training of Chairs**
**Managing Conflicts of interest**
**Advancing Additional Review Platforms and Processes**

## 10. Advancing Additional Review Platforms and Process

**Video (VED)**
**Internet (AED)**
**Editorial Board**
**Order of Discussion **

## 10. Advancing Additional Review Platforms and Processes

**Editorial Board Review**
- Heavy triage based on innovation and potential science transformation by a small study section of distinguished, broad-science reviewers (**the editors**)
- Specific science reviewed by appropriate reviewers (**the editorial board**)
- Final ranking by the editors

## 10. Advancing Additional Review Platforms and Process

***New Transformative R01***
***Betsy Wilder***

## 10. Advancing Additional Review Platforms and Process

***Results of Asynchronous ***
***Electronic Discussion Survey***
***	******	******Andrea Kopstein***

## NIH Enhancement of Peer Review

**The Charge from Dr. Zerhouni:**** **
***	******“******Fund the best science, by the best scientists, ***
***	******  ******with the least administrative burden...”***

**http://enhancing-peer-review.nih.gov**

**Two advisory committees to the NIH Director**

## The NIH Director’s Peer Review Initiatives

## The NIH Director’s Peer Review Initiatives

**Corporate NIH Peer Review Enhancement:  Implementation**
**	****Raynard Kington**

## This is CSR

**September 2008**

## Recognizing Outstanding Reviewers

**Going above and beyond the call of duty**
**Having a significant impact on the quality of peer review**
**Educating and inspiring other reviewers**
**Setting the highest standard  of peer review excellence **

## The First Marcy Speer Award, 2007 

**Marcy Speer**

## 2008 Marcy Speer Outstanding CSR Reviewer Award 

## The 2nd Marcy Speer Award, 2008