# Presentation: 045240

##                                    CSR   Where we are and where we would like to be  

**National Institutes of Health**
**Department of Health and Human Services**

**Toni Scarpa**

**                                   ****CSR****   ****Where we are and where we would like to be**** **** **

**PRAC, August 2006**

## Changes in CSR Operations   Our Vision for Peer Review

## This is CSR

## Changes in CSR Operations

**Increased Communication and Transparency**
**Increase Uniformity**
**Increase Efficiency **

**Changes in CSR Operations**

## Changes in CSR Operations

**1.  Increased Communication and Transparency**
**     ****Within CSR**
**Biweekly meetings with SRAs and other staff**
**New comprehensive communications plan**
**     ****With NIH and other Agencies**
**     ****With the Scientific Community**
**Expanded Peer Review Notes newsletter**
**Six Open House Meetings with leaders of scientific societies and disease groups**

## Changes in CSR Operations

**2.  Increase Uniformity**
**     ****Summary Statements **
**Post all within 1 month of meeting ****(97.3%)**
**Post new investigator summary statements within 1 week**
**Produce complete and structured resumes**
**     ****Unscoring**
**Unscore 50% ****(49.7%)**
**    ****Best Practices Committee Assessments**
**Appropriate Rosters**
**Types of Meetings**
**Structure of Summary Statements**

## Changes in CSR Operations

**3. Increase Efficiency **
**     ****Electronic Submission**
**     ****Text Fingerprinting, Artificial Intelligence Software**
**Assigning applications to Integrated Review Groups or Study Sections**
**    ****Major pilot to directly assign applications to Integrated**
**    ****Review Groups and/or Study Sections will begin in**
**    ****October, with implementation by February 2007.**

## Scoring – Current Situation

**Compression of scoring**
**“****Approach is King” and “Significance” has been downgraded**
**Reasons for scoring often not clear in summary statements**
**Scoring of A2 applications may be inappropriately influenced by reviewers knowing A1 score**

## IRG and Study Sections Realignment

**Increasing concern from applicants, societies and diseases groups about the variance in the breadth of science covered by study sections.**

## Monitoring IRGs and Study Sections

**Every IRG is being assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders.  **
**Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. **
**Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee.**
**Most study sections visited by CSR’s Director and senior staff at least once a year. **
**All retiring study section chairs called by CSR’s Director to learn about problems and possible improvements. **
**All summary statements  read by CSR’s Director.**
**Multiple outreach efforts now gather fresh input.**

## IRG Review Schedule

| Scheduled 2007 -- 9 IRGs |
| --- |
| Biology of Development and Aging (BDA) |
| Infectious Diseases and Microbiology (IDM) |
| Biobehavioral and Behavioral Processes (BBBP) |
| Cell Biology (CB) |
| Musculoskeletal, Oral and Skin Sciences (MOSS) |
| Oncological Sciences (ONC) |
| Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) |
| Respiratory Sciences (RES) |
| Renal and Urological Sciences (RUS) |

| Scheduled 2006 --14 IRGs |
| --- |
| Biological Chemistry and Macromolecular Biophysics (BCMB) |
| Cardiovascular Science (CVS) |
| Bioengineering Sciences and Technologies (BST) |
| AIDS and Related Research (AARR) |
| Risk, Prevention, and Health Behavior (RPHB) |
| Genes, Genomes and Genetics (GGG) |
| Digestive Sciences (DIG) |
| Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) |
| Brain Disorders and Clinical Neuroscience (BDCN) |
| Integrative, Functional and Cognitive Neuroscience (IFCN) |
| Molecular, Cellular and Developmental Neuroscience (MDCN) |
| Hematology (HEME) |
| Immunology (IMM) |
| Health of the Population (HOP) |

## Monitoring IRGs and Study Sections

**Every IRG is being assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders.  **
**Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. **
**Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee.**
**Most study sections visited by CSR’s Director, Deputy Director, and senior staff at least once a year. **
**All retiring study section chairs called to learn about problems and possible improvements. **
**Multiple outreach efforts now gather fresh input.**
**Open house meetings to be held.**

## Broad Scientific Areas

- Biomolecular (4): Biological Chemistry and Macromolecular Biophysics (BCMB); Bioengineering Sciences and Technologies (BST); Cell Biology (CB); Genes, Genomes and Genetics (GGG)
- Integrated Biological (5): Immunology (IMM); Hematology (HEME); Cardiovascular Sciences (CVS); Respiratory Sciences (RES); Biology of Development and Aging (BDA)
- Integrated Biological (4): Digestive Sciences (DIG); Musculoskeletal, Oral and Skin Sciences (MOSS); Renal and Urological Sciences (RUS) Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR)
- Disease-based (4): AIDS and Related Research (AARR); Infectious Diseases and Microbiology (IDM); Oncological Sciences (ONC);  Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB)
- Neurological (3): Brain Disorders and Clinical Neuroscience (BDCN); Integrative, Functional and Cognitive Neuroscience (IFCN); Molecular, Cellular and Developmental Neuroscience (MDCN)
- Behavioral/Social (3): Biobehavioral and Behavioral Processes (BBBP); Health of the Population (HOP); Risk Prevention and Health Behavior (RPHB)

## Our Agenda

**Shorten the review cycle**
**Address concern that clinical research is not properly evaluated**
**Improve the assessment of innovative, high- risk/high-reward research**
**Do more to recruit and retain more high-quality reviewers**

## Our Agenda

**Shorten the review cycle**

## Shortening the NIH Review Cycle, Initial Steps

**Pilot study with ~600 new investigators in 40 study sections who may revise and resubmit for the very next review cycle 4 months earlier than before **
**	****(Effective Feb 06)**
**Summary statements posted within one month after the study section meeting **

## Short Review Cycle Pilot of New Investigator R01 Applications

| 
New Investigator R01 Applications | 
  Number | 
Percent |
| --- | --- | --- |
| Reviewed in Pilot  | 631 | 100% |
| Amended and Submitted for the Next Round | 86 | 14% |

## R01 A1 Resubmissions in the Next Round 

**R01 A1 Resubmissions in the Next Round **

## Our Agenda

**Shorten the review cycle**
**Address concern that clinical research is not properly evaluated**

## 17.56

**17.56**

**22.07**

- M. Martin, CSR/NIH/DHHS

## Our Agenda

**Shorten the review cycle**
**Address concern that clinical research is not properly evaluated**
**Improve the assessment of innovative, high- risk/high-reward research**

## Our Agenda

**Shorten the review cycle**
**Address concern that clinical research is not properly evaluated**
**Improve the assessment of innovative, high- risk/high-reward research**
**Do more to recruit and retain more high-quality reviewers**

## Applications Received for all of NIH and Applications Referred for CSR Review, FY 1998 - 2006

**Applications Received for all of NIH and Applications Referred for CSR Review, FY 1998 - 2006**

**NIH Applications**

**Applications for CSR Review**

## Number of Research Grant Applications/Applicant

**Number of Research Grant Applications/Applicant**

## Growth of R01 Applications              Reviewed at CSR vs. Other ICs

**Growth of R01 Applications              Reviewed at CSR vs. Other ICs**

## Growth of R21 Applications              Reviewed at CSR vs. Other ICs

**Growth of R21 Applications              Reviewed at CSR vs. Other ICs**

## Institutes and Centers Use R21s Differently 

**R21s fund ~14 types of efforts, such as—**
**Exploratory/developmental research **
**Junior investigators**
**Phase I/II clinical trials**
**International research planning  **
**High-risk/payoff research  **
**R21s have broad parameters—**
**Award period:        1-3 years**
**Funding allowed:   $100K-$450K**
**Research Plan:       10-20 pages**

## 	Average Number of Applications                             Per Reviewer		     October Council Only

**	****Average Number of Applications ****                            Per Reviewer****	****	****     October Council Only**

## Academic Rank of All CSR Reviewers Standing and Ad Hoc Combined

**Academic Rank of All CSR Reviewers Standing and Ad Hoc Combined**

## Reviewers – Current Situation 

**Far too many reviewers on study sections**
**Broader science**
**Decrease in reviewer load **
**Unnecessary**
**Too many ad hoc reviewers**
**Different processes for enlisting chartered and ad hoc members**
**Often too many telephone reviewers**

## Challenges to Recruiting Reviewers

**Manage the need for 20,000 reviewers**
**Decrease the number of reviewers and increase the quality**
**Increase the number of applications  reviewed without extra workload**
**Recruit and retain the best reviewers**
**Have smaller study sections with fewer ad hoc reviewers**

## Possible Approaches for Recruiting and Retaining the Best Reviewers

**Require less travel by using electronic review modes**

## Expanding Peer Review Platforms

**Electronic Reviews**
**Telephone Enhanced Discussions**
**Video Enhanced Discussions**
**Asynchronous Electronic Discussions**

**     ****Our Goal:  10% of all reviews to be electronic in 2007**

## Electronic Review Methods               Number of Applications Reviewed by Year

**Electronic Review Methods               Number of Applications Reviewed by Year**

## Possible Approaches for Recruiting and Retaining the Best Reviewers

**Require less travel by using electronic review modes**
**Have shorter meetings**
**Use Various Review Platforms**
**Unscore 40% of the F32s (postdoctoral fellowships)**
**Shorten Applications**

## Size of Grant Applications

**RO1 **
**Will increase number of applications reviewed by reviewers**
**Will decrease the number of reviewers in a study section**
**May be combined with a change in format of the application, more consonant to review criteria**
**Strong support by councils and scientific leadership**
**May result in better focus on significance (and innovation)**

## Trans-NIH Committee to Shorten the Application 

**Chairs**
**Robert Finkelstein, Ph.D. **
**(Co-Chair), NINDS **
**Donald Schneider, Ph.D. **
**(Co-Chair), CSR **
**Members**
**Mary Custer, Ph.D.**
**CSR **
**Ann A. Hagan, Ph.D.**
**NIGMS **

**Craig A. Jordan, Ph.D. ****NIDCD **
**Sherry Mills, M.D., M.P.H.**
**OER**
**Philip F. Smith, Ph.D. ****NIDDK**
**Barbara Spalholz **
**NCI**
**Betsy Wilder, Ph.D.**
**NIDDK**
**Alan L. Willard, Ph.D.**
**NINDS **

## Special Thanks