# Presentation: 337639

## CSR INTEGRATED REVIEW GROUP REVIEWS

**CSR INTEGRATED REVIEW GROUP REVIEWS**
**[peer review of peer review]**
**Don Schneider**

**National Institutes of Health****U.S. Department of Health and Human Services**

**November 3, 2008**

## CSR is committed to peer review of peer review

**Need to respond to changing science**
**Trace roots to Panel on Scientific Boundaries for Review (PSBR)**
**Examine Integrated Review Group-specific issues**
**Discuss general issues**

## The PSBR issued a report January 2000 outlining reorganization philosophy and expectations.

**URL - **_**[http://cms.csr.nih.gov/NewsandReports/ReorganizationActivitiesChannel/BackgroundUpdatesandTimeline/FinalPhase1Report.htm](http://cms.csr.nih.gov/NewsandReports/ReorganizationActivitiesChannel/BackgroundUpdatesandTimeline/FinalPhase1Report.htm)**_
**Be monitored continuously: assess peer review system at regular intervals, at the level of the IRG, and globally by the Advisory Committee **** **

## Working Groups Assess IRGs[Peer review of peer review]

**Peer reviewers report on study section operations**
**Feedback from reviewers condensed by staff into a brief 5-10 page white paper report**
**Subset of peer reviewers meet as working group to edit/confirm essence of report**
**Report is shared with Advisory Committee and used as basis of taking action for improvements**

## IRG Working Groups, Round 1, provided baseline, 2000-2002

**Round 1 working groups – one-cycle visit of each study section, often by single pre-selected reviewer**
**Pre-selection probably resulted in perturbations**
**Nonetheless, useful information obtained that informed reorganization**
**PSBR Reorganization occurred step-wise one IRG at a time from June 2003 to March 2005**

## IRG Working Groups, Round 2, were initiated in 2005 and are currently in progress

**Eliminate perturbation –peer reviewers of peer review selected post-study section meetings**
**Solicit input from multiple reviewers each cycle**
**Solicit input from reviewers over multiple, at least two, cycles**
**Share white papers with Chairs prior to reporting to Advisory Committee**
**Provide substantial basis for future actions**

## Schedule for IRG Reviews

**July 2008**
**Brain Disorders & Clinical Neuroscience (BDCN)**
**Molecular, Cellular & Developmental Neuroscience (MDCN)**
**September 2008**
**Integrative, Functional & Cognitive Neuroscience (IFCN)**
**Emerging Technologies & Training in Neuroscience (ETTN)**
**Biology of Development & Aging (BDA)**
**January 2009**
**Biobehavioral & Biobehavioral Processes (BBBP)**
**Risk, Prevention & Health Behaviors (RPHB)**
**March 2009**
**Healthcare Delivery & Methodologies (HDM)**
**Epidemiology & Population Sciences (EPS)**

## Cont., Schedule for IRG Reviews

**May 2009**
**Immunology (IMM)**
**Infectious Diseases & Microbiology (IDM)**
**July 2009**
**Oncology 1 – Basic Translational (OBT)**
**Oncology 2 – Translational Clinical (OTC)**
**September 2009**
**Bioengineering Sciences & Technologies (BST)**
**Surgical Sciences, Biomedical Imaging & Bioengineering (SBIB)**
**November 2009**
**AIDS & Related Research (AARR)**
**Endocrinology, Metabolism, Nutrition & Reproductive Sciences (EMNR)**

## Cont., Schedule for IRG Reviews

**January 2010**
**Vascular & Hematology (VH)**
**Cardiovascular & Respiratory Sciences (CVR)**
**March 2010**
**Musculoskeletal, Oral & Skin Sciences (MOSS)**
**Digestive, Kidney & Urological Systems (DKUS)**
**May 2010**
**Genes, Genomes & Genetics (GGG)**
**Biological Chemistry & Macromolecular Biophysics (BCMB)**
**July 2010**
**Interdisciplinary Molecular Sciences & Training (IMST)**
**Cell Biology (CB)**

## Format for IRG Reviews

**Late afternoon**
**Present Peer Review Issues & Current Initiatives (CSR Director)**
**Discuss; discuss further over dinner**
**Morning**
**Present Working Group White papers**
**Describe study sections and specific issues/metrics (Chiefs)**
**Discuss and improve study section guidelines (Chairs)**
**Forecast future issues & trends in science**
**Afternoon**
**Consider the critical role of the study section chair (CSR Director, in closed session with Chairs)**

## IRG specific issues and actions are noted

_**Molecular, Cellular & Developmental Neuroscience IRG**_
**One study section split for second time**
**Initial attempt to charter a neurotechnology panel led to PRAC requested crosscutting look and eventual formation of two study sections, as well as stimulating formation of crosscutting neuroscience IRG and contributing to momentum for new division**

## Cont. IRG-specific issues & actions

_**Brain Disorders & Clinical Neuroscience IRG**_
**Split two study sections, each into two**
**Resolve referral issue so that brain tumor applications focusing on neurological aspects are assigned to BDCN, with those focusing on cancer being assigned to Oncology IRG**

## Cont. IRG-specific issues & actions

_**Integrative, Functional & Cognitive Neuroscience IRG**_
**No issues & no actions**
_**Biology of Development & Aging IRG**_
**Clustering of International & Cooperative Projects in BDA, with great breadth of topics – perhaps pilot editorial board review**
_**Emerging Technologies & Training in Neuroscience IRG**_
**First applications reviewed winter 2008, no revisions/actions yet**
**One new study section growing rapidly, on verge of needing to split**

## General Review Issues & Actions

**Reviewers**
**Seniority**
**Ad hoc/temporary**
**Telephone**
**Mail**
**Registry/College**
**Scoring**
**Streamlining/unscoring**
**Score compression**
**Scoring consistency throughout meeting**

## IRG Issues & Enhancing Peer Review

**Reviewers**
**Flexible service**
**Systematic training**
**Shorter applications and critiques**
**Scoring**
**Smaller scale, 7 or perhaps 9 bins**
**Scores for all applications**
**Promote score spreading, ranking or order of review**
**Emerging scientific trends**
**Selecting appropriate reviewers**
**Updating study section guidelines**

## The End