# Presentation: 411151

## CSR Best Practices Committee

**CSR Best Practices Committee**

**Cheryl A. Kitt, Ph.D.**
**Deputy Director**
** **

**National Institutes of Health****U.S. Department of Health and Human**** ****Services**

**December 3, 2007**

## CSR’s Mission 

***Ensure NIH grant applications receive fair, independent, expert, and timely reviews***
**To ensure that the process continue to operate at a high level, the CSR Best Practices Committee was formed.**

## Charge to the CSR Best Practices Committee:

**Goals:**
**Review existing policies and procedures**
**Ensure alignment with FACA and other regulations**
**Ensure consistency of practices and policies across study  sections**
**Promote the sharing of best practices across sections at CSR**

## The Committee Authored Five CSR Peer Review Best Practices Guidelines:

**Criteria for Selecting Reviewers and Assembling Rosters**
**Conduct of Study Section Meetings – Roles and Responsibilities**
**Roles and Responsibilities for Summary Statement Production **
**Telephone Reviewers at Face-to-Face Review Meetings**
**Mail Reviews**
********These documents are companion pieces to the SRA Handbook, a critical compendium of peer review policies and practices***

## 1. Criteria for Selecting Reviewers and Assembling Rosters

**Core Values****:**
**The quality of scientific evaluation of a grant application is a reflection of the quality of the reviewers on the roster.**
** **
** ****SROs must ensure that the study section remains responsive to emerging areas of science and shifting scientific boundaries, and maintain a balanced membership with respect to geographic, gender, and minority representation. **
**Balance is needed between recruiting generalists for their    breadth and specialists for specific niche research areas. **

## Reviewer Selection 

**General Considerations: **
**Independent and Established Investigator **
**Respected by peers (e.g., department chair, journal editor, clinical status, plenary lectures, keynote speaker, etc.) **
**Quality of research accomplishments – Impact on a field **
**Independent publications in peer-reviewed journals **
**Honors (e.g., Awards, Memberships, etc.)**
**Research Support (e.g., NIH, VA, NSF, foundations, industry, foreign, etc.) **
** **

## For Fellowship Committees: 

**All of the above **
**Broad perspective **
***Reviewer’s track record of mentoring*****: e.g., students, pre- and/or postdoctoral fellows; institutional or unit (departmental, division; lab) mentoring- years and resources **
**For Small Business Committees: **
**If from academia, criteria listed above **
**Rank in the small business concern (e.g., CSO, section head, directorship, group/project leader) **
**SBIR/STTR funding **
**Publication record in peer reviewed, high visibility journals or Patents **
**Recognized as prominent practitioner in a professional field (e.g., surgeon, engineer) **

## Sources for Reviewer Recruitment: 

**Research databases (e.g. CRISP, PubMed, PsychInfo, etc.) **
**Professional societies **
**Scientific community consultation **
**NIH Program Staff **
** ****Conference attendance **
**Study section chairs **
**Study section members **

## Maintaining Reasonable Application to Reviewer Workload 

***Best Practice Statement: ***
**For reviewers attending the meeting, the goal is *****7 assignments *****per reviewer on average, which includes reader assignments (see exceptions below). **
**A workload of 7 gives the SRO flexibility to bring in a limited number of reviewers with light loads (e.g., field leaders, reviewers for a small group of specialized applications), balanced by the rest of the study section who are assigned a more substantial number of applications to review. **
****Exceptions: ***
**For small numbers of temporary members, member SEPs, re-review SEPs, multidisciplinary clinical trials, etc., that must be approved by the IRG Chief****. **

## 2. Conduct of Study Section Meetings 

**Goal: **
**To provide consistent and clear guidance to SROs regarding the expectations of *****conduct before and at study section meetings*****, and to provide guidance for continued oversight of this critically important function. **
**********An effective peer review process requires an SRO to be active at the meeting, as well as in the weeks before the meeting.***

## Conduct of Study Section Meetings

**How: **
***Establish contact with reviewers and stay in touch with them *****during the weeks leading up to the meeting. **
**Foster a team environment with *****ongoing interactions between SRO, IRG Chief, and Division Director *****to enhance the quality of peer review.**
**Use Pre- and Post-Meeting Checklists**

## 3. Summary Statement Production Best Practices

**High quality summary statements depend both on the reviewer and SRO. **
**Reviewers effectively communicate in writing the critique of an application. **
**SRO provides orientation and oversight before and during meetings to avoid discrepancies and misleading statements.**
** ****SRO monitors critiques posted in IAR and provides an opportunity to detect and correct problems before the meeting, enhancing the review process and subsequent summary statement. **
**The SRO uses written critiques and input from study section discussions to assemble summary statements that are concordant with the scores. **

## 4. Telephone Reviewers at Face-to-face Review Meetings 

**Limit number of telephone reviewers initially to 10% and not to exceed 20% for regular face-to-face meetings, barring unforeseen emergencies. **
**Telephone reviewers should have prior review experience. **
**SROs must focus on handling the meeting and taking notes. Thus, they should be minimally involved in communication logistics. **
**Telephone reviewers who are not regular members preferably should not lead off the discussion. **
**Telephone reviewers submit their scores in a secure manner, preferably via IAR or Meeting One. **

## 5. MAIL REVIEWS 

**In balancing expert coverage, SROs will on occasion have an application where a small part of the proposed project involves a special expertise or technique not covered by the existing members of the panel. **
**If the size of the need for the expertise does not warrant addition of another member to the meeting then a mail review may be appropriate. **

## Parking Lot Issues

**Conflict of Interest**
**Streamlining**
**Appeals/Rebuttals**
**Deferral for Re-Review**
**Selection of Study Section Chairs**
**Training of Reviewers and Chairs**
**Role of Program Staff at Meetings**

## Committee Members

**Shirley, Mariela (NIAAA-Program Director)**
**Bradley, Eileen (CSR-IRG Chief)**
**Byrnes, Noni (CSR-IRG Chief)**
**Cooper, Cathleen (CSR-SRO)**
**Edwards, Emmeline (NINDS-Dep. Dir. DEA)**
**Etcheberrigaray, Rene (CSR-IRG Chief)**
**Fisher, Suzanne (CSR-Division Director)**
**Gibson, Joy (CSR-IRG Chief)**
**Khan, Mushtaq (CSR-IRG Chief)**
**Krishnan, Krish (CSR-SRO)**
**Panniers, Richard (CSR-IRG Chief)**
**Pyper, Joanna (CSR-Dep. IRG Chief)**
**Rigas, Marc (CSR-SRO)**
**Schneider, Donald (CSR-Division Director)**
**Kitt, Cheryl  (CSR), Chair**

## kittc@csr.nih.gov

**kittc@csr.nih.gov**
**301-435-1112**