# Presentation: 263069

## Peer Review: The Role of the Scientific Review Administrator

- Francisco O. Calvo. Ph.D.
- Chief, Review Branch
- National Institute of Diabetes and Digestive and Kidney Diseases
- National Institutes of Health
- Department of Health and Human Services

## The NIH Extramural Team

**Review**

**Program**

-    **Grants**
**Management**

**Notes:**
.The NIH Extramural Team 

consists of review, program, and grants management segments.  

   The NIH seeks to ensure integrity and accountability in it grants administration processes by 
Relying on system of checks and balances
separation of responsibilities

None of theses entities are responsible for or report to the other.  Although these segments are separate and distinct, but do overlap in small but significant ways.  

## Peer Review of NIH Research Grant Applications

**Center for Scientific Review **
**National Institute of **
**Diabetes and Digestive and **
**Kidney Diseases-Review Branch**

## Applications Submitted to NIH

- Approximately  46,000 grant applications are submitted to NIH each year, of which 25-30% are funded
- Competing grant applications are received for three review cycles per year

## Review Process For A Research Grant

**Research Grant**
**Application**

**School or Other**
**Research Center**

**National Institutes of Health**

**RFA/PI Initiates**
**Research Idea**

**Conducts Research**

**Submits**
**application**

**Allocates**
**Funds**

** **

**Center for Scientific Review**

**Scientific Review Group/**
** ****IC Review Branch**

**Institute-NIDDK**

**Advisory Council-NDDK**

**Institute Director**

**Assign to**

**IC and IRG**

**Review **
**  ****for**

**Scientific**
**Merit**

**Evaluate for**

**Relevance**

**Recommend**

**Action**

## Peer Review

- CSR and IC Study Sections are managed by a Scientific Review Administrator (SRA) who is a scientist, usually at the Ph.D. level, whose scientific background is close to the expertise of the study section
- Each CSR/IC standing study section has 12-24 members who are primarily from academia
- Institute review is also different in that Special  Emphasis Panels are constituted specifically to review applications that have been received in response to an RFA or other Institute specific grant mechanisms
- As many as 60-100 applications are reviewed at each study section meeting, again this depends on the complexity of the review

## Scientific Review Administrator

- Performing administrative and technical review of applications to ensure completeness and accuracy
- Selecting reviewers based on broad input
- Managing study section meetings
- Preparing summary statements
- Providing any requested information about study section recommendations to institutes and national advisory councils/boards

**Designated Federal official with overall **
**     ****responsibility for the review process, including:**

## Selection of Peer Reviewers

**Non-Doctoral**

**Scientific Community**

**Non-Research**

**Research**
**Capability**

**Active and Productive**
**Researchers**

## Criteria for Selection of Peer Reviewers

**Scientific Community**

**Active and Productive Researchers**

**Non-Doctoral**

**Non-Research**

**Active & Productive Researchers**

**Research Capability**

**Doctoral or Equivalent Degree**

**Interest in Serving**

**Expertise in Discipline of Review Group and Specialization Needed**

**Meet Internal Administrative Considerations**

**Geography**

**Institutional**

**Affiliation**

**Non-Federal**

**Employment**

**Gender &**

**Ethnic**

**Status**

## Study Section Meeting

## Certification of  No Conflict of Interest

- This will certify that in the review of applications and proposals by _(study section_) on _(date)_, I did not participate in the evaluation of any grant or fellowship applications from (1) any organization, institution or university system in which a financial interest exists to myself, spouse, parent,child, or collaborating investigators; (2) any organization in which I serve as officer, director, trustee, employee or collaborating investigator; or (3) any organization which I am negotiating or have any arrangements concerning prospective employment or other such associations.
-    __________________     __________________
-    __________________     __________________
-    __________________     __________________
-    __________________     __________________

**SIGNATURES**

## Confidentiality

- Review materials and proceedings of review meetings represent privileged information to be used only by consultants and NIH staff.
- At the conclusion of each meeting, consultants will be asked to destroy or return all review-related material.
- Consultants should not discuss review proceedings with anyone except the SRA.
- Questions concerning review proceedings should be referred to the SRA.

**K185pp.46**

## Review of Research Grants

- REVIEW CRITERIA:
-   Significance
-   Approach
-   Innovation
-   Investigator
-   Environment
-   Overall Evaluation & Score Reflects Impact on Field

## Review Criteria (Continued)

- Significance: Does the study address an important problem?  How will scientific knowledge be advanced?
- Approach: Are design and methods well-developed and appropriate?  Are problem areas addressed?
- Innovation: Are there novel concepts or approaches?  Are the aims original and innovative?
- Investigator: Is the investigator appropriately trained?
- Environment: Does the scientific environment contribute to the probability of success?  Are there unique features of the scientific environment?

## Scientific Review Group or Study Section Actions

- Scored, Scientific Merit Rating (priority scores and percentiles)
- Unscored (lower half)
- Deferral

## Action

- Scored -- Scientific Merit Rating 1.0 to 	   approximately 3.0
- Based on the relevant review criteria,  the application is judged to be in the upper half of applications reviewed by the study section or scientific review group.  The recommendation can be for the requested time and amount or for an adjusted time and amount.  A priority score is provided, and a summary statement prepared that incorporates the written critiques plus a resume and summary of the discussion.

## Action

- Unscored
- Application is unanimously judged to be in the lower half of applications reviewed by the study section or scientific review group.  No priority score is assigned.  The summary statement provided to the applicant is a compilation of reviewers’ comments prepared prior to the meeting.

## Action

- Deferral
- The study section cannot make a recommendation without additional information.  This information may be obtained by a project site visit or by submission of additional material by the applicant.

## Post Scientific Review Group Actions

- Calculations of priority scores and percentile rankings
- Preparation of summary statements
- Removal of applications from national advisory council / board consideration

## Summary Statement

- Once applications are reviewed, the results are documented by the SRA in a summary statement and forwarded to the IC Program Director (and the PI) where a funding decision is made:
- _The summary statement contains:_
- Overall Resume and Summary of Review Discussion
- Essentially Unedited Critiques
- Priority Score and Percentile Ranking
- Budget Recommendations
- Administrative Notes

## Preparation of an Application

## PHS Research Grant

***    ******Application Kit (form PHS 398)***

**Mail Completed Forms To:**
**CENTER FOR SCIENTIFIC REVIEW**
**NATIONAL INSTITUTES OF HEALTH**
**ROCKLEDGE II ROOM 1040 MSC-7710**
**BETHESDA MD 20892-7710**

## When Preparing an Application

- Read and understand the Instructions in the PHS398 Form
- Read and understand the Instructions in the PHS398 Form
- Read and understand the Instructions in the PHS398 Form
- Read and understand the Instructions in the PHS398 Form
- Read and understand the Instructions in the PHS398 Form

## When Preparing an Application

- Read instructions
- Never assume that reviewers “will know what you mean”
- Refer to literature thoroughly
- State rationale of proposed investigation
- Include well-designed tables and figures
- Present an organized, lucid write-up
- Obtain pre-review  from faculty at your institution

## Common Problems in Applications

- Lack of new or original ideas
- Absence of an acceptable scientific rationale
- Lack of experience in the essential methodology
- Questionable reasoning in experimental approach
- Uncritical approach
- Diffuse, superficial, or unfocused research plan
- Lack of sufficient experimental detail
- Lack of knowledge of published relevant work
- Unrealistically large amount of work
- Uncertainty concerning future directions

##                                          for GrantsNIH GUIDE                   and ContractsU.S. Department of Health and Human Services

- Announces NIH Scientific Initiatives
- Provides NIH Policy and Administrative Information
- Available on the NIH Web Site :        http://www.nih.gov

## Program Announcement

- Invites grant applications in a given  research area
- May describe new or expanded interest in a particular extramural program
- May be a reminder of a continuing interest in a particular extramural program
- Generally has no funds set aside
- Applications reviewed in CSR along with unsolicited grant applications

## Requests for Applications (RFA)

- Announcement describing an institute initiative in a well-defined scientific area
- Invitation to the field to submit research grant applications for a one-time competition
- Set-aside of funds for a certain number of awards
- Applications generally reviewed within the issuing institute, in this case the NIDDK

## NIH Information Sources

## Information on the World Wide WebSelected Sites of Interest   

- _National Institutes of Health_ (http://www.nih.gov)
- Office of Extramural Research (http://www.nih.gov/grants/oer.htm)
- Grants Policy (_[http://www.nih.gov/grants/policy/policy.htm](http://www.nih.gov/grants/policy/policy.htm)_)
- _Center for Scientific Review_ (http://www.csr.nih.gov)
- Referral and Review (http://www.csr.nih.gov/refrev.htm)
- CSR Study Section Rosters (http://www.csr.nih.gov/committees/rosterindex.asp)
- NIH Peer Review Notes  (_[http://www.csr.nih.gov/prnotes/prnotes.htm](http://www.csr.nih.gov/prnotes/prnotes.htm)_)
- _NIDDK (http://www.niddk.nih.gov/fund/fund.htm)_

## Office of Extramural Research

- Handles requests for grant applications, program guidelines, and general information regarding grant applications
**Office of Extramural Research**
**National Institutes of Health**
**6701 Rockledge drive, suite 6095**
**Bethesda, Maryland 20892-7910**
***Phone*****:   301-435-0714**
***Fax:*****         301-480-0525**
***E-mail:*****      grantsinfo@nih.Gov**

## There is no grantsmanship that will turn a bad idea into a good one, but........There are many ways to disguise a good one.

- Dr. William Raub,
- Former Deputy Director, NIH