# Presentation: 915892

## Evaluation and Evolution of CSR Study Sections and Integrated Review Groups: Principles and PhilosophyDon Schneider, PhDDiscussants Drs. Pugh, Leinwand, Sassaman, & RammPeer Review Advisory CommitteeMay 22, 2006National Institutes of HealthDepartment of Health and Human Services

**Evaluation and Evolution of CSR Study Sections and Integrated Review Groups: Principles and Philosophy**Don Schneider, PhDDiscussants Drs. Pugh, Leinwand, Sassaman, & RammPeer Review Advisory CommitteeMay 22, 2006National Institutes of HealthDepartment of Health and Human Services

## Study Section Organization

- Core Values of Peer Review
- Fair, expert, & timely
- Free from influence/conflict
- Organizational Approach:
- 	Integrated Review Groups (IRGs)
- Right size work unit
- Cluster related science

## Scientific Peer Review at DRG/CSR

- 1945-1998: Case-by-case changes
- Study sections formed in response to increases in workloads on a case-by-case basis
- 1998/9: Broad changes
- With scientific community participation, merger of Alcohol, Drug Abuse and Mental Health Administration (ADAMHA) with NIH
- AIDS IRG formed
- Panel on Scientific Boundaries for Review (PSBR) meetings to develop plan for study section organization with involvement of scientific community

## CSR: IRG Organization2006

- IRGs with focus on organ system or disease serve NIH health mission (17 IRGs)
- IRGs with focus on basic scientific discovery ensure foundation for future discoveries (3 IRGs: BCMB, CB, & GGG)
- IRGs with focus on development of methods and crosscutting science are needed (3 IRGs: BDA, BST, & SBIB)

## Study Section Evaluation:Philosophical views

- Assurance of Core Values of peer review
- Continuous monitoring of peer review processes and organization
- Prompt evolution of organization as scientific opportunities change
- Promotion of open communications and transparency with the applicant community and NIH staff

## Procedures for Systematic Evaluation:

- Identifying issues, concerns, and problems
- Collecting and analyzing data
- Involving applicants, reviewers, relevant advocacy groups and scientific societies, and NIH staff

## Five Year AssessmentsWorking Groups

- Full cycle of assessments completed 2001
- Second cycle started with Neuroscience IRGs
- SBIR study sections also underway
- Other IRGs will follow in order of reorganization, AIDS and Behavioral/Social Science IRGs, then HEME (2003) and ending with BCMB and CB (2005)
- Working Groups meet by email, telephone/video conference, asynchronous electronic discussion, or face-to-face
- Results are presented to PRAC (a FACA requirement)

## Additional Working Group Activities

- May address trans-IRG issues resulting from excessive overlap or emergence of new areas
- May evaluate broader principles and effectiveness of CSR structure by involving senior members of broad spectrum of areas similar to Bruce Alberts-led PSBR (eight years ago)

## Continuous Monitoring

- Study section workloads are analyzed every cycle, e.g., to assess staffing needs
- CSR supervisors, IRG Chiefs and senior management, including Dr. Scarpa, attend study sections
- CSR reviews one IRG a month, completing all 23 every two years

## Monthly IRG Review

- Initiated February 2006
- Review sequence determined by:
- Known/emerging issues
- Scientifically related areas
- For local matter, such as overload in one study section, CSR develops plan of action
- For substantial matter, e.g., formation of new study section, a Working Group is formed, and the issue is presented and discussed at PRAC

## Cont. Continuous Monitoring

- Contact retiring Study Section Chairs
- Attend scientific society meetings and present peer review talks
- Host visiting officials from scientific societies
- Invite society representatives of broad scientific areas to CSR for discussion
- About 30 societies per broad area
- See Dr. Scarpa’s 9:00 AM Remarks

## Summary

- Adhere to core values of peer review, in support of NIH health mission
- Allow scientific developments, with input from senior advisors, to influence/shape design organization philosophy
- Monitor review units every 5 years, every 2 years, and continuously
- Promote communication and transparency with community, staff, and PRAC

## DISCUSSANTS

- Dr. Edward Pugh
- Dr. Leslie Leinwand
- Dr. Anne Sassaman
- Dr. Louise Ramm
- PRAC