# Presentation: 709769

## Oversight of CSR Study Sections that Review Small Business Applications

- Elliot Postow

## CSR Advisory Committee Discussion

- May 2003 the CSR Advisory Committee (AC)
- discussed the possibility of chartering Small Business Study Sections.
- Reasons for chartering:
- “regularize” study section activities.
- provide additional oversight.
- increase the consistency of review.

## CSR Advisory Committee DiscussionContinued

- Reasons for not recommending chartering:
- Difficulty in obtaining long-term commitment from reviewers who come from small businesses.
- Variability in expertise needed:
- from meeting to meeting.
- over the four-year term of membership.
- Recidivism close to that of chartered study sections.
- Increased administrative burden for:
- review staff.
- committee management staff.
- Very few “frequent flyers”.

## Today’s Question

- How can we provide additional oversight and increase the consistency of review in Small Business Study Sections without chartering them?

## Slide 5

## Applications Reviewed by CSR

| ACTIVITY  | 2004/05 | 2004/10 | 2005/01 |
| --- | --- | --- | --- |
| Small Business | 2,287  | 2,243  | 2,056  |
| R01 | 9,899  | 9,336  | 9,563  |
| Other | 4,432  | 4,860  | 4,434  |
| Total | 16,618    | 16,439  | 16,053  |

## Demographics of CSR Study SectionsGender and Age

| Type of Study Section | Number of Reviewers | Percent Females | Average Age of all Reviewers |
| --- | --- | --- | --- |
| Small Business | 937 | 21.7 | 48.7 |
| Chartered | 5191 | 27.8 | 49.1 |

## Demographics of CSR Study SectionsMinorities

| Type of Study Section | Number of Reviewers | All Minorities Number | All Minorities Percent | Under-represented Minorities Number | Under-represented Minorities Percent |
| --- | --- | --- | --- | --- | --- |
| Small Business | 937 | 148 | 15.8 | 42 | 4.5 |
| Chartered | 5191 | 951 | 18.3 | 297 | 5.7 |

## Demographics of CSR Study SectionsGeographic Distribution

| Type of Study Section | East | South | Central | West | Foreign |
| --- | --- | --- | --- | --- | --- |
| Small Business | 26.2% | 32.2% | 23.2% | 17.3% | 1.1% |
| Chartered | 29.1% | 28.8% | 21.6% | 19.3% | 1.6% |

## Demographics of CSR Study SectionsReviewer Stature

| Type of Study Section | Assistant Professor | Associate Professor | Professor | Other |
| --- | --- | --- | --- | --- |
| Small Business | 15.9% | 27.9% | 35.8% | 20.3% |
| Chartered | 9.3% | 29.3% | 49.7% | 11.7% |

## Demographics of CSR Small Business Study Sections

| Academic* | Small Business | Large Business | Government |
| --- | --- | --- | --- |
| 80.4% | 16.7% | 1.5% | 2.1% |

- *Includes individuals with both academic appointments and small business affiliations

## Scoring Behavior of Small Business Study Sections

| Type of application | Number reviewed | Percent with priority score < 200 |
| --- | --- | --- |
| All Small Business | 4,498 | 15.6 |
|     Fast Track | 158 | 17.1 |
|     Phase I | 3,644 | 13.5 |
|     Phase II | 680 | 25.9 |
| R01 | 678 | 28.3 |
| Other | 551 | 22.0 |

## Proposed Oversight Mechanisms

- Annual “State of the Study Section” Report to IRG Chief and Division Director.
- Working Group Evaluation of Small Business Study Sections in a Division.

## State of the Study Section Report

- Prepared by the SRA.
- Similar in concept to the nomination slate
- Covers the previous year.
- Documents changes in review responsibilities.
- Discusses review workload.
- Discusses relationship with other Small Business Study Sections.
- Lists Institutes supported.
- Raises issues to CSR management.
- Discusses reviewer participation

## Reviewer ParticipationHistory and Plans

- Demographics
- Women
- Minorities
- Under represented minorities
- Geographic distribution
- Business Experience
- Small Business
- Large Business
- End Users
- Length of Service
- Chair
- Members
- Prior Service

## Small Business Working Group

- Examination of CSR Study Sections every five years.
- All Small Business Study Sections within a Division will be evaluated simultaneously.
- Evaluation will be framed by a series of questions about the Study Section’s operations.
- Input for questions will be obtained from:
- Retiring Chairs.
- Other study section members.
- Program staff.
- CSR staff.
- Reports of prior IRG Working Groups.
- Reviewer Satisfaction Surveys.
- After each of three consecutive meetings, a reviewer will be selected from each Study Section to participate as an evaluator.

## Evaluators

- Evaluators will be experienced reviewers.
- Occasionally a new reviewer, one with no prior NIH review experience, may also be selected to share perspectives on training and integrating first-time reviewers into the deliberations of the Study Section.
- Division Director may invite additional individuals to attend Study Sections to supplement the evaluation process.
- Evaluators will address a series of questions about the operations of the Study Section.
- Input will be solicited from Chairs.

## Evaluation Process

- After the third review meeting all individuals who served as Chair during the year will be asked a similar set of questions.
- Evaluators for each Study Section will discuss their observations and develop a report on the status of the Study Section.
- The Division Director will convene a group consisting of one representative from each team of Study Section evaluators.
- This group will review and discuss the individual Study Section evaluations and draw more global conclusions and recommendations about the operations of Small Business Study Sections in the Division.
- A dialog with the SRAs will be integrated into the evaluation process.
- With assistance from the Division Director, a summary report will be prepared.
- Feedback from program staff will be solicited.
- Final report will be conveyed to the PRAC and to the Director, CSR.

## Thanks to:

- Lawrence Boerboom
- Valerie Durrant
- Michael Marino
- Marian Wachtel
- Teresa Lindquist
- Diane Bronzert
- Sherwood Githens
- Kathleen Shino