Center for Scientific Review Advisory Committee Meeting

National Institutes of Health

January 28-29, 2002

The Center for Scientific Review (CSR) convened the 28th meeting of its
Advisory Committee at 8:30 a.m. on Monday, January 28, 2002, in
Conference Room 9100, Rockledge II Building. The entire meeting was
convened in open session. Dr. Karen Matthews presided as Chair.

## Members

Karen Matthews, Ph.D., Chair Susan Berget, Ph.D.

Shu Chien, M.D., Ph.D. Michael Colvin, M.D.

James Kushner, M.D. Edward Pugh, Jr., Ph.D.

Lucia Rothman-Denes, Ph.D.

## Ad Hoc Advisors

Leonard Epstein, Ph.D. Michael Leon, Ph.D.

David Soybel, M.D. Marvin Wickens, Ph.D.

David Williams, Ph.D., M.P.H.

Dr. Howard Schachman was the official observer, and Dr. Brent Stanfield
was the Executive Secretary for the meeting.

## Welcome and Opening Remarks 

Dr. Matthews welcomed members of the CSR Advisory Committee (CSRAC) and
asked everyone at the table to introduce themselves for the benefit of
new members. She then asked the Committee to consider the minutes from
the May 14-15, 2001, CSRAC meeting. Dr. Shu Chien moved that the minutes
be approved. Dr. Lucia Rothman-Denes seconded the motion, and the
minutes were approved.

**CSR Update**

*Budget Developments*

Dr. Ellie Ehrenfeld, Director, CSR, noted that Congress recently passed
the fiscal year 2002 budget for the National Institutes of Health (NIH).
CSR\'s allocation will be sufficient to continue hiring new staff and to
support ongoing initiatives. The 2003 NIH budget appears to be in accord
with the 5-year plan to double NIH funding by 2004.\
A significant portion of future increases will likely be earmarked for
targeted research areas, such as bioterrorism. CSR, however, should be
able to advance its reorganization and hiring efforts, continue its
internship program, and further the electronic receipt and review of
applications.

*Response to the September 11 Tragedy*

There were several consequences of the September 11 attack for NIH peer
review. For study section meetings held during the subsequent weeks,
reviewers were offered reimbursement for an extra night lodging, etc. if
they opted to drive or travel by train instead of flying. However, more
reviewers than usual participated via telephone. In the end, almost all
reviewers did attend their fall-round meeting. Dr. Ehrenfeld expressed
appreciation for the commitment and dedication of these reviewers. She
added that the recent anthrax mailings led CSR to issue a new policy
concerning the delivery of grant applications. Individuals may no longer
hand deliver applications, etc. but must use the U.S. Postal Service or
a licensed courier. CSR may need to consider developing contingency
plans for sustaining the review process in times when it may be
impossible to hold meetings. Any proposal likely to impact broadly on
peer review would be presented to CSRAC for discussion.

*Review of Stem Cell and Other New Applications*

On November 7, 2001, NIH posted an Internet registry of existing stem
cell lines that may be used in Government-sponsored research in accord
with guidelines established by President Bush. CSR has since received
nine applications seeking to use these cells and expects to receive more
before the current submission deadline. CSR also expects to receive
applications in response to two Requests for Applications (RFAs) issued
by the new National Institute on Biomedical Imaging and Bioengineering
(NIBIB).

Dr. Ehrenfeld explained that CSR will review these applications in the
June review round. Since NIBIB intends to develop a small review
division, CSR will work with its staff to develop guidelines for
determining NIBIB and CSR review responsibilities. When these efforts
advance, she may invite NIBIB\'s Director to discuss them with CSRAC
members.

*Evaluation of the Neuroscience Study Sections*

Dr. Ehrenfeld reported on efforts to evaluate the neuroscience study
sections that were reorganized when NIH integrated the National
Institute of Mental Health, the National Institute on Drug Abuse, and
the National Institute on Alcohol Abuse and Alcoholism. CSR hired an
experienced contractor and developed surveys for applicants and
Institute staff in collaboration with the Institutes, external
consultants, and CSRAC members. These surveys have since been
administered, with an 84 percent response rate from applicants and a 75
percent response rate from Institute staff. Results will be presented at
a future CSRAC meeting. She said that member input will be valuable in
assessing the merits of future surveys and determining how they may be
improved.

*Master Reviewers*

Members were reminded of their previous discussions on increasing the
numbers of senior scientists in the peer review process. Dr. Ehrenfeld
explained that CSR is proposing to establish a new category of
\"master\" reviewers, who would be offered reduced terms of service.
These reviewers could help set the proper tone at meetings and help
train new reviewers. To be eligible, an individual would be required to
have served a regular study section term. She noted that some Scientific
Review Administrators (SRAs) have no problems recruiting senior
reviewers. This recruitment tool will thus be offered only as an option
for SRAs who feel it would be useful. She has surveyed eight study
section chairs and received an overwhelmingly positive response to the
proposal. CSR recently received approval for creating this new reviewer
category. Dr. Ehrenfeld asked CSRAC members if they approved the general
concept.

Dr. Howard Schachman expressed enthusiasm for this proposal, noting that
he has supported efforts to recruit senior reviewers for many years. He
said there were many who will be willing to participate. Dr. James
Kushner added his support. He briefly discussed the history of CSRAC\'s
interest in this issue, which arose from the impression that the
reviewers on study sections are more junior than they used to be. As
long as study sections retain an appropriate balance, he recommended
implementation of the proposal. Dr. Rothman-Denes asked about
experienced researchers who have never served as reviewers. Dr.
Ehrenfeld explained that they could still be temporary reviewers.
Limiting master reviewers to those who have served full terms will
ensure that they are experienced reviewers. This requirement also will
facilitate their acceptance by regular reviewers. Dr. Chien endorsed the
proposal, although he disliked the term \"master\" reviewer. He proposed
calling these reviewers \"special\" reviewers.

Dr. Ehrenfeld agreed that the name was problematic and encouraged
members to provide alternatives. Dr. Leonard Epstein asked if SRAs may
assess the composition of other study sections to determine how well
their own study section is balanced in terms of experience. Dr.
Ehrenfeld explained that the Chiefs of the Integrated Review Groups
(IRGs) and the Division Directors can provide relevant data or
assistance to the SRAs.

*Other CSR Initiatives*

[CSR\'s internship program]{.underline}: Dr. Ehrenfeld explained that
four interns were hired last fall from the NIH intramural community, and
everyone seems pleased with the program. Six new interns will be hired
this year. If the program continues to be successful, it will be opened
up to scientists outside of NIH.

[Mock study section video]{.underline}: CSR is advancing its efforts to
produce a video of a mock study section to educate new reviewers and
applicants about the peer review process.

[Efforts to improve adherence to application format
regulations]{.underline}: About 2 percent of the applications submitted
for the October 2001 Council round were returned for resubmission in the
next round because of format problems. CSR was persuaded by community
input to allow applicants 4 days to revise and resubmit noncompliant
applications. Dr. Ehrenfeld explained that compliance has been
improving. Compliance for small business and fellowship applications,
however, is poor. CSR will consider outreach efforts to educate these
communities.

[CSR\'s relations with professional societies]{.underline}: In response
to a question from

Dr. Matthews, Dr. Ehrenfeld explained that CSR generally has very
productive relations with many professional societies. Reorganization
efforts have improved these relations, and CSR is working to increase
communications with societies.

**Reorganization Issues**

*Overview of Activities*

Dr. Donald Schneider, Director, CSR Division of Molecular and Cellular
Mechanisms, provided an overview of CSR\'s reorganization activities. In
January 2000, CSRAC accepted the report of the Panel on Scientific
Boundaries for Review (PSBR). This panel encouraged CSR to redesign its
study sections into 24 IRGs that recognized the growing importance of
basic and molecular science to organ/disease-specific research. Since
the seven neuroscience and behavioral science IRGs were recently
reorganized, attention was focused on the remaining 17 IRGs. CSR has
initiated or completed work on seven of these IRGs. Steering committees
of NIH staff were formed for each of them to identify the subject areas
to be covered by their study sections. These committees also worked with
professional societies to identify external experts who would serve with
a small number of NIH and CSR staff members on Study Section Boundary
(SSB) Teams. These teams used a mock sort of application abstracts from
the May 2000 Council round to design the boundaries of their IRG\'s
study sections. The resulting study section guidelines are being posted
on the CSR Web site for a 90-day period for public comment. After
readjustments made and following consideration by CSRAC, CSR will devote

1 year to recruiting staff and reviewers. CSR plans to begin work on the
remaining 10 IRGs within the next 1 1/2 years.

*Development of the Hematology IRG*

To illustrate the changes being made, Dr. Schneider discussed the new
Hematology IRG. Previously, there were two nearly identical hematology
study sections. The SSB Team designed three distinct study sections that
incorporate more expertise in the basic sciences: (1) Hematology A to
focus on blood systems, including hemoglobin structure and
crystallography, gene therapy, molecular biology, and biochemistry; (2)
Hematology B to focus on basic and applied aspects of hematopoiesis,
including stem cell biology; and (3) Hematology C to focus on vascular
elements, coagulation platelets, and vascular biology. The public
comments received have highlighted areas that overlap those covered by
other IRGs: graft versus host disease, vascular biology, and parasitic
infections such as malaria. Dr. Schneider explained that such overlaps
were expected, and PSBR thought it good for many applications to have
two homes.

*Challenges Ahead*

Dr. Schneider said that participants have enjoyed the process, which
often facilitated better communications between NIH Institutes and
Centers (ICs). The guidelines produced often went well beyond the scope
of the science in the abstracts provided, and the resulting study
sections tended to be small, with less than the optimal 60-80
applications. He concluded by summarizing the challenges ahead: (1)
coming to closure on areas of shared interest, (2) adjusting boundaries
to create study sections of optimal size, and (3) managing the
implementation of all the new study sections. He also expressed concerns
about the single mock sort. Since much has changed in the 2 years since
this sort was made, CSR will conduct an additional mock sort of
application abstracts from the October 2002 Council round to see how
well the new guidelines work. CSR will continue to seek CSRAC advice as
it works to overcome these challenges.

*Surgery, Applied Imaging, and Applied Bioengineering (SAIAB) IRG*

Dr. Elliott Postow, Director, CSR Division of Clinical and
Population-Based Studies, discussed how PSBR recommended that the
reorganization focus on diseases and organ systems. PSBR only briefly
mentioned the clustering of applications according to crosscutting areas
of science. He then described how these two approaches recently came
into conflict in developing the SAIAB guidelines. While the current two
surgery study sections (Surgery, Anesthesiology and Trauma; and Surgery
and Bioengineering) now review 160-170 applications per round, the SAIAB
IRG received only 63 comparable applications in the mock sort. Members
of the SSB Team, however, were familiar with the current IRG and
designed four study sections that retained and expanded the current
crosscutting approach. In addition, they designed three imaging study
sections and one for biomedical computing and health informatics. Dr.
Postow presented four options for addressing the difference in
philosophy: (1) build the redundant study sections and see how the
applications sort out, (2) give primacy to the disease and organ system
areas,

\(3\) give primacy to the crosscutting areas, or (4) allow the
communities to inform a decision via public comments. Building redundant
study sections would be costly, while following approaches 2 or 3 would
likely upset the research communities. Dr. Postow recommended posting
the SAIAB guidelines on CSR\'s Web site and basing a final decision on
the resulting comments.

Dr. Chien began discussions, noting how PSBR took a crosscutting
approach in giving almost half of the IRGs crosscutting titles. He
supported the idea of using community comments to determine the areas to
be covered by SAIAB. He suggested that CSR carefully craft the
presentation of the SAIAB guidelines in order to get the most useful
comments. He also suggested that CSR carefully consider the approach it
takes if it decides to conduct a second mock sort so that the results
will be more representative of the true situation.

Dr. Pedro J. del Nido, a pediatric cardiologist and cardiac surgeon at
Harvard Medical School, spoke as a member of the Cardiovascular Sciences
SSB Team. He reported difficulties in placing surgery-related
applications in multiple IRGs and questioned how well these applications
would fare in review. Dr. del Nido stated that the imposed trauma of
surgery is a common link that distinguishes surgery as an important
crosscutting field. While he advocated the clustering of surgery-related
applications, he said that leeway should be provided to certain
subgroups. Surgical oncologists and medical oncologists have worked well
together reviewing applications. Neurosurgeons and neurobiologists also
have worked well together, as have vascular surgeons and vascular
biologists.

Dr. del Nido emphasized the importance of defining the criteria for the
next sort so that these concerns may be addressed.

*Taking the Crosscutting or Disease/Organ-Specific Approach*

Dr. Kushner suggested that neither the organ-specific approach nor the
crosscutting approach to reviewing applications was perfect. He
emphasized the importance of developing a mechanism for dealing with
overlapping and conflicting areas.

Dr. Matthews noted that a fifth of the applications in the mock sort
could have been assigned to more than one IRG. She suggested examining
these applications to identify areas for further evaluation before
initiating a second mock sort. She also emphasized the importance of
developing mechanisms to keep the system flexible. Dr. Ehrenfeld said it
was important to use the same mock sort in designing the new IRGs. The
second sort could allow CSR to see how the science has evolved in the
last 2 years, assess round-to-round variations, and collect new
information on alternative or overlapping assignments. She expects this
exercise will allow CSR to identify and address crosscutting issues
better. Dr. Ehrenfeld added that she would value CSRAC discussions on
maintaining a flexible review system and developing a mechanism to
resolve IRG design questions.

Dr. Edward Pugh said that more could be done to help applicants refer
their own applications. He suggested adding assignment data to the CRISP
database, which contains abstracts of funded applications. This
information could help investigators identify the most appropriate study
section for their applications. Dr. Ehrenfeld agreed that this
information could be useful, but she noted that such historical
information would have limited value as new IRGs are created. She agreed
that more could be done to educate applicants about the roster and
boundary information on CSR\'s Web site.

Dr. Susan Berget asked how the reorganization has affected the degree to
which study sections are IC specific. Dr. Ehrenfeld said that PSBR
encouraged CSR to create study sections that handle applications
assigned to multiple ICs. Its endorsement of an organ/disease approach
to creating IRGs, however, has increased the problem, since the ICs are
defined by the same approach. Dr. Brent Stanfield, Deputy Director, CSR,
noted that the diversity of IC assignments was increased in the
reorganization of the neurosciences and the behavioral and social
sciences IRGs.

Dr. David Soybel stated that investigators could likely sort out their
own referral problems within 5 years, but he thought some small fields
like surgery, radiology, anesthesiology, and reproductive medicine could
suffer in the process. Dr. Epstein suggested that the key to identifying
these kinds of problems is to determine how many reviewers are needed to
provide an informed review of a particular application.

Dr. Ehrenfeld explained that the number of applications in an area
typically determines the number of reviewers recruited in that area. She
added that it was still difficult to decide at what point clustering
should occur.

Dr. David Williams asked about mechanisms that would allow investigators
to have more input into where their grants are assigned. Dr. Ehrenfeld
explained that applicants are free to submit a written request for a
particular study section assignment. These requests are honored unless
the study section lacks the required expertise or there is a conflict of
interest. She noted that applicants frequently request an inappropriate
study section. CSR therefore plans to increase efforts to inform
applicants about the practice once the reorganization is complete.

In summarizing the discussion, Dr. Matthews focused on the importance of
assessing overlapping or crosscutting areas by soliciting additional
input from investigators and conducting a new mock sort with the help of
IRG Chiefs and referral staff. In developing recommendations for
addressing these situations, she noted the importance of considering IRG
workloads, the diversity of Institutes served, and the education of
applicants.

Dr. Schneider asked the Committee for its thoughts on the charge to be
given for the second mock sort. Dr. Matthews emphasized the importance
of identifying crosscutting areas that could be candidates for
clustering. Dr. del Nido suggested developing an initial list of
potential crosscutting areas that could be used in conducting the sort.
In addition, Dr. Schneider emphasized the critical role of the Committee
in making final recommendations to the Director about IRG and study
section boundaries, particularly in selecting from the options for
resolving shared interests.

Dr. Postow then asked members for guidance on what should be done with
the guidelines developed for SAIAB. Dr. Ehrenfeld noted that
representatives of the surgery community have made a convincing case for
reviewing their crosscutting applications together. She added that she
has also received e-mails from others, such as neurosurgeons, who want
their applications to be reviewed with others in their specific field.
Dr. Epstein cited previous discussions on innovation and noted that
community members might not recognize the value of setting up a
crosscutting venue for reviewing state-of-the-art research that crosses
disciplines. Dr. Matthews suggested adding a preamble to the proposed
SAIAB guidelines that explained concerns about the placement of surgery
applications. It would discuss the advantages of creating a core for
reviewing crosscutting applications while encouraging community comments
on specialty areas that should be reviewed in organ/disease-specific
review groups. Dr. Chien said that such a preamble would be useful. The
Committee agreed and formed a subcommittee, which included Drs. Matthews
and Soybel, who will consult with Dr. del Nido. This subcommittee\'s
draft preamble will be e-mailed to all CSRAC members for a final review.
The Committee also formed a subcommittee to develop options for
completing the Hematology guidelines. Dr. Michael Colvin and Dr. Kushner
volunteered to assist Dr. Michael Martin, Director, CSR Division of
Physiological Systems, in this effort.

**Multidisciplinary Applications**

Dr. Gillian Einstein, SRA, CSR Molecular, Cellular and Developmental
Neuroscience 2 Study Section, explained how an internal CSR committee
developed a response to the recommendations on multidisciplinary
applications discussed at the last CSRAC meeting. This committee
included SRAs from all CSR Divisions and was cochaired by

Dr. Einstein and Dr. Jean Sipe, SRA, CSR Bioengineering Research
Partnerships and Grants Special Emphasis Panel. Members of this
committee agreed on a working definition of multidisciplinary
applications, i.e., applications that could be assigned to more than one
IRG. They also recognized the existence of complex/multidisciplinary
applications that will always stretch the boundaries of existing study
sections. While the original recommendations called for modifying the
practices of existing study sections, this committee felt the overall
goals could be better achieved by developing special emphasis panels
(SEPs) for multidisciplinary applications. This mechanism was seen as
flexible and not likely to disrupt or be hindered by the culture of
established study sections. Creating SEPs for multidisciplinary
applications may also foster a greater appreciation for these
applications. Dr. Janet Newburgh, Deputy Director, CSR Division of
Receipt and Referral, conducted a pilot study that provided a
preliminary assessment. Her data indicates that multidisciplinary
applications make up approximately 20 percent of the total received.
Emerging areas include host-pathogen interactions, general
bioengineering, biomaterials, general surgical research, neuromuscular
interactions, genetics, epidemiology, behavioral sciences, and
neurobiology.

Dr. Newburgh then elaborated on her plans to assess the frequency and
fate of multidisciplinary applications better. She took a random sample
of 682 applications and found 232 of them could be assigned to two
different IRGs and 20 of them could be assigned to three IRGs. These
applications will be studied further to assess which IRG reviewed them
and how well they fared. Dr. Newburgh is also keeping track of
multidisciplinary applications that come to her attention because of
assignment difficulties. Dr. Einstein emphasized how these efforts will
be useful in piloting multidisciplinary reviews between IRGs and
developing appreciation for multidisciplinary applications in the
standing study sections.

Dr. Berget began discussions by saying there were two classes of
multidisciplinary applications: those that can be reviewed once the
expertise is located in the IRGs and those that require the recruitment
of reviewers from fields that have not been covered by NIH, such as
bioinformatics. She indicated that developing SEPs would be an
appropriate means to incorporate these fields. Dr. Michael Leon
commended the effort and mentioned the difficulty of reviewing an
application when no one reviewer understands all of its components. Dr.
Einstein suggested that the current model of a primary reviewer as
having primary responsibility for an application should be set aside to
review these applications. Reviews will need to be more group-based. Dr.
Colvin added his support to the effort and asked if the proposed SEPs
would have temporary or standing members. Dr. Einstein explained that
CSR might consider developing standing SEPs that would serve multiple
IRGs with a stable core of reviewers thus providing continuity of
review.

**Streamlining**

Dr. Zakir Bengali, Chief, CSR Biochemical Sciences IRG, described the
current practice of streamlining. The SRA asks assigned reviewers to
nominate applications for streamlining---usually those that would have a
priority score of three or worse. At the start of the study section
meeting, the nominations for streamlining are presented to the
committee. If all study section members concur, the applications are not
discussed and not scored. Applicants receive summary statements that
only contain reviewer critiques. Dr. Bengali then summarized concerns
CSRAC has raised about this practice:

\(1\) reviewers in conflict remain in the room during the unscoring
process, (2) unassigned reviewers are uncomfortable with unscoring
applications they have not discussed,

\(3\) program staff are not told why applications are unscored and
cannot properly advise applicants, and (4) study sections that strictly
force unscoring at the 50 percent level may unscore meritorious
applications.

Responding to these concerns, CSR recently completed a pilot test of
modified streamlining procedures. Nine study sections participated in
this pilot. Prior to the meeting, the reviewers were asked to send their
nominations for streamlining. As each of these applications came up for
review on the meeting agenda, the reviewers briefly presented their
reasons for unscoring the application. Reviewers in conflict left the
room. SRAs later added a brief resume on the unscoring decision to the
summary statement. In discussing the results, Dr. Bengali explained that
some study sections strongly approved of the revised process while
others strongly disapproved. Overall, the study section members felt
more comfortable voting to streamline applications when they understood
the reasons for doing so. Discussing the reasons for streamlining,
however, did not lead reviewers to change their critiques. Most study
sections did not feel these extra discussions were a burden, though a
few thought they could be a burden to study sections that review large
numbers of applications. Program staff largely approved of the piloted
changes. CSR has not yet received enough feedback to know if these
changes helped applicants in revising unscored applications. SRAs were
evenly divided in their opinions of the revised practices. They also had
concerns about the burden to study sections that review large numbers of
applications, the limits of producing resumes that do not discuss
positive attributes, and the usefulness of the abbreviated discussions.

Dr. Colvin suggested that the revised practices could address some
community concerns about streamlining. Dr. Berget emphasized the
importance of assessing the value of these revised practices on the
applicants, particularly first-time applicants. She also suggested
having the primary reviewers provide scores so applicants could better
gauge reviewer sentiment. Dr. Matthews noted that there was a need to
help young investigators understand that an unscored application is not
a rejected application.

Dr. Williams discussed how unscoring affects senior reviewers as well.
Without scores, they are less able to know if it would be worthwhile to
revise and resubmit their applications.

Dr. Leon said that problems are inherent in any compromise; however, he
suggested that the new Internet-assisted review system could allow
reviewers to read critiques of applications proposed for streamlining
and submit preliminary scores. Providing summary statements to these
applicants sooner would allow them to revise and resubmit applications
for the next round. Dr. Ehrenfeld agreed that it would be good to
complete all summary statements in time to permit applicants to resubmit
them in the next round. She explained, however, that it takes time to
carefully edit reviewer critiques. Speeding the completion of summary
statements for unscored applications is complicated by the fact that the
ICs want CSR to release the best applications first so they can be
quickly reviewed via their electronic council system. She emphasized the
importance of doing what can be done and educating the community. Dr.
Chien also endorsed the idea of conducting a preliminary scoring round
and suggested writing the resume for unscored applications ahead of
time. They would only be discussed at the meeting if disagreements
existed. Dr. Soybel added that applicants really want to know why their
applications were unscored. He suggested devising a checklist that
reviewers could use to indicate why an application was unscored:
problems with preliminary data, track record, or resources and
environment. Such information could help an applicant decide if
resubmission was worthwhile.

Dr. Matthews summarized the discussions, saying that the Committee
remains concerned that the streamlining mechanism is used in the best
fashion possible. The Committee appreciated CSR\'s effort to evaluate
modifications to the current practice. She noted its limited successes
and discussed the possibility of securing preliminary scores through the
electronic review system. She emphasized the importance of providing
written reviews to applicants so that they can determine whether or not
their unscored application can be revised successfully. Dr. Ehrenfeld
added that the study sections surveyed in the pilot were relatively
small, and larger study sections could find the proposed changes more
burdensome. She suggested that it might be worth discussing new
procedures that would not consume review meeting time, such as giving
reviewers checklists or producing resumes ahead of time.

**Identifying Innovation in Grant Applications**

Dr. David Armstrong, Acting Chief, Brain Disorders and Clinical
Neuroscience IRG, updated members on efforts to advance the
identification of innovative grant applications. CSR took a number of
actions after the Committee\'s last meeting, which included a talk on
this topic by Dr. Robert Sternberg of the Yale Center for Psychology of
Abilities, Competencies, and Expertise. Drs. Ehrenfeld and Stanfield
spoke with the directors of three neuroscience Institutes: the National
Institute of Neurological Disorders and Stroke, the National Institute
of Mental Health, and the National Institute of Drug Abuse. All of them
agreed to seek the approval of their advisory councils to participate in
an experiment to solicit and review innovative grant applications. Drs.
Ehrenfeld and Stanfield also asked the IC Directors to nominate
community representatives for a workshop to obtain their ideas on this
initiative.

Setbacks, however, occurred in the wake of the September 11 tragedy. The
issue was not discussed at the respective council meetings. In addition,
the directors of the three participating Institutes have left the NIH
for other positions. Dr. Armstrong, however, continued to work with the
CSRAC subcommittee formed to advance this initiative:

Drs. Pugh, Epstein, and Rothman-Denes. In addition, Dr. Armstrong worked
with associate directors and program staff from the three participating
Institutes.

One of the challenges to fostering innovation is to define the term more
clearly.

Dr. Armstrong explained how NIH guidelines for evaluating innovation in
grant applications are tautological: \"Does the project employ novel
concepts, approaches, or method? Are the aims original and innovative?\"
He also noted problems in how Program Announcements define the review
criteria for exploratory/development grants (R21s). Innovation is
often not the key criteria, but one of many. Investigators therefore may
propose to carry out important work that by its nature is not
innovative.

Dr. Armstrong then presented the criteria the subcommittee proposed for
the preparation and receipt of innovative applications: (1) Grants would
be limited to \$200,000 in direct costs per year for no more than 3
years. (2) Applications would be limited to five pages, and no
preliminary data would be necessary. (3) Grants would not be renewable,
with the understanding that the principal investigator (PI) could
compete for R01 funding.

\(4\) There would be two receipt dates a year. (5) The review should be
expedited.

\(6\) Amended applications could only be submitted if requested by the
peer review panel.

Dr. Armstrong asked CSRAC members for advice on addressing outstanding
questions: (1) Should the definition of innovative research be redefined
to aid applicants and reviewers? (2) Can a system be developed for
evaluating the innovative qualities of applications? (3) Should
innovative applications be reviewed in standing or new study sections?
(4) Should institutions be limited in the number of innovative
applications per review cycle? (5) Should the target community for this
initiative be universities or industry?

Dr. Armstrong then asked members if this initiative should advance, if a
pilot with neurosciences research was best, and if two or three review
groups should be formed simultaneously. He proposed organizing a
workshop including innovative neuroscientists and appropriate community
leaders to discuss innovation and how NIH can best nurture it.

Dr. Pugh said that ordinary study sections were too conservative to
review innovative grant applications appropriately, and therefore a new
review venue is necessary. He also raised a concern about RFAs, which he
said tended to operate in a \"closed system\" that favored certain
institutions. Dr. Williams emphasized the need to recruit workshop
members who are not tied to the system. Dr. Colvin echoed these
comments, explaining that NIH needs to encourage researchers to submit
innovative ideas that will not be dismissed simply because they are
difficult to prove.

Dr. Schachman pointed to the field of structural biology and emphasized
how it has advanced dramatically through many incremental steps and NIH
awards to investigators who did excellent work. Creative leaps such as
the development of polymerase chain reaction (PCR) are rare. Dr.
Schachman suggested that current practices should not be downgraded and
cautioned about the risks of funding unproductive research. All
investigators who submit an R01 application believe they are proposing
to make some technological or conceptual innovation. He indicated that
the problem is that current study sections have a hypercritical attitude
that needs to be addressed. Dr. Epstein added that there would be a
larger payoff if something is done to improve how standing study
sections view innovation instead of setting up special review groups.
Dr. Williams suggested that this was not an either/or situation and
risking small amounts of money as proposed in the pilot was worth
considering.

Dr. Ehrenfeld focused on the key question before the Committee: is it
worthwhile to conduct the proposed pilot? She explained that the only
way to receive and review higher-risk and higher-impact applications is
to set up a new application and review process. Applicants need to be
convinced that NIH is serious about funding innovative research, and
reviewers of these applications need to be freed from the conservative
nature of existing study sections that are geared to reviewing
traditional R01 applications.

Dr. Ehrenfeld noted the setbacks Dr. Armstrong mentioned earlier, and
she added that large portions of the budget are being dedicated to
bioterrorism and related initiatives. She suggested that this might not
be the right time to proceed with this pilot, though things may open up
later.

Dr. Matthews asked the Committee if it thought the proposed pilot was
worth pursuing when the time is right. After an informal poll, she
concluded that the Committee was fairly enthusiastic. She encouraged Dr.
Ehrenfeld to make an executive decision.

Dr. Ehrenfeld said that she and Dr. Armstrong would examine the research
community to be targeted in this pilot. She explained that AIDS might be
a more promising area to target. CSR will examine the possibilities and
monitor the overall developments at NIH over the next few months.

**IRG Working Group Reports**

Dr. Kushner discussed the development of IRG Working Groups. Their
purpose was to provide advice on the organization, management, and
leadership of each IRG. These groups were made up of 5-10 eminent
scientists and veteran reviewers. They observed and participated in
study section meetings and then reported their findings to CSRAC. All
the Working Groups have now submitted their reports. CSR has since
produced a summary of these reports. Dr. Kushner explained that CSRAC
members were being asked to review the summary to see if it should be
modified before being released.

Dr. Martin provided an overview of the summary of the IRG Working Group
reports.

*Scope and Breadth of the Science Reviewed*

The Working Groups agreed that the optimal number of applications
assigned to a study section was between 60 and 80. Study sections with
less than 50 or more than 90 applications functioned less well or had
many management problems. A few study sections consider two or three
distinct and quite unconnected areas of science. Only those with a good
chair and SRA were successful in dealing with this challenge. Concerns
were raised about IRGs that cover the neurosciences, endocrinology,
nutrition, and metabolic science areas with study sections that are too
small. Dr. Martin explained that he will encourage the SSB Team for
endocrinology, metabolism and reproductive sciences to develop
appropriate-sized study sections when it meets in April. He added that
CSR has implemented a number of Working Group recommendations. The
Pathology C Study Section was created; workloads within the Molecular,
Cellular, and Developmental Neuroscience IRG were rebalanced; and other
adjustments are being made. Larger changes are expected as the
reorganization process advances.

*Appropriateness, Qualifications, and Stature of Reviewers*

The Working Groups found that reviewers overall were of a high caliber.
The study sections had senior reviewers, but more should be recruited.
Increased efforts were also needed to increase reviewer diversity. Dr.
Martin said that CSR has been working with the NIH Committee Management
Office to develop flexible service terms for recruiting reviewers.

*Policies, Procedures, and Management of the Meeting*

CSR was encouraged to involve the chair and SRA more in training new and
temporary reviewers and to provide training to the chairs in consensus
building. Dr. Martin said that CSR also was encouraged to involve the
chairs more in assigning applications, while not letting reviewers
select the applications they review. The Working Groups found that
review quality drops dramatically when reviewers are assigned more than
10 reviews. Problems also arose when more than four reviewers attended
the meeting via the telephone and when reviewers were not acquainted
with outside reviewers. The Working Groups favored Internet-assisted
peer review, but they had concerns about the practice of not scoring
applications and the use of modular budgets. Study sections tended to
have problems complying with new policies for reviewing clinical
research when a third or less of its applications involved this kind of
research. Some study sections also had problems with scores clustering
near the payline. CSR is exploring training options, advancing a best
practices document for SRAs, and working with the NIH Office of
Extramural Research to address concerns about modular budgets. Later in
the meeting, Dr. Soybel proposed a mechanism for spreading clustered
scores: creating subcategories where reviewers are required to assign
scores.

*Accommodation of New Directions and Emerging Areas*

The Working Groups reported that the study sections saw few novel
applications, yet the ones they did see were reviewed fairly. Dr.
Martin, however, noted that the Working Groups were concerned about the
tendency of newer reviewers to dwell excessively on methodological
concerns.

*Fairness of the Reviews for All Grant Mechanisms*

Applications for R21 and R03 grants were found to be reviewed fairly.
The Working Groups encouraged CSR to clump the review of applications
for R15 area awards. SRAs attempt to cluster the review of these
applications when reviewer and program staff schedules permit it. In
response to Working Group recommendations, CSR has developed study
sections dedicated to review fellowship applications, which had been
reviewed in regular study sections.

*Questions to Consider*

Dr. Martin then posed questions for CSRAC to consider: Was this effort a
success? Have all of the important topics been covered appropriately?
Does the summary report capture them? How do we reach out to the
research community so it can understand and appreciate this effort? He
also reminded the Committee that seven Working Group reports await their
final consideration: (1) Behavioral and Bio-Behavioral Processes;

\(2\) Endocrinology and Reproductive Sciences, and Nutritional and
Metabolic Sciences; (3) Infectious Diseases and Microbiology; (4)
Pathophysiological Sciences; and

\(5\) Social Sciences, Nursing, Epidemiology, and Methods, (6) AIDS and
AIDS-Related Research, and (7) Risk, Prevention and Health Behavior.

Dr. Kushner said the summary addresses the important areas covered by
the Working Group reports. It adequately describes how CSR has brought
closure to some issues, and it appropriately states options being
considered to address remaining issues. He thus recommended posting the
summary on the CSR Web site. Dr. Leon agreed and suggested that e-mails
containing the corresponding URL should be sent to Working Group
participants, reviewers, and other interested parties, such as those at
various organizations.

Dr. Pugh also agreed that the document should be released. He suggested
that more emphasis could be given to the selection and training of study
section chairs who can skillfully balance the dynamics of a review
meeting. He also recommended a greater emphasis on the need to encourage
new investigators. Dr. Rothman-Denes added that it was important to
define "new investigators." Dr. Martin explained that CSR's best
practices document defines them in terms of where they are in their
career.

Drs. Rothman-Denes and Leon approved of this approach, but proposed that
the Committee discuss it at a future meeting. Dr. Chien pointed to a
sentence in the document that said chairs were "less accomplished at
developing consensus." He noted that there are many skilled chairs and
suggested saying that "some" chairs were less accomplished in this area.

Dr. Marvin Wickens asked about the distribution of the Working Group
reports.

Dr. Martin explained that the chairs and SRAs of the study sections
should receive copies if they have not already received them. He
emphasized that, once the summary document is complete, it will be
disseminated to all Working Group and study section members so that the
community will understand the important issues that have been identified
and the actions CSR has taken.

Dr. Mathews said the document shows how well communications have been
fostered between constituencies and how well CSR has responded to their
concerns. She then asked members to consider the Working Group reports
completed since the last meeting. One new issue that surfaced is keeping
reviewers focused on the science while assessing factors related to the
use of human subjects and the inclusion of women, minorities, etc. A
second issue relates to ensuring that reviewer talents are fully used
while keeping the process fair. Dr. Martin explained the best practices
document will address this latter issue. He added that Congress has
proposed a new review format that will require reviewers to discuss and
make specific comments on defined aspects of each application. Dr.
Matthews suggested that the Committee discuss this proposed change, and

Dr. Ehrenfeld said that such input would be appreciated. Dr. Pugh said
the reports under consideration could be improved with the addition of
subheadings and links to an acronym glossary.

Dr. Wickens asked what happens after the Working Group reports are
distributed to the IRGs and chairs. Dr. Ehrenfeld explained that CSR
follows up on each recommendation, with IRG Chiefs and Division
Directors addressing recommendations that are specific to their IRGs or
study sections. She also calls the Working Group chairs after their
reports are sent to them to discover if there were important issues that
did not get in the report. CSR has not sent out additional
communications to Working Group members, but

Dr. Ehrenfeld acknowledged that more could be done. She agreed that Dr.
Pugh had a good idea when he suggested that master reviewers could play
an important role in future Working Group evaluations. Dr. Ehrenfeld
then explained how the Working Group reports are being used to inform
the SSB Teams as they design new IRGs.

Dr. Matthews said that the Committee seemed to be in consensus on
releasing the summary report with only minor modifications. She then
focused on the new Working Group reports. They looked good to her and
were consistent with the ideas the Committee has heard before. She asked
members if they had additional comments.

Dr. Epstein agreed with her assessment. With no objections, Dr. Matthews
said that the Committee appeared to accept all of the Working Group
reports.

Dr. Soybel noted that the minutes of the last CSRAC meeting mentioned
that 40 percent of researchers who receive R01 funding do not receive a
renewal. Considering the amount of training a researcher needs to get
funded, this dropout rate is significant. He suggested that future
Working Groups investigate the situation and that NIH should collect
career data on these individuals. Dr. Ehrenfeld agreed that such data
could be useful; however, she explained that the NIH Office of
Extramural Research is the appropriate office to address this issue.

Dr. Matthews noted that a number of Working Groups made fairly strong
recommendations about changes to the composition of study sections. She
suggested that the proposed summary include mention of these
recommendations. She concluded the discussion by saying that all of the
Working Groups did a commendable job and they should be pleased with
CSR\'s response.

***Tuesday, January 29***

**Electronic Review Administration**

Dr. Richard Panniers, SRA, CSR Physiological Chemistry Study Section,
provided an update on three electronic initiatives at NIH: (1) CD-ROMs
that replace paper duplicates given to reviewers, (2) the Commons-2
database that will permit e-grant submissions, and

\(3\) the Internet-assisted review system that allows reviewers to post
and view critiques.

*CD-ROMs*

The CDs contain scanned copies of a reviewer\'s applications, which can
be sorted by PI or application number. They also contain Program
Announcements, RFAs, and reviewer guidelines related to the meeting, as
well as a user guide. Dr. Panniers explained that reviewers can search
for keywords in the applications and navigate through them using
bookmarks associated with the table of contents. He explained that
applications are automatically scanned when they are copied on NIH
digital copiers. The resulting images are thus identical to black and
white photocopies. A total of 17 study sections are currently using CDs,
and additional study sections will use them as capabilities are
developed. Dr. Panniers discussed data from a survey of reviewers in the
AIDS IRG who are using CDs. Ninety-nine percent of the 156 reviewers
surveyed liked them. Twenty-two percent said they would read more
applications on a CD than they would on paper; 29 percent said they
would read fewer applications using a CD; and 55 percent said using a CD
would not affect how many applications they would read. He explained how
scanning applications is a prelude to electronic application submission
and discussed plans for piloting the use of e-books for reviewing
grants.

*Commons-2 and E-Grant Applications*

The database CSR uses to process applicant data (IMPAC II) will be
integrated with the Commons-2 database being developed. The current
Commons-1 database permits about 200 institutions and 2,000 individuals
from those institutions to access information on the status of their
applications. The new Commons-2 database will also give them access to
their summary statements. Plans call for Commons-2 to open registration
to all institutions later this year. A feature is being built to allow
registrants to submit type 5 progress reports. The external and internal
experts on the Commons Working Group are also redesigning the 398 grant
application form to facilitate electronic submissions via the Commons-2
system. Plans call for a pilot to be conducted in 2003 with a bigger
ramp-up in 2004. Dr. Panniers noted a few outstanding questions: How
will appendices be handled? \[The Commons Working Group has since
decided that the appendix will be transmitted as a PDF file.\] Should
CSR still give reviewers paper copies of assigned e-applications? Will
reviewers be reimbursed if they want to print their applications? Will
reviewers download their applications from a secure server? Should CSR
supply all reviewers e-books when screen technologies advance? Dr.
Panniers suggested that e-grants will speed parts of the referral and
review processes. For instance, the data-entry process will no longer be
needed, and there might be more self-referral. Some applicants may
receive their summary statements sooner and thus be able to resubmit
their applications in the next round. Since some study sections meet
later in the cycle than others, quick turnaround times cannot be offered
to all applicants, which might generate potential inequity.

*Internet-Assisted Review*

Seventy-seven of the 155 large study sections are using the
Internet-assisted review system for submitting critiques via the
Internet. After a certain deadline, reviewers who have submitted
critiques via this system can view all the others except those where a
conflict exists. Reviewers can edit their critiques after the meeting,
and the grants technical assistants can capture the critiques to
assemble summary statements. A total of 136 study sections are expected
to use this system in the next review round. Dr. Panniers explained how
he surveyed the SRAs currently using the system. Eighty-three percent of
those who responded said their reviewers were satisfied or very
enthusiastic about it. Fifteen percent had a more mixed response, and
there was a cadre of reviewers who did not want to change their routine.
There was a compliance rate between 60 and

100 percent. Twenty-eight SRAs reported that their reviewers are able to
focus their discussions better. Fifteen SRAs reported that their
reviewers are more confident about streamlining or they have increased
the number of applications they streamline.

Dr. Panniers then discussed some of the negative comments. A few SRAs
reported that some reviewers were changing their opinions without
sharing their reasoning. Two SRAs said that the system utilized more of
their time, and a more elaborate system could demand even more. All in
all, he thought most of the SRAs would say they and their reviewers were
better prepared for their meetings.

*Future Considerations*

Dr. Wickens said that he was pleased with the work that had been done.
He was particularly interested in how electronic review administration
may speed the review cycle. Dr. Panniers noted that improvements to the
IMPAC II system have eliminated a 1-day delay in securing paper copies
of summary statements. The new Commons-2 database will also provide
applicants rapid access to summary statements via the Web. Drs. Leon and
Chien also expressed enthusiasm for these efforts. Dr. Wickens asked how
the Internet-assisted review system affects review discussions. He
specifically wondered if it might reduce the value of the discussion.
Dr. Panniers agreed that this was an issue worth considering further.
Dr. Chien suggested that the Internet-assisted review system will allow
reviewers to focus their discussions better. If primary and secondary
reviewers see they agree an application falls either at the high or low
end, they can reduce the time spent discussing it and devote more time
to the grants that need more discussion.

Dr. Berget mentioned difficulties her office of research had in helping
PIs deal with the new electronic submission systems used by the National
Science Foundation (NSF) and the Howard Hughes Medical Institute. She
emphasized the importance of having a competent 24/7 help desk. Dr.
Stanfield noted that NIH has recruited an individual who was
instrumental in implementing NSF\'s system, and NIH hoped to benefit
from the lessons he learned. Dr. Berget suggested that CSR provide
advance information on the new system to the institutions so that they
will be better prepared to assist their PIs.

Dr. Stanfield said that NIH has involved representatives from the
sponsored research offices at major universities. Dr. Ehrenfeld
explained that one of the reasons why NIH is taking so long to implement
this system is that it wants to be better prepared. She said that the
scanning initiative has been an important transitional effort, since CSR
has learned a great deal about how to handle large volumes of electronic
material.

Dr. Matthews reminded the Committee how it recommended increased support
for NIH electronic administration in September 2000. She said she was
pleased with the progress that has been made since.

**Survey Activities**

Dr. Michael Micklin, Chief, CSR Risk, Prevention and Health Behavior
IRG, provided members draft copies of two follow-up surveys and related
materials. He developed them with a CSRAC subcommittee, which also
included Drs. Matthews and Epstein as well as Dr. Yvette Davis, SRA, CSR
Social Sciences, Nursing, Epidemiology and Methods 2. The first survey
will assess the time reviewers spend preparing for a study section
meeting. Four hundred reviewers will be selected randomly from the IRGs
and asked to complete a diary-based survey to assess the time spent (1)
preparing for the review, (2) reading applications and taking notes, and
(3) writing and revising critiques. The second survey will seek to
identify the incentives and barriers to study section service. Two
hundred current study section members and 300 R01 grantees with 3 or
more years of funding will be selected randomly from NIH and CSR
databases. They will be asked about their response to being invited to
serve on a study section and to future invitations to serve as an ad hoc
or regular reviewer. Pilot studies will be conducted to test and perfect
these two instruments.

Dr. Leon questioned how valid the resulting data would be. He said that
respondents are apt to lie about why they decline study section service.
In the case of the time-allotment diary, respondents are likely to
complete it after the fact. Dr. Leon suggested, however, that accurate
numbers might not be necessary. Dr. Matthews said that the
time-allotment form was designed to maximize compliance. Dr. Pugh also
questioned this study, suggesting that the time required to review
applications is less important than other factors in recruiting
reviewers. In addition, he suggested studying how time allotment affects
the quality of reviews. Dr. Micklin said that there might be ways to
assess review quality.

Dr. Soybel said that it was important to know how much time reviewers
spend preparing for their meetings. It may be useful to reviewers
contemplating service and also to CSR in assessing the value of the
services it receives and the compensation it provides.

Dr. Soybel then proposed focusing on particular grants and assessing the
time the assigned reviewers devoted to them. Increased data points would
allow CSR to investigate additional factors, such as the influence of
years of service. He added that the burden of this study could be
reduced if reviewers were only asked to report on the time devoted to
reviewing one grant application. Dr. Chien expressed concerns about
compliance and then suggested a few format changes to the
incentives/barriers survey. He also suggested that an additional barrier
be added to the list: \"service related to professional societies.\" Dr.
Stanfield suggested another: \"clinical responsibilities.\"

Dr. Williams encouraged CSR to consult with survey administration
experts.

Dr. Matthews concluded the discussion by saying that members had made
some excellent suggestions. Noting that there were no emphatic
objections, she suggested that CSR proceed with plans to develop pilots.
She invited members to examine the survey materials more closely and
forward any comments to Drs. Micklin and Davis.

**Fellowship Study Sections**

Dr. Richard Rodewald, SRA, F05 Fellowship Study Section, summarized the
events that led to the creation of new fellowship study sections. A
CSR-commissioned report by

Dr. Maxine Lineal of the Fred Hutchinson Cancer Research Center
recommended that fellowship applications be reviewed in dedicated study
sections. A CSR fellowship committee was established, with members from
the NIH Training Advisory Committee, the ICs, and CSR. Mock sorts of
fellowship applications by the CSR Division of Receipt and Referral led
to the development of 12 dedicated fellowship study sections. These
review groups recently completed their first review round. A total of
687 fellowship applications were received. Eighty-two percent of them
were reviewed by CSR, and

88 percent of these were reviewed in the new fellowship study sections.

A number of concerns have surfaced. Dr. Rodewald noted the difficulty of
covering a broad area while having adequate expertise to score
applications. Another difficulty is dealing with applications that do
not fit into any of the fellowship study sections, such as clinical
research applications. He also mentioned the difficulty of achieving
scoring consistency across the fellowship study sections. There were
tremendous variations in scoring among the fellowship study sections.
Three ICs have requested percentiles for their fellowship applications
to inform their secondary review, and the NIH Training Advisory
Committee may soon encourage all ICs to calculate percentiles. He said
that there was a debate as to whether percentiles should be released to
the applicants or kept confidential for internal IC use only.

The NIH Training Advisory Committee expects to propose an NIH-wide
policy on percentiling fellowship applications. Dr. Rodewald then
emphasized the importance of evaluating the new fellowship study
sections. CSR will begin by arranging a meeting of the fellowship SRAs
and other CSR and IC staff members, including representatives of the
Training Advisory Committee. Additional input will be sought from
reviewers, applicants, and the scientific community. CSR will likely
perform a formal evaluation of these study sections once they become
more firmly established.

Dr. Berget suggested that releasing percentile scores would help new
applicants see the review process as something fair and transparent. She
noted that there may not be enough applications to maintain all the
fellowship study sections. CSR should thus monitor the number of
incoming applications to determine if modifications are necessary. Dr.
Berget then cited the large formatting problems associated with these
applications and suggested that CSR work with post-doc associations or
develop targeted Web pages to educate applicants. Dr. Soybel cautioned
NIH about releasing percentiles. The same percentile scores may not
represent the same quality in two study sections, and releasing
percentile and payline information may be problematic. Dr. Chien noted
that 5 of the 12 fellowship study sections were devoted to the
neurosciences, but none of them seemed to cover crosscutting areas, such
as bio-computing, bioimaging, and bioengineering.

Dr. Ehrenfeld explained that the study sections were developed after
long discussions with many individuals, including CSRAC members. She
expects that CSRAC will be consulted as these study sections are
evaluated and modified according to evolving needs.

Dr. Matthews encouraged CSR to assess how well fellowship applications
fare when they are reviewed in regular study sections and when they are
reviewed in the new fellowship study sections. Dr. Soybel complemented
Dr. Rodewald on the work that has been done to develop these study
sections. He added that his review group has found it difficult to deal
with a diverse group of applications. As the diversity will vary from
round to round, it will be all the more difficult to develop a standing
panel of reviewers who are well equipped to handle them. Dr. Matthews
recalled the difficulty regular study sections had in reviewing
fellowship applications. CSR was moving in the right direction, and she
said she looked forward to hearing how this initiative progresses.

**Document on How Scientists Are Selected for Study Section Service**

A draft document on How Scientists Are Selected for Study Section
Service was distributed to members for comment. Dr. Ehrenfeld explained
that CSR intended to format this document for posting on CSR\'s Web
site. Doing so is part of an ongoing effort to help the community
understand how CSR works. Dr. Berget suggested modifying the document to
include a general statement on conflict of interest. Dr. Chien proposed
mentioning that one of the requirements for service is a commitment to
the NIH mission. Dr. Matthews said that this document would be very
useful in educating junior colleagues. Dr. Ehrenfeld told members that
CSR would incorporate the comments made by Drs. Berget and Chien before
posting it.

**Agenda Items for the Next Meeting**

Dr. Matthews reviewed possible agenda items for the next meeting: (1)
the development of the new Hematology IRG, (2) useful practices for
making streamlining work,

\(3\) training and selecting of study section chairs, (4) defining and
identifying new investigators, (5) CSR contingency plans for
emergencies, (6) an update on the innovation initiative, (7) the mock
study section video, (8) the longevity and retention of PIs, (9) format
options for reviewing applications that involve the use of human and
animal subjects and the recruitment of women and minorities, (10)
addressing the special needs of postdoctoral applicants, and (11)
shortening the review cycle so applicants can resubmit for the next
round.

With no other business to address, Dr. Matthews thanked everyone for
their efforts and adjourned the meeting at 10:52 a.m.

We do hereby certify that, to the best of our knowledge, the foregoing
minutes are accurate and complete. The minutes will be considered at the
29th meeting of the Advisory Committee, and any corrections or comments
will be made at that meeting.

> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Brent Stanfield, Ph.D.
>
> Executive Secretary
>
> Center for Scientific Review Advisory Committee
>
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Ellie Ehrenfeld, Ph.D.
>
> Director
>
> Center for Scientific Review
>
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Karen Matthews, Ph.D.
>
> Chair
>
> Center for Scientific Review Advisory Committee