Center for Scientific Review Advisory Committee Meeting

National Institutes of Health

May 14-15, 2001

The Center for Scientific Review (CSR) convened the 27th meeting of its
Advisory Committee at 8:30 a.m. on Monday, May 14, 2001, in Conference
Room 9100, Rockledge II Building. The entire meeting was convened in
open session. Dr. Karen Matthews presided as Chair.

## Members

Karen Matthews, Ph.D., Chair Susan Berget, Ph.D.

Shu Chien, M.D., Ph.D. Michael Colvin, M.D.

James Kushner, M.D. Raphael Pollock, M.D., Ph.D.

Lucia Rothman-Denes, Ph.D. Tadataka Yamada, M.D.

## Ad Hoc Advisors

Leonard Epstein, Ph.D. Michael Leon, Ph.D.

Edward Pugh, Jr., Ph.D. Marvin Wickens, Ph.D.

Dr. Howard Schachman was the official observer, and Dr. Robert Eisinger
was the Executive Secretary for the meeting.

## Welcome and Opening Remarks 

Dr. Matthews welcomed members and asked everyone at the table to
introduce themselves for the benefit of new members. She then asked Dr.
Ellie Ehrenfeld, Director, CSR, to provide her update.

**CSR Update**

Dr. Ehrenfeld noted that the new Secretary of the Department of Health
and Human Services (DHHS), the new chairman of the House appropriations
committee, and new members of Congress recently visited the National
Institutes of Health (NIH). She then discussed how the new
administration has questioned the funding of human stem cell research.
DHHS instructed CSR to postpone the April 25 meeting of the Pluripotent
Stem Cell Review Group, which was established under the CSR Advisory
Committee (CSRAC) to ensure that any Government-supported study that
would use these cells complies with Federal guidelines. Dr. Brent
Stanfield, CSR Deputy Director, explained that DHHS is conducting an
internal review of the legal opinion that previously permitted funding
of human stem cell research. The Department also has asked the NIH
Office of Science Policy to submit a report on the potential uses of
these cells. DHHS hopes to make a decision in June 2001. Dr. Ehrenfeld
said that she would advise members of future developments in this area.

She then focused on the new National Institute of Biomedical Imaging and
Bioengineering (NIBIB). The Head of CSR\'s Receipt and Referral Office,
Dr. Suzanne Fisher, has met with

Dr. Donna Dean, Acting NIBIB Director, to begin addressing grant
referral issues.

Dr. Ehrenfeld proposed inviting Dr. Dean to a future CSRAC meeting when
she is more able to discuss NIBIB\'s impact on CSR review activities.

Dr. Ehrenfeld continued by explaining that CSR has requested a 13.5
percent increase in its fiscal year (FY) 2002 budget to address its
increased workloads due to growing NIH budgets, new Institute
initiatives, and the increasing complexity of the scientific areas it
services. Additional resources are also needed to implement and evaluate
CSR\'s ongoing reorganization of its study sections and to develop new
business practices for the electronic receipt and administration of
grant applications. She noted that the NIH Electronic Research
Administration Committee received an extra \$30 million last year, and
its FY 2002 budget request seeks a substantial investment above its FY
2001 budget.

Dr. Ehrenfeld reported on CSR\'s efforts to recruit new Scientific
Review Administrators (SRAs) to handle its expanding workload and
address the usual staff turnover. Approximately a dozen new and highly
qualified SRAs have been hired since the beginning of the year. She then
provided an update on the new 2-year internship program for training
individuals interested in research administration. These trainees will
offer CSR flexible workload assistance and serve as a potential source
of trained applicants for future SRA positions. A pilot program has been
initiated within the NIH intramural community. CSR received applications
from 29 highly qualified applicants and is interviewing 10 of them with
the intention of hiring 6 trainees. The program will begin on August 1.
If it is successful, CSR will hire an additional 6 trainees next year
and work to open the program to scientists from the extramural
community.

An update was also provided on CSR\'s efforts to enforce format
requirements for grant applications. Dr. Ehrenfeld noted that CSRAC
members earlier approved a plan developed in concert with NIH Institutes
and Centers (ICs) to address increasing reviewer complaints about
applications that use inappropriately small font sizes and crowded
spacing. She also noted that format compliance is critical to advancing
efforts for scanning and processing applications electronically. Before
the last review round, CSR informed applicants via an *NIH Guide* notice
that it would return incompliant applications and defer them to the next
round. Of the total 14,000 applications received in this round, 220 were
returned for format problems. Dr. Ehrenfeld explained that complaints
from affected applicants and congressional staffers led CSR to modify
its enforcement policy. When noncompliant applications are received,
investigators will be informed and given 4 business days to resubmit
reformatted applications or they will be deferred. If reviewers or CSR
staff later identify noncompliant applications, they may still be
returned or deferred. Drs. Tadataka Yamada and Shu Chien emphasized the
value of having a consistent and fair policy and expressed concern about
the message CSR is providing applicants by changing this policy. Dr.
Ehrenfeld cited the hardships that resulted from deferring applications
and explained that CSR was trying to send the message that it is
responsive and accommodating. Drs. Lucia Rothman-Denes and Matthews
joined the other CSRAC members in supporting the original policy. Dr.
Ehrenfeld acknowledged their sentiments before discussing efforts to
revise the instructions for the Public Health Service (PHS) 398 form.

The Office of Management and Budget recently approved revised
instructions that NIH developed for the PHS 398 form. Dr. Ehrenfeld had
hoped that the revisions could have been reviewed more thoroughly by the
outside community. She was, however, able to have a copy sent to Dr.
Matthews a few days before the document was completed. Dr. Matthews
explained that she found the document to be very thorough and that she
was only given the opportunity to identify a few minor changes.

Dr. Ehrenfeld then focused on CSR\'s ongoing external reviews of all its
study sections. Experts are being used to evaluate the study sections
within each integrated review group (IRG). These working groups are
assessing how well emerging fields of research are handled and how well
the study sections are functioning. She explained that evaluations have
been completed for 14 of the 20 existing IRGs, and CSRAC will receive
reports from a couple of working groups at its next meeting. To date,
these evaluations have been very useful for adjusting the boundaries of
the study sections and improving the review process in general.

In concluding her update, Dr. Ehrenfeld discussed efforts to improve the
review of fellowship applications. She reminded members they had
previously endorsed a plan to reestablish separate fellowship study
sections for reviewing these applications. After meeting with
representatives from the Institutes and others, CSR is working to
establish approximately 10 scientifically dedicated fellowship study
sections, which will be piloted in October 2001.

**Comments of the Chair**

Dr. Matthews said she was pleased to see progress on some of the issues
CSRAC members have addressed previously. She then called for the
approval of the minutes from the January 22-23, 2001, CSRAC meeting. Dr.
Chien seconded the motion, and the minutes were approved.

**Panel on Scientific Boundaries for Review (PSBR) Activities and
Implementation Issues**

Dr. Michael Martin, Director, CSR Division of Physiological Systems,
summarized efforts to develop the new Hematology IRG. A steering
committee of NIH staff was convened last July to identify this IRG\'s
scientific areas and nominees in these areas who could serve on the
Hematology Study Section Boundary (SSB) Team. A number of different
societies proved helpful in identifying nominees. SSB team members were
successfully recruited and given (1) a set of application abstracts CSR
referral staff sorted out of the May 2000 council round, according to
PSBR guidelines, for hematology sciences, (2) other abstracts the IRG
Chiefs thought should be considered from a review of the abstracts in
their areas, (3) comments the IRG Chiefs had on relevant abstracts, and
(4) the PSBR report. The SSB team met in February 2001. After a broad
discussion of the IRG\'s possible subject areas, the team wrote an
overview statement of what the new IRG would review, specific referral
guidelines for each of the proposed study sections, and shared interest
statements. Copies of these documents were provided to CSRAC members.
These documents and other information on this effort were posted for a
12-week period on the CSR Web site for public comment.

Dr. Martin explained that SSB teams would soon meet to develop three
more of the proposed IRGs: Muscle, Bone, Connective Tissue and Skin;
Oncological Sciences; and Biology of Development and Aging. Meanwhile,
steering committees are being established to begin work on three
additional IRGs: Cardiovascular Sciences; Surgery, Applied Imaging and
Applied Bioengineering; and Fundamental Bioengineering and Technology
Development.

Dr. Howard Schachman emphasized the importance of involving professional
societies in these activities. He has met researchers at various
professional meetings who are apprehensive about the proposed changes
and concerned that they have not been given opportunities to
participate.

Dr. Martin noted that IC program staff who work with the research
communities have helped CSR inform and involve appropriate societies in
these efforts. Dr. Ehrenfeld described how CSR is using its database of
120 professional societies to send periodic PSBR updates to them.

Dr. Martin continued his presentation by focusing on efforts to identify
and address overlapping areas that arise in reorganizing the IRGs. He
listed the individuals who will play vital roles in identifying
overlaps: (1) CSR referral staff, (2) IRG Chiefs, (3) steering committee
members from the Institutes, and (4) the research community, which can
submit comments when proposed IRG guidelines are posted on the CSR Web
site. The steering committees and Division Directors collect all
comments and determine if it is necessary to reconvene the SSB team or a
subgroup to clarify overlapping areas. He explained that the resulting
IRG proposals would be presented to CSRAC members for final discussions
and recommendations.

The presence of significant overlap may make it difficult to maintain a
2-year schedule for implementing some of the IRGs. Dr. Martin discussed
several options for dealing with cases where IRGs have significant areas
of shared interest. When SSB teams with overlapping areas have disparate
start dates, CSR could delay implementation until all relevant SSB teams
can meet. Another option would be to encourage steering committee
members and CSR staff to address problems of overlap and propose a
resolution for the SSB team and CSRAC to review. Dr. Martin emphasized
that CSR would take a case-by-case approach to addressing such problems.
He explained that ongoing work on the Oncological Sciences IRG involves
efforts to deal with overlapping areas, and the recently defined
Hematology IRG has three areas that overlap those of other proposed
IRGs. CSR will need to be persistent in fostering communications with
the societies and study section members to resolve these issues
successfully.

Dr. Martin then focused on the proposed practice of clustering
crosscutting areas of science in individual study sections. The PSBR
report recommended that crosscutting areas of science should represent
30 percent of the applications in a given study section. A number of
groups have asked CSR to clarify this recommendation. Dr. Ehrenfeld
explained that PSBR suggested dividing the IRGs according to an organ
system or disease model with appropriate IRGs covering fundamental
sciences. Clustering crosscutting applications was seen as a way to
prevent them from being scattered among the study sections. She
emphasized the value of remaining flexible in clustering and made an
example out of aging, a significant research area that crosscuts many
organ- and disease-specific areas. Instead of scattering or dividing
aging applications among different study sections in 30 percent
portions, it would be more appropriate to cluster related aging
applications in one place. Dr. Ehrenfeld continued by discussing the
importance of clustering broad general areas of research instead of
creating clusters of small and specific research areas.

Dr. Rothman-Denes initiated discussion by questioning how the
implementation would be timed. She stated that it was very important to
figure out the most appropriate time to initiate a new IRG. Dr. Martin
focused on the new Hematology IRG and explained that CSR would need
advice on how to deal with areas that overlap between this IRG and
Cardiovascular Sciences and the Oncological Sciences IRGs. He again
acknowledged the difficulty of implementing IRGs with overlapping
research areas and emphasized the importance of proceeding in a
step-wise manner. If necessary, however, implementation in these cases
could be delayed until all the relevant groups meet and resolve their
differences. Dr. Yamada explained how cardiovascular scientists could
become concerned by the way hematologists incorporate vascular biology
in the proposed Hematology IRG. Communication between the different
communities will be very important, and he emphasized the value of
involving all stakeholders in the process. He suggested that
stakeholders should be on the SSB teams or be invited to a separate
meeting to discuss the new IRGs. Dr. Martin responded by explaining how
CSR was reaching out to the different societies and recruiting
representatives to provide input. Dr. Matthews concluded the session by
noting that CSRAC would discuss ways to enlist the aid of professional
societies later in the meeting.

**Muscle Biology Review Group**

Dr. Martin discussed how there has been growing community interest in
having CSR consolidate the review of skeletal muscle biology
applications into one study section. Over the past summer, members of
Congress urged CSR to consider establishing a muscle biology study
section. Congressional staff were advised that the ongoing
reorganization called for a new IRG on bone, muscle, connective tissue
and skin. CSR, however, was encouraged to take an interim step and
establish a muscle biology study section. CSR worked with NIH staff to
develop a CSRAC Working Group of outside experts from the relevant
research communities, i.e., the geriatrics, rehabilitation, exercise,
muscle metabolism, and diabetes communities. Approximately 24 members
were recruited, including a chair, Dr. Leslie Leinwand, from the
University of Colorado at Boulder. This Working Group attempted to
define the boundaries for the new Skeletal Muscle Biology Special
Emphasis Panel (SEP) by examining approximately 75 application abstracts
from those used and tracked in the reorganization process. Dr. Martin
discussed how he and the Working Group dealt with certain abstracts from
the Respiratory and Applied Physiology Study Section as boundaries for
this SEP were defined. He noted how these problems were resolved through
discussions with representatives of the relevant communities. These
discussions led Dr. Martin to believe that the proposed review group
will have community support. He explained that, if CSRAC accepts the
Working Group\'s report, it will be posted on CSR\'s Web site and the
appropriate societies and reviewers will be notified. He added that he
would attend all the affected study sections this summer and discuss the
changes with them. The reviewers, however, already know that eight of
them will be moved from their existing study sections into the new one.
In the meantime, the SRA for this new SEP, Dr. Paul Wagner, is
recruiting additional members to prepare for the June 1, 2001, receipt
date for applications.

Dr. Susan Berget expressed concern about using applications from just
one round to design the new review group. Dr. Martin noted that the same
practice was being used in the reorganization process. He explained that
there would be a core of reviewers with broad expertise to review a
large portion of the applications referred to the SEP. He acknowledged
the difficulty of predicting the number of applications in some areas;
however, all study sections face this problem and fill emerging needs
with temporary reviewers. Dr. Berget asked if CSR would be tracking how
the new SEP will affect the other study sections and their workload. Dr.
Martin explained that the Respiratory and Applied Physiology (RAP) and
Lung Biology and Pathology Study Sections would be followed closely. He
noted that the number of applications referred to the RAP study section
is expected to decrease significantly. The skeletal muscle blood flow
applications will remain in the RAP Study Section, and he noted concerns
about how well they would fit there. Dr. Martin explained that these
study sections would be reviewed by a working group this summer. Any
imbalances will be addressed via discussions with reviewers, applicants,
and community representatives. He emphasized that this group was a SEP
and that it would be terminated when this area is addressed by the
ongoing reorganization process.

Dr. James Kushner asked which segments of the community felt it
necessary to create this new review group. Dr. Martin noted the interest
of patient-orientated groups as well as the interest of some of the
field\'s cellular and molecular biologists, specifically those studying
skeletal muscle pathology. He acknowledged that the field was large and
that applications dealing with muscle components generically could be
better reviewed in other molecular study sections. Dr. Kushner
questioned how well disease-specific applications would fare since the
SEP would require many reviewers whose primary expertise is in molecular
mechanisms. After acknowledging this concern, Dr. Martin explained that
approximately two thirds of the applications that fell in this SEP were
classified as cellular and molecular, while the remaining third was
classified as integrated physiology, which involved whole-animal or
clinical research. He explained that most of the clinical research in
this area is supported through program project applications, which are
reviewed by the NIH Institutes themselves. Dr. Ehrenfeld noted that Dr.
Martin has examined the success rate of muscle applications that were
reviewed by many different study sections. He found that these
applications fared better than average in whatever study section they
were in. For this reason, CSR had planned to wait for this field to be
addressed by the ongoing reorganization process. Nevertheless, Dr.
Ehrenfeld acknowledged how this process could be aided by insights
gained through implementing the proposed SEP.

Dr. Matthews asked CSRAC members if they approved the interim plan for
creating the Skeletal Muscle Biology SEP. Dr. Michael Colvin moved that
the committee accept the plan, and his motion was approved.

**Best Practices for Recruitment and Retention of Study Section
Members**

Dr. Arnold Revzin, SRA, CSR Biophysical Chemistry Study Section,
initiated a discussion of the best practices for recruiting and
retaining study section members. He spoke on behalf of CSR staff who
have headed efforts in this area. Dr. Revzin focused on three key
issues. First, he briefly discussed the degree to which CSR should
publicize the flexible service options it may offer reviewers in order
to recruit and retain them. A transparent process was seen as desirable,
but there was concern that advertising all the potential options could
prove disruptive. There thus was a consensus that CSR should continue
its current practice. It should seek a 4-year commitment to attend 12
meetings but tell potential reviewers that they could miss a meeting if
they needed to do so. Dr. Revzin then discussed the idea of having
floating reviewers in an IRG who would move from study section to study
section as needed. He noted that a pilot of this practice would be
conducted by Dr. Carole Jelesma, Chief, CSR Molecular, Cellular and
Developmental Neuroscience IRG. Finally, he explained that CSR is
proposing to develop a group of distinguished senior reviewers who have
already completed a term of service to NIH review and who would commit
to serve as reviewers for at least one meeting per year for an extended
period of time. It is hoped that membership would carry prestige in the
community and thereby allow CSR to recruit more senior reviewers.

Dr. Raphael Pollock noted legal constraints to extending the study
section term to 12 meetings in 5 years and to dropping a meeting. He
agreed that there were disadvantages to advertising options for study
section service. He, however, emphasized the importance of informing the
SRAs of all the options they could have in recruiting and retaining
reviewers. Dr. Pollock also endorsed the concept of establishing a
distinguished reviewer group. If such a program were to be initiated or
tested, it should not require senior reviewers to make a long-term
commitment. He suggested that their terms be limited to 2-3 years.
Potential reviewers might otherwise perceive service more as a burden
than an opportunity.

Dr. Berget expressed support for flexible service terms, noting that
qualified professors at many institutions have teaching commitments
which cannot be changed easily. In addition, she endorsed the idea of
developing a cadre of senior reviewers and listing their names publicly.

Dr. Marvin Wickens also endorsed the ideas presented. He cited previous
discussions on recruiting a cadre of experienced reviewers and suggested
that there were reservations about calling members of such a cadre
\"distinguished\" or \"senior\" reviewers. Dr. Wickens suggested calling
them \"principal\" reviewers. In addition, he endorsed the concept of
having floating reviewers.

Dr. Michael Leon suggested that CSR could meet its need for reviewers by
requiring grantees to serve on study sections. Dr. Matthews noted that
such a requirement would be illegal.

Dr. Ehrenfeld explained that this idea has been discussed many times,
but it has never been advanced successfully. She suggested that the
concept could be inherently flawed because subjecting applicants to this
requirement would mean judging their applications by criteria that has
nothing to do with scientific merit. Requiring study section service
could, therefore, diminish the regard given to the peer review process.
Dr. Leon then suggested that reviewers could be rewarded with extra
funding for their service. Dr. Ehrenfeld admitted that providing such an
incentive would make CSR\'s job easier, but it would be expensive and
involve the commitment of funds without peer review. Dr. Edward Pugh
noted how reviewers may be the best recruiters since they report
remarkable satisfaction with their service. He suggested that CSR
develop a mechanism for communicating the excitement and enjoyment
reviewers experience to potential reviewers. Dr. Pugh contended that a
\"biology of altruism\" exists within scientific communities. He also
suggested that CSR might recruit more senior reviewers if it asked
scientists to nominate respected colleagues to be distinguished
reviewers.

Dr. Schachman endorsed the three proposals presented by Dr. Revzin. He
focused primarily on the proposal to develop a cadre of experienced
reviewers. He also suggested that these individuals not be called
\"senior\" or \"distinguished\" reviewers. He then recommended that CSR
recruit enough of these individuals so that one could attend each study
section meeting.

Dr. Leonard Epstein addressed the broad issue of recruitment by
suggesting that the SRAs use \"peer pressure\" by enlisting their study
section chairs to initiate contact with prospective recruits. Dr. Leon
added that program staff in the Institutes could also be effective
recruiters.

After Dr. Yamada endorsed the three proposals, Dr. Matthews summarized
the consensus that had formed: (1) prospective reviewers should be
informed that there was some flexibility in their terms of service if
special circumstances arose, and all SRAs should know the many options
available; (2) CSR should continue efforts to assess floating study
section reviewers; and

\(3\) efforts to recruit a cadre of senior reviewers should advance,
although CSR should assess the appropriate number of years for service.

**CSR Responses to Study Section Member Satisfaction Survey**

Dr. Anita Miller Sostek, Chief, CSR Behavioral and Biobehavioral IRG,
provided an overview of the comments she received on the Study Section
Member Satisfaction Survey from the IRG Chiefs and additional CSR staff.
She explained that these comments focused on both the merits of
conducting a reviewer satisfaction survey and the merits of the one that
was conducted. Overall, there was variation in the perceived goals of
the survey. Dr. Sostek briefly discussed these goals: to help reviewers
feel involved, to get multiple perspectives on procedures and
performance, and to evaluate CSR practices. Having better-defined goals
would be helpful in deciding if another reviewer satisfaction survey
should be conducted.

Dr. Sostek then focused on problems with the recently completed survey
that should be addressed if future surveys are planned. (1) It took
almost a year to get the results from this survey. Reviewer and staff
turnover that occurred in this period made the results less valuable.
(2) A number of key terms used in the survey lacked clear definitions,
e.g., \"objectivity,\" \"collaborative interaction,\" and
\"knowledgeable.\" (3) The rating scale lacked useful anchors. A
majority of the scores were in the \"satisfactory\" and \"very
satisfactory\" range. A wider spread in the ratings would have been more
useful. (4) The survey did not address nonapplicable responses.
Reviewers who did not use mail-in reviews or review applications
involving human subjects often responded to questions involving these
issues. (5) The survey did not differentiate the data collected from
regular and temporary reviewers. The differences between these groups
could have confounded the results. Dr. Sostek noted also that different
review groups sometimes rated the usefulness of initial materials
differently even though they received the identical materials. She
suggested that these differences might be explained by the fact that
some review groups were new. All of the IRG Chiefs interviewed agreed
that any future survey should be well designed, employ good survey
techniques, have psychometric validity and reliability, and be
thoroughly piloted and evaluated.

Some of the IRG Chiefs, however, questioned the value of conducting
future reviewer satisfaction surveys. Those who had experience with the
working groups seemed to believe that they were a far more valuable way
of evaluating their study sections and staff. Dr. Sostek noted also that
some IRG Chiefs thought the survey mainly corroborated what they had
already observed. She suggested that future surveys should focus on
information that can only be collected from reviewers themselves: (1)
the amount of time they devote to reviews and how it correlates with the
amount of time they expected to expend, (2) the adequacy of their
assignments, (3) the influence of review presentations on their
decisions, (4) the adequacy of their review group\'s composition, and
(5) the degree to which they feel their service affects the review of
their own grant applications. Future surveys should also take into
account the length of time a study section has been in existence and the
experience of the SRA. In addition, administering surveys in February
could ensure that at least some reviewers would have attended two
meetings and most of the respondents would receive the results while
they are still reviewers. Dr. Sostek emphasized the importance of
determining whether a future survey would provide enough valuable
information to justify the financial cost as well as staff and member
time and effort. One of the suggestions she received was to administer
surveys a year before IRG working group members meet so that they could
use the survey results in their evaluations. Another of the IRG Chiefs
she interviewed recommended that future surveys ask reviewers what could
be done to recruit them again and to recruit underrepresented
individuals.

Dr. Yamada emphasized the importance of determining the parameters for
deciding if and when additional reviewer satisfaction surveys should be
conducted. For example, he asked if these surveys should be conducted
every 3 years as a matter of practice or if they should only be
conducted if CSR noticed significant changes in reviewer recruitment
rates.

Dr. Pugh said he was struck by the degree of satisfaction reviewers
reported, and he suggested that CSR capitalize on this finding in
recruiting new reviewers. He then emphasized the importance of assessing
the substantive dimensions of reviewer satisfaction. For example,
reviewers could be asked if they had a much better picture of their
field since joining their review group. Dr. Pugh also suggested asking
reviewers for their thoughts on resources that could help them do their
jobs better. He urged CSR to conduct regular surveys that could better
identify what could be done to improve their performance and their
ability to do their jobs.

Dr. Ehrenfeld emphasized the experimental nature of the reviewer
satisfaction survey. It was the first broad survey of all CSR reviewers.
She agreed that it was not an ideal survey and acknowledged the value of
reassessing the purpose and value of future surveys. Dr. Sostek
explained that CSRAC would discuss next steps later in their meeting.
Dr. Matthews concluded the discussion by expressing interest in the
proposal to collect satisfaction data a year before conducting working
group evaluations.

**Evaluation of Innovative Ideas**

Dr. Matthews noted that study sections are supposed to assess innovation
in the applications they review, but reviewers tend to get entrenched in
certain kinds of ideas and sometimes appear to have difficulty
identifying innovative applications. She explained that several CSRAC
members have repeatedly expressed an interest in inviting an innovation
expert to help the committee address the challenges to identifying
innovative grant applications. CSR responded by inviting such an expert:
Dr. Robert Sternberg of the Yale Center for Psychology of Abilities,
Competencies, and Expertise. He has been highly honored for his work in
the psychology of creativity and related areas since he earned his Ph.D.
in psychology from Stanford University in 1975.

Dr. Sternberg explained that researchers need creative skills to
generate new ideas, analytic skills to test them, and practical skills
to design the needed studies and to disseminate the ideas effectively.
He suggested that current evaluation mechanisms tend to emphasize the
analytic component while the creative component is by far the least
emphasized. Dr. Sternberg proposed a \"triarchic model\" for evaluating
grant proposals in these three areas. Focusing on the creative
component, he listed eight types of creativity that could be assessed.
The first four types of creativity accept the existing paradigm: (1)
conceptual replication, (2) redefinition for interpreting data by using
existing paradigms, (3) forward incrementation for advancing a field of
research to the next logical step, and (4) advance forward
incrementation for rapidly advancing a field of research many steps
forward. The next three types of creativity reject the existing
paradigm: (1) redirection to a different paradigm, (2) regression and
redirection to a paradigm previously proposed, and (3) reinitiation,
where the starting point of a previous paradigm is viewed as wrong. The
final type of creativity synthesizes different paradigms.

Dr. Sternberg proposed a mechanism for assessing the type of creative
movement expected from applications being reviewed and assessing the
novelty and quality of their ideas with a seven-point scale, which would
increase discrimination and variance, reduce ceiling effects, and lessen
the likelihood of \"blackballing.\" He emphasized also the importance of
assessing the scientific importance of the research problem and its
solution. Dr. Sternberg then briefly discussed proposed criteria for
assessing grant applications for their analytical and practical
qualities. He concluded by explaining that a triarchic evaluation
mechanism could permit reviewers to better assess the overall quality
and potential of proposed research and that such a mechanism could be
implemented relatively quickly.

Dr. Matthews acknowledged the importance of having a balanced research
portfolio and asked Dr. Sternberg how CSR could improve how its review
groups assess novelty. He said it would be important to inform
applicants that NIH wanted to receive creative applications and that it
had a framework for reviewing them. In addition, NIH should work to
reorient its reviewers through training sessions or an Internet-based
self-study program. Dr. Ehrenfeld noted that CSR is concerned about how
to review multidisciplinary applications appropriately. Dr. Sternberg
suggested creating one or more \"wild card\" groups of generalists to
review such applications. Dr. Ehrenfeld said that most applications sent
to NIH are paradigm accepting and are not necessarily considered
creative. She continued by explaining that reviewers now are required to
assess applications for innovation, which is defined in terms of
changing paradigms, either rejecting or integrating paradigms. Focusing
on the first four types of creativity, therefore, would not be as
valuable as focusing on the other types. Dr. Sternberg suggested that
NIH could best decide how much it needs to do in these areas. He added
that it is important for both the reviewers and applicants to understand
what NIH wants in terms of innovative applications. Applicants may find
it difficult to believe that NIH is truly open to innovative ideas, and
review groups can be hindered in identifying creative ideas because of
an ingrained tendency to seek consensus. He emphasized the importance of
attacking the problem systemically, with training, an assessment
mechanism, and public relations. Dr. Leon said that it is often
difficult for reviewers to discuss innovation, and he explained that the
assessment mechanism proposed by Dr. Sternberg could help reviewers
think about innovation in an organized way. Dr. Leon later focused on
the fears that prevent applicants from submitting innovative
applications. Many fear submitting an application that would fail and
count against them and others fear that good ideas in a \"half-baked\"
application could be stolen. He proposed allowing applicants two extra
pages in their applications that could be used to request a small amount
of extra risk capital to pursue more risky research. Dr. Matthews
returned to Dr. Sternberg\'s suggestion of creating \"wild-card\" review
groups and suggested that a pilot could be conducted with a subset of
neuroscience applications. Dr. Sternberg agreed that this could be done,
but he emphasized the importance of making a sustained commitment. Dr.
Matthews then focused on streamlining the review of grant applications
and asked if this practice could be improved using a seven-point rating
scale.

Dr. Sternberg explained that expanding the rating scale could permit
more variance that could spark useful review group discussions.

After thanking Dr. Sternberg for his input, Dr. Matthews noted that Dr.
Pollock had completed his CSRAC term. She and Dr. Ehrenfeld thanked him
for the many valuable insights and contributions he has provided over
the years.

**Follow-up Survey Activities**

Dr. Michael Micklin, Chief, CSR Risk, Prevention and Health Behavior
IRG, discussed efforts of the subcommittee addressing some issues
stemming from the Study Section Member Satisfaction Survey. The
committee included Drs. Matthews and Epstein as well as Dr. Micklin and
Dr. Yvette Davis, SRA, CSR Social Sciences, Nursing, Epidemiology and
Methods 2 Study Section. This subcommittee has considered ways to better
assess the time reviewers spend preparing for their meetings and to
identify the incentives and disincentives for study section service. Dr.
Micklin noted that sociological research conducted by Dr. John Robinson
shows that individuals asked to estimate retrospectively the time they
spent on a task often grossly over-estimate it. To get more accurate
estimates, the subcommittee developed a time-log survey for collecting
real-time data. Reviewers would be asked to log the time they spent (1)
reviewing an application to determine if they have a conflict or the
appropriate expertise, (2) reading and taking notes on an application,
and (3) writing, editing, reviewing, and submitting their reviews.

Dr. Micklin then discussed the survey the subcommittee has drafted to
identify incentives and disincentives for study section service. He
explained that the survey will examine various types of individuals who
may be asked to serve: those asked for the first time to serve as an
temporary member of a review group, those first asked to serve as a
regular member of a study section, and those with recent service who are
asked to serve again.

Dr. Matthews emphasized that these were draft surveys and then asked
CSRAC members for their input. Dr. Rothman-Denes suggested that the
latter survey could be very useful since it would include the views of
individuals who had not served on study sections. She then stressed the
importance of surveying a balanced group of individuals with different
academic positions, including senior researchers who have or have not
served on study sections and junior researchers. Dr. Micklin explained
that this survey would only be given to researchers who have had R01
support within the last 3 years or so.

Dr. Wickens asked for more information on the purpose of the reviewer
time log. Dr. Micklin explained that a more accurate estimate of the
time reviewers spend preparing for study section meetings could improve
the guidance given to potential reviewers. Dr. Ehrenfeld acknowledged
the importance of this discussion and added that the time-log survey may
help NIH better determine a reasonable workload for reviewers. Drs.
Rothman-Denes and Leon, however, suggested that reviewer perception of
their service may be a more critical factor than the absolute number of
hours served. Dr. Matthews returned to Dr. Micklin\'s point by
explaining that potential reviewers may give study section service
greater consideration if this new survey showed that service hours have
been overestimated. It was noted that reviewer recruitment is likely to
be more influenced by personality variables than an estimate of service
hours.

Dr. Matthews explained that the second survey would help assess the
reasons why researchers would be more or less interested in becoming
reviewers. Later in the meeting, Dr. Yamada suggested that a more
accurate measure of reviewer effort could be helpful to CSR management
in balancing reviewer workload and effort. It was noted that periodic
assessments could be useful in managing future changes in the grants NIH
receives or in the way they are reviewed.

Dr. Ehrenfeld explained that this is the main reason why she would want
this kind of data.

Dr. Pollock focused the discussion on the proposed survey of reviewer
incentives/disincentives by emphasizing the value of surveying those who
have served as reviewers before and who might be asked to serve again.
In addition, he suggested examining clinical responsibilities as a
limiting factor for study section service. Dr. Ehrenfeld asked how CSR
should identify and survey individuals who have consistently declined to
serve on study sections. Dr. Matthews explained that her subcommittee
hopes that these individuals could be identified through a sampling of
the researchers who have received NIH grants in the last 3-5 years but
who have not served on study sections. Dr. Berget questioned whether
this method would work. Dr. Yamada suggested surveying individuals who
refuse an invitation to serve on a study section. He also noted that the
survey does not include \"total commitment of time required\" as a
reason for not serving. The value of collecting data on this factor,
however, was questioned, because there would likely be deeper reasons
why someone would cite it as a reason for not serving.

Dr. Matthews explained that these issues could be worked out, and she
asked members if they thought this survey could be useful in identifying
the common reasons researchers have for serving or not serving on study
sections and if the resulting data would be useful to recruiting and
retaining reviewers. Drs. Wickens and Schachman thought it was confusing
to mix reasons for and against becoming a reviewer in the third section
of the survey. Dr. Micklin explained that the reasons were mixed to
prevent respondents from falling into positive or negative tracks in
rating these reasons. Dr. Schachman noted that he has counseled many
young scientists, and those interested in serving want to learn more
about grantsmanship, contribute to the community, be exposed to cutting
edge science and methods, and work with wonderful people. In addition,
Dr. Schachman expressed concern about selecting survey participants and
reviewers based on whether they had R01 grant support. He noted that 40
percent of those who receive an R01 grant never receive a second one,
and these individuals would be the wrong ones to survey or enlist as
reviewers.

Dr. Ehrenfeld asked members to consider whether they thought the survey
could yield data that would help CSR recruit reviewers. Dr. Pollock
suggested that CSR may be able to do something if it learned that a
majority of those who refused to serve did so because of clinical or
teaching responsibilities. He added that data on how successful
reviewers are in getting their own research funded could be useful to
recruitment if serving on study sections increased their chances of
being funded. Dr. Matthews suggested that work continue on the two
surveys with a little more focus on their goals.

*Tuesday, May 15*

**Transparency of the Review Process**

Dr. Jelsema discussed a proposed Web page to explain how scientists are
selected for study section service. A draft of this document was
developed by a subcommittee and distributed to CSRAC. Members of this
subcommittee include Drs. Berget and Leon and three CSR staff members in
addition to Dr. Jelsema: Dr. Jeanne Ketley, Chief, CSR Cardiovascular
Sciences IRG; Dr. Christine Melchior, Chief, CSR Integrative, Functional
and Cognitive Neuroscience IRG; and Dr. Gordon Johnson, SRA, CSR
Cardiovascular Study Section. Dr. Jelsema summarized the NIH
requirements for study section members. They must be highly qualified
scientists at the professor or associate professor level and have
current NIH grant support or demonstrate other evidence of having
stature in their field. In addition, attempts are made to ensure that
review groups are balanced in terms of geographic, racial, and ethnic
diversity.

Dr. Jelsema then discussed selection criteria used at the study section
level. She noted the importance of recruiting individuals who have the
appropriate breadth or depth of experience to fill current and expected
needs. She continued by emphasizing the importance of recruiting
individuals who have excellent communication and interpersonal skills as
well as the abilities to be fair, objective, and respectful of other
reviewers and the applicants. In addition, recruits should be able to
defend their positions but compromise when necessary. They should also
be accepted leaders who can facilitate, focus, or finalize discussions,
and who will ensure that scoring is fair and consistent. Temporary
reviewers often provide needed expertise as well as fresh perspectives
to the review groups.

The names of potential study section members come from a number of
sources: (1) the Computer Retrieval of Information on Scientific
Projects (CRISP) database, which lists grantees and their areas of
expertise, (2) current or former study section members and chairs, (3)
NIH program staff, (4) NIH advisory councils, (5) department chairs and
established scientists at different academic institutions, (6) SRAs
through their knowledge of researchers in their scientific communities,
and (7) individuals who nominate themselves. Dr. Jelsema described a
second level of analysis the SRAs perform in evaluating potential
reviewers. Potential study section members are checked for previous
committee service. Their grant and publication histories and status
within the university are also checked. If they meet all criteria, they
will be invited to serve as an temporary reviewer and then further
assessed for permanent service.

Dr. Jelsema then described how the SRAs put together nomination packages
and how they are cleared through the relevant IRG Chief and Division
Director, and the CSR committee management office. Once approved by the
CSR Director, the packages are submitted to the central NIH committee
management office before being sent to the NIH Director. If a problem
arises at any of these review levels, the package is returned to the SRA
for revision.

Dr. Donald Schneider, Director, CSR Division of Molecular and Cellular
Mechanisms, noted that, in recent months, one in three nomination slates
has been returned by the NIH Director\'s office for revision. Dr.
Ehrenfeld explained that diversity concerns are the primary reasons for
these returns. A broad range of diversity issues can be involved, e.g.,
racial, ethnic, gender, geographic, and institutional diversity. She
encourages CSR staff to seek, analyze, and improve diversity
representation on its review groups. She noted, however, that there were
no numerical quotas, since it is understood that the different
scientific fields have different levels of diversity.

Dr. Pollock commended the Government for doing as much as it has in
fostering diversity.

Dr. Matthews asked if the intended readers would benefit from all the
detailed information in the document. Dr. Chien suggested developing a
shortened version with links to additional information. Dr. Ehrenfeld
explained that applicants seeking to assess their reviews and
researchers interested in becoming reviewers often ask CSR staff how
study section members are selected. The proposed document would be used
to address these questions as well as make the process more transparent
to all interested parties. Drs. Colvin and Berget endorsed the idea of
creating a shortened document with links to more detailed information
for principal investigators.

Responding to questions about temporary members, Dr. Jelsema explained
that these individuals must meet similar requirements, though the same
documentation and nomination process is not used. Temporary members are
recruited to meet specific needs in a particular round by the SRAs, who
typically only have 3 weeks to fill these slots. About a third of each
study section\'s members are temporary. Dr. Jelsema emphasized that the
feedback the SRAs get from their chairs and other members also helps
them assess whether these individuals would be suitable as permanent
members of their study sections.

Dr. Matthews recapped the discussion by stating that the committee likes
the document, and it would like to see something done to reduce the
detail on the nomination process when the document is posted on the Web.

*Reference Book for Reviewers*

Dr. Johnson summarized the development of a reference book for
reviewers, which was discussed at the last CSRAC meeting. He noted that
the committee previously recommended that (1) this book should be sent
to reviewers with a personalized cover letter directing them to the most
relevant sections, (2) CD and Web versions could be useful, but hard
copies should be provided for the most frequently used documents, and
(3) an index, quick guide, or executive summary should be provided to
make the book easier to use. CSRAC members were sent an

e-mail seeking additional feedback. Dr. Leon subsequently recommended
that the book be refocused to help reviewers better perform fair and
competent reviews. He specifically suggested that a revised book should
discuss how reviewers can comply with the rules to ensure that reviews
are fair, and he also proposed that the book have a section that
included sample summary statements.

Dr. Johnson discussed how the draft reference book has been reorganized.
He provided members with a proposed table of contents and explained that
the first part of the book would contain material that reviewers would
need for each meeting while the second part would contain information
that could be useful but might not be necessary. In addition, he
provided members with a proposed preface that could be included to guide
reviewers to the information they may need in the book.

In responding to Dr. Leon\'s suggestions, Dr. Melchior explained that
providing reviewers an example of a critique would be worthwhile, and
she noted that the SRAs frequently provide examples of an R01 critique.
She added that producing sample critiques for every grant mechanism,
however, would take considerable time and effort. A committee member
noted that the reference book now appeared to have three different
objectives. Originally, it was conceived as a ready reference for
regular study section members. It now was also being considered as a
training tool for new reviewers, and it could also be used for making
the review process more transparent to the public. Dr. Matthews
explained that the latter objective was considered less important when
the committee originally discussed the reference book, and she suggested
that the committee examine orientation materials for new study section
members at a future meeting.

**Update on Conflict-of-Interest Rules and Implementation Issues**

Dr. Lawrence Yager, SRA, CSR Bacteriology and Mycology 2 Study Section,
acknowledged the members of the CSR working group addressing
conflict-of-interest issues: Dr. Jean Paddock, CSR Associate Director
for Program Coordination; Dr. Russell Dowell, SRA, CSR Cardiovascular
and Renal Study Section; Dr. Lee Mann, SRA, CSR Risk, Prevention and
Health Behavior 3 Study Section; Dr. Angela Ng, SRA, CSR Metabolic
Pathology Study Section;

Dr. Nancy Shinowara, SRA, CSR Special Reviews 5 Study Section; and Dr.
Cheryl Corsaro, SRA, CSR Genome Study Section. CSRAC was provided a
draft document prepared by this working group: \"Guidelines for
Determining Whether a Conflict of Interest Exists for Reviewers Who Are
Not Federal Employees.\"

Dr. Yager explained that he and Dr. Corsaro would discuss broad
conflict-of-interest problems that hinder reviewer recruitment. He began
by focusing on collaborations. More and more researchers have
conflict-of-interest issues because the number, size, and complexity of
collaborations have increased, particularly with increases in
multidisciplinary and interdisciplinary research. There also has been an
increase in ties between academia and industry. Problems can be acute
when trying to recruit reviewers in a unique or new field, where there
is a scarcity of expertise and there are close ties between the few
experts that exist.

Dr. Yager explained that current conflict-of-interest policies need to
be modified to accommodate different levels of collaboration and
competition. He pointed to the clustering of large groups of researchers
within the same university departments. Many of the researchers within
these large departments do not collaborate with each other, but current
rules do not allow members from the same academic department to
concurrently serve on the same study section. However, a comparable
group of researchers subdivided into smaller departments at another
university would be eligible for service on the same study section. Dr.
Yager asked if policies should be changed to deal with large departments
and with different levels of collaboration.

Another conflict-of-interest concern relates to competition and the
possibility that a reviewer would reap a gain or loss from the outcome
of a specific review. Dr. Yager explained that competition issues are
particularly difficult when dealing with Small Business Innovation
Research (SBIR) or other business-related applications. He noted that
researchers from companies who have SBIR applications before a SEP
cannot serve on it, while researchers from universities that have
applications before a study section may serve on it, though they must be
out of the room when their university\'s application is reviewed. Dr.
Yager asked if this policy should be changed. He then noted that
researchers who serve on company advisory boards and set broad
scientific goals and policies cannot serve on SEPs that consider
applications from their companies. He asked if such individuals could be
unbiased in reviewing applications from other companies and suggested
that efforts should be made to better define the kinds of indirect
involvement that may or may not truly represent conflicts of interest.

Dr. Corsaro summarized efforts to follow-up discussions at the last
CSRAC meeting on the need to discourage applicants from submitting
letters of support from other researchers that disqualify these
individuals from being reviewers. She explained that CSR is looking into
developing a notice that will be posted on its Web site. Program staff
in the ICs are being encouraged to include a similar notice in their
program announcements and requests for applications.

Dr. Corsaro then discussed how multidisciplinary and collaborative
research has increased the number of conflicts for potential reviewers.
She noted how some ICs encourage investigators from many sites to pool
their resources or data. In one instance, a genetic analysis application
named 20 collaborating institutions. She noted also how some
publications can list 20-100 authors. Advances in telecommunications may
only increase the number of collaborations and conflicts. Dr. Corsaro
asked CSRAC members for recommendations, since SRAs need more
flexibility in defining direct and indirect collaborations and
determining whether real conflicts of interest exist. She added that
increased ties between academic researchers and industry has raised the
number of reviewers with possible financial conflicts, and she asked if
CSR should be more stringent with respect to these kinds of conflicts.

Dr. Colvin suggested that loosening some of the conflict-of-interest
restrictions could be helpful. Dr. Matthews stated that the proposed
document provides a good overview of the current issues and policies.
She then asked Dr. Corsaro to amplify her request for assistance in
making the guidelines more flexible. Dr. Corsaro explained that SRAs
need more flexibility in interpreting conflict-of-interest policies and
making gray-area judgments. Suggestions from CSRAC on how this could be
done appropriately would be helpful. Drs. Chien and Yamada proposed
making financial disclosure more robust and using this information to
assess conflict-of-interest issues better. It was noted that reviewers
are considered consultants instead of temporary employees, and NIH
cannot ask consultants to make detailed financial disclosures.

Dr. Berget emphasized the importance of public confidence in the review
process and questioned the wisdom of relaxing conflict-of-interest rules
or even appearing to relax them. She suggested seeking ways to work
around the problems. Dr. Pugh added that perceived conflicts of interest
can undermine credibility, and he emphasized the importance of
preventing a group of scientists from having undue influence. He then
suggested that a standing committee be formed to provide the advice and
guidance the SRAs need in addressing conflict-of-interest problems. This
committee could collect and analyze case histories that could be useful
in addressing future conflict-of-interest problems. Dr. Ehrenfeld noted
that the difficulties SRAs face are significant, and they cannot do
their jobs by simply taking a conservative stance towards
conflict-of-interest problems. She explained that Dr. Pugh\'s suggestion
of forming a committee to address these problems was a good one, and she
noted CSRAC\'s preference for moving forward in a conservative fashion.
Dr. Matthews suggested forming a follow-up study group to examine these
issues further. She expressed interest in exploring whether or not
potential reviewers could be asked to make more robust disclosures and
looking at the existing rules and policies that define conflicts of
interest.

**Other Best Practices Issues Relevant to Multidisciplinary
Applications**

Dr. Gillian Einstein, SRA, CSR Molecular, Cellular and Developmental
Neuroscience 2 Study Section, summarized efforts to implement CSRAC
recommendations to promote best practices for reviewing
multidisciplinary applications. CSR has revised its Web site
(http://www.csr.nih.gov/EVENTS/AssignmentProcess.htm) to instruct
applicants who propose multidisciplinary research to identify the main
disciplinary or methodological thrust of their applications. An internal
CSR committee was established to consider the other recommendations
CSRAC made at its last meeting. She added that CSR is setting up an
NIH-wide reading group on multidisciplinary applications, and efforts
are underway to invite a speaker to provide a formal presentation to CSR
staff on multidisciplinary research and the review process.

Dr. Jean Sipe, SRA, CSR Bioengineering Research Partnerships and Grants
SEP, described two SRA training sessions on reviewing multidisciplinary
applications. The first session was on the review of bioengineering
research partnerships, which combine engineering and quantitative
expertise with clinical and biological expertise to study clinical or
biomedical problems. Case studies were used, with an emphasis on
identifying the main thrust of an application and the expertise needed.
The second session considered the broad range of multidisciplinary
applications. SRAs selected a case study from a group of applications
from each of CSR\'s divisions, analyzed it in a breakout group, and then
reported back to the main group. The SRAs discussed the main thrusts of
their applications, the key areas of science involved, and the expertise
needed to review them. The SRAs also discussed whether their
applications could be reviewed in standing study sections and if
additional expertise would be needed.

Drs. Chien and Wickens complemented Drs. Einstein and Sipe and their
colleagues for the way they are implementing many of CSRAC\'s
recommendations. Dr. Sipe was asked if the internal CSR committee on
multidisciplinary reviews would develop instructions for reviewers.
There was a concern that an individual reviewer may be unable to cover
all of the issues that need to be addressed in the review of a
multidisciplinary application. Dr. Sipe explained that reviewers are
urged to evaluate the overall application from the point of view of
their expertise and identify key questions. CSR, however, will consider
developing additional instructions.

**Interaction of CSR with Professional Societies**

Dr. Marcia Steinberg, Chief, CSR Cell Development and Function IRG,
noted that interactions between CSR and the professional societies have
increased significantly with the ongoing reorganization activities. CSR
has developed a database of 200 societies, which receive periodic
updates. She added that societies frequently contact Dr. Ehrenfeld with
their concerns. To better assess current interactions, the IRG Chiefs
were asked if they contact societies for input in selecting reviewers.
Nine of the thirteen IRG Chiefs who responded said yes and two explained
that they planned to seek such guidance in the future. SRAs were also
asked how many mock study sections and grant workshops they held in the
last year. A third of the SRAs responded: 14 held mock study sections
and 46 made presentations on NIH and the grant review process at
national meetings. It was noted that the Endocrinology Society
recognizes its members who serve on study sections at its annual
meeting. Dr. Steinberg emphasized the value of these interactions, which
serve to educate applicants and to help CSR recruit highly qualified
reviewers. She noted, however, that formal requests to society officials
are rarely as productive as personal interactions in identifying the
best reviewers. As for future activities, Dr. Steinberg discussed the
possibility of holding a reception at a national meeting to honor
individuals who have served on study sections. She also raised the
possibilities of (1) submitting articles for society newsletters that
would encourage members to consider study section service, (2) meeting
with society officers to build more productive relationships, and (3)
contacting editorial boards.

Dr. Rothman-Denes said she has spoken with individuals associated with
the American Society for Biology and the American Association for
Microbiology and has learned that the agendas for their national
meetings are usually packed. It would be difficult to add additional
activities as proposed. She emphasized the value of using newsletter
articles to encourage study section service and of contacting editorial
boards, which often are familiar with the good reviewers in their
fields.

Dr. Chien expressed approval of ongoing interactions. He acknowledged
the value of reviewer recognition programs, such as the one used by the
Endocrinology Society. After noting the difficulties of scheduling
receptions, he recommended using breakfasts at society meetings to
recognize past and current reviewers and to identify and recruit future
reviewers. He also endorsed the idea of using articles in society
newsletters to inform members about general NIH/CSR news, reviewer
recruitment, and other issues. Dr. Kushner suggested that CSR work with
the NIH Institutes in making joint presentations at national meetings.
Dr. Matthews noted that much could be done at minimal expense since many
societies are located in or near Washington, D.C., and they often
convene their meetings nearby. Dr. Ehrenfeld explained that recognition
efforts could be useful and inexpensive. She explained how presentations
were valuable in educating young or new investigators about grants and
the review process. She did not, however, believe that such
presentations would significantly improve reviewer recruitment. Having
SRAs attend society meetings is in itself a valuable means of
identifying and recruiting good reviewers. Dr. Schachman commended CSR
for its efforts to communicate and work with scientific communities, and
he emphasized the value of future efforts, particularly in regards to
the ongoing reorganization of the study sections. Dr. Ehrenfeld
acknowledged that there were many good reasons to increase interactions
with the scientific communities beyond improving reviewer recruitment.
Dr. Matthews then summarized these discussions: While there are a number
of options for formally seeking assistance from the societies in
recruiting reviewers, they may not be particularly productive ones.
Interactions with societies are nonetheless important for many reasons.

**Further Considerations**

Dr. Matthews returned to the issue of innovation by recalling some ideas
that rose from the committee\'s discussions: (1) providing reviewers
additional guidance for identifying what is and is not innovative, (2)
including a provision for risk capital in grants to advance innovative
research, (3) training SRAs and review group chairs to recognize
innovation, and (4) examining R03 and R21 mechanisms for fostering
innovative research. Dr. Epstein noted that members have often talked
about modifying the Public Health Service 398 grant application form.

Dr. Matthews explained that changes to this form were recently approved
and the process is quite lengthy. Dr. Anthony Demsey, Senior Advisor for
Policy, NIH Office of Extramural Research,

added that instructions to the form could be more readily changed. Dr.
Yamada noted that there were two issues to be considered: how reviewers
can become more accepting of innovation and how applicants can be
encouraged to submit more innovative applications. He emphasized the
importance of proceeding cautiously. Dr. Ehrenfeld explained that many
NIH Institutes have set aside funds for high-risk, high-impact research,
but they have not received the applications. She stated that CSR could
work with some of the NIH Institutes to set up a special study section
for innovative research. Dr. Yamada explained that he has reviewed
innovative grants for a Rockefeller Foundation program, which limits
applications to two pages, reviews submissions quarterly, and has no
requirements for prior experience or knowledge. He suggested that NIH
could gain from the experience of this program, but he again emphasized
the importance of proceeding carefully.

Dr. Matthews then asked members if they liked the idea of having outside
speakers come to their meetings. A number of members and CSR staff
expressed their approval. Dr. Ehrenfeld noted that the CSR
multidisciplinary committee wanted to invite a speaker in its area, and
if CSRAC members had an interest, this speaker could be invited to
address them and CSR staff.

Dr. Yamada expressed interest in hearing a presentation on conflicts of
interest. Drs. Matthews and Leon agreed that CSRAC and the SRAs might
benefit from a presentation on multidisciplinary research.

**Summary of Major Agenda Items for the Next Meeting**

Dr. Matthews noted changes in CSRAC\'s responsibilities and asked if it
should meet two or three times a year. To frame discussions, she
reviewed possible agenda items for the next meeting: (1) PSBR
activities, reports, and concerns, such as how to identify hot areas for
the Study Section Boundaries Teams ahead of time and how to avoid
problems that could arise from the long-term nature of the
reorganization; (2) efforts to foster the submission and review of
innovative applications; (3) orientation materials for new reviewers;
(4) conflict-of-interest rules; (5) the review of complex and
multidisciplinary applications; (6) the pilot using floating reviewers
at the IRG level; and (7) the streamlining experiment.

After noting that efforts in a number of these areas will not be ready
for discussion in September and other efforts will not require extensive
discussion, CSRAC members decided to forgo their September 2001 meeting
and reconvene in January 2002. Dr. Matthews then thanked

Dr. Marjam Behar, SRA of the SSS-6 study section in CSR\'s Biophysical
and Chemical Sciences IRG, for her dedication to CSR. She is retiring
after 21 years of service.

With no other business to address, Dr. Matthews thanked everyone for
their efforts and adjourned the meeting at 11:35 am.

We do hereby certify that, to the best of our knowledge, the foregoing
minutes are accurate and complete. The minutes will be considered at the
28th meeting of the Advisory Committee, and any corrections or comments
will be made at that meeting.

> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Robert W. Eisinger, Ph.D.
>
> Executive Secretary
>
> Center for Scientific Review Advisory Committee
>
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Ellie Ehrenfeld, Ph.D.
>
> Director
>
> Center for Scientific Review
>
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> Karen Matthews, Ph.D.
>
> Chair
>
> Center for Scientific Review Advisory Committee