##### FY 2002 REPORT 

#####  NSF COMMITTEES OF VISITORS (CoVs)

**February 25 -- 27, 2002**

**Physiology & Ethology Cluster**

**Integrative Biology & Neuroscience**

**Biological Sciences**

**Number of actions reviewed by COV: [124]{.mark}**

***N.B.** The core questions and report template used by this Committee
of Visitors' Report were not finalized at the time the CoV met. Use of
these draft questions and template was authorized by Lorretta Hopkins on
2/19/02.*

**[Part A. Integrity and Efficiency of the Program's Processes and
Management]{.smallcaps}**

A.1 Questions about the quality and effectiveness of the program's use
of merit review procedures.

+-------------------------------------------------------------+--------+
| **[Quality and Effectiveness of Merit Review                | *      |
| Procedures]{.smallcaps}**                                   | *[Yes, |
|                                                             | No, or |
|                                                             | Data   |
|                                                             | Not    |
|                                                             | Availa |
|                                                             | ble]{. |
|                                                             | smallc |
|                                                             | aps}** |
+-------------------------------------------------------------+--------+
| Is the review mechanism appropriate? (panels, ad hoc        | *      |
| reviews, site visits)                                       | *Yes** |
+-------------------------------------------------------------+--------+
| Is the review process efficient and effective?              | *      |
|                                                             | *Yes** |
| *The Program Directors have an enormous number of duties    |        |
| with insufficient time to carry them out. Assistants could  |        |
| carry out many of these duties. NSF should provide more     |        |
| science assistants to the Program Directors in order to     |        |
| allow them to concentrate on the scientific and             |        |
| decision-making duties for which they have been selected    |        |
| (see comments at the end of this section).*                 |        |
+-------------------------------------------------------------+--------+
| Is the time to decision appropriate?                        | *      |
|                                                             | *Yes** |
| *It is clear that considerable effort goes into providing a |        |
| clear, thorough and timely response. This process works     |        |
| extremely well.*                                            |        |
+-------------------------------------------------------------+--------+
| Is the documentation for recommendations complete?          | *      |
|                                                             | *Yes** |
| *The documentation as provided in the jackets and in Form 7 |        |
| was very thorough and complete. Members of the committee    |        |
| were very favorably impressed with the care and insight     |        |
| demonstrated in this very complicated process.*             |        |
+-------------------------------------------------------------+--------+
| Are reviews consistent with priorities and criteria stated  | *      |
| in the program's solicitations, announcements, and          | *Yes** |
| guidelines?                                                 |        |
|                                                             |        |
| *The files we reviewed indicated that in all cases the      |        |
| reviews were consistent with intellectual merit portions of |        |
| the program's solicitations, announcements and guidelines.  |        |
| Review with regard to Criterion II is discussed in section  |        |
| A.2.*                                                       |        |
+-------------------------------------------------------------+--------+

Discuss issues identified by the CoV concerning the quality and
effectiveness of the program's use of merit review procedures.

It became clear to the Committee, as we investigated the merit review
process within the cluster, that a great deal of the workload falls upon
the Program Directors. These duties include identification and contact
of ad hoc reviewers, selection of panel members, convening of the panel
meetings, analysis and interpretation of the reviews obtained leading up
to the funding decision, and contact with the PIs following the funding
decisions. These diverse and essential duties lie at the heart of NSF's
role in fostering scientific discovery, and enhancing physical
infrastructure and human resources in the sciences. In addition to these
duties that require their technical and intellectual expertise in a
particular area of science, the Program Directors also spend an
inordinate time on more mundane duties. These include looking up the
e-mail addresses of potential ad hoc reviewers, entering the e-mail
addresses and areas of scientific expertise of the ad hoc reviewers in
the NSF database system, contacting ad hoc reviewers, and more
specifically reminding them that the deadlines for their responses are
approaching. These duties are tangential to the most critical duties of
the Program Directors, but they are very time-consuming.

We urge NSF to seriously consider mechanisms to relieve the Program
Directors of some of the time-consuming but largely clerical duties they
face. One suggestion is to provide additional personnel who can be
assigned to these duties by the Program Directors. Two of the programs
(IAB and IPB) have only one Program Director each and share one
assistant between them. Additional assistance would allow the Program
Directors to concentrate on scientific and management issues. The
Program Directors told us that Science Assistants were particularly
useful since they had sufficient expertise to glean material from the
web.

Two members of the Committee have extensive duties in an editorial
capacity with scientific journals. They face similar problems in
identifying, contacting and reminding reviewers. They have found the
commercial computer programs "Journal Manager" and "Aires" to be very
helpful. The Program Directors mentioned that personal messages
reminding the *ad hoc* reviewers of approaching deadlines were more
effective than automatic pre-formatted reminder notes. While this may be
true, the Committee felt that most reviewers simply needed a reminding
nudge. Under any circumstances, the form notices can be formatted to be
more personal, and the time saved by the Program Directors can be put to
more valuable use.

Another issue of concern in the cluster is the ratio of permanent to
rotating Program Directors. Although there are plans to hire one
permanent Program Director in the near future, at present, there are
none. We strongly feel that two of the 5 Program Director positions in
the cluster should be permanent. This will allow the cluster to maintain
institutional memory while at the same time continuing to bring new
ideas and intellectual replenishment into the cluster. A ratio
significantly different from this would not be in the best interests of
the cluster. The idea of identifying promising minority candidates for
Program Directors is commendable. However, given the size of the pool of
such scientists in these disciplines, it will be difficult to find
appropriate candidates with previous experience with NSF (as a grant
recipient, reviewer, panelist, etc). We do not feel that filling 2 of
the 5 Program Director positions in the cluster with permanent officers
will significantly impede the search for minority representatives in
rotating, and future permanent positions.

A. 2 Questions concerning the implementation of the NSF Merit Review
Criteria (intellectual merit and broader impacts) by reviewers and
program officers.

+----------------------------------------------------------+-----------+
| **[Implementation of NSF Merit Review                    | **[%      |
| Criteria]{.smallcaps}**                                  | Revi      |
|                                                          | ews]{.sma |
|                                                          | llcaps}** |
+----------------------------------------------------------+-----------+
| What percentage of *reviews* address the [intellectual   | \>98%     |
| merit]{.underline} criterion?                            |           |
|                                                          |           |
| *Some of the ad hoc reviews were superficial. All of the |           |
| panel reviews were very thorough in their review of the  |           |
| intellectual merit of the proposals.*                    |           |
+----------------------------------------------------------+-----------+
| What percentage of *reviews* address the [broader        | approx.   |
| impacts]{.underline} criterion?                          | 30%       |
|                                                          |           |
| *Many of the ad hoc reviews failed to address Criterion  |           |
| II at all. In recent years, all of the panel reviews     |           |
| addressed Criterion II. (see comments below).*           |           |
+----------------------------------------------------------+-----------+
| What percentage of *review analyses* (Form 7's) comment  | 100%      |
| on aspects of the [intellectual merit]{.underline}       |           |
| criterion?                                               |           |
+----------------------------------------------------------+-----------+
| What percentage of *review analyses* (Form 7's) comment  | 100%      |
| on aspects of the [broader impacts]{.underline}          |           |
| criterion?                                               |           |
+----------------------------------------------------------+-----------+

Discuss any concerns the COV has identified with respect to NSF's merit
review system.

> By and large the ad hoc reviews addressed the issue of the
> intellectual merit of the proposals. Occasionally, the ad hoc reviews
> were too superficial or did not adequately address substantive,
> scientific issues. For example, in some cases in which a proposal was
> rated as excellent, no basis for this evaluation was given. In every
> case we examined in which this was a problem, the panel rejected these
> reviews from consideration. These superficial reviews therefore did
> not adversely affect the final decision on the proposals. Such reviews
> are very regrettable, however, because they reduce the external
> expertise available to the panel in reviewing the proposals. It would
> be helpful to provide instruction to *ad hoc* reviewers encouraging
> them to highlight and detail why they think a project is excellent, as
> well as detailing criticisms and problems. Such instructions are
> already routine in the guidelines for reviewers of scientific journal
> articles.

Part of our latter analysis included following the same proposal over
several years. Although the fate of the proposal varied, it was noted
that the quality and consistency of the reviews was excellent, as were
the responses to comments by the PI. Clearly, judging from our sampling,
the review process is being used effectively to identify the highest
quality science, while also fostering improvement in less stellar
proposals.

> Insufficient information is provided in the panel summary to the PIs
> whose proposal is not funded, about whether the proposed work is novel
> and should be resubmitted or whether it may not be worthwhile to
> pursue (i.e., some indication of whether continued pursuit of the work
> is encouraged or discouraged). For example, the panel comments rarely
> include specific guidance to the PI in terms of whether a resubmission
> would be welcomed, or whether the panel found the proposal fatally
> flawed by a lack of novelty or some other fundamental weakness
> unlikely to be rectified by revision. Such information is often
> connoted by the language of the reviews, but in several instances
> proposals that had received excellent reviews were denied funding,
> leaving the PI little sense of whether a resubmission would be
> warranted or welcome, or little guidance as to revision. The Form 7
> review analyses were felt to provide much of this information. Perhaps
> a "cleansed" (in that identity of reviewers or information about
> conflicts of interest are removed) version of Form 7 could be sent on
> to the PI to increase the information to PIs about why decisions were
> reached, and the best approach for future submissions or revisions.

Well below half of the files we examined had substantive *ad hoc*
reviews of Criterion II. This situation was found to be markedly
improving during the period of time we reviewed (1999-2001). We did
find, however, that all of the recent panel reviews addressed Criterion
II. We feel that the importance of addressing Criterion II has been made
clear to the panels and they are dealing with it in an effective manner.

The problem remains that the *ad hoc* reviewers are not adequately
addressing Criterion II. We make the following recommendation: On the
FASTLANE form used by the *ad hoc* reviewers, reviewers are asked to
respond to the question: **What are the broader impacts of the proposed
activity?** We feel that this question should be replaced by: **What are
the impacts of the proposed activity with regard to scientific
discovery, integration of research and education, connections between
discovery and service to society, and diversity of the workforce?** This
wording is very likely to solicit much more substantive responses by the
*ad hoc* reviewers, and eventually by the writers of proposals, with
regard to Criterion II**.**

A.3 Questions concerning the selection of reviewers.

+-----------------------------------------------------------+----------+
| [Selection of Reviewers]{.smallcaps}                      | [Yes,    |
|                                                           | No, or   |
|                                                           | Data Not |
|                                                           | Availa   |
|                                                           | ble]{.sm |
|                                                           | allcaps} |
+-----------------------------------------------------------+----------+
| Did the program make use of an adequate number of         | Yes      |
| reviewers for a balanced review?                          |          |
|                                                           |          |
| *In the vast majority of cases an adequate number of ad   |          |
| hoc reviewers was obtained. Often, however, this required |          |
| Herculean efforts by the Program Directors. (see comments |          |
| below)*                                                   |          |
+-----------------------------------------------------------+----------+
| Did the program make use of reviewers having appropriate  | Yes      |
| expertise and/or qualifications?                          |          |
|                                                           |          |
| *(see comments below)*                                    |          |
+-----------------------------------------------------------+----------+
| Did the program make appropriate use of reviewers to      | Yes      |
| reflect balance among characteristics such as geography,  |          |
| type of institution, and underrepresented groups?         |          |
|                                                           |          |
| *It was difficult, given the information we received and  |          |
| the time constraints under which the committee operated,  |          |
| to determine if the ad hoc reviewers were balanced with   |          |
| regard to these characteristics. We examined the panels   |          |
| carefully and found that they were well balanced with     |          |
| regard to the above criteria.*                            |          |
+-----------------------------------------------------------+----------+
| Did the program recognize and resolve conflicts of        | Yes      |
| interest when appropriate?                                |          |
+-----------------------------------------------------------+----------+
| Did the program provide adequate documentation to justify | Yes      |
| actions taken?                                            |          |
|                                                           |          |
| *The committee was extremely impressed with care, detail  |          |
| and thoughtfulness evident in the Form 7 documents.*      |          |
+-----------------------------------------------------------+----------+

Discuss any concerns identified that are relevant to selection of
reviewers in the space below.

The Cluster made use of an adequate number of reviewers with appropriate
expertise, but we noted that achieving this goal required a very high
effort. Sometimes as many as 15-20 reviews had to be solicited to
receive 3 ad hoc reviews. This places an undue workload on the Program
Directors, which is compounded by an inadequate database of email
addresses, requiring additional time for search and location of
addresses. We feel that a better database, and automated review
reminders would be effective in increasing the numbers of reviews and
response rate. We encourage further automation of soliciting reviews,
and reminding reviewers to return their reviews, to increase the rate of
return of reviews and simultaneously reduce workload on Program
Directors.

The programs generally did a very good job of recognizing and resolving
conflicts of interest (i.e., removing from consideration those reviews
with a conflict of interest).

Over the three days of the CoV, there was much discussion concerning the
selection, recruitment and successful solicitation of *ad hoc* reviews,
and the difficulties encountered coaxing reviews from obviously
heavily-committed colleagues. The committee as a whole was unanimously
supportive of the idea of relieving these burdens by 10 making more
personnel available and 2) employing an automated mailing system to
relieve the clerical duties of the Program Directors. A minority opinion
was also provided:

> A CoV member suggested that agreeing to function as an *ad hoc*
> reviewer might be made an obligatory condition of submitting a
> proposal for review, even more emphatically if an award is made. If
> one is willing to burden the system for one's own purposes, then one
> should be willing to contribute to the whole. Such a reviewing burden
> could be weighed in order to fairly distribute the burden. For
> example, any submission, whether awarded or declined, should obligate
> the PI to some number of reviews over the ensuing 2-3 years (the
> duration of the proposed research). PIs whose proposals were denied
> funding would profit by seeing other proposals, an opportunity
> universally acknowledged as the single most useful way to learn how to
> construct a successful proposal. Similarly, scientists enjoying the
> financial rewards of participating in a publicly-supported scientific
> endeavor are singularly beholden to that system, and accordingly
> singularly obligated to fostering its vigor and propagation. For
> example, obligating a PI to some number (1-3?) of ad hoc reviews a
> year for every \$100K received would go a long way towards relieving
> the bottleneck of limited ad hoc review returns. As an extension of
> this idea, this obligation could also be discharged by serving on a
> review panel.

It should be noted that this suggestion was received with skepticism by
the CoV in general. The opposing arguments provided were 1) that the
Program Directors should be able to avoid reviewers whose comments have
repeatedly been biased or superficial, whether they are funded or not,
2) that forced review is not conducive to the solicitation of unbiased
and fair opinions, and 3) that such a procedure, where funded
researchers might dominate the reviewing process, could contribute to a
regrettable status quo in the reviewing and funding process compared to
the current system in which Program Directors seek to achieve an
equitable distribution of reviewers with regard to experience,
geography, gender, minority status, and indeed , funding.

A.4 Questions concerning the resulting portfolio of awards under review.

+------------------------------------------------------+---------------+
| **[Resulting Portfolio of Awards]{.smallcaps}**      | **            |
|                                                      | [Appropriate, |
|                                                      | Not           |
|                                                      | Appropriate,  |
|                                                      | or Data Not   |
|                                                      | Available]{   |
|                                                      | .smallcaps}** |
+------------------------------------------------------+---------------+
| Overall quality of the research and/or education     | **            |
| projects supported by the program.                   | Appropriate** |
+------------------------------------------------------+---------------+
| Are awards appropriate in size and duration for the  | **            |
| scope of the projects?                               | Appropriate** |
+------------------------------------------------------+---------------+
| Does the program portfolio have an appropriate       | **            |
| balance of:                                          | Appropriate** |
|                                                      |               |
| > High Risk Proposals                                |               |
+------------------------------------------------------+---------------+
| > Multidisciplinary Proposals                        | **            |
|                                                      | Appropriate** |
+------------------------------------------------------+---------------+
| > Innovative Proposals                               | **            |
|                                                      | Appropriate** |
+------------------------------------------------------+---------------+
| Of those awards reviewed by the committee, what      | **\>95%**     |
| percentage of projects address the integration of    |               |
| research and education?                              |               |
|                                                      |               |
| *Of the proposals we examined, virtually all of them |               |
| supported the education of undergraduate students,   |               |
| graduate students or postdoctoral fellows. In every  |               |
| case, the role of PI's as mentors of these           |               |
| individuals was discussed. In many cases, the        |               |
| proposal also outlined the PI's role in teaching and |               |
| mentoring other students.*                           |               |
+------------------------------------------------------+---------------+

Discuss any concerns identified that are relevant to the quality of the
projects or the balance of the portfolio in the space below.

*See comments in table above.*

##### [Part B. Results: Outputs and Outcomes of NSF Investments]{.smallcaps}

**B.1.a COV Questions for PEOPLE Goal**

**[NSF OUTCOME GOAL for PEOPLE]{.underline}: Developing "a diverse,
internationally competitive and globally engaged workforce of
scientists, engineers, and well-prepared citizens."**

+------------------------------------------------------+---------------+
| **People Goal Indicators**                           | **Program     |
|                                                      | Achievement** |
|                                                      |               |
|                                                      | *             |
|                                                      | *Significant, |
|                                                      | Not           |
|                                                      | Significant,  |
|                                                      | Does Not      |
|                                                      | Apply, or     |
|                                                      | Data Not      |
|                                                      | Available**   |
|                                                      |               |
|                                                      | ***(select    |
|                                                      | one)***       |
+------------------------------------------------------+---------------+
| Development of well-prepared scientists, engineers   | **            |
| or educators whose participation in NSF activities   | SIGNIFICANT** |
| provides them with the capability to explore         |               |
| frontiers and challenges of the future;              | **Akay        |
|                                                      | 0120792**     |
| ***IBN 0120792 Akay, Metin; Dartmouth College.       |               |
| International Summer School on Biocomplexity.** This | **Cannatella  |
| award fulfills the given people goal indicator to    | 0078150**     |
| develop scientists, engineers, and educators by      |               |
| exposing undergraduate, graduate, and postdoctoral   |               |
| "biological science and bioengineering students to   |               |
| the relatively new approaches of nonlinear dynamical |               |
| analysis and biosignal interpretation in complex     |               |
| biological systems, and to facilitate rapid          |               |
| diffusion of these mathematical and computational    |               |
| tools in the biological sciences. These methods are  |               |
| helpful, also, to both students and post-doctoral    |               |
| fellows in computer science and mathematics who are  |               |
| interested in pursuing research in biology and       |               |
| bioengineering, since the summer school provides     |               |
| exceptional insights into the fundamental challenges |               |
| in biological sciences." Taken from Final Project    |               |
| Report (11/02)*                                      |               |
|                                                      |               |
| ***IBN 0078150 Cannatella, David; U of Texas Austin. |               |
| IRCEB: The Phylogenetics and Functional Integration  |               |
| of Complex Phenotypes Regulating Social/Reproductive |               |
| Interactions.** This award fulfills the given people |               |
| goal indicator to develop scientists, engineers, and |               |
| educators because one of the senior collaborators    |               |
| (Nestor Basso) sequenced several genes for the       |               |
| project and presented a published paper based on his |               |
| findings at a scientific meeting in July 2001. Also, |               |
| three graduate students contributed more than 160    |               |
| hours to the project. Two of these students were     |               |
| newly trained for the project. Taken from Annual     |               |
| Report (6/01)*                                       |               |
+------------------------------------------------------+---------------+
| Improved science and mathematics performance for     | **Does not    |
| U.S. K-12 students involved in NSF activities;       | apply**       |
+------------------------------------------------------+---------------+
| Professional development of the SMET instructional   | **Does not    |
| workforce involved in NSF activities;                | apply**       |
+------------------------------------------------------+---------------+
| Contributions to development of a diverse workforce  | **            |
| through participation of underrepresented groups     | SIGNIFICANT** |
| (women, underrepresented minorities, persons with    |               |
| disabilities) in NSF activities;                     | **Crews       |
|                                                      | 9723617**     |
| ***IBN 9723617 Crews, David; U of Texas Austin.      |               |
| Temperature -- Not Chromosomes -- Determines the     | **Smith       |
| Gender of Certain Lower Vertebrates.** This award    | 011041**      |
| fulfills the people goal indicator of contributing   |               |
| to the development of a diverse workforce by         |               |
| involving many underrepresented groups. There were   |               |
| 21 undergraduate students (six Hispanic Americans    |               |
| and one Native American), five graduate students     |               |
| (two Hispanic Americans), and three postdoctoral     |               |
| associates involved in the project. Taken from Final |               |
| Project Report (11/00)*                              |               |
|                                                      |               |
| ***IBN 0111041 Smith, C. Michael; Kansas State       |               |
| Universit. A Plant Genomics Research and Training    |               |
| Career Advancement Program.** This award fulfills    |               |
| the people goal indicator of contributing to the     |               |
| development of a diverse workforce because the PI is |               |
| a member of an underrepresented minority group. No   |               |
| information is given in the electronic jacket about  |               |
| other senior staff, post-doc, graduate, or           |               |
| undergraduate students that might be working on the  |               |
| project. Taken from Certification of Eligibility for |               |
| Minority Research Planning Grant (1/01)*             |               |
+------------------------------------------------------+---------------+
| Participation of NSF scientists and engineers in     | **            |
| international studies, collaborations, or            | SIGNIFICANT** |
| partnerships;                                        |               |
|                                                      | **Mok         |
| ***IBN 0119252 Mok, Machteld; Oregon State           | 0119252**     |
| University. U.S.-Czech Regional Workshop on the      |               |
| Plant Hormone Cytokinin.** This award fulfills the   | **Buckendahl  |
| given people goal indicator of participation of NSF  | 074957**      |
| scientists and engineers in international studies,   |               |
| collaborations, or partnerships by sponsoring a      | **Kroodsma    |
| workshop in Prague to foster existing and initiate   | 9408520**     |
| new collaboration between U.S. and Czech scientists. |               |
| There was a particular emphasis on collaborating     |               |
| between junior faculty, postdoctoral associates, and |               |
| graduate students, with the goal of extending        |               |
| research partnership to the next generation of       |               |
| scientists. Taken from Final Project Report (2/02)*  |               |
|                                                      |               |
| ***IBN 0074957 Buckendahl, Patricia; Rutgers         |               |
| University New Brunswick. The Effects of Deletion of |               |
| DBH on Osteocalcin.** This POWRE award fulfills the  |               |
| given people goal indicator of participation of NSF  |               |
| scientists and engineers in international studies,   |               |
| collaborations, or partnerships by establishing      |               |
| collaboration between a U.S. scientist and one at    |               |
| the Slovak Academy of Sciences. Taken from Project   |               |
| Summary*                                             |               |
|                                                      |               |
| ***IBN 9408520 Kroodsma, Donald; U of Mass. Amherst. |               |
| Ecology of Passerine Vocal Development.** This award |               |
| fulfills the given people goal indicator of          |               |
| participation of NSF scientists and engineers in     |               |
| international studies, collaborations, or            |               |
| partnerships by providing funds for the PI to study  |               |
| the relationship between vocal learning and various  |               |
| features of natural history, such as nesting,        |               |
| habitat selection and adult singing in a group that  |               |
| lives in a variety of habitats from Canada to Tierra |               |
| del Fuego. Taken from Project Summary*               |               |
+------------------------------------------------------+---------------+
| Enhancement of undergraduate curricular, laboratory, | **            |
| or instructional infrastructure;                     | SIGNIFICANT** |
|                                                      |               |
| ***This category is particularly well supported in   | **Young       |
| this cluster in the form of RUIs.***                 | 0116266**     |
|                                                      |               |
| ***IBN 0116266 Young, Jeffery; Western Washington    |               |
| University. RUI: Genetic and Physiological           |               |
| Characterization of Arabidopsis Plasma Membrane      |               |
| H+-ATPase Mutants***                                 |               |
|                                                      |               |
| *This award fulfills the given people goal indicator |               |
| to enhance undergraduate curricular, laboratory, or  |               |
| instructional infrastructure by allowing nine        |               |
| undergraduates to work on the project, eight of whom |               |
| worked over 160 hours. Taken from Annual Report      |               |
| (3/02)*                                              |               |
+------------------------------------------------------+---------------+
| Awardee communication with the public in order to    | **            |
| provide information about the process and benefits   | SIGNIFICANT** |
| of NSF supported science and engineering activities. |               |
|                                                      | **Pepperberg  |
| ***IBN 9603803 Pepperberg, Irene; University of      | 9603803**     |
| Arizona. Evaluation of Avian Cognitive Abilities.**  |               |
| This award fulfills the given idea indicators        |               |
| because the principle investigator, graduate         |               |
| student, and undergraduate student presented posters |               |
| at the Southwestern Rocky Mountain Division of the   |               |
| American Association for the Advancement of Science  |               |
| Annual Meeting in Grand Junction, Colorado. (Also    |               |
| presented at the Ninth Annual Undergraduate Biology  |               |
| Research Conference, Tucson, AZ; and the             |               |
| Undergraduate Research Forum, Tucson, AZ). Also, the |               |
| three of them were invited to Colloquia and Outreach |               |
| Talks to provide information about the process and   |               |
| benefits of NSF supported science and engineering    |               |
| activities. Taken from Annual Report                 |               |
| 01/1998-12/1998*                                     |               |
+------------------------------------------------------+---------------+

B.1.b COV Questions related to PEOPLE Areas of Emphasis

+-------------------------------------------+--------------------------+
| **People Areas of Emphasis**              | **[Demonstrates          |
|                                           | Likelihood of Strong     |
|                                           | Performance in           |
|                                           | Future?]{.smallcaps}**   |
|                                           |                          |
|                                           | **[Yes, No, Does Not     |
|                                           | Apply or Data Not        |
|                                           | Available]{.smallcaps}** |
|                                           |                          |
|                                           | **[(select               |
|                                           | one)]{.smallcaps}**      |
+-------------------------------------------+--------------------------+
| K-12 Education -President's Math and      | **Does not apply**       |
| Science Partnership                       |                          |
+-------------------------------------------+--------------------------+
| **Learning for the 21^st^ Century:**      | **Does not apply**       |
|                                           |                          |
| Centers for Learning and Teaching (CLT)   |                          |
|                                           |                          |
| NSF Graduate Teaching Fellows in K-12     |                          |
| Education (GK-12)                         |                          |
+-------------------------------------------+--------------------------+
| **Broadening Participation**              | **Does not apply**       |
|                                           |                          |
| Minority-Serving Institutions (MSI)       |                          |
| programs                                  |                          |
|                                           |                          |
| Graduate Student Stipends                 |                          |
|                                           |                          |
| Increasing stipends for GRF, IGERT, and   |                          |
| GK-12                                     |                          |
+-------------------------------------------+--------------------------+

Comment on steps that the program should take to improve performance in
areas of the PEOPLE goal.

*Does not apply.*

**B.2.a COV Questions for IDEAS Goal**

**[NSF OUTCOME GOAL for IDEAS]{.underline}: Enabling "discovery across
the frontier of science and engineering, connected to learning,
innovation, and service to society."**

+-------------------------------------------------------+--------------+
| **[Ideas Indicators]{.smallcaps}**                    | **[Program   |
|                                                       | Ac           |
|                                                       | hievement]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[Sig       |
|                                                       | nificant,]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[Not       |
|                                                       | Significant, |
|                                                       | Does Not     |
|                                                       | Apply, or    |
|                                                       | Data Not     |
|                                                       | Available]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[(Select   |
|                                                       | One)]{.      |
|                                                       | smallcaps}** |
+-------------------------------------------------------+--------------+
| Discoveries that expand the frontiers of science,     | **S          |
| engineering, or technology;                           | IGNIFICANT** |
|                                                       |              |
| ***IBN 0078155 Holbrook, Noel; Harvard University.    | **Holbrook   |
| Mechanisms of Embolism Reversal in Vascular Plants:   | 0078155**    |
| Studies at the Level of Individual Vessels.** This    |              |
| This award fulfills the given idea indicators to      | **Ram        |
| expand the frontiers of science, engineering, or      | akrishnaneia |
| technology by using 7 T magnet to demonstrate the use | 0103660**    |
| of image analysis of regions of the xylem to          |              |
| determine cavitation; allowing to sample at a higher  |              |
| frequency and to work with plants with smaller        |              |
| diameter vessels. Phloem girdling in intact,          |              |
| transpiring branches results in a short-term decrease |              |
| in xylem hydraulic conductance that appears to be     |              |
| mediated by a decrease in xylem sap concentration. It |              |
| is thought that phloem girdling reduces ion loading   |              |
| into the xylem. Paper submitted: Plant Physiology.    |              |
| Individual vessel study of xylem cavitation           |              |
| thresholds in A. saccharum demonstrates that younger  |              |
| vessels are more resistant to cavitation. This        |              |
| demonstrated that the shape of vulnerability curves   |              |
| is determined in large part by this age effect.       |              |
| Manuscript in preparation. Taken from Annual Report   |              |
| 08/2000-05/2001*                                      |              |
|                                                       |              |
| ***IBN 0103660 Ramakrishnaneia, Naren; VA Polytechnic |              |
| Inst St U. NGS: A Microarray Experiment Management    |              |
| System.** This award fulfills the given idea          |              |
| indicators to expand the frontiers of science,        |              |
| engineering, or technology by implementing Expresso:  |              |
| a microarray experiment management system. This       |              |
| experiment involves a large number of stages,         |              |
| typically performed over a period of weeks or months. |              |
| Expresso supports all of these stages                 |              |
| computationally, including design pro-cessing, both   |              |
| manually and robotically, of hundreds or thousands of |              |
| distinct biological materials (DNA and RNA);          |              |
| retrieving the experimental results as scanned        |              |
| images; extracting meaningful information from these  |              |
| images; analyzing the results of an experiment as a   |              |
| whole; and providing the biologists with tools to     |              |
| explore and mine these results. Taken from Project    |              |
| Summary.*                                             |              |
+-------------------------------------------------------+--------------+
| Discoveries that contribute to the fundamental        | **S          |
| knowledge base;                                       | IGNIFICANT** |
|                                                       |              |
| ***IBN 9709000 Breznak, John; Michigan State          | **Breznak    |
| University. Physiological Role and Phylogenetic       | 9709000**    |
| Diversity of Termite GutSymbionts.** This award       |              |
| fulfills the given idea indicators that contribute to | **Roer       |
| the fundamental knowledge base that enhances the      | 0114597**    |
| training of scientists at various levels, from        |              |
| postdoctoral students, to graduate students, to       |              |
| technicians. Even undergraduates are hired as         |              |
| laboratory attendants to gain first-hand knowledge of |              |
| fundamental techniques employed in a research         |              |
| laboratory (sterile technique, preparation of media   |              |
| and solutions, etc.). Several participants are        |              |
| underrepresented minorities: (Kim and                 |              |
| Hashimoto;Asian/Pacific Islander), Harris (African    |              |
| American), and hence represent a contribution to      |              |
| diversity of the human resource base. Of course, new  |              |
| concepts and findings are integrated into the         |              |
| educational process by virtue of the fact that a      |              |
| major role of the PI at his home institution is in    |              |
| the education of undergraduates and graduates in the  |              |
| classroom. Taken from Annual Report 03/1998-02/1999.* |              |
|                                                       |              |
| ***IBN 0114597 Roer, Robert; UNC-Wilmington. The Blue |              |
| Crab Exoskeleton: A Model for studying the Control of |              |
| Biomineralization***. *Mineralized skeletons, whether |              |
| they are bones, teeth, mollusc shells or crustacean   |              |
| exoskeletons, are all comprised of two structural     |              |
| components: an organic matrix and the minerals that   |              |
| impregnate it. The organic matrix is a complex        |              |
| mixture of proteins and carbohydrates; sometimes the  |              |
| proteins have carbohydrates attached to them and are  |              |
| referred to as glycoproteins, proteoglycans or        |              |
| mucins. The minerals may be crystalline forms of      |              |
| calcium phosphate, like the hydroxyapatite of bones   |              |
| and teeth, or calcium carbonate, like the calcite of  |              |
| crab exoskeletons. One of the central and most basic  |              |
| questions in the field of skeleton formation          |              |
| (biomineralization) is which of the complex of matrix |              |
| molecules are the ones that actually control the      |              |
| initiation of skeletal hardening and control the      |              |
| location and form of the mineral. This project        |              |
| utilizes the blue crab as a model for the control of  |              |
| biomineralization. Because crabs molt in order to     |              |
| grow, they provide an ideal system in which to study  |              |
| the biological control of mineralization and the      |              |
| interaction between the organic and mineral           |              |
| components of the skeleton. Taken from the Award      |              |
| Abstract.*                                            |              |
+-------------------------------------------------------+--------------+
| Leadership in fostering newly developing or emerging  | **S          |
| areas;                                                | IGNIFICANT** |
|                                                       |              |
| ***IBN 9728608 Page, Robert; University of            | **Page       |
| California-Davis. Genetic, Developmental, and         | 9728608**    |
| Environmental Determinants of Honey Bee Foraging      |              |
| Behavior.** This award fulfills the given idea        | **Bennett    |
| indicators that lead in fostering newly developing    | 9905980**    |
| and emerging areas that will have neurobiologists     |              |
| rethinking the relationships of \'motivation\' and    |              |
| learning performance. It also redefined motivation in |              |
| terms of response thresholds and reward values and    |              |
| developed some molecular tools for identifying        |              |
| molecular markers that will be used for genetic       |              |
| mapping and population genetic studies. Taken from    |              |
| Annual Report 03/1999-02/2000*                        |              |
|                                                       |              |
| ***IBN 9905980 Bennett, Albert; University of         |              |
| California-Irvine. Experimental Evolutionary Studies  |              |
| of Temperature Adaptation: Evolution in a Constant    |              |
| Thermal Environment***                                |              |
|                                                       |              |
| *This award fulfills the given idea indicators that   |              |
| lead in fostering newly developing or emerging areas  |              |
| in which bacterial lineages have developed to be      |              |
| useful in other disciplines in biology, such as       |              |
| microbiology or molecular biology, in providing       |              |
| systems for the molecular and genetic analysis of     |              |
| environmental adaptation. Taken from Annual Report    |              |
| 8/1999-7/2000*                                        |              |
+-------------------------------------------------------+--------------+
| Connections between discoveries and their use in      | **S          |
| service to society;                                   | IGNIFICANT** |
|                                                       |              |
| ***IBN 9870793 Espey, Lawrence; Trinity University.   | **Espey      |
| Analysis of Ovarian Gene Expression during            | 9870793**    |
| Ovulation.** This award fulfills the given idea       |              |
| indicators of connections between discoveries and     | **Zuk        |
| their use in service to society by providing the      | 9902255**    |
| basic information about a major event in fertility    |              |
| and reproduction, namely the ovulatory process.       | **West       |
| Therefore, the findings can increase the chance of    | 9809883**    |
| finding more effective means of regulating fertility  |              |
| in humans. It also has the potential of leading to an |              |
| increase in production of livestock and other food    |              |
| resources. Therefore, in an indirect way, the project |              |
| is related to population growth, and the ultimate     |              |
| problems that mankind will likely face during the     |              |
| next century as population pressures continue to      |              |
| rise.*                                                |              |
|                                                       |              |
| ***IBN 9902255 Zuk, Marlene; U of California,         |              |
| Riverside. DISSERTATION RESEARCH: The Effects of a    |              |
| Phonotactic Parasitoid Fly on Geographic Variation in |              |
| Calling Behavior of the Field Cricket, Teleogryllus   |              |
| oceanicus.*** *This award fulfills the idea indicator |              |
| of expanding the frontiers of science, engineering,   |              |
| or technology through discoveries in the following    |              |
| way: the parasitoid fly has recently received a great |              |
| deal of attention from neurobiologists studying how   |              |
| animals process directional information from acoustic |              |
| signals. This work and that of other behavioral       |              |
| biologists has drawn attention to this fly as a       |              |
| valuable tool for studying responses to acoustic      |              |
| signals. It is hoped that neurobiologists will soon   |              |
| develop specialized hearing aids for human use based  |              |
| on the principles deduced from studying this unique   |              |
| parasitoid. Taken from Final Project Report (5/01)*   |              |
|                                                       |              |
| ***IBN 9809883 West, Meredith J.; Indiana University. |              |
| Experiential Influences on Perceptual, Communicative, |              |
| and Neural Capacities**. The following excerpt        |              |
| explains the relevance of the award to the above idea |              |
| indicator: the results of the work should advance     |              |
| knowledge of the effects of early social environments |              |
| on vocal and perceptual learning, learning that is    |              |
| fundamental to an individual's success. This topic    |              |
| has broad implications for many species, including    |              |
| humans, as avian vocal learning is one of the premier |              |
| models for identifying potential mechanisms of        |              |
| language learning. Of special significance is the     |              |
| possibility of identifying previously ignored sources |              |
| of instruction including visual stimulation. The      |              |
| coordinated studies of brain and behavior also set    |              |
| the stage for measuring how behavioral experiences    |              |
| affect brain growth itself. Taken from Abstract       |              |
| (6/98)*                                               |              |
+-------------------------------------------------------+--------------+
| Connections between discovery and learning or         | **S          |
| innovation;                                           | IGNIFICANT** |
|                                                       |              |
| ***IBN 9904432 Wagner, Allan R.; Yale University.     | **Wagner     |
| Mechanisms of Associative Learning.***                | 9904432**    |
|                                                       |              |
| *This award is an example of the idea indicator of    | **Trotter    |
| connections between discovery and learning or         | 9723260**    |
| innovation. The project has provided research         |              |
| training to two graduate students and two             |              |
| undergraduates, who were involved in the grant        |              |
| research. Edgar Vogel is pursuing his Ph. D. under    |              |
| PI's direction, has coauthored a number of articles   |              |
| that have been published or are in print, and has had |              |
| an intellectual involvement in all of the projects of |              |
| the laboratory during the last year. Karen Myers has  |              |
| completed her predissertation research under PI's     |              |
| direction and collaborated in several other projects  |              |
| as reported. She also plans to complete her Ph. D.    |              |
| and pursue an academic career. Daniel Gottleib did    |              |
| his senior thesis in the laboratory and has gone on   |              |
| to graduate school in experimental psychology. Anh    |              |
| Chung, who also did his senior thesis in the          |              |
| laboratory, has taken a teaching positon for next     |              |
| year. Taken from Annual Report (5/00)*                |              |
|                                                       |              |
| ***IBN 9723260 Trotter, John A.; University of New    |              |
| Mexico Health Sciences Center. Molecular Organization |              |
| in Mutable Collagenous Tissues**. This award is an    |              |
| example of a connection between discovery and         |              |
| learning or innovation because their work has led to  |              |
| a biomimetic approach to producing a novel material   |              |
| that would have regulated mechanical properties. This |              |
| approach has been funded by DARPA as a                |              |
| multidisciplinary program. Taken from Final Project   |              |
| Report (6/01)*                                        |              |
+-------------------------------------------------------+--------------+
| Partnerships that enable the flow of ideas among the  | **S          |
| academic, public or private sectors.                  | IGNIFICANT** |
|                                                       |              |
| ***IBN 9981974 Mok, David W.; Oregon State            | **Mok        |
| University. Regulation of cytokinin metabolic         | 9981974**    |
| genes.** This award enables the flow of ideas among   |              |
| the academic, public, or private sectors through the  | **Feder      |
| involvement of the following individuals:*            | 0097876**    |
|                                                       |              |
| > ***University of Antwerp, Belgium.** Dr. Harry van  |              |
| > Onckelen\'s lab. Quantification of cytokinin        |              |
| > content in transgenic plants.; University of        |              |
| > Missouri-Columbia. Dr. Zhan-yuan Zhang,             |              |
| > collaboration on the generation of transgenic maize |              |
| > and soybean.;* ***Czech Academy of Sci.Inst.Exp.    |              |
| > Bot.** Czech Academy of Sciences, Institute of      |              |
| > Experimental Botany. Dr. R. Vankova\'s laboratory   |              |
| > (US NSF-Czech International Collaborative Project): |              |
| > effects of modulating cytokinin metabolic genes on  |              |
| > plant development.; **Pioneer Hi-Bred International |              |
| > Inc.** Dr. Jeff Habben, isolation of cytokinin      |              |
| > genes in maize.*                                    |              |
|                                                       |              |
| *Taken from Annual Report (2/02)*                     |              |
|                                                       |              |
| ***IBN 00987876 Feder, Martin E.; Society for         |              |
| Integrative and Comparative Biology. Symposium: Plant |              |
| and animal physiological ecology, comparative         |              |
| physiology/biochemistry, and evolutionary physiology: |              |
| opportunities for synergy.** This award is relevant   |              |
| to the idea indicator that encourages the flow of     |              |
| ideas among the academic, public, or private sectors  |              |
| because the symposium would be part of a broader      |              |
| initiative to promote greater interaction and         |              |
| collaboration between plant ecophysiologists (as      |              |
| represented by the Physiological Ecology Section of   |              |
| the Ecological Society of America)and animal          |              |
| ecophysiologists (as represented by the Division of   |              |
| Comparative Physiology and Biochemistry of the        |              |
| Society for Integrative and Comparative Biology). The |              |
| symposium itself is intended as a scholarly program   |              |
| of such interest that both plantand animal            |              |
| ecophysiologists would attend the meeting and thereby |              |
| begin interacting. Taken from Project Summary 2/02*   |              |
+-------------------------------------------------------+--------------+

B.2.b COV Questions related to IDEAS Areas of Emphasis

+--------------------------------------------------------+-------------+
| **[Ideas Areas of Emphasis]{.smallcaps}**              | **[D        |
|                                                        | emonstrates |
|                                                        | Likelihood  |
|                                                        | of Strong   |
|                                                        | Performance |
|                                                        | in          |
|                                                        | Future?]{.s |
|                                                        | mallcaps}** |
|                                                        |             |
|                                                        | **[Yes, No, |
|                                                        | Does Not    |
|                                                        | Apply or    |
|                                                        | Data Not    |
|                                                        | Av          |
|                                                        | ailable]{.s |
|                                                        | mallcaps}** |
|                                                        |             |
|                                                        | **[(select  |
|                                                        | one)]{.s    |
|                                                        | mallcaps}** |
+--------------------------------------------------------+-------------+
| Biocomplexity in the Environment                       | **SI        |
|                                                        | GNIFICANT** |
| ***IBN 0110666 Houck, Lynne D.; Oregon State           |             |
| University. IRC: The Evolution of a Pheromone          | **Houck     |
| Signaling System: From Molecules to Mating.** This     | 0110666**   |
| award supports research in biocomplexity in the        |             |
| environment. Terrestrial salamanders have a unique     |             |
| system of chemical communication. In their subsequent  |             |
| evolutionary history, plethodontids modified this      |             |
| pheromone delivery system in a variety of ways,        |             |
| including delivery methods in which the pheromone is   |             |
| \"injected\" into the female's circulatory system.     |             |
| This chemical communication system -unique among       |             |
| vertebrates- has been retained by diverse groups of    |             |
| plethodontids throughout their evolutionary radiation. |             |
| Taken from Abstract (6/01)*                            |             |
+--------------------------------------------------------+-------------+
| Information Technology Research                        | **Does not  |
|                                                        | apply**     |
+--------------------------------------------------------+-------------+
| Nanoscale Science and Engineering                      | **Does not  |
|                                                        | apply**     |
+--------------------------------------------------------+-------------+
| Interdisciplinary mathematics                          | **Does not  |
|                                                        | apply**     |
+--------------------------------------------------------+-------------+

**B.3.a COV Questions for TOOLS Goal**

**[OUTCOME GOAL for TOOLS:]{.underline} Providing "broadly accessible,
state-of-the-art and shared research and education tools."**

+-------------------------------------------------------+--------------+
| **[Tools Indicators]{.smallcaps}**                    | **[Program   |
|                                                       | Ac           |
|                                                       | hievement]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[Sig       |
|                                                       | nificant,]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[Not       |
|                                                       | Significant, |
|                                                       | Does Not     |
|                                                       | Apply, or    |
|                                                       | Data Not     |
|                                                       | Available]{. |
|                                                       | smallcaps}** |
|                                                       |              |
|                                                       | **[(Select   |
|                                                       | One)]{.      |
|                                                       | smallcaps}** |
+-------------------------------------------------------+--------------+
| Provision of facilities, databases or other           | **S          |
| infrastructure that enable discoveries or enhance     | IGNIFICANT** |
| productivity by NSF research or education             |              |
| communities;                                          | **Holekamp   |
|                                                       | 0113170**    |
| ***IBN 0113170 Holekamp, Kay E.; Michigan State       |              |
| University. LTREB: Integrated study of behavioral and | **Bart       |
| morphological development in free-living              | 0097004**    |
| carnivores.** Dr. Holekamp has maintained excellent   |              |
| working relationships with all relevant government    |              |
| officials in Kenya at both local and national levels. |              |
| The PI has research clearance from the Office of the  |              |
| President of Kenya to conduct fieldwork, as well as   |              |
| all necessary permits for export of skulls, blood,    |              |
| tissue, and fecal material to the MSU laboratories.   |              |
| The PI works closely with high-ranking individuals at |              |
| the University of Nairobi and the Kenya Wildlife      |              |
| Service. NSF support of this research has also        |              |
| yielded myriad benefits to lay and scientific         |              |
| communities, including an annual undergraduate field  |              |
| course in Kenya that uses equipment borrowed from the |              |
| PI's research camp                                    |              |
| (http://www.msu.edu/\~szykmanm/beamweb), and several  |              |
| publications and films for lay audiences (eg., Smale  |              |
| & Holekamp 1991, 1993; Holekamp & Smale 2000b;        |              |
| Discovery Channel film provided to program officer).  |              |
| The Holekamp camp has served as a field base for      |              |
| scientists studying other African mammals (eg., Van   |              |
| Valkenburgh 1996; Blanchong et al. 1999; Blanchong &  |              |
| Smale 2000); they have assisted African wildlife      |              |
| managers concerned with "problem" hyenas, and have    |              |
| made available to many investigators samples of hyena |              |
| blood, DNA, or skeletal material (eg., Biknevicius    |              |
| 1996; Biknevicius & Leigh 1997). The PI has also made |              |
| unpublished DNA sequences from Crocuta available to   |              |
| other investigators, and has contributed to the MSU   |              |
| Museum 61 complete hyena skulls; for 43 of these,     |              |
| exact age was known at time of death. Taken from      |              |
| Generated Facilities (no date given) and Project      |              |
| Description (no date given)*                          |              |
|                                                       |              |
| ***IBN 0097004 Bart, Henry L.; Tulane University.     |              |
| Improvements to the Tulane Fish Collection: Final     |              |
| phase.** The following passage explains why this      |              |
| award is an example of the above idea indicator: we   |              |
| completed the project ahead of schedule and began     |              |
| verifying the collection against the database. The    |              |
| collection is now fully computerized and              |              |
| georeferenced. Computerization has significantly      |              |
| improved data access for staff and outside            |              |
| researchers, and has allowed us to undertake a        |              |
| thorough inventory of all lots in the collection.     |              |
| Tulane University provided the computer network       |              |
| necessary to complete this project, and is providing  |              |
| continuing financial support in the form of a         |              |
| dedicated ISDN data line to connect the Museum to the |              |
| Internet.* *Taken from Project Description (no date   |              |
| given)*                                               |              |
+-------------------------------------------------------+--------------+
| Provision of broadly accessible facilities, databases | **Does not   |
| or other infrastructure that are widely shared by NSF | apply**      |
| research or education communities;                    |              |
+-------------------------------------------------------+--------------+
| Partnerships, e.g., with other federal agencies,      | **S          |
| national laboratories, or other nations to support    | IGNIFICANT** |
| and enable development of large facilities and        |              |
| infrastructure projects;                              | **Kochian    |
|                                                       | 0129844**    |
| ***IBN 0129844 Kochian, Leon V.; Boyce Thompson       |              |
| Institute. The Molecular Basis for Heavy Metal        |              |
| Accumulation and Tolerance in the Hyperaccumulating   |              |
| Plant Species, Thlaspi caerulescens**. This project   |              |
| was funded through the Joint Program on               |              |
| Phytoremediation, co-sponsored by the Environmental   |              |
| Protection Agency, the National Science Foundation,   |              |
| the Office of Naval Research, and the Strategic       |              |
| Environmental Research and Development Program.       |              |
| Contamination of soils with toxic heavy metals is a   |              |
| serious worldwide problem both for human health and   |              |
| agriculture. Cleanup of hazardous wastes by the       |              |
| currently used engineering-based technologies has     |              |
| been estimated to cost at least \$400 billion in the  |              |
| U.S. alone. Recently, there has been considerable     |              |
| interest in the use of terrestrial plants as an       |              |
| alternative, \"green technology\" for the remediation |              |
| of surface soils contaminated with toxic heavy        |              |
| metals. A major factor behind the recent interest in  |              |
| phytoremediation of metal polluted soils has been the |              |
| growing awareness by the scientific community of the  |              |
| existence of a number of plant species that not only  |              |
| can tolerate high levels of toxic heavy metals in the |              |
| soil, but actually can accumulate these metals to     |              |
| very high levels in the easily harvested above-ground |              |
| shoot biomass. The ultimate goal of this research is  |              |
| to develop transgenic plants that both are metal      |              |
| hyperaccumulators and produce high shoot biomass ,    |              |
| and thus will be well suited for the phytoremediation |              |
| of metal contaminated soils.*                         |              |
+-------------------------------------------------------+--------------+
| Use of the Internet to make SMET information          | **Does not   |
| available to the NSF research or education            | apply**      |
| communities;                                          |              |
+-------------------------------------------------------+--------------+
| Development, management, or utilization of very large | **Does not   |
| data sets and information-bases;                      | apply**      |
+-------------------------------------------------------+--------------+
| Development of information and policy analyses that   | **Does not   |
| contribute to the effective use of science and        | apply**      |
| engineering resources.                                |              |
+-------------------------------------------------------+--------------+

B.3.b COV Questions related to TOOLS Areas of Emphasis

+------------------------------------------+---------------------------+
| **[Tools Areas of                        | **[Demonstrates           |
| Investments]{.smallcaps}**               | Likelihood of Strong      |
|                                          | Performance in            |
|                                          | Future?]{.smallcaps}**    |
|                                          |                           |
|                                          | **[Yes, No, Does Not      |
|                                          | Apply or Data Not         |
|                                          | Available]{.smallcaps}**  |
|                                          |                           |
|                                          | **[(select                |
|                                          | one)]{.smallcaps}**       |
+------------------------------------------+---------------------------+
| Major Research Equipment (MRE)           | Does not apply            |
+------------------------------------------+---------------------------+
| Major Research Instrumentation (MRI)     | Does not apply            |
| Program                                  |                           |
+------------------------------------------+---------------------------+
| Science & Engineering information,       | Does not apply            |
| reports, and databases                   |                           |
+------------------------------------------+---------------------------+
| Scientific databases and tools for using | Does not apply            |
| them                                     |                           |
+------------------------------------------+---------------------------+
| National SMETE Digital Library           | Does not apply            |
+------------------------------------------+---------------------------+

B.4 Please comment on any program areas in need of improvement.

The COV was impressed by the excellent job the Cluster is doing in
funding innovative and high quality research in all of the Programs. The
average size of awards for the 1999-2001 period was significantly higher
than for the previous review period. As a result, the award amounts were
significantly closer to the actual amounts requested by the PI's than in
the previous review periods. This clearly is a positive development for
researchers funded by the Cluster.

The Program Directors have worked hard to maintain a good mix of high
risk and multidisciplinary proposals. It was noted that there are a
relatively small number of multidisciplinary proposals that cut across
several Programs/Divisions, but it was felt this is appropriate for this
Cluster. The research funded by the cluster is by its very nature
integrative, often encompassing the full breadth of molecular, cellular
and organismal enquiry. The small number of grants officially designated
as multidisciplinary may therefore be small, but the number of proposals
using diverse techniques and with broad implications is not.

With regards to the appropriateness of the subject matter funded, it
appears that the Cluster is supporting a good mix of physiologically-
and behaviorally-oriented research that ranges from the level of the
gene all the way to the whole organism level.

It was noted that the IPB program is supporting research on a wide range
of plant species. A significant number of proposals involve research
employing the model plant species, Arabidopsis thaliana. This is due in
part to the impact of the Arabidopsis 2010 Program, which has had a
positive impact in redistributing the portfolio of IPB resources. This
has led to an increased diversity of research projects in IPB, The
increasing use of resources and knowledge generated by NSF-funded
Arabidopsis genome research is leading to a better understanding of
other plant species, including a number of economically important crop
species.

B.5 Comment as appropriate on the program's performance in meeting
program-specific goals and objectives which are not covered by the above
questions.

*No comment.*

**B.6 NSF would appreciate your comments for improvement of the CoV
review process, format and report template.**

When the committee members agreed to serve on the Committee of Visitors,
those members that had not previously served on a CoV were uncertain of
the role we were to play. Although we were provided with a huge
collection of supporting documents (see below) we did not have a copy of
the previous report, or a committee charge that might serve to guide our
evaluation of the data provided. We respect NSF's desire and right to
provide information at the time and in the order that they prefer. We
would suggest, however, that at a minimum, the report template be
provided to committee members early in the process so they can begin to
understand the issues they will be exploring.

Our Committee of Visitors was provided with the supporting documents and
data required for our analysis approximately five days (three working
days) before the date we traveled to Washington for our meeting. The
data were provided on the World Wide Web. Clearly, three working days is
not sufficient time for a thorough review. Some members of the committee
printed out all the information provided on the web, others read as much
as they could and printed out some sections. Upon arrival at NSF
headquarters, the committee was provided with two complete printed
copies of the material provided on the web. For some committee members,
some parts of this material had not been accessible in their web
versions, apparently for technical reasons.

It was striking to members of the committee that the printed material
was easier to read, easier to navigate within, and readily portable
about the room. Tabs provided insights into the various topics
available, and it was easy to move from one section to another. The take
home message from these observations is that printed material is much
easier to read and assimilate than is material on the web. Even if the
material is printed off, it becomes a huge stack of paper, not as
clearly organized as a notebook with sections, tabs, etc.

It may be that NSF wishes to put the material on the web to avoid the
personnel, printing and mailing costs associated with a hard-copy
version. If material continues to be provided on the web, NSF should
provide a table of contents with links allowing the reader to move
quickly to this section. A hastily-prepared, web-based system in which
information is provided without navigational aids makes the Committee's
task unnecessarily more difficult than that faced by previous
committees.

Similarly, when we arrived at NSF headquarters and began to peruse the
proposals, computer-based investigation of the proposals was very
cumbersome. In some cases this was due to the fact that files more than
two years old had not been entered into the system. Even more recent
proposals could not be reviewed because computers would crash and had to
be rebooted with passwords from NSF personnel. This may be due in part
to the committee members' unfamiliarly with the system and the need for
passwords to navigate within it. Therefore while the system may (or may
not) function well for in-house users, it should be remembered that the
Committees of Visitors are not such in-house personnel and the system
functioned very poorly for their needs. The committee quickly learned to
ignore the computers and work with the proposal jackets, a bookkeeping
system that was astonishingly efficient, thorough and reliable.

**Appendix A: Summary and Overview**

Many portions of the form provided above request the Committee's
comments on concerns identified. We have provided those as requested. We
wish, however, to also provide some summary comments which express the
overwhelming positive impression with which we were left, following our
evaluation of the Physiology and Ethology Cluster.

It was the unanimous opinion of the Committee of Visitors that the
Cluster is very effective in carrying out the programs it has been
assigned to administer. There is ample evidence of enormous efforts made
to review the grants fairly, thoroughly and with a maximum input from
scientists who are diverse both in knowledge and background. Although
the procedures followed are extensive, the reviews are carried out in a
very timely way. Although the proposal load experienced by the Cluster
has risen substantially during the time period under review, the quality
of review did not suffer and indeed can be seen to improve in a number
of areas described above.

Members of the committee who had no previous experience with NSF's
review procedures were very favorably impressed with the clear evidence
of fairness, scientific expertise, and care demonstrated by the review
procedures. The cluster is doing an excellent job of fostering and
promoting scientific discovery, enhancing scientific infrastructure, and
supporting the training of scientific personnel in the areas of
Physiology and Ethology.

Finally, the frank and direct answers from each of the NSF officials
with whom we met, the willingness to solicit and listen to criticism,
coupled with the motivation to improve; all reflect very positively on
both the individuals involved, and on the institutional gestalt and
culture. In an organization where funding decisions are based on
clear-eyed review of merit, it is heartening that the personnel and
programs are equally open to introspection and review.

Our Committee of Visitors was instructed to review the Physiology and
Ethology Cluster as a whole, and in the material presented above we have
attempted to do so. There are, however, some aspects of the individual
programs that are best discussed individually, program by program. We
have therefore included such comments as appendices, one for each
Program in the Cluster.

# Appendix B: Animal Behavior Program

The Program in Animal Behavior is a vigorous program that plays a
central role in the integration of research across all levels of biology
from genes to whole organisms. As such, it includes numerous
subdisciplines that touch on many other programs and clusters in the BIO
directorate. Moreover, research funded by this Program does not involve
one or two model organisms, but includes many organisms of diverse taxa
across a variety of systems and environmental conditions, thereby
providing a much broader perspective than that in many other programs.

We examined approximately 30 grants that included: awarded; declined;
and recommended for funding but not funded. This review of the operation
of the program indicated that it was being accomplished fairly and
conscientiously, with informative panel summaries. The Panels generally
provided sufficient information to the PIs regarding the actions taken.
Jacket documentation was clear, and understandable. In general, there
seemed to be sufficient numbers of ad hoc reviewers, an issue raised by
the COV reported 3 years ago. However, we noted that in many cases, a
large number of reviewers had to be contacted to obtain a sufficient
number of ad hoc reviews.

We feel it is important to point out that NSF is the primary, and often
the sole, source of funding for scientists wishing to address new
principles and concepts in Animal Behavior. Despite the necessity of
this funding to continue growth of this field, the data on funding
levels and rates among the 4 programs in the Physiology and Ethology
cluster clearly indicate that funding is not commensurate with vigor of
activity, interest and quality of science conducted in the Animal
Behavior area. ***The Animal Behavior program is the most underfunded
program relative to numbers of proposals submitted, percentage funded,
and size of awards in IBN.*** Within the P&E cluster, the proportion of
proposals funded was lowest (and 10% lower than other programs) for
Animal Behavior, despite Animal Behavior having the second highest
number of proposal submissions (and only 9 less than the number one
area). The 10% difference in funding rate between programs yields
upwards of 20 more proposals being funded in other programs. Yet, the
annual reports clearly indicate the high quality and vigor of research
in animal behavior, as indicated by the many papers published in
*Science* and *Nature* each year as well as the tremendous media
attention. The underfunding of this program has many trickle-down
effects, including lower funding of new, young investigators, fewer new
projects, and reduced support for graduate students and undergraduates
in this active and exciting field. Moreover, discoveries in the area of
animal behavior provide new paradigms for many other programs in sensory
biology and neurosciences. Reduced funding in Animal Behavior therefore
creates a constraint on the gateway to new research programs in these
neuroscience and sensory programs. This underfunding occurs in an
environment where technological advances (e.g., genetics, sensory
systems) that are needed to do cutting-edge science are increasing in
cost. As a result, low funding constrains the more creative and
imaginative investigations that cost more.

Of the four programs in the Physiology and Ethology Cluster, Animal
Behavior showed the largest increase in the proportion of female
investigators funded relative to proposals submitted. We view this
strong support for female investigators as a strong positive component
of the program. In addition, our review indicates that many research
programs of individual PIs have included training of a diverse work
force, including minorities, females, and students from under-developed
countries. These too are very strong components of the Animal Behavior
program.

The Animal Behavior program seems significantly understaffed relative to
the extent of work required. As noted above, obtaining sufficient
numbers of ad hoc reviews is a critical element and requires significant
time because we noted that it commonly requires contact of 15-20
potential reviewers. This contact of reviewers and follow-up reminders
to get sufficient ad hoc reviews needs greater support from added
Science Assistants. In addition, the work load impacts the ability of
the Program Directors to be able to continue their own research program,
having a significant negative impact on their program and, the
consequence is that it has become difficult to recruit new program
directors. A review and modification of the current situation is needed
whereby increased staff help will reduce the work load to allow greater
time for Program Directors to maintain their home research, and possibly
some form of monetary support for the research while at NSF should be
provided. In addition, we recommend that one of the Program Directors
become a permanent position rather than having two rotator positions to
increase the efficacy of the program processes across time.

**Appendix C: Integrative Plant Biology**

A subcommittee of three (Klee, Kochian, Mok) reviewed the IPB Program.
In addition to annual reports provided for the three fiscal years (1999,
2000, 2001), 30 folders (15 awards and 15 declines, five of each
category per year) were randomly selected for detailed review.

The subcommittee feels the selection/decision process was fair and well
executed. The choice of panel and ad hoc review members was appropriate,
covering diverse fields of expertise. Funded projects were
representative of a broad range of research areas, geographic
distribution and types of institutions. Funding level per project was
appropriate. However, if additional funds become available, more worthy
proposals, not funded due to budget constraints, could have been
supported. Other plant systems, in additional to the model plant
Arabidopsis, were well represented, resolving a major concern raised by
the previous PECOV review. The 2010 projects have had a positive impact
in redistributing the portfolio of IPB resources.

The output of funded proposals met the GPRA goals. The subcommittee
endorses the examples highlighted in the annual reports as evidence of
success. Additional examples of the IPB program fulfilling a broad range
of NSF goals are illustrated by the following folders:

> **9817594**: cross-program funding with Metabolic Biology resulting in
> new findings in sulfur metabolism.
>
> **9904910**: funding high-risk topics in an under-researched area with
> productive outcomes in understanding the biochemistry of novel
> metabolites.
>
> **9808197**: supported anticipated need for development of novel model
> plant systems (Medicago genomics) with successful outcome.
>
> **0119306**: effective integration of research and education at an
> undergraduate institution.
>
> **0074617**: POWRE grant to support a mid-career changes resulting in
> successful career improvement.
>
> **0116266**: integration of Arabidopsis 2010 project with
> undergraduate training.
>
> **0114131**: effective communication of panel suggestions to PI
> resulting in successful outcome upon resubmission.
>
> **0090766**: investment in high risk proposal by established PI
> resulting in significant advances in plant defense mechanisms.

The average number of ad hoc reviews per proposal is 3.5, an improvement
from the previous P&E CoV review, and is the result of diligent
solicitation by the Program Directors. The funding of REUs (nine in
2001) is to be commended as it is indicative of the Program's effort in
the training of undergraduates in plant biology. The number of IPB
proposal has been increasing steadily (128, 136 and 159) and is high
within the cluster. Support staff is not adequate for the work load. A
science assistant should be provided. IPB supports over 50% of basic
plant research in the country and represents the only substantial
funding source for investigator initiated (vs team) and idea-driven (vs
technology/theme) research projects.

##### Appendix D: Integrative Animal Biology

A subcommittee of three (A. G. Campbell, T. Brady, P. Dickinson)
reviewed the IAB program as part of the cluster COV review. In addition
to the information provided in the annual reports for each of the three
years, we examined 26 folders, 6 randomly selected from each of three
groups: clear awards, clear declines, and borderline cases, as well as 8
declines with unusually long time-to-notifications.

The Integrative Animal Biology program supports work in a variety of
sub-fields, including comparative animal physiology and biochemistry
(including endocrinology), biomechanics and functional morphology, and
comparative immunology. Some of these areas are making or poised to make
significant moves forward, so continued NSF support is very important.
For most of the research supported by this program, NSF is the only
source of support; for others, it provides at least 70% of the support
for the area.

Over the three years we reviewed, the number of proposals submitted to
the IAB program increased by 50% (from 104 to 155), a trend indicative
of the vitality of the field. Over the same time, the average award
amount increased from \$72,000 to \$99,000/year (a 37.5% increase), with
no decrease in the number of proposals funded or duration of funding.
Thus, the program budget has clearly increased, which is highly
commendable. Further increases will be needed to keep pace with the
increasing number and quality of proposals submitted.

Another consequence of the increased number of submissions is an
increase in workload for the sole Program Director in IAB. In spite of
this, the current Program Director managed to decrease the time-to-award
notification over that in the other years. This was accomplished partly
by the added efficiencies made possible by FastLane submissions, but
partly by the Program Director putting in an inordinate amount of time,
particularly in the weeks immediately after each panel meeting. In
addition, he was able to take less than half the allotted research time
due to the demands of the Program Director position. This is not a
sustainable situation for the long-term.

We recognize that the proportion of underrepresented minority scientists
submitting proposals in IAB is low (about 6% of total proposals), so
that it is difficult to have them represented in large numbers on review
panels. In the face of the small numbers of possible panelists, the IAB
Program Directors have done remarkably well in finding minority
panelists (15% in both 1999 and 2001). We encourage IAB to continue to
seek proposals from, and panel participation by, underrepresented
minorities, women and researchers from a wide array of institutions. We
commend their success in this area and encourage further efforts.

In the long term, however, the only way to increase the representation
is to increase the pool of available PIs from these groups. We thus
encourage IAB to continue to emphasize Criterion II because of the
possible positive impact such funding could have on the entry of members
of these groups into the field.

##### 

##### Appendix E: Ecological and Evolutionary Physiology

A subcommittee of three CoV members (Davis, Bradley & Dykens) pursued
several strategies while evaluating the EEP program. To gather an
overall impression of the breadth and depth of the successful proposals,
the topics of all successful proposals for the past 3 years was
reviewed, with an eye towards discerning trends. This analysis indicated
that the program has effectively supported a commendably diverse array
of research projects, at a commendable wide variety of levels. Research
ranging from the molecular through the organismal to the population
levels from an astonishing broad phylogenetic range has been supported.
This is perceived as a healthy trend, given the prevailing drives in
many fields towards molecular mechanistic reductionism. Simultaneously,
as some areas of research progress, the utility of molecular techniques
increases, and EEP has done a good job of discerning these trends, and
supporting molecular approaches where most appropriate. The CoV members
believe that the context of molecular experiments is as crucial as the
molecular data, and that the EEP program is to be commended for their
continued support for appropriate integrative and organismal approaches,
regardless of technique.

We noted that many of the areas of research funded depend entirely or in
large part on funding from NSF. In this context, EEP is doing an
excellent job of promoting groundbreaking, scientific research in
critical areas of Ecology, Evolution and Physiology.

To investigate the range of science and quality of proposals that
resulted in funding or failure, we reviewed the individual jackets of 27
proposals ranging in quality from outstanding that resulted in awards,
to those deemed only fair by the panel and subsequently denied funding.
This entailed reading the complete left page materials, especially the
ad hoc reviews, the panel summaries, and Form 7s. The consensus, reached
independently by all three COV members, was that the review panel and
Program Directors decisions accurately reflected the quality of the
proposals, and that subsequent decisions appear to be both in accord
with the reviewers' detailed comments and equitably distilled into the
panel comments. One CoV member who had not previously served on NSF
review panels commented on the seriousness of purpose, quality of the
reviews and obvious commitment to the scientific endeavor reflected by
the detailed and thoughtful nature of the reviews. Likewise, the sense
of fairness and attempts to maximize possibilities for success for all
submitted proposals was commendable.

Over the three years of our review, the EEP Program has carried the
largest burden of proposals within the cluster, no doubt a reflection of
the breadth and diversity of the fields falling under the auspices of
EPP. The doubling of Program Directors is a reasonable response. It was
our sense, in talking with the Program Directors, that these
administrators need help in handling the volume of proposals if they are
to give each the careful scrutiny they deserve. The quandary arises when
the exact nature of that assistance was discussed. Clerical help
diminishes one sort of burden, but does little to relieve the bottleneck
of selecting and pursuing relevant and qualified reviewers, decisions
that need to be informed by the professional expertise of the Program
Directors.

The consensus of the CoV members examining the EEP program was that
there was a healthy and insightful mix of high risk awards, RUI, cross
cutting efforts and other multidisciplinary awards within EEP. In a
program and agency fueled by individual investigator research
initiatives, it is expected that most awards would be for such research.
That concerted efforts to expand this format are being acknowledged by
funding bespeaks a flexibility and willingness to support good ideas and
science regardless of the logistics.