Response to Committe of Visitors Report for the Graduate Research
Fellowship Program

The Committee of Visitors (COV) for the Graduate Research Fellowship
Program (GRFP) met on June 17 and 18, 2003 and submitted its report on
July 17, 2003. The COV felt strongly that the stipend awarded to GRF
fellows increases the responsibilities of the applicants and the
subsequent fellowship holders, and that these additional
responsibilities should include mentoring potential applicants,
testimonial support, and encouraging and advising new applicants,
especially with the goal of increasing the number of applicants from
underrepresented groups for the GRF. This was a strong theme in the
report and the program office welcomes this guidance. This guidance has
been incorporated into our planning for the next GRFP competition.

The COV also made a set of specific recommendations. These
recommendations and program office responses are summarized as follows:

# Application Process

-   Reword items 15 and 16 on the application form to request evidence
    of the applicant\'s leadership potential and to provide reviewers
    with useful evidence of past experience in research.

> Response: This refers to two of the essays in our application. Since
> NSF has been stressing to reviewers that GRFP is looking for future
> leaders in STEM, it is important that the application makes this clear
> to applicants. The application form will be modified for the 2005
> competition to reflect this recommendation.

-   Provide a more structured format for the research proposal that
    would include the title, key words, hypothesis, research plan,
    literature citations, and, most importantly, a statement of
    integrity attesting to the originality of the applicant's proposal.

> Response: The program office will modify the instructions and format
> for the proposed plan of research essay for the 2005 competition in
> order to provide applicants a better idea of what we are looking for
> in this essay.

-   Require research advisors to provide a similar statement of the
    integrity attesting to the originality of the applicant's proposal.

> Response: The program office will consider modifications to the 2005
> application that would reflect this recommendation.

-   Provide a structured format for the reference letters, and eliminate
    the "check box" rating of candidates with regard to other students
    the referee has known. Referees should be asked to provide this
    information in narrative form by comparing the applicant with former
    successful applicants for the GRF fellowships and/or other
    outstanding students.

> Response: This recommendation is in line with discussions that the
> program office has been having to make the reference process simpler
> and more effective. Modifications will be made to the 2005 reference
> report to reflect this recommendation.

-   Provide explicit instructions to the referees about the importance
    that the letters play in the panel's ranking of the GRF candidates.

Response: This is an area that the program office has considered working
on, and we welcome the COV's guidance. References are most effective
when they are informed and the instructions to referees should be
improved. The program office will develop explicit instructions for
referees for the 2005 competition.

# Review Process

-   Above all, continue the strong emphasis on diversity of the panels.

Response: The program office is committed to continue to seek diversity
(gender, race/ethnicity, institution type, etc) in its panelists.

-   When possible, encourage more participation on the panels of faculty
    from schools who receive large numbers of the GRF awardees.

Response: The program office has initiated discussions with some of the
schools and programs that attract and produce many of our fellows to try
to encourage their faculty to participate in the review process. These
faculty are invited but tend to decline. We will work to improve the
acceptance rate for these top schools for the 2004 competition.

-   Develop well-grounded criteria for assessing leadership in an
    application.

Response: The program will develop instructions to reviewers based on
well-grounded criteria to reflect this recommendation. This development
will begin with the 2004 competition.

-   Expand panel preparation exercises by including advice on how to
    ascertain leadership potential from the application materials and
    insist on delaying the assignment of scores for applications until a
    panelist has read the first four or five.

Response: The COV has given good guidance in considering ways to improve
the GRFP review process and to better utilize the calibration exercise
at panels. This recommendation will be incorporated into the ongoing
planning for the 2004 competition review panels.

# Other Recommendations

-   Freeze the current stipend level at \$27,500 for the next three
    review periods.

Response: The NSF request for FY04 included a stipend increase to
\$30,000. However, stipend, cost-of-education, and general funding
issues will be addressed in a workshop planned by EHR/DGE for FY04. This
workshop will invite stakeholders from government and academia, and the
COV comments will be considered in the discussions.

-   Increase the number of awards by 100 per year for the next three
    review periods to reach a goal of 1200 awards per year.

Response: The FY04 budget request included an increase in the number of
graduate students supported by EHR/DGE programs (GRFP, IGERT, and
GK-12). Some of this increase will go to GRFP. The program office will
also consider approaches to vary the number of awards depending on
application pressure.

-   Initiate a new type of award for applicants ranked in the QG2 and
    Honorable Mention categories that will provide \$3-5,000 one time
    awards to be used for travel to professional meetings, membership in
    professional organizations, journal subscriptions, or for purchase
    of educational materials.

Response: Giving more meaning to Honorable Mention status may be a good
way to broaden the impact of the program; however, this would create a
significant increase in the GRFP budget. The program office has not
decided whether or not to implement this recommendation.

-   Increase access and awareness of the GRF program through contacts
    with various organizations that represent MSIs.

Response: The program office and its Application Processing Contractor
have increased program interactions of this sort, including
participation in meetings of NSBE, the Council of Historically Black
Graduate Schools, SACNAS, SHPE, and with visits to MSIs and to meetings
of NSF-supported alliances such as LSAMPs and AGEPs. We attribute some
of the increase in success of underrepresented minorities (URMs) to this
outreach, in which we do not just seek to increase the number of URM
applications but to provide level access and awareness so that the
success rates for all applicants is similar.

-   Recognize baccalaureate institutions with first-time successful
    applicants.

Response: The program office will discuss the practicality of this idea.
Recognition could be in the form of a mentorship-type grant to encourage
sustaining the school's production of Fellows, or as a posted or
otherwise highlighted recognition.

-   Improve the awareness of the GRF program in general and maintain
    access to the program from underrepresented groups by providing GRF
    fellow mentors to prospective applicants and grant-writing workshops
    when and wherever possible.

Response: The program office has considered how to encourage current and
past fellows to mentor potential applicants. We recently asked a current
fellow to give an outreach talk to students at his institution and have
fellows have also helped in talks and workshop given by NSF or our
contractor. We will expand the use of fellows -- current and past -- in
our outreach whenever possible.

-   Encourage more intra-directorate communication and activity,
    especially between the REU and GRF program directors and with other
    NSF programs that impact or overlap with the mission and goals of
    the GRF program.

Response: The program office has been working to strengthen these
connections and the Division of Graduate Education is in the process of
developing a more formal approach to doing so.

-   Provide incentives and support for the GRF fellows to participate in
    service and teaching opportunities, as well as encouraging their
    mentors to appreciate the value of having talented students who
    might choose to pursue more nontraditional, nonacademic, or
    professional career trajectories.

Response: The program office will work on modifications to the program
guidelines to encourage current and past fellows to participate in more
service and teaching activities than they currently do. We will also
work with institutions to encourage them to provide positive guidance in
the careers of their graduate students.

-   Identify persons at MSIs willing to serve as mentors for students
    seeking GRFs.

Response: The program office has initiated this on a small scale by
searching for former fellows on the faculty of MSIs. So far, a small
number have been invited as reviewers. We agree that these former
fellows teaching at MSIs as well as other interested mentors would be
powerful allies in developing strong applications from MSIs, and will
expand the use of such mentors.

-   Develop a communication scheme for organizing and informing mentors
    about cogent aspects of the GRF application wherein mentors can
    facilitate students in the process of writing quality applications.

Response: The program office is exploring effective ways to communicate
better with the community of potential mentors, and to improve the
materials available on-line that assist applicants.

-   Increase the interface with a diversity of organizations that can
    assist in informing students, especially minorities, about GRF as
    well as other appropriate NSF programs.

Response: As noted above, the program office and its Application
Processing Contractor have increased its interactions of this sort. We
will continue to try to improve our effectiveness in this area,
particularly in ways that level the awareness of the program and what we
are looking for in the applications.

-   Seek ways of encouraging inter-institutional collaboration between
    GRF recipient institutions and MSIs that train GRF awardees.

Response: The program office has looked into a variety of ways to do
this. The initial approach is to connect institutions that traditionally
host many GRFs with various existing capacity building programs
supported by NSF, such as LSAMPs. Initial discussions have begun, and
will continue. This may be an effective pathway for increasing the
access for underrepresented minorities s to graduate education at top
institutions, even beyond the effects on GRFP.