Professor C. Lineberger

Department of Chemistry and Biochemistry  

University of Colorado at Bolder 215  

Boulder, CO 80309-0215   USA

March 6, 2005

Dear Professor Lineberger:

On behalf of the 2005 Committee of Visitors (COV) for the Division of
Materials Research (DMR) of the National Science Foundation (NSF), I
submit on the attached templates the compiled findings of our review.
This note provides a summary of our most important observations.

[Overall Findings of the Committee of Visitors]{.underline}

The COV finds that the DMR has assembled a portfolio of world-class
materials-inspired research, which exemplifies scientific excellence and
breadth. DMR researchers are making scientific breakthroughs, which are
as fundamental as the discovery of new forms of matter, and others,
which have immediate applications as diverse as homeland security,
advanced electronics and information technology, and new medical
technologies. Central to these advances is the investment of the
Foundation and the Nation in developing and sustaining the scientific
workforce. Each year, DMR-funded researchers have trained more than 1500
undergraduate students, 2000 graduate students, and 500 postdoctoral
associates, and have communicated the centrality and importance of
materials in daily life through outreach to their communities. DMR
manages this complex, broad and successful scientific endeavor in an
admirably efficient and innovative manner. DMR-funded programs provide
the Nation an exceptionally high return on its investment.

[Findings as to People, Ideas and Tools:]{.underline} The Program
Directors of the DMR are doing an excellent job managing a very
important part of the Nation's funding for science, technology and
education. In spite of the exceptional stress in today's funding
environment, the Division is running an outstanding program, which is
consistent with the NSF's goals with regard to People, Ideas, Tools and
Management Excellence. We were very impressed with the thoroughness and
fairness of the reviewing process, the demonstrated technical expertise
of the program managers, the level and breadth of science, technology
and education supported by the program, and the high quality of the
research results achieved in the DMR program. The discretion of the
Program Directors to use their scientific knowledge, connection to the
scientific community and professional experience in making programmatic
and funding decisions is at the heart of this highly successful program.
Program Directors seldom deviated from review recommendations. When it
did happen, we found these decisions to be universally well reasoned and
to result in more effective use of program funds.

[DMR's response to COV 2002]{.underline}: The DMR management responded
in a reasonable manner to the requests from the previous COV. We commend
the Director for using his Reserve Funds to boost the average funding
level for grants. We also applaud his actions to move the average grant
duration towards four years. The concern of the 2002 COV regarding the
heavy workload of the Program Directors and the diversity of the group
has been addressed by hiring three new individuals and increasing the
fraction of woman and minorities in the Program Directors' ranks. Yet
the increase in staffing has not kept pace with the increase in proposal
submission rates, and the workload continues to be exceptionally high.
Given the critical importance of the Program Directors in the reviewing
and funding process, we urge the Division to maintain an acceptable
workload. The fraction of industrial reviewers has been increased in
response to the 2002 COV concern. Some of the COI limitations on the COV
process were addressed by DMR, but other COI constraints seem to be
beyond its immediate control. We encourage the Division to continue
working on these issues.

[DMR within MPS and NSF]{.underline}: The COV is struck by the fact that
the DMR's budget expansion has not kept pace with those of MPS and NSF
as a whole. While NSF and MPS saw their total budgets over the past
seven years increase by \~68% and \~53%, respectively, the DMR budget
increased by only \~43% over the same period. We do not understand the
rationale for this trend and the differentially poor participation of
DMR in the past budgetary expansion of NSF and MPS. As noted above we
judge the DMR to be a highly efficient, well managed, innovative
organization supporting high-quality research with a broad impact --
from fundamental science to national security applications. Research
funded through the DMR impacts \~50% of all of the physical sciences,
and makes contact with an array of other disciplines. Its research
covers very fundamental aspects of matter and creates the basis for
future technologies. It is ill-advised to permit the relative funding
level for these vital programs to erode.

[Imminent DMR Budget cuts]{.underline}: In the view of the COV the
biggest challenge facing the DMR is the imminent cut of its budget by 4%
for FY 2005. This will further restrict the Division's ability to
support new grants. Continued budget erosion will have a major negative
long term impact on national productivity in science, technology and
education.

To the degree to which program cuts must be performed, we support the
intent of the Director to maintain funding for each grant at a viable
level. The COV feels strongly that budget reductions should not
disproportionately erode the fraction of individual investigator grants
in DMR's funding spectrum. This sector of the DMR research portfolio
contains some of the most creative and high-risk projects but did not
proportionately participate in the past funding growth.

[Balance between Individual Investigator Grants, Group Grants, Centers
and Facilities:]{.underline} The previous COV was unable to investigate
the appropriateness of the relative funding level between individual
investigator grants, group grants, centers and facilities and
recommended it to be performed in 2005. Due to the limited time of our
visit, the present COV also did not perform an in-depth evaluation of
this balance; however, based on the information examined by the COV, the
group as a whole views the distribution of funds between these different
sectors as roughly appropriate. The COV realizes the benefits of
facilities to many individual investigators. It supports the funding of
groups to foster collaborative research and of centers to have a focused
impact on a field. It also recognizes the highly creative contribution
to science from individual investigators. The view among the COV members
is that it is important to stop the continuing relative erosion of
support for individual investigators and at least maintain the present
percentage level of funding for this sector.

[Balance between Programs within DMR:]{.underline} An often heard
concern in the DMR science community relates to the relative funding
level of the programs within DMR. We solicited information from the
Director, viewed available data over the past seven years and had in
depth discussion with several Program Directors. Based on the data,
there have been no major disproportionate shifts between programs during
this time period. We also investigated the relative success rates of
proposals submitted to the various programs within DMR over the past
seven years. We have learned that these data are not easily interpreted
due the accounting procedure used for acceptances and declinations from
panel reviews. We urge the DMR to generate a process that provides
easily interpretable data for success rates in core programs. Given the
present wide fluctuation in year-to-year success rates within each
program, it is difficult to extract trends, but it is obvious that
average acceptance rates have drastically decreased during the past two
years.

In conclusion, the DMR will face severe challenges in the coming years
in maintaining its excellent research portfolio in the face of a
difficult budget environment. We have confidence that the DMR staff has
the necessary expertise and perspective to make difficult decisions and
respond to new opportunities using the effective and transparent
processes that they have in place. The COV urges NSF to do everything
possible to maintain the support of the excellent programs managed
through DMR and to provide the resources to take advantage of the
opportunities emerging across the broader materials research community.

With best wishes

Horst Stormer

##### CORE QUESTIONS and REPORT TEMPLATE

#####  for 

##### FY 2005 NSF COMMITTEE OF VISITOR (COV) REVIEWS

**Guidance to NSF Staff:** This document includes the FY 2005 set of
Core Questions and the COV Report Template for use by NSF staff when
preparing and conducting COVs during FY 2005. Specific guidance for NSF
staff describing the COV review process is described in Subchapter
300-Committee of Visitors Reviews (NSF Manual 1, Section VIII) that can
be obtained at
[http://www.inside.nsf.gov/od/gpra/.](http://www.inside.nsf.gov/od/gpra/. )

NSF relies on the judgment of external experts to maintain high
standards of program management, to provide advice for continuous
improvement of NSF performance, and to ensure openness to the research
and education community served by the Foundation. Committee of Visitor
(COV) reviews provide NSF with external expert judgments in two areas:
(1) assessments of the quality and integrity of program operations and
program-level technical and managerial matters pertaining to proposal
decisions; and (2) comments on how the outputs and outcomes generated by
awardees have contributed to the attainment of NSF's mission and
strategic outcome goals.

Many of the Core Questions are derived from NSF performance goals and
apply to the portfolio of activities represented in the program(s) under
review. The program(s) under review may include several subactivities as
well as NSF-wide activities. The directorate or division may instruct
the COV to provide answers addressing a cluster or group of programs --
a portfolio of activities integrated as a whole -- or to provide answers
specific to the subactivities of the program, with the latter requiring
more time but providing more detailed information.

The Division or Directorate may choose to add questions relevant to the
activities under review. NSF staff should work with the COV members in
advance of the meeting to provide them with the report template,
organized background materials, and to identify questions/goals that
apply to the program(s) under review.

**Guidance to the COV:** The COV report should provide a balanced
assessment of NSF's performance in two primary areas: (A) the integrity
and efficiency of the ***processes*** related to proposal review; and
(B) the quality of the ***results*** of NSF's investments in the form of
outputs and outcomes that appear over time. The COV also explores the
relationships between award decisions and program/NSF-wide goals in
order to determine the likelihood that the portfolio will lead to the
desired results in the future. Discussions leading to answers for Part A
of the Core Questions will require study of confidential material such
as declined proposals and reviewer comments. *COV reports should not
contain confidential material or specific information about declined
proposals**.*** Discussions leading to answers for Part B of the Core
Questions will involve study of non-confidential material such as
results of NSF-funded projects. It is important to recognize that the
reports generated by COVs are used in assessing agency progress in order
to meet government-wide performance reporting requirements, and are made
available to the public. Since material from COV reports is used in NSF
performance reports, the COV report may be subject to an audit.

*We encourage COV members to provide comments to NSF on how to improve
in all areas, as well as suggestions for the COV process, format, and
questions.*

##### FY 2005 REPORT TEMPLATE FOR

#####  NSF COMMITTEES OF VISITORS (COVs)

+-----------------------------------------------------------------------+
| ##### Date of COV: February 16-18, 2005                               |
+-----------------------------------------------------------------------+
| **Program/Cluster: Metals, Ceramics and Electronic Materials**        |
+-----------------------------------------------------------------------+
| **Division: Division of Materials Research**                          |
+-----------------------------------------------------------------------+
| **Directorate: Mathematical and Physical Sciences**                   |
+-----------------------------------------------------------------------+
| **Number of actions reviewed by COV**[^1]**: 70 Awards: 42            |
| Declinations: 28 Other: 0**                                           |
+-----------------------------------------------------------------------+
| **Total number of actions within Program/Cluster/Division during      |
| period being reviewed by COV**[^2]**: 2151 Awards: 331 Declinations:  |
| 1134 Other: 686**                                                     |
+-----------------------------------------------------------------------+
| **Manner in which reviewed actions were selected: 90 jackets were     |
| provided. Six Committee members focused on specific technical areas   |
| (two each to three areas) and the other three members reviewed        |
| jackets from all three technical areas.**                             |
+-----------------------------------------------------------------------+

**PART A. INTEGRITY AND EFFICIENCY OF THE PROGRAM'S PROCESSES AND
MANAGEMENT**

Briefly discuss and provide comments for *each* relevant aspect of the
program\'s review process and management. Comments should be based on a
review of proposal actions (awards, declinations, and withdrawals) that
were *completed within the past three fiscal years*. Provide comments
for *each* program being reviewed and for those questions that are
relevant to the program under review. Quantitative information may be
required for some questions. Constructive comments noting areas in need
of improvement are encouraged.

A.1 Questions about the quality and effectiveness of the program's use
of merit review procedures. Provide comments in the space below the
question. Discuss areas of concern in the space provided.

+-----------------------------------------------------------+----------+
| **QUALITY AND EFFECTIVENESS OF MERIT REVIEW PROCEDURES**  | **YES,   |
|                                                           | NO,**    |
|                                                           |          |
|                                                           | **DATA   |
|                                                           | NOT      |
|                                                           | AV       |
|                                                           | AILABLE, |
|                                                           | or NOT   |
|                                                           | APPLICAB |
|                                                           | LE**[^3] |
+-----------------------------------------------------------+----------+
| 1\. Is the review mechanism appropriate? (panels, ad hoc  | Yes      |
| reviews, site visits)                                     |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The NSF has a highly regarded, fair review process that   |          |
| plays a key role in the decision process. Panels work     |          |
| best for topically homogenous proposals which need to be  |          |
| compared. Mail reviews by individual reviewers give the   |          |
| most rigorous reviews.                                    |          |
+-----------------------------------------------------------+----------+
| 2\. Is the review process efficient and effective?        | Yes      |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The COV thinks that the review process is very effective. |          |
| A broader reviewer base is obtained with individual       |          |
| reviewers versus panel reviews, which are more expensive. |          |
| Proposals that are funded by multiple divisions require   |          |
| more reviewers than those that are funded by a single     |          |
| division. However, the Program Directors notify reviewers |          |
| that have not completed their reviews when the Division   |          |
| has received enough reviews to make a decision, which the |          |
| COV feels is a very good practice.                        |          |
+-----------------------------------------------------------+----------+
| 3\. Are reviews consistent with priorities and criteria   | Yes      |
| stated in the program's solicitations, announcements, and |          |
| guidelines?                                               |          |
|                                                           |          |
| Comments:                                                 |          |
+-----------------------------------------------------------+----------+
| 4\. Do the individual reviews (either mail or panel)      | Yes      |
| provide sufficient information for the principal          |          |
| investigator(s) to understand the basis for the           |          |
| reviewer's recommendation?                                |          |
|                                                           |          |
| Comments:                                                 |          |
+-----------------------------------------------------------+----------+
| 5\. Do the panel summaries provide sufficient information | Yes      |
| for the principal investigator(s) to understand the basis |          |
| for the panel recommendation?                             |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| Sometimes panels conclude that some proposals should be   |          |
| funded, but they were not because of inadequate           |          |
| resources.                                                |          |
+-----------------------------------------------------------+----------+
| 6\. Is the documentation for recommendations complete,    | Yes      |
| and does the program officer provide sufficient           |          |
| information and justification for her/his recommendation? |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The documentation records are very complete. The COV      |          |
| commends the staff for the clarity and organization of    |          |
| the administrative case files.                            |          |
+-----------------------------------------------------------+----------+
| 7\. Is the time to decision appropriate?                  | Yes      |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The COV was favorably impressed with the rapid response   |          |
| for the programs reviewed in the decision process.        |          |
+-----------------------------------------------------------+----------+
| 8\. Discuss any issues identified by the COV concerning   |          |
| the quality and effectiveness of the program's use of     |          |
| merit review procedures:                                  |          |
|                                                           |          |
| Many proposals which have very positive reviews were not  |          |
| funded, but the COV felt that the decisions were          |          |
| reasonable, given the funds available.                    |          |
+-----------------------------------------------------------+----------+

A.2 Questions concerning the implementation of the NSF Merit Review
Criteria ([intellectual merit]{.underline} and [broader
impacts]{.underline}) by reviewers and program officers. Provide
comments in the space below the question. Discuss issues or concerns in
the space provided.

+----------------------------------------------------------+-----------+
| **IMPLEMENTATION OF NSF MERIT REVIEW CRITERIA**          | **YES,    |
|                                                          | NO,**     |
|                                                          |           |
|                                                          | **DATA    |
|                                                          | NOT       |
|                                                          | A         |
|                                                          | VAILABLE, |
|                                                          | or NOT    |
|                                                          | APPLICA   |
|                                                          | BLE**[^4] |
+----------------------------------------------------------+-----------+
| 1\. Have the individual reviews (either mail or panel)   | Yes       |
| addressed both merit review criteria?                    |           |
|                                                          |           |
| Comments: NSF stresses that intellectual merit is        |           |
| necessary, but not sufficient; however, broader impact   |           |
| can take multiple venues, such as outreach, technical    |           |
| relevance, scientific foundations, etc. Reviewers should |           |
| be encouraged to consistently evaluate the broader       |           |
| impact of the proposal.                                  |           |
+----------------------------------------------------------+-----------+
| 2\. Have the panel summaries addressed both merit review | Yes       |
| criteria?                                                |           |
|                                                          |           |
| Comments: Panels frequently do a more consistent job of  |           |
| addressing both merit review criteria than individual    |           |
| reviewers.                                               |           |
+----------------------------------------------------------+-----------+
| 3\. Have the *review analyses* (Form 7s) addressed both  | Yes       |
| merit review criteria?                                   |           |
|                                                          |           |
| Comments: Definitely.                                    |           |
+----------------------------------------------------------+-----------+
| 4\. Discuss any issues the COV has identified with       |           |
| respect to implementation of NSF's merit review          |           |
| criteria.                                                |           |
|                                                          |           |
| See A. 2-1 and A. 2-2 above.                             |           |
+----------------------------------------------------------+-----------+

A.3 Questions concerning the selection of reviewers. Provide comments in
the space below the question. Discuss areas of concern in the space
provided.

+---------------------------------------------------------+------------+
| Selection of Reviewers                                  | YES , NO,  |
|                                                         |            |
|                                                         | DATA NOT   |
|                                                         | AVAILABLE, |
|                                                         |            |
|                                                         | or NOT     |
|                                                         | APPL       |
|                                                         | ICABLE[^5] |
+---------------------------------------------------------+------------+
| 1\. Did the program make use of an adequate number of   | Yes        |
| reviewers?                                              |            |
|                                                         |            |
| Comments: All cases examined by the COV had been        |            |
| decided based on an appropriate number of reviews.      |            |
+---------------------------------------------------------+------------+
| 2\. Did the program make use of reviewers having        | Yes        |
| appropriate expertise and/or qualifications?            |            |
|                                                         |            |
| Comments: The COV thought that the expertise and        |            |
| qualifications of the reviewers were very high.         |            |
+---------------------------------------------------------+------------+
| 3\. Did the program make appropriate use of reviewers   | Yes        |
| to reflect balance among characteristics such as        |            |
| geography, type of institution, and under-represented   |            |
| groups?                                                 |            |
|                                                         |            |
| Comments: The COV found that the geographical balance   |            |
| was appropriate, but that more reviewers from industry  |            |
| would be beneficial. It is clear that this program      |            |
| group has taken steps to increase the number of women   |            |
| and under-represented minorities that review proposals. |            |
+---------------------------------------------------------+------------+
| 4\. Did the program recognize and resolve conflicts of  | Yes        |
| interest when appropriate?                              |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| Several examples of COI resolution were noted in the    |            |
| files. The NSF guidelines on COI were appropriately     |            |
| communicated to the pool of reviewers.                  |            |
+---------------------------------------------------------+------------+
| 5\. Discuss any issues the COV has identified relevant  |            |
| to selection of reviewers.                              |            |
|                                                         |            |
| The diversity of the reviewer pool has been improved by |            |
| use of faculty at HBCUs and minority-serving            |            |
| universities. Efforts to increase utilization of        |            |
| faculty from these institutions should continue.        |            |
| Although the industrial representation in the reviewer  |            |
| pool has increased compared to 2002, it would be        |            |
| desirable to increase the fraction of the reviewer pool |            |
| from industry.                                          |            |
+---------------------------------------------------------+------------+

A.4 Questions concerning the resulting portfolio of awards under review.
Provide comments in the space below the question. Discuss areas of
concern in the space provided.

+------------------------------------------------------+---------------+
| **RESULTING PORTFOLIO OF AWARDS**                    | **A           |
|                                                      | PPROPRIATE,** |
|                                                      |               |
|                                                      | **NOT         |
|                                                      | APPROPRIA     |
|                                                      | TE**[^6]**,** |
|                                                      |               |
|                                                      | **OR DATA NOT |
|                                                      | AVAILABLE**   |
+------------------------------------------------------+---------------+
| 1\. Overall quality of the research and/or education | Appropriate   |
| projects supported by the program.                   |               |
|                                                      |               |
| Comments: Research and educational programs          |               |
| supported by this program group are excellent and    |               |
| frequently determine the leading edge.               |               |
+------------------------------------------------------+---------------+
| 2\. Are awards appropriate in size and duration for  | Appropriate   |
| the scope of the projects?                           |               |
|                                                      |               |
| Comments: With the high quality of the proposals and |               |
| the inability of the DMR to fund all of the good     |               |
| proposals due to limited funding, the size and       |               |
| duration are appropriate. DMR has been moving in the |               |
| proper direction by increasing the average grant     |               |
| size and duration.                                   |               |
+------------------------------------------------------+---------------+
| 3\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   High risk projects?                              |               |
|                                                      |               |
| Comments: The number of projects in this class seems |               |
| appropriate but remains at a level where further     |               |
| expansion could be supported. The COV commends the   |               |
| Program Directors in the use of SGER grants in this  |               |
| area. The use of the division reserve to promote the |               |
| funding of programs in this class is also            |               |
| appropriate and supported by the COV. In responding  |               |
| to this question, the COV is referring to proposals  |               |
| for high payoff, high risk research for which both   |               |
| the proposal and the research team are of excellent  |               |
| quality.                                             |               |
+------------------------------------------------------+---------------+
| 4\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Multidisciplinary projects?                      |               |
|                                                      |               |
| Comments: The multidisciplinary aspects of the       |               |
| portfolio appear to be increasing in multiple areas  |               |
| and this trend is encouraged. This has been an area  |               |
| of growth in the program portfolio and has the       |               |
| potential for increasing near-term scientific and    |               |
| technological impact.                                |               |
+------------------------------------------------------+---------------+
| 5\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Innovative projects?                             |               |
|                                                      |               |
| Comments: The proposal review process ensures        |               |
| innovative proposals and research results.           |               |
+------------------------------------------------------+---------------+
| 6\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Funding for centers, groups and awards to        |               |
|     individuals?                                     |               |
|                                                      |               |
| Comments: The appropriateness of the balance is      |               |
| difficult to evaluate and probably should be done in |               |
| a larger context than at the Division level;         |               |
| however, if the DMR experiences continuing budget    |               |
| reductions, the COV feels that the individual        |               |
| investigator and small group proposal areas, which   |               |
| have declined slightly as a fraction of the program  |               |
| portfolio, should not be reduced at a higher rate    |               |
| than other areas of the portfolio.                   |               |
+------------------------------------------------------+---------------+
| 7\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Awards to new investigators?                     |               |
|                                                      |               |
| Comments: The program portfolio seems to have a      |               |
| reasonable fraction of awards to new investigators.  |               |
| The goal that 30% of the awards be to new            |               |
| investigators is unrealistically high and is not     |               |
| demographically sustainable.                         |               |
+------------------------------------------------------+---------------+
| 8\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Geographical distribution of Principal           |               |
|     Investigators?                                   |               |
|                                                      |               |
| Comments: The geographical distribution looks very   |               |
| reasonable; it is clear that the Program Directors   |               |
| have developed processes to insure appropriate       |               |
| geographical distribution of Principal Investigators |               |
| and follow these processes.                          |               |
+------------------------------------------------------+---------------+
| 9\. Does the program portfolio have an appropriate   | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Institutional types?                             |               |
|                                                      |               |
| Comments: In addition to funding for programs at     |               |
| major research universities, the COV was pleased to  |               |
| see significant funding for undergraduate-only       |               |
| institutions.                                        |               |
+------------------------------------------------------+---------------+
| 10\. Does the program portfolio have an appropriate  | Appropriate   |
| balance of:                                          |               |
|                                                      |               |
| -   Projects that integrate research and education?  |               |
|                                                      |               |
| Comments: Integrated research and education is part  |               |
| of the essential fabric on the program and is        |               |
| reflected in essentially every proposal funded.      |               |
+------------------------------------------------------+---------------+
| 11\. Does the program portfolio have an appropriate  | Appropriate   |
| balance:                                             |               |
|                                                      |               |
| -   Across disciplines and subdisciplines of the     |               |
|     activity and of emerging opportunities?          |               |
|                                                      |               |
| Comments: The COV saw a healthy increase in the      |               |
| number of interdisciplinary programs funded. The     |               |
| Division has been particularly effective in          |               |
| developing joint programs with other Divisions,      |               |
| including Engineering, and the COV commends the      |               |
| Program Directors for their effort in this area.     |               |
+------------------------------------------------------+---------------+
| 12\. Does the program portfolio have appropriate     | Appropriate   |
| participation of underrepresented groups?            |               |
|                                                      |               |
| Comments: The program portfolio and nuggets          |               |
| demonstrate that the DMR has a high level of         |               |
| appreciation of the importance of this area, and the |               |
| COV feels that this emphasis should be continued.    |               |
+------------------------------------------------------+---------------+
| 13\. Is the program relevant to national priorities, | Appropriate   |
| agency mission, relevant fields and other customer   |               |
| needs? Include citations of relevant external        |               |
| reports.                                             |               |
|                                                      |               |
| Comments: This DMR program group is well-aligned     |               |
| with priorities for materials and nanoscale research |               |
| documented in reports of the National Science Board, |               |
| President\'s Committee of Advisors in Science and    |               |
| Technology, and the Interagency Working Group on     |               |
| Nanotechnology.                                      |               |
+------------------------------------------------------+---------------+
| 14\. Discuss any concerns relevant to the quality of |               |
| the projects or the balance of the portfolio.        |               |
|                                                      |               |
| The Program Directors are handling the growth in new |               |
| multidisciplinary program areas very competently.    |               |
| The COV strongly praises the depth and breadth of    |               |
| the programs supported by the Division, including    |               |
| the high level of innovative projects in the program |               |
| portfolio.                                           |               |
|                                                      |               |
| The COV thinks that the disparity of the proposal    |               |
| success rate within the DMR (Ceramics \~13% compared |               |
| to the DMR average of \~23%) should be examined to   |               |
| understand these differences and determine if any    |               |
| action should be taken.                              |               |
+------------------------------------------------------+---------------+

A.5 Management of the program under review. Please comment on:

+-----------------------------------------------------------------------+
| 1\. Management of the program.                                        |
|                                                                       |
| Comments: The COV assessed the management of these programs as        |
| excellent.                                                            |
+-----------------------------------------------------------------------+
| 2\. Responsiveness of the program to emerging research and education  |
| opportunities.                                                        |
|                                                                       |
| Comments: Excellent; The Program Directors have been very proactive   |
| in fostering new research areas and advancing the coordination of     |
| research and education.                                               |
+-----------------------------------------------------------------------+
| 3\. Program planning and prioritization process (internal and         |
| external) that guided the development of the portfolio.               |
|                                                                       |
| Comments: Excellent. The Program Directors play a key role in this    |
| area.                                                                 |
+-----------------------------------------------------------------------+
| 4\. Additional concerns relevant to the management of the program.    |
|                                                                       |
| The DMR responded to the recommendation of the 2002 COV visit by      |
| adding two Program Directors and one technical staff. The 2005 COV    |
| feels that, even with this added staff, the DMR needs additional      |
| Program Directors because of the significant increase in the number   |
| proposals.                                                            |
+-----------------------------------------------------------------------+
|                                                                       |
+-----------------------------------------------------------------------+

##### PART B. RESULTS : OUTPUTS AND OUTCOMES OF NSF INVESTMENTS

NSF investments produce results that appear over time. The answers to
the first three (People, Ideas and Tools) questions in this section are
to be based on the COV's study of award results, which are direct and
indirect accomplishments of projects supported by the program. These
projects may be currently active or closed out during the previous three
fiscal years. The COV review may also include consideration of
significant impacts and advances that have developed since the previous
COV review and are demonstrably linked to NSF investments, regardless of
when the investments were made. Incremental progress made on results
reported in prior fiscal years may also be considered.

The following questions are developed using the NSF outcome goals in the
NSF Strategic Plan. The COV should look carefully at and comment on (1)
noteworthy achievements of the year based on NSF awards; (2) the ways in
which funded projects have collectively affected progress toward NSF's
mission and strategic outcomes; and (3) expectations for future
performance based on the current set of awards. NSF asks the COV to
provide comments on the degree to which past investments in research and
education have contributed to NSF's progress towards its annual
strategic outcome goals and to its mission:

-   To promote the progress of science.

-   To advance national health, prosperity, and welfare.

-   To secure the national defense.

-   And for other purposes.

Excellence in managing NSF underpins all of the agency's activities. For
the response to the Outcome Goal for Organizational Excellence, the COV
should comment, where appropriate, on NSF providing an agile, innovative
organization. Critical indicators in this area include (1) operation of
a credible, efficient merit review system; (2) utilizing and sustaining
broad access to new and emerging technologies for business application;
(3) developing a diverse, capable, motivated staff that operates with
efficiency and integrity; and (4) developing and using performance
assessment tools and measures to provide an environment of continuous
improvement in NSF's intellectual investments as well as its management
effectiveness.

**B.** **Please provide comments on the activity as it relates to NSF's
Strategic Outcome Goals. Provide examples of outcomes (nuggets) as
appropriate. Examples should reference the** **NSF award number, the
Principal Investigator(s) names, and their institutions.**

+-----------------------------------------------------------------------+
| **B.1** **[OUTCOME GOAL for PEOPLE]{.underline}:** **Developing "a    |
| diverse, competitive and globally engaged workforce of scientists,    |
| engineers, technologists and well-prepared citizens."**               |
|                                                                       |
| Comments: All of the programs reviewed have a strong component of     |
| education and career development of undergraduate and graduate        |
| students and post doctoral researchers. The important role of the NSF |
| in educating graduate students is well documented. This section calls |
| attention to other areas of educational impact by the DMR programs.   |
|                                                                       |
| A good example of outreach to high school students is the extensive   |
| workshop on organic light emitting devices for local high schools     |
| organized by Penn State (DMR-0103068; Joan Redwing).                  |
|                                                                       |
| The program run at Coe College (an undergraduate-only institution) by |
| PI's Steve Feller and Mario Affitagato (DMR-0211718) is a fine        |
| example of leading-edge scientific research being done by             |
| undergraduates. Over the last 3 years alone, more than 20 students    |
| have participated. A high fraction have been launched on careers in   |
| science, having gone off to PhD programs at major universities.       |
|                                                                       |
| DMR 0403004 sponsored a successful workshop, led by PI's Bill         |
| Fahrenholtz and Greg Hilmas, on ultrahigh temperature materials that  |
| are critical for hypersonic flight and rocket propulsion. The         |
| workshop brought together people from academia, industry, and         |
| government labs from the U.S. and abroad to exchange ideas and plan   |
| future basic research.                                                |
|                                                                       |
| DMR 0224642 supported innovative research by a young female professor |
| at SUNY Stony Brook, Perena Gouma, on electronic ceramics doped with  |
| biological molecules, which have potential for microsensors for       |
| medical diagnostics and environmental monitoring.                     |
|                                                                       |
| DMR 0139081 Georgia Tech REU/RET Nano Site engaged 97 REU students    |
| from 41 institutions (including 16 minority institutions) and 29      |
| teachers from Atlanta schools (predominantly minority) in a program   |
| on structure-property correlations across micro to nano length        |
| scales.                                                               |
|                                                                       |
| Impressive results have come from programs where the PI(s) are        |
| intimately involved with the mechanisms that they have proposed to    |
| actively influence the broader impact of their work.                  |
|                                                                       |
| DMR-0207643, Alison Baski, Virginia Commonwealth University: The      |
| outreach program has delivered physical science lessons to 200        |
| disadvantaged students in the National Youth Sports Program each      |
| summer, as well as sponsored visits to nearby Richmond City           |
| Elementary Schools. These venues serve a broad range of young people  |
| and potential scholars, including a significant population of         |
| underrepresented minority students. In turn, young scholars at the    |
| undergraduate level participate in scientific activities through      |
| which they not only receive encouragement but are exposed to the      |
| competitive aspects of research that will be a fact of their lives as |
| professional scientists/engineers in the global environment;          |
| DMR-0210704, Rachel Goldman, Univ. of Michigan- had two students      |
| selected as Finalists in the Intel Research Contest for Undergraduate |
| Students (2003-04).                                                   |
+-----------------------------------------------------------------------+
| **B.2** **[OUTCOME GOAL for IDEAS]{.underline}:** **Enabling          |
| "discovery across the frontier of science and engineering, connected  |
| to learning, innovation, and service to society."**                   |
|                                                                       |
| Comments: A good example of an innovative program which has potential |
| to generate significant change in more than one technology area is    |
| DMR-0210321: Caroline Ross, Hank Smith, Carl Thompson, and Frances    |
| Ross of MIT and IBM entitled \"Nanostructured Surfaces with Long      |
| Range Order for Controlled Self Assembly\". The project developed     |
| methods and processes to control the position and geometry of arrays  |
| of nanostructures over large areas with precise long range order.     |
| Such structures can provide a matrix for growth of a wide array of    |
| devices, including SiGe quantum dots and optical or magnetic devices. |
|                                                                       |
| Jay Switzer (U. Missouri-Rolla, DMR 007135) has accomplished a        |
| remarkable example of organic/inorganic \"nanoengineering\". In this  |
| project, copper oxide films with a clear crystallographic             |
| \"handedness\" were deposited on an ordinary gold substrate, using an |
| organic molecular template. The result is a new material that can     |
| help catalyze specific \"chiral\" organic synthesis and sensors.      |
|                                                                       |
| Peter Davies (U. Penn, DMR 0213489) has made important advances in    |
| development of electronically active ceramics that are critical to    |
| wireless communication devices. In carefully doped barium zinc        |
| niobates, this group has discovered ways to improve microstructure    |
| and lower fabrication temperatures, both of which should lead to      |
| improved device performance at lower cost.                            |
|                                                                       |
| DMR-01002755, Paul Thibado, Univ. of Arkansas: Used tensile-strained  |
| ScN to emit light in the visible spectrum with electro-mechanical     |
| response greater than perovskites (the basis for the piezoelectric    |
| industry); this work is moving toward the realm of "smart materials". |
| This service-to-society research could be the basis for a new class   |
| of materials functionality.                                           |
|                                                                       |
| DMR-0237400: The ability to directly correlate macroscopic mechanical |
| response to microscopic and atomic processes has been demonstrated in |
| nano-grained fcc metals. These studies support the possible           |
| expectation of nano-grained structural systems in functional MEMS     |
| devices.                                                              |
+-----------------------------------------------------------------------+
| **B.3** **[OUTCOME GOAL for TOOLS:]{.underline}** **Providing         |
| "broadly accessible, state-of-the-art S&E facilities, tools and other |
| infrastructure that enable discovery, learning and innovation."**     |
|                                                                       |
| Comments: Robert Marzke, Arizona State University (DMR-0116361), in   |
| work jointly funded by AFOSR, developed difficult and innovative      |
| in-situ, high temperature spectroscopic methods that are being        |
| applied to study the structure and dynamics of molten oxides at       |
| temperature as high as 2400 deg. C.                                   |
|                                                                       |
| DMR-0430190, John Abelson, Univ. of Illinois at Urbana-Champaign is   |
| using fluctuation electron microscopy to study medium range order in  |
| group IV and amorphous materials; this new tool has particular        |
| applications to nanoscale media.                                      |
|                                                                       |
| DMR-0207295, Michael Downer, Univ. of Texas at Austin used recently   |
| developed/refined techniques for surface spectroscopy using nonlinear |
| second order harmonic and sum frequency gain (SHG/SSFG) as well as    |
| linear reflectance-difference spectroscopy (RDS); obtained            |
| convergence of theory (ab initio pseudopotential and full potential   |
| all-electron approaches) with aforementioned for the purpose of       |
| developing these experimental techniques for wide utility to the      |
| surface science community.                                            |
|                                                                       |
| DMR-0134706 (Mark Hersam, Northwestern University,) a CAREER award    |
| which developed a new technique for characterizing the electronic and |
| optical performance of organic light emitting diodes resulting in a   |
| patent filing and Dow / Kodak interest. Seven undergraduate students, |
| including three women and two underrepresented minorities,            |
| contributed to this project while participating in the REU program    |
+-----------------------------------------------------------------------+
| **B.4** **[OUTCOME GOAL for ORGANIZATIONAL EXCELLENCE]{.underline}:** |
| **Providing "an agile, innovative organization that fulfills its      |
| mission through leadership in state-of-the-art business practices."** |
|                                                                       |
| Comments: The management philosophy of using the Director\'s reserve  |
| to stimulate research initiatives in emerging and innovative areas,   |
| as well as to increase involvement with under-represented groups and  |
| industry, is excellent.                                               |
|                                                                       |
| The programs reviewed by the COV demonstrated results indicative of   |
| an agile, innovative organization as the Program Directors moved the  |
| program into new technology and interdisciplinary areas and balanced  |
| the program portfolio in the process.                                 |
|                                                                       |
| Another example of organizational agility and excellence is the very  |
| high percentage of proposals which are acted upon within the          |
| six-month goal of the NSF.                                            |
+-----------------------------------------------------------------------+

##### PART C. OTHER TOPICS

C.1 Please comment on any program areas in need of improvement or gaps
(if any) within program areas.

The DMR needs additional budget for its programs; the data show that the
fraction of high quality proposals that can be funded is decreasing
significantly. As noted earlier, only \~13% of the proposals in ceramics
were funded, and the fraction of proposals that could be funded in both
electronic materials and metals decreased significantly between 2003 and
2004. In addition, it is increasingly important to understand and design
materials at the atomic and molecular level and fabricate them with
nanoscale control to produce materials with new properties for specific
applications.

Supporting undergraduate research is an important mission for the NSF.
The introduction of undergraduate students to research and the impact
that NSF is having through its programs in this area are not
sufficiently publicized. Approaches should be explored to encourage more
complete reporting from PIs to capture the level of participation and
outcomes/impacts of these programs.

C.2 Please provide comments as appropriate on the program's performance
in meeting program-specific goals and objectives that are not covered by
the above questions.

The COV feels that the performance of the programs has been very
effective in meeting their program-specific goals and objectives.

**C.3 Please** **identify agency-wide issues that should be addressed by
NSF to help improve the program\'s performance.**

Capability for mid-career changes in research direction or area are
important and should be continued (ADVANCE, Planning Grants, etc.).
Divisional flexibility by retaining these funds at the Divisional level
would be helpful in addressing this need.

C.4 Please provide comments on any other issues the COV feels are
relevant.

The COV feels that NSF needs to respond aggressively to the changing
needs of the U.S. workforce in today\'s increasingly competitive, high
tech global economy, in which a larger fraction of the workforce needs
advanced degrees in science, engineering, and technology areas. Many
good outreach and undergraduate research programs are being developed,
and better coordination among these programs has the potential for
increasing the number of U.S. students who receive bachelor\'s degrees
and advanced degrees in math, science and engineering.

Acceleration of technology transfer from NSF-funded scientific and
engineering advances to industrial uses is increasingly important for
the national economy. Approaches to stimulate this acceleration of tech
transfer need to be explored, perhaps in conjunction with other
government agencies.

**C.5 NSF would appreciate your comments on how to improve the COV
review process, format and report template.**

> The COV feels that limited executive sessions (without NSF staff)
> would be beneficial for the efficacy review process. (The COV
> recognizes that current federal law currently prohibits this practice,
> but feels it shouldn\'t.)
>
> Adding a brief (\~30 minute) summary of the programs to be reviewed to
> begin the small group sessions could potentially be beneficial to the
> review process.

**SIGNATURE BLOCK:**

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

For the COV - Division of Materials Research

Horst Stormer

Chair

##### CORE QUESTIONS and REPORT TEMPLATE

#####  for 

##### FY 2005 NSF COMMITTEE OF VISITOR (COV) REVIEWS

**Guidance to NSF Staff:** This document includes the FY 2005 set of
Core Questions and the COV Report Template for use by NSF staff when
preparing and conducting COVs during FY 2005. Specific guidance for NSF
staff describing the COV review process is described in Subchapter
300-Committee of Visitors Reviews (NSF Manual 1, Section VIII) that can
be obtained at
[http://www.inside.nsf.gov/od/gpra/.](http://www.inside.nsf.gov/od/gpra/. )

NSF relies on the judgment of external experts to maintain high
standards of program management, to provide advice for continuous
improvement of NSF performance, and to ensure openness to the research
and education community served by the Foundation. Committee of Visitor
(COV) reviews provide NSF with external expert judgments in two areas:
(1) assessments of the quality and integrity of program operations and
program-level technical and managerial matters pertaining to proposal
decisions; and (2) comments on how the outputs and outcomes generated by
awardees have contributed to the attainment of NSF's mission and
strategic outcome goals.

Many of the Core Questions are derived from NSF performance goals and
apply to the portfolio of activities represented in the program(s) under
review. The program(s) under review may include several subactivities as
well as NSF-wide activities. The directorate or division may instruct
the COV to provide answers addressing a cluster or group of programs --
a portfolio of activities integrated as a whole -- or to provide answers
specific to the subactivities of the program, with the latter requiring
more time but providing more detailed information.

The Division or Directorate may choose to add questions relevant to the
activities under review. NSF staff should work with the COV members in
advance of the meeting to provide them with the report template,
organized background materials, and to identify questions/goals that
apply to the program(s) under review.

**Guidance to the COV:** The COV report should provide a balanced
assessment of NSF's performance in two primary areas: (A) the integrity
and efficiency of the ***processes*** related to proposal review; and
(B) the quality of the ***results*** of NSF's investments in the form of
outputs and outcomes that appear over time. The COV also explores the
relationships between award decisions and program/NSF-wide goals in
order to determine the likelihood that the portfolio will lead to the
desired results in the future. Discussions leading to answers for Part A
of the Core Questions will require study of confidential material such
as declined proposals and reviewer comments. *COV reports should not
contain confidential material or specific information about declined
proposals**.*** Discussions leading to answers for Part B of the Core
Questions will involve study of non-confidential material such as
results of NSF-funded projects. It is important to recognize that the
reports generated by COVs are used in assessing agency progress in order
to meet government-wide performance reporting requirements, and are made
available to the public. Since material from COV reports is used in NSF
performance reports, the COV report may be subject to an audit.

*We encourage COV members to provide comments to NSF on how to improve
in all areas, as well as suggestions for the COV process, format, and
questions.*

##### FY 2005 REPORT TEMPLATE FOR

#####  NSF COMMITTEES OF VISITORS (COVs)

+-----------------------------------------------------------------------+
| ##### Date of COV: 16-18 Feruary 2005                                 |
+-----------------------------------------------------------------------+
| **Program/Cluster: Condensed Matter Physics, Polymers, Material       |
| Theory, Solid State Chemistry**                                       |
+-----------------------------------------------------------------------+
| **Division: Division of Material Sciences**                           |
+-----------------------------------------------------------------------+
| **Directorate: Mathematical and Physical Sciences**                   |
+-----------------------------------------------------------------------+
| **Number of actions reviewed by COV**[^7]**: Awards: 85 Declinations: |
| 50 Other:5**                                                          |
+-----------------------------------------------------------------------+
| **Total number of actions within Program/Cluster/Division during      |
| period being reviewed by COV**[^8]**: 3699 Awards: 754 Declinations:  |
| 1389 Other: 1556**                                                    |
+-----------------------------------------------------------------------+
| **Manner in which reviewed actions were selected:**                   |
|                                                                       |
| Jackets were selected in equal numbers from the Condensed Matter      |
| Physics, Polymers, Materials Theory, and Solid State Chemistry        |
| programs, and contained a mix of individual investigator and          |
| collaborative proposals.                                              |
+-----------------------------------------------------------------------+

**PART A. INTEGRITY AND EFFICIENCY OF THE PROGRAM'S PROCESSES AND
MANAGEMENT**

Briefly discuss and provide comments for *each* relevant aspect of the
program\'s review process and management. Comments should be based on a
review of proposal actions (awards, declinations, and withdrawals) that
were completed within the past three fiscal years. Provide comments for
each program being reviewed and for those questions that are relevant to
the program under review. Quantitative information may be required for
some questions. Constructive comments noting areas in need of
improvement are encouraged.

A.1 Questions about the quality and effectiveness of the program's use
of merit review procedures. Provide comments in the space below the
question. Discuss areas of concern in the space provided.

+-----------------------------------------------------------+----------+
| **QUALITY AND EFFECTIVENESS OF MERIT REVIEW PROCEDURES**  | **YES,   |
|                                                           | NO,**    |
|                                                           |          |
|                                                           | **DATA   |
|                                                           | NOT      |
|                                                           | AV       |
|                                                           | AILABLE, |
|                                                           | or NOT   |
|                                                           | APPLICAB |
|                                                           | LE**[^9] |
+-----------------------------------------------------------+----------+
| 1\. Is the review mechanism appropriate? (panels, ad hoc  | Yes      |
| reviews, site visits)                                     |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| It was noted that there is a wide variety of programs,    |          |
| due dates and criteria, each needing its own review       |          |
| process. This complex process is handled very well.       |          |
+-----------------------------------------------------------+----------+
| 2\. Is the review process efficient and effective?        | Yes      |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| Even though it is sometimes difficult to find reviewers,  |          |
| the process is executed efficiently and works quite well. |          |
| The review process is critical to selecting top proposals |          |
| in a competitive environment, and in maintaining the      |          |
| overall excellence of the portfolio.                      |          |
+-----------------------------------------------------------+----------+
| 3\. Are reviews consistent with priorities and criteria   | NO       |
| stated in the program's solicitations, announcements, and |          |
| guidelines?                                               |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The broader impact section is not consistently addressed  |          |
| or weighted by the reviewers. We suggest clarifying or    |          |
| redefining the broader impacts section of the review,     |          |
| especially given the importance of specific feedback on   |          |
| points contained within this section to the review        |          |
| process. A number of the reviews did not evaluate the     |          |
| proposal based on the stated program guidelines.          |          |
+-----------------------------------------------------------+----------+
| 4\. Do the individual reviews (either mail or panel)      | YES      |
| provide sufficient information for the principal          |          |
| investigator(s) to understand the basis for the           |          |
| reviewer's recommendation?                                |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The reviewers generally did an outstanding job of         |          |
| communicating their views on the proposals.               |          |
+-----------------------------------------------------------+----------+
| 5\. Do the panel summaries provide sufficient information | YES      |
| for the principal investigator(s) to understand the basis |          |
| for the panel recommendation?                             |          |
|                                                           |          |
| Comments:                                                 |          |
+-----------------------------------------------------------+----------+
| 6\. Is the documentation for recommendations complete,    | YES      |
| and does the program officer provide sufficient           |          |
| information and justification for her/his recommendation? |          |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| The documentation is very complete and the analysis of    |          |
| the program director is uniformly thoughtful and nuanced. |          |
| In cases where funding is declined, detailed and written  |          |
| rationale is not provided to the investigator. However,   |          |
| the PI is invited to call the program officer for more    |          |
| information. The COV observes that the reviewing process  |          |
| is commendably transparent.                               |          |
+-----------------------------------------------------------+----------+
| 7\. Is the time to decision appropriate?                  | YES      |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| Though exact timing varied between different types of     |          |
| proposals, decisions were usually made in 4-6 months. It  |          |
| is notable that DMR response time is generally shorter    |          |
| than that of NSF.                                         |          |
+-----------------------------------------------------------+----------+
| 8\. Discuss any issues identified by the COV concerning   |          |
| the quality and effectiveness of the program's use of     |          |
| merit review procedures:                                  |          |
|                                                           |          |
| In general, the COV felt that the reviewers of individual |          |
| investigator proposals engaged more deeply on the         |          |
| scientific merits of the research than the combined panel |          |
| and mail reviews of interdisciplinary and group           |          |
| proposals, whose reports seem more general. This is       |          |
| problematic, as intense peer scrutiny is critical to      |          |
| assuring the overall excellence of the research           |          |
| portfolio. In part, this is an unresolved problem         |          |
| specific to the review of interdisciplinary research, not |          |
| necessarily addressed by simply broadening the reviewer   |          |
| pool.                                                     |          |
+-----------------------------------------------------------+----------+

A.2 Questions concerning the implementation of the NSF Merit Review
Criteria ([intellectual merit]{.underline} and [broader
impacts]{.underline}) by reviewers and program officers. Provide
comments in the space below the question. Discuss issues or concerns in
the space provided.

+----------------------------------------------------------+-----------+
| **IMPLEMENTATION OF NSF MERIT REVIEW CRITERIA**          | **YES,    |
|                                                          | NO,**     |
|                                                          |           |
|                                                          | **DATA    |
|                                                          | NOT       |
|                                                          | A         |
|                                                          | VAILABLE, |
|                                                          | or NOT    |
|                                                          | APPLICAB  |
|                                                          | LE**[^10] |
+----------------------------------------------------------+-----------+
| 1\. Have the individual reviews (either mail or panel)   | Yes       |
| addressed both merit review criteria?                    |           |
|                                                          |           |
| Comments:                                                |           |
|                                                          |           |
| Reviewer comments on intellectual merit are more         |           |
| extensive and focused than those on broad impact.        |           |
+----------------------------------------------------------+-----------+
| 2\. Have the panel summaries addressed both merit review | Yes       |
| criteria?                                                |           |
|                                                          |           |
| Comments:                                                |           |
|                                                          |           |
| Panels provide more consistent evaluation of both        |           |
| intellectual merit and broad impact components than do   |           |
| individual reviewers. Conversely, reviewers on           |           |
| individual investigator proposals generally engaged more |           |
| deeply with the intellectual merit criteriia than the    |           |
| panel and mail reviewers for interdisciplinary and group |           |
| programs.                                                |           |
+----------------------------------------------------------+-----------+
| 3\. Have the review analyses (Form 7s) addressed both    | Yes       |
| merit review criteria?                                   |           |
|                                                          |           |
| Comments:                                                |           |
|                                                          |           |
| Program director summaries provide thoughtful analysis   |           |
| and clear rationalizations of recommendations.           |           |
+----------------------------------------------------------+-----------+
| 4\. Discuss any issues the COV has identified with       |           |
| respect to implementation of NSF's merit review          |           |
| criteria.                                                |           |
|                                                          |           |
| \- Reviewers do not seem to have a uniform or consistent |           |
| understanding of the boundaries delimiting \"broad       |           |
| impact\". Therefore implementation of these criteria can |           |
| vary widely on a case by case basis. Metrics for         |           |
| performance in the \`broad impact\' area are much less   |           |
| well defined than those for intellectual merit.          |           |
|                                                          |           |
| \- NSF program managers are central and critical to the  |           |
| success of the review process. Their work is far beyond  |           |
| simply tallying ratings of reviewers to arrive at        |           |
| funding decisions.                                       |           |
+----------------------------------------------------------+-----------+

A.3 Questions concerning the selection of reviewers. Provide comments in
the space below the question. Discuss areas of concern in the space
provided.

+---------------------------------------------------------+------------+
| Selection of Reviewers                                  | YES , NO,  |
|                                                         |            |
|                                                         | DATA NOT   |
|                                                         | AVAILABLE, |
|                                                         |            |
|                                                         | or NOT     |
|                                                         | APPLI      |
|                                                         | CABLE[^11] |
+---------------------------------------------------------+------------+
| 1\. Did the program make use of an adequate number of   | YES        |
| reviewers?                                              |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| There were typically 4-5 reviews per proposal. In some  |            |
| cases (CAREER, NIRT) additional written and panel       |            |
| reviews were provided.                                  |            |
+---------------------------------------------------------+------------+
| 2\. Did the program make use of reviewers having        | YES        |
| appropriate expertise and/or qualifications?            |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| This is perhaps the most important part of the entire   |            |
| review process. Considerable effort has been expended   |            |
| to find experts in the field for these reviews. DMR     |            |
| program directors also tracks performance of reviewers  |            |
| over time to ensure high quality.                       |            |
+---------------------------------------------------------+------------+
| 3\. Did the program make appropriate use of reviewers   | YES        |
| to reflect balance among characteristics such as        |            |
| geography, type of institution, and underrepresented    |            |
| groups?                                                 |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| There seems to be a good balance, but the statistics    |            |
| collected on demographics is rather sketchy because     |            |
| only 14% of DMR reviewers responded to this question in |            |
| 2004 (380 out of 2836 in the GPRA Stats, p.3.) Out of   |            |
| this limited sampling 29% identified themselves as      |            |
| being from under-represented groups (111 out of 380,    |            |
| based on the same source.) The DMR COV Reviewer         |            |
| Distribution Table (on p.15) cites stats for industrial |            |
| and government reviewers in the four program areas of   |            |
| interest (CMP, MT, P, SSC) that indicate a doubling of  |            |
| their participation relative to the prior COV report    |            |
| that cited \~5%. For example, CMP listed 4% ind. and 9% |            |
| gov\'t; MT was lower with 1.8% and 5.2%, resp.; P was   |            |
| 9.2 and 4.4%; and SSC with 3.1 and 6.8%, resp.          |            |
+---------------------------------------------------------+------------+
| 4\. Did the program recognize and resolve conflicts of  | YES        |
| interest when appropriate?                              |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| The jackets show cases where otherwise qualified        |            |
| reviewers were eliminated due to COI.                   |            |
+---------------------------------------------------------+------------+
| 5\. Discuss any issues the COV has identified relevant  |            |
| to selection of reviewers.                              |            |
|                                                         |            |
| The previous COV highlighted the under-utilization of   |            |
| reviewers from U.S. industry and the national labs.     |            |
| Impressively, the number of industrial and national lab |            |
| reviewers has doubled from 5% to 10% since the last COV |            |
| report. It would be interesting to know how the rate of |            |
| industrial reviewers declining to review proposals      |            |
| compares with that for academic reviewers. In certain   |            |
| disciplines the insight from industrial reviewers could |            |
| be extremely helpful to DMR.                            |            |
+---------------------------------------------------------+------------+

A.4 Questions concerning the resulting portfolio of awards under review.
Provide comments in the space below the question. Discuss areas of
concern in the space provided.

+------------------------------------------------------+---------------+
| **RESULTING PORTFOLIO OF AWARDS**                    | **A           |
|                                                      | PPROPRIATE,** |
|                                                      |               |
|                                                      | **NOT         |
|                                                      | APPROPRIAT    |
|                                                      | E**[^12]**,** |
|                                                      |               |
|                                                      | **OR DATA NOT |
|                                                      | AVAILABLE**   |
+------------------------------------------------------+---------------+
| 1\. Overall quality of the research and/or education | APPROPRIATE   |
| projects supported by the program.                   |               |
|                                                      |               |
| Comments: The process is so competitive and so well  |               |
| administered by DMR that an excellent outcome is     |               |
| almost ensured                                       |               |
+------------------------------------------------------+---------------+
| 2\. Are awards appropriate in size and duration for  | APPROPRIATE   |
| the scope of the projects?                           |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| Average award size has increased, albeit slowly,     |               |
| over the past several years, due largely to the      |               |
| Nanoscience and ITR initiatives. Given the           |               |
| relatively greater rigor with which they are         |               |
| reviewed, we would like to see single investigator   |               |
| programs maintain their importance within the        |               |
| portfolio, and recommend that their average size,    |               |
| duration, and number be maintained. The current      |               |
| average duration for all programs is slightly        |               |
| greater than 3 years. This duration should ideally   |               |
| be 4 years, the time it takes students to complete   |               |
| the research portion of their PhD.                   |               |
+------------------------------------------------------+---------------+
| 3\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   High risk projects?                              |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| We encourage the DMR to ensure that the high-risk    |               |
| portion of the research portfolio is sufficiently    |               |
| insulated from budget fluctuations.                  |               |
+------------------------------------------------------+---------------+
| 4\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Multidisciplinary projects?                      |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The ever increasing interest in growing areas like   |               |
| Nanoscience has ensured that many multidisciplinary  |               |
| projects are being supported. Also the introduction  |               |
| of specific programs requiring multidisciplinary     |               |
| activity has increased the proportion of this type   |               |
| of research in the portfolio. It is important to     |               |
| ensure that the primary criteria for funding         |               |
| decisions remain intellectual merit and broad        |               |
| impact..                                             |               |
+------------------------------------------------------+---------------+
| 5\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Innovative projects?                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The NSF does an excellent job of funding innovative  |               |
| proposals, although we have noted a number of        |               |
| excellent and innovative proposals which were not    |               |
| funded, presumably for lack of funds.                |               |
+------------------------------------------------------+---------------+
| 6\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Funding for centers, groups and awards to        |               |
|     individuals?                                     |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The portfolios of these programs contains primarily  |               |
| groups and individuals. In terms of this balance,    |               |
| things have remained largely constant with a small   |               |
| incremental growth in small focused groups.          |               |
+------------------------------------------------------+---------------+
| 7\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Awards to new investigators?                     |               |
|                                                      |               |
| Comments:                                            |               |
+------------------------------------------------------+---------------+
| 8\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Geographical distribution of Principal           |               |
|     Investigators?                                   |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The distribution of awards for the most part seems   |               |
| to mirror the population. In the range of 8-13% of   |               |
| the awards went to EPSCoR states.                    |               |
+------------------------------------------------------+---------------+
| 9\. Does the program portfolio have an appropriate   | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Institutional types?                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The statistics within the academic community were    |               |
| not readily available in Materials Theory and Solid  |               |
| State Chemistry, but were for in Condensed Matter    |               |
| Physics and Polymers. For CMP 93% of awards are to   |               |
| Doctoral Granting Research Universities and the      |               |
| remainder to Masters Granting Research Universities  |               |
| between 2002-2004. For Polymers of 178 active grants |               |
| 24 are in EPSCoR states, 7 RUI, 4 HBCU, and 3 REU    |               |
| sites.                                               |               |
+------------------------------------------------------+---------------+
| 10\. Does the program portfolio have an appropriate  | APPROPRIATE   |
| balance of:                                          |               |
|                                                      |               |
| -   Projects that integrate research and education?  |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The overall program has a good balance of research   |               |
| and education. Virtually all DMR proposals combine   |               |
| research and education, however there is             |               |
| considerable variation among individual proposals    |               |
| with some minimizing the educational outreach        |               |
| component, while others are extremely innovative     |               |
| (c.f. 0406127 Cebe).                                 |               |
+------------------------------------------------------+---------------+
| 11\. Does the program portfolio have an appropriate  | APPROPRIATE   |
| balance:                                             |               |
|                                                      |               |
| -   Across disciplines and subdisciplines of the     |               |
|     activity and of emerging opportunities?          |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| A large portion of the present balance in the        |               |
| program portfolio is determined by historical trends |               |
| and past commitments. The reviewers and program      |               |
| directors do an excellent job in recognizing and     |               |
| pursuing emerging areas within these constraints.    |               |
+------------------------------------------------------+---------------+
| 12\. Does the program portfolio have appropriate     | APPROPRIATE   |
| participation of underrepresented groups?            |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| DMR strives to improve the diversity of the          |               |
| supported researchers, and substantial progress has  |               |
| been made since the 2002 COV review. We note, in     |               |
| particular, that there is no appreciable disparity   |               |
| in the funding rate based on gender or minority      |               |
| status.                                              |               |
+------------------------------------------------------+---------------+
| 13\. Is the program relevant to national priorities, | APPROPRIATE   |
| agency mission, relevant fields and other customer   |               |
| needs? Include citations of relevant external        |               |
| reports.                                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The program does a good job of fostering basic       |               |
| science and of preparing the next generation of      |               |
| scientists. A number of these projects have          |               |
| implications for applications as well. There have    |               |
| been an impressive number of strategic workshops in  |               |
| the program areas of interest, i.e. \`Future         |               |
| Directions in Solid State Chemistry\', (Prog. In     |               |
| Sol. Stat Chem 30, 1-101 (2002) The program          |               |
| specifically addresses priority areas such as        |               |
| nanoscale science in the form of funded NIRT         |               |
| proposals as well as individual PI proposals.        |               |
|                                                      |               |
| Frechet (DMR - 0317514) has been exploring           |               |
| dendrimers and other highly branched macromolecules  |               |
| to see if they can be used at molecular              |               |
| nanoreactors. Frechet, J. M. J. \"Dendrimers and     |               |
| other dendritic macromolecules: from building blocks |               |
| to functional assemblies in nanoscience and          |               |
| nanotechnology\" J. Polym. Sci. A., 2003, 41, 3713). |               |
| This work is an excellent example of the use of      |               |
| advanced chemical techniques to engineer dendritic   |               |
| nanoreactors. This work is generating a great deal   |               |
| of interest in the polymer science community         |               |
+------------------------------------------------------+---------------+
| 14\. Discuss any concerns relevant to the quality of |               |
| the projects or the balance of the portfolio.        |               |
|                                                      |               |
| We view the range of activities (Single              |               |
| investigators, small groups, and centers) as a       |               |
| primary strength of the DMR portfolio. Consequently, |               |
| we are concerned that further growth especially in   |               |
| facility support but also in collaborative research  |               |
| should not be at the expense of the single           |               |
| investigator part of the portfolio. Future budget    |               |
| changes should maintain the balance of the           |               |
| portfolio.                                           |               |
+------------------------------------------------------+---------------+

A.5 Management of the program under review. Please comment on:

+-----------------------------------------------------------------------+
| 1\. Management of the program.                                        |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| The COV was impressed with the management of the program by the       |
| Program Directors. The increase of new initiatives has resulted in a  |
| larger number of proposals to be managed. Assistance should be        |
| provided to the Program Directors in identifying new reviewers,       |
| especially from industry and government laboratories.                 |
+-----------------------------------------------------------------------+
| 2\. Responsiveness of the program to emerging research and education  |
| opportunities.                                                        |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| The reviewers and program directors exhibit excellent judgment in     |
| recognizing and pursuing emerging areas, both in funding proposals    |
| and in organizing topical conferences and workshops. Division         |
| director reserves are deployed appropriately to optimize              |
| institutional responsiveness.                                         |
+-----------------------------------------------------------------------+
| 3\. Program planning and prioritization process (internal and         |
| external) that guided the development of the portfolio.               |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| The COV agrees that initiatives should develop in part from ideas     |
| brought forward by investigators, both in proposals and also through  |
| workshop participation. The Program directors also seem well informed |
| about scientific and social needs, and this serves as an additional   |
| factor for determining programmatic priorities. Examples of this      |
| second mode include the Nanoscience and International Initiatives.    |
+-----------------------------------------------------------------------+
| 4\. Additional concerns relevant to the management of the program.    |
+-----------------------------------------------------------------------+

##### PART B. RESULTS : OUTPUTS AND OUTCOMES OF NSF INVESTMENTS

NSF investments produce results that appear over time. The answers to
the first three (People, Ideas and Tools) questions in this section are
to be based on the COV's study of award results, which are direct and
indirect accomplishments of projects supported by the program. These
projects may be currently active or closed out during the previous three
fiscal years. The COV review may also include consideration of
significant impacts and advances that have developed since the previous
COV review and are demonstrably linked to NSF investments, regardless of
when the investments were made. Incremental progress made on results
reported in prior fiscal years may also be considered.

The following questions are developed using the NSF outcome goals in the
NSF Strategic Plan. The COV should look carefully at and comment on (1)
noteworthy achievements of the year based on NSF awards; (2) the ways in
which funded projects have collectively affected progress toward NSF's
mission and strategic outcomes; and (3) expectations for future
performance based on the current set of awards. NSF asks the COV to
provide comments on the degree to which past investments in research and
education have contributed to NSF's progress towards its annual
strategic outcome goals and to its mission:

-   To promote the progress of science.

-   To advance national health, prosperity, and welfare.

-   To secure the national defense.

-   And for other purposes.

Excellence in managing NSF underpins all of the agency's activities. For
the response to the Outcome Goal for Organizational Excellence, the COV
should comment, where appropriate, on NSF providing an agile, innovative
organization. Critical indicators in this area include (1) operation of
a credible, efficient merit review system; (2) utilizing and sustaining
broad access to new and emerging technologies for business application;
(3) developing a diverse, capable, motivated staff that operates with
efficiency and integrity; and (4) developing and using performance
assessment tools and measures to provide an environment of continuous
improvement in NSF's intellectual investments as well as its management
effectiveness.

**B. Please provide comments on the activity as it relates to NSF's
Strategic Outcome Goals. Provide examples of outcomes (nuggets) as
appropriate. Examples should reference the NSF award number, the
Principal Investigator(s) names, and their institutions.**

+-----------------------------------------------------------------------+
| **B.1 [OUTCOME GOAL for PEOPLE]{.underline}: Developing "a diverse,   |
| competitive and globally engaged workforce of scientists, engineers,  |
| technologists and well-prepared citizens."**                          |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| NSF continues to be the premier national organization for the         |
| promotion of science, education, and outreach. In the past three      |
| years, DMR programs supported each year approximately 1500            |
| undergraduate students, 2000 graduate students, and 500 postdoctoral  |
| associates, on a par with the number in each year of the previous 3   |
| year period. In addition to student support provided through grants,  |
| the DMR also sponsors a number of innovative and effective projects   |
| geared towards both recruiting and retaining students to divisional   |
| disciplines, and science in general.                                  |
|                                                                       |
| The Summer Research Program in Solid State Chemistry (DMR 0303450     |
| Hwu) continues to be particularly effective in providing an authentic |
| research experience for undergraduates and faculty members from small |
| undergraduate institutions. DMR investigators continue to be very     |
| active and effective in utilizing the REU program, and we consider    |
| the NHMFL program noteworthy for its scope and for the range of       |
| backgrounds of the students participating. The Boulder Summer School  |
| for Condensed Matter and Materials Physics (DMR- 9987640 Radzihovsky) |
| brings 60 beginning graduate students together for a month of         |
| intensive learning and interaction with each other and expert         |
| practicioners in focused research areas.                              |
|                                                                       |
| The DMR Polymers Program has placed special emphasis on attracting a  |
| broad diversity of                                                    |
|                                                                       |
| scientists and educators and helping them develop their scientific    |
| careers. In 2003, three                                               |
|                                                                       |
| African-American grantees received top international recognitions in  |
| their fields. Professor                                               |
|                                                                       |
| Larry Dalton (U. Washington and USC,DMR-0092380) received the         |
| Materials Chemistry Award from the American Chemical Society.         |
| Professor Joshua Otaigbe (U. Southern Mississippi, DMR-0242754) was   |
| elected Fellow of the United Kingdom Institute of Materials.          |
| Professor Valerie Sheares Ashby, a CAREER                             |
|                                                                       |
| grantee at Iowa State University (DMR-9733837) had a full-page        |
| article written about her superb teaching, mentoring, and scholarly   |
| accomplishments in the ACS Chemical & Engineering News.               |
|                                                                       |
| NSF awards.                                                           |
|                                                                       |
| The DMR Condensed Matter Physics Program has initiated a multiyear    |
| award (DMR 0400699 Jackson) to the National Society of Black          |
| Physicists to support their well attended annual conferences, an      |
| important resource for universities seeking to increase diversity in  |
| their graduate programs.                                              |
|                                                                       |
| Solid State Chemistry supports an undergraduate research program for  |
| Native Americans (DMR 0071672 Porter). Students in this program have  |
| a graduation rate of 100%, in contrast to the 10% mean for this group |
| overall.                                                              |
+-----------------------------------------------------------------------+
| **B.2 [OUTCOME GOAL for IDEAS]{.underline}: Enabling "discovery       |
| across the frontier of science and engineering, connected to          |
| learning, innovation, and service to society."**                      |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| NSF is to be commended for its enhanced support of interdisciplinary  |
| research during the period since the last COV report. Materials       |
| research is an ideal vehicle for these collaborations, as it          |
| intrinsically spans a variety of scientific and technical frontiers,  |
| enabling advances in other disciplines. This is one of the major      |
| strengths which DMR brings to the national scientific endeavor. For   |
| example, the discovery and development of advanced materials will be  |
| crucial for space exploration, new medical technologies, new          |
| information technologies such as quantum computing, and advanced      |
| accelerator design. DMR continues to define world class fundamental   |
| research.                                                             |
|                                                                       |
| The distributed nature of the DMR portfolio and the breadth of        |
| expertise supported optimize the possibility of unanticipated         |
| research breakthroughs, while enabling rapid response to take         |
| advantage of these new discoveries.                                   |
|                                                                       |
| In the period since the last COV report, a number of significant      |
| breakthroughs have been supported by DMR. Research highlights from    |
| the period since the last COV report include:                         |
|                                                                       |
| -control of the hydrophobicity of polymer surfaces, using with uv     |
| irradiation or novel y-shaped molecules, with important implications  |
| for applications in adhesion, friction, lubrication, and switchable   |
| surfaces( DMR-0208825 Ober, DMR-0308982 Tsukurk).                     |
|                                                                       |
| -Synthetic membranes developed with polymeric analogs of proteins     |
| (DMR 0203755 Safinya)                                                 |
|                                                                       |
| -Design and subsequent virtual synthesis of new multiferroic          |
| compounds (DMR 0312407 Spaldin)                                       |
|                                                                       |
| -Magnetic field driven metal-semiconductor transitions in carbon      |
| nanotubes (DMR 0134770 Bezryadin, DMR 0134058 Kono)                   |
|                                                                       |
| -Further advances in molecular electronics and materials for          |
| informational technology, such as wave function entanglement in two   |
| dimensional electron gases (DMR-0102153 Kastner).                     |
|                                                                       |
| -theory of novel electronic phases found near quantum phase           |
| transitions in magnets (DMR 0312495 Coleman).                         |
|                                                                       |
| -Self assembled polymer nanofibers for nerve repair (DMR 0108342      |
| Stupp)                                                                |
|                                                                       |
| -shape controlled nanostructures of metals (DMR 0413366 Xia)          |
|                                                                       |
| A DMR supported researcher, Anthony Leggett, shared the 2003 Nobel    |
| Prize in Physics with Abrikosov and Ginzburg for primary              |
| contributions to the theories of superconductors and superfluids. His |
| continued work in quantum dissipation is widely relevant to our       |
| understanding of systems as diverse as quantum fluids and supersolids |
| (DMR 0207071 Chan) , and to quantum electronics and its application   |
| to information theory.                                                |
|                                                                       |
| DMR supported researchers increasingly make important                 |
| interdisciplinary contributions to other disciplines like biophysics  |
| (DMR 0203963 Marko and DMR 01055903 Zocchi), fluid mechanics (DMR     |
| 0352777 Nagel), and atomic physics (DMR 0304906 Grier). These         |
| projects exemplify the increasing importance and impact of            |
| interdisciplinary projects, and the central role of DMR research in   |
| forging this new science.                                             |
+-----------------------------------------------------------------------+
| **B.3 [OUTCOME GOAL for TOOLS:]{.underline} Providing "broadly        |
| accessible, state-of-the-art S&E facilities, tools and other          |
| infrastructure that enable discovery, learning and innovation."**     |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| DMR supported researchers have developed a broad range of novel       |
| techniques and tools for experimental, computational, and theoretical |
| use. Some highlights include:                                         |
|                                                                       |
| -Synthesis of novel nanomaterials using atomic manipulation by STM    |
| (DMR- 0114246 Ho, DMR 0210893 Ho DMR 0304314 Smith), and also         |
| molecular manipulation (DMR 0308045 Yazdani)                          |
|                                                                       |
| -Development of the dynamical mean-field theory, and its application  |
| to electronic instabilities such as the metal-insulator transition    |
| and moment formation in correlated electron systems (DMR - 0312478    |
| Kotliar)                                                              |
|                                                                       |
| -The development of femtosecond fluorescent spectroscopy tools to     |
| study the behavior of polymers and dendrimers for nanoscale           |
| optoelectronics (DMR 0134691 Goodson).                                |
|                                                                       |
| -Optoelectronic spin control and imaging of nuclear spins, with       |
| implications for ultrahigh frequency spintronics (DMR -0305223        |
| Awschalom). Awschalom was a co-recipient of the 2005 Buckley Prize    |
| for his accomplishments in this field.                                |
|                                                                       |
| \- Novel computing architecture for materials computation based on a  |
| cluster of playstation consoles (DMR - 0325939 Johnson)               |
|                                                                       |
| Researchers have made important advances due to the general           |
| availability of novel research tools.                                 |
|                                                                       |
| For instance, the availability of advanced scattering beamlines at    |
| NIST and at the NSLS has enabled a wide variety of basic explorations |
| in fields as diverse as polymer physics (DMR 0405432 Hsaio) to        |
| emergent behavior in correlated electron materials.                   |
+-----------------------------------------------------------------------+
| **B.4 [UTCOME GOAL for ORGANIZATIONAL EXCELLENCE]{.underline}:        |
| Providing "an agile, innovative organization that fulfills its        |
| mission through leadership in state-of-the-art business practices."** |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| We note that the quality and depth of analysis which support          |
| decisions on proposals and maintains the scientific strength of DMR   |
| research rests squarely on the efforts of the program directors. We   |
| encourage NSF to continue augmenting the number of Program Directors, |
| particularly as the recent increase in number of submitted proposals  |
| has all but restored the work load of the Program Directors to the    |
| unacceptably high level found by the previous COV. Despite this,the   |
| proposal response time in DMR is well below the agency average, which |
| contributes to the ability of the supported researchers to respond to |
| new and fast-developing fields. We view the use of divisional reserve |
| funds in DMR as effective in supporting a competitive environment     |
| within DMR, also contributing to responsiveness to new initiatives.   |
| The FASTLANE system has greatly simplified proposal submission,       |
| review, and internal management.                                      |
+-----------------------------------------------------------------------+

##### PART C. OTHER TOPICS

C.1 Please comment on any program areas in need of improvement or gaps
(if any) within program areas.

The COV is concerned about the weaker growth of DMR, relative to other
divisions, and of the Foundation as a whole. A fundamental challenge for
the materials community is to identify crosscutting grand challenges
which motivate increased investment in both manpower and
instrumentation. DMR can play a central role in helping the materials
community to develop an accurate, simple, and compelling message which
motivates this greater investment. Materials research is distinct from
other scientific areas in its great diversity of expertise and approach,
and its eschewing of monolithic goals. We view the over-reliance on
applications for making this argument as inconsistent to the core
scientific values and activities of DMR researchers, not capturing the
fundamental intellectual worth and excitement of our endeavor. We
believe that the NSF undervalues the importance of materials researchers
as leaders in exploring the interdisciplinary theme of emergent behavior
and complexity which form the bases of both the physical and life
sciences. Through the diversity of expertise and approaches, materials
researchers offer a unique and promising approach.

C.2 Please provide comments as appropriate on the program's performance
in meeting program-specific goals and objectives that are not covered by
the above questions.

Specific performance metrics for reviewing the \`broader impact\'
components of projects should be developed. In this way, there is an
opportunity to channel the efforts of a diverse group of researchers
into a collective and defining effort which would be of general benefit
to the materials community, and to the public.

**C.3 Please identify agency-wide issues that should be addressed by NSF
to help improve the program\'s performance.**

We are concerned that further growth especially in facility support but
also in collaborative research should not be at the expense of the
single investigator part of the portfolio. Future budget changes should
maintain the overall balance of the portfolio. Even in the face of
decreasing resources, it is important not to decrease grant size from
the present level, consistent with the recommendations of the National
Science Board.

The NSF should study ways to implement the open dissemination of
published results, such has been done by NIH (Medline) and the e-print
archives. As improvements in IT become available, it would be of great
value to make a broader range of experimental and theoretical results
and associated tools freely available, such as is done in Astronomy and
Genomics.

C.4 Please provide comments on any other issues the COV feels are
relevant.

**C.5 NSF would appreciate your comments on how to improve the COV
review process, format and report template.**

Advance instruction about the process, and more targeted preparatory
reading would have been most useful in making effective use of the very
limited time during the visit itself. The COV had considerable
difficulty in correlating information with the template questions, and
some indexing would have been very helpful. The information, while
comprehensive, would have been more useful if it had a more uniform
level of synthesis and presentation. The COV appreciates the
responsiveness of the DMR staff in providing orientation to the
information as well as additional information during the visit itself.
We suggest that next time there should be brief presentations from DMR
staff to give an initial overview of the program.

**SIGNATURE BLOCK:**

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

For the COV - Division of Materials Research

Horst Stormer

Chair

##### CORE QUESTIONS and REPORT TEMPLATE

#####  for 

##### FY 2005 NSF COMMITTEE OF VISITOR (COV) REVIEWS

**Guidance to NSF Staff:** This document includes the FY 2005 set of
Core Questions and the COV Report Template for use by NSF staff when
preparing and conducting COVs during FY 2005. Specific guidance for NSF
staff describing the COV review process is described in Subchapter
300-Committee of Visitors Reviews (NSF Manual 1, Section VIII) that can
be obtained at
[http://www.inside.nsf.gov/od/gpra/.](http://www.inside.nsf.gov/od/gpra/. )

NSF relies on the judgment of external experts to maintain high
standards of program management, to provide advice for continuous
improvement of NSF performance, and to ensure openness to the research
and education community served by the Foundation. Committee of Visitor
(COV) reviews provide NSF with external expert judgments in two areas:
(1) assessments of the quality and integrity of program operations and
program-level technical and managerial matters pertaining to proposal
decisions; and (2) comments on how the outputs and outcomes generated by
awardees have contributed to the attainment of NSF's mission and
strategic outcome goals.

Many of the Core Questions are derived from NSF performance goals and
apply to the portfolio of activities represented in the program(s) under
review. The program(s) under review may include several subactivities as
well as NSF-wide activities. The directorate or program may instruct the
COV to provide answers addressing a cluster or group of programs -- a
portfolio of activities integrated as a whole -- or to provide answers
specific to the subactivities of the program, with the latter requiring
more time but providing more detailed information.

The Program or Directorate may choose to add questions relevant to the
activities under review. NSF staff should work with the COV members in
advance of the meeting to provide them with the report template,
organized background materials, and to identify questions/goals that
apply to the program(s) under review.

**Guidance to the COV:** The COV report should provide a balanced
assessment of NSF's performance in two primary areas: (A) the integrity
and efficiency of the ***processes*** related to proposal review; and
(B) the quality of the ***results*** of NSF's investments in the form of
outputs and outcomes that appear over time. The COV also explores the
relationships between award decisions and program/NSF-wide goals in
order to determine the likelihood that the portfolio will lead to the
desired results in the future. Discussions leading to answers for Part A
of the Core Questions will require study of confidential material such
as declined proposals and reviewer comments. *COV reports should not
contain confidential material or specific information about declined
proposals**.*** Discussions leading to answers for Part B of the Core
Questions will involve study of non-confidential material such as
results of NSF-funded projects. It is important to recognize that the
reports generated by COVs are used in assessing agency progress in order
to meet government-wide performance reporting requirements, and are made
available to the public. Since material from COV reports is used in NSF
performance reports, the COV report may be subject to an audit.

*We encourage COV members to provide comments to NSF on how to improve
in all areas, as well as suggestions for the COV process, format, and
questions.*

##### FY 2005 REPORT TEMPLATE FOR

#####  NSF COMMITTEES OF VISITORS (COVs)

+-----------------------------------------------------------------------+
| ##### Date of COV 16-18 February 2005                                 |
+-----------------------------------------------------------------------+
| **Program/Cluster: Centers/Facilities/Instrumentation/Special         |
| Programs**                                                            |
+-----------------------------------------------------------------------+
| **Program: Division of Materials Research**                           |
+-----------------------------------------------------------------------+
| **Directorate: Mathematical and Physical Sciences**                   |
+-----------------------------------------------------------------------+
| **Number of actions reviewed by COV**[^13]**: 115 Awards: 55          |
| Declinations: 50Other: 10**                                           |
+-----------------------------------------------------------------------+
| **Total number of actions within Program/Cluster/Program during       |
| period being reviewed by COV**[^14]**: 1319 Awards: 314 Declinations: |
| 658 Other: 347**                                                      |
+-----------------------------------------------------------------------+
| **Manner in which reviewed actions were selected: Jackets were        |
| selected in roughly equal numbers from the MRSEC, Facilities, Special |
| Programs and Instrumentation programs. Jackets corresponding to clear |
| \'fund\' or \'do not-fund\' decisions were looked at, but more time   |
| was spent on jackets corresponding to proposals that were close to    |
| the fund/do not fund line.**                                          |
+-----------------------------------------------------------------------+

**PART A. INTEGRITY AND EFFICIENCY OF THE PROGRAM'S PROCESSES AND
MANAGEMENT**

Briefly discuss and provide comments for *each* relevant aspect of the
program\'s review process and management. Comments should be based on a
review of proposal actions (awards, declinations, and withdrawals) that
were *completed within the past three fiscal years*. Provide comments
for *each* program being reviewed and for those questions that are
relevant to the program under review. Quantitative information may be
required for some questions. Constructive comments noting areas in need
of improvement are encouraged.

A.1 Questions about the quality and effectiveness of the program's use
of merit review procedures. Provide comments in the space below the
question. Discuss areas of concern in the space provided.

+-----------------------------------------------------------+----------+
| **QUALITY AND EFFECTIVENESS OF MERIT REVIEW PROCEDURES**  | **YES,   |
|                                                           | NO,**    |
|                                                           |          |
|                                                           | **DATA   |
|                                                           | NOT      |
|                                                           | AV       |
|                                                           | AILABLE, |
|                                                           | or NOT   |
|                                                           | A        |
|                                                           | PPLICABL |
|                                                           | E**[^15] |
+-----------------------------------------------------------+----------+
| 1\. Is the review mechanism appropriate? (panels, ad hoc  | YES      |
| reviews, site visits)                                     |          |
|                                                           |          |
| Comments: The MRSEC/NSEC process is multi-step including  |          |
| pre-proposals, full proposals, panels and reverse site    |          |
| visits. The level of investment in these Centers          |          |
| justifies the complex and multi-step process. Facility    |          |
| proposals are similarly complex as is appropriate for the |          |
| scale of the proposed research. IMR and OSP processes are |          |
| far less involved, reflecting the narrower scope and size |          |
| of the projects.                                          |          |
+-----------------------------------------------------------+----------+
| 2\. Is the review process efficient and effective?        | YES      |
|                                                           |          |
| Comments: The process is clearly effective, as the        |          |
| quality of science and education program supported is     |          |
| very high. The process is also as efficient as can be     |          |
| expected, given the complexities of the programs under    |          |
| review.                                                   |          |
+-----------------------------------------------------------+----------+
| 3\. Are reviews consistent with priorities and criteria   | YES      |
| stated in the program's solicitations, announcements, and |          |
| guidelines?                                               |          |
|                                                           |          |
| Comments:                                                 |          |
+-----------------------------------------------------------+----------+
| 4\. Do the individual reviews (either mail or panel)      | YES      |
| provide sufficient information for the principal          |          |
| investigator(s) to understand the basis for the           |          |
| reviewer's recommendation?                                |          |
|                                                           |          |
| Comments: Most reviews are thoughtful and expressive.     |          |
+-----------------------------------------------------------+----------+
| 5\. Do the panel summaries provide sufficient information | YES      |
| for the principal investigator(s) to understand the basis |          |
| for the panel recommendation?                             |          |
|                                                           |          |
| Comments: Panel summaries are quite thorough and of high  |          |
| utility to the PI in all cases reviewed.                  |          |
+-----------------------------------------------------------+----------+
| 6\. Is the documentation for recommendations complete,    | YES      |
| and does the program officer provide sufficient           |          |
| information and justification for her/his recommendation? |          |
|                                                           |          |
| Comments: The documentation is generally extensive.       |          |
| Decisions on borderline cases when funds are too limited  |          |
| to support all recommended proposals are difficult for a  |          |
| PI to understand. The decision-making process in this     |          |
| situation needs to be communicated carefully (while       |          |
| preserving aspects of anonymity) so that the reasons for  |          |
| borderline rejections are understood by the PIs.          |          |
+-----------------------------------------------------------+----------+
| 7\. Is the time to decision appropriate?                  | YES      |
|                                                           |          |
| Comments:                                                 |          |
|                                                           |          |
| Most proposals are decided within 6 months of submission. |          |
| IMI proposals require about 8 months because of the need  |          |
| for international cooperation in review. MRSEC and NSEC   |          |
| proposals require about 9 months. The review of these     |          |
| proposals involves pre-proposals; pre-proposal panel;     |          |
| preparation of full proposals (a very significant task);  |          |
| full proposal panel; and a reverse site visit.            |          |
| Consequently, a significant proportion of the 9 months is |          |
| to provide finalists time to prepare full proposals and   |          |
| for the reverse site visit. This time line places         |          |
| significant stress on both the PIs and reviewers and it   |          |
| is possible that a slight increase in the turn around     |          |
| time would be advantageous. It is possible that a survey  |          |
| of PIs who have gone through this process (both           |          |
| successfully and unsuccessfully) might provide guidance   |          |
| for adjustments in the timing of the various steps in     |          |
| future competitions.                                      |          |
+-----------------------------------------------------------+----------+
| 8\. Discuss any issues identified by the COV concerning   |          |
| the quality and effectiveness of the program's use of     |          |
| merit review procedures:                                  |          |
|                                                           |          |
| no additional issues                                      |          |
+-----------------------------------------------------------+----------+

A.2 Questions concerning the implementation of the NSF Merit Review
Criteria ([intellectual merit]{.underline} and [broader
impacts]{.underline}) by reviewers and program officers. Provide
comments in the space below the question. Discuss issues or concerns in
the space provided.

+----------------------------------------------------------+-----------+
| **IMPLEMENTATION OF NSF MERIT REVIEW CRITERIA**          | **YES,    |
|                                                          | NO,**     |
|                                                          |           |
|                                                          | **DATA    |
|                                                          | NOT       |
|                                                          | A         |
|                                                          | VAILABLE, |
|                                                          | or NOT    |
|                                                          | APPLICAB  |
|                                                          | LE**[^16] |
+----------------------------------------------------------+-----------+
| 1\. Have the individual reviews (either mail or panel)   | yes       |
| addressed both merit review criteria?                    |           |
|                                                          |           |
| Comments: Adherence to addressing these two categories   |           |
| in all reviewers\' reports has been very high.           |           |
+----------------------------------------------------------+-----------+
| 2\. Have the panel summaries addressed both merit review | yes       |
| criteria?                                                |           |
|                                                          |           |
| Comments:                                                |           |
|                                                          |           |
| Most panel summaries include these two categories.       |           |
+----------------------------------------------------------+-----------+
| 3\. Have the *review analyses* (Form 7s) addressed both  | YES       |
| merit review criteria?                                   |           |
|                                                          |           |
| Comments:                                                |           |
|                                                          |           |
| Form 7s invariably address both criteria. In general,    |           |
| the Form 7s that were reviewed were excellent sources of |           |
| information relevant to understanding the decision to    |           |
| fund or not fund.                                        |           |
+----------------------------------------------------------+-----------+
| 4\. Discuss any issues the COV has identified with       |           |
| respect to implementation of NSF's merit review          |           |
| criteria.                                                |           |
|                                                          |           |
| The balance of emphasis on the two merit review criteria |           |
| varies according to the particular program, and          |           |
| rightfully so. Programs that are intended to increase    |           |
| the presence of science and technology in otherwise      |           |
| unexposed communities are appropriately appreciated for  |           |
| their contribution to education and outreach while not   |           |
| sacrificing scientific rigor. In these programs, it is   |           |
| appropriate to place greater emphasis on the broad       |           |
| impact. On the other hand, in Center programs,           |           |
| demonstration of both intellectual merit and broader     |           |
| impact are critical for a successful proposal.           |           |
+----------------------------------------------------------+-----------+

A.3 Questions concerning the selection of reviewers. Provide comments in
the space below the question. Discuss areas of concern in the space
provided.

+---------------------------------------------------------+------------+
| Selection of Reviewers                                  | YES , NO,  |
|                                                         |            |
|                                                         | DATA NOT   |
|                                                         | AVAILABLE, |
|                                                         |            |
|                                                         | or NOT     |
|                                                         | APPLI      |
|                                                         | CABLE[^17] |
+---------------------------------------------------------+------------+
| 1\. Did the program make use of an adequate number of   | YES        |
| reviewers?                                              |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| Special Programs used 126 reviewers over the last 3     |            |
| years to review approximately 240 proposals.            |            |
|                                                         |            |
| In 2002 the total number of referees used in the MRSEC  |            |
| program was 390. For full proposals, each IRG of a      |            |
| MRSEC is reviewed by at least 5 reviewers, with as many |            |
| as 15 or more reviewers for the entire proposal. This   |            |
| number is often required to deal with the very broad    |            |
| scientific program proposed for the Centers.            |            |
|                                                         |            |
| Instrumentation for Materials Research (IMR) utilized   |            |
| 321 reviewers over three years to review about 600      |            |
| proposals. Each proposal was reviewd by at least 3      |            |
| reviewers.                                              |            |
+---------------------------------------------------------+------------+
| 2\. Did the program make use of reviewers having        | YES        |
| appropriate expertise and/or qualifications?            |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| The answer is a resounding \'Yes\', and one of the      |            |
| great strengths of the programs we reviewed. It is a    |            |
| daunting task to find well qualified scientists with    |            |
| the expertise needed to review Center proposals that    |            |
| involve very broad ranging science programs. The DMR    |            |
| staff maintains a remarkable grasp of the materials     |            |
| research community in this country, and does a superb   |            |
| job of assigning proposals to appropriate reviewers.    |            |
| Reviewers are chosen not only for their scientific      |            |
| expertise, but also on the basis of the type of         |            |
| institution. Attention is also paid to obtaining        |            |
| reviews from underrepresented minorities.               |            |
+---------------------------------------------------------+------------+
| 3\. Did the program make appropriate use of reviewers   | YES        |
| to reflect balance among characteristics such as        |            |
| geography, type of institution, and underrepresented    |            |
| groups?                                                 |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| All facility-related programs, and those from the       |            |
| Office of Special Programs and MRSEC/NSEC Programs, pay |            |
| close attention to the statistical distribution of      |            |
| reviewers, and the metrics show that this area is       |            |
| covered well. For example, EPSCoR representation        |            |
| (17/126, 13% for OSP and 46/423, 11% for Centers) is    |            |
| used to measure geographical representation, and this   |            |
| fraction is appropriate.                                |            |
|                                                         |            |
| In tracking referee diversity, sometimes personal       |            |
| knowledge of referees was used to adjust                |            |
| under-represented groups numbers, especially when the   |            |
| referee does not volunteer the tracked information,     |            |
| again a necessary and effective practice. Female and    |            |
| minority reviewers in OSP were well represented at      |            |
| 39/126 (31%). While this number is good for diversity,  |            |
| it represents a larger burden on the attending          |            |
| demographic slice. As a benchmark, the Materials        |            |
| Research Society membership consists of approximately   |            |
| 25% female; there are no extant data for MRS            |            |
| underrepresented groups.                                |            |
+---------------------------------------------------------+------------+
| 4\. Did the program recognize and resolve conflicts of  | YES        |
| interest when appropriate?                              |            |
|                                                         |            |
| Comments:                                               |            |
|                                                         |            |
| The NSF staff are clearly very careful to insure that   |            |
| all potential conflicts of interest and identified and  |            |
| the proper steps taken to avoid them.                   |            |
+---------------------------------------------------------+------------+
| 5\. Discuss any issues the COV has identified relevant  |            |
| to selection of reviewers.                              |            |
|                                                         |            |
| Intellectual diversity in the review community is       |            |
| highly prized at NSF, and this factor is measured by    |            |
| several highly tracked metrics, from geography to field |            |
| of expertise. This practice pushes DMR toward a         |            |
| possible unintended consequence: Limited demographic    |            |
| slices (such as women and minorities) are pushed toward |            |
| an increasing review load. This trend could potentially |            |
| have a negative impact on the careers of these people   |            |
| by taking their energies away from tasks more central   |            |
| to their long term success.                             |            |
+---------------------------------------------------------+------------+

A.4 Questions concerning the resulting portfolio of awards under review.
Provide comments in the space below the question. Discuss areas of
concern in the space provided.

+------------------------------------------------------+---------------+
| **RESULTING PORTFOLIO OF AWARDS**                    | **A           |
|                                                      | PPROPRIATE,** |
|                                                      |               |
|                                                      | **NOT         |
|                                                      | APPROPRIAT    |
|                                                      | E**[^18]**,** |
|                                                      |               |
|                                                      | **OR DATA NOT |
|                                                      | AVAILABLE**   |
+------------------------------------------------------+---------------+
| 1\. Overall quality of the research and/or education | Yes           |
| projects supported by the program.                   |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| Overall the quality of the research supported by the |               |
| Programs through a variety of centers and national   |               |
| facilities is excellent.                             |               |
|                                                      |               |
| The MRSECs support a very broad range of cutting     |               |
| edge research and high-impact educational and        |               |
| outreach programs. They seem to be an especially     |               |
| good mechanism for fostering effective               |               |
| interdisciplinary research. They also exemplify how  |               |
| innovative projects can be carried out at relatively |               |
| low cost by taking advantage of a variety of local   |               |
| expertise (an example is the MRSEC at the University |               |
| of Southern Mississippi where the merging of         |               |
| existing high quality, but disjoint, efforts in      |               |
| polymer research has led to a strong focused program |               |
| on polymer films). The MRSEC seed programs are       |               |
| excellent for supporting innovative small scale      |               |
| projects and for the funding of innovative projects  |               |
| carried out by young investigators. Several of the   |               |
| MRSECs do an excellent job of promoting interactions |               |
| between theoretical and experimental research, an    |               |
| interaction judged by be very important for the      |               |
| long-term health of quality materials research.      |               |
|                                                      |               |
| The portfolio of activities in the area of education |               |
| and outreach that are funded under the MRSEC         |               |
| umbrella is extremely diverse with very broad        |               |
| impact, including support for undergraduate research |               |
| and teacher programs, partnerships with local        |               |
| science museums (Chicago Museum of Science, New      |               |
| Jersey's Liberty Science Center), programs for local |               |
| schools K-12, presentations to the general public,   |               |
| and many others. In their reports the centers        |               |
| estimated that 13,500 pre-college students were      |               |
| impacted by these education outreach programs.       |               |
|                                                      |               |
| The new PREM program has proven an excellent         |               |
| mechanism for involving minority-serving             |               |
| institutions in research partnerships with more      |               |
| established research universities.                   |               |
|                                                      |               |
| The user facilities supported within DMR achieve a   |               |
| huge impact by providing unique tools to excellent   |               |
| researchers. For instance, CHESS is an example of an |               |
| outstanding user facility with an impressive track   |               |
| record of superb research and technical innovation.  |               |
| It has also played a crucial role as                 |               |
| cross-disciplinary training ground for young         |               |
| researchers working on synchrotron-based science.    |               |
+------------------------------------------------------+---------------+
| 2\. Are awards appropriate in size and duration for  | Yes           |
| the scope of the projects?                           |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The level of funding is generally quite appropriate  |               |
| to the scope of the project. The increase of MRSEC   |               |
| awards to six years duration is a positive change    |               |
| given the scale of these projects and the length and |               |
| complexity of the review process.                    |               |
|                                                      |               |
| The trend towards larger award size and longer       |               |
| duration is to be commended. On the other hand, in   |               |
| the present climate of shrinking or at best steady   |               |
| budget this is often achieved at the cost of         |               |
| decreasing the number of awards. This can have       |               |
| demoralizing effect across the community, with young |               |
| researchers particularly hard hit. The committee     |               |
| suggests exploration of a program of small seeds     |               |
| grants for new PIs as a mechanism to increase the    |               |
| relatively low success rate for young investigators  |               |
| in some programs.                                    |               |
|                                                      |               |
| National facilities require a long life span to      |               |
| justify the investment in constructing the facility. |               |
| Use of these facilities for production of excellent  |               |
| research requires a certain minimum level of support |               |
| to maintain excellent support staff, user programs   |               |
| and necessary updates and repairs of                 |               |
| instrumentation. It is pointless to build a          |               |
| world-class facility and then starve it with         |               |
| inadequate funding levels. The facilities program    |               |
| appears to recognize this, and the facilities, as    |               |
| far as we could tell from the information available, |               |
| are adequately supported. Nevertheless, in an era of |               |
| tight budgets, better ways of judging the impact of  |               |
| these facilities on the production of world class    |               |
| research may be needed. The Program should try to    |               |
| develop more uniform quantitative measures of the    |               |
| integrated impact and utilization of the large user  |               |
| facilities and ask the directors of these facilities |               |
| to use these metrics in their reports (e.g., number  |               |
| of unique users; number of user groups; number of    |               |
| resulting publications in refereed journals). The    |               |
| COV experienced some frustration in attempting to    |               |
| compare different facilities due to the different    |               |
| metrics used by different centers. In particular the |               |
| definition of \'user\' seemed to vary from facility  |               |
| to facility.                                         |               |
+------------------------------------------------------+---------------+
| 3\. Does the program portfolio have an appropriate   | Yes           |
| balance of:                                          |               |
|                                                      |               |
| -   High risk projects?                              |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| An effort has been made to support high-risk         |               |
| projects, although the number of awards in this      |               |
| class could be increased. The use of seed funds at   |               |
| the MRSEC can provide a good mechanism for funding   |               |
| highly innovative projects. Perhaps the proportion   |               |
| of the budget used for this (currently about 10% of  |               |
| centers budget goes to \'high risk\' or seed         |               |
| projects) should be encouraged to grow within        |               |
| MRSECs.                                              |               |
+------------------------------------------------------+---------------+
| 4\. Does the program portfolio have an appropriate   | Yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Multidisciplinary projects?                      |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The portfolio includes a strong support of           |               |
| nanoscience and a clear growth of support in the     |               |
| area of biomaterials. The centers seem to be highly  |               |
| effective in their support of interdisciplinary      |               |
| science. Several of the MRSECs reviewed are          |               |
| excellent examples of very effective collaborations  |               |
| across disciplines, including physics, material      |               |
| science, chemistry, computer science and biology.    |               |
|                                                      |               |
| Both MRSECs and large user facilities also are the   |               |
| ideal venue for effective large-scale outreach       |               |
| programs. An example of a high-impact outreach       |               |
| program is the one run by the University of Chicago  |               |
| MRSEC in partnership with the Chicago Museum of      |               |
| Science.                                             |               |
|                                                      |               |
| In the context of the IMR/MRI program the            |               |
| interdisciplinary nature of many large scale         |               |
| facilities (e.g., microscopy tools that may serve a  |               |
| broad community from material scientists to          |               |
| biologists) provides a unique opportunity for cost   |               |
| sharing with other agencies. Some of this has in     |               |
| fact been done (.e.g co-funding with ONR for         |               |
| microscopy at Northwestern)                          |               |
+------------------------------------------------------+---------------+
| 5\. Does the program portfolio have an appropriate   | yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Innovative projects?                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| It is not clear how an innovative project should be  |               |
| defined. The COV felt that any project supported by  |               |
| NSF should be innovative. Even by that standard, the |               |
| programs reviewed appear to pass muster.             |               |
+------------------------------------------------------+---------------+
| 6\. Does the program portfolio have an appropriate   | yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Funding for centers, groups and awards to        |               |
|     individuals?                                     |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| This is a very broad and important question that     |               |
| goes beyond the charge of this particular            |               |
| subcommittee and addresses the budgetary balance of  |               |
| the entire Division. That said, we find that the     |               |
| MRSEC program constitutes a very effective use of    |               |
| funds and almost uniformly the outcome of these      |               |
| collaborative research centers is much more than the |               |
| sum of their parts. This subcommittee also feels     |               |
| that inquiry-driven single investigator research     |               |
| remains a key characteristic of many research areas, |               |
| especially in theory. It is important that core      |               |
| programs be nurtured and preserved and that the      |               |
| funding of single PI not be decreased. We saw no     |               |
| evidence to suggest an inappropriate balance.        |               |
+------------------------------------------------------+---------------+
| 7\. Does the program portfolio have an appropriate   | not           |
| balance of:                                          | applicable    |
|                                                      |               |
| -   Awards to new investigators?                     |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| Few investigators that have not had NSF grants in    |               |
| the past apply for center or facility grants. In     |               |
| most cases this is also true of the programs funded  |               |
| in instrumentation, education, or international      |               |
| collaborations. These are generally not the first    |               |
| grant an investigator starting his or her career     |               |
| will apply for.                                      |               |
+------------------------------------------------------+---------------+
| 8\. Does the program portfolio have an appropriate   | Yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Geographical distribution of Principal           |               |
|     Investigators?                                   |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| Generally there is a good geographical distribution  |               |
| of MRSEC awards. The facilities are too few in       |               |
| number to provide a statistically significant answer |               |
| to this question. The REU sites seem to track the    |               |
| population density and number of submissions.        |               |
+------------------------------------------------------+---------------+
| 9\. Does the program portfolio have an appropriate   | Yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Institutional types?                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The portfolio includes a very broad range of         |               |
| institutions. A clear effort has been made by the    |               |
| Program to increase the support to non-PhD granting  |               |
| institutions.                                        |               |
+------------------------------------------------------+---------------+
| 10\. Does the program portfolio have an appropriate  | Yes           |
| balance of:                                          |               |
|                                                      |               |
| -   Projects that integrate research and education?  |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The integration of research and education /outreach  |               |
| is one of the mandates of the MRSECs. These centers  |               |
| have responded in a very active and creative way to  |               |
| this charge. Some of the National facilities (CHESS, |               |
| NHMFL) have also developed effective programs in     |               |
| this area. It is clear that these multidisciplinary  |               |
| research centers are in an ideal position for        |               |
| running high-impact programs in the area of          |               |
| outreach, much more so than individual PIs.          |               |
+------------------------------------------------------+---------------+
| 11\. Does the program portfolio have an appropriate  | Yes           |
| balance:                                             |               |
|                                                      |               |
| -   Across disciplines and subdisciplines of the     |               |
|     activity and of emerging opportunities?          |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The MRSEC portfolio reflects the increasing focus on |               |
| nanoscience and biological materials seen throughout |               |
| the materials research community. The IMR/MRI have   |               |
| shown an increase of tools for novel microscopies.   |               |
| These trends reflect the priorities of current       |               |
| research and are therefore appropriate.              |               |
|                                                      |               |
| The COV wishes to commend the activities of OSP,     |               |
| which supports a very broad array of interesting     |               |
| initiatives. In particular the IMI initiative has    |               |
| provided a mechanism for full-scale support of       |               |
| international collaborations both with developed and |               |
| developing countries. The Princeton IMI *US Africa   |               |
| Materials Institute* is exemplary for this type of   |               |
| collaboration.                                       |               |
+------------------------------------------------------+---------------+
| 12\. Does the program portfolio have appropriate     | Not there     |
| participation of underrepresented groups?            | yet, but      |
|                                                      | making good   |
| Comments:                                            | progress      |
|                                                      |               |
| There has been a modest, but steady increase in the  |               |
| participation of women in leading positions at the   |               |
| research centers. Currently two of the MRSEC and     |               |
| three of the NSEC Directors are women and another    |               |
| nine are leaders or co-leaders of IRGs. The increase |               |
| in the number of minority researchers has been much  |               |
| more modest. Interesting programs have been          |               |
| developed, such as the summer research program for   |               |
| faculty at historically black colleges at the        |               |
| University of Alabama MRSEC. But, to date, the       |               |
| impact of these programs has been relatively modest. |               |
|                                                      |               |
| The PREM program - that reaches out to               |               |
| minority-serving undergraduate institutions (e.g.,   |               |
| California State University at LA partnered with     |               |
| Caltech, University of Puerto Rico at Humacao        |               |
| partnered with the University of Pennsylvania) and   |               |
| involves them in cutting edge research - is an       |               |
| excellent effort aimed as attempting to diversify    |               |
| the pool of potential scientists. It is too soon to  |               |
| assess the effectiveness of this program.            |               |
|                                                      |               |
| Very effective programs have been put in place for   |               |
| undergraduate research (REU) and teacher training    |               |
| (RET), but there is a gap at the high school level   |               |
| as very few programs exist involving high school     |               |
| students in research projects (an example is the     |               |
| Young Scholars Program organized by the Chicago      |               |
| MRSEC, although this focuses on math) . The          |               |
| development of summer internship programs for high   |               |
| school juniors could serve as an effective tool for  |               |
| recruiting a new and diverse body of college         |               |
| students into the sciences. Materials science        |               |
| provides hands-on science training and the MRSECs    |               |
| are ideally positioned to undertake this endeavor.   |               |
| The NSF should consider earmarking a modest amount   |               |
| of funds for such initiatives.                       |               |
+------------------------------------------------------+---------------+
| 13\. Is the program relevant to national priorities, | Yes           |
| agency mission, relevant fields and other customer   |               |
| needs? Include citations of relevant external        |               |
| reports.                                             |               |
|                                                      |               |
| Comments:                                            |               |
|                                                      |               |
| The committee feels that the varied portfolio        |               |
| pursued by the Program is highly relevant to         |               |
| national priorities and to the needs of the          |               |
| scientific community. DMR is doing the best job it   |               |
| can in fulfilling its mission with limited           |               |
| resources.                                           |               |
|                                                      |               |
| The MRSECs have been extremely effective at carrying |               |
| out their multifaceted mission of pursuing           |               |
| world-class interdisciplinary research in material   |               |
| science, while educating the public at large on      |               |
| technical matters and encouraging young people to    |               |
| enter scientific fields (C. L. Bryant, MRS Bulletin, |               |
| August 2002).                                        |               |
|                                                      |               |
| Several recent external reports exist assessing the  |               |
| relevance and impact of the national facilities,     |               |
| such as the *Report on the Status and Needs of Major |               |
| Neutron Scattering Facilities and Instruments in the |               |
| United States*, Office of Science and Technology,    |               |
| June 2002; the 2004 NRC Committee Report on          |               |
| *Opportunities in High Magnetic Field Science*, and  |               |
| several others.                                      |               |
+------------------------------------------------------+---------------+
| 14\. Discuss any concerns relevant to the quality of |               |
| the projects or the balance of the portfolio.        |               |
|                                                      |               |
| It was noted that some variations exist from program |               |
| to program in the boundary below which a proposal is |               |
| not funded. It is clear that in borderline cases the |               |
| program directors incorporate all available criteria |               |
| in making their decision. In those cases reviewed by |               |
| the committee, the result of these deliberations     |               |
| appeared to be very well justified and usually       |               |
| clearly spelled out in the Form 7.                   |               |
|                                                      |               |
| It is good to see MRSEC's awards going to            |               |
| Institutions that have not received prior            |               |
| large-scale funding (e.g., University of Southern    |               |
| Mississippi and University of Alabama). The COV also |               |
| hopes to see an increase in the number of such       |               |
| awards in the future.                                |               |
|                                                      |               |
| The NSF should explore new ways of co-funding user   |               |
| facilities and instrumentation programs with other   |               |
| agencies and scientific organizations, whenever      |               |
| appropriate.                                         |               |
+------------------------------------------------------+---------------+

A.5 Management of the program under review. Please comment on:

+-----------------------------------------------------------------------+
| 1\. Management of the program.                                        |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| Facilities, MRSECS and NSECS involve an unusually interactive         |
| management process. Each program is managed differently as is         |
| appropriate to the needs of the program. Review and feedback ranges   |
| from regular proposal review, to site visits and visiting committees. |
| All have significant post award management which includes annual      |
| reports which are reviewed for both programmatic and financial        |
| information. In addition, the national facilities have an annual site |
| visit by the program director, PD, with or without a site visit       |
| committee. MRSECs have a mid-term site visit by an external panel,    |
| director meetings twice per year which include topical areas, and a   |
| site visit by the PD every other year. NSECs have site visits by the  |
| PD alone and an external panel in alternating years. This complex     |
| process appears appropriate and effective in maintaining the highest  |
| quality effort in these large centers and facilities.                 |
+-----------------------------------------------------------------------+
| 2\. Responsiveness of the program to emerging research and education  |
| opportunities.                                                        |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| Programs adjust their priorities to respond to perceived needs.       |
| IMR/MRI have responded to the needs for shared instrumentation. In    |
| order to encourage instrument development, NSF has dropped the        |
| requirement for cost sharing on such proposals. MRSECs actively       |
| participate in NSF-wide initiatives and respond to changing trends in |
| proposal submissions. The MRSEC system is designed to be responsive.  |
| For instance, centers are allowed to change IRGs to take advantage of |
| new research opportunities.                                           |
+-----------------------------------------------------------------------+
| 3\. Program planning and prioritization process (internal and         |
| external) that guided the development of the portfolio.               |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| For the national facilities, program planning involves a long term    |
| process where new ideas are encouraged through direct interaction     |
| with the PD who uses white papers to initiate the planning process    |
| through the NSF divisions. The interactive and deliberate nature of   |
| this process is appropriate given the complexity of developing a new  |
| national facility.                                                    |
|                                                                       |
| MRSECs are designed to be responsive to new proposal trends. In       |
| addition, PD\'s interact extensively with the scientific community    |
| with direct meetings, presentations at conferences and workshops to   |
| encourage the development of new ideas for centers. Calls for NSEC    |
| proposals will include areas of emphasis.                             |
|                                                                       |
| OSP was created in response to a perceived need for managing          |
| cross-cutting programs including educational, research, and           |
| work-force development. The increase in international programs within |
| DMR was a major part of that perceived need.                          |
+-----------------------------------------------------------------------+
| 4\. Additional concerns relevant to the management of the program.    |
|                                                                       |
| The most pressing concern discussed was the work load of the PD\'s,   |
| especially in the MRSEC and NSEC Programs. In a time of limited       |
| financial resources, the importance of thorough and fair proposal     |
| review cannot be overstated. The requirement for post award           |
| management is also very important to the success of the centers. Both |
| of these areas require significant efforts by the PD\'s. This will    |
| become even more true as additional multi-element centers are funded. |
|                                                                       |
| In addition, continuity may be an issue for a complicated program     |
| such as the MRSECs which has 1 permanent and two temporary PDs. The   |
| tenure of a temporary PD is only three years - much less than the     |
| lifetime of a MRSEC. With the large number of centers, the current    |
| staffing model is in danger of failing to maintain the current high   |
| quality of the process.                                               |
+-----------------------------------------------------------------------+

##### PART B. RESULTS : OUTPUTS AND OUTCOMES OF NSF INVESTMENTS

NSF investments produce results that appear over time. The answers to
the first three (People, Ideas and Tools) questions in this section are
to be based on the COV's study of award results, which are direct and
indirect accomplishments of projects supported by the program. These
projects may be currently active or closed out during the previous three
fiscal years. The COV review may also include consideration of
significant impacts and advances that have developed since the previous
COV review and are demonstrably linked to NSF investments, regardless of
when the investments were made. Incremental progress made on results
reported in prior fiscal years may also be considered.

The following questions are developed using the NSF outcome goals in the
NSF Strategic Plan. The COV should look carefully at and comment on (1)
noteworthy achievements of the year based on NSF awards; (2) the ways in
which funded projects have collectively affected progress toward NSF's
mission and strategic outcomes; and (3) expectations for future
performance based on the current set of awards. NSF asks the COV to
provide comments on the degree to which past investments in research and
education have contributed to NSF's progress towards its annual
strategic outcome goals and to its mission:

-   To promote the progress of science.

-   To advance national health, prosperity, and welfare.

-   To secure the national defense.

-   And for other purposes.

Excellence in managing NSF underpins all of the agency's activities. For
the response to the Outcome Goal for Organizational Excellence, the COV
should comment, where appropriate, on NSF providing an agile, innovative
organization. Critical indicators in this area include (1) operation of
a credible, efficient merit review system; (2) utilizing and sustaining
broad access to new and emerging technologies for business application;
(3) developing a diverse, capable, motivated staff that operates with
efficiency and integrity; and (4) developing and using performance
assessment tools and measures to provide an environment of continuous
improvement in NSF's intellectual investments as well as its management
effectiveness.

**B. Please provide comments on the activity as it relates to NSF's
Strategic Outcome Goals. Provide examples of outcomes (nuggets) as
appropriate. Examples should reference the NSF award number, the
Principal Investigator(s) names, and their institutions.**

+-----------------------------------------------------------------------+
| **B.1 [OUTCOME GOAL for PEOPLE]{.underline}: Developing "a diverse,   |
| competitive and globally engaged workforce of scientists, engineers,  |
| technologists and well-prepared citizens."**                          |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| The MRSEC program has done a remarkable job in integration of         |
| research and education, development of work force, and outreach to    |
| industry and the general public. It is successful in many respects.   |
|                                                                       |
| • Researchers at centers won numerous prestigious awards, including 5 |
| elected to NAS and 9 to NAE.                                          |
|                                                                       |
| • In Year 2004 alone, center programs involved 953 graduate students, |
| 567 undergraduates and 194 pre-college teachers, and impacted 13,500  |
| pre-college students.                                                 |
|                                                                       |
| • Centers had successful outreach programs for K-12 students and the  |
| general public. Example: University of Wisconsin (DMR 0425880)        |
| developed nanoscale education kits for K-12 students and the general  |
| public, and these have been widely adopted by other organizations.    |
|                                                                       |
| • The PREM program helps enhance diversity in research community by   |
| supporting research partnership between minority-serving institutions |
| and MRSEC centers. Example: University of Puerto Rico, Humacao, and   |
| University of Pennsylvania (DMR 0353730) worked jointly on polymers   |
| and materials for sensors and actuators.                              |
|                                                                       |
| Among the MRSEC participants, 12% of the faculty, 20% of the          |
| postdocs, 30% of the graduate students, and 40% of the undergraduates |
| are women, but only 3% of the faculty, 3% of the postdocs, 4% of the  |
| graduate students, and 12% of the undergraduates are                  |
| under-represented minorities. With available resources, MRSEC perhaps |
| could attempt to devote more effort to attract minority students at   |
| the K-12 and undergraduate levels into science and technology.        |
|                                                                       |
| NSF facilities provide on-site training for specialists in areas      |
| associated with the facilities. They are much needed in the           |
| scientific research community. Examples: CHESS (DMR 0225180) has      |
| trained synchrotron scientists and engineers who now occupy positions |
| in almost every synchrotron facility worldwide. Close to 40% of the   |
| graduate students at CHESS are women. CHESS facility was also         |
| instrumental for Rod MacKinnon\'s protein crystallographic work (2003 |
| Nobel Prize in Chemistry). NHMFL (DMR 9713424) houses the strongest   |
| magnets in the world and trains scientists and engineers in this      |
| special area. It also hosted annual workshops and summer programs     |
| involving 1200 teachers in 12 states and conducted outreach to \~4000 |
| K-12 students.                                                        |
|                                                                       |
| IMR-MIP program helps educate or train students on relatively         |
| sophisticated instruments in many cases, ranging from basic knowledge |
| transfer to lower-level student use, design and construction of       |
| instruments.                                                          |
|                                                                       |
| OSP coordinates and supports cross-discipline education of            |
| undergraduates. In FY2004, DMR supported 39 REU sites, in addition to |
| the 27 REU sites incorporated in the existing MRSEC, one in each NSEC |
| and 2 other REU sites associated with facilities. It also supported   |
| 140 pre-college teachers to gain research experiences at 26 REU       |
| sites.                                                                |
|                                                                       |
| IMI program supported by OSP has a goal to develop a new generation   |
| of scientists and engineers with international leadership             |
| capabilities. This is an important and laudable goal. The program is  |
| still young; the outcomes are yet to be seen.                         |
|                                                                       |
| In general, MRSECs and OSP activities significantly aided the NSF\'s  |
| attempts to achieve a connection between science and engineering      |
| breakthroughs and learning, innovation, and service to society. In    |
| several instances these projects embrace underrepresented groups,     |
| encourage the development of young scientist and engineers, and serve |
| society through the production of an effective science and            |
| engineering workforce. An excellent example of this is the University |
| of Alabama, MRSEC: 0213985, Butler. They have achieved a strong HBCU  |
| connection with strong scientific accomplishment.                     |
+-----------------------------------------------------------------------+
| **B.2 [OUTCOME GOAL for IDEAS]{.underline}: Enabling "discovery       |
| across the frontier of science and engineering, connected to          |
| learning, innovation, and service to society."**                      |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| MRSECs:                                                               |
|                                                                       |
| MRSECs continue to produce many significant discoveries that lead to  |
| an expansion of the frontier of science and engineering. Several      |
| examples of how the MRSECs contribute to the science and engineering  |
| world include:                                                        |
|                                                                       |
| Molecular storage by charge trapping: MRSEC: 0213985, Butler. This    |
| MRSEC demonstrated charge trapping in a 2 nm thick film of a designed |
| charge-storage arylamine dendrimer vapor deposited on silicon         |
| substrates.                                                           |
|                                                                       |
| Multiple superconducting gaps in MgB2 were identified by the          |
| University of Wisconsin, Madison MRSEC. The impact of this idea is    |
| that MgB2 may allow significant increase in the upper critical        |
| magnetic field that can be exploited for technological benefit. This  |
| MRSEC also reported rapid real time detection of proteins binding to  |
| receptors using liquid crystals and reported these results in Science |
| (Dec. 19th).                                                          |
|                                                                       |
| In a third MRSEC, Ingber and Whitesides at Harvard University         |
| demonstrated cell shape manipulation by controlling transmembrane     |
| integrin receptors through soft lithography.                          |
|                                                                       |
| Facilities:                                                           |
|                                                                       |
| In general, NSF funded facilities produce capabilities and support    |
| that no single investigator (or small team) can produce or sustain.   |
| At CHESS, the 2003 Nobel Prize for Chemistry (Rod MacKinnon) was      |
| awarded for membrane protein structure determination; MacKinnon made  |
| 31 visits to CHESS from 1998 onward that enabled his research.        |
| MacKinnon acknowledges that his accomplishments would not have been   |
| possible without the infrastructure and expertise provided by CHESS.  |
|                                                                       |
| Investigators using the NSF user facility at the Synchrotron          |
| Radiation Center at UW-M also studied MgB2 with similar impact as the |
| UW-M MRSEC. Investigators at the SRC also revealed inclusions in      |
| ancient rocks too small to be seen with any other instrument that     |
| proved that the earth had continents and water 4.4 Billion years      |
| ago - earlier than previously expected.                               |
|                                                                       |
| The Impact of NSF user facilities goes beyond the NSF family of       |
| investigators: The 2004 Compton award went to M. Blume for his work   |
| on magnetic scattering that started at CHESS.                         |
|                                                                       |
| Instrumentation:                                                      |
|                                                                       |
| A spin-polarized field effect transistor (spin-FET)                   |
| MRI:0215872:Thibado exhibited 100,000 times higher switching density  |
| than conventional transistors.                                        |
+-----------------------------------------------------------------------+
| **B.3 [OUTCOME GOAL for TOOLS:]{.underline} Providing "broadly        |
| accessible, state-of-the-art S&E facilities, tools and other          |
| infrastructure that enable discovery, learning and innovation."**     |
|                                                                       |
| Comments:                                                             |
|                                                                       |
| Comments: DMR funding of National Facilities, Instrumentation, and    |
| Centers/Institutes clearly enables broad and open access to unique    |
| tools and infrastructure.                                             |
|                                                                       |
| NAF: Provides facilities with unique national capabilities that are   |
| critical to a large number of researchers (3600 users, including 2000 |
| students, 800 research reports in 2004). Major awards include the     |
| 2003 Nobel for Chemistry (MacKinnon, ion channels in membranes, using |
| CHESS), the 2003 Warren award in crystallography (Egami - atomic PDFs |
| in disordered crystals, using CHESS), and the Compton award (Blume,   |
| magnetic resonance scattering, using CHESS). An example of a novel,   |
| societal use for a new tool is the confocal x-ray fluorescence        |
| microscope at CHESS, being used to see hidden paint layers in         |
| historic art. The new 900 MHz ultra-wide-bore, high field magnet for  |
| NMR at the NHMFL represents a world-leading leap forward in NMR       |
| spectroscopy/imaging capability for materials and biology. Examples   |
| of research highlights at the NHMFL include magnetic condensation of  |
| \"triplons\" in the Han-dynasty purple pigment material, analysis of  |
| protein encapsulation kinetics in the HIV virus, and identification   |
| of mysterious electron glasses inside otherwise normal Silicon        |
| transistors. At CHRNS, a new multi-critical point in superconducting  |
| niobium was discovered that defies explanation by existing theory.    |
| Also at CHRNS, the structure of \"chaperonins\", proteins that        |
| oversee the folding processes of other proteins, has been elucidated  |
| to great extent.                                                      |
|                                                                       |
| IMR/MIP: Develops new analytical capabilities, including: nano-Raman  |
| spectroscopy with 30 nm resolution (Sokolov, Akron, 215966); STM      |
| imaging of insulators using single electron tunneling (Williams, U.   |
| Utah, 216711), a multi-ion FIB, which has very wide-ranging utility   |
| in nanofabrication (Erskine, Harvard, 216297); and a height-scale     |
| calibrated AFM for simultaneous mapping of surface forces with 3D     |
| structure.                                                            |
|                                                                       |
| MRSEC: Each MRSEC makes available state-of-the art instrumentation in |
| a number of laboratories, which crosscut a range of interdisciplinary |
| activities, to researchers of all career levels, especially students. |
| Annual MRSEC facilities budget a total of more than \$5M for          |
| facilities/instrumentation, and this provides access for 2700         |
| researchers nationwide. An important aspect is that MRSEC budgets     |
| include technician support for maintenance and operation of critical  |
| university facilities, which is rarely supported by other funding     |
| modes.                                                                |
|                                                                       |
| OSP: It cannot be ignored that REUs and RETs provide undergraduates,  |
| high school students, and K-12 teachers with access to state-of-the   |
| art instrumentation that plays a key role in stimulating new          |
| scientific careers.                                                   |
+-----------------------------------------------------------------------+
| **B.4 [OUTCOME GOAL for ORGANIZATIONAL EXCELLENCE]{.underline}:       |
| Providing "an agile, innovative organization that fulfills its        |
| mission through leadership in state-of-the-art business practices."** |
|                                                                       |
| Comments: DMR maintains an excellent environment for support of a     |
| broad range of activities that nurture and support the materials      |
| research community.                                                   |
|                                                                       |
| Funding mechanisms provide flexibility to react to new opportunities, |
| identify highly innovative ideas and find ways to support them.       |
|                                                                       |
| Examples include:                                                     |
|                                                                       |
| DMR\'s Office of Special Programs - commendable international effort  |
| in *sustained* international collaboration: Princeton and Africa, UC  |
| Santa Barbara and Asia/Americas.                                      |
|                                                                       |
| Some MRSEC\'s also have *sustained* international components.         |
|                                                                       |
| IMR\'s MIP - Instrumentation fund created to fill a void in           |
| intermediate funding range.                                           |
|                                                                       |
| NSEC - created to fill nano niche, based on MRSEC and Engineering     |
| Research Center models.                                               |
|                                                                       |
| DIRECTOR\'s RESERVE discretionary fund (as pointed out in previous    |
| COV) to fund higher risk or uncategorizable ventures.                 |
|                                                                       |
| PREM program commendable for addressing a stratum of society to       |
| integrate into national science and technology. Judging criteria of   |
| intellectual and societal impact are appropriately adjusted.          |
|                                                                       |
| MRSEC\'s outreach and OSP\'s REU/RET programs lauded for efforts in   |
| involving more citizens in science. There\'s an apparent              |
| discontinuity in further engaging high school students who respond    |
| positively to the outreach efforts. What is the NSF or DMR funding    |
| for high-school internships?                                          |
+-----------------------------------------------------------------------+

##### PART C. OTHER TOPICS

C.1 Please comment on any program areas in need of improvement or gaps
(if any) within program areas.

Although a number of new programs have been developed, and the impact of
enhanced investment in nanoscience is continuing to grow, much of DMR\'s
structure has not changed substantially in 7 years. Given the pace of
change in materials science, the committee felt that a re-examination of
the balance and definition of programs may be warranted.

The support of DMR, even in times of tight budgetary constraints, seems
inadequate to the scope and scale of questions that its research
programs are addressing. Under NSF\'s overarching goals, DMR is poised
to provide large, continuing returns. The COV feel that materials
research is the leading enabler for a number of grand challenges,
including innovation-driven prosperity, national security, and energy
security. No other scientific discipline has primary impact on all three
of these national needs.

DMR is also making an impact in the science and engineering work force.
The aging research workforce in physical sciences, and in particular, in
DHS, DoD, and DOE, is under current study by an Interagency Working
Group (IWG). The demographics point to a long term shortage of trained
scientists and engineers in the coming decades. DMR\'s Centers offer a
steep return on investment through support of students and postdocs. The
COV observes that the \'people\' impact from Centers is especially great
because they foster a unique, interdisciplinary training environment for
large numbers of early career scientists. The internal staffing of the
Centers Program must follow the program size and balance, and the
current staffing is insufficient to handle current Centers load, not
even considering further growth.

DMR has been forward looking in leading the Foundation in the creation
of Centers, Midscale Instrumentation, and Special Programs. Making
difficult investment-dis-investment decisions at times of budgetary
stress draws attention by the community and requires management courage.

C.2 Please provide comments as appropriate on the program's performance
in meeting program-specific goals and objectives that are not covered by
the above questions.

DMR has done a splendid job in meeting its program goals and those of
the scientific community that it serves within the limits of the
resources available. It has led the way within the agency at developing
a diverse and creative portfolio and at taking advantage of targeted
initiatives such as those in nanoscience and information technology. It
has achieved a good balance of centers, groups and individual funding.
The COV feels, however, that the core programs are in some danger in the
present funding climate. The success rate in some areas is becoming
dangerously low as some programs are forced to turn down excellent
proposals for lack of funds. We wish to make a strong statement on the
need to preserve the core programs that are the natural niche for
innovative, curiosity-driven research carried out by individual PIs.

**C.3 Please identify agency-wide issues that should be addressed by NSF
to help improve the program\'s performance.**

The ever increasing workload of NSF staff is a growing concern. In a
time of shrinking financial resources, the importance of thorough and
fair proposal review can not be overestimated. The requirement for post
award management is also important to the success of the science and
engineering centers and must be maintained. Both of these areas require
significant efforts by the PD\'s. This will become even more critical as
more multi-element centers are added. In addition, continuity may be an
issue for a complicated program such as the MRSECs which has 1 permanent
and two temporary PDs. For example, the tenure of a temporary PD is only
three years which does not match the length of a MRSEC award. With the
large number of centers, the current staffing model is in danger of
failing to maintain the current high quality of the process.

Increases in DMR budget has not kept pace with that of MPS or NSF.
Between 1998 and 2004, the overall NSF budget increased by almost 70%
whereas the DMR budget only increased by a little over 40%. The central
importance of materials research within all fields (physics, chemistry,
biology, engineering, etc.) and the increasing importance of
interdisciplinary centers should cause NSF to examine its budget
priorities.

C.4 Please provide comments on any other issues the COV feels are
relevant.

One of the goals of MRSEC is to promote interactions and collaborations
among researchers in various disciplines. In presenting the outcomes, we
suggest that the centers put more emphasis on accomplishments that would
not be possible without the setting of MRSEC.

**C.5 NSF would appreciate your comments on how to improve the COV
review process, format and report template.**

To immerse the COV quickly and efficiently into the DMR/NSF world, it
would be useful for future COV panels to be provided with some
additional information:

Short overviews by NSF staff of each of the programs to be reviewed.

> Additional graphical representations of the interactions among
> programs, such as Venn Diagrams of MRSEC, facilities, Instrumentation
> and core program funding and co-funding, as well as special programs,
> Independent Investigator programs and others to show the
> interrelationship of these funding vehicles
>
> We felt that a histogram of project funding size would be a helpful
> graphic to better display the relative funding levels of small grants,
> individual investigator projects, instrumentation proposals and
> centers. A simple statement of the average size of award was not
> helpful when the size of individual awards spanned from a few thousand
> dollars to millions.

When a committee member does not wish to plug into the NSF network, he
should be given a red sticker for his laptop without waiting in a long
line and powering up his computer.

After hours access to conference rooms should be made possible.

**SIGNATURE BLOCK:**

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

For the COV - Division of Materials Research

Horst Stormer

Chair

[^1]: To be provided by NSF staff.

[^2]: To be provided by NSF staff.

[^3]: If "Not Applicable" please explain why in the "Comments" section.

[^4]: In "Not Applicable" please explain why in the "Comments" section.

[^5]: If "Not Applicable" please explain why in the "Comments" section.

[^6]: If "Not Appropriate" please explain why in the "Comments" section.

[^7]: To be provided by NSF staff.

[^8]: To be provided by NSF staff.

[^9]: If "Not Applicable" please explain why in the "Comments" section.

[^10]: In "Not Applicable" please explain why in the "Comments" section.

[^11]: If "Not Applicable" please explain why in the "Comments" section.

[^12]: If "Not Appropriate" please explain why in the "Comments"
    section.

[^13]: To be provided by NSF staff.

[^14]: To be provided by NSF staff.

[^15]: If "Not Applicable" please explain why in the "Comments" section.

[^16]: In "Not Applicable" please explain why in the "Comments" section.

[^17]: If "Not Applicable" please explain why in the "Comments" section.

[^18]: If "Not Appropriate" please explain why in the "Comments"
    section.