# Presentation: 690137

## EMSL:
Our Path Forward as a 
National Scientific User Facility

- Our Path Forward as a
**National Scientific User Facility**

**Allison A. Campbell**
- EMSL Director
- July 10, 2006

## Presentation contents

- The Environmental Molecular Sciences Laboratory’s (EMSL’s) response to the recommendations of the U.S. Department of Energy (DOE) Biological and Environmental Research Advisory Committee (BERAC).
- EMSL’s “Dashboard.”

## 2005 BERAC Review Committee

- Michelle Broido, PhD
- Carolyn Cassady, PhD
- David Cowburn, PhD
- Paul V. Dressendorfer, PhD
- Jeremy  M. Hales, PhD
- Janet Hering, PhD
- Inez Hua, PhD
- Lynn Katz, PhD
- Chad Rienstra, PhD
- Ralph Roskies, PhD
- David Russell, PhD
- Charles Wade, PhD
- Mavrik Zavarin, PhD

## 2005 BERAC Review Report Summary

- Positive findings
- Science performed at EMSL is very high and often cutting-edge quality
- EMSL capabilities are unique
- Additional funding is needed to maintain state of the art
- Endorsed focus around scientific themes.
- Cross-cutting areas for improvement
- Shared mission
- Scientific leadership
- Strategic planning and science focus
- Radiological capabilities
- Advisory committees
- External peer review
- Capital revitalization
- Research needs to cut across facilities
- User counts and survey
- User outreach.
- Facility-specific areas for improvement
- Focus on science themes
- Training of users
- Opportunities for staff professional development.

## Recommendation: Establish a shared mission

- 	**Shared** **Mission**
- 	EMSL, a national scientific user facility at the Pacific Northwest National Laboratory,
- 	provides integrated experimental
- and computational resources
- 	for discovery and technological innovation
- 	in the environmental molecular sciences
- 	to support the needs of DOE
- and the nation

**Notes:**
We need to verbally transition here. Our first step was to come to consensus about what we are.  That then drives how we get there (next slide)

## Recommendation: Strengthen scientific, business, and project management leadership

**Recommendation: Strengthen scientific, business, and project management leadership**

**Stewards:**
- Project management
- Financial management and reporting
- User training and tracking
- Outreach
- Utilization and access tools

**Stewards:**
-  Cross-cutting scientific leadership
-  Strategic direction/partnerships
-  Scientific themes
-  Instrumentation gaps/needs
-  Capability needs

**Stewards:**
-  Safety
-  Security
-  Environmental compliance
-  Construction projects

**Notes:**
Stress
Leadership
New functions
Facilities
Sr management team

## Recommendation: Refresh EMSL’s Strategic Plan and focus on science themes

- Focuses on mission as a national scientific user facility
- Developed with stakeholder involvement
- 5-year strategy
- Outlines EMSL science themes
- Atmospheric aerosol chemistry
- Geochemistry/biogeochemistry and subsurface science
- Biological interactions and interfaces
- Science of interfacial phenomena
- Guide to set priorities and make decisions on funding, equipment, and staffing
- Establishes timeline and measures for success.

* ****Publications***

***Proposals***

***Publications***

* ****Patents***

***Utilization >2 systems***

***Data checks***

***Meetings***

***Staff survey***

***  ******DART, TRC***

***Construction***

***Down time***

* ****Publications/project***

* ****Increased proposal***
***pool***

***Survey response/***
***satisfaction***

***>2 process improvement***

## Recommendation: Two committees with redefined charters and revitalized membership

- Developed new charters for Science Advisory Committee and User Advisory Committee.
- Science Advisory Committee
- New Chair Appointed – Len Spicer
- Identified and solicited potential new members
- Meeting date set – August 3
- Focused on science and strategy
- Member of PNNL’s Laboratory Advisory Committee
- Staff liaison established.
- User Advisory Committee
- Current committee to serve as nominating committee
- Focused on user needs
- User advocate
- Planned Fall 2006 meeting
- Staff liaison established.

*Len Spicer*

## Recommendation:  Establish consistent policies and practices for external peer review across all EMSL facilities

- All user proposals externally peer reviewed
- Consistent review criteria now in place
- Science merit
- Alignment with EMSL mission, science themes, and resources
- Potential for publications
- EMSL Proposal Review System developed to make peer-review process consistent across all facilities

## Recommendation: Establish formal calls to focus research on high-impact science and encourage cross-facility utilization

- Methods for bringing research to EMSL
- Formal calls for proposal
- Science theme calls
- Facility-focused calls
- Open call; may be submitted any time.
- Measures
- Increase proposals submitted
- Increase proposals that will use multiple systems
- Increase newly submitted proposals from external users
- Increase proposals from new users.

- 2006 recommendation
- Replace facility-focused calls with an EMSL-wide capability-based call.

## Response to science theme call

| Science
Theme | Total | Non–PNNL Lead Principal Investigators | New User
Principal Investigators  |
| --- | --- | --- | --- |
| Aerosol | 8 | 5 | 2 |
| Biology | 14 | 8 | 6
 |
| Subsurface | 22 | 11 | 7
 |
| Interfacial | 55 | 34 | 9 |
| Total | 99 | 58 | 24 |

- Principal Investigator Information

## Previous user definition
Counted all proposal participants
Development of new user definition
Benchmarked against several user facilities
New user definition
An individual, not in the EMSL organization, who makes use of the facility as part of an active user proposal in the EMSL Usage System
Onsite
Remote operating equipment
Remote not operating equipment.
Individuals are counted once in a reporting period.

- Counted all proposal participants
- Development of new user definition
- Benchmarked against several user facilities
- New user definition
- An individual, not in the EMSL organization, who makes use of the facility as part of an active user proposal in the EMSL Usage System
- Onsite
- Remote operating equipment
- Remote not operating equipment.
- Individuals are counted once in a reporting period.

**Recommendation:  Define “user” through benchmarking and maintain statistics accordingly**

## Revised survey
Shortened survey (eliminated redundant questions)
Clarified questions
Grouped by category
Send out every 6 months.
Measures
Target 45% response rate 
“Satisfied" or “Very Satisfied" user responses = 90%
Continuous improvement.
FY06 response
48.6% response rate (224 respondents)
91.9% “Satisfied” or “Very Satisfied”

- Shortened survey (eliminated redundant questions)
- Clarified questions
- Grouped by category
- Send out every 6 months.
- Measures
- Target 45% response rate
- “Satisfied" or “Very Satisfied" user responses = 90%
- Continuous improvement.
- FY06 response
- 48.6% response rate (224 respondents)
- 91.9% “Satisfied” or “Very Satisfied”

**Recommendation: Benchmark and revise user survey**

## Recommendation: Establish radiological capabilities

- Exploring options for a radiochemical annex
- Will serve a new user community
- Facility to mitigate safety and contamination concerns
- Address Environmental Remediation Sciences Program objectives
- A “second” use of EMSL equipment from recapitalization effort
- Radiological Nuclear Magnetic Resonance (NMR) Spectrometry Meeting first step in defining the facilities
- Approximately 40 participants from 11 institutions and 6 countries attended
- Emphasized that NMR only part of the answer, other instrumentation would be critical to success.
- General desire to have facilities that allow work with relatively high levels of radioactivity.
- Will include sample preparation, receiving/storage, analytical capabilities as determined via workshops/input for advisory committees and user community.
- Designed to accommodate diverse samples (from biological to subsurface science).

*Radiological NMR Spectrometry Meeting*
*May 1-2, 2006*

## Recommendation: Develop Recapitalization Plan

- Recapitalization Workshop (August 1-2):  Define key scientific challenges in EMSL’s four science themes and the investments appropriate for EMSL as a national scientific user facility
- Eighteen plenary speakers
- Include Gordon Brown (Stanford), Barbara Finlayson-Pitts (University of California Irvine), Mike White (University of Texas), Vicki Grassian (University of Iowa), Larry Kasmerski (National Renewable Energy Laboratory), John Sidles (University of Washington), and Alan Marshal (Florida State University).
- Broad outreach
- Announcement sent to approximately 3000 scientists and others associated with EMSL.
- Excellent response; auditorium full, webcast planned
- 95 participants to date, including 57% from external entities.
- Future science challenges form the framework
- Other considerations include enhanced methods for probing interfacial reactivity, new techniques for material and sample preparation, enhanced capabilities for studying complex dynamical systems, methods for enhanced sample throughput, etc.
- Output: EMSL Recapitalization Plan document.

## Goals of outreach
Attract new users to EMSL (e.g., minority institutions, top universities)
Increase EMSL name recognition and enhanced reputation.
Activities
Broadly distribute user calls and establish relationships with external peer reviewers
Increase EMSL’s presence at national professional society meetings
Attend/host workshops and symposia at regional/national professional meetings
Attend/host user meetings
Actively engage EMSL advisory committee members. 

- Attract new users to EMSL (e.g., minority institutions, top universities)
- Increase EMSL name recognition and enhanced reputation.
- Activities
- Broadly distribute user calls and establish relationships with external peer reviewers
- Increase EMSL’s presence at national professional society meetings
- Attend/host workshops and symposia at regional/national professional meetings
- Attend/host user meetings
- Actively engage EMSL advisory committee members.

**Recommendation: Develop consistent model for ****attracting users**

*AAAS 2006 Annual Meeting*

## Recommendations: Environmental Spectroscopy and Biogeochemistry Facility

- Increase visibility of facility to users
- Enhanced outreach.
- Provide staff scientific growth opportunities
- EMSL Laboratory-Directed Research and Development opportunities
- Provide resources for attendance at professional meetings.
- Increase “in-house” technology development
- EMSL instrument needs in FY07 Small Business Innovative Research call
- Additional capability development funds available
- Increase EMSL’s Instrument Development Laboratory activities.
- Radiological capabilities important to this facility
-  Radiological annex.

**Recommendations: Environmental Spectroscopy and ****Biogeochemistry Facility**

*Metrics – New users, increased proposals*

## High-Field Magnetic Resonance Facility

**High-Field Magnetic Resonance Facility**

- Facility needs to focus on science areas
- Alignment with EMSL strategy and science themes
- No longer submit standalone calls for proposals.
- Seek out users of marquee capabilities across science disciplines
- Attend scientific meetings in science areas other than NMR fields.
- Radiological capabilities would be an enhancement
- Radiological NMR Spectrometry Workshop.

*Metrics – New users, proposals in science themes,*
*publications on marquee instruments, etc.*

## High-Performance Mass Spectrometry Facility

- Shorten sample turnaround times
- EMSL has increased throughput by adding capabilities.
- Reach out beyond the mass spectrometry community
- Attend meetings in other science areas.
- Explore radiological capabilities
- Recapitalization Workshop.

**High-Performance Mass Spectrometry Facility**

*Metrics – New users, increased proposals in science themes, etc*

## Interfacial and Nanoscale Science Facility

- Prioritize and focus activities across the facility
-  Science themes, capability-based calls
-  EMSL Strategic Plan.
- Policy for handling resources with shared ownership
-  EMSL Utilization Policy.
- Ability to handle radiological samples
- Radiological Annex.

**Interfacial and Nanoscale Science Facility**

*Metrics – Science themes proposals, utilization, etc.*

## Molecular Science Computing Facility

**Molecular Science Computing Facility**

- Consider alternative to current one-machine model
- Developing an alternatives analysis.
- Align research activities with EMSL and DOE
- The facility is part of the science theme call
- DOE/EMSL working to ensure full alignment with new proposal round.

*Metrics –Increased proposals, downtime, publications, etc.*

## Chemistry and Physics of Complex Systems Facility

**  **Training of users
- All EMSL facilities will develop training guidelines for instruments where appropriate.
-  Expand user access and outreach
- Set specific goals for new external users
- Provide funding based on response to calls for proposals and outreach efforts
- Ensure openness of facility by external peer-review process
- Ensure that collaborative relationships with PNNL programs are not required.

**Chemistry and Physics of Complex Systems Facility**

*Metrics – New users, external users, increased proposals, etc.*

## Slide 24

## Slide 25

| Tenet | Goal | Objectives | Metric  | FY Target | Q2 YTD Target | Actual YTD | Method used for calculating metric |
| --- | --- | --- | --- | --- | --- | --- | --- |
| High-Impact Science & Marquee Capabilities | Challenging Scientific Problems | Science Themes | Science Theme Proposals | 30% | 0% | 0% | Submitted science themes proposals /total submitted proposals |
|  |  |  | Science Theme Utilization | 20% | 0% | 0% | ST usage on systems/total utilized hours of system usage |
|  |  | High Impact Science | Peer-Reviewed Publications | 372 | 154 | 182 | Count |
|  |  |  | Marquee Research on Marquee Instruments | 3 | 0 | 0 | Count |
|  |  | Grand Challenges | Grand Challenge Utilization | 15% | 6% | 4.3% | GC usage on major systems/ total utilized hours on major systems |
|  | Scientific Forefront | State of the Art Capabilities | Patents | 2 | 1 | 1 | Count |
|  |  | Problem Solving | Utilization of Multiple Systems | 15% | 15% | 16% | # of proposals that use multiple systems / total active proposals |
|  |  |  | Co-Authored Publications | 39% | 37% | 39% | # of EMSL staff co-authored pubs / total pubs |
| Mgmt &    Operations | Optimize Mgmt | Mgmt Systems | Reports Accurate & Timely | 95% | 95% | 100% | Reports delivered on-time and defect free / # reports due |
|  |  |  | EMSL User System & EMSL Resource System Data Integrity Checks | 95% | 0% | 0% | # of data items that were correct / total number of data items checked |
|  |  |  | Performance Management Plan | 100% | 50% | 50% | % complete of project plan |
|  |  | Committees | Advisory Committees | 100% | 30% | 34% | % complete of project plan |
|  |  | Staff Development | Diversity | 75% | 75% | 100% | # of measures successful / # of goals where activity occurred |
|  |  |  | Publications per Science & Engineering Staff | 2 | .80 | .98 | Pubs w/ EMSL S&E staff as author / # of EMSL S&E staff (71) |
|  | Operate with Excellence | Infrastructure | Raised Floor Construction | 10% | 10% Var | 11% | Cost Variance - BCWP - ACWP |
|  |  | Operations | Total Recordable Cases | 0 | 0 | 0 | Count |
|  |  |  | Off Normal Events | 0 | 0 | 0 | Count |
|  |  | Efficient Utilization | Unscheduled Maintenance (Broken) | <10% | < 10% | 3% | Downtime on major systems / available time on major systems |
|  |  |  | Instrument Down Time (# of instruments) | 3 | < 3 exceed 10% | 4 | Count of major systems where downtime / available time > 10% |
| Outreach  &           Service | Attract Best User Research | Select Best Science | Publications per Project | 0.25 | .13 | .19 | Total publications / total active user proposals |
|  |  | User Outreach | Increase Proposal Pool | 265 | 132 | 239 | Count of total submitted user proposals |
|  |  |  | Increase Submitted Proposals from New Participants | 20% | 10% | 23% | # of submitted proposals w/ only new participants / # of submitted proposals |
|  |  |  | Increase Submitted Proposals from Non-PNNL Participants | 30% | 15% | 38% | # of submitted proposals w/ only external (non-PNNL) participants / # of submitted proposals |
|  | User Service | User Satisfaction | Survey Response | 45% | 45% |   | # of survey responses / # of surveys sent |
|  |  |  | User Satisfaction | 90% | 90% satisfied | 94% | # of responses marked satisfied or very satisfied / total number of survey responses |
|  |  | Seamless Systems | Operating Hours | 95% | 95.00% | 100.00% | # of hours open / # of hours planned to be open |

## 2006 BERAC recommendations

- Remove obstacles that** **could hamper EMSL’s ability to be a model user facility.
- Facility Business Plans, Operations Manual, Strategy Hearings
- Build a vision of a “single” EMSL with coordinated, scheduled proposal calls. Continue increasing the user base and monitoring quality user experiences. Reevaluate *user *definition* *for consistency with other user facilities.
- Continue science theme calls
- Establish EMSL-wide capability-based calls
- Continue metrics for tracking responses, user satisfaction, etc.
- Redefine definition of *user* that is compatible with other user facilities.
- Prioritizing investments through science themes must be matched, at some level, with the budget/performance expectations for each facility. BER, Pacific Northwest Site Office (PNSO), and EMSL should agree on mechanisms for assessment (e.g., internal evaluation, external review, EMSL Science Advisory Committee).
- Align EMSL budget allocations and metrics with science themes proposals
- Hold discussions with PNSO, BER on mix of oversight and review.

## 2006 BERAC recommendations (contd)

- EMSL’s mission (i) should not preclude the pursuit of high-level science by its staff; (ii) should mandate that when non-BER funds are requested for equipment, EMSL leadership is involved in such requests early in the process; and (iii) should maximize outreach to all non-BER funding agencies.
- EMSL Operations Manager signs all capital appropriation requests
- Time established for EMSL staff science.
- Ensure that ongoing benchmarking of best practices continues as part of the documented assessment process.
- EMSL’s annual Performance Management Plan

## Accomplishments since our last update

- Shared mission for EMSL
- Renewed and revitalized relationship with DOE and PNSO
- Enhanced and refreshed advisory committees
- Excitement from Office of Science front office on EMSL’s model translating to other user facilities
- Institutionalized policies and practices (e.g., external peer review, instrument utilization, proposal calls)
- Removed from the Office of Science Watch List
- Increased PNNL investment in EMSL as a user facility (e.g., space for capabilities and users, Laboratory-Directed Research and Development investments for staff)
- Project management rigor applied to EMSL.

## Questions?