# Presentation: 785287

## Report of the BERAC ARM Facility Review Panel

- Joyce E. Penner, Eugene W. Bierly, Robert Dickinson, Charles Miller1, Nelson A. Seaman1, Anne-Marie Schmoltner2, Paul Try2
- 1Non-ARM funded
- 2 Participated in 2005  review

## Charge: Assess the effectiveness of the ARM facilities as a national user facility

- How well are they serving the needs of the open (non ARM-funded) user community?
- What factors are driving the costs of maintaining and operating the facilities?
- Were critical recommendations from the 2005 review implemented?

## ARM Background

- ARM program funds ARM research and ARM infrastructure (sites and services)
- 2004:  ARM Climate Research Facility designated as a national user facility
- ACRF Science Board reviews requests for field programs at the fixed sites and use of the Mobile Facility
- ACRF does not fund research, but provides instrumentation, data, and expertise

## ARM as a User Facility: External Users

- The climate modeling community gains from knowledge generated by the data taken at ARM: Cloud/aerosol interactions; Convective precipitation
- Non-ARM funded direct use of the facility:
- Mobile facility: 3 out of last 4 users were not ARM-funded
- 60% of accepted proposals are non ARM-funded P.I.’s
- 1500 users; 931 use archive, 69% are non-funded
- Several satellite validation exercises benefit NASA, NOAA

## ARM as a User Facility: Possible extensions

- Initiate programs to train new scientists (advanced graduate students and beginning post-docs) in the use of the ACRF
- Initiate a process of outside review of the documentation of data provided in the archive

## Cost Effectiveness of the ARM facility: Metrics

- Metrics for cost effectiveness useful for DOE and OMB, but don’t tell the whole story
- Scientific excellence metrics useful to capture the impact of the ACRF on the national and international research community.

## Cost Metrics

**Cost/Product:**  goal is $100 per product file (counted as the highest level product) (has trended down, but leveled off)
**Leveraged Science Costs**: Since external users not coming out of ARM, a measure of the costs of these campaigns would show broader impact

## Science Metrics

**Uptime** (98%) or Number of Instrument Operation Hours
**Number of Publications** Citing ARM or ARM Data: 883 during 2002-2006
**Number of Proposals Received/Number of Proposals Approved** (68%); would be nice to have stratified by cost
- Only 17% of Mobile Facility requests approved: Strong unaccomodated desire
**User and Data Use Statistics**: 931 data users: need better feedback about available documentation and need error bars on products

## Cost effectiveness and trends

- Largest cost factor is labor:
- Have worked to keep this cost category flat and/or declining: Savings of $6M compared to projections using 3% inflation
- Have also trained staff to maintain instruments (cost savings relative to contracting this out)
- Cost saving in internet services through use of higher technology

## ACRF is approaching a period of optimal productivity

- Risk that ARM Infrastructure will not be able to sustain continued growth in service to the user community under the current funding profile
- Scope of ACRF responsibilities has increased with their support of climate data records: Must emphasize data quality

## Implementation of recommendations from prior review (2005)

- Management team established a formal process to address each recommendation:
- Emphasis was given to:
-  improvement of data quality,
- documentation and dissemination,
- enhancement of communications to the user community

## Web Technology, Data Products and Information

- Massive improvement to the ARM web site and organization of data
- A more pro-active approach to feedback from users would help
- Data quality especially error characteristics could be made more accessible
- Linking and cross-cutting of data sets in time could be improved
- Grouping of data sets for particular problems has started and should continue

## Other recommendations

- Revival of the Instrument Development Program is probably not necessary
- The criteria for deployment is certainly better enunciated especially with the Mobile Facility
- Constructive outreach is being pursued:
- A limited number (one to three) of signature data sets should be advertised
- More focused attention could be given to applications of the world-wide, regionally diverse, collective data from the ACRF sites to global climate research
- Communication of ACRF Accomplishments needs to continue

## Other recommendations

- Data quality efforts have been expanded, though no easy way to track the improvements
- Management/Leadership approaches appear to be evolving to remain effective though we did not spend effort on this in the review

## Conclusions

- ACRF is effectively used by the broader scientific community
- The ACRF Management has worked aggressively to decrease costs of running the ARM facility: can widen metrics to better quantify the value of the facility
- ACRF management also aggressively pursued implementation of the 2005 review recommendations, though we had a few suggestions for further improvement