# Presentation: 224393

## Workshop report,Status and next steps Grid Storage Management WGPeter Kunszt CERN

***2004 September 22***

## Intellectual Property Statement

- The GGF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the GGF Secretariat.
- The GGF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this recommendation. Please address the information to the GGF Executive Director (see contact information at GGF website).

## Content

- Berkeley Workshop report
- SRM interface specification status
- GSM Documents and schedules

## Berkeley Workshop

**Meeting Location: LBNL**
**Sept. 1-2, 2004**

**Olof Barring, CERN**
**Jean-Philippe Baud, CERN**
**James Casey, CERN**
**Peter Kunszt, CERN**
**Andy Kowalski, Jlab**
**Don Petravick, Fermilab**
**Timur Perelmutov, Fermilab**
**Matteo Melani, SLAC**
**Junmin Gu, LBNL**
**Alex Sim, LBNL**
**Arie Shoshani, LBNL**

**http://sdm.lbl.gov/gsm/meetings-040901.html**

## Agenda

- _Wednesday, September 1:_
- 9:00 am - 12:30 pm _Session 1:_ _SRM-Basic_
- 	Presentations and discussions on the desirability and features of an SRM-Basic. This is a concept submitted to GGF in the GSM-WG charter. We will also discuss what features should SRM-Advanced include. The relationship of SRM-Basic and SRM-Advanced to SRM v1.1 and v2.1, and future SRM versions will be discussed as well.
- 10:30 am – Break
- 12:30 pm - 2:00 pm Lunch Break
- 2:00 pm - 5:30 pm _Session 2: Experience with SRM implementations_
- 	Presentations and discussions on implementations of SRMs, including SRM v1.1 and SRM v2.1. Discussion should include: missing features, unnecessary features, enhancement to current specs, features that are hard to implement, etc.
- 3:30 pm - Break
- 5:30 pm - Adjourn

- _Thursday, September 2_:
- 9:00 am - 12:30 pm _Session 3: SRM v3.0_This session is open for discussion of advanced features we currently do not have in SRM v2.1. This includes, but not limited to the interaction of SRMs with Virtual Organization Managers (VOMS), authorization, ACLs, accounting, and storage usage optimization.
- 10:30 am - Break
- 12:30 pm - 2:00 pm Lunch Break
- 2:00 pm - 5:30 pm _Session 4: SRM and WSRF_WSRF is a framework that includes multiple services (such as notification service). Specifying an SRM interface in this framework depends on which services one assumes to use. The issues to be discussed are how to specify generic SRM interfaces, and have specific WSRF interfaces interoperate. We may also discuss the possibility of interoperability of web-services-based SRM interfaces and WSRF-based interfaces.
- 3:30 pm - Break
- 5:30 pm - Adjourn

## Implementation Experience Reports

**Don Petravick, Timur Perelmutov (FNAL)**
- Have a complete implementation of 1.1 over the local storage (2+ flavors).
- Job openings to work on SRM at FNAL.
- Worked on an abstraction layer of the SRM interface that manages all the request queues in a Postgresql database and hooks into the underlying storage through a very simple interface

## Implementation Experience Reports

**Olof Barring (CERN)**
- Have a good experience with SRM 1.1 for CASTOR
- Implemented a GSI plugin for GSOAP to support delegation
- Deployment of a load-balanced SRM at CERN

## Implementation Experience Reports

**Olof Barring (CERN): problems found**
- Interoperability problems:
- Due to problems with the SRM specification
- Due to assumptions in SRM or SOAP implementations
- Due to GSI incompatibilities
- The debugging of GSI incompatibilities is by far the most difficult and time consuming task
- Suggestion to have a reference implementation and set of test suites to facilitate interoperability debugging

## Implementation Experience Reports

**Andy Kowalski (Jefferson Labs): **
- Have moved to SRM v2.1 recently, dropped SRM1 support
- Also have an abstraction layer over the local MSS, similar to FNAL – will check for commonalities
**Alex Sim (Berkeley Labs):**
- SRM 1.1: Java wrapper around our existing HRM/DRM
- For clients to talk to our SRM, for our SRM to talk to other SRMs
- Using FNAL SRM client (srmcp)
- SRM 2.1: will be ready by end of September
- To replace the current working HRM/DRM with v.2.1.1 or latest
- Better client tool in GUI

## Implementation Experience Reports

**Jean-Philippe Baud, James Casey (CERN):**
- Experience report on issues found with SRM 2.1 specification
- Suggestions how to move forward with SRM

## Accounting

**Matteo Melani (SLAC):**
- Why do accounting?
- Resource providers want to perform cost-benefits analysis
- Resource providers wants to improve planning
- Resource providers want better security
- Resource providers want to improve QoS (priorities, debugging...)
- Economic models
- Currently not easy to correlate resource usage to groups, users. Monitoring not enough.
- For SRM: want to keep track of
- Storage usage by groups/people (GB/Day/storage type)
- Network data transfer rate

## Accounting

**Arie Shoshani (LBNL): (see document on webpage)**
- Involve VO on the accounting since its policies apply
- Information needed also on amount of requested (reserved) space and not only on actual space used
- How the VO applies policy is outside of SRM scope but all information should be there so that many policies are possible

## Towards SRM 3

**Peter Kunszt:**
**Suggestion to define not a single advanced interface but several ‘feature sets’**

- Basic is ‘simpler’ to implement
- Better version management
- Can evolve each interface differently
- Can plug modules easier
- Can have different things implemented
- Easier integration

- SRM basic

- SRM advanced
*copy*

- SRM advanced
*naming*

- SRM advanced
*space reservation*

- SRM advanced
*admin*

- SRM advanced
*access control*

- SRM advanced
*proxying*

## Basic/Advanced Features

| SRM feature | Basic | Advanced |
| --- | --- | --- |
| Get/put multiple files | Yes | Yes |
| Lifetime per file (TURL) | Negotiated | Negotiated |
| Reserve space | Implicit | Explicit/Negotiated |
| Find out space assigned | No | Yes |
| SRM service status | Yes | Yes |
| User service status | No | Yes |
| Release file | Yes | Yes |
| Lifetime per space  | No | Negotiated |
| Release space | Implicit | Yes |

## Basic/Advanced Features

| SRM feature | Basic | Advanced |
| --- | --- | --- |
| Multi-file streaming | Optional | Yes |
| Transfer protocol negotiation | Yes | Yes |
| Namespace (directory) support | No  | Yes |
| File metadata discovery | Yes | Yes |
| File and space types | Volatile, permanent | All |
| Get/put from/to remote sites | No | No |
| Copy from/to remote sites | No | Yes, Copy&get,
Put&copy  |

## Feature Groups

- Remote copy
- Namespace management
- Space Management
- Adminstrative status
**Future feature groups**
- Authorization – ACLs
- Accounting

## SRM Basic

- Minimize the requirements for an SRM.
- Very simple to implement (especially for local SRM-basic and not supporting “data streaming”, i.e. no queuing of requests are needed)
- It can respond to requests for files it manages or files that can be put into its system.
- Hope to see more implementations!

## Interface specification status

- Work on SRM 3 specification has started
- Draft is being written
- SRM Interface version 3 is:
- Backward-compatible to previous versions v.2.1.1 and v.1.1. (clients)
- v.3.0 and up consists of a “core” and “advanced” set of  methods.
- The core set will be referred to as SRM Basic.
- The core methods are all included in the advanced implementations, but could be enhanced.
- The advanced methods are grouped into “feature sets”.
- V3.0 includes all the methods of v.2.1.1, with the corrections of ill-defined methods
- New: srmCopyAndGet and srmPutAndCopy
- New: file streaming model.

## Next Steps

- Finish draft and circulate on mailinglist for comments.
- Initial implementations expected soon thereafter.
- One more final round of polishing possible before next GGF
- Submit as GGF document for GGF13