# Presentation: 189269

## Storage Resource Manager

**Don Petravick**
**Timur Perelmutov**
**Fermi National Accelerator Lab**
- 4/15/04

## SRM Motivation

- Grid Architecture requires Reservation and Scheduling of the Following _**Shared**_ Resources
- Computing Resources
- Network Resources
- _Storage Resources_ (often neglected)
- SRM provides Reservation and Scheduling of the Storage Resources

## SRM Motivation 

- High Energy Physics Collaborations span multiple institutions where
- A large variety of types of Storage Systems exist
- Single Disk / Raid
- Robotic Tape Storage System (Enstore, HPSS)
- Distributed Disk Cache (dCache)
- Hierarchical Storage System (dCache - Enstore)
- Heterogeneous environments and proliferation of custom Mass Storage Systems (MSSs) exist
- User applications often need to access data at multiple institutions on multiple MSSs
- SRM provides Standardized Uniform Access to Heterogeneous Storage

## Access to Multiple MSS

## Uniform Access via SRM

## Storage Classification 

- Storage systems can be classified by:
- Persistence of data
- Permanent
- Temporary
- Data access availability
- Data immediately available
- Data needs advanced reservation before utilization (tapes need to be mounted, files need to migrate to disks, etc.)
- Supported transfer protocols
- File transfer protocols
- POSIX like access protocols
- Need management interface that supports all of the above

## Storage Resource Managers

- SRMs Are Middleware Components That Manage Shared Storage Resources on the Grid and Provide:
- Uniform Access to Heterogeneous Storage
- Protocol Negotiation
- Access to Permanent and Temporary Type of Storage
- Advanced Space and File Reservation
- Reliable Transfer Services

## SRMFile and Space Types

**   ****Volatile                     Durable                    Permanent**

**   ****Volatile                      Durable                    Permanent**

**Space**
**Types**

**File**
**Types**

- Blah

- Blah

- Blah

- Blah

- Volatile

- File Type

- Permanent

- Durable

- Blah

- Blah

- Blah

- Blah

- No

- Archived

- Yes

- No

- Blah

- Blah

- Blah

- Blah

- Yes

- Lifetime

- No

- Yes

- Blah

- Blah

- Blah

- Blah

- Yes

- AutoDeleted

- No

- No

- Property

## SRM Groups of Functions

- SRM interface consists of the following groups of functions:
- Space Management Functions
- Data Transfer Functions
- Directory Functions
- Permission Functions
- Status Functions

## SRM Interface Details

_**Space Management Functions**_
- SrmReserveSpace
- SrmReleaseSpace
- srmUpdateSpace
- srmCompactSpace
- srmGetSpaceMetaData
- srmChangeFileStorageType
- srmGetSpaceToken
_**Directory**_
- SrmMkdir
- srmRmdir
- srmRm
- srmLs
- srmMv

_**Data transfer**__ __**functions**_
- srmPrepareToGet
- SrmPrepareToPut
- srmCopy
- SrmRemoveFiles
- srmReleaseFiles
- srmPutDone
- srmAbortRequest
- srmAbortFiles
- srmSuspendRequest
- srmResumeRequest

_**Status Functions**_
- srmStatusOfGetRequest
- srmStatusOfPutRequest
- srmStatusOfCopyRequest
- srmGetRequestSummary
- srmExtendFileLifeTime
- SrmGetRequestID
_**Permission**_
- srmSetPermission
- srmReassignToUser
- srmCheckPermission

## Srm Example - srmPrepareToPut

- User

- SRM
- Node

- dCache
- HeadNode

- dCache
- PoolNode

- dCache
- DoorNode

## Srm Example - Robust Replication

## Slide 13

## Grid File Access Libraryand SRM

- Physics Application

- Replica Manager

- Client

- SRM

- Client

- Local

- File I/O

- rfio I/O

- open()

- read()

- etc.

- dCap I/O

- open()

- read()

- etc.

- Grid File Access Library (GFAL)

- SRM

- Service

- dCap

- Service

- rfio

- Service

- RM

- Services

- MSS

- Service

- Local

- Disk

- POSIX I/O

- GridFTP

- Service

- Wide Area

- Access

## Role of SRM in Grid

## US-CMS DATAGRID

- Compact Muon solenoid (CMS) is an experiment at the lepton Hadron Collider (LHC) at CERN in Geneva, Switzerland.
- US-CMS is building data grid to facilitate physics data analysis at academic institutions across the united states.
- Data grid 3 tier architecture.
- Tier 0 CERN, Geneva, Switzerland.
- Tier 1 consists of 5 regional Centers, FERMILAB in Batavia, IL is a north American Center.
- Tier 2 consists of 25 Centers, 5 of these are in north America.
- SRM copy is used as a management protocol and reliable replication service for movement of  data from tier 0 to tier 1 and from tier 1 to tier 2.

## CMS SC2003 DEMO

## US-CMS Data Challenge 2004

- Event Reconstruction at the Tier-0 Center at 25Hz for the period of a month
- Transfer of raw and reconstructed data to distributed Tier-1 centers
-  Access of events at distributed center for analysis-type applications

## Client Access to CMS-DATA

- ? (dCache, dFarm, DRM, NeST)

**Client**

**Client**

**Logical**
**Query**

**Request **
**Interpreter**

**Request**
**Executer**

**Logical Files**

**site-specific**
**file requests**

**site-specific **
**files**

**SRM**

**SRM**

**SRM**

**request**
**planning**

**pinning & file**
**transfer requests**

**property**
**index**

**Replica**
**Catalog**

**MDS**

**Site hosting **
**the Application**

**Network**

- Tier1

- Tier2

- dCache

- Enstore

## Resources

- The Storage Resource Manager Collaboration Web Site  http://sdm.lbl.gov/srm-wg/
- The Storage Resource Manager Interface Specification, version 2.1, Edited by Junmin Gu, Alex Sim, Arie Shoshani, LBL, http://sdm.lbl.gov/srm/documents/joint.docs/SRM.spec.v2.1.final.doc.
- Dcache documentation by Patrick Fuhrmann, DESY, Germany, www.dcache.org
- US-CMS http://www.uscms.org
- US-CMS Data Challenge 2004  http://www.uscms.org/s&c/dc04