# Presentation: 878285

## SAM-SRM design / specification outline.

- Baranovski, R. Mathur

- 06/29/06

## Plot

- Apply SAM data handling policies over the generic storage system managed by SRM
- Premise ?
- Similarity in many supported operations :
- transferring file, removing file, building directory listing, and retrieving file metadata
- But there are incompatibilities ...

## SRM outline

- Get file. Translates logical file path into transfer url (location access cookie) compatible with requestor transport.
- Put file. Allocates space and translates file path into location access cookie compatible with requestor transport.
- Release file. Deallocates resources associated with calls 1) and 2)
- Copy file. Duplicate data file between 2 storage elements
- Must be able to create a file path on the storage.
- Must be able to free storage space by removing a file.
- May limit number of concurrent accesses by number that can be less than total number of requests in the system.
- Must allow data read access by credentials not originally used to copy, put, or get a file. An important requirement meant to coordinate and ensure shared storage management.
- Must allow querying for file and directory metadata.

## So , what are the differences ?

- SAM managed disks need no explicit resource allocation outside of the program context.
- A file may implicitly (i.e. without explicit action from the storage user) “disappear” or change its “cost of access” category.
- explicit release op invoked on a resource handle
- SRM differentiate cost of data access on a per file basis.
- Data management is not in the same API and parameter space as actual data read and data write.

## Summary of the idea

- Reuse SAM “cache” disks as the concept to describe SRM resource to SAM.
- SAM disks are the objects that contain memory of files. The same way, reuse disks to keep memory of files requested by SAM via SRM. - uniform data management policies over durable, permanent and volatile storages.
- Loose binding between storage and SAM.
-  Adopt a mechanism to handle uncertain state of the files.
- Make SAM responsible for managing user load of SRMs.
- allocation/deallocation of the SRM handlers, keeping track of timeout conditions, and actual logical to transfer url conversion.

## Types of storage

**Volatile storage**
- may displace its own files
- SAM cache should expect that data delivery/pre-staging history may not be entirely accurate.
- finite size + implicit space cleanup policies.
- advisory explicit file erasure requests from SAM.
**Durable storage**
- Durable storage stores and removes file only by explicit concent.
- Explicitly manage the SRM space by issuing erase requests
**Permanent storage (dCache)**
- Permanent storage never displaces files.
- the srm location must remain registered in the DB reflecting permanency property of the SRM.
- Data access cost of the permanently stored files may change!
- “active” and “passive” request prioritization
- “Active” request prioritization will use SRM to learn of access costs.
- “Passive” request prioritization will use SAM cache. Better cache hit rates while reducing SRM overhead.

## Logical file location -> transfer url mapping.

- SAM dispatches transfer urls not file logical path. ( see slide 4 )
- Interpret logical transfer request to invoke “get or “copy”
-  Analyze request source location
- If local to storage srmGet
- Otherwise use srmCopy
- SAM to SRM request will reference valid SRM location, transfer urls and request id.
- The outcome is cabled to SAM and to the analysis job.

## Location mapping

- How not to overload storage directory structure ?
- Implement location mapping !
- The namespace replication implements common requirements on maximum number of files per location across all supported storage elements.
- How ?
- Append one of the source paths to the "base directory“ at the destination. But do it smart.
- Use the exact source path
- always-increasing path lengths
- Use the most suitable directory structure from the list of source locations!
- Use the path that is similar to the current protocol

## Security model

- Problem ?
- Data must be shared between storage users.
- Common data handling can be enacted on behalf of the experiment group.
- Solution
- SAM will act on behalf of the unique principal to stage and remove experiment data.
- SAM will proxy actual access requests using user’s own principals.
- Ensures that SRM correctly prepares transfer urls in agreement with credentials that are normally presented by the analysis job at a time data is read.
- SAM use MyProxy to delegate user’s credentials to SRM.