# Presentation: 439253

## Replica Manager and Replica Location Service

*An Overview*

## Firewall

**Worker Nodes**

**Permanent Storage System**
**at Tier1 Center**

- Michael Ernst

## Replication: Copying ~automatically

- GDMP: Grid Data Mirroring Package
- Born in CMS
- Implements subscription-based replication

- Lots of
- New files

- Replica catalogue

- Michael Ernst

## Data Grid Architecture incl. RLS

- Michael Ernst

## Use Case: Dataset Replication

- User

*replicateDataSet (LDN, SE)*

*query (LDN)*

*LFN []*

*getSFNs (LFN0)*

*SFN0[]*

*reserveSpace (space)*

*getAccessCost (SFN0 [])*

*SFN0Costs []*

*copyFile (SFN0,nSFN0)*

*add (LFN0, nSFN0)*

*notify job finished*

- replicaManagement
-    Service

- metaData
- Service

- replicaLocation
- Service

- replicaOptimization
- Service

- SE:
- storageElement

*Use Case: Dataset Replication*

- Michael Ernst

## . . .

- SRM

- FS

- SRM

- Grid
- FTP

- SRMpreparetoGet( SURL)

- TURL

- Open

- Read

- Read

- Close

**SE**

- LAN

- Grid

- Grid
- Middle
- Ware

- Permanent
- Store

- Worker Nodes

- SRMreserveSpace

- SFNs

- 1.1

- SRMcopy

- 1.2

- 2.1

- 2.2

- 3.1

- 3.2

- 3.4

- GridFTP

- 4.2

- SRMcopy 4.1

- Michael Ernst

- GridFTP protocol for secure, efficient WAN data transfer
- File Transfer service for reliable transfer of files between storage systems
- Replica Location Service (RLS)
- a reliable replication service that provides coordinated, fault-tolerant data movement and RLS updates
- a metadata service containing information that describes logical files
- one or more higher level data management services (master copy management, workflow management, etc.)
- application-oriented data services that implement application-specific semantics and policies such as preferences for replica selection or access

- Michael Ernst

## EDG TB2: Replica Management Services

**“****Reptor”**
**Replica ****Management ****Services**

**Optimization**

**Replica Metadata**

**Subscription**

**Client**

**Replica Location**

**File Transfer**

- Michael Ernst

## Replica Manager Componentsand name changes...

**ERM: EDG Replica Manager client interface and API**
- Entry point for all clients
**ROS: Replication Optimization Service**
- Replica selection based on network metrics (WP7)
**RSH: Replication Storage Handler **(what was GDMP)
- Subscription-based replication
**RLS: Replica Location Service **(replacing replica catalogue)
- Local Replica Catalog services LRC: Logical to Physical file mappings
- Replica Location Index services RLI: index on Logical names
**RMC:Replication Metadata Catalogue**
- Similar to Spitfire with RDBMS backend and specialized schema

- Michael Ernst

## Changes: Replica Catalogue to RLS

- edg-replica-catalogue being phased out
- For Replica Location Service (RLS)
- collaboration with Globus
- Local Replica Catalogs (LRCs) on the SEs hold the actual GUID -> PFN mappings   [GUID is what used to be LFN]
- Replica Location Indices (RLIs) redirect inquiries to LRCs actually having the file
- LRCs are configured to send index updates to any number of RLIs
- Much more scalable architecture
- The lookup time for an entry is independent of the number of catalogs. Tested for up to 108 entries.
- The catalog withstands simultaneous user queries of over 1000 queries or inserts per second.

- Michael Ernst

## Changes: Web services

- Most software has been redesigned to use web services
- Much of the server-side stuff now written in Java
- Retain security: GSI-enabled web-services
- Services have been modified to expose an API in WSDL
- For client programming, the client API libraries are auto-generated from the WSDL
- For command-line, the tools are still there, but now talk to the server using web services.
- What the applications user sees should not have changed as a result of adopting web services!

- Michael Ernst

## Replica Manager Architecture

- Michael Ernst

## File mappings

- Logical

- Physical

- LFN0 / Grid Unique ID

- Michael Ernst

## Replica Catalogue

- LFN -> PFN (1:many)
- One logical file name mapping to many physical instances of the file
- With appropriate utility functions, applications might never need to know the PFN. Use the LFN, and the middleware does the mapping for you in the background.

- Michael Ernst

## LFNs, PFNs, GUIDs

- Due to application requirement from LCG, a couple of changes:

- PFN1, Fermilab

- PFN2, CERN

- PFN3, Bologna

- GUID

- 1223423-ASSDF4-11223-35465464

- LFN1

- LFN2

- LFN3

- Michael Ernst

## Replica location problem

- Replica Location Service:
- maintains information on the physical location of files
- maintains mapping between the logical identifier of data and all its physical instances
- provides access to this information

- Given a unique logical identifier for some given data, determine the physical location of one or more physical instances of this data

- Michael Ernst

## Meta-data storage

- Spitfire meta-data storage
- Two faces
- Spitfire browser
- Spitfire client API
- Spitfire-browser allows a client to use web-browser to view the results of canned queries from a database or make canned inserts into the database.
- Client uses their grid cert embedded in their web browser to authenticate (and then authorize) to the service.

- DB

- Michael Ernst

## RLS Requirements 

- Versioning & read only data
- distinct versions of files can be uniquely identified
- data published to the community are immutable
- Size
- scale to hundreds of replica sites, 50 x 108  LFNs, 500 x 108  PFNs
- Performance
- 200 updates/second, average response time < 10ms
- Security
- knowledge and existence of private data must be protected
- storage system protects integrity of data content
- Consistency
- view of all available PFNs most likely not consistent
- Reliability
- no single point of failure,
- local and global state decoupled,
- failure of remote component does not hinder access to local component

- Michael Ernst

## Giggle Framework

- Giggle: A Framework for Constructing Scalable Replica Location Services
- Joint collaboration between WP2 and Globus
- Independent local state maintained in Local Replica Catalogues  : LRCs
- Unreliable collective state maintained in Replica Location Indices : RLIs
- Soft state maintenance of RLI state
- relaxed consistency in the RLI, full state information in LRC
- Compression of soft states
- compress LFN information based on knowledge of logical collections
- Membership and partitioning information maintenance
- RLS components change over time : failure, new components added
- Service discovery and system policies

- Michael Ernst

## Local Replica Catalogue (LRC)

- Maintains replica information at a single site
- Complete locally consistent record
- Queries across multiple sites not supported
- Maintains mappings between LFNs and PFNs on associated storage systems
- Coordinates  its contents with those of the storage system
- Responds to the following queries:
- Given an LFN, find the set of PFNs associated with that LFN
- Given a PFN, find the set of LFNs associated with that PFN
- Supports authentication and authorisation when processing remote requests
- Periodically sends information about its state to the RLIs

- Michael Ernst

## Replica Location Index (RLI)

- Index structure needed to support queries across multiple sites
-  One or more RLIs to map LFNs to LRCs
- Structure w.r.t LRCs can be freely defined
- redundancy, performance, scalability
- Geographical partitioning – all PFNs of a set of LRCs are indexed
- Namespace partitioning 1 – for load balancing purposes
- Namespace partitioning 2 – only LFNs adhering to a specified pattern are indexed for all LRCs
- Many identical RLIs may be set up for load balancing

- Michael Ernst

## RLS Architecture (1)

- A 2 level RLS layout: The RLIs contain pointers to LRCs only.

- Michael Ernst

## RLS Architecture (2)

- A hierarchical RLS topology: The RLIs point to LRCs and RLIs

- Michael Ernst

## Performance results

- Time to add/create/delete/read an LFN entry
- Number of entries in LRC from 0 -> 1000k
- ~ 15-16 ms , no noticeable increase in time with database size
- Time to perform soft state update
- Increases linearly with the number of entries in the LRC
- ~8 secs for 1000 entries in LRC
- ~10000 secs for 1000k entries in the LRC
- ~1667 queries/sec, 67 updates/second

- Michael Ernst

## Replica Management Architecture

- Michael Ernst

## Replica Management- A Proposal for CMS

- HRM: dCache/Enstore

- Michael Ernst

## Generalizing the Concept: Need to Standardize APIs

- Michael Ernst

## Action Items

- Investigate and understand
- RLS and RLI functionality and their interfaces (APIs, etc.) to
- CMS applications
- Data transport mechanisms according to the “Grid File Access” proposal
- Provide a presentation/paper describing the results
- Depending on the availability/maturity of the RLS/RLI code
- install various components required to provide a prototype service for CMS applications, i.e.
- Reptor, Optor, Giggle, RepMeC ...
- test of functionality and performance
- using application substitutes (simply testing APIs, interfaces etc.)
- understand relationship/test with POOL
- Provide a presentation/paper describing the results

- Michael Ernst