# Presentation: 439246

## The functionality of a Replica Registration Service

**Sept 18, 2003**

**The functionality of a**** Replica Registration Service**

_**Attendees**_
**Michael Haddox-Schatz, JLAB**
**Ann Chervenak, USC/ISI**
**Jean-Philippe Baud, CERN**
**Timur Perelmutov, Fermi**
**Don Petravick, Fermi**
**Doug Olson, LBNL**
**Alex Sim, LBNL**
**Arie Shoshani, LBNL**

## Generalized Replica ManagementService

**Replicate {LFN} ******** site**
**Lookup LFNs – find out which RLS to use**
**Planning: select PFNs / SURLs for accessing files **
**Allocate space**
**Move files robustly**
**Register files**

## Simple Replica Management: focus on three services 

**Dynamic Space Allocation**
**Now space is pre-allocated**
**Future: request space reservation**
**Multi-file replication**
**Managing the entire multi-file request**
**Use file transfer services**
**Guarantee robustness**
**Replica Registration**
**Into one or more catalogs**
**Can register into standard Replica Catalogs**
**Can register into specialized experiment file catalogs**

## Replica ManagementArchitecture

**Replica Management**
**Service**

**Multi-file Replication**
**Service**

**Replica Registration**
**Service**

**Client**

**File Transfer**
**Services**

**Replica/File catalog**
**Services**

## Replica Managementfor STAR Experiment

**Replica Management**
**Service**

**Replica Registration**
**Service**

**Client**

**GridFTP**
**(bbftp)**

**STAR**
**File Catalog**

## Replica Managementfor STAR Experiment

**Replica Registration**
**Service**

**Client**

**GridFTP**
**(bbftp)**

**STAR**
**File Catalog**

**DataMover**

**HRM**

**HRM**

## Generalizing the Concept:Need to Standardize APIs

## Three use cases

**Client generated a bunch of files**
**Call a RepReg service to register to one or more RepCats**
**Client want to replicate a bunch of files (or directory)**
**Call RepReg with different modes**
**Want to delete one or more files**
**Should a RepReg service to cleanup catalogs?**
**---------------------------------------------------------------------------------------**
**Question: how the RepCats chosen?**
**We propose to use the MDS for that**
**Qusetion: should RRS be a separate (external) service**
**Yes, because it is too much to impose on underlying RepCats**
**Yes, because it is a common functionality to all RepCats**
**Yes, because it has value as a separate service**

## Replica ManagementArchitecture

**Replica Management**
**Service**

**Multi-file Replication**
**Service**

**Replica Registration**
**Service**

**Client**

**File Transfer**
**Services**

**Replica/File catalog**
**Services**

**Monitoring &**
**Discovery **
**Service**

- Default
- RepCats

- Find RepCats
- (optional)

## Functionality ofReplica Registration

**Registration modes**
**File-at-a-time registration**
_**Mode 1**_**: register all the files that were replicated successfully, and report which failed.**
_**Mode 2**_**: stop registration process if there is a failure in registration, report which failed.**
**Global registration: after the entire multi-file replication finished**
_**Mode 3**_**: register only if all file replications are successful.**
_**Mode 4**_**: register all files that were successfully replicated, and report which failed, if any.**
**Failures are in terms of registering to a RepCat**
**E.g. RepCat database is full or down**
**Assume – immutable files, LFN-PFN duplication is error**
**Register to any catalogs**
**Standard catalogs – e.g. RLS**
**Specialized catalogs – e.g. experiments file catalogs**
**Issue: need a standard for RepCat interface**
**In the meantime we need have a plug-in type service**

## Functionality ofReplica Registration (Cont’d)

**Queue registration requests**
**Request token **
**Multi-file reg capability**
**Permit multiple registration modes**
**File-a-time (as quickly as you can): mode 1 or mode 2**
**Bulk (do it all or do none => roll back): mode 3 or mode 4**
**Failure: no-response, no space, file-already-registered**
**Permission – security: GSI proxy + user name for now**
**Access control – specify r/w for catalog (limited today, ACLs later)**
**Abort and Roll back registration**
**Commit registration**
**Global reg request (commit implied)**
**Robustness**
**Recover from transient failures**
**Retry per request – with time limit (in seconds)**

## Functionality ofReplica Registration (Cont’d)

**Register to multiple catalogs simultaneously**
**Ability to specify which catalogs to register to**
**Refer to RepCats by URLs**
**Manage multiple registration requests for multiple clients**
**Implementation dependent**

## Functionality ofReplica Registration (Cont’d)

**Asynchronous service**
**Provide dynamic status on registration**
**Success, failed, pending – per file per RepCat**
**Failed: already-in-catalog, ill-formed (keep RepCat error string), not-authorized**
**RepCat not responding – per RepCat**
**RepCat database full – per RepCat**
**Waiting for commit – per request**
**Request timed-out – per request**
**Full Success / partial success (but completed) – per request**
**Provide statistics and failure information**
**Summary per request**
**(History activity information)**

## Un-register

**Problem: want to delete a file and un-register it from all RepCats that know about it.**
**1) Need to find out where file is registered or****2) broadcast to all RepCat in the VO**
**Prefer 2) because we avoid keeping a state of where files are registered**
**Who issues the notification to un-register a file?**
**SRMs – need to have permission**
**Maybe RepCats should check with SRM before removal**
**Should this be a service of RRS?**
**Probably, but not in first version**