# Presentation: 527653

## DRM for Tier1 and Tier2 centers 

*Michael Ernst*
*Fermilab*
*February 3, 2003*

## Production Data Flow Today ...

- Tier1 at FNAL

## Typical Tier2 Today ...

- GridFTP, bbcp

- Head/Storage Node

- Worker Nodes

## Tier2 w/dCache

## Tier2 w/dCache

- dCache

**GridFTP**

- dCache

**GridFTP**

- dCache/
- Enstore

**GridFTP**

- CASTOR

- FNAL

- CERN

- UCSD

- Florida, ...

- Catalog

- Catalog

- Catalog

## dCache Placement

- Application (e.g. dccp)

- dCap library

- xxxFTP

- GRID Access Method(s)

- Applications

*dCache*

- Enstore

- OSM

- HSM X

- Local Disk

- PNFS Namespace
- Manager

- Tertiary Storage
- Systems

## Distributed Pool Architecture

- Topic
- Caches

- Tertiary Storage

- Enstore, OSM, ...

- Super Cluster Caches
- (Experiment)

- Central Cache

- Host
- Caches

- Cluster Caches
- (Working Group)

- externally enforced attraction

- destination determined attraction

## What is dCache ?

*dCache* is/provides
- Compact system making access to HSM systems more efficient
- Generic tool for caching, storing and easily accessing peta byte scale datasets distributed among a large set of heterogeneous caching nodes
- Able to run with and without HSM backend (can be used as scalable file store)
- Very flexible HSM Backend
- Supports multiple instances of same HSM or different HSMs within same dCache instance
- Rate Adaption (tape  disk)
- Deferred Write (aggregates small files up to threshold (time, space))
- Staging
- Read Ahead
- dCache allows to relax performance requirements on HSM storage components (e.g. tape drives and robots) w/o overall performance penalty
- Attraction schemes used to optimize data placement
- Load Balancing
- Automated Replication (to prevent hot spots)

## dCache Data Access 

- Local
- dccp (req. mounted pnfs fs), kerberized and non-kerberized
- dcap lib (API)
- dcap Preload lib
- URL-style Addressing (e.g. dccp dcap://door.do.name:port#/pnfs/do.name/path/to/file)
- GridFTP (Server embedded in dCache, client with globus-url-copy)
- Remote
- URL-style Addressing
- GridFTP

## The dCap Library

- Posix open/read/write/close

- dCache System

- Door Node

- Mover Node

- Door Node

- NFS Node

- pnfs

- native FS

- libc.o

- libdcache.so

- Application

- Data

- Namespace Operations

## Current Status

- Access to Mass Storage Systems (dCache/Enstore) at FNAL
- Local
- dccp / encp  (req. mounted pnfs fs)
- URL-style Addressing
- GridFTP (Server embedded in dCache, client with globus-url-copy)
- Remote
- URL-style Addressing
- GridFTP
- in Service today at
- CERN (lxcmsa)
- UCSD (cms-dcache-serv, t2cms0)
- Using kerberized certificates when communicating w/dCache @ FNAL
- requires installation/configuration of
- NMI kx509/KCA bins,libs
- Kerberos bins, libs, krb5.conf
- Mass Storage elsewhere
- dCache up&running at UCSD
- dCache @ CERN w/interface to CASTOR will be next
- Still Missing
- Global Dataset Catalog, Replication Management

## Storage Management

- Evaluating Catalogs and Storage Managers
- There is no LHC Data/Storage Management System, nor a Global Dataset Catalog as of yet
- US CMS is looking into SRB (developed by SDSC)
- SRB servers are now running on US CMS servers at UCSD, Caltech, Fermilab, and at CERN soon

**Distributed Storage Resources**

- (database systems, archival storage systems, file systems, (Grid)FTP, http, ...)

**Application**

**SRB Server**

**MCAT**

- HRM        DB2,     Oracle,      Illustra,   ObjectStore

-          HPSS,           ADSM,   dCache

** ****The Storage Resource Broker is a middleware**
** ****SRB is a Distributed Filesystem**
** ****It virtualises resource access**
** ****It mediates access to distributed heterogeneous resources**
** ****It uses a MetaCATalog to facilitate the brokering**
** ****It integrates data and metadata**

## Global Data Management w/SRB

- ...

- Global Catalog

- Site A

- Site B

- Site N