# Presentation: 506535

## SAM: Tevatron Experiments Using the Grid

- CDF and D0 Need the Grid
- Requirements, the CAF and SAM
- Grid from the User Perspective
- Grid to Meet the Need
- How SAM works
- SAM usage by D0 and CDF
- Near Future: SAMGrid

- Rick St. Denis, University of Glasgow

- Getting Ready for the Grid

- 11 March 2004

## Spokespersons’ Requirements for CDF

- International Finance Committee (fiscally)
- FNAL PAC (for its physics merit)

- Maximize physics output @ low Lumi
- L3 output rate: 80 -> 360Hz by 06

- 50% computing outside FNAL

- CDF needs the Grid

- Getting Ready for the Grid

- 11 March 2004

## Scale of CDF Requirements

|  | THz | %offsite | CPU
Speed | #duals |
| --- | --- | --- | --- | --- |
| FY04 | 3.7 | 25% | 3GHz | 150 |
| FY05 | 9.0 | 50% | 5GHz | +360 |
| FY06 | 16.5 | 50% | 8GHz | +220 |

- 6-7 sites, 100Duals each, by 2006
-  + 700 @FNAL

- Getting Ready for the Grid

- 11 March 2004

## CDF Computing Model

- Develop Analysis on desktop
- Access to all CDF data from anywhere
- Large scale processing on batch clusters
- Submission from anywhere
- interactive tools: ls,top,head/tail/cat
- Output to scratch space or desktop

- Implemented Now with
-  CAF (not Grid standard)

- Exists Now

- Getting Ready for the Grid

- 11 March 2004

## Central Analysis Facility

- CAF is a pile of PC’s with a pile of disks. (1200 processors and 100TB)
- This can be implemented anywhere as dCAF: Decentralized CAF.
- Output of jobs can go to desktop or a scratch area
- Need a password for this: authentication (kerberos).

- Getting Ready for the Grid

- 11 March 2004

## Sequential Access through Metadata

**Metadata:** SAM allows groups of files to be identified into datasets using attributes (metadata) such as production pass version or top quark mass to associate them.
**File Retrieval:** SAM moves files to users as they request them.
**File Storage:** SAM allows output files to be stored with new metadata.

- Getting Ready for the Grid

- 11 March 2004

## Metadata

- File Type:  SAMMC Data File
- File Name:  Bs_conc_4o5_3.root
- File ID:  2494282
- File Size:  530926740 [B]
- File Start Time:  01/29/2004 16:00:00
- File End Time:  01/29/2004 17:00:00
- Application Family:  generator
- Application Version:  1.00
- Description:  BsDspi_phipi MONTE CARLO Dataset 4o5 part 3
- Run Number:  167634

- [sam@nglas08 ~]$ sam get metadata --file=Bs_conc_4o5_3.root

- totalevents = 7290

- Work Group:  cdf

- Node Name:  cdfsam.cnaf.infn.it

- dataset = BsMC-lucchesi_test

- html = http://www.pd.infn.it/~lucchesi

- Getting Ready for the Grid

- 11 March 2004

## Use Cases

- User Level MC Production
- All Users have access
- No data on site -> write to tape at FNAL
- User Level Data Access
- All users have access
- Selected samples automaticaly copied on site

- SAM provides this

- Getting Ready for the Grid

- 11 March 2004

## Functionality

- User selects a place to run, saying what dataset they will use
- System checks they can do this (privileges)
- User access to data at any place
- User output is stored on any disk or back to tape at FNAL and results are made available for transfer to any site for others to analyse.

- Getting Ready for the Grid

- 11 March 2004

## User Perspective

- CAF Gui/CLI

- Analysis program

- Grid

- Toronto

- Korea

- Italy

- Taiwan

- FermiCAF

- UK

- CAF Gui/CLI

- Only Fermilab

- Uses SAM

- Outside Lab

- Grid

- Uses SAM

- Uses SAM

- Getting Ready for the Grid

- 11 March 2004

## Meeting the Needs

- SAM: How it works
- Progress in SAM
- CDFGridWorkshop: “Nerd’s Paradise”
- D0 and CDF Usage

- Getting Ready for the Grid

- 11 March 2004

## Fcdfdata016

- Disk/Cache

- Station
- central-analysis
- Daemon
- (smaster)

- Stager
- Daemon
- (stagerng)

- FSS
- (Deamon)
- (fss)

- Stager
- Daemon
- (stagerng)

- Disk/Cache

- Disk/Cache

- Disk/Cache

- Stager
- Daemon
- (stagerng)

- Stager
- Daemon
- (stagerng)

- Stager
- Daemon
- (stagerng)

- Getting Ready for the Grid

- 11 March 2004

## A Farm: Station with Stagers and Caches

- Cache

- Node2

- Cache

- Node3

- Cache

- Node4

- Cache

- Node5

- Cache

- Station
- smaster

- Stager
- stagerng

- Stager
- stagerng

- Stager
- stagerng

- Stager
- stagerng

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## What can 20 duals and 6 TB do?

| Stream | Events | Days | Input Size |
| --- | --- | --- | --- |
| Top,W/Z | 20.5 M | 10.3 | 4.5TB |
| Hadronic B and charm | 156M | 78.3 | 34.2TB |

- Need to transfer 0.6 GB/min or 1 TB/Day

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Disks/Cache

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## sam dump station --disks

- <nglas08> sam dump station --disks
- *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 53 minutes 25 seconds, admins: jozwiak terekhov
- Known batch systems: lsf
- Default batch system: lsf
- No replica selection criteria
- There are 0 authorized transfer groups
- Minimum delivery is 1KB; external deliveries are unconstrained
- STATION DISKS:
- disk 7844 nglas08.fnal.gov:/sam/test9/jozwiak/dev/chris, 29947KB/20GB free
- disk 8064 nglas08.fnal.gov:/sam/test10/jozwiak/dev/chris, 93110KB/20GB free
- *** END OF STATION DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## sam dump station --groups

- *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 57 minutes 3 seconds, admins: jozwiak terekhov
- Known batch systems: lsf
- Default batch system: lsf
- No replica selection criteria
- There are 0 authorized transfer groups
- Minimum delivery is 1KB; external deliveries are unconstrained
- AUTHORIZED GROUPS:
-  group test: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/150, disk: 13054803KB/40GB, locks:0B/0KB
-  group test1: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/40, disk: 1714466KB/30GB, locks:0B/0KB
-  group test2: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/50, disk: 7170234KB/40GB, locks:0B/0KB
- *** END OF STATION DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## sam dump station --projects

- *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 4 minutes 49 seconds, admins: jozwiak terekhov
- Known batch systems: lsf
- Default batch system: lsf
- No replica selection criteria
- There are 0 authorized transfer groups
- Minimum delivery is 1KB; external deliveries are unconstrained
- PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited
- NO PROJECTS
- *** END OF STATION DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>sam submit
- --script=userscript
- --group=groupname
- --cpu-per-event=
- --defname=

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- >>>>>> Starting project with
- the Station Master
- Station Master contacted, result:
-  Started project 49008
- (49008_sam_) for group test
- Waiting for the project to
- initialize...

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Callback from server:
- 'OK|Project is ready'

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- >>>>>> Submitting the job to
- the batch system.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- Getting Ready for the Grid

- 11 March 2004

## Sam dump station --projects

- *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 12 minutes 44 seconds, admins: jozwiak terekhov
- Known batch systems: lsf
- Default batch system: lsf
- No replica selection criteria
- There are 0 authorized transfer groups
- Minimum delivery is 1KB; external deliveries are unconstrained
- PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited
- STATION PROJECTS:
- project 49205_sam_(49205) user jozwiak.test started 01 Nov 14:08:45 UNIX pid 158400787 still wants/currently uses 5/0 files
- *** END OF STATION DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## sam dump project –project=<project name>

- *** BEGIN GPM DUMP ***
- Input files: 1003853..1011900
-   1003853: sim.ztautau.1000evts.017-1442-c5.01, size=0K, unbuffered yet
-   1003854: sim.ztautau.1000evts.017-1442-c5.02, size=0K, unbuffered yet
-   1011651: sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.267_1553, size=0K, unbuffered
-   1011900: sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.276_1152, size=0K, unbuffered
- Cached (not buffered) files: (none)
- Buffered files: (none)
- External files with delivery problems: (none)
- Umer contexts (name, state, join time, nSeen): (no umers)
- Proc contexts (ID: name, state, join time [, current|last]): (no procs)
- Processes waiting for call back:(none)
- *** END GPM DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- Optimizer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- eworker

- eworker

- eworker

- eworker

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- eworker

- eworker

- eworker

- eworker

- encp

- encp

- encp

- encp

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- eworker

- eworker

- eworker

- eworker

- encp

- encp

- encp

- encp

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- eworker

- eworker

- eworker

- eworker

- encp

- encp

- encp

- encp

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user> PSUSP

- eworker

- eworker

- eworker

- eworker

- encp

- encp

- encp

- encp

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  PSUSP

- eworker

- eworker

- encp

- encp

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## SAMManager:sam  Getting next input file...

- SAMManager:sam  Project master will call back.

- Getting Ready for the Grid

- 11 March 2004

## Sam dump project –project=49225_sam_

- *** BEGIN GPM DUMP ***
- Input files: 1099393..1099756
-   1099417: d0g.test_file_1G_a_dev.0001_001, size=0K, unbuffered yet
-   1099418: d0g.test_file_1G_a_dev.0002_001, size=0K, unbuffered yet
- Cached (not buffered) files: (none)
- Buffered files: (none)
- External files with delivery problems: (none)
- Umer contexts (name, state, join time, nSeen):
-   36422: jozwiak(test-harness:1), active, 05 Nov 13:59:09, 31
- Proc contexts (ID: name, state, join time [, current|last]):
-   144663: jozwiak(test-harness:1)@nglas08, wait, 05 Nov 13:59:10, 1099415
- Processes waiting for call back:
-   CID=36422: 144663@nglas08.fnal.gov:11872 (05 Nov 20:53:43)
- *** END GPM DUMP ***

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- eworker

- eworker

- encp

- encp

- Enstore

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- rm

- rm

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Optimizer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- eworker

- eworker

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- eworker

- eworker

- rcp

- rcp

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- eworker

- eworker

- rcp

- rcp

- Other Cache

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- eworker

- eworker

- rcp

- rcp

- Other Cache

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- eworker

- eworker

- rcp

- rcp

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>
- Job <52554> is submitted to
- queue <sam_lo>.

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)
- 52554 <user>  RUN

- samscript.sh

- userscript

- consumer

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Project
- pmaster

- Batch (LSF)

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Batch (LSF)

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>sam submit....
- <fcdfdata016>sam submit....
- <fcdfdata016>sam run project...

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Batch (LSF)

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Batch (LSF)
- 52668 <user1> RUN
- 52675 <user2> RUN
- 52756 <user3> PSUSP

- Project
- pmaster

- Project
- pmaster

- Project
- pmaster

- samscript.sh

- samscript.sh

- userscript

- userscript

- consumer

- consumer

- eworker

- eworker

- rcp

- encp

- Other Cache

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Station
- central-analysis
- smaster

- Disks/Cache

- Stager
- stagerng

- Batch (LSF)
- 52668 <user1> RUN
- 52675 <user2> RUN
- 52756 <user3> PSUSP

- Project
- pmaster

- Project
- pmaster

- Project
- pmaster

- samscript.sh

- samscript.sh

- userscript

- userscript

- consumer

- consumer

- eworker

- eworker

- rcp

- encp

- Other Cache

- Enstore

- Getting Ready for the Grid

- 11 March 2004

## SAM Animation

- _[worldScenerio](file:///worldScenerio.html)__[.html](file:///worldScenerio.html)_

- Getting Ready for the Grid

- 11 March 2004

## Storing Files

- Getting things to tape from Glasgow

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>

- Disks

- FSS
- Central-analysis
- fss

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## Sam dump fss

- FSS version v3_2_2 at station central-analysis running on fcdfdata016.fnal.gov 6 hours 57 minutes 34 seconds
- No routing (all transfers are direct)
- Configuration for operation retrial (count, interval/timeout)
- DBS contact: 3, 1 hours
- Opter contact: 1, 1 hours
- Authorization receipt:1, 1 hours
- Stager contact: 1, 1 hours
- Transfer (retrials upon timeout and upon failure): 3, 6 hours
- Relay (multi-stage routing only): 3, 1 hours
- File Storage Server Dump:
- Stagers are known at nodes: fcdfdata016.fnal.gov
- No requests ever submitted

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>sam store
- descrip.py
- --source=<file loc>
- [--dest=/pnfs.....]

- Disks

- FSS
- Central-analysis
- fss

- Stager
- stagerng

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>sam store
- descrip.py
- --source=<file loc>
- [--dest=/pnfs.....]

- Disks

- FSS
- Central-analysis
- fss

- Stager
- stagerng

- Descrip.py

- Metadata
- Info about file

- Sam checks info,
- checks location,

- Getting Ready for the Grid

- 11 March 2004

## fcdfdata016

- <fcdfdata016>sam store
- descrip.py
- --source=<file loc>
- [--dest=/pnfs.....]

- Disks

- FSS
- Central-analysis
- fss

- Stager
- stagerng

- eworker

- encp, rcp,
- bbftp

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From Really
- Far Away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- Routing:
- fcdfdata016

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- sam store      enstore

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- Routing:
- fcdfdata016

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- eworker

- bbftp         fcdfdata016

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- eworker

- bbftp         fcdfdata016

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- eworker

- encp

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- eworker

- encp

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- rm

- Getting Ready for the Grid

- 11 March 2004

## Enstore

- Far Away

- Disk

- Fss
- From really
- Far away

- Stager

- fcdfdata016

- Fss
- central-analysis

- Stager

- Tmp Disk

- Getting Ready for the Grid

- 11 March 2004

## D0 Sam

- D0 relies entirely on SAM for analysis

- Getting Ready for the Grid

- 11 March 2004

## D0

- Getting Ready for the Grid

- 11 March 2004

## D0 Files

- 4000-8000 Files/Day

- Getting Ready for the Grid

- 11 March 2004

## D0 Data Volume

- 1TB-3TB/day

- Getting Ready for the Grid

- 11 March 2004

## D0 Files Per Month By Year

- 1999

- 2000

- 2001

- 2002

- 2003

- 100,000 files

- Run II Start

- Getting Ready for the Grid

- 11 March 2004

## D0 Total Files

- 2.5Million Files Served

- Getting Ready for the Grid

- 11 March 2004

## D0 Data Per Month By Year

- 50 TB per month

- 1999

- 2000

- 2001

- 2002

- 2003

- Run II Start

- Getting Ready for the Grid

- 11 March 2004

## D0 Total Data Moved

- 700TB moved

- Getting Ready for the Grid

- 11 March 2004

## Progress in SAM: CDF

- All 800,000 CDF data files are in SAM
- Sam is in beta testing on the CDF CAF (1200 cpus): passed 20TB/Day delivery
- Karlsruhe uses SAM routinely
- Minos uses SAM for its Data Handling
- Steve Mrenna (Phenomenology) depositing ALPGEN files in SAM for common CDF/D0 use.

- Getting Ready for the Grid

- 11 March 2004

## Florida workshop:

- 11 installations in about 2 hours. Integrated with dCAF in 2 cases in 2 days.
- 3 in Asia, 4 in Europe
- 6 sites committed to summer 2004 usage of their facilities for all of CDF (mostly MC)
- Sam installation now: initsam cdf <stationname>
- Follow-up on April 1.
- Each site has a local user support person to reduce load on core development team.
- Generally: Security ate 80% of the effort!

- Now 20!

- Getting Ready for the Grid

- 11 March 2004

## CDF

- Getting Ready for the Grid

- 11 March 2004

## Florida Workshop: After 2 Days

- Getting Ready for the Grid

- 11 March 2004

## 2TB/Day: Karlsruhe

- Getting Ready for the Grid

- 11 March 2004

## CDF Dcache on CAF

- ALL CDF on CAF
- reads 25TB/Day

- NonGrid Running

- Getting Ready for the Grid

- 11 March 2004

## CDF Files in a Month

- Karlsruhe: 1500 files/Day

- Getting Ready for the Grid

- 11 March 2004

## CDF Events Transfer in a Month

- Karlsruhe: 5-10M Evt/Day

- Getting Ready for the Grid

- 11 March 2004

## All CDF Files Moved by SAM

- 300K Files

- 2002

- 2003

- D0: 2.5M files

- Getting Ready for the Grid

- 11 March 2004

## Total CDF Data Moved

- 200 TB

- 2002

- 2003

- D0:700TB

- Getting Ready for the Grid

- 11 March 2004

## Advantage of Local Processing

- Karlsruhe processes 2TB/day.  Rest of CDF on Central Cluster processes 25TB/day.
- 5 users actively at Karlsruhe: 100 for rest of CDF: getting 1.6 X the resources.
- They pin the datasets that are of interest and when new ones come, copy them in automatically.

- Getting Ready for the Grid

- 11 March 2004

## Dcache and SAM

- Dcache shapes traffic into disk: If a SAM cache is large, need to use Dcache instead of nfs mounts
- Dcache gives the user what is requested.  1TB gets same priority as 1GB: CDF users must send email requesting data to be staged.
- SAM examines consumption rate before staging next files – No EMAIL needed.
- SAM uses Dcache for its Caching at FNAL.
- This needs further work with SRM

- Getting Ready for the Grid

- 11 March 2004

## In the near term future:JIM

- Adding Grid Standard Tools

- Getting Ready for the Grid

- 11 March 2004

## CDF Grid Strategy

- 25% of CDF Computing from external resources.  All CDF computing on CDF Grid by April 15: Utilize resources fully controlled by CDF: Kerberos/fbsng: dCAF + SAM
- October 15, 2004: JIM to capture shared resources
- June 2005: 50% of Computing resources external

- Getting Ready for the Grid

- 11 March 2004

## Desktop

- Anywhere

- Condor
- Submitter

- @regional centers

- SAM DB
- Condor
- Matchmaker

- @FNAL

- Globus GK
- CAF Submitter
- SAM Station

- @ each site

- WN

- Private LAN

- Private LAN

- dCache

- June 2004
- testing

- June 2005
- required

- Simple JIM

- Getting Ready for the Grid

- 11 March 2004

## Detailed JIM

- Getting Ready for the Grid

- 11 March 2004

## Conclusions

- CDF has embraced the need for the Grid to achieve its physics mission
- SAM is working for D0 and growing in CDF

- Getting Ready for the Grid

- 11 March 2004