# Presentation: 810842

## SAMGrid for CDF

- Computing and Data Handling to Meet CDF Needs
- SAMGrid goals for Summer 2004: Transition to efficient operation, expand resources, changes to enable long term support
- SAMGrid for 2005: Fully enter the world of the Grid to enable CDF to access a vast range of global computing resources

*Rick St. Denis, University of Glasgow*

- Sam and the Grid at CDF

- 14 Sep 2004

## Spokespersons’ Requirements

- International Finance Committee (fiscally)
- FNAL PAC (for its physics merit)

- Maximize physics output @ low Lumi
- L3 output rate: 80 -> 360Hz by 06

- 2004:25% computing outside FNAL

- CDF needs the Grid

- 2005:50% computing outside FNAL

-  for CDF

- Sam and the Grid at CDF

- 14 Sep 2004

## Summer 2004 Goal: Expand Resources, More Efficient Operations

- SAM on (D)CAFs
- Reduce DH operations load: EMAIL/Fair Tape Share
- Pin Datasets Remotely via SAM
- MC Data Import:
- Automate to reduce workload
- Replace DFC with SAM
- 04 Goal was >25% offsite computing load
- Met this goal (35% of CDF collaboration-wide cpu capacity is now available offsite)

- Sam and the Grid at CDF

- 14 Sep 2004

## 1.8 of 5.0 THz is now offsite

- Sam and the Grid at CDF

- 14 Sep 2004

## 2004 Goals: Achievements So Far

- MC Data Import: will be in 5.3.4
- SAM on (D)CAF:
- stress testing/fix bugs: need Beta Testers to do real analysis: used 20% of CAF reading golden Datasets (20TB/Day)
- V6 schema adopted, product depoyment now underway
-  Datasets Pinned and available
- _[http://hexfm1.rutgers.edu/DATA_INFO/sam_data/](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/)_
- DCAF utilization: few high-intensity users so far but no problems in principle
- Provided useful cpu capacity for summer conferences
- Now need next phase of data handling and grid submission

- Sam and the Grid at CDF

- 14 Sep 2004

## Screen Shot of Web pagehttp://hexfm1.rutgers.edu/DATA_INFO/sam_data/

**CDF Datasets on SAM stations**
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-cnaf.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-cnaf.html)__[cnaf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-cnaf.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-fzkka.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-fzkka.html)__[fzkka ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-fzkka.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-knu.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-knu.html)__[knu ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-knu.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-rutgers.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-rutgers.html)__[rutgers ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-rutgers.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-sdsc.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-sdsc.html)__[sdsc ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-sdsc.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-taiwan.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-taiwan.html)__[taiwan ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-taiwan.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-toronto.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-toronto.html)__[toronto ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-toronto.html)_
- _[cdf](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-ttu.html)__[-](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-ttu.html)__[ttu ](http://hexfm1.rutgers.edu/DATA_INFO/sam_data/cdf-ttu.html)_

- Sam and the Grid at CDF

- 14 Sep 2004

## Datasets Stored Locally on cdf-cnaf

- Sam and the Grid at CDF

- 14 Sep 2004

## User Perspective

- Sam and the Grid at CDF

- 14 Sep 2004

## User Perspective: SAM on DCAFs

- Sam and the Grid at CDF

- 14 Sep 2004

## User Perspective: JIM

- Sam and the Grid at CDF

- 14 Sep 2004

## User Perspective: Task Submission & Execution

- CAF Gui/CLI

- Analysis program

- Grid

- Sam and the Grid at CDF

- 14 Sep 2004

## CDF Data Handling: Dcache on CAF

- ALL CDF on CAF
- reads 25TB/Day

- NonGrid Running

- Sam and the Grid at CDF

- 14 Sep 2004

## Total CDF Files To User

- 1000 TB

- 2002

- 2003

*D0:700TB*

- 2004

- Sam and the Grid at CDF

- 14 Sep 2004

## 3-7 TB/Day: Karlsruhe

- 60 processes/3000 files: jpmm0c

- X J/

- Sam and the Grid at CDF

- 14 Sep 2004

## CPU from GridKa (Biggest present off-site SAM user)

- May 1-6: 650
- May 7-17:704
- May 18-27:604
- May 28-31:710
- May total 492,860 cpu hrs, 1THz roughly
- June 1-7: 740, 8-14 780, 15 power out, 16-30 700
- June total 507,360 cpuhrs, 1THz roughly

- Cluster not CDF-exclusive -
- Need Grid to make this
- resource available
- to full CDF collaboration!

- Sam and the Grid at CDF

- 14 Sep 2004

## CDF Grid Strategy: Outlook and Goals

- Currently 35% of CDF collaboration-wide open computing capacity from external resources.
- Utilizes only resources fully controlled by CDF so far: Kerberos/fbsng/CDF Condor dCAF
- SAM used and available on ALL resources
- December 15, 2004: JIM/Grid3-OSG/LCG comparison ends (Mainly MC)
- By end of 2005: 50% of computing resources from external sources, broader use of Grid

- Sam and the Grid at CDF

- 14 Sep 2004

## CPU Growth OK, Disk Growth Slower: Need network and/or use offsite for MC

- July

- Dec

- CPU

- 04

- 04

- CPU Growth OK, Disk Growth Slower: Need network and/or use offsite for MC

- Sam and the Grid at CDF

- 14 Sep 2004

## Conclusions

- CDF making good progress toward providing increased off-site computing and DH capacity.
- Can capture many more resources using Grid to achieve physics mission.
- SAM is working _now_ for CDF and will reduce operational loads, improve user experience.
- To make progress, add new software tools and move to capabilities like those supported for/by the LHC and other global grid efforts.

- Sam and the Grid at CDF

- 14 Sep 2004

## What can 20 duals and 6 TB do?(Example of Physics Datasets)

| Stream | Events | Days | Input Size |
| --- | --- | --- | --- |
| Top,W/Z | 20.5 M | 10.3 | 4.5TB |
| Hadronic B and charm | 156M | 78.3 | 34.2TB |

**Need to transfer 0.6 GB/min or 1 TB/Day**

- Sam and the Grid at CDF

- 14 Sep 2004

## CDF SAM Deployment

- Sam and the Grid at CDF

- 14 Sep 2004

## CDF Events Transferred per Month

- Karlsruhe: 5-10M Evt/Day

- Sam and the Grid at CDF

- 14 Sep 2004

## CDF Files in a Month

- Sam and the Grid at CDF

- 14 Sep 2004

## All CDF Files Moved by SAM

- 300K Files

- 2002

- 2003

*D0: 2.5M files*

- Sam and the Grid at CDF

- 14 Sep 2004

## Scale of CDF Offsite Requirements

|  | THz | %offsite | CPU
Speed | #duals
offsite |
| --- | --- | --- | --- | --- |
| FY04 | 3.7 | 25% | 3GHz | 150 |
| FY05 | 9.0 | 50% | 5GHz | 360 more |
| FY06 | 16.5 | 50% | 8GHz | 220 more |

- 6-7 sites, 100 duals each (or larger
- number of smaller sites), by 2006
-  + equivalent capacity @FNAL

- Sam and the Grid at CDF

- 14 Sep 2004