# Presentation: 265060

## CDF Grid Workshop	

- CDFSOFT and the Grid

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What do we do now

- NFS distribution to all the worker nodes

**Does not scale!**

**Not an option****in the Grid world!**

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What are our options?

- Stop relying on CDFSOFT
- Install CDFSOFT on each and every Grid site
- Use a scalable mechanism to distribute CDFSOFT

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What are our options?

- Stop relying on CDFSOFT
- Install CDFSOFT on each and every Grid site
- Use a scalable mechanism to distribute CDFSOFT

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Getting rid of CDFSOFT

- Official MC production does not rely on it anymore
- Mostly at least
- There are still a few “setup cdfsoft2” in the scripts that can fail without consequences

- Can be done for standardized code
- Can be done for general analysis?

- I am afraid not
- But I may be wrong.

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What are our options?

- Stop relying on CDFSOFT
- Install CDFSOFT on each and every Grid site
- Use a scalable mechanism to distribute CDFSOFT

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## CDFSOFT on every Grid site

- 100s of Grid sites on each continent

**A lot of work!**

**Not all Grid sites****will give us 10GB**
**for CDFSOFT**

**Consistency****nightmare!**

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What are our options?

- Stop relying on CDFSOFT
- Install CDFSOFT on each and every Grid site
- Use a scalable mechanism to distribute CDFSOFT

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What about AFS?

- AFS is a scalable way to distribute code
- Designed with this in mind
- Does local caching
- AFS is indeed a perfect match
- Used in the CNAF CAF
- Problem solved?

- Unfortunately not!
- Not all worker nodes have an AFS client installed and you cannot install it by yourself

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Other distributed file systems?

- There are a variety of options

**All need to be at least****installed by the root user!**

**We cannot**
**use them**

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Are we at a dead end?

- What else we can do?
- Change the user scripts to load the code from the Net.

- Too much work!
- Too many existing scripts in the wild

- Use a simulator of file systems

- Use parrot

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## What is parrot?

- A file system simulator
- Attaches to the job debug interface and intercepts all the system calls
- I/O operations are translated to appropriate actions
- Everything else is just pass trough
- Can talk different protocols
- The author (D. Thain) is committed to make it work for CDF

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Parrot at a glance

- (POSIX Interface)

- Whole File I/O
- (get/put)

- Integration
- with Castor

- File levelprotocols

- Allocation
- and Mgmt

- Full UNIX
- Semantics

- Name Resolution
- and Security Policies

- (Ptrace trap)

- Wide-Area
- HTTP

- Partial File I/O
- (open,close,read,write, lseek)

- LocalDisk

**Intercepts****I/O calls**

**User level****application**

**Talks several**
**protocols**

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Parrot for CDF

**Submitter**

**Condor**

- Grid
- nodes

**Condor**

- Globus

- Grid
- nodes

**Condor**

**Condor**

- Globus

- Grid
- nodes

**Condor**

**Condor**

- Globus

- Grid
- nodes

**Condor**

**Condor**

**Condor**

- Globus

- CDF
- SOFT

- HTTPd

**Squid**

**Glidekeeper**

**Squid**

**Squid**

**Condor**

**Condor**

**Condor**

**Condor**

- Reuse the HTTPd idea

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Current Parrot status

- First tests mostly successful
- A few bugs found and solved
- Negligible overhead
- <5% overhead observed
- More tests needed
- Igor Sfiligoi did most the testNo more time since moving to CMS
**New forces needed**

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006

## Conclusions

- We have a problem
- Remote mounting of CDFSOFT not an option in the Grid world
- Installing everywhere difficult
- Making all jobs CDFSOFT-free hardly an option
- We may have a solution with Parrot
- Need manpower to test and validate it

- Igor Sfiligoi - CDFSOFT and the Grid - Elba 2006