# Presentation: 548836

## CDF Grid Workshop

- Glide CAF over multiple sites

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## GlideCAF overview

- Batch queue

- GlideCAF (Portal)

- Collector

- Main Schedd

- Submitter
- Daemon

- Negotiator

- Glidekeeper
- Daemon

- Glide-in
- Schedd

- Monitoring
- Daemons

- Globus

- Grid Pool

- Globus

- Batch queue

- Grid Pool

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Where is the problem?

- Collector

- Schedd

- Available Batch Slot

- Startd

- Globus

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## We have a solution

- Collector

- Schedd

- Available Batch Slot

- Startd

- Available Batch Slot

- Startd

- Globus

- Available Batch Slot

- Startd

- Globus

- Firewall

- WAN

- TCP

- TCP

- TCP robust
- over WAN

- With GCB, firewalls
- are no longer a problem

- GCB

- Globus

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Why we used UDP in the past?

- Condor comes with UDP by default
- Works very nicely in LANs
- TCP/IP communication is much more expensive
- Head node will scale less on same hardware
- Should not be a problem for CDF numbers(less than 2000 VMs per head on WAN)

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Condor GCB(Generic Connection Brokering) 

- Firewall

- Collector

- Schedd

- Grid Pool

- GCB Broker

- Startd

- Globus

- 4) Schedd address forwarded

- 6) A job can be sent to the startd

- GCB must be on a
- public network

- TCP

- Just a proxy

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Status of GCB 

- GCB distributed with Condor since v6.7.18
- Startd-GCB interactions still have some problems when heavily loaded
- Must deploy many GCBs to keep GCB load low – Annoying but not a show stopper
- Condor team working on the fix
- Being used in NAmCAF

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## What it takes to use GCB?

- Need to install one or more GCB(s)
- Configure glidein

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## What it takes to use GCB?

- Need to install one or more GCB(s)
- A standard Condor daemon
- Need to use a different condor_config (an example in CVS)
- Very little changes needed (machine name)
- GCB should run on its own machineUses lots of IP ports!
- Configure glidein

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## What it takes to use GCB?

- Need to install one or more GCB(s)
- Configure glidein
- Just an option when creating glideins	./create_entry.sh ... -b <GCB-IP> -T 1
- It really is just a Condor configuration switch	# GCB configuration	NET_REMAP_ENABLE = true	NET_REMAP_SERVICE = GCB	NET_REMAP_INAGENT = <GCB-IP>	# enable TCP/IP	UPDATE_COLLECTOR_WITH_TCP = 1

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Network load

- WAN bandwidth much lower than LAN
- Cannot afford to transfer glidein and job tarballs  for every batch slot
- We are pushing for a Squid (HTTP Cache) at each and every Grid site
- Easy for CDF-supported sites (CNAF, Lyon, San Diego, MIT, etc.)
- Trying to make it a standard Grid service
- Not there yet, though

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## The complete picture

- Collector

- Schedd

- Available Batch Slot

- Startd

- Available Batch Slot

- Startd

- Globus

- Available Batch Slot

- Startd

- Globus

- WAN

- Globus

- GCB

- Glide-in
- Schedd

- HTTPd

- HTTPProxy

- HTTPProxy

- HTTPProxy

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006

## Conclusions

- The GlideCAF can be used to gather resources from many sites
- Gliding over WAN does not come for free
- Potentially less scalable
- Need several nodes to host GCBs
-  NAmCAF is the first “production” pool
- Need real users for final validation

- Igor Sfiligoi - Glide CAF over multiple sites - Elba 2006