# Presentation: 794723

## A Sneak Peak of What’s New in Globus GridFTP

- John Bresnahan
- Michael Link
- Raj Kettimuthu (Presenting)
- Argonne National Laboratory and
- The University of Chicago

## GridFTP

- A secure, robust, fast, efficient, standards based, widely accepted data transfer protocol
- We supply a reference implementation:
- Server
- Client tools (globus-url-copy)
- Development Libraries
- Independent implementations interoperate
- Fermi Lab has a home grown server that work with ours
- Lots of people have developed clients independent of the Globus Project

## GridFTP

- Two channel protocol like FTP
- Control Channel
- Communication link (TCP) over which commands and responses flow
- Low bandwidth; encrypted and integrity protected by default
- Data Channel
- Communication link(s) over which the actual data of interest flows
- High Bandwidth; authenticated by default; encryption and integrity protection optional

## GridFTP

- DPI

- SPI

- DPI

- SPI

- CPI

## Striping

- GridFTP offers a powerful feature called striped transfers (cluster-to-cluster transfers)

## Topics for discussion

- Performance enhancement
- GridFTP over UDT
- Ease of Use enhancements
- GridFTP over SSH
- GridFTP Where there’s FTP
- Resource Management in GridFTP
- Future directions

## GridFTP over UDT

- UDT is an application-level data transport protocol that uses UDP to transfer data
- Implement its own reliability and congestion control mechanisms
- Achieves good performance on high-bandwidth, high-delay networks where TCP has significant limitations
- GridFTP uses Globus XIO interface to invoke network I/O operations

## GridFTP over UDT

- XIO framework presents a standard open/close/read/write interface to many different protocol implementations
- including TCP, UDP, HTTP -- and now UDT
- The protocol implementations are called drivers.
- A driver can be dynamically loaded and stacked by any Globus XIO application.
- Created an XIO driver for UDT reference implementation
- Enabled GridFTP to use it as an alternate transport protocol

## GridFTP over UDT

|                   
 | Argonne to NZ Throughput in Mbit/s | Argonne to LA Throughput in Mbit/s |
| --- | --- | --- |
| Iperf – 1 stream | 19.7 | 74.5 |
| Iperf – 8 streams | 40.3 | 117.0 |
| GridFTP mem TCP – 1 stream | 16.4 | 63.8 |
| GridFTP mem TCP – 8 streams | 40.2 | 112.6 |
| GridFTP disk TCP – 1 stream | 16.3 | 59.6 |
| GridFTP disk TCP – 8 streams | 37.4 | 102.4 |
| GridFTP mem UDT  | 179.3 | 396.6 |
| GridFTP disk UDT | 178.6 | 428.3 |
| UDT mem | 201.6 | 432.5 |
| UDT disk | 162.5 | 230.0 |

## Alternate security mechanism

- GridFTP traditionally uses GSI for establishing secure connections
- In some situations, preferable to use SSH security mechanism
- Leverages the fact that an SSH client can remotely execute programs by forming a secure connection with SSHD

## GridFTP over SSH

- sshd acts similar to inetd
- control channel is routed over ssh
- globus-url-copy *popens* ssh
- ssh authenicates with sshd
- ssh/sshd remotely starts the GridFTP server as user
- stdin/out becomes the control channel

## SSHFTP Interactions

- sshd

- CPI

- GridFTP
- Server

- 2811

- Port 22

- ROOT

- exec

**USER**

- ssh

- popen

- Authenticate

- Stdin/out

## GridFTP Where there’s FTP (GWFTP)

- GridFTP has been in existence for some time and has proven to be quite robust and useful
- Only few GridFTP clients available
- FTP has innumerable clients
- GUI Clients?
- Windows Clients?

## GWFTP

- GWFTP - created to leverage the FTP clients
- A proxy between FTP clients and GridFTP servers
- Not secure from client to proxy
- Run on a trusted net (127.0.0.1)
- Data channel routed or direct
- If 3pt it is direct and secure
- If 2 party must route through proxy, or be insecure

## GWFTP (3pt)

- DPI

- SPI

- DPI

- SPI

- Your
- Client

- FTP 959
-  (not secure)

- GSI Credential

- GSI Delegated Credential

- gwtftp

- GSI Credential

## GWFTP (2pt routed)

- DPI

- SPI

- Your
- Client

- FTP 959
-  (not secure)

- GSI Credential

- gwtftp

- GSI Credential

- DPI

## GWFTP (2pt direct)

- DPI

- SPI

- Your
- Client

- FTP 959
-  (not secure)

- No Security

- gwtftp

- GSI Credential

- DPI

## Resource management

- Fork/Exec is safer service model
- sandboxes leaks/segfaults/security/etc
- If 1 session dies service exists
- Transient state
- We need permanent & shared state between sessions

## GFork

- Server Host

- Server

- GridFTP
- Plugin

- GridFTP Server
- Instance

- GridFTP Server
- Instance

- GridFTP Server
- Instance

- State Sharing  Link

- Client

- Inherited
- Links

- Control Channel Connections

- Client

- Client

## Dynamic Backends

- Dynamic list of available backends (DPIs)
- Frontend (SPI) listens for registration
- Backends register (and timeout)
- Select backend(s) to use for a transfer
- Backend failure is not system failure
- Resources can be provisioned to suit load

## Dynamic Backends

- Frontend Host

- GFork
- Server

- GridFTP
- Plugin

- Frontend
- Instance

- Lookup available backend

- Backend Host

- GFork
- Server

- GridFTP
- Plugin

- Backend
- Instance

## Future directions

- Resource Properties
- GridFTP server expose state via resource properties
- Server load
- Connection limits
- Act as a WS-MDS provider
- Firewall traversal
- Simultaneous open
- Capability to make use of dynamic firewall port opening