# Presentation: 324673

## Probe Plans and StatusSciDAC KickoffJuly, 2001

- Dan Million
- Randy Burris
- ORNL, Center for Computational Sciences

## Overview

- Proposal’s plans for Probe involvement
- HPSS overview
- Probe as a “place to be”
- SciDAC applications with Probe connections
- Technology on hand

## Plans as stated in the proposal

- Five research thrusts
- High-bandwidth transfers of very large datasets
- Optimize blocksizes and configurations
- TCP-friendly UDP service
- Test AIX 5L for improvements in TCP/IP stack
- Evaluate ATM’s effect on TCP Performance
- Web100
- Enhancements to HSI (Hierarchical Storage Interface)
- Evaluate Gigabyte System Network and trunked Ethernet
- Investigate new protocols (ST, Storage over IP)
- Adapt and utilize simulation model of HPSS
- Provide “place to be” – testbed for other SDM ETC activities

## HPSS overview

- High Performance Storage System
- Targets petabyte capacities and gigabyte/second transfer rates
- Core servers manage metadata and control movers
- Movers communicate over data network and move data from network to storage and vice versa
- Distributed architecture – core servers and movers can be replicated and run on hundreds of nodes

## HPSS overview (continued)

- ORNL is a development partner
- Can establish HPSS systems without limit (4 now)
- Production
- HPSS/AIX, most current version
- HPSS/Sun, next version
- HPSS/AIX, next version
- Can modify HPSS itself

## Probe – “Place to be”Overview of ORNL Probe Cell, June 2001

- Stingray
- RS/6000 S80

- Marlin
- RS/6000
- H70

- STK Silo

- 300 GB SCSI
- RAID Disks

- Sun
- E250

- Compaq
- DS20

- 360 GB Sun
- FibreChannel
- Disks

- 360 GB STK
- FibreChannel
- Disks

- FibreChannel Switch

- GSN Switch

**Origin 2000**
**Reality Monster**

- RS/6000
- B80

- External
- Esnet Router

**To NERSC Probe**

- Sun
- Ultra 10

- STK Silo

**IBM SP**

**Compaq Sierra**

- 3494 Library

- GSN Bridge

- RS/6000
- 44P-170

- Probe
- Shared with Production

- Intel
- Dual Xeon
- Linux

- Sun
- E450

- IBM
- F50

- SGI
- Origin
- 200

- SN6000

- Gigabit Ethernet

## ORNL Probe Cell, June 2001 – equipment for agent infrastructure

- Stingray
- RS/6000 S80

- Marlin
- RS/6000
- H70

**Origin 2000**
**Reality Monster**

- RS/6000
- B80

- External
- Esnet Router

**To NERSC Probe**

**IBM SP**

**Compaq Sierra**

- Probe
- Shared with Production

- Intel
- Dual Xeon
- Linux

- Sun
- E450

- Gigabit Ethernet
- Jumbo

**Agents might run on any machine; variety of platforms shown.**
**CPU intensive agents might run on Stingray or  some supercomputer**.

- Sun
- E250

- Compaq
- DS20

- IBM
- F50

- SGI
- Origin
- 200

## ORNL Probe Cell, June 2001 – equipment for file-transfer work

- Stingray
- RS/6000 S80

- Marlin
- RS/6000
- H70

- STK Silo

- 300 GB SCSI
- RAID Disks

- 360 GB Sun
- FibreChannel
- Disks

- GSN Switch

**Origin 2000**
**Reality Monster**

- RS/6000
- B80

- External
- Esnet Router

**To NERSC Probe**

- Gigabit Ethernet
- Jumbo

- STK Silo

**IBM SP**

**Compaq Sierra**

- 3494 Library

- GSN Bridge

- Probe
- Shared with Production

- Intel
- Dual Xeon
- Linux

- Sun
- E450

- SN6000

**Variety of hosts**
**Variety of storage**
**    ****equipment**
**Variety of links**

- Sun
- E250

- Compaq
- DS20

- 360 GB STK
- FibreChannel
- Disks

- FibreChannel Switch

- IBM
- F50

- SGI
- Origin
- 200

**Notes:**
I’d like us to get MPI/IO into ORNL but I know doodly about it. Similarly GRID. So might as well ask *them*, then discuss the next morning.

## ORNL Probe Cell, June 2001 – equipment for data research

- Stingray – S80
- 6-processor RS/6000
- 2GB
- Excellent CPU and
- I/O capability

- Marlin – H70
- 4-processor
- RS/6000
- 2 GB
- Oracle, DB2

- STK Silo

- 300 GB SCSI
- RAID Disks

- 360 GB Sun
- FibreChannel
- Disks

- 360 GB STK
- FibreChannel
- Disks

- FibreChannel Switch

- External
- Esnet Router

**To NERSC Probe**

- Gigabit Ethernet
- Jumbo

- STK Silo

**IBM SP**

**Compaq Sierra**

- 3494 Library

- Probe
- Shared with Production

- Intel
- Dual Xeon
- Linux

- Sun
- E450

- Oracle, DB2

- HPSS

**CPU intensive work might run on Stingray, Marlin or  some supercomputer**.
_**Available**_**:**
**  ****HPSS/IBM,**
**  ****HPSS/Solaris,**
**  ****Oracle 8i and DB2**

- HPSS

## Application involvements with Probe

- Climate
- Goal: production use of WAN bulk transfers
- Probe provides:
- Some storage space (source of data)
- Access to new transfer mechanism and associated equipment
- Development and testing to expedite their work
- Astrophysics
- Goal: high-bandwidth transfers between heterogeneous platforms
- Probe expects to provide:
- ~300 GB storage (disk and tape)
- Access to S80 or perhaps other machines
- Access to GSN equipment
- Assistance using the above

## Climate: Wide-Area bulk transfers

- WAN bulk transfers are a vital element of SDM ETC
- Wide-area bulk data transfer requires:
- High sustained rate
- Asynchronous transfer (so user isn’t tied up)
- Suitability for many source/sink pairs
- “Remote Mover” architecture works
- Designed for Climate – one direction, one source/sink pair
- Not suitable for SciDAC needs
- Considering implementing a front-end for HPSS
- Currently testing ESnet III
- ORNL/NERSC performance poor (bandwidth quadrupled but latency doubled; other problems too)

## ORNL GSN Configuration

- ODS/Essential
- GSN Switch
- 6 ports

- Genroco
- Bridge

- Stingray
- RS/6000 Model S80
- AIX 4.3.3

- Capsicum
- Origin 2000
- Reality Monster
- IRIX 6.5.10

- Compaq Alpha
- DS20
- Tru64 4.0f

- 2 Gigabit Ethernet blades

- 2 FibreChannel blades

- Cave
- Visualization
- Equipment

- Sun, Clariion or Compaq disks

- IBM, Compaq use
- Genroco 6466 NICs
- (64 bit 66 MHz)

- Marlin
- RS/6000 Model H70
- (50 MHz slots)

- Storage
- Equipment

- Earl
- RS/6000 Model B80
- (50 MHz slots)

- Alternative nodes

## ORNL Probe Cell, June 2001 – Possible equipment for supernova visualization

- Stingray
- RS/6000 S80

- STK Silo

- 300 GB SCSI
- RAID Disks

- Compaq
- DS20

- 360 GB Sun
- FibreChannel
- Disks

- 360 GB STK
- FibreChannel
- Disks

- FibreChannel Switch

- GSN Switch

**Origin 2000**
**Reality Monster**

- RS/6000
- B80

- External
- Esnet Router

**To NERSC Probe**

- Gigabit Ethernet
- Jumbo

- STK Silo

**IBM SP**

**Compaq Sierra**

- 3494 Library

- GSN Bridge

**ORNL Backbone**

- Probe
- Shared with Production

- Sun
- E450

- AIX 5L, GSN

- SGI
- Origin
- 200

- GSN

- RAIT

- HPSS

- SN6000

- Marlin
- RS/6000
- H70

- HPSS, GSN

**CPU intensive work might run on Stingray, Marlin or  some supercomputer**.

## Technology on hand and available

- On hand
- HPSS (unlimited instantiations) and HPSS development license
- ESnet III OC12
- Oracle 8i and DB2 (current developer’s editions)
- Inter-HPSS hsi application
- TCP-friendly UDP service
- OPNET modeling product
- Variety of current storage and networking hardware and software
- RAIT (scheduled for trial in August)
- In plan:
- ORNL GRID node (MICS funded)
- Web100 participation (MICS funded)
- MPI/IO software

## Questions?