# Presentation: 417821

## CplantTM Release 0.5 Update

- Rolf Riesen, et al.
- April 4, 2000

## Highlights

- Bug fixes
- Meminfo() to get at /proc/meminfo
- New PBS policy (already on Alaska)
- Portals 3.0
- More (will send list when Release comes out)

- Cplant Users Meeting, April 4, 2000

## Portals 3.0

- Data movement layer from SUNMOS,  Puma, and Tflops
- 3.0 is a complete redesign to make it suitable for full-featured OS’s (such as Linux) and COTS hardware
- API intended for library writers, not application programmers
- Tech report SAND99-2959

- Cplant Users Meeting, April 4, 2000

## Portals 3.0 & Release 0.5

- We’re behind, due date was March 17
- 3.0 is already better than 2.0, but some known bugs remain
- Want to release it to first few test users without any known bugs
- Fix whatever test users uncover, then release
- First release will be on Siberia ~45s, 100MB/s under MPI
- Fixes problems seen under 2.0 on Alaska

- Cplant Users Meeting, April 4, 2000

## Portals 3.0 Phase 1 (Release 0.5)

- MPI Library

- Application

- Runtime
- environment
- yod, pct, etc.

- Std I/O Library

- User space

- Kernel space

- P3 Module

- Old P2 Module

- RTS/CTS Module

- MCP on Myrinet card

- Cplant Users Meeting, April 4, 2000

## Portals 3.0 Phase 2

- MPI Library

- Application

- Runtime
- environment
- yod, pct, etc.

- Std I/O Library

- User space

- Kernel space

- P3 Module

- RTS/CTS Module

- MCP on Myrinet card

- Cplant Users Meeting, April 4, 2000

## Portals 3.0 Phase 3

- MPI Library

- Application

- Runtime
- environment
- yod, pct, etc.

- Std I/O Library

- User space

- Kernel space

- Cplant Module

- P3 on Myrinet card

- Cplant Users Meeting, April 4, 2000

## Interface Concepts

- One-sided operations
- Put and Get
- Zero copy message passing
- Increased bandwidth
- OS Bypass
- Reduced latency
- Application Bypass
- No polling, no threads
- Reduced processor utilization
- Reduced software complexity

- Cplant Users Meeting, April 4, 2000

## MPP Network: Paragon and Tflops

- Memory

- Processor

- Processor

- Memory
- Bus

- Network

- Message passing or computational co-processor

- Network interface is on the memory bus

- Cplant Users Meeting, April 4, 2000

## Commodity:  Myrinet

- Memory
- Bus

- PCI Bus

- Bridge

- Processor

- Memory

- NIC

- Network

- Network is far from the memory

- OS Bypass

- Cplant Users Meeting, April 4, 2000

## “Must” Requirements

- Common protocols (MPI, system protocols, I/O)
- Portability
- Scalability to 1000’s of nodes
- High-performance
- Multiple process access
- Heterogeneous processes (binaries)
- Runtime independence
- Memory protection
- Reliable message delivery
- Pairwise message ordering

- Cplant Users Meeting, April 4, 2000

## “Will” Requirements

- Operational API
- Zero-copy MPI
- Myrinet
- Sockets implementation
- Unrestricted message size
- OS Bypass, Application Bypass
- Put/Get

- Cplant Users Meeting, April 4, 2000

## “Will” Requirements

- Packetized implementations
- Receive uses start and length
- Receiver managed
- Sender managed
- Gateways
- Asynchronous operations
- Threads

- Cplant Users Meeting, April 4, 2000

## “Should” requirements

- No message alignment restrictions
- Striping over multiple channels
- Socket API
- Implement on ST
- Implement on VIA
- No consistency/coherency
- Ease of use
- Topology information

- Cplant Users Meeting, April 4, 2000

## Portal Addressing

- Portal Table

- Match List

- Memory Descriptors

- Event Queue

- Memory Region

- Portal API Space

- Application Space

- Operational Boundary

- Cplant Users Meeting, April 4, 2000

## Portals 3.0 Status

- Currently testing Cplant Release 0.5
- Portals 3.0 kernel module using the RTS/CTS module over Myrinet
- Port of MPICH 1.2.0 over Portals 3.0
- TCP/IP reference implementation ready
- Design of port to LANai begun

- Cplant Users Meeting, April 4, 2000

## http://www.cs.sandia.gov/cplant

- Cplant Users Meeting, April 4, 2000