# Presentation: 324692

## MPI (I/O) Perspectives

*Rajeev Thakur*
- Mathematics and Computer Science DivisionArgonne National Laboratory

## Outline

- What is MPI-IO?
- How does it address the scalable parallel I/O problem?
- What does it enable you to do?
- Why is it important?
- What is the status of implementations?
- What are the performance and scalability issues?

## What is MPI-IO?

- A standard, portable API for parallel I/O
- Part of the MPI-2 Standard, released July 1997
- Designed to support a programming model in which multiple processes perform I/O to the same file concurrently
- Defined by a broadly based group including vendors, users, and I/O researchers from academia and laboratories
- Incorporates results of research over the last 8--10 years in the parallel I/O field

## How does it address the scalable parallel I/O problem?

- Defines a standard, portable API that enables various optimizations that are essential for  high-performance parallel I/O:
- collective I/O
- noncontiguous file accesses
- user-supplied hints
- multiple consistency modes
- Eliminates the performance (and portability) problems of using the Unix API for parallel I/O

## What does it enable you to do?

- Write application programs in a portable way and also achieve high performance
- Leave I/O optimizations to the MPI-IO implementation
- Library writers can exploit the features of MPI-IO to deliver high performance along with the convenience of the higher-level library (e.g., HDF, DMF, Exodus, Silo)

## Why is it important?

- With the advent of MPI-IO, the I/O API is no longer the problem
- Given sufficient I/O hardware and a good file system, the MPI-IO implementation *can *deliver high performance to applications
- Performance is limited by what is  realistically achievable on the given hardware and file system
- I have seen 1 Gbyte/s on NEC SX-4,                800 Mbytes/s on Argonne Origin2000, and 150 Mbytes/s on one of the NCSA Origins

## What is the status of MPI-IO implementations?

- I/O was the first part of MPI-2 to be implemented
- Several implementations are available
- ROMIO, developed at Argonne, is a portable implementation that runs almost everywhere
- PMPIO from NASA Ames, MPI-IO on HPSS from Livermore
- Several vendors (HP, SGI, NEC) have incorporated ROMIO into their MPI impl.

## What is the status of implementations?

- IBM, Compaq, Fujitsu, Hitachi have MPI-IO implementations in various stages of completion
- On clusters:
- ROMIO has been implemented on the PVFS file system from Clemson (and also on NFS)
- ROMIO works on SGI’s XFS, therefore should work directly on CXFS
- ROMIO is already being used on IBM GPFS at Livermore
- Should also work on GFS (Minnesota) because ROMIO works on any file system with a Unix/POSIX API

## What are the performance and scalability issues (on clusters)?

- On large clusters with sufficient number of I/O servers, the aggregate disk bandwidth is quite high
- The network, then, becomes the bottleneck
- Scalability and performance depends on scalability of network and networking software
- TCP, even on fast gigabit networks, may not be good enough
- We are working with the Clemson folks to redesign PVFS to use multiple communication mechanisms: TCP, VIA, Myrinet, ST, ...

## MPI (I/O) Perspectives

*Rajeev Thakur*
- Mathematics and Computer Science DivisionArgonne National Laboratory