UserLevel Scheduled Communications for MPI Composability is one of seven reasons for the longstanding and continuing success of MPI. Extending MPI by composing its operations with userlevel operations provides useful integration with the progress engine and completion notification methods of MPI. However, the existing extensibility mechanism in MPI (generalized requests) is not widely utilized and has significant drawbacks. MPI can be generalized via scheduled communication primitives, for example, by utilizing implementation techniques from existing MPI3 nonblocking collectives and from forthcoming MPI4 persistent and partitioned APIs. Nontrivial schedules are used internally in some MPI libraries; but, they are not accessible to endusers. Messagebased communication patterns can be built as libraries on top of MPI. Such libraries can have comparable implementation maturity and potentially higher performance than MPI library code, but do not require intimate knowledge of the MPI implementation. Libraries can provide performanceportable interfaces that cross MPI implementation boundaries. The ability to compose additional userdefined operations using the same progress engine benefits all kinds of general purpose HPC libraries. We propose a definition for MPI schedules: a userlevel programming model suitable for creating persistent collective communication composed with new applicationspecific sequences of userdefined operations managed by MPI and fully integrated with MPI progress and completion notification. The API proposed offers a path to standardization for extensible communication schedules involving userdefined operations. Our approach has the potential to introduce eventdriven programming into MPI (beyond the tools interface), although connecting schedules with events comprises future work. Early performance results described here are promising and indicate strong overlap potential.

Detecting and Reproducing ErrorCode Propagation Bugs in MPI Implementations We present an approach to automatically detect and reproduce error code propagation bugs in MPI implementations. Specifically, we combine static analysis and program repair for bug detection, and apply fault injection to reproduce error propagation bugs found in MPI libraries written in CC. We demonstrate our approach on the MPICH libraryxe2x80x94one of the most popular implementations of MPI, and the MPICHbased implementation MVAPICH2, uncovering 447 previously unknown bugs. We discovered that 31 of these bugs result in program crashes, and 60% of the MPICH test suite is susceptible to crashing due to failures to propagate error codes. Moreover, 95 bugs produce undesirable behavior that has been confirmed dynamically, causing tests to fail, hanging processes, or simply dropping error codes before reaching user applications.NDaniel DeFreezUniversity of California, DavisUnited StatesAntara BhowmickUniversity of California, Davi