Abstract-We present and analyze two new communication libraries, cudaMPI and glMPI, that provide an MPI-like message passing interface to communicate data stored on the graphics cards of a distributed-memory parallel computer. These libraries can help applications that perform general purpose computations on these networked GPU clusters. We explore how to efficiently support both point-to-point and collective communication for either contiguous or noncontiguous data on modern graphics cards. Our software design is informed by a detailed analysis of the actual performance of modern graphics hardware, for which we develop and test a simple but useful performance model.
Lawlor has proposed cudaMPI REF that provides an MPI-like interface for communication between remote GPUs.
Message passing for GPGPU clusters: CudaMPI
2009 IEEE International Conference on Cluster Computing and Workshops
2009 IEEE International Conference on Cluster Computing and Workshops
Computer Science