HighPerformance Adaptive MPI Derived Datatype Communication for Modern MultiGPU Systems The recent advent of the NVLink interconnect and Peripheral Component Interconnect express (PCIe) switch has resulted in the creation of extremely dense Graphics Processing Unit (GPU) systems like Cray CSStorm and NVIDIA DGX. In addition to the extremely high computational capability and communication capacity within a single machine, these systems expose novel capabilities like performing loadstore operations from remote GPU memory across interconnects. While researchers have proposed solutions that take advantage of loadstore semantics to provide support for highperformance datatype processing on CPUs, there exists no scholarly work on how one can orchestrate such highperformance datatypebased communication for GPUresident data. In this paper, we take up this challenge and propose highperformance and architectureaware designs for GPUbased noncontiguous datatype processing that uses the loadstore semantics exposed by modern dense GPU systems. We demonstrate that the proposed solutions can reduce the overhead of datatype processing by up to 4.7X compared to the stateoftheart schemes for GPUbased MILC communication kernel on an NVLink2enabled dense GPU system. For a weather forecast application kernel, the proposed designs demonstrate up to 9.9X faster HaloExchange kernel among 64 GPUs over stateoftheart designs. The proposed adaptive scheme also reports 10% higher throughput than existing designs for a 2D Jacobi solver on 16 GPUs. To the best of our knowledge, this is the first scholarly work that takes advantage of the zerocopy based loadstore semantics to perform highperformance GPU to GPU derived datatype communication on modern dense GPU systems.

A general purpose contention manager for software transactions on the GPU Abstract   The Graphics Processing Unit (GPU) is now used extensively for general purpose GPU programming (GPGPU), allowing for greater exploitation of the multicore model across many application domains. This is particularly true in cloudedgefog computing, where multiple GPU enabled servers support many different end user services. This move away from the naturally parallel domain of graphics can incur significant performance issues. Unlike the CPU, code that is hindered from execution due to blockingwaiting on the GPU can affect thousands of threads, rendering the advantages of a GPU irrelevant and reducing a highly parralel environemnt down to a serial one in the worst case. In this paper we present a solution that minimises blockingwaiting in GPGPU computing using a contention manager that offsets memory conflicts across threads through thread reordering. We consider conflicts of memory not only to avoid corruption (standard for transactional memory) but also in the semantic layer of application logic (e.g., enforcing ordering to ensure money drawn from bank account occurs after all deposits). We demonstrate how our approach is successful across a number of industry benchmarks and compare our approach to the only other related solution. We also demonstrate that our approach is scalable in terms of thread numbers (a key requirement on the GPU). We believe this is the first work of its kind demonstrating a generalised conflict and semantic contention manager suitable for the scale of parralel execution found on a GPU.