qCUDA: GPGPU Virtualization for High Bandwidth Efficiency The increasing demand for machine learning computation contributes to the convergence of highperformance computing and cloud computing, in which the virtualization of Graphics Processing Units (GPUs) becomes a critical issue. Although many GPGPU virtualization frameworks have been proposed, their performance is limited by the bandwidth of data transactions between the virtual machine (VM) and host. In this paper, we present a virtualization framework, qCUDA, to improve the performance of compute unified device architecture (CUDA) programs. qCUDA is based on the virtio framework, providing the paravirtualized driver and the device module for performing the interaction with the API remoting and memory management methods. In our test environment, qCUDA can achieve above 95% of the bandwidth efficiency for most results by comparing it with the native. Also, qCUDA has the features of flexibility and interposition. It can execute CUDAcompatible programs in the Linux and Windows VMs, respectively, on QEMUKVM hypervisor for GPGPU virtualization.

Performance Optimization for InfiniBand Virtualization on QEMUKVM The emergence of machine learning applications has brought the new demands for high performance computing in cloud environment. Besides accelerators, such as GPU or TPU, fast interconnection among and within computers becomes more and more important to achieve efficient training and learning. One of the highbandwidth, lowlatency interconnection architectures is InfiniBand. However, software based virtualization of InfiniBand in QEMUKVM suffers large performance degradation owing to virtualization overhead and memory allocation problem. In this paper, two techniques, doorbell mapping and memlink, are proposed to optimize the performance the InfiniBand virtualization on QEMUKVM. Doorbell mapping allows the applications in guest userspace to access the doorbell memory page directly so that the virtualization overhead is minimized. Memlink ensures memory contiguity after virtualization, which is a critical requirement for zerocopy between guest and host. Experiments show that the virtualized InfiniBand with mmap and memlink can achieve near native performance for large data transmissions. Comparing to the previous InfiniBand virtualization on QEMUKVM, our implementation obtains over 3.5 times performance improvement in various benchmarks.