Evaluation of modern GPGPU technologies for image processing

Image processing has high computational requirements. As many papers (including [Frisk 2010; N. Zhang and Wang 2010; Yang et al. 2008]) have indicated, a number of image processing operations can be optimized by heavy parallelization of the computation. Currently, one of the best options for parallelized image processing is using general purpose computation on graphics processing units (GPGPU). In the area of GPGPU, a rather wide range of APIs is available and finding an appropriate choice for a project is challenging. In the course of being able to provide guidance on selecting the right API in an image processing context, four GPU programming models were compared according to their platform independence, usability, and performance. The four investigated APIs are CUDA, OpenCL 1.2, Vulkan, and SYCL. To gain information on the metrics usability and performance a test project was created, implementing a number of image processing tasks using the four GPU programming models. The test project was designed as a library, where the 4 investigated APIs could be replaced at compile-time and new operations could be easily added. The implemented reference algorithms form a pipeline for processing images from polarization cameras. The targeted platforms were mid- to high-end desktop PCs as well as embedded single board platforms ranging from Odroid to Nvidia Jetson. On all devices, either MSVC 19.x or GCC 7+ were used as host compilers. CUDA tests were compiled with NVCC and SYCL was represented by ComputeCpp and hipSYCL depending on the device. The direct comparison of the APIs, done for this work, cumulates into a decision matrix to aid organizations in their process of selecting an offloading API. It shows, for example, that CUDA and SYCL are comparable in terms of development cost, support of modern C++ and ease of integration in existing code bases. When considering modern standard adoption and GPU portability Vulkan has advantages, while OpenCL and SYCL can run on CPUs and FPGAs. In the tooling category, CUDA has the most specialized and effective tools whereas Vulkan has almost no compute specific tools available. CUDA and Vulkan provide the best performance (on the tested devices), while Vulkan was able to outperform CUDA on a Nvidia Jetson. It becomes clear that all four APIs have their advantages and may be suitable for different applications. Vulkan is the API of choice in mobile consumer applications. CUDA is convenient for fast development but only if it is feasible to support Nvidia GPUs only. For AMD GPU support the CUDA-like HIP can be used, although lacking Windows support and the tool quality on the AMD side. OpenCL might be chosen if a considerable number of different desktop GPUs and FPGAs should be supported. Although some metrics still show room for improvement, SYCL seems to grow into a good all-rounder with all the available implementations. It could benefit from weakening its tight integration with OpenCL to improve support on some platforms. A Vulkan implementation, for example, would open opportunities to target mobile and Apple devices, while hipSYCL could gain the status of a standard-conforming implementation. While these results provide a general and first overview of the differences, the pros, and cons of those APIs, it would be interesting to receive feedback from the Khronos developer community on their experiences and build more mature metrics on it.