# An Evaluation of Directive-Based Parallelization on the GPU Using a Parboil Benchmark
###### Abstract  
Heterogeneous architectures consisting of both central processing units and graphics processing units are common in contemporary computer systems. For that reason, several programming models have been developed to exploit available parallelism, such as low-level CUDA and OpenCL, and directive-based OpenMP and OpenACC. In this paper we explore and evaluate the applicability of OpenACC, which is a directive-based programming model for GPUs. We focus both on the performance and programming effort needed to parallelize the existing sequential algorithms for GPU execution. The evaluation is based on the benchmark suite Parboil, which consists of \(11\) different mini-applications from different scientific domains, both compute- and memory-bound. The results show that mini-apps parallelized with OpenACC can achieve significant speedups over sequential implementations and in some cases, even outperform CUDA implementations. Furthermore, there is less of a programming effort compared to low-level models, such as CUDA and OpenCL, because a majority of the work is left to the compiler and overall, the code needs less restructuring. GPU computing; directive-based parallelization; parallel algorithms; parallel programming; performance analysis +
Footnote †: journal: _electronics_  
Accelerators). OpenMP is a well-known, proven programming model for multicore CPUs. GPU support was added to the 4.0 specification. Programmer uses directives to annotate regions of code that should be automatically parallelized by the compiler. The main sources of parallelism in sequential codes are loops. To allow parallel execution, loop iterations should be independent. Based on ideas from the OpenMP directive-based programming model for multicore CPUs, the OpenACC programming model for GPUs was introduced in 2011. OpenMP introduced basic support for GPUs in version 4.0 in 2013, while more enhanced support was added later with versions 4.5 and 5.0. In the OpenACC programming model, a set of compiler directives is used to annotate sequential code regions that should be offloaded for execution on the GPU. Basically, compiler directives are used to offload loops for GPU execution and manage the data movement between CPU and GPU. Although such an approach cannot always yield the performance of low-level programming models, it can obtain a reasonably good performance with much less programming effort. Also, performance portability remains high, as not all hardware details are exposed to the programmer. Many programmers find the directive-based approach useful, as it preserves the flexibility of running parallel code both on the CPUs and the GPUs, while also avoiding vendor-specific language lock-in [10]. In studies, such as [11; 12], it is suggested that OpenACC offers a more descriptive programming model than OpenMP, which is described as prescriptive. In OpenACC, much work on parallelization is performed automatically by the compiler, while OpenMP requires the programmer to specify more precisely how to parallelize the code. OpenACC is considered a more mature programming model for the GPUs in several studies in terms of compiler and platform support [11; 13]. The descriptive nature of the programming model and the maturity of the whole OpenACC ecosystem were the main reasons we chose to evaluate OpenACC in this work. In this paper, we re-implemented applications from the Parboil GPU benchmark suite [14] using OpenACC to evaluate the effectiveness of the directive-based parallelization on the GPU. The Parboil benchmark consists of eleven different computing applications from various scientific domains, including image processing, graph theory, biomolecular simulation, fluid dynamics, and astronomy. It was carefully selected to represent diverse workloads in typical high-performance computing applications and exhibit different algorithmic approaches, memory access patterns, and scalability issues. To the best of our knowledge, no such complete OpenACC implementation of the Parboil benchmark is available in the open literature.