# An Evaluation of Directive-Based Parallelization on the GPU Using a Parboil Benchmark
## 2 Related Work  
The OpenACC programming model has been used to parallelize scientific applications in numerous fields. OpenACC has been extensively used to parallelize simulations in nuclear physics [10; 15]. A hybrid MPI/OpenACC implementation of an iterative eigensolver for many-body calculations can be found in [10]. The implementation was derived from a previous OpenMP-based solution. The authors denote that those architectural differences between CPU and GPU changed the way OpenACC directives were inserted in the original code compared to the OpenMP implementation, especially for sparse matrix-vector multiplications with multiple vector operation. They report that the GPU implementation achieved significant speedup over the CPU implementation. Another example is the radiation transport mini-application presented in [15], based on the combined MPI (Message Passing Interface)/OpenACC implementation. It performs slightly better than the CUDA implementation of the same algorithm. The study [12] presented a novel methodology to implement scientific applications with MPI, OpenMP, and OpenACC They demonstrated the semi-automatic methodology on four real-life applications in the domain of physics and material science. The OpenACC implementation slightly outperformed CUDA Fortran in one of the examples used. Benchmarking multicore CPU and manycore GPU architectures was the focus of numerous research efforts in the past. Most of the benchmark suites are based on the concept of mini-applications (mini-apps) [16]. Mini-apps are parts of much larger, real-life applications from various domains that consist only of core algorithms rather than complete solutions from their respected fields. Still, they possess much of the performance characteristics of the original applications, but they are easier to maintain and experiment with and also explore the parameter space defined by the choices of the hardware platform, programing model, runtime environment, compiler, etc. One of the first benchmark suites for GPU computing was Rodinia [17]. Rodinia introduced mini-apps based on the dwarf application taxonomy from Berkley. In version 3.1, it consists of 23 different mini-apps implemented in CUDA, OpenCL, and OpenMP. Another notable example is the Parboil benchmark [14] which consists of 11 different applications in sequential and several parallelized forms, together with a well-developed testing framework in Python. The NAS Parallel benchmark is a standard benchmark suite for parallel computing based on MPI and OpenMP. The NAS benchmark consists of five kernels and three mini-applications. However, the GPU version of the suite, based on CUDA, became available only recently [18], and directive-based implementations for the GPU are not available. Performance comparisons of the OpenACC and CUDA implementations of different applications have been studied in the open literature [19; 20; 21]. However, most of the studies focused on one type of application, such as computational fluid dynamics (CFD) solvers [19; 21], molecular modeling code [20], and a combustion modelling simulation [22]. Four mini-apps from physics and materials science domains were ported to OpenACC and OpenMP and evaluated on several heterogeneous supercomputing systems in [11]. Those mini-apps utilized common methods found in the domain of high performance computing. The experiences from the study suggest that porting efforts are dependent on compiler maturity in implementing both sets of directives on a given platform. The OpenMP, OpenACC, and Kokkos implementations of the complete Navier-Stokes flow solver mini-application were compared in [13]. Navier-Stokes flow solvers are frequently used for computational fluid dynamics problems. A performance portability study [23] evaluated OpenMP, OpenACC, Kokkos, and RAJA directive-based models with a newly developed metric based on the data reported in 324 case studies from the open literature. It reports a performance portability of over 80% cases.