# Acceleration of a Production-Level Unstructured Grid Finite Volume CFD Code on GPU
## 1 Introduction
However, the irregularity of the computation and indirect addressing in inner loops due to the unstructured meshes makes this a daunting challenge (2018). Current methods for handling such data races lead to reduced parallelism and suboptimal performance. Particularly for GPUs that have increasing core/thread counts, reducing data movement and exploiting memory locality is vital for gaining good performance (2018). Moreover, the inherent sequential properties of most implicit schemes make it more difficult for the algorithm to achieve parallel acceleration. For example, the lower-upper symmetric Gauss-Seidel (LU-SGS) algorithm is the most popular implicit method for solving large sparse linear equations in CFD. However, the strong data dependency during the forward and backward sweep makes it a tough challenge for shared-memory parallelization (2018). In-depth shared memory parallelization studies for production-level CFD applications are still relatively limited and are a focus of active research. [PERSON] accelerated NASA's overset grid CFD solver OVERFLOW by moving a portion of the code to the GPU. However, due to the dependencies of the iteration, the Jacobian algorithm used to solve the sparse linear system was ported to the GPU instead of the original SSOR algorithm and demonstrated a 25% improvement in performance over the CPU alone (2018). [PERSON] et al. improved the performance of the famous open-source CFD code OpenFOAM by GPU. However, they only ported the linear solver part of the code, achieving a 2.5 times improvement. Utilizing NVIDIA's Tesla K20 GPU compared to a parallel CPU implementation on ten cores of an Intel Xeon E5-2670 v2 (2018), [PERSON] et al. presented a CUDA C++ implementation of FUN3D's thermochemical nonequilibrium flow simulation capabilities targeting GPUs. They mainly used atomic operations to address the race condition issue (2018). In summary, when targeting GPU architecture and complex production-level unstructured CFD software, comprehensive research and analysis are needed in terms of parallelization methods and memory access optimization techniques. The motivation of this paper is to accelerate a production-level unstructured CFD software on GPU. NNW-FlowStar (FlowStar for short) is an industrial CFD software developed by China Aerodynamics Research and Development Center (CARDC) [20]. Its solver is developed based on the unstructured finite volume method and MPI parallel technology. To adapt to the new trend of HPC development, it is necessary to further develop MPI+X hierarchical parallel technology, the CUDA programming model of GPU is a kind of X. The remainder of this paper is organized as follows. Section 2 details the governing equations for the finite volume method for unstructured grids and the data racing problem due to irregular memory access patterns. Section 3 introduces the GPU parallelization implementation for face-loop kernels with data conflicts problem and the implicit method of LU-SGS. Section 4 describes some optimization methods to improve memory access efficiency. Section 5 presents the performance and efficiency results analysis for practical engineering problems. Section 6 provides the conclusion of this work and a plan for future work.