Parallelization of DirectForcing Immersed Boundary Method Using OpenACC A parallelization of Poisson equation solver is developed by using the parallel algorithm of redblack SOR method performed on GPU by OpenACC. We use the parallel computing solver to solve the fluidstructure interaction problem with directforcing immersed boundary method. The speedup for the parallel computing solver is up to 8.62 times. In addition, we present that the optimization of the redblack SOR (RBSOR) algorithm for the problem with the complexgeometry objects is to allocate the same memory on two variables which is to take advantages of accessing data by random indexes and formatted indexes. The implementation of parallel computing RBSOR with OpenACC is efficient to delegate the simulation of fluidstructure interaction problem on the GPU.

GPU Acceleration of Communication Avoiding Chebyshev Basis Conjugate Gradient Solver for Multiphase CFD Simulations Iterative methods for solving large linear systems are common parts of computational fluid dynamics (CFD) codes. The Preconditioned Conjugate Gradient (PCG) method is one of the most widely used iterative methods. However, in the PCG method, global collective communication is a crucial bottleneck especially on accelerated computing platforms. To resolve this issue, communication avoiding (CA) variants of the PCG method are becoming increasingly important. In this paper, the PCG and Preconditioned Chebyshev Basis CA CG (PCBCG) solvers in the multiphase CFD code JUPITER are ported to the latest V100 GPUs. All GPU kernels are highly optimized to achieve about 90% of the roofline performance, the block Jacobi preconditioner is redesigned to extract high computing power of GPUs, and the remaining bottleneck of halo data communication is avoided by overlapping communication and computation. The overall performance of the PCG and PCBCG solvers is determined by the competition between the CA properties of the global collective communication and the halo data communication, indicating an importance of the internode interconnect bandwidth per GPU. The developed GPU solvers are accelerated up to 2x compared with the former CPU solvers on KNLs, and excellent strong scaling is achieved up to 7,680 GPUs on the Summit.