# Multi-GPU Acceleration for Finite Element Analysis in Structural Mechanics
## 1 Introduction  
Nowadays, parallel processing has become the dominant paradigm in scientific computing to address the ever-increasing complexity and fidelity of the finite element models [1]. [PERSON]'s law [2] is still valid today, but semiconductor companies use these increasing computational resources to increment the number of processors (multi-core and many-core architectures) rather than making faster cores [3]. The limitations to obtaining faster cores despite the increasing number of transistors in integrated circuits are mainly due to three technical walls: the power wall, the Instruction-Level Parallelism (ILP) wall, and the memory wall. The power wall refers to the difficulty of scaling the performance due to the affordable power delivery and dissipation, which limits the operating frequency to a certain threshold. The ILP wall relates to the difficulty of finding enough parallelism in a single instruction stream to keep a single-core processor busy [4]. The memory wall refers to the problems that arise when the processor speed improvement exceeds the memory speed improvement [5]. All these facts allow us to affirm that serial computing has reached its zenith in performance [6]. Multi-core architectures exploit the task or function parallelism, where we divide the problem into smaller ones, and the code is then distributed across multiple processors and executed in parallel. On the other hand, the many-core architectures exploit data parallelism using Single Instruction Multiple Data (SIMD), where the same code executes on different pieces of distributed data across parallel computing nodes. Function parallelism permits flexible parallel computing, whereas data parallelism provides high speedups. For this reason, we often combine both techniques to exploit the computing capabilities of the architectures properly. Many-core computing in modern high-performance systems refers to non-graphical computing using graphics cards, which is becoming an integral part of modern high-performance systems. Computing nodes of most modern supercomputers include potent CPUs for task parallelism and powerful GPUs for massively parallel computing [7]. Workstation vendors also adopt a similar configuration by including co-processors that permit the use of General-Purpose computing on Graphics Processing Units (GPGPU) [8; 9]. However, exploiting these massively parallel architectures requires algorithms with abundant fine-grained or data-level parallelism. The memory limitations in GPU computing using only the device memory of one graphics card restrict the problems that can be addressed using many-core computing. By introducing function parallelism in GPU systems [10; 11], whether using shared (multi-GPU systems) or distributed (GPU clusters) memory approaches, we can remove the device memory limitations, usually at the cost of reducing the GPU computing performance due to data transfers between memory systems. Multi-GPU computing is becoming a hot topic to speed up libraries, tools, and applications limited by the memory specifications of GPU devices. We can mention the mesh-free peridynamic simulation using a multi-GPU parallel scheme [12] in the context of fracture mechanics applications, the simulation of the dynamics of gigamolecules [13] using GPU clusters, and the 3D turbulent flow simulations [14] using a multi-GPU implementation for a gas kinetic scheme. In the context of generic linear algebra applications, we can mention the acceleration of Sparse Matrix-Vector (SpMV) multiplication operations for iterative-solving algorithms used in large-scale problems [15] and the dynamic work balancing in multi-GPU systems through task migration [16], which is evaluated using Block-Sparse General Matrix Multiplication (BSpGEMM), Block General Matrix Multiplication (GEMM), and Block Cholesky Factorization. In the context of solid mechanics problems, we can accelerate the resolution of the linear system arising from the finite element method [17] using GPU computing [18].