## 2 Matrix Sign Function-Based Solvers
A description of the unblocked version called from inside the blocked one can be found in [17]; for simplicity, we hide the application of pivoting during the factorization, but details can be found there, as well. ### Parallel Implementations of the Gauss-Jordan Elimination Algorithm

Three high performance implementations were developed for the matrix inversion, one for each class of target platform: general-purpose multicore processor, hybrid CPU-GPU systems and hybrid multi-GPU platforms. Using these implementations, we can obtain parallel and efficient solvers for each matrix equation and architecture under study. **Multicore processors**. The multicore routines rely on high-performance multi-threaded implementations of BLAS that efficiently exploit the hardware concurrency (superscalar/pipelined processing of instructions, SIMD (Single Instruction Multiple Data) floating-point units, multiple cores, _etc._) on this type of architecture. The key to success here is the use of a highly-efficient implementation of the matrix-matrix product to perform the updates of the corresponding blocks in Algorithm 5--GJEBLK, as well as a correct evaluation of the algorithmic block size. Some additional performances can be regained by replacing the unblocked algorithm that factorizes the panel at each iteration by a blocked implementation. **Hybrid CPU-GPU systems. ** These implementations exploit the massively parallel architecture of the GPU to reduce the time-to-response of costly data-parallel computations (e.g., large matrix-matrix products), while operations that feature a reduced computational cost or present fine-grain parallelism are executed on the CPU. In matrix inversion via GJE (see Algorithm Algorithm 5--GJEBLK), all operations are performed on the GPU, except for the factorization of panel \([A_{01};A_{11};A_{21}]^{T}\), which is done on the CPU. Assuming that the complete matrix initially lies in the GPU memory, this panel is transferred to the CPU at the beginning of each iteration and processed there, and the results are sent back to the GPU. This version benefits from the use of architecture-specific parallel BLAS, like Intel's MKL (Math Kernel Library) for the CPU and NVIDIA's CUBLAS for the GPU. Additionally, these implementations include several common optimization techniques:

* _Padding_ adds rows at the bottom of the matrix until their number is a multiple of 32. Padding slightly increases the memory requirements, but not the computational cost, as we do not access/operate with the elements in these rows. On the other hand, the GPU-memory accesses are notoriously accelerated due to coalescing. * _Look-ahead_ advances the computation of operations in the critical path. In our case, a key element in this path is the column panel factorization performed at each step of the algorithm. To advance this computation, we partition the update of \([A_{02};A_{12};A_{22}]^{T}\) into two parts: the first \(b\) columns of this block (which will form blocks \([A_{01};A_{11};A_{21}]^{T}\) in the next iteration) are first updated. Next, while this part is transferred to the CPU and factorized there, the rest of the elements of the matrix are updated in the GPU. * _Recursive blocking_ detaches the block sizes used by the GPU and CPU. While the GPU usually attains high performance using a large algorithmic block-size, the CPU requires a smaller value for this parameter. Furthermore, the unblocked algorithm that factorizes \([A_{01};A_{11};A_{21}]^{T}\) is replaced by a blocked implementation, which uses an inner algorithmic block-size. More details on these techniques can be found in [18]. **Hybrid multi-GPU systems. ** Hybrid CPU-GPU implementations offer remarkable performance, but the dimension of the problems that can be tackled with them is constrained by the size of the GPU memory (typically, 4-6 Gbytes). Hybrid implementations that combine a CPU with multiple GPUs partially overcome this problem, since, as the number of GPUs grows, the aggregated size of the memory is also increased.