Performance Evaluation of the Mixed Precision GMRES(𝑚)Method using FP64 and FP32

1

Yingqi Zhao

Takeshi Fukaya

Takeshi Iwashita

Hokkaido UniversityJapanyqzhao2016@hotmail.com

Hokkaido University, JST PRESTOJapan
fukaya@iic.hokudai.ac.jp

Hokkaido UniversityJapan
iwashita@iic.hokudai.ac.jp

INTRODUCTION

Traditionally, FLOPS (Floating-point Operations Per Second) for
double-precision floating-point number (FP64) operation has beenused as a metric for the performance of a computer system. However, it is getting difficult to improve FP64 FLOPS of a single processor due to the limitation of power budget. Besides, some new
applications gradually accept low precision computing.
Under this circumstances, it is necessary to develop a new methodthat can exploit low precision computing and provide computed
results with the same accuracy as by traditional methods using only
FP64. The key idea here is mixed precision computing, in which
both FP64 and low precision computing (e.g., FP32) are combined.
In this paper, the target problem is a system of linear equations
𝐴𝑥 = 𝑏, where 𝐴 ∈ R𝑛 is large and sparse. For this problem, amixed precision variant of the GMRES(𝑚) method using FP64 andFP32 (MP-GMRES(𝑚)) is discussed, and its numerical behaviour isinvestigated in detail through numerical experiments.

2MIXED PRECISION GMRES(𝑚) METHODS

The mixed precision variant of the GMRES(𝑚) method (Algorithm 1)
has the mathematical background related to the iterative refinement
scheme for the solution of linear system ; this relation is clearly
explained in the study by Imakura et al. [3] The effectiveness of
MP-GMRES(𝑚) has been confirmed in some cases [1, 4], however,its numerical behaviour is sill not clear.
Algorithm 1 Mixed Precision GMRES(𝑚)Require: 𝑥 0 : initial guess, 𝜖 : convergence criterion, 𝐴; coefficient matrix, 𝑏 : righthand side vector, maximum number of iterations
1: 𝐴(L) = Low (𝐴)⊲ precision converts from standard to low
2: repeat
3: 𝑟 0 = 𝑏 − 𝐴𝑥 0 , 𝛽 = ∥𝑟 0 ∥ 2
4:
if 𝛽/ ∥𝑏 ∥ 2 ≤ 𝜖 then return 𝑥 0
5:𝑣0 = 𝑟 0 /𝛽(L)6:𝛽 (L) = Low (𝛽) , 𝑣0 = Low (𝑣0 )
(L)
(L)
(L)
7: Compute 𝑉𝑚 and 𝐻˜ 𝑚 by the 𝑚 -step Arnoldi process with 𝐴(L) and 𝑣0 . ⊲using low precision arithmetic
(L)
(L)
8:
Compute 𝑦𝑚 from 𝛽 (L) and 𝐻˜ 𝑚 .
⊲ using low precision arithmetic(L)(L) (L)
9: 𝑧𝑚 = 𝑉𝑚 𝑦𝑚⊲ using low precision arithmetic(L)
10: 𝑧𝑚 = Std (𝑧𝑚 )⊲ precision converts from low to standard
11: 𝑥 0 = 𝑥 0 + 𝑧𝑚
12: until attain the maximum number of iterations
Ensure: 𝑥 03

NUMERICAL EXPERIMENTS

In order to clarify the numerical behaviour of MP-GMRES(𝑚) ,numerical experiments with the matrices obtained from the SuiteSparse Matrix Collection [2] are conducted, in which MP-GMRES(𝑚)6000
FP64-GMRES(m)MP-GMRES(m)

5000

4000

3000

2000

10000050100200300

400500Figure 1: The change of the number of the total inner iterations in FP64-GMRES(𝑚) and MP-GMRES(𝑚) for memplus.
is compared with conventional GMRES(𝑚) using only FP64 (FP64GMRES(𝑚)). The number of iterations, the achievable accuracy andthe execution time are investigated. The detail of the numericalresults will be presented in the poster.
In terms of the number of iterations, when 𝑚 is small, the numberof iterations of the two methods are almost equal. However, when𝑚 becomes larger, the number of iterations of FP64-GMRES(𝑚)basically decreases, but that of MP-GMRES(𝑚) tends to increase.
Take matrix named memplus for example, Figure 1 illustrates thechange of the number of the total inner iterations with 𝑚.
For the achievable accuracy, both FP64-GMRES(𝑚) and MPGMRES(𝑚) attained the convergence criterion (∥𝑏 − 𝐴𝑥 ∥ 2 /∥𝑏 ∥ 2 ≤10−10 ) in most cases. If both two methods did not converge, thereis almost no difference in the final accuracy.
When 𝑚 is small, the execution time of both methods tends tobe short. For many test matrices, when the number of iterations
is almost equal, MP-GMRES(𝑚) outperforms FP64-GMRES(𝑚) interms of the execution time. Even in the case that larger 𝑚 reduces
the number of iterations in FP64-GMRES(𝑚), e.g., matrix named
wang4, MP-GMRES(𝑚) is still faster, in which about three times theiterations are required.

REFERENCES[1] Hartwig Anzt, Vincent Heuveline, and Björn Rocker. 2011. An Error Correction
Solver for Linear Systems: Evaluation of Mixed Precision Implementations. In
High Performance Computing for Computational Science – VECPAR 2010. 58–70.
[2] Timothy A. Davis and Yifan Hu. 2011. The University of Florida Sparse Matrix
Collection. ACM Trans. Math. Software 38, 1 (Dec. 2011), 1:1–1:25.
[3] Akira Imakura, Tomohiro Sogabe, and Shao-Liang Zhang. 2012. An Efficient
Variant of the GMRES(𝑚 ) Method Based on the Error Equations. East Asian
Journal on Applied Mathematics 2, 1 (2012), 19–32.
[4] Neil Lindquist, Piotr Luszczek, and Jack Dongarra. 2022. Accelerating Restarted
GMRES with Mixed Precision Arithmetic. IEEE Transactions on Parallel andDistributed Systems 33 (2022), 1027–1037. Issue 4.