Scalable and Multilevel Iterative Methods

OF THE DISSERTATION Scalable and Multilevel Iterative Methods by Wing Lok Wan Doctor of Philosophy in Mathematics University of California, Los Angeles, 1998 Professor Tony Chan, Chair In this dissertation, we analyze three classes of iterative methods which are often used as preconditioners for Krylov subspace methods, for the solution of large and sparse linear systems arising from the discretization of partial di erential equations. In addition, we propose algorithms for image processing applications and multiple right-hand side problems. The rst class is the incomplete LU factorization preconditioners, an intrinsic sequential algorithm. We develop a parallel implementation of ILU(0) and devise a strategy for a priori memory allocation crucial for ILU(k) parallelization. The second class is the sparse approximate inverse (SPAI) preconditioners. We improve and extend its applicability to elliptic PDEs by using wavelets which converts smoothness, often found in the Green's functions of PDE operators, into small wavelet coe cients, thus producing a more e cient approximation. We also give analytic estimates for the wavelet SPAI preconditioners. The third class is the multigrid preconditioners. We propose and analyze new techniques for the basic components of multigrid: smoothing, coarsening and interpolation. First, we present a new class of parallel SPAI smoothers which o er more robustness than the relaxation smoothers by improving their quality using more nonzeros in the approximate inverse. Second, we propose an interface preserving coarsening, a new technique for solving discontinuous coe cient PDEs. It selects coarse grid points in such a way that the interfaces will align with or are resolved by all the coarse grids, allowing linear interpolation to give fast multigrid convergence. Finally, we propose and analyze a new robust interpolation method based on the concepts of energy minimization and approximation. In one dimension, we prove a super-optimal convergence rate independent of both the mesh size and the PDE coe cients. In two dimensions, we prove a two level convergence and show that the resulting multigrid is e ective for rough coe cient problems. The energy minimization principle can be also applied to inde nite problems. x As an application, we propose the use of fast transform preconditioned conjugate gradient methods as smoothers to solve di erential-convolution equations arising from PDE-based image problems by multigrid. These smoothers remove the high frequency errors which the usual relaxation smoothers cannot damp away. The resulting multigrid is e ective for the Tikhonov regularization approach. We analyze a class of projection methods for solving multiple right-hand side problems. We observe and prove two fundamental properties, namely: the superconvergence of the conjugate gradient re nement process and the automatic exploitation of the near rank de ciency of the right-hand side matrix. Furthermore, we propose a block generalization of it. xi CHAPTER 1 Introduction 1.1 Overview With continued improvements in computer hardware and architecture design, scientists and engineers are attempting numerical simulations of complex phenomena involving increasingly large number of unknowns. The underlying mathematical models are often described by partial di erential equations (PDEs), a discretization of which often results in extremely large and sparse linear systems. An e cient solution approach is the combination of two iterative method techniques, namely, Krylov subspace methods and preconditioning. While Krylov subspace methods are a subject of interest by themselves, this dissertation is mainly devoted to the study of scalable, multilevel and robust preconditioning and issues related to it. A success of iterative methods for large scale computing is their simplicity. They often just need perform simple operations such as matrix-vector product and vector arithmetics. Hence the computer storage is minimal; we need only to store the nonzero entries of the sparse matrix, if necessary, and a few number of vectors. Consequently, the structures of a matrix, for instance, sparsity and storage format, can be fully exploited. On the other hand, however, their convergence rate may be slow. Also, they may not be as robust as direct methods. A standard way to improve both the e ciency and robustness is preconditioning; a powerful technique to improve the condition number of linear systems which often determines the convergence rate of Krylov subspace methods. In fact, the development of e ective preconditioning techniques helps bring the iterative methods into widespread use. There is a wide spectrum of preconditioners. On one extreme of the spectrum, there are black box solvers; they are purely algebraic and do not require any information of the problem from which the linear system arises. Moreover, they are often quite robust with respect to variations in the coe cients of the PDEs. However, their convergence may deteriorate as the problem size increases. On the other extreme, preconditioners can be designed for a speci c class of problems and may require the analytic and geometric information of the problem being solved. They are often characterized by the optimal convergence rate which is independent of the mesh size. However, their convergence may depend on the PDE coe cients and special techniques are needed for speci c problems. Preconditioners between the two extremes, which may be known as gray box solvers, apply techniques of one 1 extreme to the other and hence possess a mixture of their advantages and disadvantages. In fact, one of the goal of this dissertation is to bridge the gap between the two extremes. In particular, we enhance the purely algebraic preconditioners for speci c and yet important problem classes, and on the other hand, generalize the problem-speci c preconditioners to a broader setting. Apart from preconditioning, there are other issues in employing iterative methods. For example, unlike LU factorization, iterative methods are not readily suitable for solving linear systems with multiple right-hand sides which arise frequently in various applications. In this dissertation, we address the issues of iterative methods and the tradeo between generality, parallelism and robustness (fast convergence) for large scale scienti c computing. We intentionally provide a good mix of theory, algorithm and application. As the main theme of the dissertation, we introduce and analyze three of the most useful and interesting scalable preconditioners ranging from algebraic and problem-independent to analytic and problem-speci c preconditioners. Applications to other interesting problems such as image processing are also studied. In addition, we analyze a class of projection methods for solving multiple right-hand side problems and propose a generalization to it. 1.2 Scalable and Multilevel Preconditioners Although there is a broad number of topics, this dissertation has a unifying theme{scalable and multilevel methods. Classical iterative methods such as Jacobi, Gauss-Seidel and SOR were evolved from the algebraic extreme of the spectrum, where a matrix is approximated by the diagonal or the triangular parts of itself. A more sophisticated and powerful example is the incomplete LU factorization preconditioners [34, 98], where the L, U factors of a matrix are approximated by sparse matrices. Since their algorithms resemble Gaussian elimination, they possess the robustness of direct methods. However, their convergence may deteriorate with the problem size. Although there are some analyses regarding the existence and convergence of ILU preconditioners for M -matrices [52, 98, 125], the theoretical aspect is still an open issue. Furthermore, their sequential character makes them hard to parallelize on modern computer architectures. Nevertheless, there is a practical need for general purpose preconditioning techniques, since linear systems arising from applications often do not have many structures other than sparsity to exploit and for those problems, specialized preconditioners may fail. E orts have been made on the parallelization aspect for the symmetric case, the incomplete Cholesky factorization, for example [82]. In Chapter 2, we generalize the parallel implementation to the ILU(0) factorization for general nonsymmetric matrices with a symmetric structure. We also discuss how to tactically add nonzeros to the approximate L 2 and U factors to improve the quality of the ILU(0) preconditioners. Another class of purely algebraic methods is the sparse approximate inverse preconditioners [7, 8, 89]. They construct a sparse approximation to the inverse of a matrix, instead of a sparse approximation to the matrix itself as the ILU factorizations do. They have not receivedmuch attention until recently, where their intrinsic parallelism is recognized. Robust algorithms and theory are still under development. Nevertheless, their potential as an alternative to ILU preconditioners has drawn a lot of attention recently. In contrast to ILU preconditioners, they are designed for parallelism. Yet they have to face other issues such as how to determine an e ective sparsity pattern a priori, and how to maintain parallelism if the sparsity pattern is chosen adaptively. More importantly, there is a potential weakness of sparse approximate inverse preconditioners for solving matrices arising from PDE problems since these inverses are dense and may not be accurately approximated by sparse matrices. In Chapter 3, we improve their e ciency for PDE problems by incorporating the wavelet techniques [41]. These wavelet sparse approximate inverse preconditioners can be considered as gray box solvers belonging to the middle of the spectrum, since they are specialized for linear systems arising from di erential equations. On the other extreme, preconditioners can be designed for a speci c class of problems and may requi