# Presentation: 016087

## Overview of Trilinos Preconditioners(in AztecOO, IFPACK and ML)...and how to get started using them

- Alan Williams, org 1543_[william@sandia.gov](mailto:william@sandia.gov)_with material and input fromMike Heroux, Marzio Sala, and othersReview&Approval: SAND 2005-7145P

## Trilinos – lots of packages,what do I need?

- To decipher all the greek package-names, check_[http://software.sandia.gov/Trilinos/capabilities.html](http://software.sandia.gov/Trilinos/capabilities.html)_
- Many packages build by default.
- Type ‘configure –help’ in main Trilinos directory
- To get started with solving and preconditioning linear systems:
- AztecOO – Krylov subspace iterative methods
- IFPACK – Incomplete factorization package (preconditioners)
- ML – Multilevel solvers/preconditioners
- Amesos direct solvers (more power for IFPACK, ML)
- Use –prefix=<dir> option with configure, then gmake, gmake install

## Preconditioning: (Just to make sure we’re on the same page...)

- Denote the linear system (user’s problem):

- Preconditioning: given M such that:

- Solve the preconditioned system:

- Where:

- The art of preconditioning:

- is close to identity, or nice properties

- can be inverted efficiently

## More basic stuff...

- In practice, usually M1 or M2 is identity, leaving right or left preconditioning, and preconditioner is often referred to as simply “M” or “P”.
- Trilinos Krylov methods:
- Aztec/AztecOO
- Preconditioning performed by Krylov implementations, user not concerned with left vs right, etc.(Aztec applies preconditioners on the right...)
- Other Krylov methods in Trilinos...
- Belos allows user to specify left or right preconditioning

## AztecOO

**AztecOO**

- Krylov subspace solvers: CG, GMRES, Bi-CGSTAB,...
- Incomplete factorization preconditioners
- Aztec is the workhorse solver at Sandia:
- Extracted from the MPSalsa reacting flow code.
- Installed in dozens of Sandia apps.
- 1900+ external licenses.
- AztecOO improves on Aztec by:
- Using Epetra objects for defining matrix and RHS.
- Providing more preconditioners/scalings.
- Using C++ class design to enable more sophisticated use.
- AztecOO interfaces allows:
- Continued use of Aztec for functionality.
- Introduction of new solver capabilities outside of Aztec.

- Developers:
-  Mike Heroux, Alan Williams, Ray Tuminaro

## Preconditioners and Aztec

**“****old-fashioned” Aztec...**
- Original Aztec interfaces allow selection of internal preconditioners by setting the contents of the options and params arrays.
- Options[AZ_precond]:	AZ_none
- 			AZ_Jacobi
- 			AZ_Neumann
- 			AZ_ls
- 			AZ_sym_GS
- 			AZ_dom_decomp
- If AZ_dom_decomp, also set options[AZ_subdomain_solve]:
- 			AZ_lu	(serial)
- 			AZ_ilut
- 			AZ_ilu
- 			AZ_rilu
- 			AZ_bilu
- 			AZ_icc

## A Simple Epetra/AztecOO Program

- // Header files omitted...
- int main(int argc, char *argv[]) {
-   MPI_Init(&argc,&argv); // Initialize MPI, MpiComm
-   Epetra_MpiComm Comm( MPI_COMM_WORLD );

-   // ***** Create x and b vectors *****
-   Epetra_Vector x(Map);
-   Epetra_Vector b(Map);
-   b.Random(); // Fill RHS with random #s

- // ***** Create an Epetra_Matrix  tridiag(-1,2,-1) *****
-   Epetra_CrsMatrix A(Copy, Map, 3);
-   double negOne = -1.0; double posTwo = 2.0;
-   for (int i=0; i<NumMyElements; i++) {
-     int GlobalRow = A.GRID(i);
-     int RowLess1 = GlobalRow - 1;
-     int RowPlus1 = GlobalRow + 1;
-     if (RowLess1!=-1)
-        A.InsertGlobalValues(GlobalRow, 1, &negOne, &RowLess1);
-     if (RowPlus1!=NumGlobalElements)
-        A.InsertGlobalValues(GlobalRow, 1, &negOne, &RowPlus1);
-     A.InsertGlobalValues(GlobalRow, 1, &posTwo, &GlobalRow);
-   }
- A.FillComplete(); // Transform from GIDs to LIDs

- // ***** Map puts same number of equations on each pe *****
-   int NumMyElements = 1000 ;
-   Epetra_Map Map(-1, NumMyElements, 0, Comm);
-   int NumGlobalElements = Map.NumGlobalElements();

- // ***** Report results, finish ***********************
-   cout << "Solver performed " << solver.NumIters()
-           << " iterations." << endl
-           << "Norm of true residual = "
-           << solver.TrueResidual()
-           << endl;
-   MPI_Finalize() ;
-   return 0;
- }

-   // ***** Create/define AztecOO instance, solve *****
-   AztecOO solver(problem);
-   solver.SetAztecOption(AZ_precond, AZ_Jacobi);
-   solver.Iterate(1000, 1.0E-8);

- // ***** Create Linear Problem *****
-   Epetra_LinearProblem problem(&A, &x, &b);

## Preconditioning AztecOO

- Epetra_Crsmatrix* A; //or Epetra_VbrMatrix
- Epetra_MultiVector *x, *b;
- AztecOO azoo(A, x, b);
- ...
- Epetra_Operator* M = // (define preconditioner here...)
- azoo.SetPrecOperator(M);
- azoo.Iterate(max_iters, tolerance);

## IFPACK: Algebraic Preconditioners

**IFPACK: Algebraic Preconditioners**

**Overlapping Schwarz preconditioners with incomplete factorizations, block relaxations, block direct solves.**
**Accept user matrix via abstract matrix interface (Epetra versions).**
**Uses Epetra for basic matrix/vector calculations.**
**Supports simple perturbation stabilizations and condition estimation.**
**Separates graph construction from factorization, improves performance substantially.**
**Compatible with AztecOO, ML, Amesos. Can be used by NOX and ML.**

- Developers:
-  Marzio Sala, Mike Heroux

## IFPACK preconditioners

- Class **Ifpack_AdditiveSchwarz**<T> defines the IFPACK preconditioner
- Overlap is specified (on or off) by a parameter
- If overlap > 0, overlapping matrix allocated rows for overlapping region only
- This class is templated with the local solver T:
- Defines Ai-1
- Must be an Ifpack_Preconditioner derived class
- Ifpack_Preconditioner is a base class of Ifpack_AdditiveSchwarz

## Available Local Solvers

- Simple point relaxation methods:
- Jacobi, Gauss-Seidel, SOR, SSOR
- Block relaxation methods:
- Blocks defined by METIS or a simple greedy algorithm
- Blocks of any size, inverse of each block applied using LAPACK or any IFPACK preconditioner
- Jacobi, Gauss-Seidel, symmetric Gauss-Seidel
- Incomplete factorizations:
- Dropping based on graph
- Dropping based on value
- Any Amesos LU factorization
- Replaces Y12M
- Local solvers (e.g., Ifpack_ILUT, Ifpack_Amesos) are specializations of Ifpack_Preconditioner
- Example: Ifpack_Preconditioner* P = new Ifpack_AdditiveSchwarz<Ifpack_ILUT>(...);

- Effectiveness
- Memory
-  requirements

|  |
| --- |

|  |
| --- |

## IFPACK Example

- Ifpack Factory;
- std::string PrecType(“ILUT”);
- int OverlapLevel = 1;
- Ifpack_Preconditioner* P = Factory.create(PrecType, A, OverlapLevel);
- Teuchos::ParameterList List;
- List.set(“fact: drop tolerance”, 1.e-9);
- List.set(“fact: level-of-fill”, 1);
- P.SetParameters(List);
- P.Initialize();
- P.Compute();
- Azoo.SetPrecOperator(P);

- packages/ifpack/example/Ifpack_ex_Factory.cpp

## Ifpack Parameters - OverlapLevel

- OverlapLevel == 0,	no overlap
- OverlapLevel > 0,	overlap

- P0

- P1

- P2

- No Overlap

- Overlap

## More Ifpack Parameters...

- Choice of local solver: PrecType
- “point relaxation”,  “block relaxation”
- “IC”, “ICT”, “ILU”, “ILUT”
- “Amesos”  (requires Trilinos be configured with –enable-amesos)
- Relaxation parameters, examples:
- List.set(“relaxation: type”, “Jacobi”);
- List.set(“relaxation: sweeps”, 3);
- List.set(“relaxation: damping factor”, 1.0);
- Factorization parameters, examples:
- List.set(“fact: level-of-fill”, 5);
- List.set(“fact: drop tolerance”, 1.e-6);

## ML: Multi-level Preconditioners

**ML: Multi-level Preconditioners**

**Smoothed aggregation, multigrid and domain decomposition preconditioning package**
**Critical technology for scalable performance of some key apps.**
**ML compatible with other Trilinos packages:**
**Accepts user data as Epetra_RowMatrix object (abstract interface). Any implementation of Epetra_RowMatrix works.**
**Implements the Epetra_Operator interface. Allows ML preconditioners to be used with AztecOO and TSF.**
**Can also be used completely independent of other Trilinos packages.**

- Developers:
-  Ray Tuminaro, Jonathan Hu, Marzio Sala

## ML parameters, options

- Useful defaults:
- ML_Epetra::SetDefaults(“SA”, MLList);
- Screen output level
- MLList.set(“output”, 1);
- Value of 0 turns off screen output, higher value gives more output
- Third-party libraries for reordering, load-balancing, etc.
- Configure options include –with-ml_metis, –with-ml_parmetis3x, –with-ml_zoltan, etc.
- Covered in User Manual
- Consult documentation, and/or ML developers Jonathan Hu, Ray Tuminaro

## ML with Epetra: Example of use

**ML with Epetra: Example of use**

- Trilinos_Util_CrsMatrixGallery Gallery(“laplace_3d", Comm);
- Gallery.Set("problem_size", 100*100*100);
- // linear system matrix & linear problem
- Epetra_RowMatrix * A = Gallery.GetMatrix();
- Epetra_LinearProblem * Problem = Gallery.GetLinearProblem();
- // Construct outer solver object
- AztecOO solver(*Problem);
- solver.SetAztecOption(AZ_solver, AZ_cg);
- // Set up multilevel precond. with smoothed aggr. defaults
- ParameterList MLList;      // parameter list for ML options
- ML_Epetra::SetDefaults(“SA”,MLList);
- MLList.set(“aggregation: type”, “METIS”);
- // create preconditioner
- ML_Epetra::MultiLevelPreconditioner * MLPrec = new
-          ML_Epetra::MultiLevelPreconditioner(*A, MLList, true);
- solver.SetPrecOperator(MLPrec); // set preconditioner
- solver.Iterate(500, 1e-12);     // iterate at most 500 times
- delete MLPrec;

**Trilinos/packages/ml/examples/ml_example_MultiLevelPreconditioner.cpp**

- Triutils

- AztecOO

- ML

- AztecOO

## Documentation

**Start with Trilinos web site, information about all packages:**
_**[http://software.sandia.gov/Trilinos](http://software.sandia.gov/Trilinos)**__** **_
**AztecOO User’s Manual:**_**[http://software.sandia.gov/Trilinos/packages/aztecoo/AztecOOUserGuide.pdf](http://software.sandia.gov/Trilinos/packages/aztecoo/AztecOOUserGuide.pdf)**__** **_
**ML User’s Guides etc.**** **
_**[http://software.sandia.gov/Trilinos/packages/ml/documentation.html](http://software.sandia.gov/Trilinos/packages/ml/documentation.html)**_
**Example programs:**
**packages/ml/examples**
**packages/ifpack/example**
**Other Preconditioners, interfaces coming in the future**
**Claps**
**Meros**
**Thyra**