# Presentation: 431515

## What’s New for EpetraMichael A. HerouxSandia National Laboratories

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

**Notes:**
Questions from audience

Can you use another matrix-vector multiply?
Are multivectors supported?
Are Petsc and Hypre available via TSF?
How does Trilinos differ from Petsc?

## Outline

- File I/O.
- Matlab connection.
- 2D objects.

## Typical Flow of Epetra Object Construction

- Construct Comm

- Construct Map

- Construct *x*

- Construct *b*

- Construct *A*

-  Any number of Comm objects can exist.
-  Comms can be nested (e.g., serial within MPI).

-  Maps describe parallel layout.
-  Maps typically associated with more than one comp object.
-  Two maps (source and target) define an export/import object.

-  Computational objects.
-  Compatibility assured via common map.

## Trilinos/packages/epetraext/test/inout/ (Part 1)

- EpetraExt::BlockMapToMatrixMarketFile("Test_map.mm", *map, "Official EpetraExt test map",
-                    "This is the official EpetraExt test map generated by the EpetraExt regression tests");
- EpetraExt::RowMatrixToMatrixMarketFile("Test_A.mm", *A, "Official EpetraExt test matrix",
-                    "This is the official EpetraExt test matrix generated by the EpetraExt regression tests");
- EpetraExt::VectorToMatrixMarketFile("Test_x.mm", *x, "Official EpetraExt test initial guess",
-                    "This is the official EpetraExt test initial guess generated by the EpetraExt regression tests");
- EpetraExt::VectorToMatrixMarketFile("Test_xexact.mm", *xexact, "Official EpetraExt test exact solution",
-                    "This is the official EpetraExt test exact solution generated by the EpetraExt regression tests");
- EpetraExt::VectorToMatrixMarketFile("Test_b.mm", *b, "Official EpetraExt test right hand side",
-                    "This is the official EpetraExt test right hand side generated by the EpetraExt regression tests");

## Trilinos/packages/epetraext/test/inout/ (part 2)

- Epetra_Map * map1;
- Epetra_CrsMatrix * A1;
- Epetra_Vector * x1;
- Epetra_Vector * b1;
- Epetra_Vector * xexact1;
- EpetraExt::MatrixMarketFileToMap("Test_map.mm", comm, map1);
- if (map->SameAs(*map1))
-   if (verbose) cout << "Maps are equal.  In/Out works." << endl;
- else
-   if (verbose) cout << "Maps are not equal.  In/Out fails." << endl;
- EpetraExt::MatrixMarketFileToCrsMatrix("Test_A.mm", *map1, A1);
- EpetraExt::MatrixMarketFileToVector("Test_x.mm", *map1, x1);
- EpetraExt::MatrixMarketFileToVector("Test_xexact.mm", *map1, xexact1);
- EpetraExt::MatrixMarketFileToVector("Test_b.mm", *map1, b1);

## Inout Summary

- Reads from/writes to Matlab or Matrix Market compatible files.
- Works for any distributed map, matrix, vector or multivector.
- If map is read in on the same number of processor it was written to, the map layout will be preserved.

## Matlab subpackage in EpetraExt

- If you have Matlab, you can:
- Put:
- RowMatrix, SerialDenseVector/Matrix, Vector/MultiVector objects into a Matlab engine.
- Send commands to Matlab as strings.
- Get:
- CrsMatrix, SerialDenseVector/Matrix, Vector/MultiVector objects from a Matlab engine.
- Supports:
- Use of Matlab preconditioner with Trilinos solvers.
- Interrogation of Epetra objects in Matlab.
- Full support for distributed Epetra objects.
- Details in Trilinos/packages/epetraext/doc/matlab.README.

## Details about Epetra Maps

- Note:  Focus on Maps (not BlockMaps).
- Getting beyond standard use case...

## 1-to-1 Maps

*1-to-1 map* (defn):  A map is 1-to-1 if each GID appears only once in the map (and is therefore associated with only a single processor).
- Certain operations in parallel data repartitioning require 1-to-1 maps.  Specifically:
- The source map of an import must be 1-to-1.
- The target map of an export must be 1-to-1.
- The domain map of a 2D object must be 1-to-1.
-  The range map of a 2D object must be 1-to-1.

## 2D Objects: Four Maps

- Epetra 2D objects:
- CrsMatrix
- CrsGraph
- VbrMatrix
- Have four maps:
**RowMap**: On each processor, the GIDs of the **rows** that processor will “manage”.
**ColMap**: On each processor, the GIDs of the **columns** that processor will “manage”.
**DomainMap**: The layout of domain objects 	(the *x* vector/multivector in *y=Ax*).
**RangeMap**: The layout of range objects 	(the *y* vector/multivector in *y=Ax*).

- Must be 1-to-1 maps!!!

- Typically a 1-to-1 map

- Typically **NOT **a 1-to-1 map

## Sample Problem

- =

*y*

*A*

*x*

## Case 1: Standard Approach

- RowMap 	= {0, 1}
- ColMap 	= {0, 1, 2}
- DomainMap 	= {0, 1}
- RangeMap 	= {0, 1}

- First 2 rows of *A*, elements of *y* and elements of *x*, kept on PE 0.
- Last row of *A*, element of *y* and element of *x*, kept on PE 1.

**PE 0 Contents**

**PE 1 Contents**

- RowMap 	= {2}
- ColMap 	= {1, 2}
- DomainMap 	= {2}
- RangeMap 	= {2}

- Notes:
- Rows are wholly owned.
- RowMap=DomainMap=RangeMap (all 1-to-1).
- ColMap is NOT 1-to-1.
- Call to FillComplete: **A.FillComplete(); // Assumes**

- =

*y*

*A*

*x*

- Original Problem

## Case 2: Twist 1

**Case 2: Twist 1**

- RowMap 	= {0, 1}
- ColMap 	= {0, 1, 2}
- DomainMap 	= {1, 2}
- RangeMap 	= {0}

- First 2 rows of *A*, first element of *y* and last 2 elements of *x*, kept on PE 0.
- Last row of *A*, last 2 element of *y* and first element of *x*, kept on PE 1.

**PE 0 Contents**

**PE 1 Contents**

- RowMap 	= {2}
- ColMap 	= {1, 2}
- DomainMap 	= {0}
- RangeMap 	= {1, 2}

- Notes:
- Rows are wholly owned.
- RowMap is NOT = DomainMap 	     is NOT = RangeMap (all 1-to-1).
- ColMap is NOT 1-to-1.
- Call to FillComplete: **A.FillComplete(DomainMap, RangeMap);**

- =

*y*

*A*

*x*

- Original Problem

## Case 2: Twist 2

- RowMap 	= {0, 1}
- ColMap 	= {0, 1}
- DomainMap 	= {1, 2}
- RangeMap 	= {0}

- First row of *A*, part of second row of *A, *first element of *y* and last 2 elements of *x*, kept on PE 0.
- Last row, part of second row of *A, *last 2 element of *y* and first element of *x*, kept on PE 1.

**PE 0 Contents**

**PE 1 Contents**

- RowMap 	= {1, 2}
- ColMap 	= {1, 2}
- DomainMap 	= {0}
- RangeMap 	= {1, 2}

- Notes:
- Rows are NOT wholly owned.
- RowMap is NOT = DomainMap 	     is NOT = RangeMap (all 1-to-1).
- RowMap and ColMap are NOT 1-to-1.
- Call to FillComplete: **A.FillComplete(DomainMap, RangeMap);**

- =

*y*

*A*

*x*

- Original Problem

## What does FillComplete Do?

- A bunch of stuff.
- One task is to create (if needed) import/export objects to support distributed matrix-vector multiplication:
- If ColMap  DomainMap, create Import object.
- If RowMap  RangeMap, create Export object.
- A few rules:
- Rectangular matrices will *always* require:
- A.FillComplete(Domain,RangeMap);
- DomainMap and RangeMap *must be 1-to-1.*

## Summary

- EpetraExt supports deterministic parallel I/O to/from Matlab and Matrix Market format.
- EpetraExt provides a coupling between Matlab/Epetra for:
- Prototyping code, e.g., preconditioners, in Matlab and
- Injecting Epetra objects into Matlab.
- 2D objects:
- Epetra provides very flexible support for:
- Placing any matrix entry on any processor, and
- Even sharing a matrix entry across processors.