# Presentation: 181195

## Thyra from a Developer's Perspective

**Roscoe A. Bartlett**
** ****Department 1411: Optimization and Uncertainty Estimation**
**Sandia National Laboratories**

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Categories of Abstract Problems and Abstract Algorithms

- Linear Problems:
- Linear equations:
- Eigen problems:

- Nonlinear Problems:
- Nonlinear equations:
- Stability analysis:

- Transient Nonlinear Problems:
- DAEs/ODEs:

- Optimization Problems:
- Unconstrained:
- Constrained:

- _Trilinos Packages_

- Belos

- Anasazi

- NOX

- LOCA

- Rythoms

- MOOCHO

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## Common Environments for Scientific Computing

** ****Serial / SMP (symmetric multi-processor)**
-  All data stored in RAM in a single local process

- Code

- Data

- Processor

** ****Out of Core**
-  Data stored in file(s) (too big to fit in RAM)

- Processor

- Disk

- Code

- Data

** ****SPMD (Single program multiple data)**
-  Same code on each processor but different data

- Code

- Data(0)

- Proc 0

- Code

- Data(1)

- Code

- Data(N-1)

- MPP (massively parallel processors)

- ...

- Proc 1

- Proc N-1

**Notes:**
These computer codes may run in a number of different computing enviroments

## Introducing Abstract Numerical Algorithms

**Introducing Abstract Numerical Algorithms**

-   An ANA is a numerical algorithm that can be expressed abstractly solely in terms of vectors, vector spaces, and linear operators (i.e. not with direct element access)

_**Example Linear ANA (LANA) : Linear Conjugate Gradients**_

_**Types of operations**_

_**Types of objects**_

- What is an abstract numerical algorithm (ANA)?

_**Linear Conjugate Gradient Algorithm**_

**Notes:**
So we looked at some different categories of numerical algorithms and some of the different computing environments these algorithms are run in but what is an abstract numerical algorithm?

## Different Platform Configurations for Running ANAs

**SPMD**

- ANA

- LAL

- Proc 0

- APP

- MPP

- ANA

- LAL

- Proc 1

- APP

- ANA

- LAL

- Proc N-1

- APP

- ...

**Master/Slave**

- ANA

- LAL

- Proc 0 (master)

- APP

- MPP

- LAL

- Proc 1

- APP

- ...

**Client/Server**
**Master/Slave**

- LAL

- Proc N-1

- APP

- MPP Server

- LAL

- Proc 1

- APP

- ...

- LAL

- Proc N-1

- APP

- LAL

- Proc 0

- APP

- Client

- ANA

**If a code can not run in these modes it is not an ANA code!**

**Notes:**
Now lets talk about some of the different kinds of computing environment configurations in which we would like to be able to run our scientific software.

What differentiates ANA software from APP and LAL software is that ANA software can be run in much more flexible ways without  dramatically impacting performance in many cases.  This type of potential flexibility does not exist for APP or LAL software.

On this slide I show some interesting computing configurations that demonstrate of the potential flexibility of  ANA software.

Another very interesting configuration is the client/server master/slave configuration.  In this case the ANA can run an entirely different computer than the APP and LAL and drive the computations.  For example, an ANA implemented in Matlab or Java on the highest level running on my laptop here could drive the numerical computations in one of the MPPs here at Sandia.  This may seem far fetched to some but with the interfaces that I describe later this is very realistic and in some cases does not significantly impart performance over SPMD.

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Trilinos Strategic Goals

**Scalable Solvers**: As problem size and processor counts increase, the cost of the solver will remain a nearly fixed percentage of the total solution time.
**Hardened Solvers**: Never fail unless problem essentially unsolvable, in which case we diagnose and inform the user why the problem fails and provide a reliable measure of error.
**Full Vertical Coverage**: Provide leading edge capabilities from basic linear algebra to transient and optimization solvers.
**Universal Interoperability**: All Trilinos packages will be interoperable, so that any combination of solver packages that makes sense algorithmically will be possible within Trilinos.
**Universal Solver RAS**: Trilinos will be:
- Integrated into every major application at Sandia (Availability).
- The leading edge hardened, scalable solutions for each of these applications (Reliability).
- Easy to maintain and upgrade within the application environment (Serviceability).

**Courtesy of Mike Heroux, Trilinos Project Leader**

**Key Point**
- Universal Interoperability will not happen automatically and if not done carefully then can compromise the other strategic goals

**Thyra** is being developed to address this issue

## Interoperability is Especially Important to Optimization

- Numerous interactions exist between abstract numerical algorithms (ANAs) in a transient optimization problem

**Iterative Linear Solvers**

**Operators and Vectors**

**Nonlinear Solvers**

**Nonlinear Optimizers**

**Key Points**
- Higher level algorithms, like optimization, require a lot of interoperability
- Interoperability must be “easy” or these configurations will not be possible
- Many real problems even more complicated

- What is needed to solve problem?
- Standard interfaces to break O(*N**2*) 1-to-1 couplings
- Operators/vectors
- Linear Solvers
- Nonlinear solvers
- Transient solvers
- etc.

**Applications**

**Transient Solvers**

- This is hard!  This level of interoperability for massively parallel algorithms has never been achieved before!

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## Interfacing Packages to Implementations : Everyone for themselves!

**Package 1**

***Package Interface 1***

**Package Interface 1 / Implementation 1 adapter**

- ...

** ****Implementation 1**

** ****Implementation 2**

**Package Interface 1 / Implementation 2 adapter**

**Package Interface 1 / Implementation N adapter**

** ****Implementation N**

- ...

**Package 2**

***Package Interface 2***

**Package Interface 2 / Implementation 1 adapter**

**Package Interface 2 / Implementation 2 adapter**

**Package Interface 2 / Implementation N adapter**

**Package M**

***Package Interface M***

**Package Interface M / Implementation 1 adpater**

**Package Interface M / Implementation 2 adapter **

**Package Interface M / Implementation N adapter**

- ...

- ...

- ...

-  Oh no, M * N adapter subclasses needed!
-  This is _not_ a “scalable” approach!

**Package == ANA**

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## Interfacing Packages to Implementations : Standard Interfaces

**Package 1**

***Package Interface 1***

- ...

** ****Implementation 1**

** ****Implementation 2**

** ****Implementation N**

- ...

**Package 2**

***Package Interface 2***

**Package M**

***Package Interface M***

- + Only M + N adapter subclasses needed!
- + This is a “scalable” approach!

***Std ANA Interface***

**Std ANA Interface / Implementation 1 adapter**

**Std ANA Interface / Implementation 2 adapter**

**Std ANA Interface / Implementation N adapter**

**Package Interface 1 / Std LANA Interface adapter**

**Package Interface 2 / Std LANA Interface adapter**

**Package Interface M / Std LANA Interface adapter**

- ...

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## Interfacing Packages to Packages : Everyone for themselves!

**Package 1**

***Package Interface 1***

- ...

**Package 2**

***Package Interface 2***

**Package Interface 2 / Package Interface 1 adapter**

**Package 3**

***Package Interface 3***

**Package Interface 3 / Package Interface 2 adapter**

**Package Interface 3 / Package Interface 1 adapter**

- -  Major assumption: “Package Interface 1” satisfies the requirements of “Package Interface 2”?

**Package M**

- - Oh no, as many as 1 + 2 + 3 + ... + M-1 = O(M2) adapter subclasses needed if only one-way interactions!

- _Concrete Examples:_
- 1) NOX/LOCA interfaces had to be extended to support Belos interfaces
- 2) NOX/LOCA vector interface does not support MOOCHO vector interface

***Package Interface M***

- ...

- - This is _not_ a “scalable” approach!

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## The Recommended Practice!

**Interfacing Packages to Packages : Standard Interfaces**

***Package Interface 1***

- ...

**Package 2**

***Package Interface 2***

** ****Implementation 1**

***Std ANA Interface***

**Std ANA Interface / Implementation 1 adapter**

**Package Interface 1 / Std ANA Interface adapter**

** ****Implementation 2**

**Std ANA Interface / Implementation 2 adapter**

**Package Interface 2 / Std ANA Interface adapter**

- ...

- + Only M adapters need
- + Each Package knows nothing of other packages

**Package M**

**Package 1**

**The Recommended Practice!**

**Notes:**
Many different kinds of numerical problems and algorithms for solving those problems are used in scientific computing

Just some of the categories of problems and algorithms are:

## Reasons to Adopt Standard Interfaces

- “Automatic” interoperability only comes through “standard” interfaces
- Only way to guarantee interoperability is even possible (without revisions)
- One set of common documentation
- Common set of unit tests for common interface objects
- More uniform access to Trilinos packages for users

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Requirements for Abstract Numerical Algorithms and Thyra

**An important consideration ******** Scientific computing is computationally expensive! **

**Abstract Interfaces for Abstract Numerical Algorithms using Thyra **_**must**_**:**
- Be portable to all ASC (advanced scientific computing) computer platforms
- Provide for stable and accurate numerics
- Result in algorithms with near-optimal storage and runtime performance
- Scientific computing is expensive!

**An important ideal ******** ****A customized hand-code algorithm in Fortran 77 should not provide **_***significant***_** improvements in storage requirements, speed or numerical stability!**

**Abstract Interfaces for Abstract Numerical Algorithms using Thyra **_**should**_**:**
- Be minimal but complete (good object-oriented design principle)
- Support a variety of computing platforms and configurations
- i.e. Serial/SMP, Out-of-Core, SPMD, master/slave and client/server
- Be (relatively) easy to provide new implementations **(Subjective!!!)**
- Not be too complicated **(Subjective!!!)**

**An important ideal ******** ****Object-oriented “overhead” should be constant and not increase as the problem size increases.**

**Notes:**
Wait, before going into detail about the abstract object-oriented TSFCore interfaces we need to clearly understand the requirements for the ANA software that we write using it.

## Foundational Thyra ANA Operator/Vector Interfaces

**The Key to success!**
**Reduction/Transformation Operators**** **
-   Supports all needed vector operations
-   Data/parallel independence
-   Optimal performance

- R. A. Bartlett, B. G. van Bloemen Waanders and M. A. Heroux. Vector Reduction/Transformation Operators, ACM TOMS, March 2004

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Three Different Types of Use Cases for Thyra ANA Interfaces

**Three Different Types of Use Cases for Thyra ANA Interfaces**

**Package i**

**Implementation k**

**Implementation l**

**Package j**

**Thyra ANA Support**

**Thyra ANA Interface**

- Thyra ANA Interoperability Interfaces
- Defines basic functionality needed for ANAs
- Critical for scalable interoperability!

- Thyra ANA Support Software
- Defines conveniences to aid in writing ANAs
- Avoids duplication of effort
- Useful but optional!

- Thyra Adapters Support Software
- Defines infrastructure support and concrete implementations to make it easy to provide concrete implementations for Thyra ANA interfaces
- Avoids duplication of effort
- Useful but optional!

**Thyra Adapter Support**

- The question should not be _**if**_ this ANA support and adapter support software should exist but instead the question should be _**where**_ this software should be placed and what relationship it will have to the Thyra ANA interoperability interfaces

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Thyra Adapter Support for APP-specific Scalar Products

- Linear operators must “obey” application-specific scalar product <x,y>
- Specifically, for the linear operator (i.e. LinearOpBase, MultiVectorBase etc.):
- the following adjoint relation must hold:
- Goal of Thyra adatpers support subclasses
-  => Separate the definition of application-specific scalar product from data-structure and computing platform concrete implementations of vector spaces, vectors, multi-vectors and linear operators as much as possible.

***VectorSpaceBase***

***LinearOpBase***

***ScalarProdBase***

**EuclideanScalarProd**

**LinearOpScalarProd**

- rangeScalarProdVecSpc, domainScalarProdVecSpc

***ScalarProdVectorSpaceBase***

***EuclideanLinearOpBase***

- op

**AppSpecificScalarProd**

- ...

**ConcreteMultiVector**

- ...

**ConcreteVectorSpace**

- ...

## Automatic Compatibility of Serial and SPMD (Multi)Vectors 

**A simple principle for compatibility of (Multi)Vectors**
-    As set of vector (or muliti-vector) objects *v**1**, v**2**, ... v**n *should be automatically and efficiently compatible, regardless of their concrete implementation, if their coefficients are co-located in the same address space, period!
- Example: If two vectors *v**1* and *v**2* in an SPMD program have their elements partitioned to processors in the same way then these two concrete implementations should be interchangable for any client software that fills/accesses these vectors.
- Anti-example: TNT vectors are not compatible with any (nontemplated) software that accepts std::vector representations.
- This type of compatibility for serial and SPMD vectors and multi-vectors is automatically supported as built in to the basic Thyra interfaces VectorBase and MultiVectorBase!

*v**2*

*v**1*

- P0

- P1

- P2

- P3

## Thyra Interface Support for (Multi)Vector Compatibility

- Automatic (Multi)Vector object compatiblity is supported through explicit cooefficient views:

- virtual void getSubVector( const Range1D& rng, RTOpPack::SubVectorT<Scalar>* sub_vec ) const;
- virtual void freeSubVector( RTOpPack::SubVectorT<Scalar>* sub_vec ) const;
- virtual void getSubVector( const Range1D& rng, RTOpPack::MutableSubVectorT<Scalar>* sub_vec );
-   virtual void commitSubVector( RTOpPack::MutableSubVectorT<Scalar>* sub_vec );

- template<class Scalar>
- class VectorBase : virtual public MultiVectorBase<Scalar> {
- public:
-   ...
-   virtual void getSubVector( const Range1D& rng, RTOpPack::SubVectorT<Scalar>* sub_vec ) const;
-   virtual void freeSubVector( RTOpPack::SubVectorT<Scalar>* sub_vec ) const;
-   virtual void getSubVector( const Range1D& rng, RTOpPack::MutableSubVectorT<Scalar>* sub_vec );
-   virtual void commitSubVector( RTOpPack::MutableSubVectorT<Scalar>* sub_vec );
-   ...
- };
- template<class Scalar>
- class MultiVectorBase : virtual public LinearOpBase<Scalar> {
- public:
-   ...
-   virtual void getSubMultiVector(
-     const Range1D &rowRng, const Range1D &colRng
-     ,RTOpPack::SubMultiVectorT<Scalar>  *sub_mv ) const;
-   virtual void freeSubMultiVector( RTOpPack::SubMultiVectorT<Scalar>* sub_mv ) const;
-   virtual void getSubMultiVector(
-     const Range1D &rowRng, const Range1D &colRng
-     ,RTOpPack::MutableSubMultiVectorT<Scalar> *sub_mv );
-   virtual void commitSubMultiVector( RTOpPack::MutableSubMultiVectorT<Scalar>* sub_mv );
-   ...
- };

- The types Sub(Multi)VectorT and MutableSub(Multi)VectorT simply store raw pointers to memory and information about where they can from the host (Multi)Vector.

- Consequence: If you know the partitioning of elements to processors then you can access the cooefficients of any (Multi)Vector object without any dynamic casting!

## MPI-based SPMD Operator/Vector Node Subclasses 

- MPIVectorSpaceBase gives the number of elements on each processor and the local offset
- Provides near optimal implementations supporting RTOp and using Teuchos::BLAS
- Concrete subclasses must just provide contiguous views of data on a processor

- (Multi)Vectors are automatically compatible if their (range) space object support the MPIVectorSpaceBase interface! => No dynamic casting of (Multi)Vector objects!

- MPILinearOpBase provides explicit access to (multi)vector elements and takes care of the range and domain spaces.

## Concrete MPI-based SPMD Operator/Vector Subclasses 

- Concrete Vector and MultiVector subclasses accept RefCountPtr-wrapped pointers to raw array data or can construct the arrays internally
- MPIVectorStd and MPIMultiVectorStd can be used as base classes when explicit contiguous storage is used by implementation.
- MPIMultiVectorStd provides highly efficient implementations of the contiguous and non-contiguous subView() functions.

- These concrete subclasses provide highly efficient, (near) optimal implementations of vectors and multi-vectors for most use cases.

## Steps to Implementing Concrete Thyra ANA Objects 

**Steps to Implementing Concrete Thyra ANA Objects **

- Using an SMP or SPMD environment?

- Can provide contiguous views of (multi)vector data?

- Using contiguous storage on each processor for (multi)vector data?

- Using MPI?

- Create concrete subclasses deriving from MPIVectorSpaceBase and MPIMultiVectorBase

- Use prewritten MPIVectorSpaceStd subclass which creates MPIVectorStd and MPIMultiVectorStd

- No

- Create concrete subclasses from VectorSpaceDefaultBase, VectorDefaultBase, and MultiVectorDefaultBase

- Create concrete linear operator subclasses by deriving from MPILinearOPBase

- Create concrete linear operator subclasses by deriving from LinearOpBase

- Yes

- Yes

- No
- (currently)

- No

- No
- (currently)

- Yes

- If: (a) Using SPMD and MPI for communication, (b) All (multi)vector data is uniquely stored in contiguous arrays on each processor
- Then: the xxxStd subclasses should be near optimal for most if not all use cases!

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Epetra/Thyra Operator/Vector Adapters 

**Epetra/Thyra Operator/Vector Adapters **

***Thyra::EpetraLinearOpBase***

- Current Epetra/Thyra adatpers are a mix of “real” adapters and “fake” adatpers

**Thyra::EpetraLinearOp**

- Interoperability interface
- * Provides Epetra_Operator views

***Epetra_Operator***

**Epetra_MultiVector**

**Epetra_Map**

- ...

***Thyra::MPIVectorSpaceStd***

***Thyra::MPIVectorSpaceBase***

- domain
- range

***Thyra::MPIMultiVectorStd***

- These _low-overhead_ conversions are performed using the Epetra/Thyra wrapper functions in Thyra_EpetraThyraWrappers.hpp

***Thyra::LinearOpBase***

- This style of adatpers:
- Allows for automatic compatibility of all *similar* (Multi)Vector object implementations
- Requires less code to maintain than the “real” adapters
- Provides greater opportunities for optimization of ANA-only operations
- Can be changed _without_ affecting clients by using wrapper functions
- Outstanding issues:  Flop counts?  Others ???

## Performance of Epetra/Thyra Adatpers vs. Pure Epetra

-                                epetra    thyra     thyra/epetra
-                                --------  --------  ------------
- FULL:      Apply Operator    : 0.153514  0.149969  0.14886
- FULL:      MvTimeMatAddMv    : 0.278658  0.27842   0.278105
- FULL:      MvTransMv         : 0.185737  0.185776  0.185522
- CONT-VIEW: Creation          : 1.3e-05   3.9e-05   3.2e-05
- CONT-VIEW: Apply Operator    : 0.151971  0.148705  0.14631
- CONT-VIEW: MvTimeMatAddMv    : 0.278541  0.278497  0.27983
- CONT-VIEW: MvTransMv         : 0.1857    0.185799  0.1864
- VIEW:      Creation          : 1.5e-05   0.116395  0.116881
- VIEW:      Apply Operator    : 0.328524  0.147952  0.146495
- VIEW:      MvTimeMatAddMv    : 0.38062   0.281685  0.279953
- VIEW:      MvTransMv         : 0.28367   0.185861  0.185714

-                                epetra     thyra      thyra/epetra
-                                --------   --------   ------------
- FULL:      Apply Operator    : 0.2445893  0.2413893  0.2429688
- FULL:      MvTimeMatAddMv    : 0.2813016  0.2760603  0.2809743
- FULL:      MvTransMv         : 0.1958322  0.189001   0.1878586
- CONT-VIEW: Apply Operator    : 0.2400045  0.238128   0.2429735
- CONT-VIEW: MvTimeMatAddMv    : 0.2892126  0.3099845  0.3125794
- CONT-VIEW: MvTransMv         : 0.1893584  0.1986764  0.2006157
- VIEW:      Apply Operator    : 0.3461629  0.7728371  0.7761013
- VIEW:      MvTimeMatAddMv    : 0.4013208  0.2982232  0.3071349
- VIEW:      MvTransMv         : 0.3055691  0.3063636  0.305717

- _Times originally reported by Chris Baker _

- _After minor modification to Thyra::SerialMultiVectorStd and Thyra::MPIMultiVectorStd_

- 1) Apply Operator:
-   V = A * X
-   where: X and V
-          are numel x n

- 2) Block multivector update
-    (MvTimeMatAddMv)
-   Z = alpha*Z + X * T
-   where: Z is numel x n, X is numel x m,
-          and T is m x n

- 3) Block inner (i.e. dot) product
-    (MvTimesMv)
-   T = X * Y
-   where: X is numel x m,
-          Y is numel x n,
-          and T is m x n
- In these tests numel=50000, m=100,
- and n=5.

- Similar performance for whole multi-vectors and contiguous multi-vector views

- Both horrible and decent performance for non-contiguous multi-vector views (i.e. first and last columns interchanged)

- Overall, better performance for Epetra/Thyra adapters!

**Key Point**
- Dedicated yet general ANA concrete implementations can out perform even heavily used implementations like Epetra

- Test program put together by Chris Baker for Anasazi

## Outline

- Quick overview of abstract numerical algorithms (ANAs), Trilinos software, interfaces, and computing environments
- Why interoperability and standard interfaces are important
- Requirements for Thyra and the Fundamental ANA Operator/Vector Interfaces
- Three different general categories of use cases for Thyra ANA interfaces and required vs. optional software and capability vs. dependency
- Thyra ANA operator/vector adapter support software
- Current Epetra/Thyra adapters or lack of?
- Trilinos Thyra package structure

## Trilinos Package Structure Related to Thyra

**Trilinos Package Structure Related to Thyra**

**thyra**

**epetra/thyra**

**anasazi/thyra**

**aztecoo/thyra**

**epetraext/thyra**

**amesos/thyra**

**capo**

**rythmos**

**rythmos/epetraext**

**capo/epetraext**

- The ‘thyra’ package apprears early in the build process
- Thyra adapters are placed in the packages where the concrete implementations reside
- Allows for scalable growth in the number of Thyra adapters
- Encourages native package developers to take ownership of their Thyra adatpers (Thanks Ansasizi developers!)
- CAPO and Rythmos are written directly in terms of Thyra interfaces!
- Guidance: Ask me or another Thyra developer to review your Thyra adapters

## Summary

**Thyra **Interfaces provide** **minimal but efficient connectivity between ANAs and linear algebra implementations and applications
**Thyra** is the critical standard for interoperability between ANAs in Trilinos
**Thyra** can be used in Serial/SMP, SPMD, client/server and master/slave
**Thyra** provides a growing set of optional utilities for ANA development and subclass implementation support
**Thyra** adapters are available for Epetra, Amesos, AztecOO, Anasazi, and Rythmos with others on the way (e.g. Belos, NOX, MOOCHO ...)
_**Trilinos website**_
-    http://software.sandia.gov/trilinos

## The Principle of Control

**The Principle of Control**

- The ANA support and adapter support software in Thyra is like the machines that support Zion:

- “If we want we can shut these machines down”

- “That’s it, you hit it, that’s control”

- “So do we really have control?”

- “So you are saying that we need the machines and they need us”

**Key Point**
- This software is for our convenience but we can throw it away if we want to and not loose _**any**_ interoperability!