# Presentation: 391021

## Terascale Simulation Tools and Technologies Center

- Brief overview extracted from presentation by
-  Lori Freitag Diachin
- August 2004

## The TSTT Center

*Goal:* To enable high-fidelity calculations based on multiple physics processes over multiple scales
*Adaptive methods*
*Composite or hybrid solution strategies*
*High-order discretization strategies*
*Participants:** *TSTT brings together many meshing and discretization tools
*Structured Grids*:  Overture, ORNL variational techniques
*Unstructured Meshes*: AOMD, MOAB, NWGrid, FronTier

## TSTT data model abstracts PDE-simulation data hierarchy

- Core Data Types
*Geometric Data*: provides a high level description of the boundaries of the computational domain; e.g., CAD, image, or mesh data
*Mesh Data*: provides the geometric and topological information associated with the discrete representation of the computational domain
*Field Data*: provides access to the time dependent physics variables associated with application solution.  These can be scalars, vectors, tensors, and associated with any mesh entity.
- Data Relation Manager
- Provides control of the relationships among two or more the core data types.  It resolves cross references between entities in different groups and provides additional functionality that depends on multiple core data types.

## Interoperability Development Plan

- Functional Tools:
- (e.g. Discretization,

- Mesquite,

- Frontier-Lite, etc.)

- Create new tools that use these interfaces to work with the underlying infrastructures interchangeably
- Mesquite mesh quality improvement
- Discretization library
- Frontier-Lite

- Use TSTT tools interoperably

## An overarching philosophy defines the boundaries of our interface definition efforts

## Mesh entities are a core building block

**Definition**
- Unique type and topology
- Type: Vertex, Edge, Face, Region
- Topology: Point, Line_Segment, Polygon, Triangle, Quadrilateral, Polyhedron, Tetrahedron, Hexahedron, Prism, Pyramid, Septahedron
- Faces and regions have no interior holes
- Higher-dimensional entities are defined by lower-dimensional entities through canonical ordering relationships
- Examples
- Vertex (0D), edge (1D) , triangular face (2D), tetrahedral region (3D)
**Capabilities**
- Return upward and downward first order adjacencies in the canonical ordering
- Support both individual and agglomerated request mechanisms
- Vertices return coordinate information in arrays of doubles
- Add, retrieve, set, and remove user-defined tag data

## TSTT meshes cover a broad spectrum of types and functionalities

**TSTT Mesh Definition**
- A collection of TSTT entities that have uniquely defined entity handles
- Entities are related through topological adjacency information in which higher-dimensional entities are defined by lower dimensional entities
- Examples
- a non-overlapping connected set of TSTT entities (Type 1)
- a collection of Type 1 meshes used to represent the computational domain.  These may or may not be overlapping meshes (Type 2)
**Capabilities for static Type 1 meshes**
- Populate the interface by string name
- Basic query capabilities
- Entity Traversal
- Connectivity Information
- Array or iterator-based
- Add, retrieve, set and remove user-defined tags

## Entity Sets provide subsetting and relations capabilities within a TSTT mesh

- Entity Set Definition
- Arbitrary groupings of TSTT mesh entities
- Multiple entity sets can be associated with a given mesh
- Relationships between entity sets
- Contained-in
- Parent/Child
- Examples:  A set of vertices, the set of all faces on a geometric face, the set of regions in a domain decomposition for parallel computing
- Capabilities
- Static mesh capabilities as before
- Set Operations
- Add and remove existing TSTT entities to the mesh set
- Union, subtract, and intersect entity sets
- Contained-in and Parent/Child relationships

## Terascale computing support under consideration

- Builds on existing, supported functionality
- In the data model (e.g., entity sets)
- In the underlying TSTT tools (e.g., PAOMD)
- Supports distributed mesh representation
- Available services
- Invariant entity handles (unique and unchanged even with mesh migration and mesh modification in parallel)
- Mesh migration: global or local
- In existing external tools (e.g., Zoltan)
- Requires additional interface work
- Local/Global map conventions and behavior
- Convenience functions for ghost node support, etc.
- Mechanisms for mesh and user-data transfer

## Preliminary interface implementations are well underway

## Performance studies will provide ‘best practices’ guidelines

- Working with CCA’s SIDL Babel team to characterize the interface performance
- Focus on entity-based access
- Single entity iterators
- Workset iterators
- Compare
- Native data structures
- Native language interfaces
- SIDL/Babel interfaces

**Workset Iterators**
**Initialize Workset(entity_type, num_entities, &wrk_iter)**
**While (!done){**
**     ****GetNextWorkset(&entity_handles[ ], &num, wrk_iter);**
**     ****GetInfo(entity_handles);**
**     ****for i=0:num-1 {**
**         ****do_something(entity_handle[i]);**
**     ****}**
**}**
**Destroy Workset(wrk_iter);**

## Preliminary Performance Results

**1**

**100**

- As workset size increases, interface overhead amortized
- 15-40% compared to native data structures
- SIDL a few percent more than native language
- Single entity access via workset iterators is expensive
- 2X-4X for native and SIDL interfaces
- Direct, single entity access (no worksets)
- Similar performance to a workset size of 5-10

## The TSTT interface and philosophy already used in a variety of ways

- Immediate
- Prototype CCA components (PNL/LLNL)
- Mesquite mesh quality Improvement (ANL, SNL)
- Frontier/AMR/mesh generation (SUNY SB/LLNL/PNL)
- Adaptivity in SLAC Omega3P code (RPI/SLAC)
- Long Term
- Deployed as CCA components for rapid application development
- Basis of the TSTT Discretization Library
- Design optimization for accelerators (w/ TOPS, SLAC)
- Read TSTT meshes directly into CEMM NIMROD code

## Roadmap for next two years

- Current year
- Release standard mesh interfaces
- Develop a prototype system with meshes, geometry, fields and data relation managers
- Basic, low-level field manipulation interfaces
- Demonstrate utility via pairwise coupling of technologies and pairwise insertion of existing technology into SciDAC apps
- Demonstrate interchangeability with compatible technologies (e.g., partitioning, visualization) and simple applications
- Next year
- Release TSTT mesh components compliant with query interface as CCA components
- Discretization interfaces and start library development/ deployment
- Demonstrate interoperability in SciDAC applications such as AST design optimization for RF cavities

## Contact Information

- TSTT Web Site: www.tstt-scidac.org
- David Brown: dlb@llnl.gov
- Lori Freitag Diachin: diachin2@llnl.gov
- Jim Glimm: glimm@ams.sunysb.edu