# Presentation: 352596

## The Terascale Simulation Tools and Technologies Center

- James Glimm
- David Brown
-  Lori Freitag Diachin
- March 2004

## The TSTT Center

*Goal:* To enable high-fidelity calculations based on multiple coupled physical processes and multiple physical scales
*Adaptive methods*
*Composite or hybrid solution strategies*
*High-order discretization strategies*
*Barrier:* The lack of easy-to-use interoperable meshing, discretization, and adaptive tools requires too much software expertise by application scientists
*   **The TSTT center recognizes this gap and is addressing the technical and human barriers preventing use of adaptive, composite, hybrid methods*

## TSTT brings a broad set of meshing-discretization communities together

- Diverse Partners
*Overture*
*AOMD/Trellis** *
*CUBIT *
*NWGrid/Phys*
* *
*Frontier *
*Mesquite  *
*Nek5000*

- Meshing Technologies
- Structured/Overset
- Mixed element hybrid
- Unstructured hex
- Tetrahedral
- Variational approaches
- Mesh quality improvement
- Adaptive Technologies
- Mesh refinement
- Front tracking
- Software Frameworks and PDE Solvers
- Significant Applications Experience

## The Challenge

- These tools all meet particular needs, but
- They do not interoperate to form hybrid, composite meshes
- They cannot be easily interchanged in an application
- In general the technology requires too much software expertise from application scientists
- Difficult to improve existing codes
- Difficult to design and implement new codes
- Our goal is to develop interchangeable and interoperable software components for meshing and discretization

## We are addressing SciDAC applications needs through a two-pronged approach

*Near term:** *deployment of current TSTT mesh and discretization capabilities by partnering with SciDAC applications
*Longer term:** *development of interoperable software tools enabling
- Rapid prototyping of new applications
- Plug-and-play insertion of mesh and discretization technology through uniform software interfaces

## TSTT mesh generation has enhanced Tau3P accelerator design capabilities

- Tau3P is a successful time-dependent electromagnetic design code for SLAC
- The basic numerical method in Tau3P (DSI scheme) is unstable on general meshes
- Instability is mesh-quality related
- Time-series filter is often successful, but unreliable
- In collaboration with SLAC scientists, TSTT has
- Custom-built meshes with attention to quality requirements
- Carefully studied dependence of instability on mesh quality metrics
- Consulted on the next generation, FEM-based code
- Investigated alternative discretization approaches
- Stable, hybrid mesh DSI schemes
- Maxwell solvers on overset grids using Overture

## TSTT-generated mesh enabled first long-time PEP-II beam heating study

- (T. Tautges SNL,
- N. Folwell, Cho-Kuen Ng , SLAC)

- Hex mesh for Tau3P generated from CAD model meeting specific requirements:
- orthogonal on the beam axis and
- overall good quality in aspect ratios and angle of elements
- made extensive use of advanced capabilities and tools in CUBIT and CGM
- First successful Tau3P run with a transit beam resulting in direct calculation of the mode spectrum of the PEP-II IR complex
*Resulted in recent decision to use Tau3P as vehicle for further PEP-II IR design studies*

- _Computed Excited Fields_

## TSTT/SLAC collaboration provides adaptive mesh capabilities for Omega3P

*Omega3P* is SLAC’s quadratic FEM eigensolver for computing normal modes in RF cavities
- Extremely accurate solutions (0.01% error) are required for requirements of accelerator design
- Using new TSTT technology developed at RPI, adaptive mesh capability has been provided for Omega3P
- Simulations achieve same accuracy with 1/3 the number of unknowns

- 3 levels of refinement

- First Refinement

- Final Adaptive Mesh

- Initial Mesh

## Omega3P adaptivity provided through interoperation of multiple tools 

## Optimal mesh generation and adaptive methods reduce error in climate applications

- Application PI: Drake (ORNL)
- TSTT PI: Khamayseh (ORNL)
*Goal:* Given an initial isotropic or anisotropic planar or surface mesh and a solution field with large gradient mountain heights, use solution based r-adaptation to minimize solution error
*Progress*:
- Several different meshing strategies developed for structured and unstructured surfaces
- Proof of principle of meshing technologies demonstrated, integrated in next generation climate codes
- Improving the prediction of rainfall, snowfall and cloud cover in regional weather models

- Orography field showing high altitude over the Himalayas and alps; ** **Structured adapted spherical mesh, hybrid geodesic mesh, and unstructured mesh based on orographic field data

## TSTT technologies used to understand microbial cell floc behavior

- TSTT PI: Trease
*Goal*: To understand the behavior of Shewanella microbe flocs in oxygen rich environments
*Progress**:*
- Floc geometry built using image reconstruction techniques from a stack of confocal images
- TSTT unstructured mesh generated using NWGrid
- Solve reaction-diffusion equations to find the concentration of oxygen in the floc

## Long Term Strategy

- Create interoperable meshing and discretization components
- Common interfaces for mesh/geometry query and modification
- Initial design will account for interoperability at all levels
- Encapsulate existing TSTT software tools into CCA-compliant components for plug and play
- Develop new technologies as needed to enable interoperability
- Mesh quality improvement for hybrid meshes
- AMR/Front tracking schemes
- High-level discretization library

## Interoperability Development Plan

- Use TSTT tools interoperably

## TSTT data model abstracts PDE-simulation data hierarchy

- Core Data Types
*Geometric Data*: provides a high level description of the boundaries of the computational domain; e.g., CAD, image, or mesh data
*Mesh Data*: provides the geometric and topological information associated with the discrete representation of the computational domain
*Field Data*: provides access to the time dependent physics variables associated with application solution.  These can be scalars, vectors, tensors, and associated with any mesh entity.
- Data Relation Manager
- Provides control of the relationships among two or more the core data types.  It resolves cross references between entities in different groups and provides additional functionality that depends on multiple core data types.

## An overarching philosophy defines the boundaries of our interface definition efforts

## Preliminary interface implementations are well underway

## The TSTT interface and philosophy already used in a variety of ways

- Immediate
- Prototype CCA components (PNL/LLNL)
- Mesquite mesh quality Improvement (ANL, SNL)
- Frontier/AMR/mesh generation (SUNY SB/LLNL/PNL)
- Adaptivity in SLAC Omega3P code (RPI/SLAC)
- Long Term
- Deployed as CCA components for rapid application development
- Basis of the TSTT Discretization Library
- Prototype edge-plasma fluid code (LLNL)
- Design optimization for accelerators (w/ TOPS, SLAC)
- Read TSTT meshes directly into CEMM NIMROD code

## Mesquite Mesh Quality Improvement

- Stand alone library of mesh quality improvement algorithms
- 2D/3D unstructured and hybrid meshes
- Triangles, Quads, Tets, Hex elements
- State-of-the-art optimization-based improvement schemes
*Joint work with TOPS created fastest mesh quality optimizer in Mesquite tool suite – improves 1M elements in 15 seconds*
- Uses TSTT interface to access mesh and geometry information
- Vertex coordinates, element connectivity, tags
- Successfully integrated with AOMD, MOAB, Overture, Cubit
- Application uses
- Surface mesh smoothing on complex geometries for accelerator modeling
- Mesh quality analysis for SLAC Tau3P code
- Mesh deformation in design optimization problems
- Anisotropic smoothing for error minimization

## TSTT developed a new interoperable AMR/Front tracking scheme

- Combine front tracking in Frontier with adaptive techniques in Overture
- Increase problem resolution using AMR
- Maintain exact interface representation capabilities
- Applications
- Richtmeyer-Meshkov instability
- Diesel jet spray break up
- Using TSTT interface to combine front-tracking schemes with NWgrid capabilities

## Summary

-    The TSTT Center focuses on interoperable meshing and discretization strategies on complex geometries
- Short term impact through technology insertion into existing SciDAC applications
- Long term impact through the development of
- a common mesh interface and interoperable and interchangeable mesh components
- new technologies that facilitate the use of hybrid solution strategies
- Working with SciDAC ISICs to ensure applicability of tools and interfaces

## TSTT Team PIs

- Mark Shephard
- Rensselaer Polytechnic
-            Harold Trease
-                    Pacific NW Natl Lab
-                  Xiao-Lin Li
-                     SUNY Stony Brook
- Joe Flaherty
- Rensselaer Polytechnic

- Jim Glimm
- Brookhaven National Lab
- SUNY Stony Brook
- David Brown
- Lawrence LivermoreNational Laboratory
- Lori Freitag Diachin
- Lawrence Livermore
- National Laboratory

- Center Co- PIs

- Site PIs

- Pat Knupp
- Sandia National Laboratories

-               Paul Fischer
-      Argonne National Laboratory

- Ed d’Azevedo
- Oak Ridge Nat’l Lab

## Contact Information

- TSTT Web Site: www.tstt-scidac.org
- David Brown: dlb@llnl.gov
- Lori Freitag Diachin: diachin2@llnl.gov
- Jim Glimm: glimm@ams.sunysb.edu