# Presentation: 706584

##  Adaptive Simulation Loop for Omega3P

**X. Li, M.S. Shephard, E. Seol, A.C. Bauer, Y. Luo**

**Scientific Computation Research Center, Rensselaer Polytechnic Institute**

**Stanford Linear Accelerator Center (SLAC), DOE and Stanford University**

**L. Ge, I. Malik, K. Ko, Z. Li**

_**A Collaboration under DOE’s SciDAC program**_

- TSTT

- Outline
- Issues, goals and overall approach
- Ingredients to constructing a serial adaptive loop
- Results to date
- Efforts on the parallel adaptive loop

## Constructing Adaptive Solution Procedures

- Components of an an adaptive mesh control loop
- High level problem definition
- Spatial discretization (mesh generation)
- Equation discretization
- Solvers to deal with the large systems of equations
- Error estimators
- Correction indicators
- Discretization enrichment methods
- Implementation
- Tightly coupled using a single set of structures
- Advantage is more optimal and computationally efficient if done well
- Disadvantage is complex algorithm and code development
- More important with “frequent” adaptation steps
- Loosely coupled building on existing components
- Advantage is ability to take advantage of existing analysis codes and adaptive tools and to mix and match tools
- Disadvantage is the overhead of multiple structures and data conversion

**Mesh/Model Relationship**

- Critical to all aspects of the simulation process
- Relationship is termed “classification”
- Mesh Classification: Unique association of a mesh entity, *M**i**d**i*, to a geometric model entity, *G**j**d**j*, where *d**i**<d**j* is denoted by
-                  *M**i**d**i         **G**j**d**j*
-          indicates the left-hand entity (or set) represents a portion of the right-hand entity in the discretization
-     Multiple *M**i**d**i* classified on a *G**j**d**j*
- Boundary mesh entities are identifiedin terms of their classifications
- Classification critical to supporting adaptive simulations and high level problem definitions

**Notes:**
The model and mesh are related using the concept of “classification”, which defines the unique association between a set of mesh entities and a model entity. 

## Adaptive Mesh Control

**Adaptive Mesh Control**

- Two approaches
- Re-meshing
- Define new mesh size field
- Provide mesh generator with domain definition and mesh size field
- Map any required history dependent solution fields for the old to new mesh
- Local mesh modifications
- Determine local operations needed to perform desired modifications
- Execute the local modifications
- Apply incremental history dependent solution field updates
- Used generalized mesh modification

## Muzzle Blast 

- t=0.0

- t=2e-4

- t=5e-4

**Notes:**
This slide shows a section of a cannon, and the adaptive simulation results of muzzle blast at different time. The right pictures show the density contours and the left pictures shows the mesh

Next page

## Mesh Adapted Using Mesh Modification Tool

- interior mesh
- on xz plane

- initial mesh

- adapted mesh

- velocity contours
- on xz plane

- inlet

- section
- B-B

- B

- B

## Field Interface Procedures 

- Fields are tensors defined in terms of basis and multipliers defined over mesh entities
- Solution information needed for queries and operations - in this case error estimation
- Maintain representation over the problem domain in a manner that is consistent with the solution process
- Operators needed to access fields at the model and mesh
- Used to support error estimation
- Used in other projects to support global and local solution transfer

## Adaptive Loop for Accelerator Design 

- Complex geometry defined in CAD modeler
- Physics modeling by the Omega3P code from SLAC
- Solves Maxwell’s equations in frequency domain
- Addresses Electric and magnetic fields
- Determine power loss for specific modes
- High level modeling accuracy needed
- E.g., 0.01% error in frequency predictions
- Adaptive mesh control needed to provide accuracy needed
- Serial adaptive loop constructed using unmodified Omega3P
- Parallel adaptive loop may require adding a parallel file

- TSTT

## Providing the Adaptive Loop Components  

- Components added to construct adaptive loop
- Automatic mesh generation directly from CAD
*a-posteriori* error indication procedures to define mesh size field
- General mesh modification procedures to construct requested mesh size field accounting for the CAD geometry
- Approach to the coupling of the components
- Interoperability procedures to provide information needed by components
- Geometry - support the integration to multiple CAD systems
- Mesh - support the integration to various mesh generation and modification procedures
- Field - support the integration with different analysis procedures
- Interoperability procedures used in this example from
- TSTT
- SCOREC
- Simmetrix

## Integration of Components

## Operator Interfaces

**Using *****geometry operators*****means alternate solid ****modelers can be inserted**

**Using TSTT *****mesh operators*****means alternate mesh generators****and mesh adaptation procedures****can be inserted**

***Operator Interfaces***

**Projection-based error estimator****used to construct new mesh size****field given to mesh modification**

**Mesh adaptation based on ****local modification linked****directly to CAD**

**Unaltered**
**SLAC code**

**Error estimators****from RPI and SLAC**

## Error Indicator/Estimator

**           **provides a stopping criterion

***Error Indicator/Estimator***

**Projection-based error Indicator**** **

- Total error
- Where
**          **error in element** **
**          **FE solution
**          **recovered gradient of FE solution

## Convert Error Field to Size Field

- The number of elements is minimized for a given error
- tolerance by equally distributing the error over the mesh

***Convert Error Field to Size Field***

- Size field       is obtained by minimizing
- and utilizing a priori error estimate
- Where
- 	Lagrange multiplier
-            desired error tolerance
-            problem dimension
- 	error in element
- 	current size of element** **

## Adaptive Results for Trispal Model

_**Frequency Convergence**_**                                    **_**Q Convergence**_** **

**        **_**Distribution of Wall-loss**_

**Level  0**

**Level  2**

**Level  1**

## Need for Parallel Adaptive - Need ~100M Elements

- Initial Mesh Generation

- Mesh Distribution

- Next step in TSTT/SLAC collaboration -
- parallel adaptive loop

***Need for Parallel Adaptive - Need ~100M Elements***

## Parallel Adaptive Solutions

**Parallel Adaptive Solutions**

- Need distributed data structures to support parallel processing
- Distributed mesh topologically
- PAOMD
- Hierarchical mesh structure used to distribute computation and data
- Partition boundary tools
- Update, traversal
- Dynamic load balancing
- Zoltan can meet the need
- Mesh and data migration
- PAOMD supports
- Need to deal with distributed files
- This need work

## Mesh Mesh Modification

- Refinement
- Can be done on processor with diagonal selection made consistent
- Snapping to boundary for curved domains -can require swaps, etc.
- Synchronize the partition data at end
-  Coarsening and swapping
- Parallization
- On processor - performed
- Off processor - held
- Migration to get operation on processor

## Initial Test of Component Tools

- Initial Mesh - 8,000 tets

- Adapted Mesh - 700,000 tets

## Closing Remarks 

- Adaptive loop constructed using available components and interoperability operators
- Software for multiple sources combined to create a adaptive loop
- Adaptive loop constructed and demonstrated in less that 3 months
- By people with little background on interface methods or the meshing components
- Included development of error estimation and correction indication
- Steps underway
- Installing and testing serial version on SLAC systems
- Parallization of the tools
- Putting components on SLAC machine
- Initial version did not resole most effective I/O issues
- Will work on the I/O issues
- Parallel adaptive planned to be part of the TSTT tools to be put together to support the design optimization activity
- Remeshing can adapt for both discretization control and to deal will bad elements when total shape changes are too large for mesh motion only