# Presentation: 711625

## CEMMAnswers to the PAC Questions

- S. C. Jardin
- June 3, 2005
- Princeton Plasma Physics Laboratory

## 1. What steps are you going to take to insure that there is more analysis of the data?

- Further enhance common HDF5/AVS-based M3D/ NIMROD viewer with:
- Poincare plots
- routines to analyze stochastic fields
- Beyond fractal dimension ?
- Institute real-time data streaming of simulation data to local sites for visualization and analysis (Klasky, et al)
- Exploring automated data management with SDM/SPA SciDAC center

## Characterizing Field Line Structure with Fractal Dimension

- The dimensionality of a field line inside the separatrix of a tokamak provides information relevant to confinement.
- Lines tracing out irrational surfaces are two-dimensional.
- Lines tracing out rational surfaces are one-dimensional.
- Stochastic field lines are space-filling and potentially three-dimensional.
- The extent to which stochastic lines fill space may give an indication of the effect of parallel heat conduction on radial transport.
- A measure of non-integer dimensions in data sets is provided by the Hausdorff-Besicovitch fractal dimension

- where *N(****)* is the minimum number of hypercubes of linear size **  necessary to
- cover all points in the set.

## Fractal Dimension: Good Flux Surfaces

*t* = 1266.17

- magnetic axis

## Fractal Dimension: Large Islands

*t* = 1795.61

- magnetic axis

## Fractal Dimension: High Stochasticity

*t* = 1839.86

- magnetic axis

## Fractal Dimension: Moderate Stochasticity

*t* = 1944.27

- magnetic axis

## 2. What kind of hardware and computing environment would you like?

- The most important thing for “research codes” (as opposed to production codes) is that there be availability to computing platforms without long queue wait times
- The NIMROD and M3D production jobs need to run for long times.   They need a queuing system where they can submit long runs (24-48 hours) without waiting a long time for the job to start.
- The codes perform and scale best on machines with high-performance networking and associated very low MPI latencies, like the SGI Altix
- Need a seamless way to get simulation data back to local site storage for subsequent analysis and visualization.  We are working on this.

## 3. What is your plan for addressing development of closures, including sub-gridscale models?

- We have “closure experts” on our team [Callen, Ramos, Hegna, Held...] who are willing to work with the computational people in developing closures that are practical for numerical implementation.
- There will be an initial closures discussion this Summer in Wisconsin to discuss this issue, and begin planning for the pre-APS workshop and the Spring 2006 workshop.
- This activity will be featured topic of the CEMM meeting this fall at APS, and we expect that CEMM will become central to such activity.
- We plan to organize a Closures Workshop for late winter/early spring of 2006.

## 4. What other Nonlinear Benchmarks will you be doing?  What physics will be compared?

- The CDX-U nonlinear benchmark is near completion.  During the next year, we will document, assess, and publish this benchmark and determine if there is a need for a follow-up nonlinear benchmark.
- The Closures Workshop and discussion may result in another non-linear benchmark calculation.

## 5.  How will you benchmark the 2-fluid capability?

- Basic wave propagation
- Two-fluid effects on magnetic reconnection
- Energy conservation and convergence tests
- Compare ELM results with BOUT
- May need to define a 2-fluid test problem.  This will be discussed in Closures Workshops.

## 6. What are your plans to optimize your codes for the Cray X-1?

- The majority of the M3D time is spent in the linear solvers, which are done with PETSc.
- Most of the solvers heavily depend on how fast the matrix-vector product can be done, particularly CG/Jacobi.  The initial improvement here by ORNL helped......however,
- Still need another 10 times improvement.  We are discussing with Cray how to do this.  May require a PETSc interface to access an optimized Cray sparse solver
- NIMROD requires either a version of SuperLU optimized to the Cray, or equivalent.
- We are doing some exploratory timing studies with co-array Fortran to better understand the optimization issues with the X1.  This could lead to new solvers, separate from PETSc.
- The M3D-C1 code should perform well since most of the time is spent in defining the matrix elements, but it also requires SuperLU availability.
- We presently do not have significant time allocated on the X1.
- We are also exploring the Red Storm and IBM Blue Gene

## 7. What are the plans to implement AMR in the flagship codes?

- We presently have 3 code lines that we will continue:

- M3D
*(SP-2F)*

- NIMROD

- AMRMHD

- M3D-*C**1*
*Implicit 2-fluid*

- Block-structured AMR, conservative finite difference

- High-order finite element, spectral in 

- Unstructured finite element, grid in 

-  Richtmeyer Meshkov
-  pellet injection
-  supersonic gas injection
-  reconnection
-  other applications ?

- NIMROD
*Implicit 2-fluid*

- Unstructured, triangles, adaptive h-refinement

- Adaptation via mesh distortion without changing connectivity

## 8. What new scientific insights/conceptual breakthroughs have been enabled by the FES SciDAC Program

-   Physics of the sawtooth cycle of a small tokamak,
-  including the nonlinear role of the higher toroidal harmonics.
-   Physics of the saturation of the Fishbone mode in a burning plasma
- , including mode chirping.
-   Physics of the redistribution of mass after pellet injection into a tokamak
-  the difference between low and high-field side injection
-   Physics of the formation of closed flux surfaces in a gun-injected spheromak
-  how to optimize the timings and ratios of the driving voltages
-   Physics of the saturation of the n=1 mode in a high- ST with co-injection
-   crucial role of the 2-fluid effects
-   Nonlinear physics of the Edge Localized Mode (preliminary)

## 9. Demonstrated utilization of terascale computing capability?

- Essentially all of the applications presented were performed on the IBM SP3 ( Seaborg ) at NERSC
- Several Million node-hours used.  Exact amount available upon request or by checking NIM web pages.

## 10. Likelihood of timely delivery of reliable computational modeling capabilities addressing burning plasma physics issues relevant to ITER?

- 7 critical problems identified in our proposal that are important for ITER
- Sawtooth,
- Neoclassical Tearing Mode
- Resistive Wall Mode
- Energetic Particle Modes
- Edge Localized Modes
- Vertical Displacement Events
- Pellet and Supersonic Gas Jet fueling
- Integration Activities will focus on ITER applications
- Integrated calculation of sawtooth stabilization/destabilization by RF (with RF SciDAC group)
- Hybrid calculation of neoclassical closures (with ORNL)
- M3D (NIMROD) are part of C.S. Cheng, et al , (R. Cohen, et al) Edge FII