Fusion Simulation Project:
Integrated Simulation & Optimization of Magnetic Fusion Systems
First report of the FESAC ISOFS Subcommittee
12 July 2002

ISOFS (Integrated Simulation & Optimization of Fusion Systems)
Subcommittee Members:
Jill Dahlburg, General Atomics (Chair)
James Corones, Krell Institute, (Vice-Chair)
Donald Batchelor, Oak Ridge National Laboratory
Marsha Berger, New York University
Randall Bramley, Indiana University
Martin Greenwald, Massachusetts Institute of Technology
Stephen Jardin, Princeton Plasma Physics Laboratory
Sergei Krasheninnikov, University of California - San Diego
Alan Laub, University of California - Davis
William Lokke, Lawrence Livermore National Laboratory
Jean-Noel Lebouef, University of California - Los Angeles
John Lindl, Lawrence Livermore National Laboratory
Marshall Rosenbluth, University of California - San Diego
David Ross, University of Texas - Austin
Dalton Schnack, Science Applications International Corporation
Harold Weitzner, New York University

FESAC ISOFS Subcommittee First Report
12 July 2002

Executive Summary
Fusion is a potentially inexhaustible energy source whose development entails
understanding complex plasmas under extreme conditions. The development of ascience-based predictive capability for high temperature, fusion-relevant plasmas is achallenge central to fusion energy science. The combination of extreme separation oftime and spatial scales, extreme anisotropy, the macroscopic effects of microscopic
physics, the importance of geometric detail, and the requirement of causality (inability toparallelize over time) makes this problem among the most challenging in computational
physics. The exponential growth of computational capability, coupled with the high costof operating large-scale experimental facilities, makes the development of a fusion
simulation initiative a timely and cost-effective opportunity.
Numerical modeling has played a vital role in magnetic fusion for over four decades,with increases in the breadth and scope of feasible simulation enabled by improvements
in hardware, software and algorithms. Currently, sophisticated computational models areunder development for many individual features of magnetically confined plasmas.
However, full predictive understanding of fusion plasmas also requires cross-coupling ofa wide variety of physical processes. While integrated models using simplified
descriptions of a number of physical processes exist and have been widely used in the
program, the capabilities needed for full predictive simulation and optimization of aburning plasma require major qualitative improvements in physics models, algorithms,
computational platforms, and the ability to integrate codes from a large number of
research teams working on different elements of the problem. Such a capability was citedin the year 2000 FESAC IPPA integrated planning document as a ten-year goal.
Worldwide progress in laboratory experiments provides the basis for a recent FESAC
recommendation to proceed with a burning plasma experiment (see FESAC Review of
Burning Plasma Physics Report, September, 2001). An integrated simulation capability
would dramatically enhance the utilization of such a facility and the optimization of
toroidal fusion plasmas in general. This undertaking represents a significant opportunity
for the DOE Office of Science to create a capability that will advance the understandingof fusion plasmas to a level unparalleled worldwide.
The ISOFS subcommittee recommends that a major initiative be undertaken, referred tohere as the Fusion Simulation Project (FSP). The purpose of the initiative is to make asignificant advance within five years toward the ultimate objective of fusion simulation –to predict accurately the behavior of plasma discharges in a toroidal magnetic fusion
device on all relevant diverse time and space scales. This is in essence the capability forcarrying out “virtual experiments” of a burning magnetically confined plasma, implying
predictive capability over many energy-confinement times, faithful representations of thesalient physics processes of the plasma, and inclusion of the interactions with the external
world (sources, control systems and bounding surfaces).
Two fundamental issues are common to many fusion physics integration areas: couplingof phenomena at disparate space and time scales, and coupling models of different
dimensionality. To solve these generic problems and achieve the integration we are
FESAC ISOFS Subcommittee First Report
12 July 2002

seeking, strong collaboration and advances in physics, applied mathematics, andcomputer science will be required. This disciplinary integration will be an essential
element of the program.
To succeed, a central feature of this initiative must also be an intensive and continual
close coupling between the calculations and experiments. The phenomena in magnetic
fusion devices, the equations describing them, and the interactions among the various
critical phenomena are sufficiently complex that developing the most effective
approximations and establishing when the models have the desired accuracy can only be
accomplished by continual iteration and testing of the models against experimental data.
We envision that there will be three major categories of activity within the Fusion
Simulation Project: 1) research to advance fundamental capabilities in fusion science,
applied mathematics and computer science that address specific program critical needs;
2) development of applications modules, the core building blocks of an integrated
modeling capability, in extended MHD, turbulence and transport, the plasma edge, andexternal sources, including development of the required computational science
framework as well as the visualization and interpretation tools; and 3) project integration,
which includes development necessary to create a comprehensive simulation including
the interconnection and interoperation of multiple applications modules, oversight ofsoftware standards and release policies, collaborative tools, program governance, andaccountability. It will likely be necessary to support more than one approach to various
research areas in all three categories listed above.
Provided below is an overview of the Fusion Simulation Project.

FESAC ISOFS Subcommittee First Report
12 July 2002

TABLE OF CONTENTS

I. Background1

II. Overview and Recommendations

1

III. Fusion Simulation Project (FSP) Elements

4

IV. Project Governance11

V. Fusion Simulation Capabilities Status

12

VI. Summary

15

VII. Acknowledgements

16

VIII. Appendix

17

FESAC ISOFS Subcommittee First Report
12 July 2002

I. BackgroundOn 22 February 2002, Dr. James Decker [then Director (acting), Department of Energy(DOE) Office of Science] issued a charge to the federal panel that provides advice to the
DOE Office of Fusion Energy Sciences (OFES), the FESAC (Fusion Energy Sciences
Advisory Committee). This charge, the letter of which is reproduced in the appendix of
this report, requested advice on the development of a capacity for integrated simulationand optimization of fusion systems (ISOFS). The charge letter, further, indicated that thecapacity should be developed by researchers from the DOE OFES in collaborative
partnership with researchers from the DOE OASCR (Office of Advanced Scientific
Computing Research).
Here we address the first two requests in the charge letter: to overview the current statusof simulation of toroidal confinement fusion systems, and to describe a vision for acomputational capacity for the full-integrated simulation of toroidal magneticallyconfined fusion systems. Subsequently, and by 1 December 2002, a comprehensive plan
for an ISOFS capability will be developed. This plan will include a detailed discussion of
the role of applied mathematics and computer science in the initiative as well as thecomputational infrastructure needs of the initiative. Thus, the later report will address theremaining elements of the charge letter.
In order to address the ISOFS charge, the FESAC formed a subpanel, the FESAC ISOFS
Subcommittee. Members of this Subcommittee are listed on the first page of this report.
During the period 22 February to the present, the Subcommittee has taken as input for its
deliberations both written and verbal contributions from members of the fusion andapplied mathematics communities, and, further, convened in a workshop environment on
23-24 May 2002, to discuss the ISOFS charge. This document, which represents theresults of the Subcommittee activity to date, is the first report of the FESAC ISOFS
Subcommittee. The Subcommittee notes that substantial community input for the final
report will be solicited at a major ISOFS workshop that is currently being planned for 1718 September 2002; details about this upcoming meeting and extended Subcommittee
activities may be found at the website: http://www.isofs.info .
II. Overview and Recommendations
Numerical modeling has played a vital role in magnetic fusion for over four decades withincreases in the breadth and scope of feasible simulation enabled by improvements in
hardware, software and algorithms. Recent developments in computers, computerscience, and theory have created an opportunity to achieve dramatic advances from anintensive effort to harness these emerging capabilities. These advances can bringsimulation to a level of sophistication enabling it to be an equal partner with basic theory
and experiments in advancing the field.
The ability to understand and predict the dynamics of high temperature fusion-relevant
plasmas is a formidable physics challenge that is central to the goals of the Fusion Energy
Sciences program. It is widely recognized that the complexity of the dynamics of these
FESAC ISOFS Subcommittee First Report
12 July 2002

systems is such that the development of computational models to understand their
behavior is critical. Numerical modeling in magnetic fusion research, as exemplified by
the fusion SciDAC projects, are providing important physics understanding and routinely
stretch the limits of available supercomputers. However, crosscutting issues crucial to
the further development of these models require a qualitative change in the approach tothese problems. In particular, we find two fundamental issues that commonly appear in
the integration of different fusion physics areas: the coupling of phenomena at disparate
time scales and the necessity of coupling models of different dimensionality.
Further, research on fusion plasmas has often been compartmentalized into specifictopical areas with little cross-talk between these topics – e.g., magnetohydrodynamics(MHD) and turbulent-transport. Since the dynamics of a high temperature plasma does
not respect those categorizations, the inability to bridge the gap between the various
models inhibits the ability to achieve the needed depth of understanding of key
phenomena. Indeed, these are key issues for any challenging simulation effort such as
climate modeling, inertial confinement fusion or astrophysics. To solve these genericproblems and achieve the integration we are seeking, strong collaboration and advancesin fusion physics models, applied mathematics, as well as computer science will be
required.
A properly designed computational initiative that advances both the understanding of the
key scientific processes which control plasma behavior, and the development ofinnovative techniques for cross linking these processes, would fundamentally advance theability to predict the behavior of and therefore optimize fusion plasma systems. Such an
initiative would:- allow us to bring together the multiple disciplines necessary to take advantage
of modern computing and advances in fusion theory;- go beyond what is scientifically achievable by the disciplines in isolation; and,- enable the full integration of theory, experiment, and simulation.
In view of the importance of establishing the feasibility of the potentially inexhaustible
energy source, fusion energy – coupled with the complexity and variety of magnetic
fusion physics, the exponential growth of computational power, the dramatic
improvement in high-temperature plasma diagnostics, and the high cost of operating
experimental facilities over the wide ranges of relevant scenarios – the development of afusion simulation initiative represents a timely and cost-effective opportunity.
The ISOFS subcommittee recommends that a major initiative be undertaken, here
referred to as the Fusion Simulation Project (FSP), of creating a comprehensive setof theoretical fusion models, an architecture for bringing together the disparate
physics models, combined with the algorithms and computational infrastructurethat enable the models to work together.
The purpose is to make a significant advance toward the ultimate objective of fusion
simulation – to predict in detail the behavior of any discharge in a toroidal magnetic
fusion device on all important time and space scales. This is in essence the capability for
FESAC ISOFS Subcommittee First Report
12 July 2002

carrying out “virtual experiments” of a burning magnetically confined plasma, implying
predictive capability over many energy-confinement times, faithful representations of thesalient physical processes of the plasma, and the interaction with the external world(sources, control systems and bounding surfaces). The initiative should be structured to
add capability incrementally and should adopt near and intermediate term objectives of:
supporting basic theoretical research; supporting the understanding, interpretation, andplanning of ongoing fusion experiments; exploration of new confinement concepts toimprove the prospects for economical fusion power; and predicting the performance of
future fusion devices.
The FSP should be comprised of three interacting levels of activities:•

Fundamentals: improvements in physics understanding in individual areas,
including all relevant physical processes, validation by comparison withexperiment and analytic models, development of advanced numerical approaches,and needed computer science advances;•

Applications modules: “stand-alone” integrated suites of codes, perhaps frommultiple developers, that address fundamental problem areas such as MHD,
turbulent transport, and external sources. Each applications module must be
developed to high software engineering standards and be formulated in such a
way so as to be compatible with other modules and seamlessly join to assemble
the comprehensive simulation capability; and•Integration: development necessary to create a comprehensive simulation
capacity, i.e. development of flexible physics and computational frameworks
which at all stages of the FSP provide all functions necessary to create acomprehensive simulation, including the inter-operation of applications modules,
oversight of software standards and release policies, collaborative tools and thelike.
We envision a vigorous program of comparison and iteration withexperiments at this level, as well as at the other levels.

The initiative should be carried out at a scale such that within five years certain goals canbe achieved:
1)

Robust computational modules are developed in each of the fundamental scienceareas representing the state-of-the-art in physics content, numerical methods, andcomputational science methods enabling efficient incorporation into theintegration framework.

2)

Approaches are developed for the fundamental problems of disparate time orspace scales, and coupling of models of processes having different
dimensionalities.

3) An initial inter-operable code capability that allows for three-dimensional
geometry is available for widespread testing as a research tool.
FESAC ISOFS Subcommittee First Report
12 July 2002

4) The effectiveness of the integration approach is demonstrated by application tointerpreting experimental data, and testing the validity of various physics models.
To bring these disparate components together will require the dedicated talents of many
accomplished physicists, applied mathematicians and computer scientists. There is no
doubt that the sociology of the FSP will be a challenge. On the one hand, a strong fusion
physics effort is required, involving a number of institutions and the relevant theory,
simulation, and experimental communities. On the other hand, setting priorities and aconsiderable degree of central direction will be essential. Even more challenging will beeffective integration of first-rate computer scientists and applied mathematicians as equal
partners with fusion physicists in this venture. The issue of project governance includes
the establishment of an effective cooperative arrangement between and within OFES andOSCAR and clear delineations of working relationships with other initiatives andactivities such as the DOE Office of Science SciDAC, OFES fusion experiments, andOSCAR computing resources. Success will require planning, leadership, and likely new
management approaches.
This initiative rests entirely on a progressing science base. Therefore it is paramount that
FSP funding be new to the program rather than redirected from present, critical areas.
Also, it must be at a level adequate to accomplish the FSP goals. To estimate funding, we
use the successful DOE Accelerated Strategic Computing Initiative (ASCI) Level 1
University Centers Program as an example, and recognize that a critical mass, as realized
in that $25M/year program, must be applied to the elements of the applied and integration
activities of FSP. This leads to the estimate that a total of $20M in new FSP funding will
be required in the first year of the project, with an increment projected for the second
year, followed by a leveling-off of funding in subsequent years of the activity. Throughthe course of the project, funding should be approximately equally allocated between
OFES and OASCR research elements.
III. Fusion Simulation Project Elements
Bringing the power of advanced computing to the comprehensive simulation of fusion
devices will entail three major categories of Fusion Simulation Project (FSP) activity:
fundamental needs; applications modules; and project integration. Work within the
category of fundamental needs addresses specific project critical needs. In some cases
this work will include the solution of basic research problems in the areas of plasma
science, applied mathematics and computer science. It will likely be necessary to support
more than one approach to the same problem at this level of activity. Applications
modules are the core building blocks of the initiative. These are integrated suites ofcodes or tools for the simulation of categories of phenomena as well as collaborative
tools or algorithmic suites, as required. The project integration activity includes allfunctions necessary to create a comprehensive simulation including the inter-operation of
applications modules, oversight of software standards and release policies, collaborative
tools, program governance and accountability.

FESAC ISOFS Subcommittee First Report
12 July 2002

A. Fundamentals
In the past fifteen years, enormous progress has been made in the fusion program inexperimental discoveries, theoretical insight, and computation. Internal transport barriers
and other advanced regimes in tokamaks have been found and qualitatively explained interms of the interaction of plasma flow shear and turbulence. Large-scale plasma
turbulence simulations now promise to provide a complete quantitative description ofthese processes. A new understanding of tokamak disruptions has emerged. New stable
MHD configurations for alternative concepts, such as stellarators, reversed-field pinches
and other devices are being designed by means of advanced computational tools.
Through MHD studies in the plasma edge, the limiting plasma pressure, density, andcurrent are being better explained. The scrape-off layer outside the magnetic separatrix isbeing well modeled in the collisional regime by fluid codes, together with calculations ofneutral-particle recycling. Instabilities driven by injected radio waves, particle beams,
and, by extension, alpha particles, are being diagnosed and interpreted by theory andsimulation. In addition to the above, there are many other examples.
Nevertheless, a number of major unsolved problems remain, which must be investigatedand resolved in the course of this simulation initiative, including, for example, inclusionof kinetic effects in MHD fluid-like models, electron kinetics and the exploration of theelectron gyroradius regime, particle and momentum transport, and the physics underlying
transient and nonlocal effects. In the edge plasma, an outstanding problem is the width of
the H-mode pedestal, which has an enormous effect on the overall plasma confinement.
Kinetic codes, valid in the long mean-free-path regime of the edge and scrape off layer
remain to be developed. Anomalous transport in stellarators is only beginning to beaddressed. These are only a few of myriad tantalizing examples of FSP-relevant
fundamental research that will lead to the modules described in the following sections.
B. Applications Elements
The applications elements are key system components. They integrate fundamental work,
perhaps from multiple developers of the same process. These modules can be viewed as“stand alone” integrated suites of codes that address problems areas e.g. MHD, plasma
microturbulence, and external sources. Each applications element must be developed tobe compatible with other elements so that it is possible to join these elements together
seamlessly to assemble the comprehensive simulation suite.
Softwareengineering/standards and collaborative tools are included in this category. Applications
elements will be built to deal with the broad concepts outlined below.
Element i. Turbulence and Transport
Plasma transport or confinement will be a major component of the integrated simulation
project. We envision a time when reliable calculations of transport fluxes, informed bytheory, and validated of experimental comparisons, will “inter-operate” with the othercomponents of the computation. Loss of plasma particles and energy across field lines
results from at least three categories of phenomena: diffusion and convection based on
FESAC ISOFS Subcommittee First Report
12 July 2002

individual orbits and collisions (classical, neoclassical), anomalous diffusion andconvection from turbulent microinstabilities (usually thought to be dominant), andphenomena that are instantaneous and nonlocal.
1) Neoclassical codes for both 2- and 3-dimensional configurations are available
subject to certain approximations. Additional work will be required to fully
implement the existing theories and to account for additional effects in theinter-operative environment, particularly in 3-D configurations (stellarators)
and in tokamaks with transport barriers.
2) Micro-turbulence-driven anomalous transport will be dealt with on three levels:a) fine-scale stand-alone gyrokinetic simulations of core plasma turbulence willcontinue to be developed, interpreted by theory, and compared with experimentto firmly validate the fundamental theory and benchmark the otherdescriptions; b) reduced simulations will be coupled directly to the transport
equation solvers. In some cases, it may be possible to directly couple the fullturbulence simulations with the transport equation solvers; and c) algebraic
models of transport will continue to be developed, again informed by theory,
experiment, and the just-described simulations. When supported by turbulencesimulations for selected cases, they will provide the most rapid parameterscans.
3) Work will continue on development of models of observed rapid and nonlocal
phenomena (e.g. avalanches, radiative transport, or global magnetic
interactions) that do not fit into the diffusive-convective approach. The
architecture must include provision for these from the outset, so they can
become incorporated when available.
A measure of success for these efforts and their validation will be the ability to predict
and model a transport barrier, a region of steep gradients where turbulence is suppressed.
Element ii. Extended MHD
Modern fusion plasmas are subject to low frequency, long wavelength, fully threedimensional hydromagnetic phenomena that can degrade confinement, and in some cases
lead to major disruptions. The only computationally tractable model to describe these
dynamics is based on fluid equations, which are derived from velocity moments of themore fundamental kinetic equation. These equations are solved simultaneously with the
low frequency Maxwell equations. The resulting model is local in space but requires
additional closure relations to obtain a self-contained system of equations and takeaccount of kinetic effects. Models of magnetized plasmas based on this approach arecollectively called (eXtended) MagnetoHydroDynamics (X-MHD).
There are two approaches to developing an appropriate system of closed Extended MHD
equations. One approach is to calculate analytic expressions for the highest ordermoments of the distribution function for use in the fluid equations. Another approach is
to solve the kinetic equations numerically to obtain the distribution function, and thenevaluate its moments to obtain the required contributions to the fluid equations. The
result of an X-MHD calculation is often that an initial equilibrium that has crossed astability threshold will exhibit unstable behavior and then relax to a saturated state in a
FESAC ISOFS Subcommittee First Report
12 July 2002

few Alfvén wave transit times. This is the case, for example, in a sawtooth oscillation.
For an integrated plasma model, the effects of this MHD profile relaxation must be
encapsulated into the description of the plasma evolution on the even longer transport
time scale.
Element iii. External Sources
External systems that add mass, momentum, or energy to a plasma have been essential
tools in the successful efforts to obtain good performance in present experiments and are
sure to be a necessary part of plasma control in any future experiments and reactors. At
present, the external sources include beams of neutral atoms that can carry energy,
particles, and angular momentum across magnetic fields; radio frequency waves (RF)whose interactions with a plasma can be used for heating, current drive or flow drive;high-speed pellets of frozen fuel gas that can deliver particles deep into the plasma core;and gas fueling which supplies particles to the plasma edge.
The computational capabilities related to neutral beam injection are very well developed
and are presently integrated into many codes. The work could be easily ported to codesdeveloped in new initiatives. Wave-plasma interactions at RF frequencies (from ion toelectron cyclotron frequencies) are the subject of intense ongoing research including
fusion SciDAC activity. However, the scope of needed work, and the ability to provide
interactive coupling with other plasma codes, extend far beyond the SciDAC activity.
This area includes many distinct problems and will be an essential element of futureplasma prediction, interpretation, and control schemes. Work on the ablation andsubsequent transport and deposition of fuel from injected pellets is in a relatively earlyphase of development but certainly amenable to computation. However the physics offueling in general is not in a satisfactory state at present and would benefit from basic
theory studies of particle transport.
Element iv. Physics of the Plasma Edge and Connection to the Core and Wall
Edge plasma, which bridges the hot plasma core and the material wall, plays a crucialrole in both overall plasma confinement and plasma-wall interactions. Our current
understanding of the physics of edge plasma and the tools to model it are rather sketchy.
We know that the physics of edge plasma is very complex and ranges from turbulent
plasma transport to atomic physics and radiation, and to surface and condensed matterphysics. Moreover, edge plasma turbulence is highly intermittent with a strong
contribution to particle and energy fluxes from large but rare events, e.g. the so-callededge-localized modes (ELMs). Therefore, the conventional separation of spatio-temporalscales for the description of turbulence and transport is questionable. New approaches
should be developed for both turbulence and transport phenomena that relax thisordering. Additional complications result from a strong variation of the plasma
parameters from highly collisional to collisionless regimes. Therefore, the simpler fluid
description is not always adequate and should be replaced by a much more complex
hybrid or kinetic one. Because of the open field lines, such a code will have different
boundary conditions from those of the core. Thus, the understanding and modeling of
FESAC ISOFS Subcommittee First Report
12 July 2002

edge plasma and its coupling to the core from first principles are a real challenge for boththeory and computing.
With large-scale efforts in this area, it is realistic to anticipate a number of advances.
These include crucial improvement (e.g., adding neutral effects, some kinetic features,
impurity transport, atomic physics, and geometry) of fluid edge turbulence codes to thelevel required to do more detailed modeling of the pedestal physics as well as to study
macroscopic transport phenomena (e.g., heat loading, plasma-wall interaction, and tritium
retention). Ultimately, edge plasma turbulence gyrokinetic codes should be advanced toa level adequate for the study of core-edge synergy effects.
Element v. Computational Science Framework
Advancing simulation capabilities in each constituent module of fusion simulation, anddeveloping the ability to couple codes from multiple modules to create integrated
simulations, will require development of new computational science research andmethodologies in at least three areas.
1) Fusion simulations present a unique characteristic when compared to other PDE
applications – the different physics modeled by different codes are often
overlapping in space, instead of being decomposed into physical sub-domains.
This implies tight coupling between modules, necessarily large communication tocomputation ratios, and close collaboration among physicists, appliedmathematicians, and computational scientists.
2) Building software infrastructure and middleware for creating multi-physics
simulations enabling dynamic interoperability of codes from different researchersand sites. Business and industrial computing now use software component
architectures for such integration, but it is still a major challenge to connect large,
distributed, and parallel physics codes.
3) Data management and sharing technologies.
The challenges range fromdeveloping self-describing and portable file formats, to full-fledged data andmeta-data frameworks such as MDS-plus which is already in extensive use by thefusion community. New scientific data systems can greatly ease the sharing ofresults between experiments and simulations and form a necessary first step
towards code inter-operability for different regimes. Storage systems are now
reliably supporting large scale (petabyte) data. Providing users with the ability tonavigate, search through, and manage the large distributed data sets that will be
generated and used in an integrated simulation is another major research problem.
The FSP software architecture will provide exciting new challenges for mathematicians
and computer scientists, and can leverage the research from other programs, such as DOE
SciDAC projects now developing numerical methods and computational infrastructure
for large distributed simulations and models being developed within the ongoing theory
program within OFES. Basic computational infrastructure has recently been deployedthat will help: reliable authentication mechanisms for security, tools for wide-areascientific collaborations, software component systems supporting efficient data transfers,
FESAC ISOFS Subcommittee First Report
12 July 2002

large-scale distributed data management systems, and capabilities that provide end-useraccess to these facilities. The FSP will, moreover, provide valuable mission-driven
impetus for the development of user-friendly modules, infrastructure, and training, which
will then be provided to code teams beyond the FSP.
Element vi. Interpretation and Visualization Tools
An integrated simulation will only be beneficial to our community if we can understandwhat it is telling us. To this end, it is imperative that the FSP have built-in a powerful
diagnostic and analysis capability from the beginning.
There are several categories of diagnostics that are needed:
1) The ability to query the calculation regarding virtually any combination ofinternal variables, be they physical or numerical in origin, and including
integral and differential operations.
2) The ability to easily graphically display any of the results of a calculation
together with similar results from previous simulations, theoretical models, or
experimental results.
3) Capabilities that facilitate experimental comparisons. Thus, we envision anumber of software packages that provide a signal that can be directlycompared with an experimental diagnostic on a given machine.
C. FSP Integration
As discussed in Sect.V of this report, MFE physics simulation presently includes
primarily two categories of codes: 1) detailed simulations of a particular physical process(e.g. micro-turbulence, macroscopic MHD, RF-plasma interactions), and 2) transport
codes that use reduced descriptions of the details to follow a significant portion of theplasma (e.g. the core or the edge) in reduced dimensionality (1 dimension for core, 2 foredge) over long timescales.
Our approach to integration is to develop an architecture that will:- promote the development of the physics modules and their validation throughexperimental comparison, beginning in the near term;- facilitate study of mutual physics interactions presently modeled in separate
codes as such interconnections become appropriate; and- increase significantly the depth and breadth of physics compared to today’stransport codes, incrementally as better modules become available.
Two fundamental issues are common to many fusion physics integration areas: couplingof phenomena at disparate time and spatial scales, and coupling models of different
dimensionality. To solve these generic problems and achieve the integration we are
seeking, strong collaboration and advances in physics, applied mathematics, andcomputer science will be required. This disciplinary integration will be an essential
element of the program.

FESAC ISOFS Subcommittee First Report
12 July 2002

To succeed, a central feature of this initiative must also be an intensive and continual
close coupling between the calculations and experiments. The phenomena in magnetic
fusion devices, the equations describing them, and the interactions among the various
critical phenomena are sufficiently complex that developing the most effective
approximations and establishing when the models have the desired accuracy can only be
accomplished by continual iteration and testing of the models against experimental data.
Hence, a continual process of testing and iteration is required to advance both modelingand the characterization of experimental results. From these objectives flow a number ofrequirements that the integration framework must satisfy:
•

•

•

•

It must be extensible.
- Easy connections can be made early in the project while more difficult ones, forexample those involving very disparate time-scales, can be added whenappropriate.
- Its architecture must permit continuous improvements and additions.
It must be flexible.
- Only the particular physics modules required for a given study need beinterconnected.
- It must be robust to changes in physics paradigms. For example a traditional
diffusive transport model will be inadequate if non-local effects turn out to be
essential.
- It must be interpretive as well as predictive. That is it must be possible to makeuse of both experimental information such as profiles, and predicted informationsuch as source rates, to interpret other needed quantities such as transportcoefficients.
- It must support choice in appropriate level of description for any of the modules
in the particular study. It must allow for 3D effects but also be capable of lower
level 1D and 2D models where appropriate.
It must support collaborative research.
- It should interface well with experimental databases and provide appropriate tools
such as synthetic diagnostics to facilitate understanding its output.
- It must include protocols for effective communication among geographically andscientifically diverse physics participants and code developers.
It must complement existing research.
- The facility must provide value to the individual involved in basic physics
research, who may himself be doing large-scale computation.
- It must not impose a significant overhead (computational or human) on the use
and development of the separate physics modules. It must provide needed
services so as to be of value even to a single module.

At this stage it is not necessary to specify how these connections are to be made. That
will be a major part of the research program and will require the involvement of appliedmathematicians and computer scientists to define.
In fact, to satisfy the above
requirements it is likely that several integration schemes will be necessary.

FESAC ISOFS Subcommittee First Report
12 July 2002

IV. Project Governance
The FSP initiative should be focused, highly interdisciplinary, and involve a significant
number of people. For these reasons it is extremely important that careful attention be
given to governance of the project. The governance structure needs to effectively balance
the professional needs of the creative and individualistic people who will carry out thework with the programmatic needs for focus and timely delivery of results. In addition
the structure needs to work effectively with the two programs offices (OASCR andOFES) that will support and manage the initiative. A sketch of a proposed governancestructure is provided below. It is recognized that extensive discussions with the relevant
communities are needed to assure buy-in of this or any other structure.
An analogous set of issues has been addressed by the Community Climate Systems
Model activity (see http://www.ccsm.ucar.edu ). While there are significant differences
between the nature of the science involved in the CCSM and the initiative discussed here,
there nonetheless are sufficient similarities that the CCSM activity can help provide theneeded structure. Three critical elements are needed: a scientific steering group, a series
of working groups including one which addresses software management issues, and an
external advisory board. The organizational chart for these groups is:

Scient ific Steering Gro up

Adv isory Boa rd

Wor king Gr oup

Wor king GroupWor king GroupThe functions of these groups are:
Scientific Steering Group: This group provides the overall scientific direction and vision
for the project. It provides oversight and coordination of scientific activities. It is the keygroup for assuring the integration described above is effected. It coordinates with
program offices on resource allocation issues.
Working Groups: Each working group is focused on one of the applied modules (seeabove) as well as a software engineering group. A working group oversees the scientificdirection of the module development including determination of required fundamental
work. It assures integration of components within the modules and compatibility of themodule with the rest of the system. This is where the real work gets done.
FESAC ISOFS Subcommittee First Report
12 July 2002

Advisory Board: This group is made up of people with scientific breadth that are notdirectly engaged in the initiative. The group will provide scientific and management
advice to both the Steering Group and the program offices.
V. Fusion Simulation Capabilities Status
This section summarizes the current status of integrated computational modeling andsimulation of toroidal confinement fusion systems. The intent in this section in thepresent document is to provide a general perspective of the status of this very active andmature field. This section responds to the explicit request in the subcommittee’s charge
letter to report on the status of fusion simulation capabilities. These capabilities form asignificant part of the critical underpinning of the FSP.
There are over 50 major toroidal physics design and analysis codes being maintained by
the magnetic fusion community. The major multi-user codes are depicted in Figure 1,
which shows how they divide into groups, and indicates with arrows the flow of
information from one code group to another.
The axisymmetric free boundary equilibrium codes solve the force balance equation by
calculating the poloidal magnetic flux in cylindrical coordinates for given pressure andcurrent profile parameterizations. These can be used to define the boundary for the
inverse equilibrium codes. There are also two major fully 3-D equilibrium codes in use:
VMEC, which assumes the existence of good magnetic surfaces a priori, and works in acoordinate system based on these, and PIES, which calculates the existence of surfaces aspart of the solution, if they exist.
The collection of linear macroscopic stability codes maintained by the MFE community
is quite mature and can assess the stability properties of a given equilibrium with respect
to both ideal and non-ideal (resistive) MHD, including the effects of an energetic particle
component.
The nonlinear codes fall into four major groupings.
In descending order of thefrequencies addressed, these are the: 1) RF Heating and Current Drive codes, 2) the
Nonlinear Gyrokinetic codes, 3) 3-D Nonlinear Extended MHD codes, and 4) the 2-D
Transport codes.
The RF Heating and Current Drive codes calculate the propagation of electromagnetic
waves of a given frequency through a prescribed background plasma, including reflection
and absorption. The codes are of two major types: ray-tracing (or geometrical optics),
and full wave (global solution). There are also depicted antenna and Fokker-Planck
codes, which are closely coupled with the RF codes, and provide boundary conditions
and background distribution functions. The RF codes are designed to calculate theinstantaneous heating and current-drive profiles for a given plasma equilibrium subject toa given RF oscillator source and antenna.

FESAC ISOFS Subcommittee First Report
12 July 2002

FIG. 1. Major U.S. toroidal physics design and analysis codes used by the plasma
physics community.

The gyrokinetic codes are based on an analytic reduction of the full 6-dimensional plus
time plasma distribution function obtained by averaging over the rapid gyro-motion ofions in a strong magnetic field, and by neglecting the displacement current in Maxwell’sequations to remove “light waves” from the system. These codes are appropriate forstudying 3-D turbulent transport in a background system with fixed profiles.
The 3-D nonlinear Extended MHD codes are based on taking velocity moments of the
Boltzmann equation to yield 3-D magneto-fluid equations for the evolution of the averageplasma velocity, density, and pressure, along with a closure procedure. These codes areappropriate for describing global stability phenomena such as sawteeth oscillations,magnetic island evolution, and plasma disruptions.
The 2-D transport codes presently form the core capability in our community for
integrated modeling. There are six major codes, with considerable overlap, that exist
largely for historical reasons. These codes are all based on the Grad-Hogan evolving
equilibrium description where the inertial terms in the momentum equation are neglected
and the remaining MHD equations are averaged over the flux surfaces, where they exist.
FESAC ISOFS Subcommittee First Report
12 July 2002

The 2-D transport codes are all very modular. They are each a collection of equilibrium
modules, transport modules, solvers, and source and sink modules representing Neutral
Beam Injection (NBI) and RF heating, pellet and gas injection, impurity radiation, andthe effects of saturated MHD activity such as sawteeth and islands. These codes haverecently benefited from the National Transport Code Collaboration (NTCC), which hasformed a modules library so that modules taken from individual codes can be exchanged
and shared (see, e.g., w3.pppl.gov/NTCC for more details on this). While these codes
address integrated modeling, the individual modules represent simplified reduced
descriptions of the full three-dimensional physical phenomena being modeled.

Typical Time Scales in a next step experimentwith B = 10 T, R = 2 m, n e = 1014 cm-3, T = 10 keV
SAWTOOTH CRASHELECTRON TRANSIT

ωLH-1 Ω -1
Ωce
ci
-10
10
10-8
-1

ENERGY CONFINEMENTTURBULENCE

τA

10-6

ISLAND GROWTH

10-4

10-2

CURRENT DIFFUSION

100

10 2

104

SEC.

Single frequency
Neglect displacement
and prescribedcurrent, average overplasma background gyroangle, (some)with electrons

Neglect displacement
current, integrate overvelocity space, neglect
electron inertia

RF Codeswave-heating andcurrent-drive

Gyrokinetics Codes

Neglect displacement
current, integrate overvelocity space, averageover surfaces, neglect
ion & electron inertia
Extended MHD Codes Transport Codes

turbulent transport

device scale stability

discharge time-scale

FIG.2. Summary of the four major code groups and the timescales being addressed.
A summary of these four major code groups and the timescales being addressed by themis given in Figure 2. The RF codes address frequencies of order the ion cyclotron
frequency, Ω ci , and above, up to the electron cyclotron frequency Ω ce. The gyrokinetics
codes typically take time steps about 10 times longer than the ion cyclotron frequency,
whose motion is analytically averaged over, and are normally run for 103 to 104 time
steps to calculate stationary turbulent fluctuation levels. Recent additions to these codes
to include some electron timescale phenomena bring in the electron transit time, which
lowers the maximum timestep. The Extended MHD codes need to resolve phenomena
occurring on the Alfvén transit time, τA, although most codes are at least partially implicit
to avoid a strict restriction on the timestep based on this. These codes can normally run
104 to 105 time steps to address MHD phenomena such as sawteeth and island growth.
FESAC ISOFS Subcommittee First Report
12 July 2002

The 2-D transport codes are very efficient and can take many long timesteps to model theentire discharge. However, the 2-D edge transport codes need to resolve the parallel
dynamics and use a time-step based on the parallel sound-wave propagation near the edge
region.
The calculations now being performed with the gyrokinetics codes, the Extended MHD
codes, and the RF codes, are straining the limits of the existing computing capabilities
and capacities. For example, recent attempts by the core-turbulence gyrokinetics codes
to include both electron and ion dynamics in a self-consistent simulation require upwardsof 104 processor-hours (over one processor-year) on the IBM SP3 at NERSC to generate
one result for a set of fixed background profiles.
Similar times are required by the
Extended MHD codes to calculate the growth and self-consistent saturation of aneoclassical tearing mode. Thus, we can take solace in the fact that the capability ismostly in place, but must deal with the fact that the computational requirements for a
fully integrated 3D comprehensive simulation capability are truly daunting.
Examples of fundamentally important experimental phenomena which involve 3D
physical processes that cross theoretical boundaries and thus cannot adequately be
addressed by the present suite of above-described codes include:
-

-

-

-

-

-

Pedestal physics – A description of the transport barrier that forms in theregion of the plasma between the core and the edge, and of the associatededge localized relaxation events;
Long time scale profile evolution – A way to self-consistently evolve theglobal profiles of plasma temperature and density on the energy-confinement
time scale from turbulent transport and in the presence of magnetic islands
and other MHD phenomena.
Edge transport: A description of long-mean-free-path particle and heattransport outside the closed magnetic flux surfaces, on the open field lines that
impact the first wall or divertor and involve multi-phase physics
Self-consistent heating and current drive: A fundamental model of theinteraction of Radio Frequency (RF) waves with plasma in the presence ofplasma turbulence.
Sawtooth phenomena – Internal MHD-type modes in the hot core of tokamak
plasmas for which fast ion and kinetic effects are clearly relevant
experimentally but are only beginning to be addressed computationally.
Island physics: incorporation of the effect of 3-D island formation onequilibrium evolution and turbulent transport

These are but some of the important problems to be addressed by the integrated
simulation initiative.
VI. Summary
The essential goal of the Fusion Simulation Project (FSP) is to develop a computational
capacity that can perform integrated simulations of toroidal magnetic confinement
FESAC ISOFS Subcommittee First Report
12 July 2002

devices.
This capability is envisioned to span fundamental dynamics of evolving
plasmas, including diagnosis and control of toroidal configurations, and the coupledproperties of fusion devices, in the presence of the widest expected range of plasma
dynamics.
The FSP is envisioned to be comprised of three major elements that are funded
approximately equally over the life of the project: fundamentals capabilities, applications
modules development, and project integration. This development will be made feasible
by close coupling of the integration initiative research with ongoing program activities intheory, experiment, simulation, computer science and applied math carried out under
OFES and OASCR.
It will be enabled fundamentally by the SciDAC efforts alsopresently under way in the DOE Office of Science. Taking as an example effective
capabilities development from the DOE NNSA Accelerated Strategic Computing
Initiative (ASCI), new funding necessary for the success of the FSP is presently estimatedat approximately $20M for the first year of deployment (FY04), ramping up to a
procurement and Centers’ formation phase in FY05, and followed by steady, personneldriven level of effort funding for FY06, FY07 and FY08 inclusive. To achieve thegreatest productivity, this new research should be split between OFES and OASCR, withfusion scientists provided by OFES and applied mathematicians, computer scientists, andthe computational toolkits provided under the auspices of OASCR.
This joint
undertaking represents a significant opportunity for the DOE Office of Science to create acapability that will advance the understanding of fusion energy to a level unparalleledworldwide.

VII. Acknowledgements
The FESAC ISOFS Subcommittee acknowledges helpful contributions to this report frommembers of the fusion and applied mathematics communities, with particular thanks to
speakers at the May 23 ISOFS workshop. The Subcommittee also thanks the Theory
Coordinating Committee (TCC) for their letter, and the members of the PSACI PAC for
the concepts and suggestions provided in their 18 June 2002 letter to this Subcommittee,
and most particularly is appreciative of their enthusiasm for the Fusion Simulation
Project.

FESAC ISOFS Subcommittee First Report
12 July 2002

VIII. APPENDIX
February 22, 2002

Professor Richard D. Hazeltine, Chair
Fusion Energy Sciences Advisory Committee
Institute for Fusion Studies
University of Texas at Austin
Austin, TX 78712
Dear Professor Hazeltine:
This letter provides a charge to the Fusion Energy Sciences Advisory Committee (FESAC) toassist the Office of Fusion Energy Sciences (OFES) in preparing a roadmap for a joint initiativewith the Office of Advanced Scientific Computing Research (OASCR). Recent reports, such as
the FESAC report “Priorities and Balance within the Fusion Energy Sciences Program,” the“Report of the Integrated Program Planning Activity” (IPPA), and the NRC report “AnAssessment of the Department of Energy’s Fusion Energy Sciences Program,” have identified a
predictive understanding as a measure of the quality of the science and the maturity of theknowledge base of a field. The IPPA report lists several challenging10-year objectives for thefusion program, including “develop fully integrated capability for predicting the performance of
externally-controlled systems including turbulent transport, macroscopic stability, wave-particle
physics, and multi-phase interfaces.” This objective, as well as several other IPPA objectives
related to innovative confinement configurations, will require significantly enhanced simulationand modeling capability. Therefore, the goal of this initiative should be to develop an improved
capacity for Integrated Simulation and Optimization of Fusion Systems .

The initiative should be planned as a 5-6 year program, which would build on the improved
computational models of fundamental processes in plasmas that are being developed in the basetheory program and in the SciDAC program. Rough estimates are that an integrated simulation
initiative would require a total funding level of about $20 million per year, with funding for theplasma scientists provided by OFES and funding for the applied mathematicians, computerscientists, and computational resources provided by OASCR. Thus, the roadmap should include
not only human resources but also computer and network resources.
Please carry out the preparation of the roadmap using experts outside of FESAC membership, as
necessary, including experts recommended by the Advanced Scientific Computing Advisory
Committee. The sub-panel of experts should obtain community input through a series ofworkshops covering at least the following questions:••

What is the current status of integrated computational modeling and simulation?
What should be the vision for integrated simulation of toroidal confinement fusion
systems?

FESAC ISOFS Subcommittee First Report
12 July 2002

2
•
•
•
•

What new theory and applied mathematics are required for simulation and optimizationof fusion systems?
What computer science is required for simulation and optimization of fusion systems?
What are the computational infrastructure needs for integrated simulation of fusion
systems?
How should integrated simulation codes be validated, and how can they best be used toenable new scientific insights?

The ultimate product should be a roadmap document similar to the one developed for the
Genomes to Life Initiative (http://www.doegenomestolife.org/roadmap/index.html). Please
conduct a workshop on the first two questions above and provide a summary document withoverall program goals and objectives, major program deliverables, and a brief description of the
OFES and OASCR funded elements of the program by July 15, 2002, so that OFES would be
able to include a description of the program in the FY 2004 OMB budget request. Please
complete work on the final roadmap by December 1, 2002, in order to provide the detailed
information needed by OFES and OASCR to develop detailed program plans, program
announcements and grant solicitations.
I appreciate the time and energy that members of FESAC and FESAC sub-panels have provided
to the continuing efforts to develop program plans and roadmaps for the OFES program. I am
confident that the Committee's recommendations on a roadmap for Integrated Simulation and
Optimization of Fusion Systems will form a sound basis for beginning a joint OFES/OASCR
program.
Sincerely,

James F. Decker
Acting Director
Office of Science

FESAC ISOFS Subcommittee First Report
12 July 2002