S12-3
Multi-scale simulations of electron- and ion-scale turbulencein magnetic fusion plasma
核融合プラズマにおける電子／イオン系乱流のマルチスケールシミュレーション
Shinya Maeyama
前山伸也
Japan Atomic Energy Agency, 178-4-4 Wakashiba, Kashiwa, Chiba 277-0871, Japan
日本原子力研究開発機構 〒277-0871 千葉県柏市若柴178-4-4
In magnetic fusion plasma researches, plasma turbulence at ion and electron scales has often been treatedseparately by assuming their scale-separation. The gyrokinetic simulation code GKV, which is highly
optimized for the K computer by developing the efficient MPI process mapping andcomputation-communication overlap techniques, allows direct numerical simulations of the multi-scale
plasma turbulence. It is demonstrated that the cross-scale interactions exist even with real ion-to-electronmass ratio, where ion and electron scales are well separated by a factor of the square root of mass ratio.

1. Introduction
The understanding of the plasma turbulence isstrongly promoted by theoretical and numericalstudies based on gyrokinetics [1]. Most of theprevious gyrokinetic simulations of plasma
turbulence treat ion-scale turbulence (a typicalwave length ~ the ion Larmor radius ρti and afrequency ~ the ion transit frequency vti/R) andelectron-scale turbulence (a typical wave length ~the electron Larmor radius ρte and a frequency ~the electron transit frequency vte/R) separately, by
assuming their scale-separation. However, since
the assumption is not trivial, it should be clarified
whether the cross-scale interactions between ionand electron-scale turbulence exist or not. Suchan analysis resolving both of ion and electron
scales requires huge computations, and therefore
we had to wait the appearance of theextreme-scale supercomputer such as the Kcomputer.
The GKV code is one of major gyrokinetic
simulation codes in fusion community, and its
physics capability is extended along with thedevelopment of simulation models and of highperformance computers: ion-scale turbulence in a
simple circular torus tokamak plasma on thePlasma Simulator (1.4TFLOPS, at NIFS) in 2006[2], ion-scale turbulence simulations in a complex
non-axisymmetric LHD plasma on the Earth
Simulator (40TFLOPS, at JAMSTEC) in 2008 [3],and ion-scale turbulence simulations with rapidelectron motions on the Helios (1.2PFLOPS, atIFERC-CSC) in 2013 [4]. For now, in 2014 theJapanese fastest supercomputer K (10PFLOPS, atRIKEN) enables multi-scale simulations fully

resolving ion and electron scales [5]. By means of
the direct numerical simulations, we analyze thedynamics of the multi-scale plasma turbulence,
which reveal the existence of the cross-scaleinteractions.
2. Optimizations of the GKV code on the Kcomputer
The GKV code solves nonlinear time evolutionof perturbed distribution functions of ions andelectrons with electromagnetic fluctuations. In thenumerical point of views, the code is regarded as acomputational fluid dynamics application in the 5D
phase space (3D configuration space and 2D
velocity space). The 2D dynamics perpendicular tothe confinement magnetic field is calculated by thespectral method using parallel 2D FFTs, and theother 3D dynamics are calculated by the finite
difference methods. The computations areparallelized by using the OpenMP/MPI hybrid
parallelization.
Multi-dimensional
domaindecomposition introduces various inter-nodecommunications: data transpose for parallel FFTs,
point-to-point communications for finite differences,and reduction communications for charge andcurrent density evaluations (integrations over thevelocity space and particle species). Since
multi-scale plasma turbulence simulations demandhigh computational performance and memory
requirements, which can be attained by extreme
scale parallelization over 100k cores, the reductionof the inter-node communication cost andimprovement of the strong scaling is of criticalimportance.
The K computer is a distributed memory super-(a) At initial saturation(b) At steady state
5100

2.550

00-2.5-50-5-100Fig. 2 Snapshots of electrostatic potentials in
multi-scale simulations.
Fig. 1 Strong scaling of multi-scale plasma
turbulence simulation with ~6×1011grids on the Kcomputer.

computer, which consists of 82,944 computation
nodes (2 GHz, 16 GFlops/core, Memory-BW 8GB/s/core, 8 cores/node) connected by Torus fusion(Tofu) interconnect (6D mesh/torus topology,Interconnect-BW 5 GB/s × 4, Multi-way
simultaneous communications: 4 sends + 4
receives). To minimize the communication cost ofthe GKV code on the K computer, we develop thesegmented rank mapping on the Tofu network
which is designed so that the various inter-nodecommunications are preformed locally andindependently on a 3D torus network [6]. In
addition, we implement the pipelined overlaps ofcomputations and communications for spectral andfinite difference calculations. Thanks to these
optimizations, the computational performance of
the GKV code is significantly improved, whichshows excellent strong scaling up to ~600k coreswith the high parallelization rate ~99.99994% asplotted in Fig. 1.
4. Multi-scale turbulence simulations
Using the highly-optimized code, we first
analyzed multi-scale plasma turbulence simulations
resolving from electron to ion scales with real
ion-to-electron mass ratio and β value. As shown in
Fig. 2, at first high-growth-rate electron
temperature gradient modes appear and create
radially elongated eddies, streamers (a). After that,
ion temperature gradient modes slowly grow up andbecome dominant at the steady state (b). At thesame time, electron-scale streamers are suppressedby the ion-scale turbulent eddies. Thus, the results
demonstrate the existence of the cross-scaleinteractions between electron- and ion-scale
turbulence even with the real mass ratio, whereelectron and ion scales are well separated by afactor of the square root of mass ratio. The analysis

with the real β value shows that electromagnetic
perturbations stabilize ion temperature gradientmodes and changes the multi-scale interactions.
5. Summary
Gyrokinetic
simulations
promotetheunderstanding of the plasma turbulence along withthe development of high performance computing.
Optimizations of the GKV code on the K computer(segmented process mapping and pipelined
computation-communication overlaps) significantly
improve the computational performance of theGKV code. The highly-optimized code allows
multi-scale plasma turbulence simulations inmagnetic fusion plasma. It reveals the existence ofthe multi-scale interactions between electron- andion-scale turbulence.
Acknowledgments
The author would like to thank Dr. Y. Idomura
and Prof. T.-H. Watanabe for their supports andhelpful discussions. This work is supported by
HPCI Strategic Program Field No. 4 and MEXT
KAKENHI Grant No. 26800283. Computations arecarried out on the K computer at RIKEN and Helios
at IFERC-CSC.
References[1] X. Garbet, Y. Idomura, L. Villard, and T.-H.
Watanabe: Nucl. Fusion 50 (2010) 043002.
[2] T.-H. Watanabe, and H. Sugama: Nucl. Fusion 46(2006) 24.
[3] T.-H.
Watanabe,H.
Sugama,andS.
Ferrando-Margalet: Phys. Rev. Lett. 100 (2008)
195002.
[4] S. Maeyama et al.: Comput. Phys. Commun. 184(2013) 2462.
[5] S. Maeyama et al.: Proc. 25th IAEA FEC, St.
Petersburg, 2014, TH/1-1.
[6] S. Maeyama et al.: Int. Conf. High Perform. Comput.
Network. Storage and Analysis (SC13), Denver,2014.