2022 CCS-KISTI Workshop
CCS, Univ. Tsukuba
Feb. 22, 2022

Development and applications of SALMON- First-principles computations in optical science -Kazuhiro YABANA
Center for Computational Sciences
University of Tsukuba

Scalable Ab initio Light‒ Matter simulator for Optics and Nanoscience

Open-Source Software project:https://salmon-tddft.jp

Computation of light-matter interaction from atomic scaleusing first-principles computational method in materials scienceContents- (Self-) Interdisciplinary aspect in developing SALMON- Light propagation from first-principles: hierarchy linkage in SALMON- Co-design in developing SALMON and large-scale computation

Contents- (Self-) Interdisciplinary aspect in developing SALMON- Light propagation from first-principles: hierarchy linkage in SALMON- Co-design in developing SALMON and large-scale computation

How did SALMON start?
(Self-) Interdisciplinary Computational Science- Multidisciplinary Computational Science- Co-Design by computational and computer scientists

I am affiliated to two application-divisions:- Quantum Condensed Matter Physics- Nuclear Physics

Nuclei
Different scale

Atoms, Molecules, Solids

10-14 m10-9 mProtonNeutronDifferentinteraction

Commonphysics law

Nuclear force between
protons and neutrons

Coulomb force between
electrons and nuclei

Quantum mechanics for the dynamics of fermions(protons and neutrons in nuclei, electrons in atoms, molecules, and solids).

Common theories are useful: (Time-Dependent) Density Functional Theory

In electronic systems, Density Functional Theory (DFT) has been quite
successful (Novel prize awarded). But only for static problem.
𝜀! 𝜙! 𝑟 ="#$𝑝# + 𝑉!%& 𝑟 + 𝑉' 𝑟 + 𝑉() 𝑟

𝜙! 𝑟

Static (eigenvalue) problem, Kohn-Sham equation

W. KohnInitiate DFT in electronic systems
1998 Nobel prize in chemistry

Extension to dynamics: Time-Dependent Density Functional Theory (TDDFT)
𝑖ℏ

*
*+

𝜓! 𝑟, 𝑡 ="#$𝑝# + 𝑉!%& 𝑟 + 𝑉' 𝑟, 𝑡 + 𝑉() 𝑟, 𝑡 + 𝑉,(+ 𝑟, 𝑡

Popular method in nuclear physics, from 70ʼs.

𝜓! 𝑟, 𝑡TDDFT simulation for nucleus-nucleus collisions

Before I entered graduate course in 1982.

Time evolution of proton andneutron orbitals.

H. Flocard, S.E. Koonin, M.S. Weiss,
Phys. Rev. 17(1978)1682.

!
!
!
¶
y i (r , t ) = h[n (r , t )]y i (r , t )¶t!
! 2n( r , t ) = å y i ( r , t )

i#

i

! !
!
!
!
!
y (r1 , r2 ,", rN , t ) = A{y 1 (r1 , t )y 2 (r2 , t )"y N (rN , t )}16O-16O collision

Spatial grid 30x28x16, time step 4x102
using CRAY-1 (80MFLOPS)

Synthesis of superheavy elements

TDDFT simulation to produce Z=120 element
K. Sekizawa, K. Yabana, Phys. Rev. C93, 054616 (2016)
238U + 64Ni

Nh=Nihonium
Discovered by Japanese team, 2004.
Z=118:
Largest element at present(proton number: 92 + 28 = 120）In electronic systems, Density Functional Theory (DFT) has been quite
successful (Novel prize awarded). But only for static problem.
𝜀! 𝜙! 𝑟 ="#$𝑝# + 𝑉!%& 𝑟 + 𝑉' 𝑟 + 𝑉() 𝑟

𝜙! 𝑟

Static (eigenvalue) problem, Kohn-Sham equation

W. KohnInitiate DFT in electronic systems
1998 Nobel prize in chemistry

Extension to dynamics: Time-Dependent Density Functional Theory (TDDFT)
𝑖ℏ

*
*+

𝜓! 𝑟, 𝑡 ="#$𝑝# + 𝑉!%& 𝑟 + 𝑉' 𝑟, 𝑡 + 𝑉() 𝑟, 𝑡 + 𝑉,(+ 𝑟, 𝑡

𝜓! 𝑟, 𝑡Popular method in nuclear physics, from 70ʼs.

We extended static DFT in material science to electronic dynamics using nuclear-theory method.

Quantum electronic dynamics in a unit cell of silicon crystal
Time-dependent Kohn-Sham equation!

%

𝜀, 𝜙, 𝑟 = &' 𝑝& + 𝑉,-# 𝑟 + 𝑉* 𝑟 + 𝑉+) 𝑟%(𝑖ℏ !" 𝑢#$ 𝑟, 𝑡 = &' 𝑝 + ℏ𝑘 + ) 𝐴 𝑡
𝜙, 𝑟&+ 𝑉* 𝑟, 𝑡 + 𝑉+) 𝑟, 𝑡

𝑢#$ 𝑟, 𝑡"

𝐴 𝑡 = −𝑐 ∫ 𝐸 𝑡′ dtʼeigenvalue -> energy band

Electron density under pulsed light
Energy band of silicon

middle 90ʼsFirst-principles DFT
Band calculation!ω = 1.55eV
I = 1.0 ×1014 W/cm 2

Contents- (Self-) Interdisciplinary aspect in developing SALMON- Light propagation from first-principles: hierarchy linkage in SALMON- Co-design in developing SALMON and large-scale computation

Traditional computational methods in optical science
QUANTUM MECHANICS (QM)

ELECTROMAGNETISM (EM)First-principles quantum mechanics
calculations of optical constants

Light propagation
solving Maxwellʼs equations

Linear and local
constitutive relation<latexit sha1_base64="TIWqOAR5NOpfZddCzIZbiObM898=">AAACpXichVHLSsNAFD3Gd31V3QhugkXbgpapiIogFFRwI/hqFayUJJ3qYJqEZFrQ0h/wB1y4UlARP8ONC7cKfoK4VHDjwts04Av1hsy9c+acO2dmdMcUnmTsoUFpbGpuaW1rD3V0dnX3hHv7Mp5dcg2eNmzTdjd1zeOmsHhaCmnyTcflWlE3+Ya+N1db3yhz1xO2tS73Hb5d1HYsURCGJgnKhVOVrF5Q56sxP7vVURlXZ9WssKSal1E1yx1PmLYVk2MyGld90sIncjSeC0dYgvmh/iySQRFBEMt2+AJZ5GHDQAlFcFiQVJvQ4NG3hSQYHMK2USHMpUr46xxVhEhbIhYnhkboHo07NNsKUIvmtZ6erzZoF5N+l5Qqhtkdu2TP7IZdsUf29muvit+j5mWfsl7XcifXcziw9vqvqkhZYvdD9adniQKmfa+CvDs+UjuFUdeXD46e12ZWhysj7JQ9kf8T9sCu6QRW+cU4W+GrxwjRAyS/X/fPIjOeSE4mJlYmIqnp4CnaMIghxOi+p5DCIpaRpn3PcYs73CtRZUlZVzJ1qtIQaPrxJZTcO+Ghn/M=</latexit>

D(r, t) =Z

dt0 ✏(tt0 )E(r, t0 )e.g, FDTD (Finite-differencetime-domain method)

They are not sufficientin current optics frontier.

e.g, GW-BS, TDDFTOne of directions in current optics:

Generate and Utilize Intense and Ultrashort Laser Pulse
Nobel Prize in Physics 2018

In SALMON, we combine Maxwell and TDDFTLight propagation solving
Maxwellʼs equations (Classical)

1 @2
4⇡r ⇥ r ⇥ A + 2 2A =
j
c @t
c

Electron dynamics by TDDFT<latexit sha1_base64="AiB6KIFxfSLvotvK9V+XTcJw9Js=">AAAC2XichVFLSxxBEK6ZxNdqdI0XwcvgogQCS48sKoKg5pKjr1VhR5eetndt7Xkw07ugwxy8idccPOSkICL5GV78Ax70H4QcFbx4sGZ2VIwkVtPdX31VX3V1t+1LESpCrjX9w8e29o7Orlx3z6fevnz/55XQawSMl5knvWDNpiGXwuVlJZTka37AqWNLvmrvfEviq00ehMJzl9Wuz9cdWndFTTCqkKrmleVSW1LDUsLhofHaiyy7ZszGxlfDqgWURWYcsY2xOPMsnwZKUInMMzZUEn/STWeZJcsXKM0C23E1XyBFkprxFpgZKEBm817+DCzYBA8YNMABDi4oxBIohDgqYAIBH7l1iJALEIk0ziGGHGobmMUxgyK7g2sdvUrGuugnNcNUzfAUiTNApQEj5Iqck1tySX6R3+Thn7WitEbSyy7udkvL/Wrf4eDS/bsqB3cFWy+q//asoAaTaa8Ce/dTJrkFa+mbe0e3S1OLI9EoOSF/sP9jck0u8AZu846dLvDFn5DDDzD/fu63YGWsaI4XSwulwsxc9hWdMATD8AXfewJm4DvMQxnPvdFA69JyekXf1w/0w1aqrmWaAXhl+o9HbiK0Qg==</latexit><latexit sha1_base64="hOgGaEZ7e1Lvkv3STCswOPW06Rc=">AAADA3ichVHLahRBFL3dRo3jI2PcCG4Kh0hUMlSPQwyCEHWTZR7OJJCOTXVZPVOk+kF1zZDY9NKNP+BCEBREJDtx6yqb/ICLfIK4jODGhXe6exQNajVdde6559y6VeUnSqaG0kPLPjFx8tTpyTO1s+fOX5iqX5zupvFAc9HhsYr1hs9SoWQkOkYaJTYSLVjoK7Hubz8Y5deHQqcyjh6a3URshawXyUByZpDy6i+l2/eZJm6gGc/chGkjmcp/ImJy4iap9CS5S1wlAuNmldjJs1aYl+QsmRsXipivGLlZiUSecazmB+QeSrXs9c31Ry1Md70MG8gLtFTGO3wsccebevUGbdJikOPAqUADqrEc19+CC48hBg4DCEFABAaxAgYpfpvgAIUEuS3IkNOIZJEXkEMNvQNUCVQwZLdx7mG0WbERxqOaaeHmuIvCX6OTwAz9RN/RI3pA9+hn+v2vtbKixqiXXVz90isSb+rZ5bVv/3WFuBro/3L9s2cDASwUvUrsPSmY0Sl46R8+eX60dmd1JrtGX9Mv2P8rekj38QTR8Ct/syJWX0ANH8D587qPg26r6cw32yvtxuL96ikm4QpchVm879uwCEuwDB3g1oR1w7plte2n9p793v5QSm2r8lyC34b98Qe9ucNT</latexit>

@
i~
@t

⇢

1 ⇣
e ⌘2
i~r + A + Vion + VH + Vxc
i =
2m
c
o
e n ⇤⇣e ⌘j=
i~r + A i + c.c.
i2mc<latexit sha1_base64="EtO5Vvw/r9FfWi6J3kQ4x6Qm5ow=">AAAC13ichZHPaxQxFMffjrZd1/5Y9SL0ElwqrWWHbCkqglD14rE/3LbStEsmzeymzfwgk12ow+BNxFtPHjwpiIh/Ri/9BwT7J4jHCl5EfDs7KLbYvpDk5Zv3eXlJvFirxFJ6VHIuXBwaHilfqlweHRufqF65uppEXSNkU0Q6MuseT6RWoWxaZbVcj43kgaflmrf7qL+/1pMmUVH4xO7FcjPg7VD5SnCLUqtqUub5ZCcj90md+YaLVGbpXJARpqVvWUpYnKitWy01EKZJXbGOxw1hIfc0J7PkDyWyPNcDZI1qd+xMziI5S4Qr3EJlWataoy7NjZx2GoVTg8IWo+oHYLANEQjoQgASQrDoa+CQYNuABlCIUduEFDWDnsr3JWRQQbaLURIjOKq7OLZxtVGoIa77OZOcFniKxm6QJDBFP9OP9Jge0k/0K/3531xpnqNfyx7O3oCVcWvi1fWVH+dSAc4WOn+pM2u24MPdvFaFtce50r+FGPC9Z6+PV+4tT6U36Tv6Det/S4/oAd4g7H0X75fk8huo4Ac0Tj73aWd1zm3cdueX5msLD4uvKMMk3IBpfO87sACPYRGaeO4X+FUaKZWdp85z54XzchDqlArmGvxjzv5vEJSy3A==</latexit>

Two frameworks connecting two descriptions
Microscopic (single-grid) vs Macroscopic (multi-grid)iMicroscopic (Single-scale) vs. Macroscopic (Multi-scale)
Microscopic Maxwell+TDDFT
Incident

Macroscopic Maxwell+TDDFT

10nm-scale grid

TargetTransmitted

Z

Reflected

z
L

0

d

z
R

nm-scale grid #1

Single-scale approach using a common spatial grid…
…

nm-scale grid #2

Multiscale approach using different spatial grid

S. Yamada et al, PRB 98, 245147 (2018).

K. Yabana et al., PRB 85, 045134 (2012).

Nonlinear light propagation: multiscale Maxwell-TDDFT simulation
✓
◆
2
2
1 @
@
4⇡
AZ (t) =JZ (t)
2
2
2
c @t
@Z
c
<latexit sha1_base64="F371M0atF2uWJUS6KBoqhUkXVo4=">AAAC7nichVHNahRBEK6ZqFnXn6wRQRDM4JKwOWTpCYsGQYh4kZzy4yYhmWTp6fRsmvTOND29C3GYB9AH0IOIKASVPEYueQEPeQTx4CGCFw/W/KBoSFLDdH/1VX1V1d2+kiI2hBxZ9tCFi5eGK5erV65euz5SuzG6HEd9zXibRTLSqz6NuRQhbxthJF9VmtOeL/mKv/Mki68MuI5FFD4zu4pv9Gg3FIFg1CDVqb3wJA9Mw/ECTVnipgnbnE5Lz1NUG0ElMn+wY7L41FkZa3kFLbrbZtJJPD9wHqedtQY6j0pZy1MCO6VFdK6Idmp10iS5OSeBW4I6lDYf1T6CB1sQAYM+9IBDCAaxBAoxfuvgAgGF3AYkyGlEIo9zSKGK2j5mccygyO7g2kVvvWRD9LOaca5m2EXir1HpwDj5Qj6TY3JI9slX8uvUWkleI5tlF3e/0HLVGXl5e+nnuaoe7ga2/6rOnNlAADP5rAJnVzmTnYIV+sHz18dLDxfHkwnygXzD+d+TI3KAJwgHP9jeAl98A1V8APf/6z4Jlqeb7v2mu9Cqz86UT1GBO3APGnjfD2AWnsI8tLHvd+uWddcas5X9yn5rvytSbavU3IR/zP70GzznvF4=</latexit>

Grid points for light propagation

Nonlinear light propagation: multiscale Maxwell-TDDFT simulation
✓
◆
2
2
1 @
@
4⇡
AZ (t) =JZ (t)
2
2
2
c @t
@Z
c
<latexit sha1_base64="F371M0atF2uWJUS6KBoqhUkXVo4=">AAAC7nichVHNahRBEK6ZqFnXn6wRQRDM4JKwOWTpCYsGQYh4kZzy4yYhmWTp6fRsmvTOND29C3GYB9AH0IOIKASVPEYueQEPeQTx4CGCFw/W/KBoSFLDdH/1VX1V1d2+kiI2hBxZ9tCFi5eGK5erV65euz5SuzG6HEd9zXibRTLSqz6NuRQhbxthJF9VmtOeL/mKv/Mki68MuI5FFD4zu4pv9Gg3FIFg1CDVqb3wJA9Mw/ECTVnipgnbnE5Lz1NUG0ElMn+wY7L41FkZa3kFLbrbZtJJPD9wHqedtQY6j0pZy1MCO6VFdK6Idmp10iS5OSeBW4I6lDYf1T6CB1sQAYM+9IBDCAaxBAoxfuvgAgGF3AYkyGlEIo9zSKGK2j5mccygyO7g2kVvvWRD9LOaca5m2EXir1HpwDj5Qj6TY3JI9slX8uvUWkleI5tlF3e/0HLVGXl5e+nnuaoe7ga2/6rOnNlAADP5rAJnVzmTnYIV+sHz18dLDxfHkwnygXzD+d+TI3KAJwgHP9jeAl98A1V8APf/6z4Jlqeb7v2mu9Cqz86UT1GBO3APGnjfD2AWnsI8tLHvd+uWddcas5X9yn5rvytSbavU3IR/zP70GzznvF4=</latexit>

Grid points for light propagation

Electron dynamics calculation
at each grid point@
i~ unk,Z1 = Hk+ ec AZ1 (t) unk,Z1@t<latexit sha1_base64="GRDxtflxVcPAo6SMnu4eHt2t2cI=">AAAC13ichVFNSxwxGH53Wuu61rq2F6GX4GJRKktGShWhsNKLR792tToyZGJWw87ODJnsgg6htyK99dRDTy2IFH+GF/9Aof4E8ajgpZS+Ozu0+FXfkOTJk/d58yTxIl/GmtKTnPXgYc+j3nxfof/xwJPB4tDTWhy2FBdVHvqhWvVYLHwZiKqW2herkRKs6flixWu87eyvtIWKZRgs651IbDTZViDrkjONlFtU0tn2mCJOXTGeOBFTWjLf/EVEG9JykyBxvDppmIlkzbWNIW/InNulyMtMK0zCTcrNGjdNG9Pjt4ndYomWaRrkJrAzUIIs5sPiATiwCSFwaEETBASgEfvAIMa2DjZQiJDbgAQ5hUim+wIMFFDbwiyBGQzZBo5buFrP2ADXnZpxquZ4io9doZLAKP1Bv9NzekwP6Sn9dWetJK3R8bKDs9fVisgd/Di8dHmvqomzhu1/qv961lCH6dSrRO9RynRuwbv69u7n86WZxdHkBf1Gz9D/V3pCj/AGQfuC7y+IxS9QwA+wrz/3TVCbLNuvy/bCq1JlOvuKPDyHERjD956CCszBPFTx3J/wO9eby1vvrPfWB2uvm2rlMs0zuBLWpz9yKLUx</latexit>@
i~ unk,Z2 = Hk+ ec AZ2 (t) unk,Z2@t<latexit sha1_base64="CHvnVfeqGPypbRXy7QH9cQkE8Gg=">AAAC63ichVHLThRBFL3dIOKgMOLGBBYVJhiMZlJN5BETE4wbljwcINKkU13UMJXpV6prJsFKbV24dWECbiQhhPAZbvwBF3yC0R0mblx4p6ejUXzcTlede+qeW6eqwiySuab03HEHBq8MXR2+Vhm5fmN0rHpzfCNPO4qLBk+jVG2FLBeRTERDSx2JrUwJFoeR2AzbT3rrm12hcpkmT/V+JnZitpfIpuRMIxVUX0i/FTJF/KZi3PgZU1qyyP5ARFvSCUxi/LBJ2va+eRbMWksekeWgT5F7pVZYw23BPbZBUTaj7/5R7LfyjHFh5uIYE76b6jyo1midFkEuA68ENShjJa2egA+7kAKHDsQgIAGNOAIGOX7b4AGFDLkdMMgpRLJYF2ChgtoOVgmsYMi2cdzDbLtkE8x7PfNCzXGXCH+FSgLT9AM9pRf0PT2jH+m3v/YyRY+el32cw75WZMHYy9vrX/+rinHW0Pqp+qdnDU1YLLxK9J4VTO8UvK/vPn99sf5wbdrcoUf0E/p/S8/pOzxB0v3Cj1fF2iFU8AG836/7MtiYrXvzdW/1QW1psXyKYZiAKZjB+16AJViGFWjgvp+dcWfCmXRj95V74L7pl7pOqbkFv4R7/B39LLyN</latexit>

K. Yabana, T. Sugiyama, Y. Shinohara, T. Otobe, G.F. Bertsch,“Time-dependent density functional theory of strong electromagnetic fields in crystalline solids”, Phys. Rev. B85, 045134 (2012).

···

Macroscopic (multi-scale) Maxwell-TDDFT: pulsed light on Si nano-film~200 grid points
for 1µm film

Macroscopic (multi-scale) Maxwell-TDDFT: pulsed light on Si nano-film
Front
Change of electron densityfrom ground state<latexit sha1_base64="KjQtzdMcVyqpvEhecWQzwkEK9ZU=">AAACmXichVHLSsNAFD2Nr1pfVTdCN8GiKGiZSlERBF+guFJrVbBSkjjV0DQJybSgoT/gDwi6UhARf0Jwo+5d+AniUsGNC2/TiKioN2Tm3DP33Dkzo9qG7grGHkJSXX1DY1O4OdLS2tbeEe3sWnOtkqPxjGYZlrOhKi43dJNnhC4MvmE7XCmqBl9XC7PV9fUyd1zdMlfFns23isqOqed1TRFE5aKp7Bw3hCKbA15WzctOZUgMypNf02HZzHnz6coHN5iLxlmC+SH/BMkAxBHEkhU9RxbbsKChhCI4TAjCBhS49G0iCQabuC14xDmEdH+do4IIaUtUxalCIbZA4w5lmwFrUl7t6fpqjXYx6HdIKaOP3bML9sxu2CV7ZG+/9vL8HlUvezSrNS23cx0HPenXf1VFmgV2P1V/ehbIY9z3qpN322eqp9Bq+vL+4XN6YqXP62en7In8n7AHdk0nMMsv2tkyXzlGhB4g+f26f4K1kURyNJFaTsWnZoKnCCOGXgzQfY9hCgtYQob2PcIVbnEnxaRpaUFarJVKoUDTjS8hpd8BKg2bgQ==</latexit>n(r, t) = n(r, t)Thin film
Silicon, 1 µm thick
Laser pulse
800nm (1.55eV=below gap)
4x1012W/cm2

nGS (r)MiddleBack~200 grid points
for 1µm film

High harmonic generation in reflection/transmission waves
Thin film
Silicon, 1 µm thick

Laser pulse
800nm (1.55eV=below gap)
4x1012W/cm2

104

104

RHHG

THHG

100
|E(ω)|2 (a.u.)|E(ω)|2 (a.u.)

100
10−4
10−8

10−4
10−8
10−12

10−12
0

10

20
30
energy (eV)

40

50

0

S. Yamada et.al, Phys. Rev. B107, 035132 (2023)

10

20
30
energy (eV)4050Contents- (Self-) Interdisciplinary aspect in developing SALMON- Light propagation from first-principles: hierarchy linkage in SALMON- Co-design in developing SALMON and large-scale computation

Development of SALMON
Co-Design- Multidisciplinary Computational Science- Co-Design by computational and computer scientists

Code development:- Physics researchers in Quantum Condensed Matter Physics- Computer researchers in High Performance Computing Systems

In-house collaboration between
computational and computer scientists.

2014.04-2018.09 Univ. Tsukuba, Grad. Student, Department of Computer Sciencesupervised by Prof. Boku.
Ph.D thesis“Co-design for first-principles electronic dynamics simulationin cutting-edge high performance computing systems”
2018.10-2020.10 Researcher, CCS, University of Tsukuba
2020.11Private company

Dr. Yuta Hirokawa

Compile optionsin SALMON

Large-scale computing using Fugaku
Microscopic (single-scale) Maxwell + TDDFT + MD
Dynamics of light EM fields, electrons,
and ions of more than 10,000 atoms(b)

Incident

Fugaku Supercomputer(a)Vacuum8.65 [nm]SiO2 slab(c)~6.6 [nm]Top 500 1st (2020.6)
415 Pflops
158,976 A64FX (48 core)

5.13 [nm]3.95 [nm]27,648 nodes, about 1/6 of full system

Performance (Weak Scaling)
SALMON: O(N2) scaling

FFT for Hartree

Achieved ~1s/iter up to 10,000 atoms.
Hamiltonian operation (Stencil, nonlocal
PS) costs about half of total time.
60 ‒ 70 % communications (halo, sum, etc).
FFT is bottleneck in view of weak scaling.

1,704

3,408
6,816
# atoms

13,632

SALMON in GPU machine- Code mostly written in fortran90- To get performance, need to carry out all calculations in GPU- Code frequently developed and modified by physics researchers

OpenACC + Unified MemoryPerformance comparison: CPU vs GPU

K-point.

13

Orbitals 102480

4 x Intel Xeon Phi(Oakforest PACS)
3TFLOPS/CPU

4 x NVIDIA Tesla V100
(Cygnus)
7TFLOPS/GPU
writing info etc.

70

calc_total_energy_periodic

60

Current(All Reduce)

50
[sec]

Spatial grid 643

Execution Time / 100 iterations

Electron dynamics calculation in 512 atom unit cell of silicon

Density(All Reduce)

40
30
20

calc_eigen_energy
writing energies
calc. projection
calculating curr
Exc_Cor routine

Stencil + nonlocal
potential

Hartree routine
calculating rho

10time propagation

0
OFP

x 2.8

CygnusGPU > CPU using OpenACC+Unified Memory

Vbox

Weak scaling performanceElectron dynamics calculation in 512 atom unit cell of silicon

K-point. 13 vs 23
Orbitals.

1024

writing info etc.
calc_total_energy_periodic

20
iterations [sec]Spatial grid

643

Execution Time / 10025

calc_eigen_energy
writing energies

15

calc. projection
calculating curr

10

Exc_Cor routine
Hartree routine

5

calculating rho
time propagation0k (1,1,1)

4 x V100k (2,2,2)

Vbox

32 x V100 (8 nodes）

Almost perfect scaling performanceSummaryhttps://salmon-tddft.jp

We develop SALMON, first-principles calculation for optics and nano-sciences.
It describes electronic dynamics induced by light.
Unique feature: light-propagation calculation from first-principles.
In the development,- (self-) interdisciplinary computational physics (nuclear method meets first-principles calculation).
- In-house co-design was successful.
- Large-scale computing, efficient use of GPU.

Acknowledgement
Financial supports

Young collaborators (SALMON project)

Atsushi Yamada
RIST

Shunsuke Yamada Takashi Takeuchi

Arqum Hashmi

Kansai Photon Sci. Inst

Kansai Photon Sci. Int.

RIKEN

Q-LEAP
Supercomputers

Mitsuharu Uemoto
Kobe Univ.

Yuta Hirokawa
Private company

Shunsuke Sato
U. Tsukuba