SALMON TUTORIAL, TSUKUBA, 2017

Basics of SALMON

Masashi Noda
Institute for Molecular Science
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/

1What is SALMON?
Scalable Ab-initio Light-Matter simulator
for Optics and Nanoscience• A software to calculate electron dynamics and optical responses• Real-time and real-space• Treatment of light-matter interaction• Dielectric functions and polarizability
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 20172

Overview of SALMON
K. Yabana and G. F. Bertsch, Phys. Rev. B 54, 4484 (1996)
ARTEDGCEED• solid/surface• laser damage• attosecond• nanostructure (multiscale)• cluster/surface/interface• optical near field• bias voltage• nanostructure (microscale)

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 20173

Characteristics of SALMON• Real-time electron dynamics• Massive parallelization• Non-linear optical response• Couple with Maxwell equation

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 20174

Development environment• Web page: http://salmon-tddft.jp• License: Apache 2.0• Mailing list: salmon-users@salmon-tddft.jp (contact address for inquiry)
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 20175

Developers• Isabella Floss (TU Wien, Austria)• Yuta Hirokawa (University of Tsukuba, Japan)• Kenji Iida (Institute for Molecular Science, Japan)• Kazuya Ishimura (Institute for Molecular Science, Japan)• Kyung-Min Lee (Max Planck Institute for the Structure and Dynamics of Matter, Germany)• Katsuyuki Nobusada (Institute for Molecular Science, Japan)• Masashi Noda (Institute for Molecular Science, Japan)• Tomohito Otobe (National Institutes for Quantum and Radiological Science and Technology, Japan)• Shunsuke Sato (Max Planck Institute for the Structure and Dynamics of Matter, Germany)• Yasushi Shinohara (University of Tokyo, Japan)• Takashi Takeuchi (Institute for Molecular Science, Japan)• Xiao-Min Tong (University of Tsukuba, Japan)• Mitsuharu Uemoto (University of Tsukuba, Japan)• Kazuhiro Yabana (University of Tsukuba, Japan)• Atsushi Yamada (University of Tsukuba, Japan)• Shunsuke Yamada (University of Tsukuba, Japan)• Maiku Yamaguchi (University of Tokyo, Japan)
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 2017

(Alphabetic order)
6

Time-dependent Kohn-Sham equations• isolated system (length gauge)𝑖ℏ𝜕𝜓 𝒓, 𝑡𝜕𝑡 𝑛12= − 𝛻 2 + 𝑉𝑒𝑥𝑡 (𝒓, 𝑡) + 𝑉𝑖𝑜𝑛 + 𝑉𝐻 + 𝑉𝑥𝑐 𝜓𝑛 (𝒓, 𝑡)

𝜓𝑛 : Kohn-Sham orbital𝑉𝐻 : Hartree potential

𝑉𝑒𝑥𝑡 (𝒓, 𝑡): External scalar potential

𝑉𝑥𝑐 : Exchange-correlation potential

𝑉𝑖𝑜𝑛 : Electron-nuclear interaction potential• periodic system (velocity gauge)
𝜕𝑖ℏ 𝑢𝑛𝒌 𝒓, 𝑡𝜕𝑡

12=𝑒−𝑖ℏ𝛻 + ℏ𝒌 + 𝑨 𝑡
𝑐2+ 𝑉𝑖𝑜𝑛 + 𝑉𝐻 + 𝑉𝑥𝑐 𝑢𝑛𝒌 𝒓, 𝑡

𝑢𝑛𝒌 : Bloch orbital𝒌 : wave vector

𝑨(𝑡): External vector potential
Physical quantities are expressed on grid.
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/SALMON TUTORIAL, TSUKUBA, 20177

Flowchart of SALMON
GS (Ground State) calculation(determination of initial states)

RT (Real-time) calculation(propagation of states)“GS-RT” (only for periodic systems)
GS calculation• Isolated system (length gauge)12− 𝛻 2 + 𝑉𝑖𝑜𝑛 + 𝑉𝐻 + 𝑉𝑥𝑐 𝜓𝑛 𝒓 = 𝜖𝑛 𝜓𝑛 (𝒓)𝜖𝑛 : 1-particle energy• periodic system (velocity gauge)
12−𝑖ℏ𝛻 + ℏ𝒌 2 + 𝑉𝑖𝑜𝑛 + 𝑉𝐻 + 𝑉𝑥𝑐 𝑢𝑛𝒌 𝒓 = 𝜖𝑛𝒌 𝑢𝑛𝒌 (𝒓)Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/SALMON TUTORIAL, TSUKUBA, 20178

Pseudopotential• Norm-conserving pseudopotential (Kleiman and Bylander separable form)
𝑉𝑖𝑜𝑛 = 𝑉𝑙𝑜𝑐𝑎𝑙 +𝑃𝑆𝑃𝑆|𝜓𝑙𝑚𝑣𝑙 ⟩⟨𝑣𝑙 𝜓𝑙𝑚|𝑙𝑚 ⟨𝜓𝑃𝑆 𝑣 𝜓𝑃𝑆 ⟩𝑙𝑚𝑙

𝑙𝑚

𝑉𝑙𝑜𝑐𝑎𝑙 : local part of pseudopotential
𝑣𝑙 : non-local part of pseudopotential𝑃𝑆𝜓𝑙𝑚: pseudowavefunction• pseudopotential files treated by SALMON Yabana-Bertsch format

 .pspnc (ABINIT format:
https://www.abinit.org/sites/default/files/PrevAtomicData/psp-links/psp-links/lda_tm)
 .cpi and .fhi (fhi98PP format:
https://www.abinit.org/sites/default/files/PrevAtomicData/psp-links/psp-links/lda_fhi)

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/SALMON TUTORIAL, TSUKUBA, 20179

Exchange-correlation functions• Adiabatic approximation• Exchange-correlation potential
𝑉𝑥𝑐 (𝒓) = 𝜖𝑥𝑐 ( 𝑛 , 𝒓) + 𝑛(𝒓)𝛿𝜖𝑥𝑐 ( 𝑛 ,𝒓)𝛿𝑛(𝒓)𝜖𝑥𝑐 : energy per electron

𝑛: electron density• Exchange-Correlation functions
Isolatedsystems

Periodicsystems

LDA-PZ (Perdew-Zunger LDA)

LSDA-PZ (Perdew-Zunger LSDA)

PAM (Perdew-Zunger LDA with modification)

TBmBJ (Tran-Blaha meta-GGA exchange withPerdew-Wang correlation)



Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201710

Parallelization (1)• isolated systems
Kohn-Sham orbital: 𝜓𝑛 (𝒓) MPI: orbital and domain
𝜓1∼10
𝜓11∼20𝜓21∼30𝜓31∼40𝜓1∼10
𝜓11∼20𝜓21∼30𝜓31∼40• periodic systems
Bloch orbital: 𝑢𝑛𝒌 (𝒓) MPI: orbital and k points

𝑢1∼10,𝟏∼𝟒 𝑢1∼10,𝟓∼𝟖𝑢11∼20,𝟏∼𝟒 𝑢11∼20,𝟓∼𝟖𝑢21∼30,𝟏∼𝟒 𝑢21∼30,𝟓∼𝟖𝑢31∼40,𝟏∼𝟒 𝑢31∼40,𝟓∼𝟖

domain division (2×2×2)

 OpenMP: domain OpenMP: orbital and k points

This parallelization is done automatically.
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201711

Parallelization (2)• multiscale calculation
 MPI:
macroscopic grids
orbital and k points (𝑢𝑛𝒌 (𝒓))in microscopic cell

 OpenMP:
orbital and k points (𝑢𝑛𝒌 (𝒓))in microscopic cellThis parallelization is done automatically.

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201712

Performance (1)Ag54@Si454

Ag146@Si345

Used computer: K computer (RIKEN)Time steps: 1000

Diameter: 2.5nm

SystemCPU performance

Ag54@Si454

12.1% (1,944 processes)

Ag146@Si345

9.2% (4,000 processes)

Calculations for product runs end in 1.4 hours.
Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201713

Performance (2)Au14144nm

Used computer: K computer (RIKEN)
Number ofprocesses

CPUperformance

7,800

11.0%

15,6009.3%

Calculations for product runs
end in 7 hours with 15,600
processes.

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201714

Performance (3)Laser Excitation Silicon Nanosphere

Computation~250 nm

SiE(t)EM Energy

Excitation Energy• World-class many-core supercomputer“Oakforest-PACS” (OFP)•••~ 24,000 [Node Hours] for Computation

Processer:• Intel Xeon Phi 7250
(68 cores 1.4GHz base clock)Number of Nodes:• 8208 nodes (use up to 8192)
Theoretical Peak Performance• 25 PFLOPS

Scalable Ab-initio Light-Matter simulator for Optics and Nanoscience
http://salmon-tddft.jp/
SALMON TUTORIAL, TSUKUBA, 201715