3.6

Software Advancement Projects,

GPGPU Implementation, andWorkshop Support385

Activity Report 2019 / Supercomputer Center, Institute for Solid State Physics, The University of Tokyo

Development of ab-initio configuration sampling toolkit(abICS) for combining first-principles calculations withextended ensemble methods
Shusuke KASAMATSU1, Yuichi MOTOYAMA2, Kazuyoshi YOSHIMI2,Yoshiyuki YAMAMOTO2, Osamu SUGINO2, Taisuke OZAKI2
1

Faculty of Science, Academic Assembly, Yamagata University,
Yamagata 990-85602

Institute for Solid State Physics, University of Tokyo, Kashiwa 277-8581

In functional materials, varying types ofon

efficientparallelalgorithms

forconfigurational disorder are often employed tothermodynamic sampling. Figure 1 shows ourobtaincomputationalfavorable

properties

foragivenscheme.

ThereareNrep

application. The ability to simulate such

Metropolis samplers running in parallel atdisorder as a function of temperature should bedifferent temperatures. Each of the samplers

immenselywithspawns parallel DFT processes (we used VASPexperiment. It would also enable prediction and[1]) to perform local structural relaxation anddesign of properties in materials with realisticenergy calculation at every Metropolis step.

levels of disorder. To achieve this, a natural

The temperatures are swapped according to theway may be to combine first-principles

replica exchange algorithm [2] to speed up thecalculations with the Metropolis Monte Carlosampling. We successfully demonstrated thatalgorithm. However, this is quite costly insufficient sampling can be performed onterms of computational power, so many

meaningfully large supercells by calculating theworkers have opted to first derive lightweight

degree of disorder in MgAl2O4 [3].

helpfulfor

comparingmodels from first-principles calculations, thenuse that model for Monte Carlo sampling. The
problem is that deriving lightweight andreliable models becomes increasingly difficult
with increasing complexity of the system(various dopants, charged defects, interfaces,etc.).
Figure 1 Our scheme for parallel
configuration sampling. Reproduced from Ref.
[3].

In last year’s ISSP joint-use project, SK
proposed to bypass such fitting and instead rely

386

Activity Report 2019 / Supercomputer Center, Institute for Solid State Physics, The University of TokyoAs this year’s Software Advancementfiles to abICS input.

Project, we proposed to develop an easy-to-use

We believe that abICS will be immenselyinterface for this computational scheme. The

useful for making efficient use of next-result

ab-Initio

generation supercomputers due to its multi-

Configuration Sampling toolkit, or abICS forlayered parallelism. We are also aiming to bringshort. It is written in Python 3 and is registered

togetherat PyPI (Python Package Index; pypi.org). It

simulation, and statistical physics using abICS

can thus be installed in most environments

as a hub. Because of modular coding practices

rathereasilythroughthecommandemployed in our project, it should be relatively“pipinstall-–userabics”.

Now,easy to implement interfaces for other solvers,

abICS can sample using the replica exchangeor to implement new sampling schemes. Please

Monte Carlo algorithm, and we are planning tohave a look at our homepage [8] and do not

add support for other extended ensemble

refrain from contacting us if you are interested

methods soon. The parameters for the samplingin using or extending this software.

hasbeenreleasedasinformation

science,materials

are controlled by an input file in TOML format,
whichisaminimalandReferences

easy-to-read

configuration format that is being used in many[1] https://vasp.at/software projects [4]. The parameters for the[2] K. Hukushima and K. Nemoto: J. Phys. Soc.

DFT calculations such as k-point sampling,

Jpn. 65 (1996) 1604.

basis set specification, convergence parameters,[3] S. Kasamatsu and O. Sugino: J. Phys.:etc. are controlled by input formats of theCondens. Matter 31 (2019) 085901.

specified solver. As the solver, abICS supports[4] https://github.com/toml-lang/toml

the use of DFT codes VASP, Quantum Espresso[5] https://www.quantum-espresso.org/[5], and OpenMX [6] as of May 2020. It also

[6] http://www.openmx-square.org/

has experimental support for aenet [7], which[7] http://ann.atomistic.net/

uses neural network potentials. Preprocessing[8] https://www.pasums.issp.u-

scripts are provided for converting structure

tokyo.ac.jp/abics/en387

Activity Report 2019 / Supercomputer Center, Institute for Solid State Physics, The University of Tokyo

GPU acceleration for solving Bethe-Salpeter equation−1e iνn ) = [χe−1χ(q,0 (q, iνn ) − Γ(iνn )] ,Time [sec]The Bethe-Salpeter equation (BSE) provides two-particle responses in interacting
many-body systems. In the context of dynamical mean-field theory (DMFT), for example,
the momentum-dependent dynamical susceptibility χ(q, iνn ) is obtained by solving the BSE.
The BSE is symbolically written as

1600

60

Time (24 threads + 1 GPU)
1400
(24 threads + 0 GPU)
Speedup Rate
120050401000
800

30600

20400
10200(1)00100

where the bold symbols indicate matrices of2 N 2 N . Here, N
size M = Nsp
sp is the numorb ω
ber of spin components, Norb is the number oforbitals, and Nω is the number of the fermionicMatsubara frequencies. Solving Eq. (1) requires computations of matrix multiplications
and the inverse of a matrix. Therefore, thecomputational cost is O(M 3 ).
The matrix size M for typical models isshown in Table 1. Here, the value of Nω isfixed at 20, but a larger value is necessary toaddress low-temperature properties. Namely,
M in Table 1 should be regarded as a minimum estimation. We see that d-orbital systems require M = 500 without spin-orbit coupling (SOC) and M = 2,000 with SOC. For
the f -orbital, we need M ≈ 4,000.
Table 1: The matrix size M for typical models.
Model
Nsp Norb NωM
1 orbital
1
1 20202 orbital
1
2 20801
3 20180
3 orbital
5 orbital
1
5 20500
2
5 20 2,000
5 orbital (SOC)
7 orbital (SOC)
2
7 20 3,920

Speedup Rate

Junya Otsuki1 and Kazuyoshi Yoshimi2
1Research Institute for Interdisciplinary Science, Okayama University, Okayama 700-8530
2Institute for Solid State Physics, University of Tokyo, Chiba 277-8581

1000

2000
Matrix Size

4000

Figure 1: Computation time for solving theBSE (left axis) and the speedup rate achievedby the use of GPU (right axis).
Figure 1 shows a benchmark. We used 1
node (24 thread) of System B in ISSP supercomputer center. Without GPU, the computational time grows in proportion to M 3 asexpected (filled squares). Use of GPU suppresses the growth of the time (filled circles).
A significant speedup is achieved for M & 100:
The speedup rate (the time without GPU divided by the time with GPU) is 13 and 52for M = 1,000 and 4,000, respectively. Thisbenchmark indicates that the GPU acceleration is highly beneficial for d and f orbitals
especially on the existence of SOC.
As a future work, our BSE code will be integrated into the open-source DMFT software
DCore developed under the support of ISSP [1].
Acknowledgment: This work was supported by support service of program portability to GPGPU.
Reference:[1] https://github.com/issp-center-dev/DCore388

Activity Report 2019 / Supercomputer Center, Institute for Solid State Physics, The University of Tokyo

Report of CCMS hands-on sessions in the 2019 fiscal year
Takahiro Misawa
Institute for Solid State Physics, University of Tokyo
Kashiwa-no-ha, Kashiwa, Chiba 277-8581

In the 2019 fiscal year, center for computational materials science (CCMS) in institutefor solid state physics (ISSP) held two handson sessions by using supercomputer (sekirei) ofISSP. A list of the hands-on sessions is shownin Table 1. In this report, we briefly summarize the hands-on sessions in the order of date.
DSQSS [1] is an open-source software package for solving quantum many-body problems using the quantum Monte Carlo methodthat sample the Feynman path integral using the worm update.
Development ofDSQSS was supported by “Project for advancement of software usability in materialsscience”(PASUMS) [2] in the 2018 fiscal year.
In the hands-on session, Naoki Kawashima andco-developers explained the basics of the quantum Monte Carlo method and gave a tutorialof DSQSS [3].
DCore [4] is a tool for performing quantum
many-body simulations based on the dynamical mean-filed theory (DMFT). Developmentof DCore was also supported by PASUMUS inthe 2017 fiscal year. In the hands-on session,
Hiroshi Shinaoka and co-developers explained
the basics of the DMFT and gave a tutorial of
DCore [5].

DateSoftware

Main lecturer

Jun. 6
Dec. 26

DSQSS
DCoreN. Kawashima
H. Shinaoka

Table 1: List of software packages used in
CCMS hands-on sessions.

389

References[1] https://ma.issp.u-tokyo.ac.jp/en/app/180[2] https://www.pasums.issp.u-tokyo.ac.jp[3] https://ccms.issp.u-tokyo.ac.jp/event/1942[4] https://ma.issp.u-tokyo.ac.jp/en/app/1004[5] https://ccms.issp.u-tokyo.ac.jp/event/2486

Activity Report 2019 / Supercomputer Center, Institute for Solid State Physics, The University of Tokyo

Supercomputer course of Computational Materials Design(CMD®) workshop
Masaaki GESHI1, Yoshitada MORIKAWA2, Tomoya ONO3
1

Institute for NanoScience Design,

Osaka University, 1-2 Machikaneyama, Toyonaka, Osaka 560-0043
2

Department of Precision Engineering, Graduate School of Engineering,Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871

3

Department of Electrical and Electronic Engineering, Graduate School of Engineering,
Kobe University, 1-1 Rokkodaicho, Nada, Kobe 657-8501
The 35rd Computational Materials Designmaterials, atomic diffusion on solid surfaces, dry(CMD®) workshop (CMD35) has been held

reforming of methane, chemical reactions infrom September 2nd to September 6th and thesolid oxide fuel cell, and so on. The participants

36th CMD® workshop (CMD36) has been done

performed the calculations and examined thefrom February 17th to February 21st at Graduate

results.

SchoolofEngineeringScience,Osaka

In CMD36 two participants took theUniversity. In this workshop we have thesupercomputersupercomputer course to train up humansupercomputer of ISSP. They got a tutorial on

resources to advance researches by using system

RSPACE developed by T. Ono. After giving theB supercomputer of ISSP, the University ofintroductory lecture of large-scale computing byTokyo.

M. Geshi and describing the calculation method

courseandused

theIn CMD35 eight participants took theof electronic states and electron conduction

supercomputer course and got a tutorial on

property using RSPACE, exercises published inSTATE-Senri developed by Y. Morikawa. Afterthe manual were carried out by Y. Egami. Then,giving the introductory lecture of large-scale

electronic state calculations were carried out forcomputing by M. Geshi and explaining how toseveral molecules, and the electronic density

use the supercomputer of ISSP and explaining

distribution was visualized. Finally, the atomic

how to use STATE-Senri, calculation models on

structure optimization of the defects in GaAs

each research subject of the participants werebulks was carried out, and the calculation of

built and their calculations were carried out.

carrier scattering property by the defects wasSpecific subjects include oxidation of solid

investigated.

surfaces, chemical reactions on two-dimensional390