Molecular Simulationon the K Computer and towards Fugaku
The 2nd R-CCS International Symposium
14:45–15:00, 17 February 2020
Nichii Gakkan Kobe Port Island Center, Kobe

RIKEN Center for Computational Science, Kobe, Japan

Takahito Nakajima

Computational Molecular Science Research Team The main interest in our team is the development of theoretical andcomputational molecular science based on quantum chemistry. Our main
background is molecular orbital (MO) theory, for which the many-electron
Schrödinger equation is approximately solved by the 1-electron equation.
 In addition, the target is to develop interdisciplinary research by thecombination of quantum chemistry with other research fields such ascondensed matter physics, biological chemistry, computer science,
informatics, data science, ….

Theoretical & Computational Molecular Science With the emergence of peta- or exa-scale computing platforms, we areentering a new period of modeling. The molecular simulations can be carriedout for larger, more complicated, and more realistic systems than ever before.

Nanomaterial

Current & Next Generations of Science & Industry

Biomaterial

 To benefit from K/Fugaku exhaustively, we are forging novel molecularscience based on original theorizing and K/Fugaku-compatible software thatcan predict outcomes of molecular experiments.

Our Research Goals
To lead the way toward a new frontier of theoretical and computational
molecular science, the project involves the novel development of theory,algorithm, and software, which will be realized through collaborative use of K
and Fugaku across the fields of computational science and computer science.
1. Innovation of Theoretical and Computational Molecular Science Based on
Next-Generation Molecular Theory
to realize an improved and updated theoretical molecular science by developing ouroriginal theorizing to handle large and complicated molecules with high accuracy

2. Development of New Quantum Chemistry Software “NTChem”
to provide the users to a high-performance software package for molecular electronicstructure calculations for general purpose

3. Establishment of Materials Informatics on Massively Parallel Supercomputers
to develop efficient schemes for materials informatics with high-throughput
calculations on K/Fugaku

Our Research Goals
To lead the way toward a new frontier of theoretical and computational
molecular science, the project involves the novel development of theory,algorithm, and software, which will be realized through collaborative use of Kand Fugaku across the fields of computational science and computer science.
1. Innovation of Theoretical and Computational Molecular Science Based on
Next-Generation Molecular Theory
to realize an improved and updated theoretical molecular science by developing ouroriginal theorizing to handle large and complicated molecules with high accuracy

2. Development of New Quantum Chemistry Software “NTChem”
to provide the users to a high-performance software package for molecular electronicstructure calculations for general purpose

3. Establishment of Materials Informatics on Massively Parallel Supercomputers
to develop efficient schemes for materials informatics with high-throughput
calculations on K/Fugaku

NTChem QC software comprises immensely useful tools in material and biologicalscience researches.
 The development of QC software in Japan falls far short of that in Westerncommunities.
World-Wide QC Programs-- Gaussian (Gaussian, Inc.)-- GAMESS (Iowa State Univ.)-- Molpro (Stuttgart, Cardiff)-- Molcas (Lund Univ.)-- NWChem (PNNL)-- Q-Chem (Q-Chem, Inc.)-- TURBOMOLE (GmbH) In the K project, we decided to develop a comprehensive new program suiteof ab initio QC made in Japan from scratch.

Features of NTChem
NTChem implements not only standard QC approaches, but also our-own original andimproved QC methods that we have developed in our research works.
 Electronic structure calc. of the ground state of molecules based on DFT&HF.
 Linear- or low-scaling DFT: RI-DFT, PS-DFT, GAPW, GFC, dual-level DFT.
 Low-scaling SCF calculation using diagonalization-free approaches.
 Excited-state DFT calculation: TDDFT, DFT-TP.
 Accurate correlation methods for ground and excited states: MP2, CC, QMC.
 Relativistic electronic structure calc. with SO interactions: DK, RA, RESC.

 Model calculations for large molecules: QM/MM, ONIOM, ADMA, MTA.
 Calculation of solvent effects: COSMO, ASEP/MD, QM/MM-MD.
 Efficient calculation for chemical reaction pathway: NEB, String.
 Ab initio molecular dynamics calculation.
 Calculation of electric and magnetic properties: Polarizability, NMR, EPR.
 Population analysis: Mulliken, Löwdin, NBO.
 Orbital interaction analysis: MIO, PIO.
 Massively parallel computing on K and PC architectures: HF, DFT, MP2, QMC.

A Wide Variety of Applications with NTChem

8

A Wide Variety of Applications with NTChem

9Performance of Parallel Computing with NTChem NTChem is designed for high-performance calculations on the computerswith several tens of thousands of nodes for DFT, TDDFT, MP2, and QMC.
36864

36864

Nanographene (C24576

Speedups

Speedups

Ideal
24576

12288

12288

000

12288

24576
Nodes

H )

150 30 2
Speedups
B3LYP/cc-pVTZ (9,840 AOs)
NodeK computer MPI/OpenMPSpeedups

36864

Total SCF (Fock matrix calc. + diagonalization)

012288

24576
Nodes

36864

Fock matrix calculation

Parallel scalability is kept up to 36,864 nodes of K: Elapsed time 95 min using 36,864 nodesHybrid Parallel MP2 Algorithm in NTChem We have proposed an efficient MPI/OpenMP hybrid parallel algorithm of theMP2 electron-correlation calculation.
Speedups

71288

Ideal
53466
35644
17822

3.5 PFLOPS(34 %efficiency)

00

17822 35644 53466 71288

Calculated heats of formation per carbon (kJ mol–1)

 The good parallel scaling is achieved with our new algorithm on “K”. Elapsed time is
11 mins using 80,199 nodes of “K” for nanographene (C150H30)2. The world’s largest
wavefunction-based electron-correlation calculation (Left Figure) By using the same MP2 algorithm on “K”, we can predict, with high accuracy, theproperty known as the heat of formation of large fullerenes. Our finding suggests thatnew interesting properties that show the character of both fullerenes and graphenes
appear if much larger fullerenes are synthesized. (Right Figure)Press release: http://www.riken.jp/pr/press/2016/20160316_1/

Development of NTChem We are currently developing a new version of NTChem. It adopts a memorydistributed sparse-matrix algorithm based on NTPoly to overcome available memory
sizes per a computer node. It also adopts an algorithm with regular communication
patterns that map to high dimensional torus network architectures to manage
significant irregular data movement.

 We hope that ab initio calculations for large molecules including 1M atoms will be
realized on Fugaku because the new version is capable of calculating molecules with
10K atoms on “K” in the current stage.
 In future, we will perform novel computational applications for realistic molecularsystems such as whole photosystem II (1.2M atoms) on Fugaku.

NewOldOur Research Goals
To lead the way toward a new frontier of theoretical and computational
molecular science, the project involves the novel development of theory,algorithm, and software, which will be realized through collaborative use of Kand Fugaku across the fields of computational science and computer science.
1. Innovation of Theoretical and Computational Molecular Science Based on
Next-Generation Molecular Theoryto realize an improved and updated theoretical molecular science by developing ouroriginal theorizing to handle large and complicated molecules with high accuracy

2. Development of New Quantum Chemistry Software “NTChem”
to provide the users to a high-performance software package for molecular electronicstructure calculations for general purpose

3. Establishment of Materials Informatics on Massively Parallel Supercomputers
to develop efficient schemes for materials informatics with high-throughput
calculations on K/Fugaku

Materials Design for Solar Cells
 To search for novel Pb-free perovskite solar cells containing non-toxic andwidely available metals, we performed a systematic high-throughput firstprinciples simulation on the K computer for 11,025 compositions ofperovskites in ABX3 and A2BB’X6 forms, where A is an organic or inorganic
component, B/B’ is a metal atom, and X is a halogen atom.
 By applying the screening procedure to all of the computed compounds, we
discovered novel candidates for environmentally friendly Pb-free perovskite
solar cells and proposed 51 low-toxic halide perovskites, most of which areproposed newly in our work.
 The potential candidates are
categorized as only 6 types basedon the combination of groups towhich two metal elements (at Band Bʹ-sites) belong in the periodic
table; group 14–14, group 13–15,
group 11–11, group 9–13, group
11–13, group 11–15.
Press release: http://www.riken.jp/pr/press/2017/20171005_1/

Materials Design for Photocatalysts
 We performed a large-scale computational screening for 29,160 perovskite
oxynitrides and oxides to explore novel visible-light water-splitting
photocatalysts. We can propose 42 potential candidates, most of which are
newly discovered in our study.
 We can suggest that the perovskite oxynitrides with d0/d10 electronic
configuration and the perovskite oxides with partially occupied d electrons
are desirable for the visible-light driven water-splitting photocatalysts withadequate band gaps and band edge positions.
 By selecting the perovskites containing
cheaper elements than vanadium fromthe 42 potential candidates, we canpropose novel 6 compounds, NaWO2N,
KWO2N,
MgWON2,CaVO2N,
CaAl1/3W2/3O2N, and CaV2/3Fe1/3O3, as
rare-metal-free photocatalysts.

Materials-Informatics Driven Design of HTMs
 We proposed new hole-transport materials (HTMs) by using the materials informatics
and the simulations with NTChem on “K”. We firstly created a machine-learningmodel to predict the power conversion efficiency of perovskite solar cells (PSCs) forthe molecules generated from structural combinations of the fragments of known
HTMs. The quantum-chemical electronic descriptors were evaluated by NTChem on“K”. Then, to select HTM with the highest PCE, we adopted the particle swarm
optimization (PSO) with the GPR model. In details, the optimal set of fragments for
HTM with the highest PCE was selected by PSO so as to optimize the acquisition
function obtained from the GPR model. We achieved the efficient search for novel
candidates for HTMs of PSCs from the vast chemical space (32,294,400 molecules).

25.7%

24.7%

Candidate HTMs predicted by materials informatics

Summary & Future Plans
 To lead the way toward a new frontier of theoretical and computational
molecular science, we are developing novel theory, algorithm, and software,
which will be realized through collaborative use of K and Fugaku across thefields of computational science and computer science.
 NTChem is a high-performance software package for molecular electronic
structure calculation for general purpose, and is a useful tool in various
computational studies for large and complicated molecular systems.
 NTChem is a community-based software package. We intend to continue
adopting users’ requests with the aim of making the program moreconvenient and user-friendly for researchers in various research fields.
User cases
・Toyobo ・Mitsubishi Chem. ・RIKEN ・Hokkaido U. ・U. Tokyo ・Kyoto U. ・Kobe U. ・YokohamaCity U. ・Gifu U. ・JAEA ・Open to the public on K and SC centers (IMS, FOCUS, Nagoya U., Kyusyu
U., Tokyo Tech.)

http://www.r-ccs.riken.jp/software_center/software/ntchem/Summary & Future Plans
 We plan to realize accelerating the discovery of novel energy materials suchas photovoltaics, photocatalysts, and polymer batteries by establishing newmaterials informatics techniques that assimilate large databases andmathematics in materials science with high-throughput calculations onFugaku. We currently have several plans to collaborate with experimental
researchers and industries in order to discover new materials that are highly
efficient, low cost, environmentally clean, and sustainable. We would like torealize the novel material designs by the co-design with experiments,
theorizing, simulation, and informatics.