タンパク質の構造・ダイナミクス・機能への細胞内環境の影響
HPCIフォーラム （2021/3/9）

杉田有治
理化学研究所開拓研究本部
理化学研究所計算科学研究センター
理化学研究所生命機能科学研究センター

BDR

Molecular dynamics (MD) simulation• MD is useful to predict molecular motions at the atomic resolutions.
• In classical MD, newton‘s equation of motion is solved iteratively.

𝐹 = 𝑚𝑎• Time step △ t should be very short to reproduce fast vibrations
in molecules. (10-15 sec). Many iterations are required to detect
protein dynamics.
1𝐩! 𝑡 + ∆𝑡 ∆𝑡
2r 𝑡𝐫! 𝑡 + ∆𝑡 = 𝐫! 𝑡 +𝑚!
1
1𝐩! 𝑡 + ∆𝑡 = 𝐩! 𝑡 − ∆𝑡 + 𝐅! 𝑡 ∆𝑡122
𝐩 𝑡 − ∆𝑡2

𝑡𝑡 + ∆𝑡r 𝑡 + ∆𝑡1
𝐩 𝑡 + ∆𝑡2𝐅(𝑡)𝐅(𝑡 + ∆𝑡)

GENESIS for High-Performance MD
We have spent more than ten years for the development.
GENESIS Development
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021

The project
started at Wako
Kobe teams
startedVer. 1.0
J. Jung T. MoriVer. 1.2

Ver. 1.6

T. MoriVer. 1.1

Ver. 1.3

K. Yagi

Ver. 1.5
H. Oshima

J. Jung C. Kobayashi

This is free software under GPLv2 License.
https://www.r-ccs.riken.jp/labs/cbrt/

GENESIS has two key features:

Ver. 1.4

FiscalYear

Ver. 2.0beta
J. Jung

Leader: Y. Sugita
Main developers (ver. 1.5):
C. Kobayashi, Y. Matsunaga,
J. Jung, T. Mori,
H. Oshima, K. Yagi

1. High performance in large-scale biomolecular simulations.
2. Efficient conformational sampling based on various multi-copy methods.

New Methods developed in GENESISGENESIS is fast MD software for multiple computational platforms.
It enables MD simulation, enhanced sampling simulation, QM/MM calculation, coarse-grained
MD simulation, data-driven simulation, and so on.
• Optimization and parallelization of GENESIS–––––––––

Inverse Lookup Table: Jung et al. JCC 34, 2414-2420 (2013).
Midpoint Cell Method: Jung et al. JCC 35, 1064-1072 (2014).
Volumetric 3D FFT: Jung et al. CPC 200, 57-65 (2016).
GPU parallelization: Jung et al. JCTC 12, 4947-4958 (2016).
Multiple program/multiple data: Jung et al. JCC 38, 1410-1417 (2017).
Kinetic energy definition: Jung et al. JCP 148, 164109 (2018).
Optimal temperature: Jung et al. JCTC 15, 84-94 (2019).
KNL parallelization: Jung et al. JCC 40, 1919-1930 (2019).
Fugaku parallelization: Jung et al. JCC (2020) in press.

• Advanced simulation methods and models––––––––––––

Surface-tension REMD: Mori et al. JCTC 9, 5629-5640 (2013).
Reaction Path Method: Matsunaga et al. JPCLett 7, 1446-1451 (2016).
DoME model: Kobayashi et al. JPCB 119, 14584-14593 (2016).
RSE-MTD: Galvelis et al. JCC 36, 1446-1455 (2015).
gREST, gREST/REUS: Kamiya et al. JCP 149, 072304 (2018).
GaREUS: Oshima et al. JCTC 11, 3584-3595 (2019).
QM/MM vibration analysis: Yagi et al. JCTC 15, 1924-1938 (2019).
Cryo-EM flexible fitting: Mori et al. Structure 27, 161-174 (2019).
Dual-basin Go model: Shinobu et al. BPPB 16, 310-321 (2019).
Implicit Solvent and Membrane: Mori et al. JCTC 16, 711-724 (2020).
gREST+FEP: Oshima et al. JCIM (2020) in press.
CGMD: Tan et al. JCP 153, 044110 (2020).

GENESIS 2.0beta: Fugaku Codesign

J. Jung C. Kobayashi

Continuous effort on the acceleration of GENESIS on Fugaku
From K to Fugaku

FUGAKU Co-design
SystemCollaboration

Basic Arch.
DesignManufacturing

Application

11.3 peta-flops

Start-up
Investigation
Development

GENESIS (2014)

X 43488 peta-flops
GENESIS (2020)

x 125 (dt=2.5fs)
The acceleration was very important for flagship 2020 project.

Speed-up of GENESIS on Intel Xeon Gold 6148 (1 node, 40 cores)
120

Time (s)100Single precision/r-RESPA

80

Optimized kernel

60
40
200

Pre-release

GENESIS
1.0

GENESIS
1.1.0

GENESIS
1.2.1

GENESIS
2.0alpha

Benchmark calculation of ApoA1 (92, 224 atoms)

OthersLong-range NB
Short-range NB
Pairlist
Integration?

GENESIS2.0 or later

4ps MDHow to optimize GENESIS 2.0beta on Fugaku?
Algorithm1 (X)Algorithm2 (O)do i = 1, 16
do j = 1, 16
f(i,j)
end doend dodo ij = 1, 256
index i and j from ij
f(i,j)
end doend do1. In principle, there is no difference between Algorithm1 and Algorithm2 because of the same operation amount.
2. On Fugaku, Algorithm2 has better performance than Algorithm1.
3. The main reason is the operand waiting time for each do loop. On Fugaku, there are non-negligible waiting time before executingcalculations in the do loop.
4. To minimize the waiting time, the most inner do loop length should be long.
5. If the most inner do loop length is small, it would be better not to vectorize when compiling.
do i = 1, 3f(i)end doJ. Jung!$ocl nosimd
do i = 1, 3f(i)end dof(1:3)!$ocl nosimdf(1:3)
f(1), f(2), f(3)C. Kobayashi

Non-bonded interaction algorithm in GENESIS 2.0beta
do ijcel = 1, cell_pair
obtain icel and jcel
do i=1,natom(icel)
do k=1,neighbor(i,ijcel)
j = list(k,ijcel)
interaction end do
end doend dodo icel = 1, cell
do i=1,natom(icel)
do k=1,neighbor(i,icel)
list(k,i,icel)
interaction end do
end doend doApoA1 on one MPI processor

Do loop length is much longer in Algorithm 2 !

Two Kinetic energy evaluations in MD
Full-time step kinetic energy

Half-time step kinetic energy))11)! 𝑡 − ∆𝑡)! 𝑡 + ∆𝑡𝐩𝐩
122𝐾"#$% (𝑡) = (+22𝑚!
2𝑚!

((𝐩! (𝑡))𝐾%*$$ (𝑡) = (2𝑚!
!&'

𝑁+ 𝑘, 𝑇 = 2 𝐾%*$$ (𝑡) −!&'()∆𝑡6( 𝐩! (𝑡) 3 𝐫! (.) 𝑡+ 𝑂 ∆𝑡 0!&') (𝑁+ 𝑘, 𝑇 = 2 𝐾"#$% (𝑡) +∆𝑡( 𝐩! (𝑡) 3 𝐫! (.) 𝑡
12!&'

1. Full- and half-time step kinetic energies have the ∆𝑡 ) perturbation terms with opposite signs.
2. The perturbation terms are highly related to the high frequency vibrational motions.

+ 𝑂 ∆𝑡 0

Optimal Temperature and Pressure in MD
J. Jung

42𝑁! 𝑘" 𝑇 = 𝐾#$%& (𝑡) + 𝐾&'%% (𝑡) + 𝑂 ∆𝑡 (33
1. By combining full- and half-time step kinetic energies, we can obtain new temperature estimation which is accurate up to thethird power of ∆t.
2. The new temperature estimation does not require further computational cost.
3. With the new temperature estimation, we can enlarge the time step (∆t) up to 3.5 fs for short-range force and 7.0 fs forlong-range force using the r-RESPA integrator. (In conventional methods, ∆t is limited up to 2.0 - 2.5 fs)

Similarly, we can estimate pressure in a more accurate way:
1X 𝑃 𝑡 = 2𝐾&'%% 𝑡 + 𝑉𝑖𝑟(𝑡)3𝑉

1O 𝑃 𝑡 = 3𝑉 2𝐾#$%& 𝑡 + 𝑉𝑖𝑟(𝑡)J. Jung, Y. Sugita, J. Chem. Phys. 153, 234115 (2020).
J. Jung, C. Kobayashi, Y. Sugita, J. Chem. Theory. Comput. 15, 84-94 (2019).
J. Jung, C. Kobayashi, Y. Sugita, J. Chem. Phys. 148, 164109 (2018).

Strong Scaling of GENESIS 2.0beta on Fugaku
11.9 ns/day
4 times better performance than(more than twice better
GENESIS1.0 on K using 1/8 nodes
performance than NAMD)

Spike-protein on the surface ofSARS-CoV-2 using Fugaku
Fastest →x 9.04

x 6.82x 4.51x 2.96x 1.75

11.5億原子系を11.9ns/day
Jung, Kobayashi, Kasahara, Tan, Kuroda, Minami, Ishiduki, Nishiki, Inoue, Ishikawa, Feig, Sugita:
J. Comp. Chem. 42, 231-241 (2021).

Spike protein（70万原子以上）を60ns/day

Progress of biomolecular simulations
From gas-phase MD to MD in water, membrane, and cellular environments.

？
From 1977
In gasFrom 1990

From 2011 (K)From 2020 (Fugaku)

In water

In cell (short)

In cell (long)Macromolecular Crowding in a Cell

Inside of the cells is highly crowded.
Macromolecular Crowding Effects
An image of E. coli cell

(Illustrated by D. Goodsell)

Blue: available, Red: excluded (Minton, 2001)• The total concentration of proteins and RNA inside • Less free-space is available for protein inside of thecell of E. coli is in the range of 300-400 mg/ml.
cell.
• Macromolecular crowding greatly affects both
• The volume exclusion effect is considered to explain
kinetics and equilibria of biochemical reactions inmost of the macromolecular crowding effect.
living systems.

Reduced Native State Stability due toProtein−Protein Interactions
Effect of hydration in crowded protein systems

M. Feig(Michigan State University)

Reduced stability of villin

R. Harada(→Tsukuba U.)

NMR spectra of Villin at different concentrations
1mM villin (green),
32mM villin (blue),
32mM villin with protein G (red)Collaboration with NMR group (Dr. Kigawa (RIKEN))

Harada et al., J. Am. Chem. Soc. 135, 3696-3701 (2013).

Slow-Down in Diffusion Correlates with Transient Cluster
Formation
Rotational Correlation Functions of crowded villin solution

Snapshots of transient oligomers in MD (in 32mM (64 villins))

P. Wang(→Kotai Bio)

I. Yu(→Maebashi Inst.)

Nawrocki, Wang et al. J. Phys. Chem. B. 121, 11072-11084 (2017).

Modeling of the Cytoplasm of M. Genitalium
A schematic view of Mycoplasma Genitalium

M. Feig(Michigan State University)

We selected this cell, because of• its minimal size
• Extensive genetic characterization• Metabolic simplicity• Availability of proteomics and metabolomics data for M. pneumoniae
M. Feig et al. J. Mol. Graph. Modeling 58, 1-9 (2015).

Homology models of protein structures

All-atom MD Simulation of the cytoplasm in M. Genitalium
Yu, Mori, Harada, Ando, Jung, Sugita, Feig: eLife 5, e19274 (2016).

The atomic model of the cytoplasm in M. Genitalium

Performance of GENESIS on K
J. Jung

I. Yu(→Maebashi Inst.)
T. Mori

M. Feig(Michigan State University)

R. Harada(→Tsukuba U.)

T. Ando(→Tokyo U. Sci.)

Protein Diffusion and Stability
Yu, Mori, Harada, Ando, Jung, Sugita, Feig: eLife 5, e19274 (2016).

Diffusion of macromolecules in the cytoplasm

Slow-down ofdiffusion

Reduced stability of PDHA in Cytoplasm

Volume Exclusion EffectProtein-protein interaction

Reduced stability ofPDHA

Toward Cellular-Scale Drug Design

Adding competitive factors in cellular environments
Competition with other ligands or substrates as ATP

Multiple target proteins
Multiple compounds

PP1-Src Simulation:
300 ns / replica
43.2 μs in total (300 ns x 144 replicas)
13ns / day with 2304 nodes on K

Longerresidence-timefor efficacy

Competition with other proteins (off-targets)Prevent
off-target bindings• Kinetics-base design• New design principle• New drug targets

Summary• GENESIS is MD software suitable to large-scale biomolecular simulations
as well as in-silico drug discovery.
• Crowded cellular environments are now being simulated using all-atom
MD simulations.
• In crowded environments, ligand diffusion, distribution, and binding
mechanisms are different from those in dilute conditions.
• Including cellular crowding effects into drug design, we may be able toexamine the competitive factors for drugs in cells.

Acknowledgments
Collaborators

RIKEN:• Dr. Hiraku Oshima (RIKEN BDR)
• Dr. Suyong Re (→ NIBIOHN (医薬基盤研))• Dr. Kento Kasahara (→ Osaka Univ. )• Dr. Jaewoon Jung (RIKEN CPR/R-CCS)• Dr. Chigusa Kobayashi (RIKEN/R-CCS)• Dr. Cheng Tan (RIKEN/R-CCS)• Dr. Takaharu Mori (RIKEN CPR)
• Dr. Kiyoshi Yagi (RIKEN CPR)
• Dr. Ryuhei Harada (→ Osaka Univ. )• Dr. Isseki Yu (→ Maebashi Inst. Tech.)• Dr. Tadashi Ando (→ Tokyo U. Sci.)• Dr. Po-hung Wang (→ Kotai Bio)
Michigan State University:• Prof. Michael Feig
• Dr. Grzegorz Nawrocki

RIKEN Sugita lab (2019)
BDR @ Kobe

Research grants:• MEXT “Priority Issue 1 on post-K computer”• MEXT “Program for Promoting Researches on
the Supercomputer Fugaku”• MEXT Grant-in-Aid for Kiban (S) (19H05645)• RIKEN Pioneering Projects (Dynamic StructuralBiology and Glycolipidologue Initiative)• Research grants by RIKEN R-CCS and BDR

R-CCS @ Kobe

Computer resources:• RIKEN HOKUSAI GreatWave• RIKEN HOKUSAI BigWaterfall• HPCI supercomputer resources (hp170254,hp180201, hp180274, hp190097, hp190181,hp200129, hp200135)• Anton2 at Pittsburgh supercomputing center(PSCA18053P) (to MF)

CPR @ Wako