Deep Generative Model for
De Novo Drug Design

2019/11/14
M2 Koki Sasamoto
1Drug discovery
Flow chart of drug discovery process

・ Compound library is used to screen hit compound.
・ Good hit compound reduces time and money.
・ Diverse and high-quality compound library is needed.
2

Chemical space

VS・ Chemical space is vast, and only a tiny fraction was collected
as compound libraries.

J. Comput.- Aided Mol. Des. 2013, 27, 675−679. 3

Construction of virtual library
Building blockcombination
J. Phys. Chem. Lett., 2011, 2, 2241-2251.

Genetic algorithm

J. Am. Chem. Soc., 2013, 135, 7296-7303.

Reaction-based rule

J. Chem. Inf. Model, 2011, 51, 3093-3098.
4

Deep generative model
GenerativeModel

Big Data

Learn

New Data

Generate

・ Generative model generates realistic data from feature of data.

→ Drug-like molecules can be generated by generative model
learning features of biologically-active compounds.
5

Contents
1. Deep learning methods in drug design- RNN- RNN with RL (ReLeaSE)- VAE- Graph / GAN (MolGAN)
2. Application in drug discovery (GENTRL)
3. Summary6

SMILES
Examples of SMILES representation

M. H. S. Segler et al., ACS Cent. Sci., 2018, 4, 120-131. 7

De novo drug design by RNN
Generation of sentence

learn English grammar

Generation of chemical structure

M. H. S. Segler et al., ACS Cent. Sci., 2018, 4, 120-131. 8

De novo drug design by RNN
Examples of generated novel molecules

・ 976327 molecules were generated.
・ 847955 molecules were novel.
・ 75% of new molecules were highly scored (“core” or “backup”)
by AstraZeneca filter.
M. H. S. Segler et al., ACS Cent. Sci., 2018, 4, 120-131. 9

De novo design cycle
Scheme“Synthesis” … molecule generation“Virtual Assay” … best molecule selection by machine learning“Design” … retraining RNN model by best molecules

6% of known active molecules were re-generated.
M. H. S. Segler et al., ACS Cent. Sci., 2018, 4, 120-131.10

De novo drug design by RNN

Data

541555 bioactive molecules

Fine-tune

25 molecules with known agonistic activity
on RXR (retinoid X receptor) and/or
PPAR (peroxisome proliferator-activated receptor)

Result1000 molecules (90% were valid and novel)
5 molecules were synthesized and tested in vitro.
G. Schneider et al., Mol. Inf., 2018, 37, 1700153. 11

De novo drug design by RNN
Synthesized novel molecules and these bioactivity

Bioactivity (EC50 / uM)

G. Schneider et al., Mol. Inf., 2018, 37, 1700153. 12

De novo drug design by RNN
Pros
・ Diverse set of molecules could be generated.
・ Generated molecules had drug-like properties.

Cons・ Chemical space was restricted by training set.
・ Properties of generated molecules couldn’t be controlled.

Reinforcement Leaning (RL)

13

Reinforcement Learning (RL)
Scheme of Reinforcement Learning
RewardAgentAction

EnvironmentStateApplication

14

De novo drug design by RL
Scheme of “ReLeaSE”
Generate
new molecules

Predictproperties

Popova et al., Sci. Adv., 2018, 4, No. eaap7885. 15

De novo drug design by RL
Target Properties
・ Tm (Melting point)

・ logP (n-octanol / water partition coefficient)・ pIC50 for JAK2 (janus protein kinase 2)

Distribution of predicted properties

Popova et al., Sci. Adv., 2018, 4, No. eaap7885. 16De novo drug design by RL
Analysis of generated molecules(SAS = synthetic accessibility score)

With reinforcement learning, the proportion of valid molecules was lowered.
Popova et al., Sci. Adv., 2018, 4, No. eaap7885. 17

De novo drug design by VAE

R. Gómez-Bombarelli et al., ACS Cent. Sci, 2018, 4, 268-276.

18

Encoder and Decoderinterpolation between twomolecules in latent space

・ VAE learns about characteristic feature of a training set.
・ Similar molecules were mapped close together in latent space.
R. Gómez-Bombarelli et al., ACS Cent. Sci, 2018, 4, 268-276.

19

Predictor

・ VAE was jointly trained with Predictor.
・ 7,500,000 molecules were generated from 250,000 samples.
R. Gómez-Bombarelli et al., ACS Cent. Sci, 2018, 4, 268-276.

20

De novo drug design by VAE

・ VAE optimized (5 * QED – * SAS).
(QED = Qualitative Estimate of Drug-likeness,
SAS = Synthetic Accessibility Score)・ Molecular optimization was achieved efficiently
by gradient-based search.
R. Gómez-Bombarelli et al., ACS Cent. Sci, 2018, 4, 268-276.

21

Problems of SMILES representation
SMILES is not designed to capture molecular similarity.

SMILES is not robust to small molecules.
invalid
representation

22

MolGAN
Scheme of MolGAN

Generate
new molecules

Judgevalid or invalid

Predict
properties
arXiv:1805.11973

23

Graph representation
Chemical Structure

Molecular Graphedge(bond)node(atom)

・ Graph … collection of nodes and edges
・ Machine learning model don’t have to learn rules of molecular
representations.
24Graph convolution

・ New vector = self +adjacent vector→ New vector includes the information of the surrounding environment.

H. Altae-Tran, et al., ACS Cent. Sci. 2017, 3, 283-293.

25

GAN
Scheme of Generative Adversarial Network (GAN)

Manufacture of counterfeit money vs Police
26

GAN

https://arxiv.org/abs/1809.11096

https://arxiv.org/abs/1611.07004
27

GAN
Scheme of Generative Adversarial Network (GAN)

Generator : generate molecules similar to training set
Discriminator : discriminate generated molecules fromtraining set
arXiv:1805.11973

28

Reward network
Scheme of Reinforcement Learning (RL)

feedback

Reward : Valid, Drug-likeness, Synthesizability, Solubility+
Generated molecules or Training set ?
arXiv:1805.11973

29

Performance of MolGAN
Results
Objective
Druglikeness

Synthesizability

Solubility

All

Algorithm
ORGAN
Naive RL
MolGAN
ORGAN
Naive RL
MolGAN
ORGAN
Naive RL
MolGANORGAN
MolGAN

Valid (%) Unique (%) Druglikeness
88.2
69.4
0.52
97.1
97.1
0.57
99.9
2.0
0.61
96.5
45.9
97.7
13.6
99.4
2.1
94.7
54.3
92.7
100.0
99.8
2.3
96.1
97.2
0.52
97.4
2.4
0.47

Synthesizability

Solubility

0.83
0.83
0.95

0.71
0.84

0.55
0.78
0.890.530.65

ORGAN (SMILES instead of graph), Naïve RL (without GAN)

・ MolGAN beats other models in terms of optimizing property.

・ Unique score of generated molecules was very low.
arXiv:1805.11973

30

GENTRL
Identification of DDR1 kinase inhibitor by GENTRL・ Day 19 ... 30000 molecules were generated by GENTRL.
・ Day 23 ... 6 molecules were selected by prioritization.

・ Day 35 … Synthesis was completed.
・ Day 46 … Activities of synthesized molecules were confirmedin cell-based assay.
A. Zhavoronkov et al., Nat. Biotechnol., 2019, 37, 1038-1040.

31

GENTRL
Creation of chemical space

regenerated
regenerated

reduction of information

Feature vector

32
A. Zhavoronkov et al., Nat. Biotechnol., 2019, 37, 1038-1040.

GENTRL
Molecular generation by Reinforcement Leaning

Reinforcement Leaning

SOM(Self Organizing Map)

Agent : generator
State : generated molecules
Reward : novelty, kinase inhibition activity, DDR1 inhibition activity
SOM : predict properties of molecules
A. Zhavoronkov et al., Nat. Biotechnol., 2019, 37, 1038-1040. 33

GENTRL
Selected molecules and inhibitory activity in vitro and vivo

high activity

A. Zhavoronkov et al., Nat. Biotechnol., 2019, 37, 1038-1040. 34

Summary
Chemical Space・ Chemical space is vast (~1060) compared to compound library size (~106, 108).
・ Generative model can generate 103 ~ 105 drug-like compounds.
・ Generative model can control properties of generated molecules by RL.
・ The role of generative model is to capture the underlying rules of a data
distribution.
・ Generative model only reconstruct the training data set.

Molecular representation
・ SMILES is not robust to small changes or mistakes.
・ By using graph representations , generative model don’t need to learncomplex syntax, but this method is not perfect.
・ There is still a need for research on the optimal molecular representation.
- Junction Tree (arXiv:1802.04364)- 3D (arXiv:1810.11347)

35

Summary
Evaluation of Model
・ The performance of each generative model is evaluated by different methods.
- Number of generated molecules- Distribution on 2D map.
- Properties of generated molecules.
- Experimental activity.
・ Evaluation method of model is needed.
・ Several benchmarks are being developed. (J. Chem. Inf. Model, 2019, 59, 1096)

Application in drug discovery
・ The generated molecules must be reduced to the number that can besynthesized.
・ The generated molecules are necessarily synthesizable.
・ SAS (synthetic accessibility score) may prevent generation of molecular diversity.
・ Generative model may prove valuable in combination with retrosynthesis AI orvirtual screening AI.

36

37

Feature space

38

NN

39

RNN

M. H. S. Segler et al., ACS Cent. Sci., 2018, 4, 120-131.

40

AZ filter
Class 1 : bland structures- Fewer than 4 carbon atoms etc.
Class 2 : reactive structures- Anhydride etc.
Class 3 : frequent hitters- Nitrophenols etc.
Class 4 : dye-like structures
Class 5 : unlike drug candidates or unsuitable fragments
Class 6 : difficult series or natural compounds
Class 7 : general ugly halogenated structures
Class 8 : general ugly oxygen
Class 9 : general ugly nitrogen
Class 10 : general ugly sulphur

Nat. Rev. Drug Discovery 2013, 12, 948−962

41

SOM

https://qiita.com/tohru-iwasaki/items/e51864269767ccc07254

42