Molecule Generation For Target Protein Binding with Structural Motifs
Authors: ZAIXI ZHANG
Yaosen Min
Shuxin Zheng
Qi Liu
Venue: ICLR 2023 poster (2023)
Keywords: Structure-based Drug Design

Abstract: Designing ligand molecules that bind to specific protein binding sites is a fundamental problem in structure-based drug design. Although deep generative models and geometric deep learning have made great progress in drug design, existing works either sample in the 2D graph space or fail to generate valid molecules with realistic substructures. To tackle these problems, we propose a Fragment-based LigAnd Generation framework (FLAG), to generate 3D molecules with valid and realistic substructures fragment-by-fragment. In FLAG, a motif vocabulary is constructed by extracting common molecular fragments (i.e., motif) in the dataset. At each generation step, a 3D graph neural network is first employed to encode the intermediate context information. Then, our model selects the focal motif, predicts the next motif type, and attaches the new motif. The bond lengths/angles can be quickly and accurately determined by cheminformatics tools. Finally, the molecular geometry is further adjusted according to the predicted rotation angle and the structure refinement. Our model not only achieves competitive performances on conventional metrics such as binding affinity, QED, and SA, but also outperforms baselines by a large margin in generating molecules with realistic substructures.

Published as a conference paper at ICLR 2023
MOLECULE GENERATION FOR TARGET PROTEIN BINDING WITH STRUCTURAL MOTIFS
Zaixi Zhang!? Yaosen Min Shuxin Zheng", Qi Liu
1: Anhui Province Key Lab of Big Data Analysis and Application,
School of Computer Science and Technology, University of Science and Technology of China 2:State Key Laboratory of Cognitive Intelligence, Hefei, Anhui, China 3:Institute of Interdisciplinary Information Sciences, Tsinghua University, 4: Microsoft Research caixi@mail.usie mail.ustt.ed. edw..edddccc minys18 18 @ wwiigs.tiinguua.co shuz@microsee muuz@mccrosoft.com qiliuql ustc.edu.cn
zaixi @
ABSTRACT
Designing ligand molecules that bind to specific protein binding sites is funda- mental problem in structure-based drug design. Although deep generative models and geometric deep learning have made great progress in drug design, existing works either sample in the 2D graph space or fail to generate valid molecules with realistic substructures. To tackle these problems, propose Fragment- based LigAnd Generation framework (FLAG), to generate 3D molecules with valid and realistic substructures Tragment-by-fragment In FLAG, motif vo- cabulary is constructed by extracting common molecular fragments (i.e., motif) in the dataset. At each generation step, 3D graph neural network first em- ployed to encode the intermediate context information. Then, our model se- lects the focal motif, predicts the next motif type, and attaches the new motif. The bond lengths/angles can be quickly and accurately determined by chemin- formatics tools. Finally, the molecular geometry is further adjusted according to the predicted rotation angle and the structure refinement. Our model not only achieves competitive performances on conventional metrics such as binding affin- ity, QED, and SA, but also outperforms baselines by a large margin in gener- ating molecules with realistic substructures. Our code publicly available at https://qithub: com ttt//aaiiiiiii http:/aaxxizaaan/AAA
INTRODUCTION
Recent years have witnessed the great success deep learning in drug design. Among the progress, deep generative models that aim to generate molecules with desirable physicochemical and pharma- cological properties are of particular importance. These models range from string-basse based (Gómez-)
Bombarelli et al. 2018) and graph-based methods (Jin et al. 2018 Xie et al. 2021 recent 3D geametry-based methods (Gebauer et al. 2019: Luo & Ji 2021).
Molecule drugs can only affect certain biological functions and pathways by binding to the target proteins. However, the complexity of the context information, geometric constraints, and molecule- protein interactions bring great challenges. Therefore, few deep learning models have been devel- oped to generate molecules that bind to specific protein binding sites (a.k.a. structure-based drug design). Early attempts modify the pocket-free models by incorporating scoring functions like dock- ing scores between generated molecules and pockets to guide the ligand generation (Li et al.) (202]). Another line of works convert the 3D pocket structures to molecular string graph representations for conditional generation (Skalic et al. (2019; Xu et al. (021a). They fail to model how molecules interact with their target proteins explicitly in 3D space. Recently, a series of 3D generative models proposed to generate 3D molecules that bind to given protein pockets (Luo & Ji 202] Liu al.) 2022 Peng et al., 2022). They use 3D graph neural networks for context encoding and achieve equivariance. However, most of these works do not consider chemical priors and may generate
"Qi Liu is the corresponding author.

---

Published as a conference paper at ICLR 2023
invalid molecules with unrealistic substructures, Their stom-wise generation scheme also leads to inefficient molecule sampling.
In this work, propose novel Fragment-based LigAnd Generation framework (FLAG) for structure-basee drug design, where the molecules are generated beemeen-by-frammeeee To generate 3D molecules, we first preprocess the dataset and extract molecular fragments with high occurrence frequencies (i.e., motif) as "building blocks" for new molecules. At each generation step, a 3D graph neural network is first employed to encode the intermediate context information including the protein pocket and the intermediate molecular graph. Secondly, our model selects the focal motif, predicts the next motif type, and attaches the new motif to the generated molecule. Attaching motifs in 3D space is a great challenge. Inspired by the fact that the flexibility of molecular geometries lies largely in the degree of rotatable bond (A bond creates two connected components of the molecule, each of which has at least two atoms) (Axelrod Gomez-Bombarelli, 2022), we employ cheminformatics tools (Bento et al. 2020) to efficiently determine the bond lengths/angles and trains neural networks to predict torsion angles. Leveraging this insight can significantly reduce the searching space of atom/motif coordinates. For example, the CrossDocked dataset (Francoeur al., 2020) has, average, m = 24 heavy atoms, corre- sponding to a dinddimnsiioal euclidean space, but only around 5 torsion angles of rotatable bonds. Furthermore, the rotation angle is predicted to further adjust the geometries of generated molecules. Inspired by force fields in physics (Rappé et al.) 1992), a novel structure refinement is finally applied to optimize the molecule structures.
molecule is rotatable if cutting this bond
We conduct extensive evaluations to evaluate our approach. Experimental results show that: (1) our method is able generate diverse drug-like molecules with high binding affinity to target proteins; (2) FLAG is much faster than most of the baseline methods at sampling new molecules; (3) thanks to the design of fragment- based generation, our method outperforms baselines by a large margin on generating valid molecules with realistic substructures.
RELATED WORK
Motif- Molecule Generation. To generate more valid molecules with realistic substructures,
many models adopt prior knowledge of chemical motifs, also known fragments rationales.
as building blocks to generate or optimize molecules (Jin et al.] 2018 2020a Podda et al. 2020
Jin al.] 2020b] Chen et al., 2021a] Seo al.) [021] Xie et al. 2021| Chen al.) 2021b] Guo
et al. 2022, Flam-Shepherd et al. (022).. For example, JT-VAE (Jin et al. 2018) first decomposes
the molecular graphs into junction trees, where each node in the tree represents a substructure of
the molecule. Then VA- adopts the variational autoencoder as the framework and learns to
reconstruct the molecular graph that lead to different properties of molecules by MCTS. Then it is trained to expand rationales to complete molecular graphs with reinforcement learning. However, the aforementioned methods cannot generate 3D molecules directly and consider the complicated context information of binding sites.
saynneettb--rragment Similarly, RationaleRL extracts rationales
3D Molecule Generation. With the development of geometric deep learning, many recent works explore 3D molecular geometry generations with given 2D molecular graphs (Mansimov et al.) 2019)
2021b Shi al. |021 Ganea al. [02 Xu al. Simm & Hernandez-Lobato 2020; Luo al.,
2021bb 2022). or from scratch (Gebauer et al., 2019: Hoogeboom et al. 2022 Nesterov et al. 2020
Gebauer al. 2022 Luo & Ji 2021] Satorras al. 2021). Comparatively, the task of structure-
based drug design is more challenging. Firstly, the 2D molecular graph is unknown. Secondly, the generated molecules should fit well with the binding pockets with high binding affinity. Finally, the aforementioned works usually deal with small organic molecules and may be insufficient to generate 3D drug-like molecules with larger molecule weights. For more detailed discussions on molecule generation, recommend readers refer to the comprehensive survey (Du et al. 2022).
Structure-based Drug Design. Structure-baset drug design aims to generate 3D molecules that bind to specific binding sites. LiGAN (Ragoza et al. 2022) first approaches this problem using conditional variational autoencoder trained on atomic density grid representations of pooein-ligaad structures. Then the molecular structures of ligands are constructed by atom fitting and bond infer- ence from the generated atom densities. As a preliminary work, LiGAN employs 3D CNN as the encoder, which does not satisfy the desirable equivariance property. The follow-up works achieve
2

---

Published as a conference paper at ICLR 2023
Tocal moll
Reterion seglle Ao
0011
Contest encoding focal moli selection
(b) Next
(0) attachments enumeration and prediction
(d) Rotation angle prediction structure refinement
Figure 1: An illustration of one generation step of FLAG. There mainly four parts whose details are shown Sec
equivariance by leveraging graph neural networks to encode the context information (Luo & Ji Liu et al. 2022 Peng et al. 2022). For example, (Luo & Ji] 2021) uses SchNet (Schütt et al., 2017 to encode the 3D context of binding sites and estimate the probability density of atom ss occur-
2021
rences in 3D space. The atoms are sampled auto-regressively until there is no room for new atoms.
GraphBP (Liu al. 2022) adopts the framework of normalizing flow (Rezende & Mohamedd 2015) and constructs local coordinate systems to predict atom types and relative positions. Pocket2Mol (Peng et al. 2022) adopts the geometrie vector perceptrons (Jing al., (202T) and the rector-based neural network (Deng et al. (021)) the context encoder. It also explicitly considers the influence of chemical bonds. However, most of the above methods do not consider the prior knowledge of chemical motifs and may generate molecules with unrealistic or distorted substructures.
We note that some recent works (Green et al. 2021 Powers et al., 2022) are similar to our model that adopts fragment-basee methods for ligand generation/optimizztioo. (Green et al.) [021] opti- mizes the binding affinity by predicting fragments to add. (Powers et al. 2022) expands a small molecule fragment into larger drug-like molecule binding to given protein pocket. Compared with these methods, FLAG does not require the information of starting fragments and can auto- matically construct the motif vocabulary. Moreover, the novel structure refinement module enables FLAG to adjust the generated molecules flexibly.
3
METHOD
3.1
OVERVIEW
Our goal to generate valid 3D molecular structures that can fit and bind to specific protein binding site. The 3D geometry of molecule (i.e., ligand) can be represented as g = {a,, rz)) Sim- ilarly, binding site of protein can be defined as a set of atoms P = {bb, {bb,,,)))T sز)] Here, we use and m to represent the numbers of atoms in the molecule and in the binding site, respectively. In 9 and P. a, and by are one-hot vectors denoting the atom types and r; 8j E I indicate the 3D cartesian coordinates. Formally, our objective is to learn conditional generative model p(G(P) to capture the conditional distribution of protein-ligand pairs.
In work, formulate the generation of molecules in the given binding pocket sequential generation process. Let our generation model be ø and the intermediate generated molecule at the t-th step be (()), the generation process can be summarized as below:
((t) = (t P), 1>1
(1)
g()) = OP)) t=
Different from previous works, we generate molecules motif-by-motif, i.e., set of atoms from the new motif are included into g(t) instead of single atom. Specifically, there are mainly four parts in one generation step, including (a) context encoding and focal motif selection, (b) next motif" prediction, (c) motif attachments enumeration and selection, and (d) rotation angle prediction and structure refinement (Figure [.. The details are demonstrated in the rest part of this section.
In this section, first introduce the motif extraction procedure in Sec. 3.2 In Sec. 3.3 and Sec. we will describe the architecture of the encoder and the 3D molecule generation process respectively. In Sec. 3.5 we derive the training objective introduces the training scheme in detail.
3

---

Published as a conference paper at ICLR 2023
Rotatable bonds
Dotach
Break into teo rings (motts)
Select motif
(a) Find all the rotatable bonds
(b) Detach and extract all the selected bonds
(c) Further Decomposttion
Motif Vocabulary
Figure 2: Illustration of Motif Extraction. More details shown in Sec 3.2
3.2 MOTIF EXTRACTION
To decompose molecules and extract motifs, a molecule can also be represented as three- dimensional graph
(V, R, E) with V atoms set, R atom coordinates & as covalent bonds set. A motif M, = (V;. R₁ E,) is defined as a subgraph of molecule G. Given a molecule, we extract its motifs Mi,... .M, such that their union covers the entire molecular graph: V = V.V. R U, R2, and = U; J,&&C E, The motif extraction mainly contains the following steps:
Firstly, extract and detach all the rotatable bonds that will not violate the chemical validity.
Then molecule G is then broken into disconnected fragments GGN.
We select G. as motif its occurrence in the whole training set is more than т. If G is not selected motif, we further decompose it into finer rings and bonds and select them as motifs in G.
After preprocessing the whole training dataset, we obtain a vocabulary of motifs VM- The motif ex- traction and vocabulary construction procedure are illustrated in Figure In the generation process, the geometry of motifs (i.e., bond lengths and angles) can be determined by cheminformaties tools such RDkit (Bento et al. 2020
3.3 ENCODER
To generate molecules conditioned the binding pocket, it is important capture the context information with the context encoder. In our work, context 3D graph ((t-1) ((t-1) UP is firstly constructed by connecting atoms within certain cutoff distances. A 3D graph neural network employed to encode ((t-1), The first layer of the encoder attributes to initial embeddings h Then, we have L feature aggregation layers in our 3DGNN. The aggregation for each atom the I-th layer (1 < < L) can be formulated as:
linear layer which maps atomic
(t.)
h 11
Σ (EM(K)
h (t,-1) O (LLE (CRBe(dat)) (duk)),
MLP'(.) 1(t-1), and @ denotes the element-wise multiplication. The embedding of pairwise distance d.. obtained with radial basis functions (CRBF(1 (),, such as Gaussian Functions (Schlichtkrull et al. 2018) and spherical Bessel functions (Gasteiger et al. 2019). Since our encoder is based on the atom attributes and pairwise distances, it is rotationally and translationally invariant. Note that more advanced 3DGNNs such as DimeNet (Gasteiger et al. 2019) and SphereNet (Liu et al. 202T can be employed as encoders in future works. We do not use them in the current version of FLAG, considering the computational efficiency and GPU memory budget.
where ((k) denotes the neighbors of the k-th atom in C(t-1),
multi-layer perceptron,
3.4 3D MOLECULE GENERATION
The following procedures applied to generate the new motif in each generation step.
Focal Motif Selection: To predict the next motif, we need to first select focal motif to which the next motif attaches with. We employ auxiliary atom-wise classifiers: protein atom classifier (for

---

Published as a conference paper at ICLR 2023
t == 1) and molecule atom classifier (for t 2 2) for the selection. (1) At the first step (t = 1), all the known context information is the binding pocket. The protein atom classifier takes the hidden representations of protein atoms imput, and predicts whether new ligand atoms can be generated within A. (2) For 1. 2 2, the molecule atom classifier selects a focal atom from the ligand atoms generated in the previous f. 1 steps. The motif that the focal atom belongs to is chosen as the focal motif. If no atom/motif is selected as focal, the molecule generation is completed.
Overall, the classifiers take the representations of atoms as input and utilize two MLPs to predict the selection probabilities. We describe how to train these two auxiliary classifiers in 3.5
Next Motif Prediction: Given the focal motif My. the label of the motif is predicted as:
Σ h))) iEM,
=
where q is the distribution over the motif vocabulary VM, Emb(My) denotes the embedding of the focal motif, and Lies, h; is the sum of the atom embeddings in the focal motif. Since there is no focal motif at the first step (t 1), we regard no motif as special motif type and also learn its embedding in training.
Motif Attachments Enumeration and Prediction: With the predicted motif, the next step is to attach the new motif to the generated molecule. Note that this step is not deterministic since there are potentially many attachment configurations (See Fi. Our goal here to select the most appropriate attachment. Specifically, we enumerate different valid attachments and form candidate set C. We employ GIN (Xu et al., 2019) as the scoring function fa over the candidate molecular graphs and train it to select the most appropriate molecule attachment:
(()) arg max fg()".
(5)
g'EC
Note that by our design, any two motifs share at most two atoms, we only need to merge at most two atoms or one bond in the process of motif attachment. By pruning chemically invalid molecules have |C| & on the CrossDocked dataset. Therefore, the attachment and scoring will not be a burden on the computational efficiency.
and merging isomorphic graphs,
After selecting the molecule attachment graph, we also need to assemble the new motif the in- termediate molecule in the 3D space. The assembling process of the first motif is challenging since there no reference motifs to attach with and the relative position the binding pocket is unknown. A straightforward method is to randomly place the first motif near the focal protein atom. However, such a strategy noisy and predicted coordinates can be far from the optimal solutions. Following (Jin et al., (202).. we use a distancee-aseed initialization strategy, which is more accurate and stable than random initialization. Specifically, a distance matrix D E RR +mm x (m'mmm) is set as:
8: 8ill (h (0)
i,j S n' i is n' n' i,j > n',
Dij
h
(6)
||r: ryll
where n' and m' denote the number of sampled protein atoms for reference and the number of atoms in the first molecular motif. The distances between the protein atoms and motif atoms can be directly calculated. For the distances between molecular and protein atoms, we use MI.Pd for prediction with the pairwise atom attributes as the input. With the distance matrix D. we can obtain the coordinates of atoms by eigenvalue decomposition of its Gram matrix (Crippen & Havel 1978):
D.j 0.5(D + Di - -Di,), D = USU
(7)
where S is a diagonal matrix with eigenvalues in descending order. The coordinate of each atom r: calculated as:
A X X Xt,)). X-UVS.
(8)
Note that even though predicted coordinates (f.) retain the original distance D. they are located in
a different coordinate system. Therefore, we apply the Kabsch algorithm (Kabsch 1976) to find
rigid body transformation (R. 1) that aligns the predicted protein coordinates {ri, ...... , Aw) } with the reference coordinates (r1, " ,Twh Lastly, the coordinates of the first motif are calculated as:
r: RÑ; i n'.
5

---

Published as a conference paper at ICLR 2023
For generation steps with t >> 1, the coordinates of the attached motifs are determined and aligned similarly with RDkit (Bento et al. 2020) and Kabsch algorithm (Kabsch] 1976). The atoms in the focal motif are used the reference atoms. If the focal motif contains rotatable bond, the new motif has an additional degree of freedom. The torsion angle should be further predicted and adjusted.
Rotation Angle Prediciton: After attaching the new motif and obtaining the initial coordinates, we apply the encoder again get the updated hidden representations. Let X, Y denote the two end atoms of the rotatable bond (specifically, let Y denote the atom connecting the new motif). The change of the torsion angle, Aa predicted as:
Aa MLP" (hx""(x, hhy, hg))mmd2r,
(10)
where xx and hy indicate the embeddings of X and Y; hig denotes the embedding of the molecule, which is obtained with a sum pooling. Since the predicted angle is based on the representations from the equivariant encoder, aa is also rotationally and translationally invariant. Finally, the coordinates of the atoms in the new motif are updated by rotating Aa around line XY. The implementation details are included Appendix
Structure Refinement: According the design of our generation scheme, each step t >> fix the coordinates of ((t-1), predict and attach the new motif. However, we find the generated co- ordinates may be inaccurate and lead to sub-optimal generated molecules for the following reasons: (1) the distance-based initialization at t = 1 may be inaccurate because the pairwise distances are predicted independently without considering higher-order interaction terms; (2) at > 1, we fix the coordinates (((-1) and ignore the interactions between ((t-1) and the new motif. Therefore, we propose to incorporate an structure refinement process at each step of generation. The refinement procedure should be equivariant as the coordinates of (t--1) and P can be rotated and translated arbitrarily. Inspired by force fields in physics (Rappé et al. 1992p. we calculate the force between atoms. Specifically, after placing the new motif at each step 6, we additionally use encoder to
compute the atom embeddings h The force between atoms in ((t) is calculated as:
(t) g g(h/)) (1) hif)) CRBF(IIr: ryll)) ||r |
T:
(11)
ryll"
where ERBF (Ilr; ryll) is the radial basis distance encoding, the function g is feed-forwer neural Ti-my
network with a scalar output, and [y---yl| is the normalized verctor from the j-th atom to the i-th
atom. For the The force between g()) and P, we only calculate the alpha carbons C2 P for computational efficiency. Similarly, we have:
g g(h (t) h CRBE(lIT: - skll)) ||: 8%
r: 8k
(12)
Finally, the coordinates of (()) is updated as:
r
(8) gis
Σ
(8) gik
(13)
g
PPc.
where [g(() denotes the number of molecular atoms at step I; PPc., denotes the number of C. atoms in the binding pocket. In the Appendix B we perform ablation studies to show the effectiveness of structure refinement.
3.5 TRAINING
In the training stage, we first extract motifs for molecules with the method described in Sec 3.2 we randomly mask the motifs of molecules and train the model to recover the masked ones. Specifically, for each pocket-ligand pair, sample mask ratio from the uniform distribution ( 1] and mask the corresponding number of structural motifs. The motifs are generated a breadth-firg order and the root motif is set the motif closest the pocket. The atoms that have valence bonds to the masked motifs are defined as focal atom candidates. If all molecular atoms are masked, the focal atoms are defined protein atoms that have masked ligand atoms within 4 Á.

---

Published as a conference paper at ICLR 2023
For the motif type prediction, For the focal atom/motif prediction, we use a binary cross entropy loss foeal for the classification of focal atoms. For the distance prediction and structure refinement, with respect to the pairwise distances. We add Gaussian noise to the ground truth coordinates in the training stage and refine the structure with the predicted force. For the motif attachment, maximize the log-likelihood of predicting the correct molecular graph:
cross-entropy loss for the classification, denoted as Lmotis-
minimize an MSE loss Ca
Lattach (Gi) + log
Σ
exp(Ja(G')),
(14)
G'EC/O.
where Ge denotes the ground truth and C is the set of possible candidates. As for the torsion angle prediction, we fit angles with von Mises distributions with La similar to (Senior et al. 2020). In the training process, aim to minimize the sum of the above loss functions:
Lmottff + G + Lattach + La + L..
(15)
EXPERIMENTS
4.1
EXPERIMENTAL SETTINGS
Dataset: Following Luo et al. a0 and (Peng et al. 2022), we use the CrossDocked dataset (Francoeur al. 2020) which contains 22.5 million protein-molecut
structures. We filter out data points whose binding pose RMSD is greater than 1 Á and molecules that can not be sanitized with RDkit (Bento et al. 2020). leading to a refined subset with around 160k data points. We use mm- seqs2 (Steinegger & Söding 2017) to cluster data at 30% sequence identity, and randomly draw 100,000 protein-ligand pairs for training and 100 proteins from remaining clusters for testing. For evaluation, randomly sample 100 molecules for each protein pocket in the test set.
We compare FLAG with four ofate-of-the-ar baseline methods: one method based on 3D CNNs (LiGAN) (Ragoza al. 2022) and another three methods based on 3D GNNs (AR, GraphBP, and Pocket2Mol) (Luo et al. 2021a Liu et al. 2022 Peng et al. 2022).
Baselines:
Model: The number of layers L in context encoder is 6, and the hidden dimension is 256. The model is trained with the Adam optimizer at learning rate of 0.0001. The batch size is 4 and the number of total training iterations is 1,000,000. The standard deviation of the added Gaussian noise the
ligand coordinates is 0.2. The cutoff distance in the context encoder is set to 10 A. The threshold motif extraction is set to 100 in the default setting.
Metrics: We choose metrics that are widely used in previous works (Luo et al. 2021a Peng et al.
2022 Liu et al. 2022) to evaluate the qualities of the sampled molecules: (1) Vina Score measures
the binding affinity between the generated molecules and the protein pockets; (2) High Affinity is
calculated as the percentage of pockets whose generated molecules have higher affinity to the ref- erences in the test set; (3) QED measures how likely molecule is potential drug candidate; (4)
Synthesizability (SA) represents the difficulty drug synthesis (the score is normalized between and and higher values indicate casier synthesis); (5) LogP is the octanol-water partition coefficient (LogP values should be between -0.4 and 5.6 to be good drug candidates (Ghose et al.
1999)): (6)
Lipinski (Lip.) calculates how many rules the molecule obeys the Lipinski's is rule of five (Lipin-)
ski et al. 2012); (7) Sim. Train represents the Tanimoto similarity (Bajusz et al., 2015) with the most similar molecules in the training set; (8) Diversity (Div.) measures the diversity of generated molecules for a binding pocket (It is calculated as average pairwise Tanimoto similarities). (9) Time records the time cost to generate 100 valid molecules for pocket. our work, the Vina Score is calculated by QVina (Trott & Olson 2010: Alhossary et al. (015)) and the chemical properties are calculated by RDKit (Bento et 2020) over the valid molecules. Before feeding Vina, the generated molecular structures are firstly refined by universal force fields (Rappé et al. 1992).
4.2 RESULTS
In Table. we show the mean values along with standard deviations of the above metrics. Generally, method achieves competitive or better performance compared with the baseline methods. Even though our method is not explicitly optimized for the binding affinity, FLAG succeeds at generating molecules with higher affinity than reference molecules 58% of the binding site, which is quite
7

---

Published as a conference paper at ICLR 2023
Table Comparing the molecular properties of the test set and the generated molecules by different methods. The best results are bolded.
Vina Score (kcal/mol, J)
High (ffinit(())
QED (1)
Lip.
Sim.
Div.
Time (L)
Methods
(1)
LogP
(1)
Train (1)
(1)
-7.180
0,483
0.710
0.932
4.373
Testset
+ 2.27
10.23
10.15
12.71
11.17
-6.129
0.214
0.395
0.612
-0.126
4.013
0.455
0.620
LiGAN
+1.63
+0.12
+0.24
+0.18
+2.58
+1.25
+0.24
+0.13
-6,231
0.236
0.489
0.676
0.261
4.757
0.417
0.675
18962
AR
11.6
10.13
0.21
0.17
1.2.3
10.42
+ 0.21
+ 0.16
19794
-7.012
0,430
0.471
0.706
0.439
4.776
0,410
0.651
1059.4
GraphBP
+1.7
10.25
10.18
10.25
12.05
10.45
10.23
10.14
1.36.7
Pocket2
-7.113
0.536
0,522
0,733
1.215
4.896
0.385
0.693
2655.1
Mol
+2.42
+0.26
+0.23
+0.22
+2.39
#0.24
+0.20
+0.18
+1740
-7.247
0.580
0.495
0.745
0.630
4.943
0.370
0.704
1047.6
Ours
+2.25
+0.24
+0.17
#0.16
+2.38
10.14
=0.22
10.15
6.1.
Ours
Reference (3bml)
Vina:
Vina: 3.2
Vna: -7.9
Vira:779
Vina:-7.7
Vina--.8 -7.6
Vina: 775
QEI: 0.50
(U/0: SA: 0.63
QUD: 0.88 0.85
QED: 0.91 5A:0.78
QED: 0.71 SAL
QFD: 0.85 SA: 0.8
Ours Ours(3haa)
Reforence (3hwf)
Vinz: 6.4
Vina: 5.4 QED: SA: 0.58
Wina: 6.3
Vinar-5 5599
Win:: QED: 0.50 SA: 0.90
QED: 0.52 SA: 0.56
QED: 0.63 SA: 0.66
QED: 0.53 0,70
QED: 0.61 S4: 0.57
QED: SA: 0.66
Figure 3: Examples of generated 3D molecules that have higher binding affinity than reference molecules. Lower Vina score indicates higher binding affinity.
impressive. When it comes to drug potentials (QED, SA, LogP, and Lipinski), we observe that FLAG obtains the best result SA and Lipinski, which indicates that molecules generated by FLAG more likely to be drug candidates. Moreover, our model manages to generate diverse molecules the molecules in the training dataset. This shows that our model
with the lowest similarities can generalize to new protein pockets and does not just memorize the training dataset. Table [ also shows the generation time comparisons among 3D GNN-based methods, Our model has comparable sampling efficiency with GraphBP, and is much faster than AR and Pocker2Mol. This is because our method generates molecules tiggeent-by-fragmeet and has much fewer generation steps. Due to page limits, more discussions of molecule generations are shown in Appendix B
In Figure 3 we provide two case studies and show several examples of generated 3D molecules that have higher binding affinity (lower vina scores) than their corresponding reference molecules. Firstly, we observe that our generated molecules with higher affinity are largely different from ref- erence molecules in structure. This further validates that model can generate diverse and novel molecules to bind target proteins, instead of just imitating or modifying reference molecules. More- over, the QED and SA scores comparable even higher than the reference molecules.
8

---

Published as a conference paper at ICLR 2023
Venta Fear
Opeen
160
(b)
(c)
(d)
Figure 4: Distributions for C-O bond lengths (a), CCO bond angles (b), CCCC dihedral angles (c), and CCCO dihedral angles (d) in the training set and generated molecules.
Table 2: The KL divergence of the bond angles (upper part) and dihedral angles (lower part) between the test and the generated molecules. The lower letters represent the atoms in the aromatic rings.
Pocket2 Mol
FLAG w/o SR
FLAG w/o T
Angles
LiGAN
AR
GraphBP
FLAG
CCC
7.26
2.17
1.67
0.88
0.30
0.25
0.24
CCO
8.31
1.95
2.02
0.93
0.17
0.19
0.12
CNC
6.90
2.36
1.26
0.24
0.22
0.18
NCC
7.53
3.59
2.10
0.68
0.29
0.26
0.21
CC=O
6.85
3.56
1.23
0.80
0.23
0.27
0.15
CCCC
1.26
0.81
0.79
0.73
0.80
0.82
0.77
cocc
6.43
8.26
5.41
4.30
0.33
0.31
0.30
CCCO
2.34
1.55
1.09
1.14
0.57
0.93
0.51
Ccce
4.98
8.27
3.13
0.58
0.51
0.48
CC=CC
6.01
5.86
3.08
3.40
1.54
1.76
1.22
4.3 SUBSTRUCTURE ANALYSIS
Substructure analysis is needed further evaluate the generated molecules Peng et al. (2022). In Table. [ we measure the bullback-Leible (KL) divergence between the distributions of the bond angles/ dihedral angles in the test set and generated molecules by different methods. The lower the values, the better the methods
capture the distributions of the dataset. In Table. - we can observe that our method has much lower KL divergences than the baseline methods over nearly all the angle types. This demonstrates that method can keep the geometric attributes of the data well and generate molecules with realistic substructures. This again shows the advantage of the fragment-wiss generation scheme. To further investigate how well model learns the distribution of substructures, we show the bond length distributions (C-O single bond), bond angle distributions (CCO chains), and two sets of dihedral angle distributions (CCCC and CCCO chains) in Figure We have the following observations: Firstly, the distributions of bond length/angle show clear clus- ters, indicating the values bond length/angle are largely fixed. This also verifies the rationality to use chemical priors and tools to determine the bond lengths/angles. Secondly, the distributions of dihedral angles are more complex and is difficult to pre-specify them when attaching new motifs. Therefore, we predict and adjust the torsion angle based on the context information in FLAG. Fi- nally, the distributions generated molecules by FLAG align well with the training set, indicating that the bond distances/angles and dihedral angles are accurately modeled and reproduced.
5 CONCLUSION
In this paper, we propose a Fragment-based LigAnd Generation framework, known as FLAG,, to generate 3D molecules with valid and realistic substructures fragment-by-fragmene FLAG not only achieves competitive performances on conventional metrics such as binding affinity but also outper- forms baselines by a large margin on generating realistic structures with speed-ups. Potential future
works include: (1) exploring more powerful context encoders such Graph Transformer Zhang
et al. (2022b); (2) building more robust and interpretable SBDD models [hang et al. 2202
2022d c (3) using pre-training further improve performance Chaiaanceehhngggg Zhhng et al. (2021dec

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
This research was partially supported by a grant from the National Natural Science Foundation of China (Grant No. 61922073).
REFERENCES
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):221-22216, 2015.
Simon Axelrod and Rafael Bonnz---omaarlli... Geom, for property prediction and molecular generation. Scientific Data, 9(1 ): 1-14, 2022.
molecular conformations
Dávid Bajusz, Anita Rácz, and Károly Héberger. Why is tanimoto index an appropriate choice for fingerprint-basee based similarity calculations? Journal of cheminformatiis, 7(1):1-13, 2015.
A Patrícia Bento, Anne Hersey, Eloy Félix, Greg Landrum, Anna Gaulton, Francis Atkinson, Louisa J Bellis, Marleen De Veij, and Andrew R Leach. An open source chemical structure curation pipeline using rdkit. Journal of Cheminformatics, 12(1):1-16, 2020.
Binghong Chen, Tianzhe Wang, Chengtao Li, Hanjun Dai, and Le Song. Molecule optimization by explainable evolution. In ICLR, 2021a.
Ziqi Chen, Martin Renqiang Min, Srinivasan Parthasarathy, and Xia Ning. deep generative model for molecule optimization via one fragment modification. Nature Machine Intelligence, 3(12): 1040-1049, 2021b.
Gordon Crippen and Timothy F Havel. Stable calculation of coordinates from distance informa- tion. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 34(2):282-288 1978.
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas. Vector neurons: A general framework for so (3)-cuvivariant networks. In CVPR, pp- 12200-12209, 2021.
Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. Molgensurvey: systematic survey in machine learning models for molecule design. arXiv preprini arXiv:2203. 14500, 2022.
Daniel Flam-Shepherd, Alexander Zhigalin, and Alán Aspuru-Guzik. Scalable fragment-basee 3d molecular design with reinforcement learning. arXiv preprint wxXiv:2202.00658, 2022.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. three-dimensional convolutional neural networks and cross-docked data set for structure- based drug design. Journal of chemical information and modeling, 60(9):4200- 4215, 2020.
Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer en- sembles. NeurlPS, 2021.
Johannes Gasteiger, Janek Groß, and Stephan Günnemann. Directional message passing for molec- ular graphs. In ICLR. 2019.
Niklas Gebauer, Michael Gastegger, and Kristof Schütt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. NeurIPS, 32, 2019.
Niklas WA Gebauer, Michael Gastegger, Stefaan SP Hessmann, Klaus-Robert Müller, and Kristof Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):1-11, 2022.
Arup K Ghose, Vellarkad N Viswanadhan, and John Wendoloski. A knowledge-bart approach in designing combinatorial medicinal chemistry libraries for drug discovery. 1. qualitative and quantitative characterization of known drug databases. Journal of combinatorial chemistry, 1(1): 55-68, 1999.
10

---

Published as a conference paper at ICLR 2023
Rafael Gómez-Bombarelle Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Timothy Hirzel,
Benjamín Sánchez- Lengheli-nngginngg Dennis Sheberla, Jorge Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268-276, 2018.
Harrison Green, David R Koes, and Jacob D Durrant. Deepfrag: a deep convolutional neural network for fragment-basee baase lead optimization. Chemical Science, 12(23):8036-8044 00078000668877,008 2021.
Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, and Wojciech Matusik. Data- efficient graph grammar learning for molecular generation. ICLR, 2022.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation 3d. In ICML., pp. 8867-8887. PMLR, 2022.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In ICML, pp. 2323-2332. PMLR, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In ICML, pp. 4839-4848. PMLR, 2020a.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In ICML, pp. 4849-4859. PMLR, 2020b.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Antibody-antigen docking and design via hierarchical structure refinement. In ICML, pp. 10217-10227. PMLR, 2022.
Bowen Jing. Stephan Eismann, Pratham N Soni, and Ron O Dror. Equivariant graph neural networks for 3d macromolecular structure. ICML, 2021.
Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallo- graphica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32 (5):922-923, 1976.
Yibo Li, Jianfeng Pei, and Luhua Lai. Structure-based de novo drug design using 3d deep generative models. Chemical science, 12(41):13664-133 2021.
Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches estimate solubility and permeability in drug discovery and develop- ment settings. Advanced drug delivery reviews, 64,4-17,201
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. ICML, 2022.
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In ICLR, 2021.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. NeurlPS, 34:6229 6239, 2021a.
Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang. Predicting molecular conformation via dy- namic graph score matching. NeurIPS, 34:19744-19755, 2021b.
Youzhi Luo and Shuiwang Ji. An autoregressive flow model for 3d molecular geometry generation from scratch. In ICLR, 2021.
Elman Mansimov, Omar Mahmood, Seokho Kang, and Kyunghyun Cho. Molecular geometry pre-
diction using a deep generative graph neural network. Scientific reports, 91)::-13, 2019.
Vitali Nesterov, Mario Wieser, and Volker Roth. 3dmolnet: generative network for molecular structures. arXiv preprint arXiv:2010.06477, 2020.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. ICMI., 2022.
11

---

Published as a conference paper at ICLR 2023
Marco Podda, Davide Bacciu, and Alessio Micheli. A deep generative model for fragment- based molecule generation. In International Conference on Artificial Intelligence and Statistics, pp- 2240-2250. PMLR, 2020.
Alexander Powers, Helen Yu, Patricia Suriana, and Ron Dror. Fragment- ligand generation guided by geometric deep learning protein-ligand structure. bioRxiv, 2022.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional receptor binding sites with deep generative models. Chemical science, 13(9):201-2273,, 2022.
Anthony K Rappé, Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. Uff, full periodic table force field for molecular mechanics and molecular dynamies simulations. Journal of the American chemical society, 11 114255):0024-10035, 1992.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In ICML, pp- 1530-1538. PMLR, 2015.
Victor Garcia Satorras, Emiel Hoogeboom, Fabian B Fuchs, Ingmar Posner, and Max Welling. (n) equivariant normalizing flows. NeurIPS, 2021.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg. Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European semantic web conference, pp. 593-607. Springer, 2018.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmicla, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: continuous- convolutional neural network for modeling quantum interactions. NeurlPS, 30, 2017.
Andrew Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Židek, Alexander WR Nelson, Alex Bridgland, al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):70--710 2020.
Seonghwan Seo, Jaechang Lim, and Woo Youn Kim. Fragment-based molecular generative model with high generalization ability and synthetic accessibility. arXiv preprint arXiv:2111.12907, 2021.
Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular confor- mation generation. In ICML, pp. 9558 9568. PMLR, 2021.
Gregor Simm and Jose Miguel Hernandez-Lobto A generative model for molecular distance ge- ometry. In ICML, pp. 8949 8958. PMLR, 2020.
Miha Skalic, Davide Sabbadin, Boris Sattarov, Simone Sciabola, and Gianni De Fabritiis. From target to drug: generative modeling for the multimodal structure-based ligand design. Molecular pharmaceutics, 16(10):4282-429) 2019.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(1 35(11):1026-1028 2017.
Oleg Trout and Arthur Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):555466 2010.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang. Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. ICLR, 2021.
Keyulu Xu, Weihua Hu. Jure Leskovee, and Stefanie Jegelka. How powerful graph neural networks? ICLR, 2019.
Mingyuan Xu, Ting Ran, and Hongming Chen. De novo molecule design through the molecular generative model conditioned by 3d information protein binding sites. Journal of Chemical Information and Modeling, 61(7):3240-355 2021a.
12

---

Published as a conference paper at ICLR 2023
Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Gomez-Bombare Bombarelli, and Jian Tang. An end-to-end framework for molecular conformation generation via bilevel program- ming. In ICML, pp. 11537-11547. PMLR, 2021b.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geomet- ric diffusion model for molecular conformation generation. ICLR, 2022.
Zaixi Zhang, Jinyuan Jia, Binghui Wang, and Neil Zhenqiang Gong, Backdoor attacks to graph neural networks. In Proceedings of the 26th ACM Symposium on Access Control Models and Technologies, pp. 15-26, 2021a.
Zaixi Zhang. Qi Liu, Zhenya Huang, Hao Wang, Chengqiang Lu, Chuanren Liu, and Enhong Chen. Graphmi: Extracting private graph data from graph neural networks. IJCAI, 2021b.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self- supervised learning for molecular property prediction. Advances in Neural Information Process- ing Systems, 34:15870-15882, 2021c.
Zaixi Zhang, Qi Liu, Shengyu Zhang, Chang- Yu Hsieh, Liang Shi, and Chee- Kong Lee. Graph self- supervised learning for optoelectronic properties of organic semiconductors. ICML, A14Science workshop, 2021d.
Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Fldetector: Defending feder- ated learning against model poisoning attacks via detecting malicious clients. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2545-2555,
2022a.
Zaixi Zhang, Qi Liu, Qingyong Hu, and Chee-Kong Kone Lee. Hierarchical graph transformer with adaptive node sampling. Advances Neural Information Processing Systems, 2022b.
Zaixi Zhang, Qi Liu, Zhenya Huang, Hao Wang, Chee-Kong Lee, and Enhong Chen. Model inver- sion attacks against graph neural networks. IEEE Transactions Knowledge and Data Engi- neering. 2022c.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Cheekong Lee. Protgnn: Towards self- explaining graph neural networks. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 36, pp. 9127-9135, 2022d.
13

---

Published as a conference paper at ICLR 2023
A IMPLEMENTATION DETAILS
A. IMPLEMENTATION OF BASELINES
To implement the baselines including LiGAN AR GraphBF and Pocket2Mo we use the open-source codes and follow their default settings. The methods are trained on the same data split for fair comparison. All the experiments are conducted on Ubuntu Linux with V100 GPUs. The codes are implemented in Python 3.8 and Pytorch 1.10.0. To sample molecules, we apply independent runs with random seeds. Our code publicly available at: hhtps:///iihhe com/ zai xi zhang/FLaS
A.2 DEFNITION OF BOND LENGTII AND BOND ANGLE
For a bond AB, its bond length is defined d(A, def ||r TBLL. where FA and TB de- note the 3D coordinates. For bonded chain ABC, the bond angle is defined as cosLABC def (MA (PA-ra)-(re---0
A.3 DEFINITION OF DIHEDRAL ANGLE
The count-clockwise (CCW) dihedral angle between two intersecting half-planes ABC and ABD with an common edge AB calculated as
(ABC, ABD) def
(b2n((1bll(bn)) b2 x ba), (br x b2, b2 x bz)),
(16)
where b = TC, b2 TB- TA, b3 =D TBB
A.4 CANONICAL DEFINITION OF TORSION ANGLE
As introduced by (Ganea et al., 2021), the torsion angle can be defined in a canonical way. Let X, Y be the coordinates of the two end atoms and {Pt). {Q₂} be the coordinates of their neighbors respectively. Let Aij def (XYP,XXQQ) denote the counter-clockwise dihedral angle. 8-j
def
[cos(Ai)), sin (), Let ei real coefficients such that s def Eis Cig8is
1R2 is not null
vector. The torsion angle is defined: def atan2(111])
In our method, instead of predicting the exact value of a, predict the change of the torsion angle Aa and update the coordinates by the rotation matrix described in Appendix A.6
A.5 CALCULATION OF KL DIVERGENCE
To calculate two distributions of bond/dihedral angles, we set one degree as a bin and calculate the normalized frequencies (180 bins for bond angles and 360 bins for dihedral angles). The KI. divergence between two distributions p(0) and q(0) is calculated as:
KL p(..) log( p(0.)
(17)
(
A.6 DETAILS OF MOTIF ROTATION
In the generation of motifs, if the focal motif is rotatable and the rotation angle Aa is known, we use the following rotation matrix Rax3 and the translation vector t3x1 to update the coordinates of the new motif. Let X.Y denote the two end atoms of the rotatable bond (Y denote the atom
ottaaa//ffabbcco///sst/////eeeee SEDD
14

---

Published as a conference paper at ICLR 2023
Table 3: The ratio of the molecules containing different rings in the test set and those generated by different methods. The results closest to the ratios of the test set are bolded.
Ring Size
Test Sct
Pocket2 Mol
LiGAN
AR
GraphBP
Ours
0.029
0.354
0.495
0.330
0.005
0.020
4
0,000
0.223
0.004
0.146
0,000
0.000
0,440
0.384
0.289
0.367
0.396
0.433
0.839
0.319
0.650
0.738
0.814
0.852
0.012
0.066
0.037
0.024
0,061
0.023
0.000
0.019
0.007
0.010
0,005
0.003
0.000
0.011
0.003
0.004
0.003
0.002
connecting the new motif) and rx and ry be their coordinates. Let n denotes the normalized directional vector Try ry=rx and Bay my and n, be its components along the æ, y and axis. Let No, yo. and zo the three components of rx- The rotation matrix and translation vector are:
n K + (os(Ax)
Vu K - nzsin(Aci) n K + cos( Ao)) ny K + ny =nsin(Aa))
n2 K + ny Aa) ] ny K nin An) n K + cos (aa)
Rax3= =
Bany K + ngsin( (dx)) n ny sysin (ax)
(18)
(00 m2 + (Nzy0 ny sini Aax)
13x1
(yo
ny My + (n ng
sin Ao)
(19)
(zo
R K + (NyT0 No: sin Aa)
where K = cos(Aar) and M ng + nyy0 + n2 The coordinates v; in the motif are
updated as:
r = Rr:
(20)
A.7 MORE DETAILS OF MOTIF ATTACHMENT
To determine the 3D coordinates of the new motif and attach it to the generated molecule, we lever- age RDkit and Kabsch algorithm. The Kabsch algorithm (Kabsch) 1976) is widely-used method to calculate the optimal rotation matrix that minimizes the RMSD (root mean squared deviation) between paired sets of points. The atoms in the focal motif are selected as the reference atoms. We first use RDkit to get the 3D coordinates of the new motif and reference atoms. Since the co- ordinates of the new motif and the intermediate molecule (t--1) are located in different coordinate frames, employ Kabsch algorithm align them. Finally, update the molecule coordinates by appending the coordinates the new motif.
B MORE EXPERIMENTAL RESULTS
B. MORE ANALYSIS OF SUBSTRUCTURES
In Table show the ratios of the molecules containing different rings in the test set and the gen- crated molecules by different methods. We can observe that the molecules generated by FLAG have the most similar ratios of rings of different sizes with the test set. This again shows the advantage of FLAG to learn and reproduce the distributions of substructures in the dataset.
B.2 ABLATION STUDIES
In Table we show the results of ablation studies regarding the structure refinement module (FLAG w/o SR) and the torsion angle module (FLAG w/o T). Specifically, in FLAG w/o SR, we remove the structure refinement in each generation step; in FLAG w/o T. do not further rotate the motif for adjustment. Table we can observe that FLAG achieves the best performance on binding affinity and likeness metrics (QED, SA, LogP, Lip.), verifying the effectiveness of the two modules.
15

---

Published as a conference paper at ICLR 2023
Table 4: Ablation studies of the structure refinement module and the torsion angle module. FLAG w/o SR and FLAG w/o T denotes removing the structure refinement and torsion angle prediction and rotation procedure respectively.
Vina Score (kcal/mol, 4)
High Affinity(1)
QED
Lip.
Sim.
Div.
Time
Methods
SA (1)
LogP
(1)
(f)
Train (4)
(1)
(4)
FLAG FLAG w/o
-7.247
0.580
0,495
0.745
0.630
4,943
0.370
0.704
1047.6
-7.016
0.437
0.465
0.710
0.692
4,925
0.396
0.651
1018.9
FLAG w/o T
-6,945
0.356
0.486
0.706
0.733
4.927
0.407
0.689
1013.5
Moreover, with these two modules, FLAG can generate more unique and diverse molecules. Admit- tedly, structure refinement and torsion prediction bring more computational overhead. However, the overhead is acceptable considering the gain generation performance.
B.3
ANALYSIS
Here, mainly discuss the influence of hyper- parameter т on the generation performance. In Fig- 3 vary T from 0 to 500 and show the average Vina scores of the generated molecules by FLAG and its variants. Lower vina scores indicate higher binding affinity and better generation quality. We have the fol- lowing observations. Firstly, our methods are robust to the choice of т. For example, FLAG has the largest vina score when than the lowest vina score at = 100. Moreover, an appropriate value important for FLAG achieve the best performance. Lower typically indicates larger size of motif vocabulary with redundant motifs. On the other hand, larger indicates smaller size of motif vocabulary with finer extracted motifs (e.g.. sin- gle bonds), which may lead too many generation steps and invalid generated molecules. Finally, an interesting observation is that FLAG w/o T has the best performance at 500. This is probably due to the reason that finer motifs offer more flexibility when the torsion angle module is removed in FLAG.
Vina Score (kcal/mool)
-7.0
-7.1
-7.2
FLAG SR
0. The value is only 0.2 larger
FLAGwo T
0
50
100
200
500
analysis with respect to т. Lower Vina scores indicate higher binding affinity and better genera- tion quality.
Figure 5:
B.4 GENERATED MOLECULES WITH LOWER AFFINITIES THAN THE REFERENCE
In Figure. 6 we show pocket (3tym) where FLAG fails generate molecules with higher affinities than the reference. FLAG fails to generate molecules with higher binding affinities probably due to the following reasons: (1) The geometry of this binding pocket (3tym) more complex and some generated molecules fail to occupy the whole pocket like the reference molecule. (2) Some generated molecules accidentally collide with the pocket, which unrealistic nature.
Ours
Reforence
9.4
Vinar-7.4 QuD: 0.71
-9.5
qit: 0.50 SA:
Ogt::
QED: 0.80 SA: 0.86
QJ0:
Figure 6: Examples of generated 3D molecules that have lower binding affinity than reference.
16


Reviews:
[{"review_id": "J8fyxEzyCp", "reviewer": "Reviewer_fSis", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new 3D molecule-generating model conditioned on a receptor. It generates a ligand molecule fragment-by-fragment, unlike other similar models that generate a molecule atom-by-atom. They first extract a set of motifs serving as building blocks of generation. Then context encoding, focal motif selection, next motif prediction and attachment, angle prediction, and refinement are followed. The experiment results show that it outperforms all the baselines in almost all metrics. Substructure analysis shows that it better captures the distributions of the dataset compared to other baselines.", "summary_of_the_review": "Since the model is novel and effective, this paper will be a good addition to ICLR.", "strength_and_weaknesses": "[Strength]\n* The overall framework is not only innovative but also effectively driven by the functional requirements of each step. For example, rotation angle prediction and structure refinement through force field are plausible and shown to be effective by the ablation study.\n* The paper is easy to read and well organized.\n\n\n[Weaknesses]\n* 3hwf case in Figure 3: All generated molecules have lower SA scores than the reference molecule. Is there any chance that the proposed model will produce molecules that cannot be synthesized?\n* Minor typo - Page 6. \"we fix the the coordinates\" -> \"we fix the coordinates\"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "8MeCk88jk3N", "reviewer": "Reviewer_xsvH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles the problem of structure-based drug design. The authors propose a framework, called FLAG, where the molecules are generated fragment-by-fragment. The authors first reconstructed their motif vocabulary and then generate molecules while considering interactions with target atoms. The authors conduct various experiments on the benchmarks of structure-based molecular generation and achieved performance improvements on some metrics.", "summary_of_the_review": "Overall, this work is borderline. The authors did a good job of combining motif-based and target-based approaches, and there are some performance improvements. However, the technical and theoretical contributions are insufficient. The ablation studies are not solid enough. Thus, I vote for a borderline rejection but may change the score during rebuttal.", "strength_and_weaknesses": "Positives:  The author provides a clear and complete formulation of the problem and conducts various experiments to demonstrate the effectiveness of his design. As the experimental results of the paper suggest, fragment-based molecular generation does yield more rational structures.\n\nNegatives: The technical contributions of this work are somewhat trivial. Motif vocabulary construction is a common strategy and has been well-explored in the field of drug design, and both predicting the torsion angle and selecting the focal motif are not proposed for the first time.  The interaction with the target was designed similarly to the previous method.\n\nQuestions:\n1. Could the authors explain why the results in Tables 1, 2 in this paper are not consistent with those reported in other benchmarks like Pocket2Mol?\n2. Could the authors provide more ablation experiments to demonstrate the effect of different parts of the FLAG on Table 2 results?\n3. If limiting the size of molecule generation is dependent on the interaction with the target, would expanding the size of the pocket affect the quality of molecule generation?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7nVbPkw3k4-", "reviewer": "Reviewer_nStG", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a model for generating molecules (represented as a 3D structure) that bind to a specific protein site (also represented in 3D). The authors propose a fragment-by-fragment sequential approach, which is summarized in the following steps:\n\n- substructure (or motif) vocabulary creation \n- context encoding (the context is initially just the binding site, then it's binding site + existing molecule)\n- focal motif selection (i.e. selecting the initial motif / the motif used to expand the existing molecule)\n- next motif prediction (i.e. selecting the motif to attach from the vocabulary\n- motif attachment and orientation\n- substructure refinement\n\nIn the experiments, the authors show that the model generates molecules with high binding affinity, sometimes even better than the reference, while also possessing high drug-likeness. ", "summary_of_the_review": "In general, I find this paper a solid contribution, probably worthy of ICLR. The most disappointing part is that I cannot look at the code, which to me makes it irreproducible. \n\nBesides this, I have several questions I'd like the authors to answer, and some comments/suggestions/corrections.\nPlease acknowledge that I am not an expert in structure-based drug design (though I know the general problem and main approaches).\n\nIn no particular order:\n- What is the vocabulary size? Do 2D-isomorphic motifs with different geometry get encoded as two different \"words\" in the vocabulary?\n- You say that after pruning chemically invalid molecules and _merging isomorphic graphs_ you reduce the set of candidates attachments. How do you handle merging isomorphic graphs, considering that it can be computationally expensive in certain cases?\n- What are the valid, novel, and unique fractions of the molecules generated for the test set?\n- Can you show examples where the model fails to generate molecules that bind to a given target (as in Fig. 3)?\n- Why did you use QVina to evaluate the Vina scores, instead of more recent alternatives (GNina)? \n- You state that all the generated molecules were further refined with universal force fields before being fed to Vina. Did the molecules generated with the baselines get the same treatment?\n- Why did you use GIN as a scoring function for candidate attachments?\n- At times, the notation is confusing. For example, in Eq. 4), the $\\mathrm{MLP}$ function has two arguments, while in Eq. 9 it has three. I thought that would mean concatenation, but then when in the equation where you define the distance matrix (which by the way is not numbered) the $\\mathrm{MLP}^d$ uses $concat$ to describe a concatenated input. Please correct or explain.\n- you say that $\\boldsymbol{e}_{\\mathrm{RBF}}$ can be either a Gaussian function or a spherical Bessel function. Which is the actual used in this work?\n- you should use \\citep instead of \\cite for the citations.\n- there is a missing closing parenthesis in Eq. 10\n\nFinally, I think there are two citations that should be considered in Section 2 when you talk about motif-based generation.\n- Podda et al. A Deep Generative Model for Fragment-Based Molecule Generation. AISTATS 2020\n- Seo et al., Fragment-based molecular generative model with high generalization ability and synthetic accessibility. https://arxiv.org/abs/2111.12907\n\n**Overall judgment**\n\nI am giving a score of 6 for the moment being, will wait for the authors' replies to see whether there's space to increase my score.\n\n**EDIT**\n\nSince the authors kindly responded to all my questions and most of the questions raised by the other reviewers, I am changing my vote to an 8.\n\n", "strength_and_weaknesses": "**Strengths**\n\n- Impressive experimental results\n- Substructure refinement appears innovative and its importance is confirmed through ablation studies\n\n**Weaknesses**\n- not reproducible\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "-vHk0w3hag", "reviewer": "Reviewer_WWrH", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In order to design ligand molecules that bind to specific protein binding sites, the authors propose a Fragment-based LigAnd Generation framework (FLAG), to generate 3D molecules with valid and realistic substructures fragment-by-fragment. A motif vocabulary is constructed by extracting common molecular fragments (i.e., motif) in the dataset. At each generation step, a 3D graph neural network is first employed to encode the intermediate context information. Then, the model selects the focal motif, predicts the next motif type, and attaches the new motif. The bond lengths/angles can be quickly and accurately determined by cheminformatics tools. Finally, the molecular geometry is further adjusted according to the predicted rotation angle and the structure refinement. In their conducted experiments, results shows that their method is able to generate diverse drug-like molecules with good binding affinity to target proteins, is faster than most of the baseline methods at sampling new molecules and outperforms baselines by a large margin on generating valid molecules with realistic substructures. ", "summary_of_the_review": "In order to design ligand molecules that bind to specific protein binding sites, the authors propose a Fragment-based LigAnd Generation framework to generate 3D molecules with valid and realistic substructures fragment-by-fragment. Some design ways of 3D molecule generation are tentative and fairly inspiring. The paper is well written and organized clearly but the novelty may be not strong enough since the authors do not illustrate key differences between this FLAG model and other fragment-based methods. And the contribution to the drug design community is limited since the authors do not provide code implementations of this model.\n\nAfter reading the responses, I raise my score a bit to 6.\n", "strength_and_weaknesses": "Strength:\n1.\tIt is an encouraging attempt since it is a quite difficult task to generate target-based molecules in 3D space.\n2.\tThis method is of fragment-based molecular generation, which is in accord with functional groups in organic chemistry.\n3.\tSince the assembling process of the first motif and rotation angle prediction are subtly hard to deal with, this paper provides a tentative solution.\n\n\nWeakness:\n1.\tThe paper does not conduct the experiment or provide sufficient evidence about target specificity of the FLAG model, i.e. the comparison between the model with target information and the model without target information. It is possible that FLAG model just fits the structural distribution of the whole ligand library rather than the distribution conditioned on the specific target. \n2.\tThe authors do not illustrate key differences between this FLAG model and other fragment-based methods, such as “Fragment-Based Ligand Generation Guided By Geometric Deep Learning On Protein-Ligand Structure” and “Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning”. Besides, the authors say, “However, in these works, starting fragments are required and the motif vocabulary needs to be constructed by experts, limiting their practical applications.” However, introducing experts’ knowledge to generate drug-like molecules in order to avoid uncommon or toxic fragments is quite reasonable in drug design. This statement is invalidated.\n3.\tAs for the motif extraction step, there are some doubts when it comes to find all the rotatable bonds. Rotatable bonds are usually just single bonds in organic chemistry. If rotatable bonds are under the condition that “where either u or v is part of a ring”, how does this model to deal with the situation that several single bonds are connected end-to-end?\n4.\tThe “Motif Attachments Enumeration” way is quite similar to JT-VAE.\n5.\tIt is not clearly demonstrated when the authors introduced the motivation of RBF in the “Encoder” part.\n6.\tIn the “Structure Refinement”, it is not well illustrated that “the force between atoms” proposed by authors has true physical meaning or not.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster