Title: Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration
Venue: NeurIPS 2023
Accepted: yes
Keywords: sturcture-based drug design; molecule generation; diffusion model
Abstract: In recent years, AI-assisted drug design methods have been proposed to generate molecules given the pockets' structures of target proteins. Most of them are  {\em atom-level-based} methods, which consider atoms as basic components and generate atom positions and types. In this way, however, it is hard to generate realistic fragments with complicated structures. To solve this, we propose \textsc{D3FG}, a {\em functional-group-based} diffusion model for pocket-specific molecule generation and elaboration. \textsc{D3FG} decomposes molecules into two categories of components: functional groups defined as rigid bodies and linkers as mass points. And the two kinds of components can together form complicated fragments that enhance ligand-protein interactions.
 To be specific, in the diffusion process, \textsc{D3FG} diffuses the data distribution of the positions, orientations, and types of the components into a prior distribution; In the generative process, the noise is gradually removed from the three variables by denoisers parameterized with designed equivariant graph neural networks.  In the experiments, our method can generate molecules with more realistic 3D structures, competitive affinities toward the protein targets, and better drug properties. Besides, \textsc{D3FG} as a solution to a new task of molecule elaboration, could generate molecules with high affinities based on existing ligands and the hotspots of target proteins.

Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration

Abstract

In recent years, AI-assisted drug design methods have been proposed to generate molecules given the pockets' structures of target proteins. Most of them are atomlevel-based methods, which consider atoms as basic components and generate atom positions and types. In this way, however, it is hard to generate realistic fragments with complicated structures. To solve this, we propose D3FG, a functional-groupbased diffusion model for pocket-specific molecule generation and elaboration. D3FG decomposes molecules into two categories of components: functional groups defined as rigid bodies and linkers as mass points. And the two kinds of components can together form complicated fragments that enhance ligand-protein interactions. To be specific, in the diffusion process, D3FG diffuses the data distribution of the positions, orientations, and types of the components into a prior distribution; In the generative process, the noise is gradually removed from the three variables by denoisers parameterized with designed equivariant graph neural networks. In the experiments, our method can generate molecules with more realistic 3D structures, competitive affinities toward the protein targets, and better drug properties. Besides, D3FG as a solution to a new task of molecule elaboration, could generate molecules with high affinities based on existing ligands and the hotspots of target proteins.

1 Introduction

The established notion in drug-target identification is that similar structures perform similar functions. This principle allows classical computer-aided drug design (CADD) to abstract protein-ligand interactions as pharmacophores, aligning similar functional groups and extracting pharmaceutical

Yufei Huang

Lirong Wu

Zhiyuan Chen

information from these structures. Fitting suitable functional groups into the pharmacophores can enhance ligand-protein interactions, thus improving drug efficiency [1, 2].

Artificial intelligence has already achieved outstanding success in protein design [3-8] and thus led to a new round of attention in drug design focused on AI-assisted drug design (AIDD). A number of methods of AI-assisted drug design on pocket-specific molecule generation are emerging [9-15], thanks to developments in deep generative models [16-21] and graph neural networks (GNNs) [22-31]. These methods usually focus on atom-level generation, which first generates atom types and positions and then assembles atoms into molecules that can bind to the protein pockets. Although significant progress has been made, they are still weak in two aspects. For one thing, it is hard for them to generate realistic functional groups that contribute pharmacological effects to the target as classical CADD methods are able to. It is shown that generating benzene rings is uncommon compared with the reference molecules, not to mention some large functional groups with complex structure constraints such as purine, indole, etc (Table. 1 in Sec. 5.2). For another, trade-offs between efficiency and sufficiency of protein context place them in a dilemma. For example, TARGETDIFF [13] employs sufficient protein context, which disassembles the amino acids into atoms, but leads to inefficiency due to a large node number (412.14 on average) in GNN's message passing. DIFFSBDD [14] simplifies the representation of protein context, by only using C α 's positions and residue types, resulting in a reduction of GNNs' node number (68.10 on average) but the insufficiency of context information.

To address the above issues, we establish a f unctionalg roup-based d i ff usion model (D3FG), including the following contributions: 1. Method Novelty. We denote the molecules' functional groups and proteins' amino acids as the same level's fragments, in which the intra-relative positions of atoms are fixed like rigid bodies, and represent single atoms as linkers. The positions and orientation of local structures and the atom type variables are generated gradually through denoising processes. The fragment-linker designation leads the binding systems to heterogeneous graphs, and thereby, two schemes are proposed as solutions, which achieve competitive performance in terms of molecule structures, binding affinity, and drug properties, and sufficiency in encoding protein context by employing more features. 2. Dataset Establishment. Though the CrossDocked2020 [32] has been a widely-used dataset for evaluation methods' performance on the task, the analysis of the molecules' functional groups of it is missing. We deeply explore the details of the inter-relative positions and types of functional groups of the molecules and establish an extendable database of common functional groups. 3. New task. Besides molecule generation, we propose molecule elaboration as another task that our model can fulfill. Fragment hotspot maps (FHM) [33, 34] are used to preprocess paired protein-molecule in CrossDocked2020 for the task. As a result, our model generates molecules with high binding affinity based on the reference.

2 Problem Statement

For a binding system composed of a protein-molecule pair (also called protein-ligand pair) as C , which contains N aa amino acids of proteins and N fg functional groups and N at single atoms of molecules, we represent the index set of the molecule's single atoms as I at , functional groups as I fg , and the protein's amino acids as I aa , where |I fg | = N fg , |I at | = N at and |I aa | = N aa . Note that a molecule can be disassembled into functional groups and single atoms other than functional groups, which we also call linkers. For a protein, the amino acids can be represented by its type, C α atom coordinate, and the orientation, denoted as s i ∈ { 1 , . . . , 20 } , x i ∈ R 3 , O i ∈ SO(3) , where i ∈ I aa . For a molecule, assuming there are M fg and M at possible types in total functional groups and linker atoms respectively, the functional groups can be represented as the three elements if the inter-relative positions are fixed, as s j , x j and O j , where s j ∈ { 21 , . . . , 21 + M fg } is the type, x j ∈ R 3 is the predefined center atom's coordinate, and O j ∈ SO(3) can also be obtained in the same way as amino acids (See Sec. 5.1 and Appendix. D.) for j ∈ I fg ; And its linkers can be represented as s k , x k and O k , with s k ∈ { 22 + M fg , . . . , 22 + M fg + M at } , k ∈ I at and O k = diag { 1 , 1 , 1 } = I .

Therefore, C = { ( s i , x i , O i ) } N aa + N fg + N at i =1 can be split into two sets as C = P ∪ M , where P = { ( s i , x i , O i ) : i ∈ I aa } and M = { ( s j , x j , O j ) : j ∈ I fg ∪ I at } . For protein-specific molecule generation, our goal is to establish a probabilistic model to learn the distribution of molecules conditioned on the target proteins, i.e. p ( M|P ) . In the following, we omit i ∈ I aa and j ∈ I fg ∪ I at by default unless specified.

3 Related Work

3D molecule generation. Previous methods on molecule generation fucus on 1D-smiles-string [35-38] or 2D-molecule-graph [39-43]. In recent years, more works concentrate on 3D molecule generation, thanks to fast development in equivariant graph neural networks [44-46] and generative models [16-21]. Molecular conformation generation aims to generate 3D structures of molecules with stability, given 2D molecule graph structure, [47-51]. Further, De novo molecular generation attempts to generate both 2D chemical formulas and 3D structures from scratch [52-54].

Fragment-based drug design. Previously, works on fragment-based molecule generation are proposed. For example, JT-VAE [55] generates a tree-structured scaffold over chemical substructures and combines them into a 2D-molecule. PS-VAE [56] can automatically discover frequent principal subgraphs from the dataset, and assemble generated subgraphs as the final output molecule in 2D. Further, DEEPFRAG [57] predicts fragments conditioned on parents and the pockets, SQUID [58] generates molecules in a fragment level conditioned on molecule's shapes. FLAG[59] autoregressively generates fragments as motifs based on the protein structures in 3D.

Structure-based drug design. Success in 3D molecule generation and an increasing amount of available structural data of protein and molecules raises scientific interests in structure-based drug design (SBDD), which aims to generate both 2D molecule graphs and 3D structures conditioned on target protein structure as contextual constraints. For example, LIGAN [60] and 3DSBDD [12] are grid-based models which predict whether the grid points are occupied by specific atoms. By harnessing 3D equivariant graph neural networks, POCKET2MOL [9] and GRAPHBP [10] generate atoms auto-regressively and model the probability of the next atom's type as discrete categorical attribute and position as continuous geometry. FLAG[59] is also fragment-based, but still generates motifs in an auto-regressive way. Recently, utilizing the diffusion denoising probability models [61-63, 53, 64], a series of SBDD methods generate ligands conditioned on the target pockets at full atom levels [13-15].

4 Method

D3FG firstly decomposes molecules into two categories of components: functional groups and linkers, and them use the diffsion generative model to learn the type and geometry distributions of the components. In this section, we describe the D3FG by four parts: (i) the diffusion model as the generative framework, in which the three variables are generated; (ii) the denoiser parameterized by graph neural networks, satisfying certain symmetries so that the generative model is SE-(3) invariant; (iii) the sampling process in which the molecules are generated by the trained models; (iv) the further problems resulted from heterogenous graph with two solutions.

4.1 Diffusion Models

Diffusion models construct two Markov processes to learn the data distributions. The first called the forward diffusion process adds noises gradually until the noisy data's distribution reaches the prior distribution; The other called the generative denoising process, gradually removes the noise from the data sampled from the prior distribution until they are recovered to the desired data distribution. Assume there are T steps in both processes, and we denote M t = { ( s t j , x t j , O t j ) } as the t -th noisy state in the forward process, with M T ∼ Prior( M T ) , and M 0 = M , where the transition distribution is denoted as q ( ·|· ) ; In the generative process, sample goes from T to 0 , in which the transition distribution is denoted as p ( ·|· ) . Here we define the forward and generative processes of s t j , x t j , and O t j .

Absorbing diffusion for functional group and linker types. Let s t j as the one-hot encoding of the type of a single functional group or linker. The forward process followed by D3PM [65] randomly transits s t j into the absorbing state K ( K = 23 + M fg + M at ) with

Division / Mergence

···

···

···

Joint Scheme

Diffusion

Generation

Diffusion

Two-stage Scheme

Generation

C

O

c1ccccc1

C(=O)O

Abosrobing Type

Initial Distribution

where ¯ Q t = Q 1 Q 2 . . . Q t and [ Q t ] mn = q ( s t j = n | s t -1 j = m ) denotes diffusion transition probabilities, with

̸

̸

β t type monotonically increases from 0 to 1, means that when t = T , all the type variables are absorbed into the K -th category. In the generative process, it first samples N at linkers and N fg functional groups whose types are all in the absorbing states, selects (1 -β t type ) × 100% of them respectively, and transforms their types from the absorbing states to the predicted ones by

where F is the type denoiser parameterized with a neural network, and F ( · , · )[ j ] predicts the probability of the types for the j -th selected functional groups or linkers. With the effectiveness of BERT-style training [66], the denoiser directly predicts p ( s 0 j |M t , P ) , leading to a training objective as

Gaussian diffusion for center atom positions. By defining the center atom in a functional group as shown in Appendix. D, and itself as the center in a linker, the Cartesian coordinate of center nodes x j represents its position. The forward transition distributions followed by DDPM [61] read

in which β t pos increases from 0 to 1 , means that the noise levels are increasing and the data's coordinate signals fade out during the forward diffusion, with α t pos = 1 -β t pos , ¯ α t pos = α 0 pos α 1 pos . . . α t pos , and finally x T j ∼ N ( 0 , I ) . Note that Eq. (5) is equivalent to the Markov process of x t j = √ ¯ α t pos · x 0 j + √ 1 -¯ α 0 pos · ϵ j , where ϵ j ∼ N ( 0 , I ) . Rather than predicting the mean value of the reverse transition distribution in the generative process, the position denoiser G approximates the added noise ϵ j with the reparameterization trick as

The training objective is thus established in a score-based way, as

SO(3) diffusion for functional group orientations. By regarding the functional groups as rigid bodies, orientations together with the center atoms' positions determine all atoms' positions. Here we represent the orientation geometry as elements in SO(3) . Following [67], we use isotropic Gaussian distribution on SO(3) [68] to formulate the process, i.e. IG so(3) ( ·| µ ori , σ ori ) , in which µ ori and σ ori are viewed as mean orientation and variance, in analogy with Gaussian distribution. The transition distribution for orientation matrices O j reads

λ ori (¯ α t ori , O 0 j ) is the geodesic flow from I to O t j by the amount ¯ α t ori , as λ ori (¯ α t ori , O 0 j ) = exp(¯ α t ori log( O 0 j )) , where exp( · ) and log( · ) are exponential and logarithm map on the SO(3) manifold. As α t ori → 0 , λ ori (¯ α t ori , O 0 j ) → I . { β t ori } T t =0 is the predefined noise level schedule ranging from 0 to 1 as t increases, α t ori = 1 -β t ori and ¯ α t ori = α 0 ori α 1 ori . . . α t ori .

In the generative process, an orientation denoiser H is used to predict the mean orientation in the isotropic Gaussian distribution, which reads

We use the same loss function as in [69] to minimize the expected discrepancy measured by the inner product between the data orientation matrices and the predicted ones, which reads

4.2 Parametrization of Denoisers with Neural Networks

Amino acid context encoding. In order to decrease the computational complexity, we denote the protein context at amino-acid levels. Besides the amino acid types, C α atom coordinate and the orientation, each atom's coordinates in the local system and three torsion angles including angles around ' N -C α ' bond, ' C α -C ' bond and ' C -N ' bond, are also used as intra-amino-acid features, which are concatenated and embedded by an MLP to create the intra-amino-acid embedding vector e i . For inter-amino-acid features, the pair of amino acid types, sequential relationships (if the two amino acids are adjacent in the protein sequence), pairwise distances between C α and inter-residue dihedrals are all embedded as inter-amino-acid embedding vector z i,j with i, j ∈ I aa . Note that these embedding vectors are all translational and rotational invariant (Appendix. C).

Denoisers with equivariance. In the setting of generative models, the learned distribution p ( M|P ) should be equivariant to translation and rotation, such that p ( T g ( M ) | T g ( P )) = p ( M|P ) for any g ∈ SE(3) , where T g is the corresponding roto-translational transformations, and T g ( M ) means each atom in the molecule is rotated and translated by T g . In the setting of diffusion models, the following proposition indicates the translational and rotational equivariance of each denoisers.

Proposition 1. Let p ( x T ) , p ( O T ) , and p ( s T ) be SE(3)-invariant distribution, and the transition distributions p ( x t -1 |M t , P ) be SE(3)-equivariant, p ( O t -1 |M t , P ) be T(3)-invariant and SO(3)equivariant, and p ( s t -1 |M t , P ) be SE(3)-invariant, then the density p ( M|P ) modeled by the reverse Markov Chains in the generative process of diffusion models is SE(3)-equivariant.

According to the Proposition 1 , while the denoisers for functional group and linker types are rototranslational invariant, the denoiser for positions should be roto-translational equivariant, and it for orientations should be translational invariant and rotational equivariant. Therefore, we employ our denoiser's network structures with IPA [3] by harnessing the expressivity of TRANSFORMER [70] and roto-translational equivariance of LOCS [71]. Denote the binding system as a graph, in which the nodes are composed of amino acids, functional groups, and linkers, and the edges are established with

Algorithm 1 Joint Generation for Molecules using D3FG

Input: Zero-centered protein { s i , x i , O i } i ∈I aa , and graph denoiser F, G, H , and node number sampler ϕ fg , ϕ at .

Sample N fg ∼ ϕ fg , N at ∼ ϕ at , leading to the index set I fg and I at .

Sample initial states of functional groups,

O

SO(3)

for

T

j

∼

Uniform for

in

t

T

1

-

, T

Sample end for

K-nearest neighbor. Let { h i : i ∈ I aa ∪ I fg ∪ I at } be the node embedding which is SE(3)-invariant, { e i : i ∈ I aa ∪I fg ∪I at } and { z i,j : i, j ∈ I aa ∪I fg ∪I at } be the previously defined intra- and interamino acid embedding, with e j = 0 and z i,j = 0 if i / ∈ I aa or j / ∈ I aa . The attention mechanism in IPA updates the embedding of node i as

where e ′ i = h i + e i , h ′ i is the updated node embedding, W q , W k , W v are learnable parameters, and N ( i ) is neighborhood of node i obtained by the edges. Because h i , e i , and z i,j are all SE(3)invariant, h ′ i is also invariant. Three heads parameterized with MLP are stacked after several layers of Transformers update the node embedding, denoted by MLP F ( · ) , MLP G ( · ) , and MLP H ( · ) is used for updating s t -1 , x t -1 and O t -1 . The LOCS updates the parameters in Eq. (3), (6) and (9) by

The output of G means predicting the coordinate deviations in the local coordinate systems and then projecting it into the global one; In H , MLP H first predicts a vector in Lie group so (3) and the exponential map on SO(3) converts it into a rotation matrix. The updating process ensures the equivariance and the invariance of the transition distributions in Proposition 1 , which is proved in [71] and [69]. However, since O t i = I for any i ∈ I at , the equivariance of G on linkers will not be preserved, so we instead use EGNN [44] to get the output of G (Appendix. C).

4.3 Sampling Process

In sampling, we first use two prior distributions as the empirical distributions ϕ at and ϕ fg drawn from the training set to sample the linker N at and functional group number N fg . As p ( x T ) , p ( O T ) and p ( s T ) should be SE(3)-invariant, p ( s T ) = 1 { K } ( s T ) , p ( x T ) = N ( x T | 0 , I ) and p ( O T ) = Uniform SO(3) ( O T ) satisfy the conditions, where 1 · ( · ) is the indicator function and Uniform SO(3) ( · ) is the uniform distribution on SO(3) . And the iteratively generative process of p ( s t -1 |M t , P ) , p ( x t -1 |M t , P ) and p ( O t -1 |M t , P ) are given in Eq. (3), (6) and (9). The detailed sampling algorithm is given in Algorithm. 1.

4.4 Heterogeneous graph: Joint or Two-stage?

Amino acids and functional groups are both fragments composed of atoms in proteins and molecules, regarded as rigid bodies, while the linkers are single atoms, regarded as mass points. Therefore, in the binding graph, nodes are of different levels and connections are of different kinds, thus leading the graph to be heterogeneous. For this reason, we propose two generative schemes: joint generation scheme and two-stage generation scheme as shown in Figure. 1.

In joint scheme, we regard amino acids, functional groups, and linkers at the same level, and use one single neural network to predict the three variables and update them. In detail, this scheme directly

{

s

t

-

j

2

-

1

j

, . . . ,

,

x

t

-

j

1

t

j

,

O

∈ I

fg

1

do

}

j

-

1

else

∈I

fg

∪I

O

at

{

T

j

s

T

j

=

,

x

I

.

as Eq. (3), (6) and (9) and update

T

j

,

O

T

j

}

j

∈I

∪I

, where fg

at

s

t

j

=

M

K

,

-

1

.

t

x

T

j

∼ N

(

0

,

I

)

,

models p ( M|P ) = p ( { s i , x i , O i } i ∈I fg ∪I at |P ) , where the parameterized transition distribution is p ( { s t -1 i , x t -1 i , O t -1 i } i ∈I fg ∪I at |M t , P ) . Note that O t i = I for any t if i ∈ I at .

In two-stage scheme, we regard amino acids and functional groups at the fragment level, and linkers at the atom level, and use two different neural networks to parameterize the transition distribution. In the first stage, the functional groups are generated, and then single atoms as linkers will be generated to connect the generated functional groups as a full molecule. The two-stage generative scheme is similar to CADD, which first determines the pharmacophores towards the target protein, fits functional groups with high activity into them, and then searches for the possible molecules with these functional groups. In specific, the generative process reads p ( M|P ) = p ( { s j , x j } j ∈I at |{ s i , x i , O i } i ∈I fg , P ) p ( { s i , x i , O i } i ∈I fg |P ) . The transition distribution of the first stage p ( { s t -1 i , x t -1 i , O t -1 i } i ∈I fg |{ s t i , x t i , O t i } i ∈I fg , P ) is parameterized by one neural network; In the second stage, the generated { s i , x i , O i } i ∈I fg is used as context, so that the other neural network models p ( { s t -1 i , x t -1 i } i ∈I at |{ s t i , x t i } i ∈I at , { s i , x i , O i } i ∈I fg , P ) .

5 Experiments

5.1 Data Processing.

In the experiments, we use CrossDocked2020[32] for evaluation. In the prevailing works, they focus on generating molecules at the atom level, differing from our functional-group-based generation, so our first target is to divide the molecules into functional groups and linkers. We use EFGs [72] to segment molecules. We select the 25 functional groups that appear most frequently with stable structures, which are partly shown in Figure. 2 For some functional groups, chirality exists in their structures, and we treat them as two functional groups. As a result, we finally build up a dataset as a corpus including 27 functional groups (two of the 25 have chirality) for Crossdocked2020, with their intra-structures fixed as rigid bodies, and assure that most molecules can be decomposed into the substructures in our corpus datasets. Details are given in Appendix. D. For linkers, we choose { B , C , N , O , F , P , S , Cl , Br , I } as representative heavy atoms. After the processing, we obtain M fg = 27 and M at = 10 in our experiments. Besides, by the fragment-linker designation of a binding graph, the node number is reduced to 53.62 on average in GNN's message-passing.

5.2 Pocket-Specific Molecule Generation

Dataset. The datasets for training and evaluation are split according to POCKET2MOL [9] and TARGETDIFF [13]. 22.5 million docked protein binding complexes with low RMSD ( < 1Å) and sequence identity less than 30% are selected, leading to 100,000 pairs of pocket-ligand complexes, with 100 novel complexes as references for evaluation.

Setup. For performance comparison, our methods are compared with baselines including LIGAN [60], 3DSBDD [12], GRAPHBP [10], POCKET2MOL [54], DIFFSBDD [14] and TARGETDIFF [13]. LIGAN as a 3D CNN-based method generates atoms on regular grids, with VAE [73] as its generative model. 3DSBDD, GRAPHBP and POCKET2MOL are all GNN-based, generating atoms in an auto-regressive way. DIFFSBDD and TARGETDIFF are two diffusion-based methods that generate molecules at the full atom level, with equivariant GNNs as the denoisers. The two schemes of generation lead to two variants of our method, written as D3FG(Joint) and D3FG(Stage). In some parts, we choose POCKET2MOL as the representative autoregressive methods, and DIFFSBDD and TARGETDIFF as the benchmarks employing diffusion models, because these three baselines are the

c1ccccc1

O

NC=O

O

HO

O=CO

N

c1ccncc1

N

N

NH

N

c1ncc2nc[nH]c2n1

latest works that show good empirical performance. We sample 100 valid molecules for each pocket for the baselines and D3FG, leading to 10,000 pairs of complexes. After the molecules are generated by the model, Openbabel [74] is used to construct chemical bonds between atoms, and Universal Force Field (UFF) minimization [75] is used for refinement. The following evaluations are based on the samples. Figure. 4 gives an example of generated molecules by different methods.

Structure analysis. We first analyze the functional groups of generated molecules by different methods. We show the 'Ratio' and 'Freq' of the included functional groups partly, where 'Ratio' means how many specified functional groups are in each molecule on average, and 'Freq' means the statistical frequency of occurrence of the specified functional group in the generated molecules. Mean absolute error (MAE) as overall metrics is calculated according to reference 'Ratio' and generated 'Ratio'; Jensen-Shannon divergence (JSD) is calculated according to reference 'Freq' and generated 'Freq'. The smaller MAE and JSD are, the better performance the method achieves. Table. 1 shows the metrics of different methods' generated molecules, demonstrating D3FG's superiority in generating molecules with realistic drug structures since distributions of the functional groups in generated molecules are more similar to the real drug molecules'. Secondly, we analyze the atom type , bond distance , bond angle , dihedral and atom type distribution. JSDs are calculated between the distributions of bond distance for reference vs. generated molecules. '-', '=',':' represents single, double, and aromatic bonds, respectively. Besides, we report MAE on reference 'Ratio' vs. generated 'Ratio' and JSD on reference 'Freq' vs. generated 'Freq' based on atom type distribution. Table. 2, 3 and Figure. 3 gives details atom type analysis. For other geometries, please refer to Appendix. E. We found that D3FG(Stage) outperforms D3FG(Joint) in generating more realistic molecules, and has competitive performance with TARGETDIFF.

Binding affinity. We secondly make a comparison of Binding Affinity. Following 3DSBDD and LIGAN, we employed two evaluation metrics including Vina docking score [76] and Gnina docking score [77]. Vina docking [78] as a classical docking tool, gives a lower score as Vina energy if the binding affinity of the molecule is better, while Gnina docking as a deep-learning-based docking tool, gives it a higher score. ∆ Affinity measures the percentage of generated molecules that have better predicted binding affinity than the corresponding reference molecules. Table. 4 shows that our D3FG

C

N

O

F

Atom Type

Frequency

0.672

0.117

0.170

0.013

0.762

0.100

0.129

0.004

0.714

0.088

0.176

0.009

0.702

0.116

0.159

0.008

0.741

0.124

0.175

0.009

0.733

0.089

0.154

0.008

Ref.

Pocket2Mol

TargetDiff

DiffSBDD

D3FG(Joint)

D3FG(Stage)

Method

MAE

(

↓

)

JSD

(

↓

)

POCKET2MOL

0.573

0.098

TARGETDIFF

0.471

0.059

DIFFSBDD

0.627

0.054

D3FG(Joint)

0.528

0.075

D3FG(Stage)

0.294

0.056

of the two-stage scheme achieves competitive affinity scores, comparable to TARGETDIFF and much better than D3FG of the joint scheme.

Drug property. Moreover, chemical properties are evaluated with RdKit [79], including QED [80] (quantitative estimation of drug-likeness), SA [81] (synthetic accessibility score), LogP [82] (the octanol-water partition coefficient, which should be between -0 . 4 and 5 . 6 for good drug candidates), and Lipinski [83, 84] (number of rules the drug follow the Lipinski's rule of five). QED, SA, and Lipinski are three metrics with preferences for atom numbers, demonstrated in DIFFBP[15]. Table. 4 demonstrates that two variants of D3FG achieve overall best performance in these metrics.

(a) Ref.

(b) D3FG (Stage)

(c) D3FG (Joint)

(d) DIFFSBDD

(e) TARGETDIFF

(f) POKECT2MOL

5.3 Pocket-Specific Molecule Elaboration

Introduction. Molecule elaboration as a sub-task of molecule optimization, aims to elaborate a fragment of existing molecules amenable to chemical modification for improving binding affinity. To fulfill it, we here first select a functional group in a ligand that contributes to binding affinity and remove it to obtain the editable fragments. Then, we attempt to use D3FG to generate the new type of functional groups with its structures, for modifying the fragments and thus building up a new molecule with higher binding affinity to the target protein.

Dataset. The selection of a functional group for replacement is the first problem. Pharmacophoric information is extracted by calculating the fragment hotspot map (FHM) [33, 34] of the target protein. In specific, FHM describes regions of the binding pocket that are likely to make a positive contribution to binding affinity. Then, by placing the molecules into the pockets, we can thus obtain each functional group's hotspot scores, according to the binding complexes. The higher hotspot score a functional group reaches, the more contributions it makes to the binding affinity. Functional groups' hotspot scores of each ligand are calculated based on 100,000 pairs of pocket-ligand complexes in Crossdocked2020, and the selection of functional groups can be based on their scores. Finally, we established our new datasets for molecule elaboration based on FHM.

Binding affinity and drug property. We here consider two schemes for molecule elaboration, the first is to remove the functional groups with the highest scores, and elaborate the remaining fragments by replacing them with the functional group generated by D3FG. We write this as D3FG(EHot). The second D3FG(ECold), on the other hand, replaces the functional groups of the lowest scores). We report the D3FG's elaborated molecules by the two schemes in Table. 4. It shows that D3FG(EHot) tends to generate more molecules with higher affinity, while molecules elaborated by D3FG(ECold) show tiny differences in binding affinity from the raw references. Besides, on other chemical properties, the elaborated molecules are very close to the raw references, since the differences lie only in one single functional group, and the major molecular skeletons are almost identical.

6 Conclusion and Future Work

In this paper, a functional-group-based diffusion model called D3FG is proposed to generate molecules in 3D with protein structures as its context. Joint and two-stage generation schemes lead to two variants of D3FG. The molecules generated by the two-stage generation scheme show more realistic structures, competitive binding performance, and better drug properties. Besides, in molecule elaboration, D3FG can also generate molecules with good binding affinity. However, limitation still exists. First, the functional group datasets are still small, which will be enlarged in the future. Second, the binding affinities of generated molecules still remain to be improved, since other diffusion models even show better binding performance.

References

T. Langer and R.D. Hoffmann. Pharmacophores and Pharmacophore Searches . 08 2006.

M. Afshar, A. Lanoue, and J. Sallantin. 4.30 - multiobjective/multicriteria optimization and decision support in drug discovery. In John B. Taylor and David J. Triggle, editors, Comprehensive Medicinal Chemistry II , pages 767-774. Elsevier, Oxford, 2007.

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon Kohl, Andrew Ballard, Andrew Cowie, Bernardino RomeraParedes, Stanislav Nikolov, Rishub Jain, Jonas Adler, and Demis Hassabis. Highly accurate protein structure prediction with alphafold. Nature , 596:1-11, 08 2021.

Zhangyang Gao, Cheng Tan, Stan Li, et al. Alphadesign: A graph protein design method and benchmark on alphafolddb. arXiv preprint arXiv:2202.01079 , 2022.

Yufei Huang, Lirong Wu, Haitao Lin, Jiangbin Zheng, Ge Wang, and Stan Z. Li. Data-efficient protein 3d geometric pretraining via refinement of diffused protein structure decoy, 2023.

Lirong Wu, Yufei Huang, Haitao Lin, and Stan Z. Li. A survey on protein representation learning: Retrospect and prospect, 2022.

Zhangyang Gao, Cheng Tan, and Stan Z Li. Pifold: Toward effective and efficient protein inverse folding. arXiv preprint arXiv:2209.12643 , 2022.

Cheng Tan, Zhangyang Gao, Jun Xia, and Stan Z Li. Generative de novo protein design with global context. arXiv preprint arXiv:2204.10673 , 2022.

Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning , 2022.

Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. In International Conference on Machine Learning , 2022.

Cheng Tan, Zhangyang Gao, and Stan Z Li. Target-aware molecular graph generation. arXiv preprint arXiv:2202.04829 , 2022.

Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3D generative model for structure-based drug design. In Thirty-Fifth Conference on Neural Information Processing Systems , 2021.

Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In International Conference on Learning Representations , 2023.

Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia. Structure-based drug design with equivariant diffusion models, 2022.

Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Z. Li. Diffbp: Generative diffusion of 3d molecules for target protein binding, 2022.

Carl Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 , 2016.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS , pages 2672-2680, 2014.

Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real NVP. arXiv preprint arXiv:1605.08803 , 2016.

Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in Neural Information Processing Systems , 31:10215-10224, 2018.

Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. Advances in Neural Information Processing Systems , 33:12685-12696, 2020.

Yoshua Bengio, Tristan Deleu, Edward J. Hu, Salem Lahlou, Mo Tiwari, and Emmanuel Bengio. GFlowNet foundations. CoRR , abs/2111.09266, 2021.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks, 2016.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry, 2017.

Meng Liu, Youzhi Luo, Limei Wang, Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang, Yi Liu, et al. DIG: A turnkey library for diving into graph deep learning research. Journal of Machine Learning Research , 22(240):1-9, 2021.

Lirong Wu, Haitao Lin, Cheng Tan, Zhangyang Gao, and Stan Z. Li. Self-supervised learning on graphs: Contrastive, generative,or predictive. IEEE Transactions on Knowledge and Data Engineering , pages 1-1, 2021.

Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations , 2021.

Lirong Wu, Haitao Lin, Zhangyang Gao, Cheng Tan, and Stan. Z. Li. Graphmixup: Improving class-imbalanced node classification on graphs by self-supervised context prediction, 2021.

Lirong Wu, Haitao Lin, Bozhen Hu, Cheng Tan, Zhangyang Gao, Zicheng Liu, and Stan Z. Li. Beyond homophily and homogeneity assumption: Relation-based frequency adaptive graph neural networks. IEEE Transactions on Neural Networks and Learning Systems , pages 1-13, 2023.

Lirong Wu, Haitao Lin, Yufei Huang, and Stan Z. Li. Knowledge distillation improves graph structure augmentation for graph neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 11815-11827. Curran Associates, Inc., 2022.

Lirong Wu, Haitao Lin, Yufei Huang, Tianyu Fan, and Stan Z. Li. Extracting low-/highfrequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework, 2023.

Lirong Wu, Haitao Lin, Yufei Huang, and Stan Z. Li. Quantifying the knowledge in gnns for reliable distillation into mlps, 2023.

Paul Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard Iovanisci, Ian Snyder, and David Koes. 3d convolutional neural networks and a crossdocked dataset for structure-based drug design. Journal of Chemical Information and Modeling , XXXX, 08 2020.

Mihaela D. Smilova, Peter R. Curran, Chris J. Radoux, Frank von Delft, Jason C. Cole, Anthony R. Bradley, and Brian D. Marsden. Fragment hotspot mapping to identify selectivitydetermining regions between related proteins. Journal of Chemical Information and Modeling , 62(2):284-294, 2022. PMID: 35020376.

Chris J. Radoux, Tjelvar S. G. Olsson, Will R. Pitt, Colin R. Groom, and Tom L. Blundell. Identifying interactions that determine fragment binding at protein hotspots. Journal of Medicinal Chemistry , 59(9):4314-4325, 2016. PMID: 27043011.

Esben Jannik Bjerrum and Richard Threlfall. Molecular generation with recurrent neural networks (rnns), 2017.

Rafael Gó mez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science , 4(2):268-276, jan 2018.

Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational autoencoder, 2017.

Marwin H. S. Segler, Thierry Kogej, Christian Tyrchan, and Mark P. Waller. Generating focussed molecule libraries for drug discovery with recurrent neural networks. ArXiv , abs/1701.01329, 2017.

Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander Gaunt. Constrained graph variational autoencoders for molecule design. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation, 2020.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation, 2019.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures, 2020.

Jiaxuan You, Bowen Liu, Rex Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation, 2019.

Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks, 2021.

Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael J. L. Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In Submitted to International Conference on Learning Representations , 2021.

Bowen Jing, Stephan Eismann, Pratham N. Soni, and Ron O. Dror. Equivariant graph neural networks for 3d macromolecular structure, 2021.

Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Gomez-Bombarelli, and Jian Tang. An end-to-end framework for molecular conformation generation via bilevel programming. 2021.

Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. Proceedings of the 38th International Conference on Machine Learning, ICML , 139:9558-9568, 2021.

Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang. Predicting molecular conformation via dynamic graph score matching. Advances in Neural Information Processing Systems , 34, 2021.

Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. GeoDiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations , 2021.

Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, and Jian Tang. Learning neural generative dynamics for molecular conformation generation. International Conference on Learning Representations, (ICLR) , 2021.

Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs, Ingmar Posner, and Max Welling. E(n) equivariant normalizing flows, 2021.

Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d, 2022.

Youzhi Luo and Shuiwang Ji. An autoregressive flow model for 3d molecular geometry generation from scratch. In International Conference on Learning Representations , 2021.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation, 2019.

Xiangzhe Kong, Wenbing Huang, Zhixing Tan, and Yang Liu. Molecule generation by principal subgraph mining and assembling, 2022.

Harrison Green, David R. Koesb, and Jacob D. Durrant. Deepfrag: a deep convolutional neural network for fragment-based lead optimization. In Chemical Science , 2021.

Keir Adams and Connor W. Coley. Equivariant shape-conditioned generation of 3d molecules for ligand-based drug design. In The Eleventh International Conference on Learning Representations , 2023.

ZAIXI ZHANG, Shuxin Zheng, Yaosen Min, and Qi Liu. Molecule generation for target protein binding with structural motifs. In International Conference on Learning Representations , 2023.

Tomohide Masuda, Matthew Ragoza, and David Ryan Koes. Generating 3d molecular structures conditional on a receptor binding site with deep generative models. arXiv preprint arXiv:2010.14442 , 2020.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. arXiv preprint arXiv:2006.11239 , 2020.

Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems , 32, 2019.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 , 2020.

Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models, 2022.

Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces, 2023.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.

Adam Leach, Sebastian M Schmon, Matteo T. Degiacomi, and Chris G. Willcocks. Denoising diffusion probabilistic models on SO(3) for rotational alignment. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning , 2022.

T. I. Savjolova. Preface to novye metody issledovanija tekstury polikristalliceskich materialov. metallurgija.

Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigenspecific antibody design and optimization with diffusion-based generative models for protein structures. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017.

Miltiadis Kofinas, Naveen Shankar Nagaraja, and Efstratios Gavves. Roto-translated local coordinate frames for interacting dynamical systems, 2022.

Peter Ertl. An algorithm to identify functional groups in organic molecules. Journal of Cheminformatics , 9, 2017.

Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022.

Noel M O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geoffrey R Hutchison. Open babel: An open chemical toolbox. Journal of cheminformatics , 3(1):1-14, 2011.

Anthony K Rappé, Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. UFF, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical society , 114(25):10024-10035, 1992.

Oleg Trott and Arthur J. Olson. Autodock vina: Improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of Computational Chemistry , 31, 2009.

Andrew T McNutt, Paul G. Francoeur, Rishal Aggarwal, Tomohide Masuda, Rocco Meli, Matthew Ragoza, Jocelyn Sunseri, and David Ryan Koes. Gnina 1.0: molecular docking with deep learning. Journal of Cheminformatics , 13, 2021.

Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with QuickVina 2. Bioinformatics , 31(13):2214-2216, 02 2015.

Greg Landrum. Rdkit: Open-source cheminformatics software. 2016.

Richard Bickerton, Gaia Paolini, Jérémy Besnard, Sorel Muresan, and Andrew Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry , 4:90-8, 02 2012.

P Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics , 1:8, 06 2009.

Arup K. Ghose, Vellarkad N. Viswanadhan, and John J. Wendoloski. A knowledge-based approach in designing combinatorial or medicinal chemistry libraries for drug discovery. 1. a qualitative and quantitative characterization of known drug databases. Journal of Combinatorial Chemistry , 1(1):55-68, December 1998.

Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced drug delivery reviews , 23(1-3):3-25, 1997.

Daniel F. Veber, Stephen R. Johnson, Hung-Yuan Cheng, Brian R. Smith, Keith W. Ward, and Kenneth D. Kopple. Molecular properties that influence the oral bioavailability of drug candidates. Journal of Medicinal Chemistry , 45(12):2615-2623, May 2002.

A Proof of Proposition 1.

First, we here denote all atom's positions in the molecules as X M ∈ R N a × 3 , and in the protein as X M ∈ R N aa × 3 , and linker and functional group type as S M ∈ R ( N aa + N fg ) × (22+ M fg + M at ) . Note that one functional group may contain several atoms so that N aa + N fg < N a .

SE(3) group as a roto-translation group in R 3 , can be divided into two groups: SO(3) as the rotation group and T(3) as the translation group. For x ∈ R 3 , and g = r + v with g ∈ SE(3) , r ∈ SO(3) , v ∈ T(3) , T g ( x ) = T r + v ( x ) = T v ◦ T r ( x )

Lemma A1. If the The equivariance and invariance of the distribution in the reverse diffusion process are listed as

Then p ( X T M , S T M | X P , S P ) is SE(3) invariant.

Proof. Since T g ( M ) = { S , T g ( X M ) } , we can write the joint generative process as

Then, let's consider a single atom's position. We here denote each atom's x t M as

where x t C is the defined center atom's position in the functional group, x t R is the relative position of the atom in the local coordinate system centering at x t C , O t C is the rotation matrices of the local coordinate system with respect to the global system. Moreover, because the functional group is regarded as rigid bodies, x t R = x R is constant. To be specific, if x t M refers to linker's position, O t C = 0 .

Proposition A2. If each atom's relative positions in the local coordinate systems are fixed, and p ( x t -1 C |M t , P ) is SE(3)-equivariant and p ( O t -1 C |M t , P ) is SO(3)-equivariant and T(3)-invariant, such that p ( T g ( x t -1 C ) | T g ( M t , P ) ) = p ( x t -1 C |M t , P ) and p ( T r ( O t -1 C ) | T g ( M t , P ) ) = p ( O t -1 C |M t , P ) , where r ∈ SO(3) , v ∈ T(3) , r + v = g ∈ SE(3) , then p ( x t -1 M |M t , P ) is SO(3)-equivariant.

Proof. According to the convolution formula in probability theory, if w = u + v , then

By using the Eq. (16), we can write every single atom's position density function as

Since and

We can obtain that

Therefore, according to Eq. (18), (20), and (17)

Proof of Proposition 1. The sufficiency of SE(3)-invariance of p ( s t -1 |M t , P ) and p ( s T ) is given in Lemma A.1 , and the sufficiency of SE(3)-equivariance of p ( x t -1 |M t , P ) , and SO(3)-equivariance and T(3)-invariance of p ( O t -1 |M t , P ) is given in Proposition A.2 . Besides, it is easy to obtain that if p ( O T ) and p ( x T ) is SE(3)-invariant distribution, then p ( x T M ) will be invariant.

B Model Comparisons.

With Pocket2Mol and GraphBP. Pocket2Mol and GraphBP are all auto-regressive models, which violate the physical rules from the perspective of energy[15], while the diffusion models which consider the global interactions are a solution to the problem. Besides, to decide which atom will be added a bond with the next atom, Pocket2Mol needs a classifier to predict the focal atom, so that the training and the prediction are not end-to-end. Finally, D3FG is a functional-group- or fragment-based method, while these two models generate molecules at the atom level.

With DiffSBDD and TargetDiff. These two methods are diffusion-based, which considers the global interactions between atoms. In the atom type generation, DiffSBDD uses continuous latent spaces generation, while Targetdiff diffuses the atom types in a uniform distribution. These two models all generate molecules at the atom level, while D3FG generates at the fragment level, and besides the types and positions, the orientations of the functional groups are generated.

With FLAG. It is still an auto-regressive model and does not support end-to-end generation since the focal atom needs to be predicted. Besides, we defined our functional groups as rigid bodies with stable intra-structures, while the motifs in FLAG are 2D smiles, with the structures generated by RDKIT. The 3D structures of functional groups are obtained by the training set in D3FG, thus avoiding the problem of distribution shift of FLAG since the training/test motif substructures may not match the RDKIT's generation.

With DeepFrag. It is a model for fragment-based lead optimization, in which the protein and the parent is used as condition and the fragment type is predicted by the model. In this way, it is more like an elaboration task defined in D3FG. Here we point out several advantages of D3FG(EHot/Cold) over Deepfrag. First, Deepfrag just predicts the fragment type, without 3D structures, so that the problem of equivariance is not included, but D3FG generates 3D positions of the molecules. Second, although the type is the SE(3)-invariant variable, the model uses CNNs as the model backbone rather than EGNN, so the invariance can also not be preserved. Instead, D3FG assures physical symmetries by using EGNN. Finally, D3FG is a generative model with stochasticity, while Deepfrag only gives the probability of the fragment types, as a discriminative model.

C Method Details

Amino acid context encoding. Several geometric or type features are embedded to encode amino acids. For the geometric features including torsion angles/dihedrals, and pairwise distances, they are all roto-translational invariant, since the geometric features are all scalars obtained from relative coordinates. Besides, the local coordinates of atoms in a single amino acid are also invariant because it is always fixed in the local frame established by C α , C and N . For the type features including amino acid types, sequential relationships, and pair of amino acid types, the translational and rotational operation is unrelated to them. Thus, the encoded amino acid contexts are roto-translational invariant, leading to the invariance of all the follow-up embeddings.

Equivariant neural network for linkers. For the roto-translational equivariance of positions for single atoms, since O t j = I , Eq. 12 will be written as G ( M t , P ) [ j ] = MLP G ( h ′ j ) , unable to satisfy the equivariance. In this way, we revised it for single atoms by using the EGNN[44] stacked in the final layer for updating the positions, which reads

where we choose C j = ∥ x j ∥ 2 +1 .

D Data Preprocessing

Local frame establishment. In 3D Euclidian space, for a rigid body including more than mass points that are not co-linear, a local frame can be established. We first choose a center node (center point) A as the origin, and a second node B , leading to ⃗ AB as the direction of x-axis. Then, a third node C is selected. By Schmidt orthogonalization of ⃗ AC with respect to ⃗ AB , the direction of y-axis can be computed. And finally, the direction of z-axis is obtained by the cross-product of the unit vectors in the x direction and y direction. By this means, the local frame is established by the three nodes, and the other nodes' local coordinates can be determined in the local frame. Because the local frame requires at least three points to establish, the functional groups including only 2 atoms are divided into two linkers.

Functional group datasets. We give detailed information on the included functional groups, including 2D graph, 3D structure, time of occurrence in the CrossDocked2020, and the center node (node A ), node B , and node C of the functional group in Table. 10.

Note that in beneze, the symmetric structures lead the frame nodes to be any three consecutive points on the hexagon. Besides, for the functional groups of 'NS(=O)=O' and 'O=CNO', two stable conformations exist, so we in practice regard them as four different types.

E Experiment Details

Platform. We use a single NVIDIA A100(81920MiB) GPU for a trial. The codes are implemented in Python 3.9 mainly with Pytorch 1.12, and run on Ubuntu Linux.

Model Details. In the diffusion of orientation and position, we employ a cosine variance schedule for ¯ α t , which reads

where s = 0 . 01 . In the diffusion of atom type, β t is set as β t = t T . For the denoiser, the layer number is set as 6, and the embedding size is set as 256. The model is trained with Adam optimizer in 5000 epochs.

Functional group and Atom type analysis. We give a detailed analysis of functional groups and atom types, in Table. 5, 6 and 7.

Bond Angle and Dihedral analysis. Besides bond distance, a detailed analysis of bond angle and dihedral is shown in Table. 8 and 9. It gives the JSD between the reference and the generated molecules, and demonstrates that D3FG generates more realistic drug molecules in comparison with other baselines.

！

Smiles

2D graph

3D structures

A

B

C

T

c1ccccc1

CH:0

CH:2 CH:1

CH:3

CH:4 CH:5

c1ccccc1

1

0

2

131148

NC=O

NH2:0

CH:1

O:2

NC=O

1

0

2

49023

O=CO

O:0

CH:1

HO:2

O=CO

1

0

2

39863

c1ccncc1

CH:0

CH:2 CH:1

N:3

CH:4 CH:5

c1ccncc1

3

2

4

15115

c1ncc2nc[nH]c2n1

CH:0

N:1

CH:2

C:3

N:4

CH:5

NH:6

C:7

N:8

c1ncc2nc[nH]c2n1

7

3

6

11369

SH:1

O:2

O:3

1

0

2

10121

O=P(O)(O)O

O:0

P:1

OH:2

OH:3

HO:4

O=P(O)(O)O

1

0

2

7451

OCO

OH:0

CH2:1

HO:2

OCO

1

0

2

6405

c1cncnc1

CH:0

CH:1

CH:3

N:4

CH:5

c1cncnc1

3

2

4

5965

c1cn[nH]c1

CH:0

CH:1

NH:3

CH:4

c1cn[nH]c1

2

3

1

5404

口

中

！

口

！

Smiles

2D graph

3D structures

A

B

C

T

O=P(O)O

O:0

PH:1

OH:2

OH:3

O=P(O)O

0

1

center(2,3)

5271

c1ccc2ccccc2c1

CH:0

CH:1

CH:2

C:3

CH:4

CH:5

CH:6

CH:7

C:8

CH:9

c1ccc2ccccc2c1

3

2

4

4742

c1ccsc1

CH:0

CH:1

CH:2

S:3

CH:4

c1ccsc1

3

2

4

4334

N=CN

NH:0

CH:1

H2N:2

N=CN

1

0

2

4315

NC(N)=O

NH2:0

C:1

H2N:2

O:3

NC(N)=O

2

1

3

4167

O=c1cc[nH]c(=O)[nH]1

O:0

C:1

CH:2

CH:3

NH:4

C:5

O:6

NH:7

O=c1cc[nH]c(=O)[nH]1

7

1

5

4145

c1ccc2ncccc2c1

CH:0

CH:1

CH:2

C:3

N:4

CH:5

CH:6

CH:7

C:8

CH:9

c1ccc2ncccc2c1

3

2

4

3519

c1cscn1

CH:0

CH:1

S:2

CH:3

N:4

c1cscn1

2

3

1

3466

c1ccc2[nH]cnc2c1

CH:0

CH:1

CH:2

C:3

NH:4

CH:5

N:6

C:7

CH:8

c1ccc2[nH]cnc2c1

5

4

6

3462

c1c[nH]cn1

CH:0

CH:1

NH:2

CH:3

N:4

c1c[nH]cn1

3

2

4

2964

O=[N+][O-]

O:0

N+ :1

-O:2

O=[N+][O-]

1

0

2

2702

人

8

口

a

8

口

B

口

Smiles

2D graph

3D structures

A

B

C

T

O=CNO

O:0

CH:1

NH:2

HO:3

O=CNO

1

0

2

2477

NC(=O)O

NH2:0

C:1

O:2

OH:3

NC(=O)O

1

0

2

2438

O=S=O

O:0

S:1

O:2

O=S=O

1

0

2

2375

c1ccc2[nH]ccc2c1

CH:0

CH:1

CH:2

C:3

NH:4

CH:5

CH:6

C:7

CH:8

c1ccc2[nH]ccc2c1

3

4

2

2301

5

10

15

20

25

Repository Size

7.6

7.4

7.2

7.0

6.8

6.6

6.4

Vina score

Vina Score

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

55.0%

60.0%

Vina

Affinity

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, a functional-group-based diffusion model called D3FG is proposed to generate molecules in 3D for target protein binding. Two generation schemes including joint and two-stage generation schemes are formulated.\n\nSTRENGTHS:\n1. The paper is well-written and easy to follow.\n2. It is the first functional-group-based diffusion model for structure-based drug design.\n\nWEAKNESSES:\n1.  The performance improvement over previous methods is limited. For example, in table.2, TargetDiff achieves the lowest JS divergence on most bond distances. In Table. 4, TargetDiff achieves the best performance on the vina score. In Figure. 3 the D3FG does not show an advantage over other baselines with respect to atom type distribution.\n2. The evaluations are not comprehensive. For example, besides divergence between bond distances, bond angles and dihedral angles should be evaluated. The influence of the size of the functional group set is not explored. \n3. The technical contribution is limited. The application of diffusion models for drug design is not new. It seems the required techniques are proposed in previous works including TargetDiff [7], DiffSBDD [8], and DiffAB [52]. \n4. Some related works are not discussed. For example, some previous methods also leverage functional groups or motifs for molecule design: JT-VAE [1], PS-VAE [2], and FLAG [3].\n\n[1] Jin et al., Junction tree variational autoencoder for molecular graph generation, ICML 2018\n[2] Kong et al., Molecule generation by principal subgraph mining and assembling, NeurIPS 2022\n[3] Zhang et al., Molecule generation for target protein binding with structural motifs, ICLR 2023\n\nQUESTIONS:\n1. It seems that the reported scores in Table. 4 are different from the original papers of baselines. Could the authors explain that?\n\nLIMITATIONS:\nYes, the authors have adequately discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPocket-specific molecule generation has received considerable attention in recent years, and the authors propose a functional-group-based diffusion model to address this task. The model considers the generation of complete molecules as the assembly of functional groups (fragments) and atoms that connect these functional groups. They employ a diffusion-based generative scheme to determine the coordinates, orientation, and types of these fragments. The experiments demonstrate the satisfactory generation performance of the model and its potential applications in molecule elaboration.\n\nSTRENGTHS:\n1. The paper is well-written, and the narrative is clear and concise.\n2. The authors address many important details, including the determination of functional groups, handling chirality, and generating heterogeneous graphs.\n3. Overall, while the idea of decomposing molecules is not novel, the authors provide a robust model and conduct experiments to validate its superiority.\n\nWEAKNESSES:\n1. In the Method section, it would be beneficial to provide an overview of the entire method before delving into the specifics of each part.\n2. The aromatic elements C:C and C:N in Table 2 could be written in lowercase to maintain consistency with convention.\n3. It would be beneficial to include more visualizations (2D/3D) of generated molecules.\n\nQUESTIONS:\n1. In the Introduction, it is mentioned that protein amino acids are represented as functional groups and linking atoms (Line 42). However, in the Method section, the amino acid space I_aa is preserved (Line 60). Why is the amino acid space still necessary if any node can be considered either a functional group or atom?\n2. The functional groups are treated as rigid fragments, and their orientations are predicted. But what if the functional groups have internal rotatable bonds and cannot be regarded as rigid (e.g., O=CNO)?  Additionally, how is the symmetry of functional groups considered when defining orientation? For example, a benzene ring can have multiple orientations resulting in the same arrangement of the six atoms. Which orientation is chosen during training?\n3. In the experiments, why did you use EFGs to segment molecules instead of BRICS composition, as done in many previous fragment-based works? Why did you choose 25 functional groups? How does this choice cover the total number of atoms in a molecule, i.e., the ratio of atoms in functional groups to the total number of atoms for a molecule? Did you consider other sizes for the set of functional groups?\n4. Does the node number include the protein in Line 220?\n5. For the molecule elaboration experiment, did you retrain the model specifically for this task, or did you use the same model as in the molecule generation experiment?\n6. In Line 288, what does the word \"them\" refer to? It seems ambiguous.\n7. Why does D3FG(EHot) generate molecules with higher affinities? In my opinion, since functional groups with high binding contributions were removed, it is likely that the generated functional groups have weaker binding, resulting in new molecules with weaker affinities. On the other hand, in the D3FG(ECold) experiments, a functional group with higher binding contribution is more likely to be generated, replacing the weaker functional group and leading to higher affinities. Why do the experimental results not align with this explanation?\n\nLIMITATIONS:\nThe authors have addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a so-called functional-group-based diffusion generative model, namely D3FG, to generate molecules with realistic substructures conditioned on the protein binding sites. D3FG represents the protein-ligand docking system as a fragment-based system. The molecule fragments are molecule substructures and the protein fragments are amino acids. Thus, the original protein-ligand docking system is reduced to a coarsen graph with molecule substructures and amino acids as nodes. The molecule substructures are connected with heavy atoms as linkers. Thus, the denoising and diffusion processes are defined with molecule substructure translation probabilities. Although this idea is interesting, the reviewer is concerned about the paper presentation, literature reviewer, and marginal performance improvement.\n\nSTRENGTHS:\nRepresenting the protein-molecule docking system as the coarsened graph reduces the computation expenses of the diffusion generative models. Recently, this scheme has drawn much attention and there are some related work has already shown the efficiency of this scheme. Moreover, molecule elaboration is also related to ligand optimization, which aim to optimize the ligand efficiency by modifying it structure.\n\nWEAKNESSES:\n1. Limited Novelty and Insufficient literature review. This paper represents the fragment-based method for ligand generation and claims no prior work employs the fragment-based generation scheme. However, there are already some methods that employ fragment-based generation. For example, Deepfrag [1], FLAG [2], and SQUID [3] are all fragment-based generative methods for target-aware molecule design. It is necessary to discuss the difference and highlight the contributions. \n\n2. The paper presentation needs to be improved. For example, what is the intuition of the definition of absorbing state in Line 105? What is the connection between the BERT-style objective and the denoising objective in Eq.4? What are the different levels of nodes in Section 4?\n\n3. Definition of \"functional group\". The functional group often refers to the molecule substructures that contribute most to the chemical properties. However, as shown in Section 5.1, the functional group in this paper refers to the most frequent substructures. Mining the most frequent substructures for molecule generation is not new and has been employed in [2,4].\n\n4. Marginal empirical improvement. Moreover, there is only marginal empirical improvement over the baseline methods as shown in Table 4.\n\n5. Unfair comparison. Since the proposed method already employs the most frequent substructures for model training, it is unfair to choose the \"'Ratio’ of the top ten functional groups with the highest frequency\" as an evaluation metric in Table 1.\n\n\n[1]. Deepfrag: a deep convolutional neural network for fragment-based lead optimization. Chemical Science.\n\n[2]. Molecule generation for target protein binding with structural motif. ICLR 2023.\n\n[3]. Equivariant shape-conditioned generation of 3d molecules for ligand-based drug design. ICLR 2023.\n\n[4]. Molecule Generation by Principal Subgraph Mining and Assembling. NeurIPS 2022.\n\nQUESTIONS:\nThe Questions are listed in the Weaknesses Section above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to generate 3D molecules using functional groups and linker atoms in a diffusion model. Using functional groups as building blocks help the model to generate realistic local structures. In the proposed diffusion model, the atom/functional group type, coordinates and orientation are predicted.  In addition, the authors experimented with two strategies, joint and two-stage, to generate heterogenous graph, which contains functional groups and linker atoms. The performance indicates the proposed method achieves SOTA performance in structure-based drug design task. \n\nThis paper also proposed a new molecular elaboration task and curated a dataset for this task.\n\nSTRENGTHS:\n1. This paper implement functional groups in diffusion-based structure-based drug design model for the first time. \n2. The performance indicates using functional groups as building blocks can improve QED and SA, which have been a challenge for previous diffusion models. \n3. This paper proposes a new task, molecule elaboration, for structure-based drug design and provides a dataset for it.\n\nWEAKNESSES:\nThe paper proposed a new molecule elaboration task, and some existing baselines (e.g. Pocket2Mol[1], 3DSBDD[2], TargetDiff[3] and DiffSBDD[4]) and can be easily adapted for this task. Currently, there is no performance comparison for molecule elaboration task, and it is recommended for the authors to evaluate their model with existing baselines which can be used for this task. \n\n\n[1] Peng, Xingang, et al. \"Pocket2mol: Efficient molecular sampling based on 3d protein pockets.\" ICML 2022.\n\n[2]Luo, Shitong, et al. \"A 3D generative model for structure-based drug design.\" NeurIPS 2021.\n\n[3] Guan, Jiaqi, et al. \"3d equivariant diffusion for target-aware molecule generation and affinity prediction.\" ICLR 2023. \n\n[4]Schneuing, Arne, et al. \"Structure-based drug design with equivariant diffusion models.\" arXiv preprint arXiv:2210.13695 (2022).\n\nQUESTIONS:\n1. How many percentage of the generated molecules are complete (all the generated atoms are connected into one molecule)?\n2. In molecular elaboration task, is the remaining fragments included in D3FG as a condition for elaboration? If not, can the proposed model guarantee that the newly generate fragment is suitable to be connected with the remaining fragments?\n3.  Why does the two-stage approach consistently outperform the joint training one? Is there any explanation?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel method for generating 3D molecules that bind to specific protein pockets based on a functional-group-based diffusion model (D3FG). The model decomposes molecules into functional groups and linkers and generates their types, positions, and orientations gradually through a denoising process. The model uses equivariant graph neural networks to parameterize the denoisers and ensure the roto-translational invariance of the molecule distribution. The paper also introduces a new task of molecule elaboration, which aims to modify existing molecules based on the fragment hotspot maps of the protein pockets. The paper evaluates the model on the CrossDocked2020 dataset and shows that it can generate molecules with realistic structures, competitive binding affinity, and good drug properties. The paper also demonstrates that the model can perform molecule elaboration and generate molecules with higher affinity than the reference molecules. The paper claims that D3FG is a novel and effective method for structure-based drug design that leverages the pharmacological information of functional groups.\n\nSTRENGTHS:\nThe paper proposes a functional-group-based diffusion model for pocket-specific molecule generation and elaboration called D3FG. The method decomposes molecules into two categories of components: functional groups defined as rigid bodies and linkers as mass points. The two kinds of components can form complicated fragments that enhance ligand-protein interactions. In the experiments, the authors claim the method can generate molecules with more realistic 3D structures, competitive affinities toward the protein targets, and better drug properties. The paper is original in its approach to generating molecules given the pockets’ structures of target proteins and its use of functional groups as basic components instead of atoms. The paper is clear in its description of the method and its results. \n\nThe paper is well-written and clearly describes the proposed method, its implementation details, and its evaluation results. The authors provide sufficient background information on related work and explain how their method differs from prior methods. The authors also provide detailed explanations of the model's components, such as the functional group decomposition, equivariant graph neural networks, and molecule elaboration. The authors use appropriate visualizations to illustrate their method's outputs and compare them with reference molecules. \n\nThe paper's originality lies in its novel approach to generating 3D molecules that can bind to specific protein pockets by leveraging functional groups' pharmacological information. The authors show their method can generate molecules with realistic structures, competitive binding affinity, and good drug properties. The paper's significance lies in its potential to improve structure-based drug design by enabling the generation of novel molecules that can bind to specific protein pockets with high affinity.\n\nWEAKNESSES:\nThe paper's main weakness is that it does not provide a detailed comparison with prior methods for generating 3D molecules that can bind to specific protein pockets. The authors briefly mention some related work, but they do not provide a comprehensive comparison of their method with prior methods in terms of performance, efficiency, and scalability. The authors also do not provide a detailed analysis of the limitations of their method, such as the types of molecules it may not be able to generate or the types of protein pockets that it may not be able to bind to. \n\nAnother weakness of the paper is that it does not provide a detailed analysis of the interpretability and explainability of its method. The authors briefly mention some visualizations and explanations of their method's outputs. Still, they do not provide a systematic analysis of how their method's components contribute to its performance and how they can be interpreted in terms of pharmacological properties. \n\nTo improve the paper, the authors could perform a more comprehensive comparison with prior methods for generating 3D molecules that can bind to specific protein pockets, including both quantitative and qualitative analyses. The authors could also perform a more detailed analysis of the limitations of their method and how they can be addressed in future work. Finally, the authors could perform a more systematic analysis of the interpretability and explainability of their method, including sensitivity analyses, feature importance analyses, and pharmacophore analyses.\n\nQUESTIONS:\n- How did the authors select the 25 functional groups used for their method? How did they ensure these functional groups cover a diverse and representative range of pharmacological properties and structures? How sensitive is their method to the choice of functional groups?\n- How did the authors evaluate the quality and diversity of the generated molecules? Did they use any metrics or criteria to measure the novelty, validity, and diversity of the generated molecules? How did they compare their method with prior methods regarding these metrics or criteria?\n- How did the authors handle the cases where the generated molecules violate chemical or physical constraints, such as bond angles, bond lengths, steric clashes, or chirality? How did they ensure that the generated molecules were chemically feasible and stable?\n- How did the authors handle the cases where the target protein pockets have multiple binding sites or modes? How did they ensure that their method can generate molecules that can bind to different sites or modes of the same protein pocket?\n- How scalable is their method to larger, more complex protein pockets and molecules? What are the computational and memory requirements of their method? How does their method compare with prior methods regarding efficiency and scalability?\n\nLIMITATIONS:\nThe authors have not adequately addressed the limitations and potential negative societal impact of their work. The authors only briefly mention some limitations of their method in the conclusion section, but they do not provide a detailed discussion of how these limitations affect their results and how they can be overcome in future work. The authors also do not discuss any potential negative societal impact of their work, such as the ethical, legal, or environmental implications of generating novel molecules that can bind to specific protein pockets.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-12"}