LinkerNet: Fragment Poses and Linker Co-Design with 3D Equivariant Diffusion
Authors: Jiaqi Guan
Xingang Peng
Peiqi Jiang
Yunan Luo
Jian Peng
Jianzhu Ma
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Linker design
generative models

Abstract: Targeted protein degradation techniques, such as PROteolysis TArgeting Chimeras (PROTACs), have emerged as powerful tools for selectively removing disease-causing proteins. One challenging problem in this field is designing a linker to connect different molecular fragments to form a stable drug-candidate molecule. Existing models for linker design assume that the relative positions of the fragments are known, which may not be the case in real scenarios. In this work, we address a more general problem where the poses of the fragments are *unknown* in 3D space. We develop a 3D equivariant diffusion model that jointly learns the generative process of both fragment poses and the 3D structure of the linker. By viewing fragments as rigid bodies, we design a fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics. Empirical studies on ZINC and PROTAC-DB datasets demonstrate that our model can generate chemically valid, synthetically-accessible,  and low-energy molecules under both unconstrained and constrained generation settings.

LinkerNet: Fragment Poses and Linker Co-Design with 3D Equivariant Diffusion
Jiaqi Guan
Xingang Peng Peking University xingang pwng@gmail com
University of Illinois Urbana Urbana-Chaman Champaign jiaqi0illinois, edu
Peiqi Jiang
Yunan Luo Georgia Institute of Technology yunanlgatech. edu
Tsinghua University
jpq200mails ts tsinghua. edu. en
Jian Peng
Jianzhu Ma
University of Illinois Urbana-Chann Chhama-hh jianpeng@ill inois.edei
Tsinghua University
maj majianzhuutsinghua edu. cn
Abstract
Targeted protein degradation techniques, such as PROteolysis TArgeting Chimeras (PROTACs), have emerged powerful tools for selectively removing disease- causing proteins. One challenging problem in this field designing a linker connect different molecular fragments to form a stable drug-candidate molecule. Existing models for linker design assume that the relative positions of the fragments are known, which may not be the case in real scenarios. In this work, we address a more general problem where the poses of the fragments are unknown in 3D space. We develop 3D equivariant diffusion model that jointly learns the generative process of both fragment poses and the 3D structure of the linker. By viewing fragments rigid bodies, we design fragment pose prediction module inspired by the Newton-Euler Euler equations in rigid body mechanics. Empirical studies on ZINC and PROTAC-DD DB datasets demonstrate that our model can generate chemically valid, puntheticall--acessst and constrained generation settings.
and lnw-energy molecules under both unconstrained
Introduction
Targeted Protein Degradation (TPD) techniques 27 71 including PROteolysis TArgeting Chimeras (PROTACs) [35), molecular glues [40,, etc., are emerging powerful tools for selectively removing discase-causing proteins. These techniques typically involve multiple molecular fragments connected by a linker, with each fragment binding a specific protein. For instance, a PROTAC consists of three components: ligand (warhead) that targets the protein of interest, another ligand that recruits E3 ubiquitin ligase, and a linker that connects two ligands [38] 9]. PROTACs induce the ubiquitination of target proteins, which is fundamental biological process where small proteins called ubiquitins are attached to target proteins, marking them for degradation by the proteasome. Unlike traditional small molecule inhibitors that temporarily inhibit protein function by binding their active sites, PROTAC techniques result complete protein elimination and offer several advantages, including increased selectivity, reduced off- off-target effects, and the potential to target previously undruggable proteins (27]. However, designing effective PROTACs remains significant challenge, particularly in optimizing the linker, which is crucial in maintaining the conformational stability and other important biological properties of the entire molecule.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(a) PROTAC-DB 146 (BRD4, CRBN
PROTAC-DB (BRD4, CRBN)
Figure 1: An example showing that the fragment poses fixed in PROTAC design. The above sub-figures show two PROTACs, both which have one fragment binding with the BRD4 target and the other fragment binding with the CRBN E3 ligase. The linkers differ with 'COC' motif.
(a) Fixed fragment design (2D) (b) Fixed fragment design (3D)
(c) Free fragment design (3D)
3D fragment graphs
3D fragment graphs
3D fragment graphs
3D linker graph fragment poses
2D linker graph
3D linker graph
Figure 2: An overview of different linker design settings. (a)/(b): Previous work focuses on 2D/3D linker design with fixed fragment poses. (c): We focus on co-designing fragment poses and linker.
In recent years, computational approaches, particularly deep learning methods, have been employed for linker design to accelerate the drug discovery process 47 22 23
20 21] Many of these
approaches utilize 3D structural information enhance their performance. For example, Delinker [22) incorporates the distance and angle between anchor atoms as additional structural information to generate the 2D linker given the fragment molecular graphs. Going a step further, 3DLinker [20] and DiffLinker (2I) operate directly in 3D space to process fragments and generate linkers with conditional VAE and diffusion model, respectively. These models commonly assume that the relative position between fragments is fixed. This assumption is reasonable in traditional fragment- based drug design, where fragments are designed bind the same protein pocket and the binding poses are largely deterministic [12]. However, in scenarios involving two proteins (drug target and E3 ligase in the PROTAC case), relative position between fragments may not be readily available due to the flexibility of mntein--roteei binding pose, as shown in Figure As result, it becomes necessary to adjust the fragment poses dynamically during the linker design process. One potential solution is to randomly sample multiple fragment poses and design a 3D linker for each pose. The drawback of this approach is that limits the design space for the linker by immobilizing the relative positions of the two molecular fragments. It may not be possible to find stable linker that can connect the two molecular fragments with fixed positions due to the narrow range in which a bond can form. To the best of our knowledge, there is currently existing computational approach that addresses this challenging 3D linker design problem in the absence fragment relative positions.
In this work, we first address the problem of co-designing fragment poses and the 3D linker. We represent each fragment as rigid body and its pose as the position its center and the rotation. The linker is represented 3D graph, consisting of atom positions, atom types, and bond types. To

---

tackle this co-designing problem, we propose LinkerNet, a 3D equivariant diffusion model, leveraging recent advancements equivariant diffusion models [9] and diffusion models on Riemann manifolds
[]] Our proposed model can jointly learn the generative process of both fragment poses and the linker, which enables the model to find stable linker and connect the fragments to form a low-energy conformation of the whole molecule. Moreover, we design fragment pose prediction module inspired by the Newton-Eule equations in rigid body mechanics. This module employs neural networks to predict atomic forces, which are then used update the fragment center positions and rotations through the aggregation of neural forces and torques. We introduce two guidance terms that restrict fragment distances and potential anchors to incorporate real-worke constraints into model. To evaluate our method, we perform comprehensive experiments on ZINC [43] and PROTAC-DE 45 datasets and demonstrate that our model can generate chemically valid, synthetically-accsssee and low-energy molecules under both unconstrained and constrained generation settings.
To summarize, our main contributions are:
Our work presents the first computational model that abandons the unrealistic assumptions in PROTAC drug design.
We propose 3D equivariant diffusion model which enables the co-design of fragment poses and the 3D linker structure in a unified framework.
We develop an effective fragment pose prediction module inspired by the Newton-Fuled equations in rigid body mechanics, allowing for the accurate adjustment of fragment center positions and rotations.
We conduct comprehensive experiments on ZINC and PROTAC-DB datasets, showcasing the superiority of our proposed model over other baseline methods in both unconstrained and constrained generation settings.
2
Related Work
Molecular Linker Design Molecular linker design is critical step in the rational compound design. SyntaLinker [47] operates on the SMILES representation of molecules and formulates the linker design as sentence completion problem. The lack of 3D structural information and the drawbacks of the SMILES representation itself limits the performance of this method. DeLinker [2) and Develop [33 overcome this limitation by operating on graphs and utilizing the distance and angle between anchor atoms as the additional structural information. However, only limited structural information is used and the generation is still in 2D space. More recently, 3DLinker (20), DiffLLinerr2111 are proposed to directly generate linkers in 3D space with conditional VAE and diffusion models, respectively. All of these models assume the fragment poses known. However, this is not always the case, especially in the emerging Targeted Protein Degradation (TPD) techniques. Instead, our model focuses on a more general linker design problem where the fragment poses are unknown.
PROTAC Linker Design PROteolysis TArgeting Chimeras (PROTAC) is promising technique with many advantages over traditional small molecule inhibitors. The first proof of concept study of PROTAC was proposed in [38). Most PROTAC linker design strategies rely on empirical optimization
of linker composition, which consists of only a few main chemical motif's [44] 33). Currently, there
are no generally accepted rules for de novo PROTAC linker design. 48 uses deep reinforcement
learning to facilitate rational PROTAC design, but it is still generating SMILES representation instead of molecules in 3D space.
Diffusion Generative Models
Diffusion generative models [4] 42 1 learn to denoise samples
from prior noise distribution and have achieved remarkable progress in generating images [36] 37]. text [8] 2) etc. Recently, diffusion models are also applied in molecular data by considering the rotation- transllaion equivariance, such as molecular conformation generation [46), 3D molecule generation [19 and structure- baaeed drug design [5
39 29]. In addition, diffusion models have also been extended to Riemann manifolds [8] 28 and many applications molecular data have emerged, including conformation generation 25 molecular docking (6), antibody design [3] and protein-ligand binding affinity prediction 24. Leveraging these advances in equivariance diffusion models and diffusion models in Riemann manifolds, propose diffusion model for the fragment
3

---

R',p',gtt -egirre
x e00)
x(0,, (00,,00 els
linear softmax
V-m1
9
b
3D linker graph
R p
fragment poses
Pose Prediction
Equivariant Layer
J
Σ (#t
AVV.
Σ
1. force
aft1
(a)
o))o.(h (N. "h .i Sel.ee e .t) 10-Imii Invier
pc) torque
Figure 3: Overview of one denoising step of LinkerNet. An equivariant GNN is applied update linker atom positions æ, atom types v and bond types b. The updated embedding and positions are then utilized by pose prediction module to predict neural forces and torques Te further update fragment poses R and p.
poses and linker co-design task. We inject task-specific network and sampling designs, including physics-inspired fragment poses prediction module and the constrained linker sampling approach.
3 Fragment Poses and Linker Co-Design
In this section, we present LinkerNet, which co-designs fragment poses and the linker with 3D equivariant diffusion. We first define notations and this problem formally in Sec. 3.1 Then, decompose the joint distribution as the product of positions, rotations and atom / bond types in Sec. 3.2 and show how the diffusion processes constructed for them. In Sec. [33] we illustrate the equivariant network and physics-inspired prediction module for denoising fragment poses and the linker. Finally, Sec. 3.4 we describe how our model can be applied with guided sampling in the constrained generation setting.
3.1 Notations and Problem Definition
In our formulation, each molccular fragment represented as a 3D graph 9F = {VF, xF.. We denote the number of atom features, the number of bond types and the number of fragment atoms as Na. .N, and NF, respectively, and then V E N represents one-hot atom types (including elements and atom charges), by IR NexxNex N represents one-hot bond types (the absence of bond treated as a special bond type), and & E RaN is the atom coordinates in the local coordinate system (i.e., conformation). Since assume fragments are rigid, Xp is unchanged in setting. We use PCA to construct the local coordinate system [14] for robustness. However, we will see the equivariance of model is independent of the choice of the local coordinate system (Sec. 3.3) The global pose of each fragment is determined by a rotation transformation R E SO(3) and a translation p. The linker is represented
transformation p T(3) ~ R33 i.e. XF R&p p
of atom types, bond types and atom positions Gr {VL, by, x where v E IR- NR-NxXM, XNa denotes linker atom types, bt IR Nv, XN denotes linker bond types and x R denotes linker atom coordinates in the global coordinate system. Here, N = Nr + NF. + NF, is the total number of atoms of two fragments and the linker.
Given two fragments (F,) whose global poses are unknown, our goal is to design the linker GL and recover the global poses of two fragments (R, and (R2, P2) connect fragments with the linker to form a conformationally stable molecule GM- Specifically, denoting the fragment rotations and translations R and p separately, aim to learn the distribution po (R,, p. Gr. Gr, Gry)) with a neural network parameterized by 0.
3.2 Diffusion Processes
diffusion probabilistic model involves two Markov chains: a forward diffusion process and a reverse generative process. The diffusion process gradually injects noise into data, and the generative process

---

learns to recover the data distribution from the noise distribution. According to the different types of variables, the joint distribution (R.R, p. Grlgr, GF,) can be further decomposed as product of fragment/ linker positions (x p). fragment rotations (R) and linker atom / bond types (vr. by).
Next, we will describe how these diffusion processes are constructed in detail.
Denote linker positions X or fragment translations p as a random variable X. The diffusion on x involves standard Gaussian diffusion kernels and has been well-studied in [7] At each time step t, small Gaussian noise is added according to a Markov chain with fixed variance schedules . BT:
Diffusion on Positions
((x:|x-1)) = N (xe: V B,x;- 8r1)
(1)
Under this formulation, we can efficiently draw samples from the noisy data distribution g(xe) xt) and compute the posterior distribution g 1 Xo) in closed-form:
q( x xo) NV vā,xo (1 a.),,, g ((x, xo)
xo), Bel),
(2)
where a₁ 3e. a IT 02 and (X,, Var-T
I-û1 X +
var(1-
1-81 X1. B3
1-
Be.
1-
Diffusion on Fragment Rotations The diffusion kernel on fragment rotation R is IGSo(3) (ce, e²), i.e, the isotropic Gaussian on SO(3) [34] 28) parameterized by a mean rotation f and scalar variance e2 The standard ZGso(3) (I, ed) can be sampled in axis-angle form, with uniformly sampled axes
@ ~ so(3) and rotation angle w E [0, π] with density
(ww - COS w
Σ (21 1)e 1(1+1) sin + 1/2)w)
sin(w/2)
To sample from IGso (Ja, 2)), we can first sample a rotation e = we and apply it to Ja to obtain the desired sample. Similar to the Euclidean diffusion process, we can also draw samples and compute the posterior distribution at any time step in closed-form:
qRR, [Ro) -I900(3) (N(var, Ro), - ait) U(R-II/R, Ro) I9s0(s) (R, Ro), Bc), (4)
where i.,(R,, Ro) 2
-x /ā--1 Ro) + (a((1- Rs) and X(7, R) exp(7 log(R)
I-a:
denotes the rotation scaling operation, which scales rotation matrices R by converting them to values in 63)3), multiplying by a scalar 7/ and converting them back 50(3) (5).
Diffusion on Atom and Bond Types Following [18, we use categorical distributions to model discrete linker atom types vy and bond types by. Take atom types as an example (same for bond types), a uniform noise across all K categories is added according to a Markov chain during the
diffusion process:
q(vilve-1) C(v.l(1 - + B,KK).
(5)
Similarly, we can compute g(v.lvo) and q(v.-1lve vo) closed-forms:
q(velvo) = C(v.lāvvo + az)/K); q(v.-1/v,, vo) C(v.-1lo vo),
(6)
where (V,, vo) = c² ck/Ek of and c* (v., vo)
3.3 Equivariant and Physics-Inspired Neural Network
The likelihood pa (R. p. G219r, Gr.) should be invariant to the global (E(3)-transformmtioo which can be achieved by composing an invariant initial distribution and an equivariant transition [46] 19].
Thus, we define the distribution on the subspace Σ P: 0, i.e., the center of fragment positions is zero. It is also consistent with the truth that the linker is around the center of fragments, and thus we can set the prior distribution of x as a standard normal distribution. For the equivariant transition, we model the atomic interaction with 3D Equivariant GNN de::
(7) To obtain GM, from [xr, Vt, b,, R2, p.] and (Gr, 9r,), ). we perform local-to-glon globba coordinate trans- formation first and compose the atoms together, i.e., Xr,; & + p. and xas, = {xr,s x Next, we will claborate on how the linker and fragment poses are denoised.
( VL.o. by. Ro. Pol = de t) (o((x,,,, VL, PL R2, Pal, SF, Gr.,).
5

---

Equivariant Linker Denoising Process 6g is a L-layer 3D equivariant GNN. The initial atom and bond embedding hy and eù are obtained by two embedding layers that encode the atom and bond information. At the 1-th layer, the atom embedding h;, bond embedding eij and linker atom positions a; are updated as follows:
Bij- = da(elj, || - ar/1l)
(8)
h 1+1 = ht +
Σ
(n nt eig,
(9)
jEVir
Cij
1+1 = cij +
Σ
dn h4 Eki, +
E
@n(h', h4, tyk,
(10)
kEV
KEVW 10}
2 o +
Σ
- = (1 4+1
Gij , .t) 1 linker
1+1
(11)
where Vas is the set of all atoms in the molecule and linker the linker atom mask. The final atom embedding hf and bond embedding efy will be fed into two multi-layer perceptrons and softmax functions to obtain [V2.,, b2.0]- In addition, they will also be used to predict the fragment poses [R. p) with physics-inspiree prediction module.
Moysics-Inspired Fragment Pose Prediction A straightforward way to predict the global pose is by predicting an invariant pose change (R,+0. Pt-s) in the local coordinate system and applying it the current pose (Re, Pt) in the global coordinate system, i.e. (R,, Pt) o (R, Pr-vo) R, +0. Pt + -- This trick has been commonly applied in protein structure prediction 26 31. By applying this trick in our setting, it will be applying the invariant pose change prediction
from hi, Ciy to the current noisy rotation R2 and translation p: to obtain the denoised ones Rg and
Po i.e.:
Ro R. (be,, eiy)
Po = P: + Re (hi, hig eij)
(12)
However, we argue that the invariant pose update limits the model's capacity since it is regardless of the geometric information of the system in the prediction phase. Considering we treat fragments as rigid 3D graphs whose local coordinates will not be changed, it is natural to take inspiration from rigid body mechanics to predict their poses.
The Newton-Euler equations describe a rigid body's combined translational and rotational dynamics. In the Center of Mass (CoM) frame, it can be expressed as the following matrix form:
- -
F
mT
1.
/d//dt /tt
-
O I.)
/dw/dt)
1ow
(13)
where and T are the total force and torque acting on CoM, and are the velocity of CoM and the angular velocity around CoM, m and I, is the mass and inertia matrix of the rigid body, which constant for a given rigid body.
In our fragment pose prediction module, the outputs of the neural network act as the forces f. on each fragment atom with which can compute the total force F and torque for each fragment:
fi = Σ (a: Z (hi, hj, Cigl [| - æz5l)
(14)
JKVr.
Σ fo, Te
Σ (a. - Pc) x fi
(15)
iEVF.
NEVE.
where 2, corresponding two fragments.
We assume the system stationary at each discrete time step, i.e. w and e 0. Thus, the Newton-Euler equations 13 can be simplified as I dw and F de For short enough di
time period At. we have the velocity and angular velocity of the fragment as w I. ' and FAt. Assuming each atom in the fragment has the unit mass and absorbing the time period At into F. and Te, the fragment pose will be updated as follows:
po.0 pt +
F.
Re (L
(16)

---

where Fr. denotes the number of atoms in F. and R denotes the operation of converting a vector in 63(3) to a rotation matrix in SO(3) (See Appendix for details).
can be seen that the predicted force and torque take advantage of the geometric information and are equivariant to the global rigid transformation. Moreover, we also show that the predicted fragment poses are equivariant to the global rigid transformation and are independent of the choices of local coordinate systems. The final training loss is the weighted sum of MSE loss of linker atom positions and fragment center positions, a discrepancy loss of rotation matrix, and KL-divergence of linker atom types and bond types. Please see the proof, more training and modeling details, and the complete training/sampling algorithm in Appendix.
3.4 Constrained Generation with Guided Sampling
The LinkerNet we introduced so far is applicable in the case where the fragment poses are fully unknown. However, in real scenarios, we have some prior knowledge, such as the fragment distance should be in a reasonable range, the anchors (the atom on the fragment to connect with the linker) should be selected from a candidate set, etc. To incorporate these constraints, we leverage the idea of classifier guidance [0] to perform guided sampling. In our formulated problem, for condition y, the diffusion score can be modified as:
V log p(GM, lv) = V log (GG.) + V logp(vllM.).
(17)
For the fragment distance constraint, we assume the expected distance range is [dmin: maax] then we have the following guidance term:
- -Vp max (pp p2ll - dmmax: 0) + max(dmin - -|pi P2l1. 0).
(18)
To generate a complete and valid molecule, at least one atom from the candidate anchor set should form bond with one linker atom, and the atoms outside the anchor set should not form bonds with any linker atoms. These principles can be formulated as the following anchor proximity guidance:
VR max(da Tmax: 0) + max (rmin - da. 0) + max (Fmxx dnay 0).
(19)
where A is the candidate anchor set, de = min;e JEA |>>; - x; and dna mingeVe JEVo, \A ||ær; - æz ||- Tmin and Tmax denote the minimum and maximum of bond length, which are set to 1.2Å and 1.9Å in practice. Besides the soft constraint on anchor proximity, we can also set hard constraint by applying a bond mask during the sampling phase, i.c. we adopt by E N. | instead of F NNNxNNN,
4 Experiments
4.1
Setup
We mainly conduct experiments in two settings: unconstrained generation and constrained generation. In the unconstrained generation setting, only 3D fragment graphs Gr, Gr,9r,, are known, and the goal validate whether the model co-design linker and fragment poses to generate molecules GM with low-energy and other desired properties. In the constrained generation setting, the candidate anchor set A on each fragment is also known besides Gr,- Gr,,, and set fragment center distance constraint. The goal is to validate whether the model can perform well in a more realistic scenario. We describe the general setting about data, baselines, and evaluation metrics follows and the detailed task-specific settings in the corresponding subsections.
Data We use subset of ZINC 43 for the unconstrained generation. Following 22 the reference conformation for each molecule is obtained by running 20 times MMFF [66 optimization using RDKit ① and selecting the one with lowest energy. We use the same procedure as 21 to create fragments- linker pairs and randomly split the dataset, which results in set with 438,610/ / 400 /400 examples. For the constrained generation, we use PROTAC-DB [45). a database collecting PROTACs from the literature or calculated by programs. The same procedure is applied to obtain reference conformations and create data pairs. We select 10 different warheads as the test set (43 examples) and the remaining the training set (992 examples).
7

---

Table 1: Unconstrained generation results on ZINC. (*) denotes additional anchor information is
utilized and thus scores are not directly comparable. (++) denotes a huge number (>100k in energy).
The mean and standard deviation values are reported by running the sampling procedure times with different random seeds.
Method
Valid,5
Unique,5 S6 Novel,%
Rec,%
QE)
SA)))
Fain (D)
RMSD (1)
Eg (
AFe ())
DeLinker
968 0.2 43.5 0.44 43.3 0.2
55.8 122
0.61 0.0
3,13 1: 0.0
40.
53.6 0.6
47.
0.9
0,55
0.00
0.0
2,42 0.01
5178.2 108.5
DillLinker
4.7400.
3.
99.5 +
0.59
0.00
0:0
179.2 +
0.00
216,8
93.3 +
Ours
83.1 +.0
14.8
11.4
24.4 0.1
0.70 0.00
3.01 + 0.00
32,7 1.2
1.44 0.00
54.8 + 10.5
67.9 + 1.3
Baselines For benchmarking, we compare our model with three baselines: Delinner [22]. 3DLineer [20] and DiffLinker [2I]. DeLinker is a 2D graph generative model, while 3DLinker and DiffLinker are 3D generative models with VAE and diffusion models, respectively. Since there is no existing generative model to perform the fragment poses and linker co-design, we randomly sample fragment rotations and add noise to fragment center positions. Then, the noisy fragments are fed to these models to generate linkers.
Evaluation Metrics For ZINC and PROTAC-DB dataset, we generate 250 and 100 samples per fragment pair respectively for further evaluation. We evaluate the generated molecules on both 2D graphs and 3D conformations. For 2D metrics, we report the standard ones including validity, uniqueness and novelty [4) the recovery rate (the percentage of generated molecules that can successfully recover the reference molecules), and property-related metrics drug-likeness (QED) 3 and synthetic accessibility (SA) [33. To evaluate 3D conformations, we first perform MMFF 16 optimization, and report the average minimum energy of generated molecules per fragment pair before optimization as Emin. and the average Root Mean Square Deviation (RMSD) of the molecule coordinates before and after optimization as RMSD. Emin indicates the best quality of the overall generated conformations and RMSD indicates the gap between generated conformations and the best possible ones. To further investigate the model's performance in generating linker and fragment poses respectively, we perform another constrained MMFF optimization by fixing the atoms in fragments. Then, we report the average median energy after optimization as Eg and the average median energy difference before and after optimization AE, Since fragment atoms are fixed, the optimization will fix the unrealistic conformation inside the linker and thus a lower Er, indicates the better fragment poses, and lower FG, means less adjustment linker atoms and thus indicates the better linker conformation.
4.2 Unconstrained Generation Results
For baseline models, we randomly sample fragment rotations and add a Gaussian noise on the fragment distance (stddev 0.1 distance). We filter out clashing initial fragments and feed each baseline with 250 valid initial fragment poses per fragment pair. We fix the number of linker atoms to be the same the reference molecule. For DeLinker and 3DLinker baselines, anchor atoms also provided to utilize their published models while they are unavailable for DiffLinker and our model.
From Table I we can first see that our model can generate much more valid molecules compared to other 3D linker generative models (3DLinker and DiffLLinerr. Although our model's uniqueness and novelty scores are lower than other baselines, we believe that is because the chemical linker design space to form a low-energy molecule is limited. Our model achieves a recovery rate of 24.5%. In contrast, DiffI Dnnee achieves zero recovery rate since it can not co-design fragment poses, while DeLinker 3DLinker utilizes the anchor information and thus achieve a higher recovery rate. In terms of QED and SA, our model clearly outperforms other models, which indicates generated linkers are more realistic, For 3D-related evaluation metrics, our model can achieve much lower energy and RMSD compared to other baselines, which justifies the effectiveness of our modeling approach.
Since we argue that our innspres fragment pose prediction module is more effective in Sec. [3.3] we conduct an ablation study (See Table 2 on it to further investigate the role of each part. We take equation 12 as the counterpart of the Newton-Buler equations-based update formula (equation T6). Firstly, we find the fragment translation prediction based on Newton equation is more effective than predicting the change of translation in the local frame. The latter approach will result in severe drift (the fragments are far away from each other) during the sampling phase, as reflected in
8

---

Table 2: Ablation study the fragment pose prediction module.
Pose Pred
Valid, %
Rec,
Emin
RMSD (U)
(L)
AE
Newton
Fuler
Place Culor Pict
(a)
81.1
47.3
(b)
97.2
70.8
180.4
2.22
2052.2
(c)
81.6
20.3
(d)
Figure 4: Rotation loss.
Fragments
Reference
LinkerNet
DiffLinker
ZINC
PROTAC-DB
Figure 5: Visualization of reference molecules and molecules generated by LinkerNet and DiffLinkerr
the huge energy and RMSD in (a) and (c), even though the model can learn to connect one linker atom with each fragment to achieve reasonable scores in 2D metrics such as validity and recovery rate. Secondly, we can see the fragment rotation prediction based on Euler equation (d) can achieve lower energy and RMSD compared to its counterpart (b), indicating our design can boost the model's capacity and make more accurate rotation predictions. We further plot the rotation training loss in Fig. which confirms our argument since the rotation loss decreases expected with Euler-equation rotation prediction while it almost remains unchanged with the counterpart design.
4.3 Constrained Generation Results
In the constrained generation setting, we mimic the real scenario in the PROTAC linker design. Since two fragments also need to bind the protein of interest and E3 ligase separately, the possible anchor can only be chosen from a subset the fragment atoms. In addition, the linker length has a critical effect on the PROTAC's selectivity, and we usually need restrict the range of fragment distance. To include these constraints, we take the atoms within two hops of the real anchor atom as the imaginary candidate anchor [0.8d, 1.2d], where is the fragment distance in the reference molecule.
set for each fragment. We set the fragment center distance constraint as
We generate samples with guided sampling and hard bond mask as described in 3.4 For baselines, the fragment distance is uniformly sampled within the constraint, and random anchors from candidate anchor sets provided. The number of linker atoms is sampled according to the approach proposed in their original paper. Since the public pretrained 3DLinker model can not support the linker generation to large molecular graphs such PROTACs, we do not include it for comparison.
Table shows the constrained generation results on PROTAC-DB. Our model achieves higher validity and recovery rate, and outperforms other baselines in three of 3D metrics (Emin. RMSD and Er.) with clear margin. DiffLinker has lower energy difference before and after linker force filed optimization than ours, which indicates molecules generated by DiffL.inker has better geometry inside the linker. makes sense since DiffLinker focuses on learning the linker distribution only, complex fragment poses and linker co-design task.
while model is trained

---

Table Constrained generation results on PROTAC-DB. The mean and standard deviation values are reported by running the sampling procedure times with different random seeds.
Method
Valid
Unique,8 S
Novel,%
Rec,
Eunia (1)
RMSD
())
AF2
DeLinker 42.8
DiflLinker
SOLO
Ours 55.5
47.9 11.6
41.4 5.4
113.7
1.55 0.02
29.9 17.1
99.1
Table Ablation study on the guided sampling.
Guidance
Valid, %
DistSuce, %
Emin (U)
RMSD (L)
E, (U)
AE, (1)
Anchor Distance
(a)
45,9
25.2
+
1.66
124.3
(b)
53.2
25.4
1249.7
1.61
432.5
1495.0
4
49.3
53.7
208.4
1.56
45.4
4971.0
(d)
55.5
52.6
115.6
1.54
18.2
610.7
To justify the effect of each guidance term, we conduct another ablation study as shown in Tab. [ Without guided sampling, the model will generate unrealistic 3D linkers for some fragment pairs and result in very large average energy Emin and AEL. Anchor proximity guidance (b) and fragment distance (c) both of benefit for meeting the constraints and achieving lower energy and RMSD. The combination of them (d) achieves the best result.
Conclusion
We introduce LinkerNet for 3D fragment poses and linker co-design, the first work which addresses this more general and challenging linker design task. The limitation of our model is that it does not directly incorporate fragment rotation constraints or explicitly consider the protein context in modeling. These aspects could be valuable directions for future research.
Reproducibility Statements can be found here: httts:///gtthhii com/ guan ja/Lingenneeesseeee
The model implementation, experimental data and model checkpoints
Acknowledgement We thank all the reviewers for their feedbacks through out the review cycles of the manuscript. This work was supported by the National Key Research and Development Program of China grants 2022YFF 2TFFF1201000 100 and 202 YFF1201600
10

---

References
[1] Rdkit: Open-source cheminformatics.
rdkit org/
[2] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:1 1s:11981-111 199919, 2021.
[3] G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
[4] Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: bench- marking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096- 108, 2019.
[5] João R Cardoso and Silva Leite. Exponentials of skew-symmetric matrices and logarithms of orthogonal matrices. Journal of computational and applied mathematics, 233(1 233(11))26772875, 2010.
[6] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint arXiv:22 10.01776 2022.
[7] Brandon Dale, Meng Cheng. Kwang- Su Park, Ümit Kaniskan, Yue Xiong, and Jian Jin. Advancing targeted protein degradation for cancer therapy. Nature reviews Cancer, 21(0)):638- 654, 2021.
[8] Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thomton, Yee whye Teh, and Arnaud Doucet. Riemannian score-based generative modeling, arXiv preprint 022,222073, 2022.
[9] Raymond Deshaies. Prime time for protacs. Nature chemical biology, 11(9))634-633, 2015.
[10] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
[11] Ethan Eade. Lie groups for 2d and 3d transformations. URL http://ethanaade. com/lie, pdf. revised Dec, 117:118, 2013.
[12] Daniel A Erlanson, Robert S McDowell, and Tom O' O'Brien. Fragment- based drug discovery. Journal of medicinal chemistry, 47(14):346333482 2004.
[13] Peter Ertl and Ansgar Schuffenhauer. Estimation synthetic accessibility score of drug-like molecules based molecular complexity and fragment contributions. Journal of cheminfor- matics, 1:1-11, 2009.
[14] Nicholas Gao and Stephan Günnemann. Ab-initio potential energy surfaces by pairing gnns with neural wave functions. arXiv preprint arXiv:2110.05064, 2021.
[15] Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. arXiv preprint arXiv:2303.03543 2023.
[16] Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 17(5-6):490-519, 1996.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840 33:6840-6851, 2020.
[18] Emiel Hoogehoom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34, 2021.
[19] Emiel Hoogehoom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. arXiv preprint 22Xiv:2203.17000 2022.
11

---

[20] Yinan Huang, Xingang Peng, Jianzhu Ma, and Muhan Zhang. 3dlinker: An e (3) equivariant variational autoencoder for molecular linker design. arXiv preprint urXiv:2205.07:099 2022.
[21] Ilia Igashov, Hannes Stärk, Clément Vignac, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia. Equivariant 3d-conditional diffusion models for molecular linker design. arXiv preprint arXiv:2210.05274 2022.
[22] Fergus Imrie, Anthony R Bradley, Mihaela van der Schaar, and Charlotte M Deane. Deep gen- erative models for 3d linker design. Journal of chemical information and modeling, 60(4):1983- 1995, 2020.
[23] Fergus Imrie, Thomas E Hadfield, Anthony R Bradley, and Charlotte M Deane. Deep generative design with 3d pharmacophoric constraints. Chemical science, 12(43):14577-1458,, 2021.
[24] Wengong Jin, Siranush Sarkizova, Xun Chen, Nir Hacohen, and Caroline Uhler. Unsupervised protein-ligand binding energy prediction via neural euler's rotation equation. arXiv preprint arXiv:2301.108/4 2023.
[25] Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. arXiv preprint arXiv:2206.0172) 2022.
[26] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger. Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-589, 2021.
[27] Ashton C Lai and Craig M Crews. Induced protein degradation: emerging drug discovery paradigm. Nature reviews Drug discovery, 16(2):101-114, 2017.
[28] Adam Leach, Sebastian M Schmon, Matteo T Degiacomi, and Chris G Willcocks. Denoising diffusion probabilistic models on so (3) for rotational alignment. 2022.
[29] Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Li. Diffbp: Gener- ative diffusion of 3d molecules for target protein binding. arXiv preprint arXiv:2211.11214, 2022.
[30] Nya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X 2017.
[31] Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen- specific antibody design and optimization with diffusion-based generative models. bioRxiv, pages 2022-07, 2022.
[32] Hannah Maple, Nat Clayden, Anne Baron, Callum Stacey, and Robert Felix. Develop- ing degraders: principles and perspectives on design and chemical space. MedChemComm, 10510):7555-174, 2019.
[33] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[34] Dmitry Nikolayev and Tatjana Savyolov. Normal distribution on the rotation group so (3). Textures and Microstructures, 29, 1970.
[35] Mariell Pettersson and Craig M Crews. Proteolysis targeting chimeras (protacs)-past present and future. Drug Discovery Today: Technologies, 31:15-27, 2019.
[36] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arX1v:2204.06125, 2022.
[37] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
12

---

[38] Kathleen M Sakamoto, Kyung B Kim, Akiko Kumagai, Frank Mercurio, Craig M Crews, and Raymond J Deshaies. Protacs: Chimeric molecules that target proteins to the skpl-culiin- box complex for ubiquitination and degradation. Proceedings of the National Academy of Sciences, (8(15):85548855) 2001.
[39] Arne Schneuing, Yuanqi Du Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, et al. Structure-based drug design with cquivariant diffusion models. arXiv preprint arXiv:2210.1366 2022.
[40] Stuart L Schreiber. The rise of molecular glues. Cell, 184(1):3-9, 2021.
[41] Jascha Sohl-Dickstein Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages
PMLR, 2015.
[42] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances Neural Information Processing Systems, 32, 2019.
[43] Teague Sterling and John J Irwin. Zinc 15 liglid discovery for everyone. Journal of chemical information and modeling, 55(11):2324 2337, 2015.
[44] Robert Ian Troup. Charlene Fallan, and Matthias GJ Baud. Current strategies for the design of protac linkers: a critical review. Exploration of Targeted Anti-tumor Therapy, 1:273-312, 2020.
[45] Gaoqi Weng, Xuanyan Cai, Dongsheng Cao, Hongyan Du, Chao Shen, Yafeng Deng, Qiaojun He, Bo Yang. Dan Li, and Tingjun Hou. Protac-db 2.0: an updated database of protacs. Nucleic Acids Research, 51(D1):D1377DD -13722, 2023.
[46] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geo- metric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923, 2022.
[47] Yuyao Yang. Shuangjia Zheng, Shimin Su, Chao Zhao, Jun Xu, and Hongming Chen. Syn- talinker: automatic fragment linking with deep conditional transformer neural networks. Chemi- cal science, 11(31):811283222 2020.
[48] Shuangjia Zheng, Youhai Tan, Zhenyu Wang, Chengtao Li, Zhiqing Zhang, Xu Sang, Hongming Chen, and Yuedong Yang. Accelerated rational protac design via deep learning and molecular simulations. Nature Machine Intelligence, 4(9):739-748, 2022.
13

---

A Exponential and Logarithmic Mapping between so(3) and SO(3)
rotation matrix has associated axis-angle representation. The transformation between them relies upon the exponential and logarithmic mapping between the Lie algebra so(3) and 50(3).
Following standard definitions
[) the logarithm of rotation matrix R is:
(R) siin
R (R-R')) R")
(20)
where Tr(R) =1 cos 0. It can be shown that the logarithm of R E 5O(3) is skew-symmetric matrix S E so(3):
0
U2
y
S = log R =
02
0
2x
(21)
Vy
Up
where v = [ex, Uy. Uz] is the rotation axis and ||la| is the rotation angle.
Correspondingly, the exponential of skew-symmetir matrix is rotation matrix:
R = exp(S) = sin ||ll2
S 1 - cos |||2 g
(22)
|2/12
Following 28 we diffuse the rotation matrix by scaling the angle of rotation along the geodesic from the identity, which can be done by logarithmic mapping the rotation matrix to values in so(3), element-wis wise multiplying by scalar, and exponentially mapping them back to SO(3), i.c. X(7, R) exp(7 log(R)) RR)). We follow the same way as [3] to pre-compute and cache discretized angle distribution to draw samples efficiently.
B Proof of Equivariance
B.1 Equivariance w.r.f Global SE(3)
We denote the global ( as Ty. which can be written explicitly as x' = =y=TT (x) = R + b, where Rg E R the rotation matrix and b R is the translation vector. Applying T will result in the same transformation on fragment poses, i.c. p' Rp., R. R.
First, considering the zero-CoM (Center of Mass) operation C; æ; (pi + p2)/22 the translation vector in the global SE(3)-tranSoormation will be cancelled out:
E = Ty (i) = Rx; + (Rp1 + + Rp2 + b)/2 = R(x; (pi + p2) = Rø, (23)
Thus, we only need to consider the rotation transformation. Next, we will prove the equivariance in the linker denoising process and fragment pose prediction.
Linker Demoising Process It is easy see the atomic distance I xz || is Tg. Thus, eij. h; esj
are also invariant since the update of them (as shown in Eq. 10 only involves invariant inputs. For H., the update formula
d(x!)
x +
Σ SEV
(af o1) 1+1 11,11 G.j ,t)
linker
(24)
After applying Tg we have
o(T(x))) T(x))
Σ (T(x)) - T(z/)) (h!+) .nth _e, ,t). Linker
I+1
JEV
Ra
Σ
R(a ( 1+1 nntt e
1+1 .). llinkr
jEV
(25)
R x +
Σ
(of (h) .//11 eet t) 1
\EV
=()(x))),
14

---

which implies that the linker atom position update is equivariant. By stacking multiple layers together, we can draw the conclusion that the denoised linker atom positions are -quivvaana.
Fragment Poses Prediction
We recap the fragment poses prediction as follows:
f. = Σ -
11 Esy 21 - -x211
JEVRc
F.
Σ fir
Te Σ
(2t (pp) f
(26)
iEVE.
iEVE,
Po.0 - +
F.,
Re.o = Ru ( R.
Vr.
where c
2, corresponding to two fragments.
First, it is easy to see fi is equivariant w.r. Tg following the similar proof about the equivariance of linker atom positions. Thus, can prove that the total force F, and torque Ta equivariant:
F²
Σ iEVE
R Rg
Σ S: F.,
iEVE.
Σ (a; Pa) x Rig:
(27)
tEVF.
Rg
Σ ( Po) x f: = RgTo.
iEVF:
Second, we follow the same inertia matrix definition as 12 I. = Eier,, | - -Pell (z,
po)(2; P.)T. After applying Tg, we have I' Rg and thus the angular velocity
equivariant:
I Ryto" R M. = Rg
(28)
Third, we notice one nice property of Lie group is that the adjoint transformation is linear: for w so R (SO(3)
Rexp(w) ex)(Adinw))R,
(29)
In the case of SO(3), the adjoint transformation for an element exact the same rotation matrix used to represent the element [11. i.e. AdjR = R. Thus, we have R exp(w) = exp(Rw)R,
As result, the predicted fragment center positions are equivariant:
1
P'. R.P.
R.F.F R.poo
(30)
[Vp
and the predicted fragment rotations are also equivariant:
R = exp(R R. Rg exp(w) R. Rg R. = R2 explw)R, = Ro
(31)
B.2 Independence w.r.t Local Coordinate System
Since always take the fraament center the origin of the local coordinate system, only need to consider the effect of the orientation change. Suppose the global coordinates remain unchanged, and the change of local coordinate system results in rotation R. applied on local coordinates, i.e. X' = R. Thus, the transformation between local coordinates and global coordinates becomes RRR X p. which implies the new rotation matrix representing the fragment pose is R.R.
Recall that the fragment rotation update is Ro (1, TT R, With the change of local coordinate system, the predicted fragment rotation is R2 (1-1 T.) R2 R Since rotation loss is defined as I Rg R Il we have
L'm R. (R. |
I Lyot
(32)
which means the training of our model is independent w.r.t the choice of local coordinate system.
15

---

C Implementation Details
C.1
Featurization
The molecular graph extended as fully-connected graph. The atom features include a one-hot element and charge indicator (H, C, N, N N O. O F. CI, Br, I, S(2), S(4), S(6) and a one-hot fragment/linker indicator. Note that the fragment/linker can be predetermined. Thus, it will only serve as the input feature without getting involved in the network's 's prediction. The edge features include a one-hot bond type indicator (None, Single, Double, Triple, Aromatic), and a 4-dim one- hot vector indicating the edge is between fragment atoms, linker atoms, fragment-linker atoms or linker-fragment atoms.
C.2
Model Details
Atom features and edge features are firstly fed to two embedding layers with node_ eemb__mb______eee_eeeeee_ 2mb_dim=256 and edge_ emb dim=64. The hidden embeddings get involved in three types of layers: atom update In each layer, concatenate the input features and update the hidden embedding positions with 2-layer MLP with LayerNorm and ReLU activation. The stack of these three layers is viewed as block, and our model consists of blocks. For the force prediction layer, we apply graph attention aggregate the message of each node/edge. The key/value/query embedding is also obtained with a 2-layer MLP.
layer, bond update layer, and position update layer as described in Eq. (9) [0] IT
We set the number of diffusion steps 500. For this diffusion noise schedule, choose to use cosine schedule suggested in [33] with s10.01
C.3 Loss Functions
For the linker atom positions and fragment center positions losses, we use the standard mean Squared Error (MSE). For the rotation loss, measure the discrepancy between the real and the predicted rotation matrices by computing the Frobenius-norm of R I||- For the atom and bond type losses, we compute the KL divergence between the real posterior and the predicted posterior. Specifically, the loss functions can be summarized as follows:
Linker = ||X LL,
(33)
Lur Σ Pe
P=.0
(34)
Lio
E R I1
(35)
No
C(VL.I. VL.0)
Latomm
C(VL,,, VL,0): log
(36)
ö(VL. VL.0)
k=1
No.
E(br.t. br.o), log E(bL,,, bL.OLe
Lband
(37)
e(br.., DI.O)&
k=1
The final loss a weighted sum of them:
L A Tinner + A2 Tus + AgTina \s Teaton + As I-bond
(38)
C.4 Training Details
The model is trained via Adam W (30) with init_iiiialae www.leeaannnnreeeeeeeeeeeeeeeeee betas= 0. 0.999) batch_size=6t and multiply factor X = 100 on the atom type loss and bond type loss. During the training phase, we add small Gaussian noise with standard deviation of 0.05 to linker atom coordinates as data augmentation. We also schedule to decay the learning rate exponentially with a factor of 0.6 and a minimum learning rate le-6. The learning rate is decayed if there is no improvement for the validation loss in 10 consecutive evaluations. The evaluation is performed for every 2000 training steps. We trained our model on one NVIDIA RTX A6000 GPU, and it could converge within 350k steps.
To balance the scales of different losses,
16

---

C.5 Overall Training and Sampling Procedures
In this section, we summarize the overall training and sampling procedures of LinkerNet as Algorithm and Algorithm] 2 respectively.
Algorithm Training Procedure of LinkerNet
Input: Linker dataset (R. p, Gr. FF,, Gr, where Gr. br, 9F (vr, 8F and p: + P2 = 0; Neural network (hg
1: while (e not converge do
2: Sample diffusion time E L((......
3: Add noise to (xL, p). R and (VL. Py) according to Eq. 2 6 respectively
4: Compose [x,, Ve, bt, RR, Pal to form Gs, by performing Xy. R,Xr, + P
5: Move the molecule make P1,t P2.*
6: Predict [$r.o, V1,0; br.0, Ro, Po] from Gıs, with de as described Sec. 3.3
7: Compute the training loss I. according to Eq. (38)
8: Update by minimizing L
9: end while
Algorithm 2 Sampling Procedure of LinkerNet
Input:
The molecular fragments GF. Grg- the learned model Φg. Optional: the fragment distance constraint [dmin d the fragment candidate anchor sets A. Output: Generated fragment poses (R. p) and linker GL
1: If the fragment distance constraint is provided, sample the number of atoms in 9r. based prior distribution summarized from the training set.
2: Sample initial fragment poses (Rr; (R);Pr) and linker 91,r
3: for in T, T - 1,...,1 do
4: Compose [x,, Ve, b,, Re, Pi] form Gm, by performing XF,t R, + Pt
5: Move the molecule to make Pi.t + P2. 0
6: Predict [& V1,0, b2.0, Ro, Po] from GM, with de as described in Sec. 3.3
Sample 1 - b R. - from the posterior according to Eq. 2
8: Optional: Compute the guidance according to Eq. (18) 19) if the corresponding constraint
provided. Update the prediction with guidance according Eq.
9: end for
D Training and Sampling Efficiency
For the training efficiency, DiffLinker converges within 300 epochs and takes 76 with one V100 GPU as the original paper reported. Our model converges within 50 epochs and takes 48 hrs with the same type GPU.
For the sampling complexity, DiffLinker finished sampling linkers for 43 PROTAC fragment pairs (100 linkers for each pair) in 132 min while our model takes 761 min with the same NVIDIA 1080 GPU.
17


Reviews:
[{"review_id": "dVox6yBW6q", "reviewer": "Reviewer_kzw2", "summary": "The authors describe a novel method for the computational design of linkers using equivariant diffusion models. This is coupled with a fragment pose prediction step that allows the design of linkers without first knowing the relative orientation of the fragments.", "questions": "- In line 92, please clarify why this is suboptimal compared to your approach. This should be changed in the text to also be used as a point of comparison with your method.\n- Is there any data or evidence to back up the claim in line 280? Otherwise, this is a big limitation of the method.", "limitations": "The authors have mentioned limitations in the conclusion, albeit with a short sentence. This should be expanded upon.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Comprehensive related works section\n- Novel and sensible approach, allows for constrained generation\n- Good empirical results support the paper's claims\n- Good ablation experiments for the pose prediction module", "weaknesses": "- The authors need to better introduce the problem for a more general machine learning conference. Though well-defined for biologists, concepts like ubiquitination need some explanation for the NeurIPS audience.\n- The writing needs to be improved. The authors need to explain the equation forms of their method, to give an intuitive sense of what their method does. This is especially critical between lines 149-164.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and suggestions. Please see below for our responses to the comments.\n\n**Q1: “The authors need to better introduce the problem for a more general machine learning conference… The writing needs to be improved … ”**\n\nA1: We thank the reviewer for the suggestions on the writing part.  We have updated our manuscript by adding more explanations of the biological terminologies for the more general ML audience and adding more descriptions between equations.\n\n**Q2: “In line 92, please clarify why this is suboptimal compared to your approach.”**\n\nA2: The SMILES representation is not an optimal choice since it fails to capture molecular similarities and suffers from the validity issue during the generation phase. A small modification of the molecule formula will significantly change the SMILES string. These drawbacks are clearly discussed in [Jin et al., 2018] and we have added more explanation in the Related Work.\n\n**Q3: “Is there any data or evidence to back up the claim in line 280?”**\n\nA3: In the original paper of 3Dlinker and DiffLinker where the fragment poses are fixed, the novelty and uniqueness for their approaches are 29% / 32% and 24% / 30% on ZINC, which are way lower than the results of 50%+ / 90%+  in our new setting. In addition, the energies of molecules generated by 3DLinker and DiffLinker in our new setting are high, indicating their molecules are less stable. This evidence indicates that the actual chemical linker design space to form a low-energy molecule is small, supporting our claim in line 280 and related results.\n\n\n**Reference:**\n\nWengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pages 2323– 2332. PMLR, 2018.\n"}, {"review_id": "4is29ziGCh", "reviewer": "Reviewer_aUNA", "summary": "The paper presents a diffusion model for molecular linker design. Given the 3D structures of two molecular fragments, the model generates a pose for each of the fragments, as well as types and positions of atoms that can be added to link the two fragments in a single molecule.", "questions": "‘in scenarios involving new drug targets… relative position between fragments may not be readily available’ – why is this the case for new targets and not older ones?\n\nHow much freedom is there really in relative fragment poses for PROTACs? It would be nice to see real pairs of examples with same heads, different relative poses, and both having high experimentally verified bioactivity. In line 274 I have no idea if the stddev of 0.1 fragment distance is realistic.\n\nLine 83 ‘this is always the case’ should be ‘this is not always the case’\n\nLine 120: What are ‘atom features’ as in ‘number of atom features’?\n\nLine 116: say what Na, Nb, NF are first, then give the other definitions that use them.\n\nWhy do linker bond types need to be explicitly generated? Can’t they be inferred from linker atom positions?\n\nDid you experiment with other samplers besides the one described in equation (2)?\n\nHow did you decide on the noise schedules for all the different things – fragment rotation, linker atom types, linker atom positions and so on?\n\nLine 147 and line 154 should say ‘compute’, not ‘computer’.\n\nLine 178: what is V_M under the summation? All the atoms in the molecule?\n\nIs the ‘baselines’ comparison, where fragment poses are randomly corrupted, fair to DiffLinker etc.? If the fragments are presented in a relative pose for which no sensible linker can be designed, then not only does this give DiffLinker an impossible task, it also presents DiffLinker with an input that is presumably out-of-distribution relative to the partially obscured real molecules that it was trained on.\n\nAre there always 2 fragments? Do you sample position and rotation for both? You could fix one without loss of generality.\n\nLine 258 and table 1: is ‘Recovery rate’ how frequently a single sample is identical to the reference, or is it how often the reference appears in the list of 250 or 100 generated samples?\n\nLine 321 and Table 4: is row (a) of table 4 just the same as row ‘ours’ of table 1, except that one is for ZINC and the other for PROTAC-DB?\n\nLine 326: ‘liker’ should be ‘linker’.\n\nWhy is the fragment pose prediction module separate? Why not use a single GNN to predict ‘forces’ on all fragment and linker atoms, with a special readout head to do the update in equation (16)?", "limitations": "The methodology seems very specific to this problem.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method and evaluation seem sound.\n\nOn the whole, the work and its relation to prior works are clearly explained.  \n\nIf you are interested in designing PROTACs, then it is a nice practical contribution.", "weaknesses": "In section 3.3 I found the description of the fragment pose prediction module hard to understand. I didn’t understand the description leading up to equation (12) and I would like to know the architecture of the neural net used for fragment pose prediction. Is it another GNN the same as the linker GNN? \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed feedback and suggestions. Please see below for our responses to the comments.     \n\n**Q1: “In section 3.3 I found the description of the fragment pose prediction module hard to understand...”**\n\nA1: Equation (12) describes a way to update poses by predicting the pose change and applying it to the current pose, i.e. $(R_t, p_t) \\circ (R_{t \\rightarrow 0}, p_{t \\rightarrow 0}) = (R_t R_{t \\rightarrow 0}, p_t + R_t p_{t \\rightarrow 0})$, which is same as the formula in Supplementary 1.8 Structure Module in the AlphaFold2 paper [Jumper et al., 2021]. The neural network predicts the rotation and translation change $R_{t \\rightarrow 0} = \\phi_R $ and $p_{t \\rightarrow 0} = \\phi_p$, which are invariant to the global rigid transformation, and apply them to the current noisy rotation and translation $R_t$ and $p_t$ to obtain the denoised ones $\\hat{R}_0$ and $\\hat{p}_0$. We have added more explanation before introducing Equation (12). It’s also worth noting that this updated formula is suboptimal and is *not* the one we used. We compare this way with our proposed physics-inspired pose prediction module in Table 2 to show the superiority of our method. \n\nThe fragment pose prediction module is one graph attention layer built upon the linker GNN, instead of a separate GNN. As Figure 2 shows, we added one graph attention layer ($\\phi_f$) on the top of $h^L$ to predict the force and torque. We have clarified it in the updated manuscript.\n\n**Q2: “‘in scenarios involving new drug targets… relative position between fragments may not be readily available’ – why is this the case for new targets and not older ones?”**\n\nA2: “Involving new drug targets” means the molecular fragments are bound with two protein targets instead of one. In this case, the relative position between fragments becomes unknown since the protein-protein binding pose has flexibility (see the example Fig.1 in the general response).  We have clarified this point in the revised manuscript. \n\n**Q3: “How much freedom is there really in relative fragment poses for PROTACs?”**\n\nA3: We find two PROTACs whose fragments bind with the same protein target (BRD4) and E3 ligase (CRBN) as Fig.1 in the general response shows. The linkers differ with a ‘COC’ motif. As we can see, the fragment poses differ a lot even with a slightly different linker. \n\n**Q4: “Why do linker bond types need to be explicitly generated? Can’t they be inferred from linker atom positions?”**\n\nA4: The heuristic approach to infer bond types is sensitive to the choices of hyperparameters and atom positions (the bond distance distribution is very sharp), and is not always reliable. Thus, we choose to predict bond types simultaneously. \n\n**Q5: “Did you experiment with other samplers besides the one described in equation (2)?”**\n\nA5: No, this is the standard choice of the diffusion process since we can directly draw samples from $q(x_t | x_0)$ and compute the posterior in closed-form.\n\n**Q6: “How did you decide on the noise schedules for all the different things – fragment rotation, linker atom types, linker atom positions and so on?”**\n\nA6: We choose to use the cosine schedule [Nichol and Dhariwal, 2021] with s=0.01 as all noise schedules. This is a common choice in DDPM. We follow this way and find it works well empirically. \n\n**Q7: “Is the ‘baselines’ comparison, where fragment poses are randomly corrupted, fair to DiffLinker etc.?”**\n\nA7: In the constrained generation setting, we randomly sample fragment poses but also make sure the candidate anchors in two fragments are facing to each other and there is no clash between fragments. This setting has been close to the real scenario for generative models which can not predict fragment poses, otherwise most of the other generative models would all fail. \n\n**Q8: “Are there always 2 fragments? Do you sample position and rotation for both? You could fix one without loss of generality.”**\n\nA8: Great question! Yes, PROTAC always involve two proteins and thus two binding molecular fragments. Although fixing one and sampling the other is equivalent to sampling two poses mathematically, it will involve the fussy coordinate system transformation for linker atoms and may make the learning process more difficult. For instance, assume the current ligand pose is [-M1+] [L] [-M2+] and the true pose is [+M2-][L][-M1+], where M1, M2 are two fragments, L is the linker and +/- denotes the two sides of fragments. The model would have to move all linker atoms in L and M2 to the other side if we fix M1, but it could simply mirror rotate M1 to achieve  [+M1-] [L] [-M2+] (same as  [-M1+] [L] [-M2+]) if we sample poses for both fragments. In preliminary experiments, we test sampling one relative pos / sampling fragment distance and two rotations, but neither of them is as effective as sampling two poses. \n\n**Q9: Line 258 and table 1: the definition of ‘Recovery rate’**\n\nA9: We sample 250 or 100 linkers for each fragment pair in the test set. If any linker is same to the reference linker, we count it as recovered for this fragment pair. The recovery rate is averaged over the whole test set. \n\n**Q10: Line 321 and Table 4: is row (a) of table 4 just the same as row ‘ours’ of table 1, except that one is for ZINC and the other for PROTAC-DB?**\n\nA10: Yes. The molecules in PROTAC-DB are much larger than the ones in ZINC. Without appropriate constrained guidance, the energy is easy to be very large. Another reason is that we applied the hard bond mask based on the candidate anchor sets as described in Sec 3.4.  Without guided sampling, the model may generate strange bond angles leading to high energy. \n\nWe thank the reviewer for pointing typos out! We will fix them in the updated manuscript.\n\n\n**References:**\n* John Jumper et al. Highly accurate protein structure prediction with alphafold. Nature, 2021.\n* Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. ICML, 2021.\n"}, {"review_id": "nXWjvTonXk", "reviewer": "Reviewer_4AKh", "summary": "The article discusses the problem of designing linkers to connect different molecular fragments in order to form stable drug-candidate molecules, specifically in targeted protein degradation techniques such as PROteolysis TArgeting Chimeras (PROTACs). One significant challenge in these techniques is that existing models for linker design assume that the relative positions of the fragments are known, which may not be the case in real scenarios. This problem is addressed by the authors through the development of a 3D equivariant diffusion model that jointly learns the generative process of both fragment poses and the 3D structure of the linker, viewing fragments as rigid bodies and designing a fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics.\n\nThe proposed 3D equivariant diffusion model, called LinkerNet, jointly learns the generative process of both fragment poses and the 3D structure of the linker. Fragments are viewed as rigid bodies and designed with a fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics. To address the problem of designing linkers when fragment poses are unknown in 3D space, LinkerNet is able to co-design fragment poses and the linker. The model represents each fragment as a rigid body and its pose as the position of its center and the rotation. The linker is a 3D graph with atom positions, atom types, and bond types. LinkerNet is a diffusion model for the fragment poses and linker co-design task, involving an equivariant network and physics-inspired prediction module for denoising fragment poses and the linker. The model is able to generate chemically valid, synthetically-accessible, and low-energy molecules under both unconstrained and constrained generation settings.", "questions": "Question:\n1. How is the training/sampling complexity compared with existing methods, especially with DiffLinker?\n\n2. I would like to see the error bars (standard deviation) presented in the table results.\n\n\nTypo: \n1. Line 154, draw samples and computer $\\rightarrow$ draw samples and compute.", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "First, the proposed method, LinkerNet, differs from previous methods in linker generation by co-designing fragment poses and the linker in a 3D equivariant diffusion model, instead of assuming that the relative positions of the fragments are known. Previous methods mostly focus on 2D or 3D linker design with fixed fragment poses, while LinkerNet addresses the more general linker design problem where fragment poses are unknown in 3D space. This perspective is new and can be a good guidance for future works on linker design.\n\nThe authors also introduce a physics-based neural network for their task, which is also novel. The physics-inspired neural network in LinkerNet is advantageous in co-designing fragment poses and the 3D linker for stable drug-candidate molecules. It allows for predicting fragment poses in a way that takes into account the molecular geometry and leverages Newton-Euler equations in rigid body mechanics, making it more effective than simply predicting an invariant update in the local coordinate system. The neural network predicts forces on each fragment atom to compute the total force and torque on each fragment, which is more effective than predicting only the change in translation in the local frame.\n\nExperimentally, the authors demonstrate that their proposed model can generate valid, synthetic-accessible, and low-energy molecules under both unconstrained and constrained generation settings, outperforming other existing generative models.\n\nOverall, the paper presents a novel good-quality work with good motivation and theoretical justification. For experiments, LinkerNet achieves satisfying results on benchmark datasets compared with existing methods.", "weaknesses": "The experimental results are not reported with error bars (standard deviation), and the training complexity, training time, and sampling time are not compared between different methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and suggestions. Please see below for our responses to the comments.\n\n**Q1: How is the training/sampling complexity compared with existing methods, especially with DiffLinker?**\n\nA1: For the training complexity, DiffLinker converges within 300 epochs and takes 76 hrs with one V100 GPU as the original paper reported. Our model converges within 50 epochs and takes 48 hrs with the same type of GPU. For the sampling complexity, DiffLinker finished sampling linkers for 43 PROTAC fragment pairs (100 linkers for each pair) in 132 min while our model takes 761 min with the same NVIDIA 1080 GPU. We thank the reviewer's suggestion and will add the training/sampling complexity analysis in the updated manuscript.\n\n**Q2: I would like to see the error bars (standard deviation) presented in the table results.**\n\nA2: We have computed the error bars for the main results by running the sampling procedure 3 times with different random seeds. The results are shown in Table 1/2 in the general response. \n\n"}, {"review_id": "YcotLdW6Bo", "reviewer": "Reviewer_ZLo5", "summary": "In this paper, the authors formulated a new linker design task where the fragment poses are unknown. The authors proposed a 3D equivariant diffusion model which enables the co-design of fragment poses and the linker structure in a unified framework.", "questions": "please see the weakness.", "limitations": "The authors have adequately discussed the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow.\n2. The authors developed an effective fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics, allowing for the accurate adjustment of fragment center positions and rotations. This is the main technical contribution compared with the previous work DiffLinker.\n3.  Comprehensive experiments on ZINC and PROTAC-DB datasets demonstrate the superiority of LinkerNet over other baseline methods in both unconstrained and constrained generation settings.", "weaknesses": "1. The authors do not show the application of LinkerNet for molecule generation conditioned on the target protein.\n2. The code is not provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and suggestions. Please see below for our responses to the comments.\n\n**Q1: The authors do not show the application of LinkerNet for molecule generation conditioned on the target protein.**\n\nA1:  Thank you for pointing this out! This is indeed one limitation as we discussed in the Conclusion section. We will consider the protein context and apply our model in a more realistic scenario in future work.\n\n**Q2: The code is not provided.**\n\nA2: We are committed to open-sourcing the data, training/inference code, and the model checkpoint once the paper is published.\n"}]

Author rebuttal:
We thank all reviewers for their efforts and time in evaluating our submission and providing valuable suggestions and feedback.  In the general response pdf, we 

* Add one example to show that the fragment poses are **not** fixed in the PROTAC design (Figure 1). Both PROTACs bind with the same protein target and E3 ligase, but the fragment poses and linkers are different. 

* Add error bars for the main results in ZINC and PROTAC-DB datasets (Table 1, Table 2). The results still support our claims in the main text. 

Please let us know if there are any additional concerns we can address for you to consider raising your initial rating, thanks!

Meta-review: This paper introduces a new and interesting method for the computational design of linkers by using equivariant diffusion models. All reviewers are positive on this paper, and they point out that the method is novel and experiments are extensive. The method is also effective as shown in the experiments. There are some minor weaknesses in the paper, such as writing, which however can be addressed in the final version. AC agrees with the reviewers, and believes this work is worth publishing in NeurIPS.
Decision: Accept (spotlight)