Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Authors: Zixing Song
Ziqiao Meng
José Miguel Hernández-Lobato
Venue: ICML 2025 poster (2025)
Keywords: PROTAC
Linker Design
Diffusion Models
Primary area: applications->health_medicine
TL;DR: We present a domain-adapted diffusion model for PROTAC linker design task.

Abstract: Proteolysis-targeting chimeras (PROTACs) are a groundbreaking technology for targeted protein degradation, but designing effective linkers that connect two molecular fragments to form a drug-candidate PROTAC molecule remains a key challenge. While diffusion models show promise in molecular generation, current diffusion models for PROTAC linker design are typically trained on small molecule datasets, introducing distribution mismatches in the chemical space between small molecules and target PROTACs. Direct fine-tuning on limited PROTAC datasets often results in overfitting and poor generalization. In this work, we propose DAD-PROTAC, a domain-adapted diffusion model for PROTAC linker design, which addresses this distribution mismatch in chemical space through density ratio estimation to bridge the gap between small-molecule and PROTAC domains. By decomposing the target score estimator into a pre-trained score function and a lightweight score correction term, DAD-PROTAC achieves efficient fine-tuning without full retraining. Experimental results demonstrate its superior ability to generate high-quality PROTAC linkers.

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Zixing Song' Ziqiao Meng?
José Miguel Hernández-Lobato
Abstract
cootellysis-targetin chimeras (PROTACs) are a groundbreaking technology for targeted protein degradation, but designing effective linkers that connect two molecular fragments to form drug- candidate PROTAC molecule remains key chal- lenge. While diffusion models show promise in molecular generation, current diffusion models for PROTAC linker design are typically trained on small molecule datasets, introducing distribu- tion mismatches the chemical space between small molecules and target PROTACs. Direct fine-tuning on limited PROTAC datasets often re- sults in overfitting and poor generalization. In this work, we propose DAD-PROTAC, a domain- adapted diffusion model for PROTAC linker de- sign, which addresses this distribution mismatch in chemical space through density ratio estima- tion to bridge the gap between mooll- and PROTAC domains. By decomposing the target score estimator into pre-trained score function and lightweight score correction term, DAD- PROTAC achieves efficient fine-tuning without full retraining, Experimental results demonstrate its superior ability to generate high-quality PRO- TAC linkers.
Fragment G%
Fragment G
Linker g'
Linker Design
PROTAC
PROTAC Linker
Figure 1: PROTAC linker design. Given two molecule fragments, the goal is to generate a linker that connects them to form a PROTAC molecule (Details in Figure 6).
binds to the target protein for degradation, another lig- and that binds to an E3 ligase, and a linker that connects two ligands (Bemis et al., 2021). Unlike traditional small molecule drugs that only temporarily suppress protein func- tion (Martín-Acosta & Xiao, 2021), PROTACs achieve com- plete protein removal with enhanced selectivity and reduced ffr-et effects. However, the design of effective PRO- TACs faces significant challenges, particularly in optimizing the linker structure (Troup et al., 2020). The PROTAC linker must ensure precise spatial arrangement of atoms in 3D space, with accurate atom and bond types, while simultane- ously meeting complex physicochemical constraints when it connected the two ligands (Figure 1).
1. Introduction
pontollsii-targgetinn chimera (PROTAC) (Gharbi & Mer- cado, 2024) has emerged as a useful technology for tar- geted protein degradation (TPD) (Zhao et al., 2022) in drug discovery (Schapira al., 2019), leveraging the ubiquitin- proteasome system to remove specific unwanted disease- relevant proteins. A PROTAC is hetero- biterobbifunctional molecule consisting of three components: ligand that
Recent advances in diffusion models have shown remarkable success in molecule generation tasks (Hoogeboom et al., 2022; Xu et al., 2023; Huang et al., 2024), making them powerful tools for PROTAC linker design (Igashov et al., 2024: Guan al., 2023; Li et al., 2024). In general, these diffusion models iteratively transform random noise into valid linker structures through learned denoising process, while preserving chemical and geometric constraints (More- head & Cheng, 2024). Hence, they can learn to gener- ate new samples that closely match the distribution of the linker in the given training dataset. Current approaches predominantly train these diffusion models on datasets con-
'Department of Engineering, University of Cambridge, Cam- bridge, United Kingdom NNational University Singapore, Singa- pore. Correspondence to: Zixing Song <zs456@ cuk.ac.uk>.
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
In this work, we introduce DAD-PROTAC, a Domain- Adapted Diffusion model for PROTAC linker generation that effectively transfers knowledge from the small molecule domain to the PROTAC domain. trained on small molecule datasets with a Gaussian dif- fusion process for continuous features (atom coordinates) and a multinomial diffusion process for discrete features (atom/bonn types). The key innovation lies in our fine- tuning strategy, which leverages density ratio estimation in chemical space to perform domain adaptation with theoreti- cal guidance. Instead of directly retraining the model on the PROTAC dataset, we decompose the optimal score estimator for the fine-tuning phase into two components: a pre- pre-trained score function from the small molecule domain and a score correction term based on the density ratio between domains. This approach not only highlights the role of density ra- tios in guiding fine-tuning but also improves efficiency by learning the score correction term as lightweight classifier rather than retraining the entire model.
1600
PROTAC Small Molocule
1400
1200
is pre-
Coint
1000
800
400
200
%
250
500
750 1000 1250 1500 1750 2000 Molecular Weight (Da)
The distributions of molecular weight in PRO- TACs versus small molecules in the chemical space.
Figure 2:
structed from the readily available small molecule domain, such as ZINC (Irwin & Shoichet, 2005), where existing molecules are artificially split into two fragments connected by a linker (Igashov et al., 2024). This simple training setup enables the models to capture the relationships between the resulting fragments and the linker, establishing a foundation for generating the linker based on the given fragments from the small molecule domain only.
Our contributions can be summarized as follows.
We propose novel domain-adapted diffusion model (DAD-PROTAC) for PROTAC linker design that ex- plicitly utilizes density ratio in chemical space between the small molecule domain and the PROTAC domain.
Therefore, a critical challenge in applying diffusion models to the PROTAC linker design task lies in the distribution mismatch in the chemical space between the small molecule domain, where the models are typically trained, and the PROTAC domain, where they are ultimately deployed. Ide- ally, training datasets would be derived directly from the PROTAC domain, such as experimentally validated ternary complexes in the PROTAC-DB dataset (Weng et al., 2021). However, because synthesizing PROTACs is expensive and time-consuming, the PROTAC-DB CD dataset is much smaller than small molecule datasets like ZINC. As illustrated in Fig- ure2 2. the chemical space of PROTACs significantly dif- fers from that of small molecules, particularly in molecular weight, reflecting the larger size of PROTACs. More discus- sions on the differences between the two domains can be found in Appendix 1. The distribution mismatch in the chemical space introduces biases to the generation of PRO- TAC linkers by diffusion models trained on small molecules, limiting their ability to capture PROTAC-specific properties. While direct fine-tuning on PROTAC datasets seems like a natural solution, the limited size of such datasets often leads to overfitting and poor generalization empirically.
We theoretically show that the score estimator for the target PROTAC domain can be decomposed into the pre-trained score function and another score correction term involved with density ratio estimation.
We present an efficient fine-tuning approach by learn- ing to approximate the score correction term guided by the density ratio, avoiding full model re-training.
2. Method
2.1. Problem Definition
Following existing works (Guan et al., 2023), we represent each molecular fragment as a 3D graph GF {V² gFF. The atom set D and the atom bond set E are associated with the one-hot atom type features A E R and the one-hot bond type features B" E NN,,,N respec- tively. Here, Ny, Na, Nb denote the number of atoms in the fragment, the number of atom types, and the number of bond types. The atom coordinates X E R of the frag- ments are in the 3D space. Hence, each molecular fragment gF is compactly represented by triple (AF, ,BF, xFF))).
This distribution mismatch highlights the need for domain- adapted diffusion model that explicitly incorporates the ratio of probability density between the small molecule and PRO- TAC domains in the chemical space. Modeling this density ratio enhances the fine-tuning process of diffusion models for general linker design, improves alignment with the PRO- TAC domain, and enables specific PROTAC linker design.
Similarly, the linker Gb = (VL, ,EL} can also be repre- sented as {A", 14). Here, A E N.NN denotes the atom type features of the linker. B E N denotes the bond type features of the linker. Here, N = N₁, + N₁, + N is the total number of atoms of two frag-
2

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
(GEIGE-))
The dataset is constructed using matched-molecular pairs (MMPs) break molecules into fragment molecule triplets { G GP3 in the small molecule domain.
St
Forward Diffusion
yi
X
A diffusion probabilistic model consists of two interrelated Markov chains: a forward diffusion process and a reverse generative process (Ho et al., 2020). The forward diffusion process gradually adds noise the linker data G", trans- forming into a pure noise distribution. In contrast, the reverse generative process is designed to learn how to take that moise-perturber data and gradually remove the noise, generating new data samples that resemble the original dis- tribution g(gh , in the source domain for small molecules. Based on the representation of the linker in Section 2.1, the distribution g(g² g can be fur- ther decomposed as a product of atom coordinates in the linker 9Xx gFi. G..gg GF2) and the linker's s atom/bond types r(A4, BL GF.) We will discuss how these diffusion processes are constructed during the pre- pre-training phase.
A
HHE Br
Br
C.H.
Fr
Fr
Reverse
Denoising
1
Estimated
gh
EGNN
Pre-training Phase
EGNN (Fixed)
g GG)
Pre-trained Score
Fine-tuning Phase
9
GG (9b)
Correction Term
96 S0- ( 4G
G
Gb
Reverse
x+
Denoising
X
2.2.1. 1. DIFFUSION ON ATOM COORDINATES X
4
Br
Forward Diffusion
CHI
Br
The diffusion process on X utilizes the standard Gaussian diffusion kernels and thus follows the well -sla-lsaall De- noising diffusion probabilistic models (DDPMs) (Ho et al., 2020). At each time step small Gaussian noise is intro- duced for X based on the previous step X XL_, with fixed variance schedules B1, ... ,3 as in Equation (1).
aICHIC
Figure 3: The proposed PRO-PROTAC model. In the pre- training phase, PAD-PROTAC performs joint diffusion process for the three components of the linker from the small molecule domain g(G) and utilizes the EGNN model to estimate the scores for the reverse denoising process. In the fine-tuning phase, DAD-PROTAC obtains the base esti- mated scores from the pre-traine pre-trained model with new samples from the PROTAC domain g(G)) the input and learns to approximate the score correction term via two lightweight neural networks W and V under the guidance of the density ratio ((tt in the chemical space between two domains. (
a( X N(X / B B.1)
(I)
This forward process admits sampling X at an arbitrary time step l. directly based on the initial given clean data sample X with closed-form solution as in Equation (2).
x(,t x - VaxKa (1-ā,)I).
(2)
Here, or 1 Br, dr 11 11= Based on the reparam- eterization trick, we have X! V + vI- - aye for (O, The posterior distribution is obtained as
ments and the linker. Note that the edge set of the linker contains both the bonds within the linker itself and its bonds connected with the two fragments. X E R3x Nr. denotes the 3D coordinates of the atoms the linker.
(X/11 Xt x()) N (X) x B.D. (3)
Here, iee (Xt Xxf)
vor-T X parl--a--- Xt. and
Given pre-trained parameterized generative model (C F =) for the linker distribution ( in the source small molecule domain, we aim to fine-tune this model for the shifted linker distribution q(g²) in the target PROTAC domain. We resort to diffusion models (Hoogeboom et al., 2022: Schneuing et al., 2024) as the backbone. The related work is extensively discussed in Appendix
3, = 1-ū Br.
The reverse generative denoising process on XL aims to invert the diffusion trajectory by approximating the original linker's coordinates Xτ based on the noisy X at the time step using neural network with parameters 0. It is more effective to predict the Gaussian noise with the loss as
_ = E,Ex E. [llea( +
f) = (4)
2.2. Pre-training in the Small Molecule Domain
The dynamics of predicting co (xF,, will later be elabo- rated Section 2.2.3. If we let so sx('
In the pre-training phase (Figure 3), the diffusion model trained on a large-scale, readily accessible dataset.
,o(X .0)
and
vi-ā,
3

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
note that the score of posterior V Vxy. / log (
Φe are both invariant to global transffaarorrmmtionn Furthermore, atom coordinates of the linker x; (i E VL) updated by dy should be SE(3)-equivariant The pro- posed DAD-PROTAC model learns to predict both so *o(X(,)) () and co(Af, () with equivariance constraints, utilizing an L-th layer E(n) Equivariant Graph Neural Network (EGNN) model de (Satorras et al., 2021), as [sg. = oo(g") t) = (o([, ,,, gh, "g .t). Hence, they are updated in the l-th EGNN layer follows. by du (clz ||x -x112)
Vi could further simplify Equation (4) as,
C² = E, Bxpixx (10(( Vxp loga (X} X6)1
(5)
2.2.2. DIFFUSION ON ATOM(BOND) TYPES (B*)))
The diffusion process of the atom bond types of the linker is similar to the one for the atom coordinates, but now we resort to multinomial diffusion (Hoogeboom et al., 2021). This is because unlike the atom coordinates data which lic in the continuous space, the atom (bond) type features of the linker are one-hot vectors as the discrete categorical data. We use atom types AL as an example from now on. The case for bond types B² follows the same manner.
1 = h/ +
Σ
on h/. 0.).1)
JEV"M(A
oly +
Σ Φn
(bk h txc,t) t)
Σ
&n (h), by esent)
REDPI(i)
NEVPND)
+ Σ (x)
Oz (1. .b
.0 ,4) Linker
(11
We use the categorical distribution for the forward process that has a Br chance of resampling a category uniformly.
JEVP
Here, Linker is the linker atom mask, a binary vector where each element indicates whether an atom belongs to the linker region or not. The initial atom and bond embeddings are encoded with simple multilayer perceptions (MI.P) based on the corresponding atom and bond types as hā -MLP(AAf [o]) and eb MLP(B/ [ūj)) respectively. A/ and B [ij] are the type of atom and the type of bond ij in the noise- noise-perturbed linker Gf, respectivell. The EGNN model de consists of three sets of parameters da. @he and Φz. which can all be instantiated with MLPs. The final embeddings {hr} and {e/} } with i.j E VL will also be fed into another two MLPs to obtain the predicted concrete scores co ( f) = MLP((h()) in Equation (9), and co (B t) = (L )). Similarly, we initialize x = =x! [ and set the score estimator sy (X(,,t) = (X/+)
+ (Af 1 = Cat (At: (1 - + (6)
N₂ is the total number of atom types. Due to the property of the Markov chain, the probability of any Af given A/ can be easily obtained in Equation (7).
(( ( Ab) Cat (A A/ + (1 (11-ā))1 an)/Na).
(7)
Here, we also have az = 1 - Br, at a The categorical posterior can be computed in the closed form as
N
g( q(4/11 1/, = Cat( AA 14-11:1:
II
Il,)
(8)
J0m1
II -I= + (1-1100 (1-aa))/N //a/ @ 14 (1-1100 (1-a)11)/aa a/No) To overcome the difficulty in predicting the noise for dis- crete atom type data, we utilize the denoising concrete score matching method (Meng et al., 2024). To generalize the score for the discrete setting of At. the concrete score of any distribution p(x) at data point is Cp(x) and is defined by the rate of change of the probabilities w.r.t. local directional changes of the input x within a neighborhood of (zx) =
2.3. Fine-tuning in the PROTAC Domain
During the fine-tuning phase (Figure 3), the proposed DAD- PROTAC model transfers knowledge from the pre- pre-trained diffusion model in the small molecule domain, and learn how to further generate samples aligned with the target distribution in the PROTAC domain. A straightforward ap- proach is directly retrain the model using the pre-trained weights as initialization. However, this naive method often leads to overfitting issues (Lutati & Wolf, 2023), as it tends to capture noise from the limited PROTAC training data samples rather than learning high-level, domain-specific features critical for PROTAC design (Gharbi & Mercado, 2024). Full fine-tuning of all parameters Φe further incurs significant computational overhead and prolonged conver- gence times (Xie et al., 2023). Freezing early layers of the EGNN denoising network can reduce computational costs and mitigate overfitting (Moon et al., 2022), but it still cannot explicitly learn high-level, domain domain-specific fea- tures necessary for better adaptability in the PROTAC linker design with theoretical guarantees.
(zn, 1/m 1i i.e., Op(z) = [ )-p(a)
p(a,,) )- pla)) p(=) J..
p(z)
p(z)
Similar to Equation (4), the training loss now becomes
L E. E EA [loo 1)
(444) (9)
The dynamics of predicting co (AF) will be elaborated next.
2.2.3. EQUIVARIANT GRAPH NEURAL NETWORKS FOR DENOISING OVER THE PERTURBED LINKER of
To incorporate the information from fragments for denoising Gf, we use the whole graph g = <V>,"" {²' as the input with VP- = F U F2 VV. We require that the atom embeddings h, (i E VP) and bond em- beddings eij (i,j E VP) output by neural network
4

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
2. CONNECTION WITH DENSITY RATIO IN THE CHEMICAL SPACE
Algorithm Fine-tuning Phase of DAD-PROTAC
input Samples from source small molecule domain ((
The fine-tuning process of the proposed DAD-PROTAC model is grounded in its theoretical connection to the den- sity ratio between PROTACs and small molecules in the chemical space, as shown in Theorem 2.1. Specifically, motivated by the previous work (Ouyang et al., 2024; Kim et al., 2024), we can show that the optimal score estimator for the target PROTAC domain during fine-tuning differs from the pre-trained score for the small molecule domain by correction term that accounts for the density ratio between these two domains in the chemical space. This correction term ensures the model adapts effectively to the distinct characteristics of PROTAC linkers.
and target PROTAC domain g(G*), pre-trained (fine- tuning) diffusion process q() g = q()) (
/ Estimate the density ratio via binary classifier W first */
1: repeat
2: Sample small molecule data G ~ g(G²) with size of m, and PROTAC data D ~ x(g")) with size of n.
3: Sample t ~ 7), and perturb the sampled data via diffiusion process g( ( = ( (t). Perform gradient descent step over the binary classi- fier W via the losses in Equations (16), (23) and (24).
5: until converged.
6: Set the density ratio in chemical space term appeared
Theorem 2.1. Let (()h) and q(G") denote the distribu- tions of linkers the chemical space of the small molecule domain and the PROTAC domain, respectively. Assume the forward diffusion process both domains is identical, i.e.
in Equations (13), (19) and (20) as ((0)) W(G..))
1-Wis/ .4)
1e Lear the score correction term for the fine-tuning phase *
7: repeat
8: Sample PROTAC data from 9b ~ (((²)) and perturb
G with noise via diffusion process q( (66).
g6)) ( = = (t))
(10)
9: Perform gradient descent step over ( t) via the losses in Equations (13), (19) and (20).
The score estimator 2z- (X.-: ,t) for the target PROTAC do- main during the fine-turing phase can be decomposed as
10: until converged
11: return Score estimator for the target PROTAC domain via Equations (15), (21) and (22).
((Xp)
83. (X/;t)= = Vxp log ( + +Vxp log E (X ((x))
pre- tre-irained score Junction
SCUVE correction
(11)
an additional neural network to approximate the density ra- tio term. This term, essentially a classifier as discussed later, substantially reduces computational costs compared to full fine
Similarly, the concrete score estimator cz- (At, 1) for the target PROTAC domain can also be decomposed as
the PROTAC domain. For another, the density correction term explicitly incorporates the density ratio between PROTACs and small molecules in the chem- ical space. By leveraging this density ratio, our method optimally fine-tunes the pre-trained model while highlight- ing the high-level distribution differences between the two domains. Therefore, our approach enhances model adapt- ability for PROTAC linker design with theoretical guidance.
(Af)) ( ( g(Ab))
Cg. (Af = (AFA + +CgAAp)
(12)
pre trained concrete acore Junction
comurete Correction
Theorem 2.1 highlights the critical role of the density ra- g(gh) tio ((Gz) in chemical space for effectively adapting the
pre-trained model to PROTAC linker design. This density x(x))
ratio serves as a correction term, V XxP log g (xF) ((x,)
2.3.2. LEARNING THE SCORE CORRECTION TERM WITH DENSITY RATIO ESTIMATION
((44)) )), allowing the proposed DAD-
(or CyCAF) (Eq(A) ((4))
PROTAC model to refine the score function V Vx2 log (({)))
We then aim to compute the (concrete) score correction terms in Equation (1I) and in Equation (12). We focus on the case of $ 3- (X t) in this section and the case of cz. (A/: t) follows the same idea discussed in Appendix B.4. Assume the density ratio 4(x,) is known, we can obtain the expec-
(or Cg(AA) (Af)) by the pre-trained model. This adjustment ensures that the model accurately captures the distributional differences between the two domains.
Decomposing the score estimator into two terms Theo- rem 2.1 offers two key advantages. For one thing, unlike conventional fine-tuning methods, our approach eliminates the need to directly fine-tune the pre-trained model on the target PROTAC domain. The score estimator is already learned during the pre-training phase, and we now only need to feed the new PROTAC training samples to obtain VX2. log logg ( (or CgLAg) (Af) as the fixed base score esti- mation, We instead shift the computational overhead to train
x(X))
tation 1, Ix!)
q(x,5 using Monte Carlo simulation, but it is hard to directly sample from x(X (XF) in the source small molecule domain during fine-tuning. Therefore, fol- lowing the previous work (Ouyang et al., 2024; Lu et al., 2023), we also build its equivalence to another score cor- rection term that is easier to sample in the target PROTAC domain g(g²).
5

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Theorem 2.2. If a parameterized neural network model 4 (XF,,) 1) takes the noise-perturbed X in the target PRO- TAC domain as the input and vo following loss as,
Our fine-tuning approach in the DAD-PROTAC model en- joys two main advantages. First, it is computationally effi- cient through the score estimator correction rather than full model retraining. Second, it explicitly estimates the density ratio in the chemical space with theoretical rigor for effec- tive domain adaptation. To sum up, the fine-tuning phase of DAD-PROTAC effectively captures the unique characteris- tics of PROTACs via the lens of density ratio in the chemical space with reduced computational overhead.
(X, 1) is trained with the
((X1000 7 2 (((())
C = B xxP)
(13)
then we have
q(Xb))
3 (xt = arg min - C* = 1x ( (
(14)
3. Experiments
3.1. Experimental Setup
From Theorem 2.2, we now build another neural network ((x))
We discuss the experimental setup for the proposed DAD- PROTAC model with more details in Appendix E.
(XX,,) t) to estimate (x(x(ix)) x)) (
by minimizing the loss function L involved with an easy sampling from the joint distribution of ((x6 X()) in the target PROTAC do- main. Instead of using the given limited PROTAC samples only (T(X))) in Equation (II)), we now use the generated noise-pert pestureeddd PROTAC samples (q(X,) in Equation (13)) to further improve the estimation of score correction term with more data samples. Therefore, the final score estimator for the target PROTAC domain can be calculated as follows.
1.1.1. DATASETS
To evaluate our model, we choose two datasets. One is a subset of ZINC (Irwin & Shoichet, 2005) from the small molecule domain for pre-training and the other one is PROTAC-DB (Weng et al., 2021; 2022) from the PRO- TAC domain for fine-tuning. The size of the PROTAC-DB dataset is significantly small compared to the ZINC dataset.
$3- (X,, Vxx. log ((xF) + Vxy log (X (15)
For the ZINC dataset during the pre-training phase, we follow the previous works (Huang et al., 2022; Igashov et al., 2024; Guan et al., 2023), and obtain the reference conformation for each molecule by running 20 times MMFF optimization (Halgren, 1996) using RDKit (Landrum, 2016) and selecting the one with the lowest energy. Then, these molecules are fragmented by enumerating all double cuts of acyclic single bonds that are not within functional groups. One molecule can therefore result in various combinations of two fragments with a linker between them. The pre- training dataset contains 438,610 samples.
To train (x('', t) via the loss in Equation (13), we still need to estimate the density ratio term x(x?" tributions perturbed at time t between PROTACs and small molecules in advance. Therefore, we need to train another binary classifier XXF, () beforehand via the cross-entropy loss in Equation (16) to determine whether the perturbed samples are from the source small molecule domain ((x)) or from the target PROTAC domain g( (.
((x/2 for the marginal dis-
Wx
1
Σ
Σ
log w(X .)
For the PROTAC-DB dataset, which is collected from the literature or calculated by programs, we follow the existing works (Li et al., 2024; Guan et al., 2023). We gather E3 ligands, the warheads (ligands that bind to targets), and the linkers of each PROTAC. We select 365 different warheads as the test set of 327 PROTAC samples, and the remaining as the training set of 2,943 samples for the fine-tuning phase.
m X x xxxx(X
(16)
Σ
Σ
log (1 w(xx .))
X)-1(()) XX-X(XXI
((x+) 1-
The density ratio is approximated as (x()) W(XF.0
The overall fine-tuning phase of DAD-PROTAC is summa- rized in Algorithm 1 with more details in Appendix D.2. It comprises two primary steps: density ratio estimation and score function correction, efficiently transferring knowledge from small molecules to PROTACs with thcoretical insights. In the density ratio estimation step, a binary classifier W is trained to quantify the distribution differences in chemical space between small molecules and PROTACs. In the score function correction step, we leverage the neural network té to approximate the optimal adjustments needed to adapt the pre-trained score function to the PROTAC domain.
3.1.2. BASELINES
For benchmarking. we compare our model with three baselines for the linker design task: 3DLinker (Huang et al., 2022), DiffLinker (Igashov et al., 2024), and Link- erNet (Guan et al., 2023). 3DLinker is the first 3D gen- erative model based on VAE models. DiffLinker is an E(3) -quivvanvva 3D conditional diffusion model. LinkerNet develops a 3D equivariant diffusion model that jointly gen- erates both fragment poses and the structure of the linker.
6

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Since they do not follow the pretrain-finetunnng pipeline, we build their pre-trained counterparts for fair comparison.
3.3. Ablation Study
We perform an ablation study on DAD-PROTAC by mod- ifying the fine-tuning components while keeping the pre- training phase unchanged. The results are summarized in Ta- ble 2. First, we evaluate standard full fine-tuning, which up- dates all parameters in the EGNN model to predict the score function. This traditional approach performs poorly due to overfitting. Partial fine-tuning, which updates only the decoder in EGNN, performs slightly better but still yields suboptimal results. Next, we test the direct score correction approximation, where the score correction term is learned via a neural network without using density ratio estimation. This approach achieves the lowest performance across all metrics, revealing that while computationally efficient, it oversimplifies the learning process. These results empha- size that density ratio estimation in the chemical space is critical for accurate score correction. Finally, we analyze the impact of using only fixed clean samples to estimate the density ratio, based on the fact that we use identical forward diffusion processes in both phases. This variant slightly degrades performance compared to DAD-PROTAC's use of noise-perturbed samples, as the latter is trained with a larger and more diverse dataset. Overall, the ablation study demonstrates the importance of each component in DAD- PROTAC, with density ratio estimation playing a pivotal role in achieving optimal performance.
3.1.3. EVALUATION METRICS
Similar to molecular generation tasks, We evaluate the gener- ated molecules using both 2D graphs and 3D conformations. We generate 100 PROTAC linker samples per fragment pair in the test set for the evaluation. For 2D metrics, we report standard metrics including validity, uniqueness, nov- elty (Brown et al., 2019), recovery rate, and property-related metrics including drug-likeness (QED) (Bickerton et al., 2012) and synthetic accessibility (SA) (Ertl & Schuffen- hauer, 2009). For 3D conformations, we perform MMFF optimization (Halgren, 1996) and report two metrics: the average minimum energy per fragment pair before optimiza- tion (Emia), reflecting the overall quality of generated con- formations, and the Root Mean Square Deviation (RMSD) of molecular coordinates before and after optimization, rep- resenting the deviation from optimal conformations.
3.2. Main Experimental Results
For fair comparison, we evaluate both bascline models trained from scratch on the PROTAC-DB dataset and their pre-trained counterparts. In the latter case, baselines are first pre-trained on the small molecule ZINC dataset and then fine-tuned with limited PROTAC data. The results in Table 1 yield three key observations. First, fine-tuned variants consistently outperform their base models, high- lighting the benefits of mre-training on the large-scale ZINC dataset. Second, DAD-PROTAC achieves the best overall performance across most metrics, notably the highest va- lidity and recovery rates. Third, while DiffLinker models perform well in uniqueness and novelty, they often generate chemically invalid structures, making DAD-PROTAC the most practical and robust model overall.
3.4. Understanding DAD-PROTAC
To illustrate the impact of fine-tuning in DAD-PROTAC, we evaluate the model's performance on the PROTAC dataset immediately after pre- pre-training without fine-tuning, Results in Figure 4(b) show the distribution of total generated PRO- TAC linkers according to the number of atoms and the pro- portion of invalid generated PROTAC structures. Compared to the results of the fine-tuned DAD-PROTAC model in Fig- ure 4(c), fine-tuning clearly yields significant improvements, with remarkably fewer invalid PROTAC structures across all possible linker lengths or the number of atoms in the linker. This is because PROTACs exhibit distinct characteristics compared to traditional small molecules in the pretraining ZINC dataset. We demonstrate that the fine-tuning phase in DAD-PROTAS effectively enhances the model's under- standing of the divergence in characteristics of two datasets.
Furthermore, we compare the fine-tuning time of DAD- PROTAC with other methods using direct fine-tuning meth- ods. The relative fine-tuning times, with DAD-PROTAC as the baseline, and the valid rate as the efficacy metric, are summarized in Figure 4(a). DAD-PROTAC achieves su- perior performance with minimal computational overhead, maintaining the highest validity score. In contrast, tradi- tional fine-tuning methods require significantly more re- sources, with computational costs ranging from 1.35x to 2.8x that of the proposed DAD-PROTAC model. These results highlight the efficiency of our insightful fine-tuning strategy, which learns to approximate the score correction term under the guidance of density ratio in chemical space, rather than re-training the entire model. In summary, our approach delivers both high performance and exceptional computational efficiency.
Additionally, Figure 5(a) compares the molecular weight distributions of true PROTACs in the test set with those of generated PROTACS. The high degree of overlap highlights the fine-tuned model's ability to generate PROTACs that closely align with the chemical space of true PROTACs, underscoring the importance of density ratio estimation in the chemical space during fine-tuning.
7

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Table 1: Performance metrics for generated PROTAC linkers the PROTAC-DB dataset.
Pre-trained
Model
Valid (
Unique (t)
Novel (1)
Recover (t)
QED (1)
(4)
Emin (4)
RMSD (U)
ZINC
3DLinker
No
21.4 0.2
25.8 0.3
27.3 0.1
3 0.2
0.44
2.98 + 0.3
$31.6 12.7
1.97 + 0.3
JDLinker-Fime-tume
Yes
58.6 0.2
49.2 0.6
$6.2 0.5
24.3 + 0.4
0.58 #.2
2.1
326.4 10.2
1.81 0.4
DiffLinker
No
0.1
9.14
98.5
0.1
0.3
2.2
420.2
0.1
Yes
6.3
98.8
99.3
0.3
0.2
2.4
253.0
2.24
LinkerNet
No
0.1
46.8
4.3
0.9
0.11
2.2
6.3
1.3
LinkerNet-Fiine
Fine-tuninng
Yes
0.3
54.6 7.0
63.7
32.8
0.69
0.11
110.7
0.2
DAD-PROTAC
Yes
94.8
69.3 0.3
71.5 0.3
45.7 +
0.74 0.1
1.63 0.1
9.54
Table Ablation study for the proposed DAD-PROTAC model.
Model
Valid %
Unique (f)
Novel% (1)
Recover%
Fiain (J)
RMSD (J)
DAD-PROTAC standard full fine-tuning
82.5 0.4
57.0
63.1 +..
022
117.4 10.4
1.66
DAD-PROTAC stundard partial line-tuning
0.6
5.84
62.2 0.4
3..
107.0 10.2
1.51
DAD-PROTAC w/ direct score correction approximation
0.8
4.4
59.7 0.3
2
203.8 12.3
2.37
DAD-PROTAC w/ density ratio estimation via clean samples PAD-PROTACC
90.0 4.70
62.8
69.3 0.2
4
4.9
94.8 0.4
6 0.3
71.5 0.3
45.7 0.6
92.4 9.5
1.43 0.2
100
DAD-PROTAC: 1x
0.12
0.12
Total Num Not Valid
Total Num
90
0.10
0.10
Not Valid
LnkerrNet-fine weatteet--ie--uiinng 2.4x
0.08
0.08
80
DAD-PROTAC- Finetnn------uuiingg 2.8x
Density
Density
0.06
0.06
70
DiffLinker- Feeeiinne--F----eeeee -tuning: 1.73x
0.04
0.04
0.02
0.02
aDLinker Linkter-ine--uiing: 1.35x
0.00
0.00
5
10
15
20
25
30
35
0
10
15
20
25
30
35
50
Number of Atoms the PROTAC, Linker
Number of Atoms in the PROTAC Linker
Relative Fine-tuning
(b): Before fine-tuning phase
(c): After fine-tuning phase
(a): Efficiency of fine-tuning phase
Figure 4: (a): Fine-tuning time comparison. (b)(c): Distribution of generated PROTAC linkers by DAD-PROTAC.
-10-3
True Generated
2.4
Density
1.8
0.6
0.0
Molecular Weight (Da)
Fragments
Reference PROTAC
PROTAC by Ours
PROTAC by LinkerNet
(a): Molecular weight
(b): Visualization of the reference and generated PROTACs given the fragments
Figure 5: (a): Distribution of molecular weight of the test and generated PROTACs. (b): Visualization results.
3.5. Visualization
LinkerNet based on the SCRDKit score. Our model pre- cisely predicts the correct atoms and bonds, greatly match- ing the reference linker, whereas LinkerNet produces incorrect structure.
Finally, we visualize examples of PROTAC linker design in Figure 5(b). Given fragment pair from the test set, we select the best-generated PROTAC from method and
8

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
4. Conclusion
says. Journal of medicinal chemistry, 53(7):2719-2740. 2010.
In this work, we propose DAD-PROTAC, a domain-adapted diffusion model for PROTAC linker design. To address the distribution mismatch between small molecules and PROTACs, DAD-PROTAC decomposes the target score estimator into pre-trained score function and a lightweight score correction term under the guidance of the density ratio in chemical space, DAD-PROTAC enables efficient fine-tuning, mitigating the overfitting issue.
Bemis, T. A., La Clair, J., and Burkart, M. D. Unraveling the role of linker design in proteolysis targeting chimeras: Miniperspective. Journal of Medicinal Chemistry, 64(12):
(042-8052, 2021.
Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S., and Hopkins, A. L. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
Despite its strengths, DAD-PROTAC has several limita- tions. First, it assumes that the number of atoms in the linker is pre-specified, which is unrealistic for real-worle applications. Second, the pre-training phase does not con- sider fragment rotation or protein context. These limitations could potentially be addressed by integrating more advanced diffusion-basee models during the pre- pre-training phase.
Brown, N., Fiscato, M., Segler. M. H., and Vaucher, A. C. Guacamol: Benchmarking models for de novo molecu- lar design. Journal of Chemical Information and Mod- eling, 59(3):1096-1108 2019. doi: 10.1021/acs, 8b00839, URL https: http:/oo.ooooooo///////ooo /1/ 10100021accssss joim. 8b00839. PMID: 30887799.
Chamberlain, P. P. and Hamann, L. G. Development tar- geted protein degradation therapeutics. Nature chemical biology, 15(10):937-94 2019.
Acknowledgements
Zixing Song and José Miguel Hernández-Lobato ac- knowledge support from EPSRC funding under grant EP/Y028805/1. José Miguel Hernández-LLobat also ac- knowledges support from a Turing Al Fellowship under grant EP/V023756/1.
Chen, X., Li, H., Tian, L.,, Li, Q.. Luo, J., and Zhang, Y. Analysis of the physicochemical properties of acaricides based on lipinski's 's rule of five. Journal of computational biology, 27(9): 139--146, 2020.
Corso, G., Stärk, H., Jing, B., Barzilay, R., and Jaakkola, T. S. Diffdock: Diffusion steps, twists, and turns for molecular docking. In The Eleventh Interna- tional Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, htt.:/eee/eeeee forum?
Impact Statement
Our work advances computational methods for PROTAC design, offering potential acceleration in therapeutic devel- opment for challenging diseases. This technology could sig- nificantly reduce drug development timelines and costs, par- ticularly benefiting areas where traditional small molecule approaches have proven insufficient. However, we acknowl- edge that powerful molecular generation tools carry inherent risks of potential misuse. To address these concerns, we rec- ommend implementing safeguards including toxicity predic- tion filters and restricted chemical space exploration using well-characterrized building blocks. We advocate for trans- parent reporting of both capabilities and limitations, and encourage ongoing collaboration between computational scientists, medicinal chemists, and bioethicists to ensure these advances benefit global health while minimizing po- tential risks.
2023. URL _d-kKF8_ K-mBbS.
Denker, A., Vargas, F., Padhy, S., Didi, K., Mathis, S., Dutordoir, V., Barbano, R., Mathieu, E., Komorowska, U. J., and Lio, P. Deft: Efficient fine-tuning of diffusion models by learning the generalised h- h-trrasffrmm 2024. URL https: / aaw.v.iiiiv org/ abs / 406.6066600 01781.
Du, W., Chen, J., Zhang, X., Ma, Z., and Liu, S. Molecule joint auto-encoding: Trajectory pretraining with 2d and 3d diffusion. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
References
Alakhdar, A., Poczos, B., and Washburn, N. Diffusion models in de novo drug design. Journal of Chemical Information and Modeling, 64(19):7238-7255 2024.
Erlanson, D. A., McDowell, R. S., and O'Brien, T. Fragment-based drug discovery. Journal of medicinal chemistry, 47(11))3463-3444 2004.
Ertl, P. and Schuffenhauer, A. Estimation of synthetic acces- sibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of chem- informatics, 1:1-11, 2009.
Baell, J. B. and Holloway, G. A. New substructure filters for removal of pan assay interference compounds (pains) from screening libraries and for their exclusion bioas-

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Gharbi, Y. and Mercado, R. A comprehensive review of emerging approaches in machine learning for de novo protac design. Digital Discovery, 3:2158-2176, 2024. doi: 10. 10.1039/D4DD0011711. URL http://dx.dd. 100.11101100//0D000111
Huang, L.,, Xu, T., Yu, Y., Zhao, P., Chen, X., Han, J., Xie, Z.,, Li, H., Zhong, W., Wong, K.-C., et al. A dual diffusion model enables 3d molecule generation and lead optimiza- tion based on target pockets. Nature Communications, 15 (1):2657, 2024.
Benba,-l R., Wei. J. N., Duvenaud. D., Hernández-Lobatte J. M., Sánchez- Sénchez-Lengeling, B., She- berla, D., Aguilera-Iparragurnee. J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzik, A. Automatic chemical de- sign using data-driven continuous representation of molecules. ACS central science, 4(2):268-217 276, 2018.
Huang, Y., Peng, X., Ma, J., and Zhang, M. 3dlinker: An E(3) equivariant variational autoencoder for molecular linker design. In ICML, volume 162 of Proceedings of Machine Learning Research, pp. 9280-9294. PMLR, 2022.
Igashow, I., Stärk, H., Vignac, C., Schneuing, A., Satorras, V. G., Frossard, P., Welling, M., Bronstein, M., and Cor- reia, B. Equivariant 3d-conditional diffusion model for molecular linker design. Nature Muchine Intelligence, pp. 1-11, 2024.
Guan, J., Peng, X., Jiang, P., Luo, Y., Peng, J., and Ma, J. Linkernet: Fragment poses and linker co-design with 3d equivariant diffusion. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 77503-77519 Curran Associates, Inc., 2023.
Imrie, F., Bradley, A. R., van der Schaar, M., and Deane, C. Deep generative models for 3d linker design. Jour- nal of chemical information and modeling, 60(4):1983- 1995, 2020.
Halgren, T. A. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmf194. Journal of computational chemistry, 17(5-6):40--51, 1996.
Irwin, J. J. and Shoichet, B. K. Zinc- a free database of commercially available compounds for virtual screening. Journal of chemical information and modeling, 45(1): 177-182, 2005.
Han, L., Li, Y., Zhang, H., Milanfar, P., Metaxas, D. N., and Yang, F. Svdiff: Compact parameter space for diffusion fine-tuning. In IEFE/CVE International Conference Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 7289-7300. IEEE, 2023. doi: 10.1 10.1 ICCVS1070.2023.00 202.00033.. URL https: / //woi.oooztt org/ 10. 1009 www11199//cvv./
Jin, W., Sarkizova, S., Chen, X., Hacohen, N., and Uhler, C. Unsupervised pootein-ligand binding energy prediction via neural euler's rotation equation. In NeurIPS, 2023.
Jing, B., Corso, G., Chang, J., Barzilay, R., and Jaakkola, T. S. Torsional diffusion for molecular conformer genera- tion. In NeurIPS, 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural In- formation Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Kim, Y., Na, B., Park, M., Jang, J., Kim, D., Kang, W., and Moon, I. Training unbiased diffusion models from biased dataset. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https: / /ppeneevi ew. net. fortt//orumiiico 3c jBed.
Hoogeboom, E., Nielsen, D., Jaini, P., Forré, P., and Welling, M. Argmax flows and multinomial diffusion: Learning categorical distributions. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, vir- tual, pp. 12454-12465, 2021.
Landrum, G. Rdkit: Open-source cheminformatics soft- ware. 2016. URL https: com/ htt/// rolkt//releasss/tta//elleeee
2016_09_4,
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Hoogeboom, E., Satorras, V. G., Vignac, C., and Welling, M. Equivariant diffusion for molecule generation in 3d. In Chaudhuri, K., Jegelka, S., Song, L.,, Szepesvári, C., Niu, G., and Sabato, S. (cds.), International Conference on Ma- chine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 8887 PMLR, 2022.
Li, F., Hu, Q., Zhou, Y., Yang, H., and Bai, F. Diffpproacs is a deep learning- based generator for proteolysis targeting chimeras. Briefings in bioinformatics, bbbs)bb 2024.
10

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Liang, L., Xu, Z., Song, Z.,, King. I., and Ye, Resnorm: Tackling long-tailed degree distribution is- sue in graph neural networks via normalization. CoRR, abs/2206.0818/ 2022.
Neeser, R. M., Akdel, M., Kovtun, D., and Naef, L. Rein- forcement learnig-drrveen linker design via fast attention- based point cloud alignment. In ICML 2023 Work- shop on Structured Probabilistic Inference & Generative Modeling, 2023. URL https: opwpreveeweeeeeeeoe net// forum? OFCTYYTG...
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization. In ICLR (Poster). OpenReview.net, 2019.
Lu, C., Chen, H., Chen, J., Su, H., Li, C., and Zhu, Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference Machine Learning, ICML 2023, 23-29 July 2023, Hon- olulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 22825-22855. PMLR, 2023. URL https: htt////oooeediggge mlr. prrss// v 10022/uu230 Lu23d. html.
Nichol, A. Q. and Dhariwal, Improved denoising dif- fusion probabilistic models. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Con- ference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Mu- chine Learning Research, pp. 8162-8171. PMLR, 2021. URL http: http:://ooceeeddec hgs. mlr, wtt.::mw......r..... ptwss/v1sss//////// nichol21a. htthollla..ml....
O' O'Boyle, N. M., Banck, M., James, C. A., Morley, C., Van- dermeersch, T., and Hutchison, G. R. Open babel: An open chemical toolbox. Journal of cheminformatics, 3: 1-14, 2011
Luo, S., Su, Y., Peng, X., Wang, S., Peng, J., and Ma, Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures. In NeurlPS, 2022.
Ouyang, Y., Xie, L., Zha, H., and Cheng, G. Transfer learning for diffusion models. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomezak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver BC, Canada, December 10 15, 2024, 2024.
Luo, W., Hu, T., Zhang, S., Sun, J., Li. Z., and Zhang, Z. Diff-instruct: A universal approach for transfer- ring knowledge from pre-trained diffusion models. In NeurlPS, 2023.
Lutati, S. and Wolf, OCD:
learning to overfit with condi- tional diffusion models. In ICML, volume 202 of Proceed- ings of Machine Learning Research, pp. 23157-23169. PMLR, 2023.
Qiang, B., Shi, W., Song, Y., and Wu, M. Proflow: An iterative refinement model for protac-induced structure prediction. arXiv preprint arXiv:2405.0665. 2024.
Ma, Y., Song, Z., Hu, X., Li, J., Zhang, Y., and King, I. Graph component contrastive learning for concept relat- edness estimation. In AAAI, pp. 13362-13323 AAAI Press, 2023.
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pp. 22500 22500-22510. IEEE, 2023.
Martín-Acosta, P. and Xiao, X. Protaes to address the chal- lenges facing small molecule inhibitors. European jour- nal of medicinal chemistry, 210:1 12993, 2021.
Sakamoto, K. M.. Kim, K. B., Kumagai, A., Mercurio, F., Crews, C. M., and Deshaies, R. J. Protacs: Chimeric molecules that target proteins to the skpl-cullin-f box complex for ubiquitination and degradation. Proceedings of the National Academy of Sciences, 98(15))85548855, 2001.
Meng, C., Choi, K., Song, J., and Ermon, S. Concrete score matching: generalized score matching for discrete data. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9781713871083
Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equiv- ariant graph neural networks. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Ma- chine Learning Research, pp. 9323-9332. PMLR, 18-24 Jul 2021.
Moon, T. Choi, M., Lee. G., Ha, J.-W., and Lee, J. Fine-tuning diffusion models with limited data. In NeurIPS 2022 Workshop on Score-Based Methods, 2022. URL httph:t//////. http://ppeneeee..ee...eee 1d= RDqRR.
Schapira, M., Calabrese, M. F., Bullock, A. N., and Crews, C. M. Targeted protein degradation: expanding the tool- box. Nature reviews Drug discovery, 18(12):949995 2019.
Morehead, A. and Cheng, J. Geometry-complete diffusion for 3d molecule generation and optimization. Communi- cations Chemistry, 7(1):150, 2024.

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Schneuing, A., Harris, C., Du, Y., Didi, K., Jamasb, A., Igashov, I., Du, W., Gomes, C., Blundell, T. L.., Lio, P., et al. Structure-based drug design with equivariant diffusion models. Nature Computational Science, 4(12): 899-909, 2024.
(D1):D1367-D1372, 10 2022. ISSN 0305-1048, doi: 10.1095/mar//kaac446 URL https://doi org///1000....///.//// 1093/naz/gkac946
Xie, E.,, Yao, L., Shi, H., Liu, Z., Zhou, D., Liu, Z., Li, J., and Li, Z. Difffit: Unlocking transferability of large diffu- sion models via simple parameter-efficien fine-tuning. In IEEE/CVF International Conference Computer Vision ICCV 2023, Paris, France, October 1-6, 2023, pp. 4207- 4216. IEEE, 2023. doi: 10.1 10.11099ICCV51107222330000 URL https: //t::://doooooo org/ 1199//1//1110/00000 2023.00392 00390.
Song, 7-,, Ma, Y., and King, I. Individual fairness in dynamic financial networks. In NeurlPS 2022 Workshop: New Frontiers in Graph Learning, 2022.
Song. Z., Zhang, Y., and King, No change, no gain: Empowering graph neural networks with expected model change maximization for active learning. In NeurlPS, 2023a.
Xu, M., Powers, A. S., Dror, R. O., Ermon, S., and Leskovec, J. Geometric latent diffusion models for 3d molecule gen- eration. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, (eds.), International Con- ference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceed- ings of Machine Learning Research, pp. 38592-38610 PMLR, 2023. URL http::tttttttteeeeeeeeee httpee///occceedeeeeeeeeo mmlr. press/v209 xu2 ww22nnhttm... html.
Song. Z., Zhang, Y., and King, I. Optimal block-wise asymmetric graph construction for graph-based semi- supervised learning. In NeurlPS, 2023b.
Song. Z., Meng, Z., and King, A ceffusion-based pre- training framework for crystal property prediction. In AAAI, pp. 8993-9001. AAAI Press, 2024.
Song, Z., King, I., and Hernández-Lobate J. M. Fededm: Federated equivariant diffusion model for 3d molecule generation with enhanced communication efficiency. WWW (Companion Volume). pp. 1752-1756. ACM, 2025.
Yang. Y., Zheng, S., Su, S., Zhao, C., Xu, J., and Chen, H. Syntalinker: automatic fragment linking with deep condi- tional transformer neural networks. Chemical science, 11 (31):8312-8322 2020.
Troup, R. I., Fallan, C., and Baud, M. G. Current strate- gies for the design of protac linkers: a critical review. Exploration of Targeted Anti-tumor Therapy, 1(5):773, 2020.
Zeng, W., Yan, Y. Zhu, Q.. Chen, Z., Chu, P., Zhao, W., and Yang, X. Infusion: Preventing customized text-to- image diffusion from overfitting. In ACM Multimedia, pp. 3568-3577, ACM, 2024.
Uehara, M., Zhao, Y., Black, K., Hajiramezanali, E., Scalia, G., Diamant, N. I.,, Tseng, A. M., Levine, S., and Bian- calani, Т. Feedback efficient online fine-tuning of diffu- sion models. In Forty-first International Conference on Machine Learning, ICMI. 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net 2024. URL https: //openreview.coo htt/:/nee///oot/to forum? d=dtV1c9ybTmm
Zhang, M., Qamar, M., Kang, T., Jung, Y., Zhang, C., Bac, S.-.., and Zhang, C. A survey graph diffusion mod- els: Generative ai in science for molecule, protein and material. arXiv preprint arXiv:2304.01565, 2023.
Zhang, Z., Zhao, Z., and Lin, Unsupervised represen- tation learning from pre-trained diffusion probabilistic models. In NeurIPS, 2022.
Vincent, A connection between score matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Zhao, L., Zhao, J., Zhong, K., Tong, A., and Jia, D. Tar- geted protein degradation: mechanisms, strategies and application. Signal transduction and targeted therapy, 7 (1):113, 2022.
Weininger, D. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-366 1988.
Zheng, S., Tan, Y., Wang, Z., Li, C., Zhang, Z., Sang, X., Chen, H., and Yang, Y. Accelerated rational protac design via deep learning and molecular simulations. Nature Machine Intelligence, 4(9):739-744 748, 2022.
Weng, G., Shen, C., Cao, D.. Gao, J., Dong, X., He, Q., Yang, B., Li, D., Wu, J., and Hou, T. Protac-db: an online database of protacs. Nucleic acids research, 49 (DI):D1381-D1333 2021.
Zheng, X., Huang, X., Mei, G., Hou, Y., Lyu, Z., Dai, B., Ouyang, W., and Gong, Y. Point cloud pre-training with diffusion models. In CVPR, pp. 22935-22945. IEBE, 2024.
Weng, G., Cai, X., Cao, D., Du, H., Shen, C., Deng, Y., He, Q., Yang, B., Li, D., and Hou, T. Protac-db 2.0: an updated database of protacs, Nucleic Acids Research,
12

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
POI
E3 ligase
linker
POI warhead
E3 warhead
Figure 6: PROTACs are molecules composed of two "warheads" and a connecting linker. The warheads bind to the E3 ligase and protein of interest (POI), while the flexible linker brings the two proteins into proximity (Qiang et al., 2024).
A. Related Work
A.1. Linker Design
A.1.1. MOLECULAR LINKER DESIGN
Molecular linker design represents fundamental challenge in rational drug design, with applications ranging from fragment- based drug discovery (Erlanson et al., 2004) to targeted protein degrader degradation (Chamberlain & Hamann, 2019), Molecular linker design addresses the challenge of connecting molecular fragments through optimal linking groups while preserving or enhancing the desired molecular properties.
Early approaches like SyntaLinker (Yang et al., 2020) treated linker design as a sequence completion problem using SMILES representation (Weininger, 1988). While innovative, this approach suffered from inherent limitations due its reliance on ID molecular representations and the absence of crucial 3D structural information.
Subsequent methods addressed these limitations by incorporating spatial information through graph-based approaches, DeLinker (Imric et al., 2020) marked significant progress by utilizing molecular graphs and incorporating basic geometric constraints such as inter-atomic distances and angles between anchor atoms. However, these methods still operated primarily in 2D space, limiting their ability to capture the full complexity of molecular interactions.
The field has recently witnessed a paradigm shift toward direct 3D linker generation. Methods like 3DLinker (Huang et al., 2022) and DiffLinker (Igashov et al., 2024) leverage advanced deep learning architectures conditional VAE and diffusion models, respectively to generate linkers directly in 3D space. LinkerNet (Guan et al., 2023) further operates under the assumption of known fragment poses.
A.1.2. PROTAC LINKER DESIGN
PROTAC (PROteolysis TArgeting Chimeras) linker design presents distinct challenges that set apart from traditional molecular linker design. Since the first proof-of-concept study (Sakamoto et al., 2001), PROTAC development has emerged as promising therapeutic strategy, distinguished by its ability to induce protein degradation rather than mere inhibition.
PROTAC linker design represents specialized subset of the molecular linker design problem with distinct requirements and additional complexity. As shown in Figure 6, a PROTAC molecule consists of three key components: a target protein ligand, an E3 ligase ligand, and linking structure. The design task requires generating linker that connects these two ligands while enabling the formation of functional ternary complex with both target proteins.
Traditional PROTAC linker design has largely relied on empirical optimization strategies, typically utilizing a limited repertoire of chemical motifs (Troup al., 2020). This empirical approach, while pragmatic, fails to fully exploit the
13

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
potential chemical space and may miss optimal linker configurations.
Recent attempts to rationalize PROTAC design through computational methods, such deep reinforcement learning approaches (Zheng et al., 2022; Neeser et al., 2023), represent important steps forward. However, these methods still primarily operate in SMILES space rather than 3D space, limiting their ability to account for the complex spatial requirements of ternary complex formation.
A.2. Difffusion Generative Models
Diffusion models have emerged as powerful generative models for molecular generation tasks, particularly in the context of 3D molecular structures (Alakhdar et al., 2024; Song et al., 2025; 2024). Inspired by nonequilibrium statistical physics, these models can generate 3D molecular structures with specific properties or requirements crucial to drug discovery. Diffusion models were particularly successful at learning the complex probability distributions of 3D molecular geometries and their corresponding chemical and physical properties, such as conformation generation (Jing al., 2022), molecular docking (Corso et al., 2023), antibody design (Luo et al., 2022) and protein-ligant binding affinity prediction (Jin et al., 2023).
The first deployment of diffusion models for 3D molecular generation is the E(3) Equivariant Diffusion Model (EDM) (Hooge- boom et al., 2022), where they use an E(n) equivariant graph neural network (EGNN) (Satorras et al., 2021) developed originally for discriminative tasks to denoise and learn molecular structure distributions. Subsequently, they become widely adapted for 3D molecular generation tasks combined with GNNs or transformer-basse models for encoding and learning molecule structures (Zhang et al., 2023).
A.3. Fine-tuning Diffusion Models
Fine-tuning pre-trained diffusion models on limited data from the target domain presents significant challenges, particularly in maintaining generation diversity while avoiding overfitting. Several approaches have been proposed to address these limitations. A novel time-aware adapter architecture within the attention blocks is developed (Moon et al., 2022), which substantially reduces the number of trainable parameters. TGDP further reduces the sample complexity for fine-tuning the pre- pre-trained diffusion model via an additional guidance network for transfer learning (Ouyang et al., 2024). The work in selective parameter fine-tuning (Xie et al., 2023) also demonstrates success by restricting updates to specific model components, including bias terms, class embeddings, normalization layers, and scale factors.
While existing tne-ttnnng methods for diffusion models have proven effective in image generation (Han et al.. 2023; Xie et al., 2023; Ruiz et al., 2023; Zheng al., 2024; Luo et al., 2023; Zhang al., 2022; Denker et al., 2024; Uehara et al., 2024; Zeng et al., 2024), they face fundamental limitations in molecular linker design. Image-basets based approaches often employ techniques such as pairwise similarity loss and high-frequency component regularization to enhance diversity. However, unlike image generation-whert minor visual artifacts may be ooleraleemmolcclla structures require strict chemical validity. Regularization strategies designed for image diversity do not inherently preserve valid chemical structures. Moreover, molecular conformations, particularly in PROTACs, involve complex geometric constraints chemical space that differ fundamentally from feature spaces in image-based models. MoleculeJAE (Du et al., 2023) addresses general molecule generation through self-supervised learning during pre-training, capturing inherent chemical structures. However, the intrinsic differences between small molecules and PROTACs still pose challenges for its adaptation to PROTAC linker design.
B. Preliminaries and Details of the Proposed DAD-PROTAC Model
In this section, we present some preliminaries and details of our proposed model DAD-PROTAC.
B.I. Small molecules and PROTACs
1.I.1. DIFFERENCE IN CHEMICAL SPACE
PROTACs, with their bifunctional nature and larger size, present distinct challenges in linker design compared to traditional small molecules. Their higher molecular weights and increased conformational flexibility are critical for effective function. Notably, the physicochemical properties and pharmacokinetic (PK) profiles of PROTACs differ substantially from those of conventional small molecules. Key differences in chemical space include molecular weight (MW), partition coefficient
14

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Small Molecule
Training Sample [ G",g
Fragment g
Fragment G
via cut of bonds
Linker gb
via silico experiments
Training Sample 1g Seggiigg Gf+}
PROTAC
Fragment 9%
Linker G²
Fragment FP2
(a): Small Molecule Datasets
(b): PROTAC Datasets
Figure 7: Differences between how small molecule datasets and PROTAC datasets are collected.
(LogP), rotatable bond count (flexibility and conformational dynamics), hydrogen bond donors (HBDs) and acceptors (HBAs), and carbon atom count.
As shown in Figure 2. small molecules typically fall within the 250-500 Da range, aligning with Lipinski's Rule of Five (Chen et al., 2020), which favors oral bioavailability. In contrast, PROTACs peak at 750-1000 Da, emphasizing their significantly larger size. The small-molecule distribution is narrow and sharply peaked, indicating a relatively uniform MW range, whereas the broader PROTAC distribution reflects greater variability in molecular size. This distinct separation underscores fundamental difference between PROTACs and traditional small molecules: their larger and more diverse structural profiles.
We include more results and analysis on key molecular descriptors in Appendix F.1 to comprehensively characterize the distinct properties of PROTACs compared to traditional small molecules. These molecular descriptor analyses highlight the fundamental differences between PROTACs and traditional small molecules in the chemical space, providing guidelines for PROTAC linker design.
B. 1.2. DIFFERENCE IN DATA COLLECTION
The construction of training datasets for small molecule linkers and PROTAC linkers differs significantly, as illustrated in Figure 7. For small molecules, linker extraction typically follows systematic fragmentation approach, where acyclic single bonds outside functional groups are algorithmically identified and double-cut to generate fragment-linker pairs. This automated strategy produces large, diverse datasets that comprehensively represent the chemical space of drug-like molecules.
In contrast, PROTAC linker datasets are curated through more specialized and deliberate methods. Rather than retrospective fragmentation, PROTAC linkers are primarily sourced from two main channels: in silico design experiments and manual extraction from published literature. This distinction reflects the rational, design-driven nature of PROTAC development. Unlike small molecule linkers, which benefit from large-scale historical databases, PROTAC linker datasets are inherently smaller and represent engineered solutions in the real world. These fundamental differences in data availability and collection methods must be carefully considered when developing domain adaptation models for PROTAC linker design.
B.2. E(n) Equivariant Graph Neural Networks
E(n) Equivariant Graph Neural Network (EGNN) (Satorras et al., 2021) is special type of graph neural network (Song et al., 2023b; Ma et al., 2023; Liang et al., 2022; Song et al., 2023a; 2022) that satisfies the equivariant constraints specified as follows. We consider point clouds X (x1, N E IR N3 with corresponding features H (hı.... .x) E R Xxf, where nf the dimension of the node feature. The features are invariant to group transformations, while the positions X
15

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
equivariant rotations, reflections, and translations. We define the function (Z 2n. [(X, H) be equivariant if for all R and have
2Z + Zn = f(RX + +t,
Considering the interactions among all atoms, we assume fully connected graph with nodes by E V. Each node vj associated with coordinates I; IR and features hRRd EGNN consists of composition of multiple Equivariant Graph Convolutional Layers (EGCL), Le., X H'+ EGCL[X', H') which are defined as follows.
maj = de (h2 Mr, dig. aij)
1 on 11. Σ
Eiymis
z Ox (n1, n2y dzyass) asj)
2/
x +
Σ j
dig
+
Here, indexes the layers of EGCL. d,j = || 22/1 is the Euclidean distance between nodes U, and vy. 0zj denotes optional edge attributes. The attention mechanism is used to infer soft estimation of the edges é. (inr(mas). ). All these learnable functions are parameterized by fully connected neural networks. Therefore, with this definition, the entire architecture of EGNN is composed EGCN layers as XX EGNN [xO. Ho..
B.3. Concrete Score
The concrete score is a generalization of the (Stein) score for discrete settings (Meng et al., 2024). Given a predefined neighborhood structure, the Concrete score of any input is defined by the rate of change of the probabilities with respect to local directional changes of the input. This formulation allows us to recover the (Stein) score in continuous domains when measuring such changes by the Fuclidean distance.
To be more precise, let p(x) be the data distribution over X. We denote N : X - * as the function mapping each sample X' to set of neighbors, such that (xx) ! Imn } and K [M(())] This neighborhood induces particular graphical structure onto the support of p(r)), which is called the neighborhood-induce graph G. Formally, G results from adding directed edge from x to each node in its neighborhood set xn, E (x) for all E supp(p(<)).
The Concrete score is constructed as the rate of change the probabilities with respect to these local directional changes in the input as defined below.
Definition B.1. (Concrete score) Let M be function mapping each data point to its set of neighbors ((x)
(Kay: ). Then the concrete score (x())()) X Rl((a) for given distribution p(x) evaluated at æ is given as
(x(m)(z) = pp(mnn) - p(x)
p(ang) - -((=)]
p(x)
p(x)
Similar denoising score matching for estimation (Vincent, 201 1), denoising counterpart for concrete score estimation is named denoising concrete score matching (Meng et al., 2024). Given discrete data distribution p(x) and a discrete noise distribution q(ī r), the perturbed data distribution p(ū) = Σx p(x)q(ī z), and the posterior
q(x: () (())) Then, the concrete score of the perturbed data distribution (()) can be obtained via ()))
(())
Σn (q(a)=) (z) i). This property leads to the following objective.
Theorem B.2. If the following objective is minimized.
DDCMM (0) p(())((a) (z) Cg(a)z) 13)
we have co co) = Cp(z) ( (8).
16

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
B.4. Learning the Concrete Score Terms
Recall that each PROTAC linker can be represented triple G (x=, AA, In Section 2.3.2, we have discussed how to learn the concrete score correction term for X for the target PROTAC domain during the fine-tuning phase. In this section, we will discuss how to learn the concrete score correction terms for A and B² in the same manner.
We show that the concrete score estimators cz- (Af f), ez. (Bt, ,t) for the target PROTAC domain during the fine-tuning phase can be decomposed as
cz. c3. (Af,t) = =c((at)(A)) + co(Af) (EuLAbLAf) g(Ab))
((Ab)
(17)
((Bb)
cg. (B, = Cy(BB) + Cg(BF) BB(BBI1B)) (BF)
(18)
q( (B())
((Ag) (V(B6) Similarly, still focus estimating the density ratio term in the chemical space ((1)) ((
since it is involved in the
concrete score correction term Equation (17) (Equation (18)).
Based on Theorem 2.2, we can figure out how the neural network 18 (Af,t) should be trained approximate the density
((45) Namely, have ratio term E By (A4) ((A))'
((A))
3 (A) arg min L BBCAAIAA (AF)
(AA)))
where the loss is set as
[b((ttt) ((A
~vA = =B Ba(Af.Af)
(19)
((Ak))
And we have,
((Bb)
4/* (B/, t) arg min L (B(111( yBbb²'
where the loss is set as
7(B,t)
h = Egcag.ng mn)
((BB,
(20)
(B())
Therefore, the final score estimators for the target PROTAC domain can be calculated follows.
33- (Af = Co(ag) co(AE) (v (A((
(21)
53- (B/, Cy(B,) + Cg(B)) (B,
(22)
Additionally, in order to estimate the density ratio term ( () via the cross-entropy loss in Equation (23) and W(B/, () via the cross-entropy
3 ((A))
in Equation (19) and 8(B (B.)
in Equation (20), we still need to
train another two binary classifiers W(AF,
loss Equation (24) to determine whether the perturbed samples are from the source small molecule domain or the target PROTAC domain.
CA Σ
Σ
log W(A
Σ
Σ
log (1 (0LAWA
(23)
Ag 1E1Ag)
Ag-a((A)) A3~Q(AE113))
WB Σ
Σ
log W(B t)
Σ
Σ
log (1 11---0 W(B,
(24)
B (B()) (B (B))
B - Bl
17

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
The density ratio is approximated as
((At) 1-W(At,1) q(B()) 1 --W(B,.) (Af) ((A q(B() = W(BF,()
C. Proofs
C.1. Theorem 2.1
Proof. We focus on proving the decomposition of the score estimator 83- (X,, t) and the proof for the case of the concrete score estimator cz- (Af: () follows the exact same manner.
Since we assume that q (6 "(g!" 9 we know
(x x((x+1x)))
Similar to the training loss for the pre-training phase in Equation (5) the source small molecule domain, the training loss for directly fine-tuning on the target PROTAC domain should be
= Ex Xxa
Vxp logg(X! x
We then have
83. (X(;,) aarg min E Ext (xf-x(x) XxxxxxxXX//xxxxxxx Xx/= -Vx log (X |xf)
|| - Vxx log x6) x6>11,]
=arg min E, ( (((((())
Then, we only need to focus on & related terms
[|oo(xtt.0) Vxp. log x( xty [1:
[| :011]]
= -B (xx6xxx)
2B 2Bx(xxxx3) ((0g(x t), V xx log x(1) Ci.
The first term is
||x( 1
1x3 / i(x6; xt) (xx( 1112 axtaxx
xxF
S xx x
(((66) dx dx!
((x (x+1)
(((x)))
- I ((xx)a(X 1x61)
*((X6)
(x
dx dx!
1xf Ixp
x(x)))
||
/|2 ((x)))
= Ex(xfx+1)
(((6)
18

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
The second term is
[sg(X f), Vxt log xf1)
BaC xx)
-fx xb xx
S (xg(X) () Vxp log X4
S
(9x(X) 1), N
Xx OxP
-Jxg JxS
Vxz ((XX)) dxpax!
9(x )
(xX1 (((t))
-S I
9(x xx,) (s t). (x; ((bb)
(xH1))) (xx)) dxp
-xx 1xx
(sa(X) 1), Vxy log logg(X x²)) [(x))
-Bx(xfxx(1
((x)))
Therefore, we have
|(x((Xt))) 1) VxF 1og xt)1?²
ang minnngggg Fxcx xx)
|xa( t) 1
=ang g( xx-)
xxt) (Sa(( t). 7xt lx [x6))| + +
x((66
(sx(( t), Vxt log (x {(xb)
$/ |xx( t)
=arg minE, xx+)
2E gxx
((x)) +2
-arg min E Exxx x [| | 1) -xx log (xx (x+))
::= ((,)
7(x))
Therefore, we have,
83. (x,,,) t) = argminkE, { Eg(xe) BacxF1x8)
[1 - Vxt lxg xt) 2 *((+))
}
((,)))
Then, we can use Importance Weighted Denoising Score Matching on the source small molecule domain and thus get the closed-form of sg. (X/, f) as follows:
V VxP
Xt log (x/1)x) ((x,) (( (
$3. (Xf
((x{)) [x(x))
E Ex(x(1xF)
Note that we also have
x log (x6(()) = Exx log ol (x(11))) + Vx log'( - Vxx log (x!)
= -xx log (x/ - Vxp log (X²),,
19

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Finally, we have that
( t)
Vx log xx
g(xi) x(Xo'
E (x+1xx)
E 1Xx
9 (
x.. log q( (xx)
=V x log ((x})) +
V x log ((},)
B (xxxx)
EE (X
6OXA ((xg)
x log (XX (()) + Vxf log logg(x())
=Vp xx!4 +
Vxt log ((x!)
E (x
7(x))
E 2xxIxx
g ((xg)
V
xp log (X. [x({)
=Vx log +
E 1x(x(1xF)
((
G V X (xFFIF))
=Vx log (x{) +
x(X6)
B xF)
= xxp log (x/g((//)>0) + (xg
Vxe log E 2xx XF)
To sum up, we prove that
$3. (X, t) VxF log (xgg(X+)++ + Vxp log E (xf(x() (X
Similarly, for the case of the concrete score estimator cz. (Af', t), we can also prove that
cz. (At, () ((()))(())) + co(Af)) (BacasInt) ((()))
((Ab)
C.2. Theorem 2.2
Proof. We first note that the objective function L can be rewritten as follows.
1 (x((x()) arg min Co
(xxx,1) (((tt) 2
arg min E Eg(x x(1)
((F))
-argmin f x
Jss/ á q(66 (x))
1 1 (x/+))
(x+t)
2
dxy. (x{) axt
((xx)
/
|| (xt, -2 ( (x(xt))) 1). /
9 (xx)
ax! + ( dx?
arg argmin
1xX q i(xb xt
9 (xx) dxo
x
(x(T+)
argmin
v (xt,t) - -Ex(xSIxF)
(xx²) ax!.
(x+)
xt
20

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Here, C is constant independent of vb Hence, we have the minimize 4 ('(f,() (0) arg argmin L satisfies the following.
② (x,+)
v (X(; arg min 2* E *x(Xx1xF
(25)
9 (x+)
Also, we can rewrite the term in the RHS of Equation (14) as follows.
Φ (xx)
- x 9 ((x))
B
(xFIxF)
dxo
9 (X))
9 (x6)
/ a ( (x)) 9 (X6) 1 (X6) dxo ạ (x()
q (X))
/ á (xt (8+) q (X6) (X6) dxo
a (xP)
(X6)
/ (xx xx) 9 (x6) dxo
(26)
(x,))
f à (
(xt) á (X+)
7 (X)) dxb
@ (X6)
q (x()
/ à (x6
x
á (x/)
dxp
9 (xt)
q (x/+)
= (x51xx))
q (X!')
Based on Equation (26) and Equation (25), we now prove
(xx)
s (x+ t) arg min L E (x(xxxx)) q (xx)
Remark C.I. Similarly, for the case of the L and L we can also prove that
a(Ab)
36 (A(,t) arg min L E BoCAfIAF (AA)'))
where the loss is set
((A()) 2d BECAg.Ap) (((A)1))
q( ((())
And we also have,
((Bb)
3 (B/ t) arg min L. (n(nttttt g( (Bb))
where the loss is set as
nn = Eqcg BB BB) ((B[, ((Bb)
(BBi)
D. Pseudocodes
D.I. Pre-training Phase of the Proposed DAD-PROTAC Model
We summarize the pre- pre-training phase of the proposed DAD-PROTAC model in Algorithm Algorithm 2. The pre-training phase operates on the small molecule domain (9). The process begins by sampling small molecule data (Ab: ) from these distributions (line 2). For each iteration, we sample time step uniformly from [1, 7] and apply diffusion noise
21

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Algorithm 2 Pre-training Phase of DAD-PROTAC
input Samples from the small molecule domain ((9) = ((4)) pre-trained diffusion process g())
)
output Score functions (x(A))(A)), (B( (BF:), and Vx log (X}:)
1: repeat
2: Sample the small molecule data from (Ap; B X (((4))((B)))(()))
3: Sample the time step from ( ~ (1, (1,17) 7).
4: Perturb the previous sampled data (Ag. B xx)) via the corresponding diffusion process g( ( based on
q(A!' A in Equation (7), ( B and q / |x6) in Equation (2).
5: Perform gradient descent step over the EGNN model (g t) to predict co(Af, 1) via the loss _ in Equation (9),
ca( (B
,7,,, 1) via the loss CB, and (x via the loss L in Equation (5).
6: until converged.
7: return Score functions C (4F) ca (Af;, t), (B (B{) co( and V Vx log logg(X!)) so(X),t) on the
small molecule domain for the subsequent fine-tuning phase
to the sampled molecules using the corresponding diffusion process q(Ge 9o) (line 4). An EGNN model deg then trained through gradient descent predict three score functions: cs (Af,,) t), co (B t) and ( t) (line 5). Upon convergence, the algorithm outputs the learned score functions for each component, which serve the foundation for the subsequent fine-tuning phase (line 7). In summary, this pre-training phase ensures that the model learns the representations of general linker structures before adaptation to the specific PROTAC domain.
D.2. Fine-tuning Phase of the Proposed DAD-PROTAC Model
We present the detailed fine-tuning phase of the proposed DAD-PROTAC model in Algorithm 3. The fine-tuning process of DAD-PROTAC comprises two primary steps: density ratio estimation and score correction, designed to efficiently transfer knowledge from small molecules to PROTACs.
In the density ratio estimation step, we begin by sampling m instances from the source small molecule domain (()) and n instances from the target PROTAC domain ((g) (line 2). We then sample a time step uniformly from interval [1, T) and apply the forward diffusion process (G)) = ((G Go) to perturb the sampled data. Note that the forward diffusion processes in both the small molecule domain and the PROTAC domain are the same (line 3). A binary classifier W is trained through gradient descent to distinguish between the domains using specified loss functions for each component of the linker G (A+, , } (line 4). Upon convergence, we compute the density ratio (G..) I-W(G..t) which quantifies the distribution differences in the chemical space explicitly (line 6).
((re) W(G..))
In the score correction phase, we leverage another neural network (G,, approximate the necessary adjustments to the pre-trained score function the score correction term. We sample data exclusively from the PROTAC domain (()) and apply noise perturbation through the diffusion process j(G Go) (line 8). The network is trained via gradient descent to learn the score corrections (line 9). The final output is a score estimator specifically adapted for the PROTAC domain, incorporating both the pre-trained knowledge and domain-specific adjustments (line 11).
E. Experimental Setup Details
E. Datasets
1. ZINC
Following the setting of Delinker (Imric et al., 2020), we also consider a subset of 250,000 molecules randomly selected from the ZINC database 166mez-Bombarelli et al., 2018; Irwin & Shoichet, 2005). We first generate 3D conformers using RDKit (Landrum, 2016) and define a reference structure for each molecule by selecting the lowest-energy conformation. Molccules are then fragmented by enumerating all double cuts of acyclic single bonds outside functional groups The resulting fragments and linkers are filtered based on atom count, synthetic accessibility score, ring aromaticity, and pan-assay interference compounds (PAINS) criteria (Baell & Holloway, 2010). Since each molecule can yicld multiple fragment- linker combinations, the final dataset comprises 438,610 samples, which we use for model pre-training. The dataset includes the
22

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Algorithm 3 Fine-tuning Phase of DAD-PROTAC in Detail
input Samples from the source small molecule domain ((9) = (a( (((44) q(
(xx)) and target PROTAC domain (g)) = ()(AA), (G(")) (()))), pre-trained (fine-tunin)) diffusion process q(G, 9o) ((G: (G))
1* Estimate the density ratio in the chemical space via binary classifier W first from lines 1-5. */
1: repeat
2: Sample the small molecule data from (Ag. B X6) ~ (q(A"), q (BB),, (Xx))) with size of m, and sample the
PROTAC data from (Ai. B xx6) ~ (q(44) (( q(x²)) with size of n.
3: Sample ~ (11, I') and perturb previous sampled data (Ah. Bb X²') via the corresponding diffusion process
9(9, Go) @ Go) based on (A! A4) in Equation (7), q q(B B)),, and q(Xp' xx)) in Equation (2).
4: Perform gradient descent step over the binary classifiers W(A,, 1) via the loss in Equation (23), W(Bf, 1) via the
loss in Equation (24), and W(X t) via the loss in Equation (16).
5: until converged.
6: Set the density ratio in the chemical space term ((AA) = 1-W(A).0) in Equation (19), q(B() ((4)) W(45.))
g(B() = 1-W(B .4) in Equa-
W(B/ .t)
tion (20), and x(x+) .t)
[ W(XF.)) in Equation (13).
1* Learn upproximate the score correction term for the fine-tuning phase from lines 7-10. */
7: repeat
8: Sample the PROTAC data from (Af. (6) (q(A"). ((x2) and perturb (45. { with noise
via diffusion process based on q(A} A4). ( B.), and ( (xF) with q(G, 90) = 9 90).
9: Perform gradient descent step over ( 1) via the loss in Equation (19), (B...) () via the loss in Equation (20) and
Tb( (x,,) I) via the loss in Equation (13).
10: until converged
11: Set score estimator for the target PROTAC domain $3. (A/,t) t) via Equation (21), sz. (B, .) via Equation (22), and
$3. (Xf,7 t) via Equation (15), Combine the pre-trained score estimators build the score estimator for PROTAC domain.
12: return Score estimator for the target PROTAC domain as (sg- (At f), 8z. (Bt, 83 (x),
following atom types: C, O, F,S, CI, Br, and
E.1.2. PROTAC-DB
PROTAC-DB 2.0 was recently published (Weng et al., 2022), which contains basic information on 3,270 PROTACs. The chemical structures, biological activities, physiochemical properties, and pharmacokinetic parameters of these compounds are manually extracted from the literature or calculated by some programs. Following the data collection method in DifiPROTAC (Li et al., 2024), the simplified molecular-input line-entry system representations for the E3 ligands, the warheads (ligands that bind to targets), and the linkers each PROTAC from the corresponding pages on the PROTAC-DB website are gathered. This process results in final dataset of 365 warheads, 82 E3 ligands, 1501 linkers, and 3,270 PROTACs, We use the PROTAC-DB dataset for fine-tuning the model.
E.2. Baselines
E.2.1. 3DLINKER
3DLinker (Huang et al., 2022) a conditional generative model for the linker design task, and is able to predict anchor atoms and jointly generate linker graphs and their 3D structures based on an B(3) equivariant graph variational autoencoder. We train it for 20 epochs the ZINC dataset. We also apply for permutation of the given two fragments to enhance the model performance. For the DDlinker-Fine-tune model, we further directly fine-tune it the PROTAC-DB dataset.
E.2.2. DIFELINKER
DiffLinker (Igashov et al., 2024) is an equivaaaia three-dimensional conditional diffusion model for molecular linker design. It can link an arbitrary number of fragments. We set the dimension of the hidden embedding as 256 with a 5-layer EGNN. Similarly, for the Diffl.inker-Fin-ee Fine-tuiinnn model, we further directly fine-tune it on the PROTAC-DB dataset.
23

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
E.2.3. LINKERNET
LinkerNet (Guan al., 2023) is another 3D equivariant diffusion model that jointly learns the generative process of both fragment poses and the 3D structure of the linker. In each layer, the imput features concatenated and the hidden embedding/positions updated with a 2-layer MLP with the LayerNorm and the ReLU activation. For the LinkerNet-Fine- tuning model, we further directly fine-tune it on the PROTAC-DB dataset.
E.3. Metrics
Our evaluation metrics include validity, uniqueness, and recovery. To assess the results, employ process where sample 100 conformations for each input ligand pair in the test set and subsequently calculate the following metrics.
E.3.1. VALIDITY
This metric evaluates the validity of generated PROTAC molecules, ensuring they remain in the chemical space. We use OpenBabel "O'Boyle et al., 201 I) to determine bond connectivity and RDKit (Landrum, 2016) to verify compliance with valeney rules. Additionally, validity requires the absence of dissociative atoms and the preservation of specified fragments. Since fragments are fixed conditions during learning, assessment focuses detecting any detached atoms.
E.3.2. UNIQUENESS
To quantitatively assess the diversity of generated molecules, we define uniqueness metric based on canonical SMILES rep- resentations. Since evaluating molecular similarity in 3D space involves complex conformational and spatial considerations, we project molecules into 2D using canonical SMILES, a standardized molecular representation. Uniqueness is computed as the ratio of unique SMILES to the total number of valid SMILES for generated PROTAC molecules. This metric ranges from 0 to where I indicates complete uniqueness, while lower values suggest structural repetition in the output.
E.3.3. NOVELTY
Since the PROTAC-DB dataset represents only tiny subset of the whole chemical space of PROTAC molecules, a model for PROTAC linker design with good coverage of chemical space will rarely generate PROTAC molecules present in the fine-tuning dataset. The metric of novelty assesses the fraction of generated PROTAC molecules that do not appear in the seen dataset (Brown et al., 2019). A higher novelty score indicates that the model can generate new, previously unseen PROTAC molecules, which crucial for discovering new compounds. Therefore, models overfitting the dataset will often obtain low scores on this metric.
E.3.4. RECOVERY
To evaluate model's ability to recreate existing PROTAC linkers, we use recovery metric based on canonical SMILES matching. This metric quantifies the model's capacity to generate molecules that are identical to known, validated structures in the training set, providing insight into the model's ability to capture the essential features of successful PROTAC linkers. The metric recovery is defined as the ratio between matched structures and total generated molecules. A recovery ratio that too low may indicate insufficient learning of essential structural features, while an exceptionally high ratio might suggest overfitting to known structures rather than meaningful generalization.
E.3.5. QUANTITATIVE ESTIMATE OF DRUG-LIKENESS (QED)
The Quantitative Estimate of Drug-likeness (QED) (Bickerton et al., 2012) serves comprehensive metric for assessing the drug-like characteristics of the molecules. QED integrates eight physicochemical parameters: molecular weight, octanol-water partition coefficient (logP), rotatable bonds, hydrogen bond donors, hydrogen bond acceptors, polar surface area, aromatic rings, and the number of structural alerts. These parameters are weighted and combined through a series of desirability functions derived from the analysis of known oral drugs, resulting in a continuous score between and 1, where higher values indicate more favorable drug-like properties.
24

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
E.3.6. SYNTHETIC ACCESSIBILITY (SA)
The Synthetic Accessibility (SA) Score aims assess the synthetic difficulty of chemical compounds (Ertl & Schuffenhauer, 2009). The score combines multiple molecular features such ring complexity, stereochemical complexity, structural symmetry, and the presence of uncommon structural elements to estimate the synthetic feasibility of target compound. Typically scaled from to 10, where lower values indicate easier synthesis, the SA score is rule-based and penalizes the occurrence of fragments rarely found in a reference dataset and the presence of specific structural features.
E.3.7. MINIMUM ENERGY EMIN
To quantitatively assess the quality 3D conformations, we perform energy minimization using the Merck Molecular Force Field (Halgren, 1996). For each generated PROTAC molecule with N conformers, we define Emin as the average minimum energy of generated molecules per fragment pair before optimization. A lower Emin indicates the better quality of the overall generated conformations.
E.3.8. ROOT MEAN SQUARE DEVIATION (RMSD)
The Root Mean Square Deviation (RMSD) is the standard metric used to quantify the conformational similarity between pre-optimization and post-optimization structures, indicating how much the atomic positions of a molecule have shifted during the optimization process, with a lower RMSD signifying greater similarity between the two structures.
E.4. Implementation Details
We view the input fragments and linkers as fully connected graphs. The atom features include a one-hot element and a charge indicator including H. C, N, N:, N O, O. CI, Br, I, S(2), S(4), and S(6). We also add one one-hot value indicate if the molecular graph is a fragment or linker. Similarly, the edge features include one-hot bond type indicator (None, Single, Double, Triple, Aromatic), and 4-dim hot vector indicating the edge is between fragment atoms, linker atoms, fragment-linker atoms, or linker-fragee fragment atoms.
During the pre-training phase, atom features and edge features are first fed into two MLP embedding layers with a dimension of 256 for nodes and a dimension of 64 for edges. These hidden embeddings get involved in an atom update layer, bond update layer, and position update layer in the EGNN model. In each EGNN layer, we concatenate the input features and update the hidden embeddings or positions with a 2-layer MLP with LayerNorm and ReLU activation function. We stack layers of EGNN for denoising. We set the number of diffusion steps as 500. For the diffusion noise schedule, we choose a cosine schedule suggested in (Nichol & Dhariwal, 2021) with s 0.01. The model is trained via Adam W (Loshchilov & Hutter, 2019) with the initial learning rate 5e-5, 31 0.99, .2 0.999. We also schedule to decay the learning rate exponentially with factor of 0.6 and a minimum learning rate of lc-6. During the fine-tuning phase, we also adopt a 4-layer MLP with 512 hidden units and SiL.U activation function as the backbone of the guidance network. We train the guidance network for 60k iterations by Adam optimizer with a learning rate of 3e-4.
F. Additional Experimental Results
E.1. Distributions of Molecular Descriptors of the Two Datasets
We further illustrate the distributional differences between small molecules and PROTAC' in chemical space. In Figure 8, we highlight additional key physicochemical distinctions, including partition coefficient (LogP), rotatable bond count (flexibility and conformational dynamics), and the number of hydrogen bond donors (HBDs) and acceptors (HBAs). PROTACs are sourced from PROTAC-DB and PROTACpedia! ' while small molecules are randomly sampled from ZINC.
F. MOLECULAR WEIGHT
As shown in Figure 2, small molecules typically range between 250-500 Da, a range optimized for oral bioavailability according Lipinski's Rule of Five. In contrast, PROTACs peak at 750-1000 Da, indicating their significantly larger size. The small-molecule MW distribution is narrow and sharply peaked, reflecting relatively uniform size range, whereas the broader PROTAC distribution indicates greater structural variability.
Inttps://protacpeee wei zmann. ac i1 httpttb/maatttt.e
25

---

Domain-Adapted Diffusion Model for PROTAC Linker Design Through the Lens of Density Ratio in Chemical Space
Logp
Rotatable Bonds
350
PROTAC
800
PROTAC
300
Small Molecule
Small Molecule
700
250
600
Count
200
Count
500
400
150
300
100
200
50
100
8.0
2.5
5.0
7.5
10.0
15.0
17.5 20.0
0
10
20
Partition Coefficient
Number of Rotatable Bonds
(a) Partition Coeflicient (LogP)
(b) Number of Rotatable Bonds
Hydrogen Bond Donors
Hydrogen Bond Acceptors
1600
PROTAC
800
PROTAC
Small Molecule
Small Molecule
1400
700
1200
600
Count
1000
Count
500
800
400
600
300
400
200
200
100
8
0
5
10
20
25
30
Number of Hydrogen Bond Donors
Number of Hydrogen Bond Acceptors
(c) Number of Hydrogen Bond Donors (HBDs)
(d) Number of Hydrogen Bond Acceptors (HBAs)
Figure 8: The distributions of various molecular descriptors of PROTACs versus small molecules in the chemical space.
1.1.2. PARTITION COEFFICIENT (LoGP)
The LogP distribution for small molecules peaks around 3-4, aligning with like-lis criteria that favor values between I and 3 for optimal oral bioavailability. PROTACs, however, exhibit broader LogP distribution, peaking around 6, indicating higher hydrophobicity. This increased hydrophobicity can impact solubility and cellular permeability, posing challenges for aqueous environments such as extracellular fluids and the cytosol.
F.1.3. ROTATABLE BONDS
Small molecules typically have 1-10 rotatable bonds, contributing to their relatively rigid structures. contrast, PROTACs peak 10-20 rotatable bonds, largely due to their flexible linker regions.
1.1.4. HYDROGEN BONDING
The HBD distribution for small molecules peaks around 1 1-2, consistent with drug-likeness criteria that prioritize membrane permeability. PROTACs, however, peak around 5 HBDs, with a similar trend observed for HBAs. These differences further distinguish PROTACs from traditional small molecules in terms of molecular interactions.
Thus, models trained on small molecule datasets like ZINC fail to capture the distinct characteristics of PROTACs, as small molecule fragments and PROTACs occupy largely non-overlapping regions of chemical space. This motivates the development of domain-adapted generative model that explicitly accounts for distributional differences between the small molecule and PROTAC domains.
26


Reviews:
[{"review_id": "dmEXGqNxc0", "reviewer": "Reviewer_gqa6", "summary": "This study focuses on domain adaptation in diffusion models for biology. The authors pretrain a diffusion model on the ZINC dataset and attempt to use the model on the PROTAC domain. For finetuning, they use density ratio estimation techniques to correct the score function on the ZINC dataset. The method looks intersting and the results are great.\n\n## Update after rebuttal\nI appreciate the authors' efforts to answer the questions, most of my concerns are resolved. Thank you.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments!\n\n# 1.  Errors for large $t$\nIn Eq.(16), we train a time-dependent classifier $W(X_t^L,t)$ with samples from potentially all steps $t$ jointly, instead of training different classifiers $W_t(X_t^L)$ for each step $t$ individually. This means that the **training set has both easy samples (t is small) and hard samples (t is large)**. This will mitigate the issue of differentiating the samples from two domains when t is large. Besides, we can easily **add more samples (with larger $m$ and $n$) to the training set to further mitigate this issue**. We also compare the classification performance to show the performance of this trained classifier on different groups of $t$ (based on its range) on an additional test set below. This table shows that **the model’s performance drop for large $t$ is acceptable**. The further **incorporation of focal loss to Eq.(16) by putting more focus on hard samples ($t$ is large)  is not necessary**. We will add it to the ablation study in the final version.\n\n| test samples (range of t) | 0% to 25%T | 25%T to 50%T | 50%T to 75%T | 75%T to 100%T |\n|---|---|---|---|---|\n| classification accuracy (with Eq.(16))(Ours) | 97.5 | 96.8 | 97.1 | 96.7 |\n| classification accuracy (with Eq.(16) + focal loss) | 97.3 | 97.2 | 97.2 | 96.9 |\n\n# 2. Model cheats by counting the number of atoms\nThe number of atoms is only one of the differences between PROTACs and small molecules. **Please refer to Figure 8 of the paper to see other differences, like LogP, which are more likely to be dependent on the distributions of atom coordinates. Therefore, we train the classifier on atom coordinates $X$ in Eq.(16) to encode all potential distribution differences in their chemical space**. We specifically design our classifier to prevent this shortcut. We use original features like $X$ as the input rather than global molecular properties like the number of atoms. We validate this by testing the classifier on a separate dataset containing molecular structures with the same atom counts but from different domains (PROTACs and small molecules). The results in the table below confirm that **classification accuracy on test samples with the same atom counts remains consistent with the overall results**. If we only use atom counts as input, the classification performance would drop significantly.\n\n|  | test samples (overall) | test samples (same atom counts) |\n|---|---|---|\n| classification accuracy (use all original features as input) (Ours) | 97.0 | 96.7 |\n| classification accuracy (use only atom counts as input) | 71.3 | 53.2 |\n\n# 3. Error propagation and Monte Carlo estimation\nFrom the results in the last point above, we can see that **the classification error in this step is quite small**, and this alleviates the error propagation issue in the first place. We further find that using MC simulation to estimate the score correction term directly may get slightly better performance, but it is **computationally prohibitive**. The extended ablation study from Table 2 is summarized below. “direct score correction approximation” refers to the model where we only run MC simulations on a few samples $X_t^L$ to obtain the corresponding score correction terms in Eq.(11) and then train another neural network to learn to approximate this score correction term. “Density ratio estimation via clean samples” refers to the model where we only use clean samples to train the classifier in Eq.(16) (namely, set t =0). More discussion is found in Sec 3.3 in the paper. From the results in the table below, **MC simulation suffers from high computational cost, our current design keeps a good balance between effectiveness and efficiency.**\n\n|  | Valid% | Unique% | Novel% | Recover% | Relative Running Time |\n|---|---|---|---|---|---|\n| DAD-PROTAC w/ direct score correction MC simulation | 95.1 ± 0.6 | 70.1 ± 0.5 | 70.3 ± 0.4 | 46.2 ± 0.4 | 131x |\n| DAD-PROTAC w/ direct score correction approximation | 81.9 ± 0.8 | 49.5 ± 0.4 | 59.7 ± 0.3 | 28.1 ± 0.7 | 0.8x |\n| DAD-PROTAC w/ density ratio estimation via clean samples | 90.0 ± 4.7 | 62.8 ± 6.1 | 69.3 ± 0.2 | 41.0 ± 0.6 | 0.95x |\n| DAD-PROTAC (Ours) | 94.8 ± 0.4 | 69.3 ± 0.3 | 71.5 ± 0.3 | 45.7 ± 0.6 | 1x |\n\n# 4. Other methods for fine-tuning diffusion models\nLoRA and DPO are promising for parameter-efficient fine-tuning. However, they assume the source and target domains share a similar latent structure, which is invalid here due to the significant chemical differences between small molecules and PROTACs (Figure 8 in the paper). Our method explicitly models the domain shift via density ratios estimation, which is specifically designed for the linker design task. That said, integrating LoRA-style methods into our framework (e.g., for the correction term) could further improve efficiency. We leave it as future work since it is out of the scope of this paper.\n\n# End of Response\nWe hope that our responses have effectively addressed all of your concerns.", "claims_and_evidence": "Yes, the claims are supported by the experiments.", "methods_and_evaluation_criteria": "Yes, the method and evaluation make sense.", "theoretical_claims": "I checked the claims.", "experimental_designs_or_analyses": "Yes, I think this study is solid.", "relation_to_broader_scientific_literature": "It would benefit the linker design area.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "**Strengths**\n\n1. The domain adaptation diffusion is novel and interesting. Score correction helps preserve the original power of the pretrained model on ZINC and prevents from overfitting on the PROTAC dataset.\n2. The results are great and outperform other methods. The ablation study proves the usefulness of the method.\n\n\n\n**Weaknesses**\n\n1. Training a classifier on x(t) may introduce some errors. For x(t) when t is large (in the early stage of reverse diffusion), will the classifier be hard to train? Since the atom number is so different (see Figure 2), will the classifier learn to cheat by counting the number of atoms? \n\n2. Since you firstly train a classifier, and then train another network with the classifier, will error propagation hurt the performance? A baseline can be use Monte Carlo estimation to directly estimate the value of the score correction in Theorem 2.1. In Table 2, you have ablated \"direct score correction approximation\" and \"density ratio estimation via clean samples\", can you provide more details?", "other_comments_or_suggestions": "NA", "questions_for_authors": "**Questions**   \nThere are other methods for \"finetuning\" a pretrained diffusion model on a specific domain, such as LoRA in LLM and diffusion DPO in image generation. Is it possible that these techniques help prevent overfitting?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1xF0J2v8At", "reviewer": "Reviewer_3rte", "summary": "This paper introduces DAD-PROTAC, a domain-adapted diffusion model for designing linkers in Proteolysis-targeting chimeras (PROTACs). The main algorithmic idea is the efficient fine-tuning strategy via density ratio estimation, avoiding full retraining of the diffusion model.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments!\n\n# 1. Figure 3\n\nWe will make Figure 3 clearer in the final version with fewer annotations and highlight more on how the score correction term is obtained. We will also explicitly annotate the input and output of two phases and each model component.\n\n# 2. Pre-specified number of atoms\nWe **follow most existing SOTA linker design methods like LinkerNet[1] and also pre-specify the number of atoms**. This is because the prediction of the number of atoms in the linker should be solved via **a deterministic model**, while the generation of the linker (atom coordinates and bonds) should be solved via another **generative model**. **The two steps of the linker design task are usually solved separately**. We focus on the generation part of this task in this work. \n\nFor the prediction of the number of atoms in the linker, **we can use a separately trained GNN to produce probabilities for the linker size**. The input is still the molecule fragments. Later, when we ​​generate the linker’s structure, we first pre-specify the linker size based on the maximum predicted probabilities by this separately trained GNN and then use our proposed method in this paper. Since the linker size prediction step and the linker structure generation step are trained separately, **no components (density ratio estimation and score correction) in the proposed DAD-PROTAC would be affected**. We include the experimental results below. As we can see, **the additional training of another neural network for linker size prediction may introduce more errors for the linker size, and thus the performance may degrade**. We leave the joint training of both linker size prediction and linker structure generation for future work, as it is currently out of the scope of this work.\n\n|  | Valid% | Unique% | Novel% | Recover% |\n|:---:|:---:|:---:|:---:|:---:|\n| DAD-PROTAC w/o pre-specified number of atoms  | 93.1 ± 0.6 | 66.2 ± 0.7 | 69.3 ± 0.8 | 40.4 ± 0.5 |\n| DAD-PROTAC (Ours) | 94.8 ± 0.4 | 69.3 ± 0.3 | 71.5 ± 0.3 | 45.7 ± 0.6 |\n\n[1] Jiaqi Guan et al., LinkerNet: Fragment Poses and Linker Co-Design with 3D Equivariant Diffusion. In NeurIPS 2023.\n\n# 3. Performance with different sizes of PROTAC datasets\n**For sample efficiency**, we analyze the performance of the proposed DAD-PROTAC with different portions of the PROTAC dataset used during the fine-tuning phase in the table below. We reduce the PROTAC dataset size from 100% (full) to 50%, 25%, and 10%. We also add an extra 20% of samples from the PROTAC-DB 2.0 database as the extended dataset. This non-linear degradation suggests our approach is reasonably robust to limited PROTAC data for fine-tuning. Note that, **even at 10% of the PROTAC dataset size, DAD-PROTAC can still outperform the baseline 3DLinker-Fine-tuning approach using 100% of the PROTAC data**. For larger datasets, our preliminary experiments with additional data show diminishing returns with 1.2x our current dataset size, suggesting we are approaching the performance ceiling of the current architecture. **We will add these results and include a learning curve figure in the final version that visualizes these results for sample efficiency analysis.**\n\n| DAD-PROTAC with different sizes of datasets | Valid% | Unique% | Novel% | Recover% |\n|:---:|:---:|:---:|:---:|:---:|\n| 10% PROTAC dataset | 64.9 ± 0.8 | 56.8 ± 0.7 | 63.8 ± 0.5 | 31.4 ± 0.8 |\n| 25% PROTAC dataset | 76.3 ± 0.6 | 59.5 ± 0.5 | 67.9 ± 0.4 | 38.2 ± 0.7 |\n| 50% PROTAC dataset | 89.9 ± 0.5 | 65.8 ± 0.4 | 70.7 ± 0.3 | 43.1 ± 0.6 |\n| 100% PROTAC dataset | 94.8 ± 0.4 | **69.3 ± 0.3** | 71.5 ± 0.3 | **45.7 ± 0.6** |\n| 100% PROTAC dataset (full) + 20% additional | **95.1 ± 0.3** | 68.1 ± 0.3 | **71.9 ± 0.3** | 45.2 ± 0.5 |\n\n# End of Response\nWe hope that our responses have effectively addressed all of your concerns.", "claims_and_evidence": "The claims made in the submission are generally supported by clear and convincing evidence:\n\n* **Claim:** Existing diffusion models for linker design, trained on small-molecule datasets, suffer from a distribution mismatch when applied to PROTACs. \n\n  **Evidence:** Figure 2 clearly shows the difference in molecular weight distributions. Appendix B.1 and Figure 7 and F.1, along with the accompanying text, provides substantial additional detail, discussing differences in data collection and various physicochemical properties (LogP, rotatable bonds, etc.).\n\n* **Claim:** DAD-PROTAC's domain adaptation, via density ratio estimation, improves performance.\n\n  **Evidence:** Table 1 shows superior performance of DAD-PROTAC compared to baselines (3DLinker, DiffLinker, LinkerNet) and their fine-tuned versions across multiple metrics (validity, uniqueness, novelty, recovery, QED, SA, Emin, RMSD). Table 2 presents a convincing ablation study, demonstrating the importance of the density ratio estimation and the use of noise-perturbed samples.\n\n* **Claim:** DAD-PROTAC is more computationally efficient than full fine-tuning.\n\n  **Evidence:** Figure 4(a) shows significantly reduced fine-tuning time compared to standard fine-tuning approaches, while achieving higher validity. The text discusses the computational cost savings of learning a classifier for the score correction term rather than retraining the entire model.\n\n* **Claim:** The generated linkers are closer to real protacs.\n\n  **Evidence:** Figure 5 gives the distribution of molecular weight of the generated linkers.\n\nOverall, the claims are well-supported by a combination of quantitative results, qualitative visualizations (Figures 1, 5, and Appendix figures), and theoretical justification (Theorems 2.1 and 2.2).", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are appropriate for the problem.\n\n* **Methods:** The core idea of using a diffusion model with domain adaptation via density ratio estimation is well-motivated and theoretically grounded. The decomposition of the score estimator is a clever way to leverage pre-trained models while adapting to the target domain. The use of EGNNs is standard and appropriate for handling 3D molecular structures.\n\n* **Evaluation Criteria:** The paper uses a comprehensive set of metrics relevant to molecular generation:\n\n* **Validity, Uniqueness, Novelty:** Standard metrics for assessing the quality and diversity of generated molecules.\n\n* **Recovery:** Measures the ability to reproduce known linkers, which is important for validating the model's ability to learn the underlying distribution.\n\n* **QED, SA:** Established metrics for assessing drug-likeness and synthetic accessibility.\n\n* **Emin, RMSD:** 3D conformation-specific metrics that evaluate the quality of the generated structures.\n\n* **Datasets:** Using ZINC for pre-training and PROTAC-DB for fine-tuning and evaluation is a reasonable choice, reflecting the availability of data in these domains. The paper clearly explains the data processing and splitting procedures.\n\n* **Baselines:** The comparison to 3DLinker, DiffLinker, and LinkerNet (and their fine-tuned versions) provides a good assessment of the proposed method's performance relative to existing state-of-the-art approaches.", "theoretical_claims": "I checked the correctness of the proofs for theoretical claims.\n\nTheorem 2.1 is correct. The decomposition is valid.\n\nTheorem 2.2 is correct. The training objective is derived in the appendix.\n\nThe demonstrations are given in Appendix C and B.4, and they are corrects.", "experimental_designs_or_analyses": "I checked the soundness/validity of the experimental designs and analyses. The ablation study (Table 2) is particularly well-designed, isolating the contributions of different components of the proposed method. The comparisons to baselines are fair, with pre-trained versions of the baselines included. The use of multiple metrics provides a comprehensive evaluation. The visualization results in Figure 5 support the claims.", "relation_to_broader_scientific_literature": "The paper is well-situated within the broader scientific literature. It clearly explains the context of PROTAC linker design and the limitations of existing approaches. The related work section (and Appendix A) provides a good overview of relevant work in molecular generation, diffusion models, and PROTAC design. The paper cites relevant papers on diffusion models (Hoogeboom et al., 2022; Nichol & Dhariwal, 2021), EGNNs (Satorras et al., 2021), molecular generation (Guan et al., 2023; Igashov et al., 2024), and PROTACs (Bemis et al., 2021; Troup et al., 2020).", "essential_references_not_discussed": "No essential references appear to be missing.", "other_strengths_and_weaknesses": "**Strengths:**\n\n* **Novelty:** The core idea of using density ratio estimation for domain adaptation in PROTAC linker design is novel and well-motivated.\n*  **Thoroughness:** The paper is very thorough, with extensive experiments, ablations, visualizations, and theoretical justifications.\n* **Clarity:** The paper is generally well-written and easy to follow. The figures and tables are informative and well-designed.\n\n**Weaknesses:**\n\n* The Figure 3 of DAD-PROTAC model is a little complicated, and is hard to understand.", "other_comments_or_suggestions": "Reference the **Other Strengths And Weaknesses** part", "questions_for_authors": "1. The current approach assumes the number of atoms in the linker is pre-specified. How challenging would it be to extend the model to generate linkers of variable lengths? How would this impact the density ratio estimation and score correction steps? This would significantly improve its real applicability.\n\n2. How does DAD-PROTAC's performance scale with the size of the PROTAC training dataset? The paper mentions the limited size of PROTAC datasets as a challenge. Could you provide some insights into how the method would perform with even smaller or larger datasets? A learning curve would help in assesing this limitation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YnOs8bBDws", "reviewer": "Reviewer_omEm", "summary": "In this work the authors explore a domain-adapted diffusion model for unconditional molecular generation, focusing on PROTAC linker design. While this is an interesting application, the novelty is somewhat limited as it does not explore one of the most compelling aspects of molecular design: conditional molecular generation. The challenge of generating molecules of this size in an unconditional setting is largely addressed in existing work, unless specific metrics to assess chemical space exploration were included (which are not present here). While the background and baseline choices are relevant, they do not introduce particularly novel insights.\n\nOverall, this paper is close to the acceptance threshold, and with additional experiments and analysis, it could become more impactful for both the machine learning and molecular engineering communities, but I have ranked it as a weak reject in its current state.\n\nSummary of Feedback and Concerns\n\n1. Reproducibility Concerns:\n    \n    - Unclear how training, validation, and/or test sets were determined (limited details in paper, and no code provided).\n        \n    - Lack of clarity on structures sampled from ZINC for pre-training (were they quasi-PROTACs?); furthermore, unclear why the molecular splitting strategy that the authors used for pre-training makes sense, was it just to generate as many splits as possible for the quasi-PROTAC pre-training data?\n        \n    - How were SMILES, and eventually conformers, in the pre-training and fine-tuning sets standardized and processed prior to training? There is a lot of ambiguity in data processing and standardization steps that is not clarified in the text, yet this is really important.\n        \n    - The statement regarding PROTAC data for fine-tuning is confusing: _\"We select 365 different warheads as the test set of 327 PROTAC samples, and the remaining as the training set of 2,943 samples for the fine-tuning phase.\"_ Unclear what is meant by this and frankly the statement does not make much sense to me. Hope the authors can clarify.\n        \n    - No code availability, making it difficult (impossible) to validate and reproduce the work.\n        \n2. Model Training and Evaluation:\n    \n    - I am concerned over the computational cost of training and sampling the diffusion model, despite the authors' claim: _\"fine-tuning approach in the DAD-PROTAC model enjoys two main advantages. First, it is computationally efficient through the score estimator correction rather than full model retraining. Second, it explicitly estimates the density ratio in the chemical space with theoretical rigor for effective domain adaptation.\"_ Could the authors provide some more details here about the computational cost for training and inference? For instance, how long does it take to generate a linker for a new structure?\n        \n    - The authors' benchmarking choices are relevant but could have been stronger, they compare against 3DLinker, DiffLinker, and LinkerNet; however, REINVENT (Link-INVENT) would have been a more suitable benchmark as it is one of the best molecular generative models publicly available, even though it does not explicitly use 3D data (see my concerns below about this).\n        \n3. Justification for the 3D Representation:\n    \n    - The authors should better justify why 3D structures are necessary for this task. It is unclear whether a 2D representation would have sufficed, and whether the complexity of using 3D models is warranted.\n        \n        \n4. Benchmarking and Evaluation Limitations:\n    \n     - The model appears to be generating PROTAC linkers unconditionally, rather than for specific POIs (proteins of interest) and E3 ligases, limiting its practical utility. Future work should focus on generating PROTACs for specific targets and evaluating performance across different targets.\n\n    - The focus on basic molecular generation benchmarks (e.g., validity) is not very insightful, as any standard generative model should meet these. Rediscovery benchmarks would have been more meaningful to demonstrate the model’s practical applicability for PROTAC linker design; however, these are only really feasible in a conditional setting, which the authors did not explore.\n        \n    - Sample efficiency should be evaluated, as PROTAC-based generative models must be sample-efficient due to the limited amount of publicly available PROTAC data. Was this explored at all?\n        \n5. Overall Summary:\n    \n    - None of the presented experiments convincingly demonstrate that this model would be useful in a prospective setting; as such, I question the utility of it. The use or need for 3D data in this setting was not really motivated, so I think the authors need to come back to this. Currently, the study feels incomplete, unless the aims of the study are reframed from PROTAC design to simply domain-adaptation. IMO, sample efficiency and rediscovery benchmarks should be prioritized to establish the model’s utility for PROTAC design.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the constructive comments!\n\n# 1. Experimental Details\nRegarding the training/test split, since the proposed model is a pretrain-finetuning model, we use different datasets for the two phases. We use the ZINC dataset (438610 small molecule samples as the training set) to pretrain the model (Sec. 3.1.1, lines 283-293 and Appendix E.1.1). We use the PROTAC-DB dataset during the fine-tuning phase. More specifically, 2943 PROTAC samples of the PROTAC-DB dataset are used for fine-tuning (including both the training and validation sets), while the remaining 327 PROTAC samples of the PROTAC-DB dataset are used as the test set (Sec. 3.1.1, lines 295-300 and Appendix E.1.2).\n\nRegarding the structures sampled from ZINC, they can be viewed as quasi-PROTACs. We follow the existing molecular splitting method on the ZINC dataset for the linker design task in DiffLinker[1]. The molecules are fragmented by enumerating all double cuts of acyclic single bonds that are not within functional groups. The resulting splits are filtered by SA, PAINS, and other criteria. Please refer to Appendix E.1.1.\n\nRegarding the data pre-processing, we do not use SMILES since we focus on 3D structure generation. The input should be conformers. We first generate 3D conformers using RDKit and define a reference structure for each molecule by selecting the lowest-energy conformations. This also follows the practice in [1]. Please refer to Appendix E.1.1.\n\nRegarding the PROTAC data for fine-tuning, we split the PROTAC dataset into training (2943 samples) and test (327 samples) sets. The **test set contains 365 unique warhead pairs not seen in the training set** to evaluate generalization. The mismatch in numbers (327 PROTACs vs. 365 warheads) arises since some warheads are shared across PROTACs. We will revise it for clarity.\n\nRegarding code, please refer to Appendix E.4, end of page 24. \n\n[1] Ilia Igashov et al., Equivariant 3D-conditional diffusion model for molecular linker design. \n\n# 2. Model Training and Evaluation\nRegarding the computational cost for training and inference, please refer to Appendix E.4, lines 1305-1308. Our model can converge within 77 hours during the pre-training phase. For the fine-tuning phase, the guidance network can converge within 31 hours. For the sampling efficiency, DAD-PROTAC can sample one PROTAC linker within 29 minutes. It is efficient compared to other baselines in Figure 4(a).\n\nRegarding the Link-INVENT, we do not include it as a baseline since it does not use the 3D molecular graphs as the input. However, we can transform the 3D graphs in the datasets to their corresponding SMILES strings or 2D graphs, and then use SMILES/2D graphs to pretrain and fine-tune the Link-INVENT. We then compare it with DAD-PROTAC below. The Link-INVENT performs well on the valid metric but performs quite badly on metrics like uniqueness and novelty. This is because it does not use 3D information and can only generate the linker in the restrained 2D space, making it difficult to generate unique or novel structures. **We will cite the Link-INVENT paper[2] and compare its performance with ours.**\n\n| Models | Valid% | Unique% | Novel% | Recover% |\n|:---:|:---:|:---:|:---:|:---:|\n| 3DLinker-Fine-tuning | 58.6 ± 0.2 | 49.2 ± 0.6 | 56.2 ± 0.5 | 24.3 ± 0.4 |\n| LinkerNet-Fine-tuning | 82.9 ± 0.3 | 54.6 ± 7.0 | 63.7 ± 3.8 | 32.8 ± 0.8 |\n| Link-INVENT-Fine-tuning | 93.6 ± 0.5 | 43.7 ± 0.9 | 52.4 ± 1.4 | 29.1 ± 1.7 |\n| DAD-PROTAC | **94.8 ± 0.4** | **69.3 ± 0.3** | **71.5 ± 0.3** | **45.7 ± 0.6** |\n\n[2] Jeff Guo et al., Link-INVENT: generative linker design with reinforcement learning. Digital Discovery, 2023.\n\n# 3. Justification for the 3D Representation\nIf we only use SMILES or 2D graphs, atoms in the 3D space that are close in the molecule structure can be far away in the SMILES string/ 2D graphs. The 3D representation contains information about relative distance and orientation between the sub-structures, which is vital to successful PROTAC linker design[3]. While 2D representations capture connectivity, they ignore torsional flexibility and spatial constraints critical for PROTAC efficacy. Most SOTA methods(Delinker, DiffLinker, LinkerNet) all use 3D representations. We will add more discussions to the paper. Please check the comparison with Link-INVENT above.\n\n[3] Fergus Imrie et al., Deep Generative Models for 3D Linker Design. Journal of Chemical Information and Modeling, 2020.\n\n# 4. Benchmarking\nRegarding conditional PROTAC linker generation, we leave it for future work (Sec. 4, lines 435-438).\n\nRegarding the rediscovery benchmark, we include the recovery rate (Appendix E.3.4) as the evaluation metric in Tables 1 and 2.\n\nRegarding sample efficiency, please refer to **the response to Reviewer 3rte: Performance with different sizes of PROTAC datasets**. \n\n# End of Response\nWe hope that our responses have effectively addressed all of your concerns. If so, we would appreciate it if you could increase your rating accordingly.", "claims_and_evidence": "Partially. See my detailed review above.", "methods_and_evaluation_criteria": "Partially. See my detailed review above.", "theoretical_claims": "Fine.", "experimental_designs_or_analyses": "Analysis is incomplete. See my detailed review above.", "relation_to_broader_scientific_literature": "Decent review of the broader scientific literature.", "essential_references_not_discussed": "Perhaps Link-INVENT https://doi.org/10.1039/D2DD00115B", "other_strengths_and_weaknesses": "The paper is overly convoluted, with a heavy focus on the math and derivations for the domain-adaptation via the density ratio, without a strong-enough emphasis on if the evaluations are the most relevant for the model. Overall the paper could be restructured for clarity.", "other_comments_or_suggestions": "Overall it is not a bad paper, I want to make that clear, I quite enjoyed reading it. Nevertheless, it can be improved through the analysis and experiments recommended above.", "questions_for_authors": "See my detailed review above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Dear authors,

thank you for submitting your paper that proposed DAD-PROTAC method. The revewiers found the topic important and timely. Your proposed algorithm is a diffusion model that leverages density ratio estimation for effective fine-tune on molecular generative models on PROTAC linker design tasks. 
Your paper addresses the domain shift between small-molecule data (e.g., ZINC) and PROTAC chemical space that contains much bigger molecules. By utilizing the correction of the score function you managed to avoid a full retraining of the generative model. 

In numerical experiments, the reviewers noted that your trained model achieves strong performance across a variety of metrics and your method demonstrates sample efficiency, and is supported by thorough ablation studies.

I would like to strongly encourage to revise your paper for camera-ready version and incorporate all the clarifications that were prepared during the rebuttal phase.

Congratulation again,

best, AC
Decision: Accept (poster)