TacoGFN: Target-conditioned GFlowNet for Structure-based Drug Design
Authors: Tony Shen
Seonghwan Seo
Grayson Lee
Mohit Pandey
Jason R Smith
Artem Cherkasov
Woo Youn Kim
Martin Ester
Venue: Accepted by TMLR ()

Abstract: Searching the vast chemical space for drug-like molecules that bind with a protein pocket is a challenging task in drug discovery. Recently, structure-based generative models have been introduced which promise to be more efficient by learning to generate molecules for any given protein structure. However, since they learn the distribution of a limited protein-ligand complex dataset, structure-based methods do not yet outperform optimization-based methods that generate binding molecules for just one pocket. To overcome limitations on data while leveraging learning across protein targets, we choose to model the reward distribution conditioned on pocket structure, instead of the training data distribution. We design TacoGFN, a novel GFlowNet-based approach for structure-based drug design, which can generate molecules conditioned on any protein pocket structure with probabilities proportional to its affinity and property rewards. In the generative setting for CrossDocked2020 benchmark, TacoGFN attains a state-of-the-art success rate of 56.0% and -8.44 kcal/mol in median Vina Dock score while improving the generation time by multiple orders of magnitude. Fine-tuning TacoGFN further improves the median Vina Dock score to -10.93 kcal/mol and the success rate to 88.8\%, outperforming all optimization-based methods.

Published in Transactions on Machine Learning Research (09/2024)
Target-conditioned GFlowNet for Structure-based Drug De- sign
Tony Shen"", Seonghwan Seo', Grayson Lee", Mohit Pandey", Jason Smith", Artem Cherkasov", Woo Youn Kimbedd Martin Ester*
Corresponding author. Contact: tony_shen 40sfu.co, ester@cs.sfu.ca
School of Compuling Science, Simon Fraser University, Burnaby, Canada.
Department of Chemistry, KAIST, Daejeon, Republic of Koreo.
Graduate School of Data Science, KAIST, Daejeon, Republic of Korea.
HITS Inc., Scoul, Republic of Korea.
Vancouver Prostate Centre, University of British Columbia, Vancouver, Canada.
Reviewed on OpenReview: https: // openreview. net/ forum? id= N8cPU95000
Abstract
Searching the vast chemical space for drug-like molecules that bind with a protein pocket a challenging task in drug discovery. Recently, structure-based generative models have been introduced which promise to be more efficient by learning to generate molecules for any given protein structure. However, since they learn the distribution of limited protein- ligand complex dataset, structure-based methods do not yet perform aptimization-basee methods that generate binding molccules for just one pocket. To overcome limitations on data while leveraging learning across protein largets, we choose to model the reward distri- bution conditioned on pocket structure, instead of the training data distribution. We design TAcoGFN, a novel GelowWeet-bssed approach for structure-based drug design, which can generate molecules conditioned on any protein pocket structure with probabilities propor- tional to its affinity and property rewards. In the generative setting for Docked2oc benchmark, TACoGFN attains state-of-the-art success rate of 56.0% and -8.44 kcal/ mol in median Vina Dock score while improving the generation time by multiple orders of mag- nitude. Fine-tuning TAcoGFN further improves the median Vina Dock score to - -10.93 kcal/mol mol and the success rate to 88.8%, outperforming all optimization-based methods.
1 Introductions
Structuro-basod drug design (SBDD) leverages target protein structures to search for high-affinity drug molecules. Due to the growing availability of protein structures from ML protein structure prediction meth- ods (Jumper et al. 2021), and many novel targets identified from high-throughput perturbation experiments [Replogle et al. ) SBDD is becoming an increasingly powerful approach in drug discovery. It currently takes 13-15 years and between US $2 billion and $3 billion for a single drug to be developed and approved (Pushpakom et al., 2018). The substantial expense and time of drug development not only impose a sig- nificant burden on healthcare systems but also amplify socictal risks during global health criscs, such as COVID-19. There an urgent need to expedite the design of novel drug candidates for new protein largets.
Traditionally, virtual screening has been used to discover binding molecules by predicting supramolecular interactions between ligands and target protein with molccular docking. Its efficacy is impeded by the exhaustive nature of its search, and by the high computational cost of molecular docking. To overcome this challenge, many optimization- based methods (Bengio et al. 2021 Fu et al.) 2022 Lee et al.) 2023) Reidenbach (2044) have been proposed to generate high-affiity molecules for one protein pocket only. These methods typically do take protein structure as input and use docking measures as the reward. Recently, structure-based generative models (Luo et. al. 2021
Peng et. al. 2022 Guan al. 2023b) have been proposed

---

Published in Transactions on Machine Learning Research (09/2024)
to design molecules (ligands) conditioned on the pocket structure. These methods learn the distribution of training dataset of protein-ligand complexes. Structure-hased generative model has the advantage of learning generalized protein-ligand interaction patterns by leveraging different pocket structures during training.
However, due to the high cost of the experiments, the size of the training datasets for SBDD, i.c. high-quality protein-ligand binding structure data, relatively small. PDBBind (Liu et al. 2014), the underlying dataset of the standard Cocke2ocke benchmark (Francoeur et al. 2020), contains only 19,443 protein-ligand complexes. After removing common biomolecules (lipids, peptides, carbohydrates, and muclcotides) and duplicates, only 4,200 unique drug-like molecules remain (Powers et al. 2023). This is only tiny fraction of the entire chemical space of drug-like molecules that is assumed to consist of 10 molccules (Lipinski et al. 1997). As a consequence, existing (tructure-based generative models relying on data-distributin learning model only a very small part of the overall chemical space and have struggled to generate novel molecules with significantly improved properties (Lee al.) )023); Therefore, existing faructurebbase generative methods do not yet outperform optimization- methods on a single target.
In summary, many current optimization- based methods cannot leverage learning from across protein pocket structures they are limited to a reward function based on affinity to one single protein target per model. On the other hand, while structure-based models can learn generalized protein-ligand interaction patterns by training on different pockets, they are restricted to modelling a limited data distribution. Therefore they are unable to effectively explore the wider chemical space for desirable molecules, and under perform optimization- based methods. Our goal in this paper is to both leverage learning across protein target structures, and overcome the challenge of limited data distribution. To the best of our knowledge, TacoGFN is the first RL model to address the challenging task of modelling a family of reward functions induced from all pocket structures.
In this paper, we frame the task of structure- baseed molecule generation as learning reward distribution instead of a training data distribution, and adopt GFlowNet Bengio et al. (2021) - an energy-based generative model for generating combinatorial objects. We propose TACOGIN, a Targel Conditioned Generative Flow Network that generates molecules conditioned on any given protein pocket structures, guided by affinity to the pocket, likeness and synthesizability measures. This formulation allows us explore the greater chemical space we are longer constrained to fixed dataset. We performed an experimental evaluation the standard CrossDocker benchmark dataset and demonstrated that TAcoGFN clearly outperforms state-of- the-art structure-bard based generative methods, improving the success rate to 56.0% from the previous best of 24.5%. TACoGFN shows even stronger performance with fine-tuning (TACOGFN++P)) resulting in -10.93 kcal/mol in median Vina Dock score and up to 88.8% in success rate, outperforming all optimization- based models.
To summarize, the main contributions of this paper are:
We point out the limitations of previous data-based distribution learning, and instead propose to frame structure-basee based molecule generation as learning from the reward distribution. To this end, we propose TACoGFN, the first application of GFlowNett for learning family of molecule distributions conditioned on protein pocket structures.
We introduced a novel pharmacophore- based affinity predictor, with improved generalization capa- bilities using a pre-trained pharmacophore representation, enabling fast affinity evaluation.
TAcoGFN clearly outperforms the state-of-the-art structure-based methods on the Cross- Docked2020 benchmark and demonstrates the benefit of reward-based distribution learning.
FFC outperforms the optimization mothods focused on a single pocket only, learning from diverse molecules for various protein pocket structures. TFT attains the best. Vina Dock score, high-affinity rate and success rate among all optimization based methods.
2

---

Published in Transactions on Machine Learning Research (09/2024)
Pharmacophore Modeling (Section
Reward Function [Section
m(alg) bar) a i lep)
a(ali hir)
In
Dooking Score Predictor (Section 4.3)
KNN graph O GVP-GNN
Softmax
T(a 9
Softmax
Tlae: pi)
Drug. Uknnas (GED)
GFlowNet
GFlowNot
Pharmacophore
(Section 4.1)
Synthetic Access bilty (SA)
Fragment to Fragment
Done
Target Pocket
Generated Molecule
Reward
Target conditioned GFlowNet (Sectoon422
Model Training)
Figure 1:
Overview of the sampling and training process of TACOGEN.
2
Related Work
Structure-based molecular generation aims generate high-affinity molecules for any target pocket. They are expected to generalize previously unseen test pockets, and therefore, do not have access to the docking oracle for the test pockets during inference or training time. The goal for evaluating the quality of molecules generated for unseen protein structures is to measure whether SBDD models have learned generalized protein-ligand interaction patterns during training. Many approaches for this problem setting has been proposed. LiGAN (Masuda et al. 2020g uses 3D CNNs to encode the protein pocket structure and prodict atom densitics from the encoded latent space. 3DSBDD (Luo et al., 2021) and Pocket2Mol (Peng) et al. 2022) adopt. an auto-regressive approach to generate molecules alom by alom. Other methods such as FLAG (Zhang et al. 2023b) and DrugGPS (Zhang & Liu 2023) build molecules fragment by fragment to leverage the chemical prior. A very recent line of research employs diffusion models (Guan et al. 2023a by Schneuing et al. 2023) for SBDD. TargetDiff (Guan et al. 2023a is diffusion- method which generates
atom coordinates and atom types in a non-autoregrressive way, and bonds are generated in a post-processing step. DecompDiff (Guan et al.) 2023b) is a diffusion model which generates both atoms and bonds with decomposed priors, which reflect the natural decomposition of a ligand molecule into arms and scaffold. A recent analysis paper (Harris et al.) 2023) questions the assumption that explicit 3D modelling of the ligand improves performance, after finding much higher occurrence of physical violations and fewer key interactions in molecules generated using 3D modelling. In our work, generate molecules in 2D space to vastly reduce the search space and compute time; Our approach is shown to effectively leverage the target pocket structure for generating molecules with high affinity.
Optimization-based molecular generation aims to generate molecules that satisfy certain optimization goals. Compared to the distributton--bseed generative model, these methods can be designed to optimize for molecules with strong binding affinity to a particular protein target. Since they have access to a docking oracle during training or inference time, they are not directly comparable to the structure- based generative problem setting. Reinforcement Learning (RL) methods such as ReLeaSE. (Olivecrona et. al. 2017 MolDQN (Zhou
let al. 2019) and REINVENT (Blaschke et al. 2020 have been proposed to guide the generation of molecules toward desirable properties. MORLD (Jeon & Kim 2020) and MoleGuLAR (Goel et al. 2021b) combine
RL and docking calculations to design novel ligands. RGA Fu et al. 2022 proposes a variant of a genetic algorithm that guided by reinforcement learning and pre-trained on multiple protein target structures.
MOOD (Lee et al. (2023) incorporates out-of-distribution and property-guided exploration in diffusion models for goal-directed molecule generation. DecompOpt Zhou et al. (2024) combines pre-trained structure-based equivariant diffusion model with a docking based greedy iterative optimization loop. EvoSBDD Reidenbach (2024) improves efficiency by performing black-box optimization over the 1D latent space using a docking oracle. In summary, optimizaation-baseed molecular generation evaluates the performance of the algorithm on optimizing for seen targets. Compared to the structure- based setting, which requires modelling the molecule distribution conditional to any protein pocket structure and generalizing to unseen protein pockets, optimizing the molecule distribution for a single protein pocket is a less challenging task. We refer readers to more detailed comparison of the problem setting in Appendix B
3

---

Published in Transactions on Machine Learning Research (09/2024)
Protein-ligand affinity prediction. Predicting the affinity or the docking score of a ligand to target, in the absence of their binding complex structure, is difficult. task. Most previous docking score prediction models have been limited to a single protein target (Bengio et al. 2021 Gentile et al. 2020). Therefore, they are not suitable for the aim of designing high-affinity molecules for any given protein pocket structure. Recently, several methods have been proposed to predict ligand affinity for arbilrary prolein targets (Zhang et al., 2023a Pandey et al. 2022]. However, these approaches are prone to memorizing the structural bias instead of learning the physics of protein-ligand binding and show low generalization ability to unseen ligands proteins (Wallach & Heifets, 2018 Chan et al., 2023). To this end, we adopt pre-trained pharmacophore representation, which only models the key interaction sites for protein pocket. This prior improves our docking score predictor's ability to learn physical interactions and generalize to unseen data.
3 GFlowNet Preliminaries
Generative Flow Networks (GFlowNets, GFN) (Bengio et al. 2021 learn a stochastic policy æ for generating a combinatorial object (such as molecular graph) z E X. The probability of constructing z, denoted as x(x), is trained to be proportional to non-negative reward function R X' R+ defined on the space X. This property of GFlowNett is ideal for generating diverse molecules with desirable properties. Conditional GFlowNet introduced in (Jain et al. 2023) simultaneously models family of reward functions. Each conditional information, denoted as c C, induces a unique reward function R(zlc). In our work, we adopt conditional GFlowNet for SBDD settings, by encoding target pocket structure as condition c, where e is a high dimensional representation of the pocket structure. Thus, a single GFlowNet models high-reward molecule distribution across all protein pockets.
Each object is constructed from sequence of actions A. In molecular scttings, molccule is constructed by inserting molecule fragments into a partially constructed fragment graph state E S (Bengio et al, 2021 Conceptually, GelowNet is acyelic graph G (S, E), with nodes S and edges E. Fach transition + s' E via action a E A corresponds to an edge in graph G. The transition function T SSA + S computes the new state s' = =((s a) given action a on state s. A special action moves state s into terminating states c S. We define the initial emply graph as sg. Construction of x can be defined over a trajectory of states (80
Following previous studies, we introduce an exponent 3 to the reward function R(zlc), thus modelling (<]e,, () x R(xle)². This steers generating probability distribution to focus the modes of R(xle), which is crucial for producing candidates that are high in reward. Adjusting 3 allows us to manage the balance between diversity and achieving higher rewards.
The forward transition probability PF(s'|s, [s, c, 3) of a GFlowNet represents the probability distribution of reaching state s' from state conditioned on context and reward temperature B. Partition function Z(c, B) X under context e. We adopt the Trajectory Balance
is the sum of the rewards R( 8 for all objects objective from Equation [ to efficiently learn forward transition policy Pr that generates object with probability proportional to its reward R(x[c)² (Malkin et al. 2023).
CTB(T,C,B:0) = log Zo 3) II Toweer Pro(o')s,,,3) R(x|c)8
(1)
0 represents the learnable parameters. Our goal is to model the probability ( B) ~ molecules E X, protein pocket contexts C and various reward temperatures 8.
R(zle)
across all
Z(e,3)
4 TacoGFN
TACOGFN is a structure-based molecular generative model that generalizes over all protein pockets with a single model. By matching the reward distribution instead of limited data distribution, our method explores the greater chemical space to generate high-affinity molccules with properties desirable as a drug candidate. Furthermore, by encoding pocket structure information, TACoGFN and its fine-tuned variant are able leverage learning from diverse protein pocket structures.
4

---

Published in Transactions on Machine Learning Research (09/2024)
Problem definition. The goal of structure- drug design (SBDD) problem is to generate molecules with both desirable properties and strong binding affinity with respect to any given protein pocket structure. The goal for SBDD models is learning generalized protein-ligand interaction patterns during training. Therefore, SBDD models take pocket structures as input and are expected to gencralize for unseen pocket structures. We refer readers to a detailed contrast of the SBDD problem setting with the opiimizaiion setting in Appendix
The pocket structure is denoted as P and will be represented as K K-nearest neighbour (KNN) residue graph. The ligand is denoted as L and will be represented either as atom graph fragment graph. We define reward function P[L[P) based on a molecule's predicted docking score, drug-likeliness and synthesizability. Our goal is to learn a molecule generation policy that constructs molecules L given protein structure P with probability matching their reward (exponentiated with 3), such that =LL)P, B) x R(L[P)².
Method overview. The protein pocket is first fcaturized as a K-ncarest neighbor residue graph and encoded using GVP-GNN (Jing et al.) 2021] (section: 4.1). Then, we use this pocket embedding as the condition for GFlowNet molecule generation (section 4.2 Finally, reward generated molecules using
docking score prediction model which leverages protein-ligand interaction priors (section 4.3). The high-level architecture of our SBDD framework is illustrated in Figure 1
4.1 Pocket structure encoder
We represent structure of the pocket P as a standard K K-nearest neighbor (KNN) residue graph GP = (VP, ,EP) following previous work in protein representation (Jing et al.) 2021). The i-th residuc node of VP is featurized using its geometrie and chemical properties. These features include the type of residue, the dihedral angles of the atoms in the residue backbone, and the directional unit vectors. An edge eiy E E is formed if the j-th residue v/ is among the K-nearest neighbors of residue i P measured by the cuclidean distance between their respective C. atoms. We set the number of neighbours K = 30. An edge is featurized with the Euclidean distance, distaance along the backbone and the direction vector between the two residues. These features sufficiently describe the features of the protein pocket.
We apply a graph neural network with geometric vector perceptrons (GVP) layers (Jing et al.) 2021 to the KNN pocket graph G to learn the node embedding h,p for each residue. The node embeddings (h,r) are then averaged to obtain embedding of the entire graph hgr. We use GVP because it encodes the protein pocket into an embedding that is invariant to rotations and translations.
4.2 Pocket conditioned GFlowNet
In this section, we discuss how to employ the pocket structure, more specifically its latent embedding, to condition the GFlowNet to generate molecules that interact with a given protein pocket. Furthermore, we describe our fragment-baser molecular generation framework.
Molecule representation. During molecular generation, we represent ligands as 2D molecular graph G² = -VV4, with node of E V representing molecule fragment, and directional edges etj E & indicating the attachment atom of fragment of that connects to fragment v/. Since a molecule's 's reward (representing its desirability as real-world drug) should be the same regardless of its predicted 3D conformation, we represent ligands as 2D graphs here.
Fragment vocabulary construction. TACoGFN generates molecules by adding one molecular fragment at a time. To create the vocabulary of fragments used, we extract common fragments from a chemical database a data-driven and chemically valid way. To obtain a fragment vocabulary, we first apply BRICS decomposition (Degen et al. 2008 to 250k ZINC20 (Trwin et al. 2020) molecules. BRICS breaks molecules
molecule's 3D conformation. However, consider the best docking score for a molecule the reward here, which is invariant to its conformation in this context. Taking the best ducking score valid appreach, bocause when compound tested for binding in the wet lab, the compound simply binds the protein with the conformations which results in the strongest affinity. Furthermore, the predicted 3D position of generated molccules from existing diffusion- based SBDD models often changes significantly upon re-docking and likely do not reflect the conformation. (Reidenbach 2024).
Docking score is dependent
5

---

Published in Transactions on Machine Learning Research (09/2024)
via retrosyuthetic rules and provides synthetically accessible building blocks, i.c. molecules that are easy to prepare (Seo et. al. 2023). To reduce the fragment vocabulary size, we further break all single bonds connecting a heavy atom to a ring structure. Next, we retain the fragments that occur in more than 50 (or 0.02%) of the molecules in our ZINC set. Only atoms (of a fragment) in the bonds decomposed by the BRICS can join to form new fragment connections during generation time; This reduces the occurrence of non-synthesizable attachment points forming bonds. This results in a set of 475 building block-like fragments. As our fragments are mined from synthetically accessible virtual library and poor synthetic accessibility is penalized in the reward function, TACOGFN achieves superior synthetic accessibility compared to all existing SBDD methods.
Molecular generation framework. We formulate molecular generation as a sequential decision process and implement it using a GFlowNet. At the t-th step, the forward action policy Pa samples an action a depending on the current molecule state O.¹ and pocket embedding hgg. The transition function T is a deterministic function which applics the action the molecular graph at stop t to produce a molecule graph al step 1+1.
at Pa(alg,, hhg)) 0441 T(a,, 0/r)
(2)
(3)
Previous autoregressive models often formulate action prediction as supervised task, where the goal is to predict the correct ground truth actions obtained from masked molecules (Peng et al. 2022). Instead, our molecule generation policy Pa aims to generate molecules with probability P(g-Tha") proportional to the
reward.
Pocket conditioned molecular generation. Here, we adopt the architecture for molecular generation in- troduced in Multi-Objective GelowNet (MO-GFN) (Jain et al. 2023). Instead of conditioning the GelowNet on multi-objective preference, we condition the GFlowNet on pocket embeddings to learn family of molec- ular distributions corresponding to family of reward functions induced from the pocket structure diversity,
We use a graph transformer (Yun et al. 2020) to model the probability (a g. hGr), by taking the partially constructed molecular graph al the 1-th time step G. and the pocket embedding hgg as input. The input feature h L(0) of molecular node of is one-hot encoding of the node's fragment type. The molecular edge (((0) is a one-hot mapping of the attachment atom index in node of which connects to node input feature eij
u/. We add an additional virtual conditioning node h h²)), featurized using pocket embedding hgr, to the graph P. (Pham et al. 2017). This virtual node connected to all other nodes and serves as a graph-level node to provide pocket information. After N Transformer layers, the set of final node embeddings {hā (h,(()) and edge embeddings {et are obtained. The final graph level embedding gL(N) is obtained via the concatenation of the global average pooling of the node embeddings and the final virtual node embedding h (M),, as seen in Equation
L(N) = Concat (AugPool ((n,am1) ,h , ((A) g
Using these final molecular graph embeddings, we follow the actions defined in previous works on fragment- based molecular generation (Bengio et al. 2021 Jin et al.) 2018). Full details can be found in Appendix
C.1.1
Reward function. A drug candidate must not only have high affinity to the pocket but also satisfy drug-like properties and synthetizability requirements to be selected for experimental validation. We design a reward function by multiplying the normalized score from all three aspects, using QED (Bickerton et al., 2012) as a measure of likelnise,,,,es, SA (Ertl & Schuffenhauer 2009)) as a measure of cese-of-sytthessiabbltty and predicted docking score as a mcasure of affinity between ligand and protein In drug discovery, is crucial to design molecules that simultaneously satisfy all of these relevant properties, despite the inherent trade-offs among these propertics. By multiplying the scores, the reward function ensures that a low score in
2Computing the reward very fast for example, computing for batch of 64 protein-ligand pairs took under 0.15 seconds.
6

---

Published in Transactions on Machine Learning Research (09/2024)
Feature Extraction
Concat MLP
h
Global Pooling
2
Pharmacophore Modeling
Feature Extraction
++E
Map
Target Packet
Pharmacophore
GNN
Ligand g
Figure 2:
Model architecture of the docking prodictor. Each pharmacophore point is represented sphere and corresponds to desired ligand characteristie for binding interaction.
any one of the factors (QED, SA, or DS) will significantly reduce the overall reward. Thercfore, multiplying the scores is
appropriate choice than summing the scores here.
We just have to optimize QED and SA up to certain threshold for molecule to be suitable as drug candidate optimizing them further does not bring additional utility (Coley 2021). Therefore, we clip the reward for the QED or SA component to 1 when they achieve their respective threshold EoED and tSĄ- For example, the reward function will prioritize more on optimizing QED if IQED is higher, at the expense of other properties such as affinity (See details of reward function in Appendix C.1.2)
4.3 Docking score predictor with pharmacophore prior
The exploration of chemical space with a GFlowNet requires evaluating binding affinities for millions of molecules sampled during training. However, using molecular docking for evaluation is computationally expensive. Here, propose fast ML-based docking score predictor that generalizes well across molecule and protein pocket distributions, as described in Figure 2 and Equation 5 (See details in Appendix C.2
We leverage PharmacoNet (Seo & Kim 2023), a recent deep learning method to obtain the pharmacophore from a pocket structure P.To enrich the representation, we obtained the embedding of the pocket z" and the emboddings of pharmacophore points (h{) from PharmacoNet. The 1D representation vectors of the pharmacophore z" is computed by the global pooling of (h.)-
During docking score prediction, we represent ligand as an atom-level molccular graph. We apply Graph Isomorphism Network (Hu et al. 2019) the atom graph to obtain the node embeddings (h²). zf is a ID representation vector the ligand, obtained by the global pooling of node embeddings {hf). Then, incorporate pairwise interaction map I- computed from the outer product of (h and (h/)- This preserves the structural topology and binding interaction details. Notably, our docking score predictor uses the pharmscophore points involved in binding instead of aloms amino acids to obtain the interaction map. It improves the generalization ability at a reduced computational cost through coarse grained modelling of the pocket at the pharmacophore level.
E (Concatt ,,,,,,,,,' ,²,z²)) + SumPool(
(5)
3Pharnacophore point sct, where each point deseribes the desirable motif propertics (such being aromatic) ligand should possess this geometric position energetically entropically favourable interactions (e.g. n--r stacking) with the protein target. (Wermuth al. 1998 Yang 2010).

---

Published in Transactions on Machine Learning Research (09/2024)
5 Experiments
5.1 Dataset
We train and evaluate TACOGFN on the commonly used CrossDocked benchmark Francoeur et al. 2020
We first apply the splitting and processing protocol on the CrossDocked dataset obtain the same train and test split of the 100k protein-ligand pairs as previous methods (Luo al., 2021; Peng al. 2022 Guan et al., 2023b). (See details in Appendix D.I). We then train docking score predictor on the training split of the protein-ligand pairs to predict their corresponding Vina Dock scores. Since TACoGFN is trained on the same set of protein-ligand pairs and evaluated on the same unseen pockets, our experimental results can be directly compared against the published results of existing structure- generalive models.
5.2 Training
We train one TACOGFN model to generate molecules conditional to any protein target structures using predicted affinity, Synthetic Accessibility (SA), and drug-likeness (QED) as the reward. We also adopt Double GFN (Lau et al. 2023) to improve exploration in sparse reward domains and high-dimensional states, by initializing two networks to model the action policy: an online network and target network.
For each training trajectory, one protein pocket. is randomly drawn from the Cross training set first. Then, molecule is generated through sampling sequence of actions using our target network which models the forward action policy Ptarget- The reward is then computed for that molecule with respect to the protein target. The Trajectory Balance loss (Equation I is used to train the online network modelling the action policy Pontine- This loss has the objective for our model to generate objects with probabilities proportional to the rewards (consisting of predicted docking score, QED and SA). Periodically, the weights of the target network are updated by the weights of the online network using a delayed strategy. This strategy reduces training instability and promotes explorations of the larger chemical space.
5.3 Evaluation
In all evaluations, each saructurobbase generative model tasked produce 100 molecules (ligands) for each of the 100 unseen protein pockets from the CrossDock-1000 test set.
Evaluation metrics. We adopt the following commonly used metrics from Guan et al. 2023a and Rei- denbach (2024): (1) Validity is the percentage of unique generated molecules free of reconstruction errors and disconnections as determined by RDKit. (2) Vina Dock approximates the binding energy between a generated molccule and a protein pocket, where a lower docking score indicates a higher binding affinity. (3) High Affinity measures the percentage of generated molecules with higher affinity than the reference molecule. (4) QED is a measure of drug-likeness, estimating a molecule's suitability as an oral drug based on its properties (Bickerton et al. (2012). (5) Synthetic Accessibility (SA) estimates difficulty of synthe- sizing the molecule (Ertl & Schuffenhaner 2009). The score is normalized between 0 and 1 using the formula (10 SA)/9. (6) Diversity is calculated as the average pairwise fingerprint Tanimoto distance bet between molecules generated for a pocket. (7) Success Rate is the percentage of molecules which pass the same criteria (QED> > 0.25, SA > 0.59, Vina Dock < -8.18) as in Long et al. (2022 Guan et al. (0023b); Zhou et al. 2024); Reidenbach 2024 (8) Time the average runtime (in seconds) for generating 100 unique
and valid molecules for a pocket.
Baselines and problem settings. Similar to Zhou et al. (024)) and Reidenbach 2024 existing methods by their problem definitions:
we separate
1) Gencrative methods are expected to generalize for pocket structures unseen during training. They generale molecules for test pockets at inference time without optimization loops or access to docking pro- grams. Therefore, methods under this setting are only allowed to generate 100 molecules for each pocket in ono-shot. We compare TacoGFN trained on CrossDockod-1000 against the following generative models:
liGAN (Ragoza et al. 2022). GraphBP (Liu al. 2022). AR (Luo et al. 2021). Pocket2Mol (Peng
al. 2022), Tifget Guun et al. 2023a DiASSBDD (Schneuing al.
2023 DecompDiff Guan

---

Published in Transactions on Machine Learning Research (09/2024)
Table 1: Comparison of the properties of molecules generated in the generative problem setting for the CrossDocked test set pockets. The reference molecules are from the CrossDocked test set. The best results are in bold. The average and median values are calculated over the averages for each pocket. The prior method results are taken from their publication.
Validity (1)
Vina Dock (L)
High Affinity (*)
QED (1)
SA (t)
Diversity (1)
Success Rate (?)
Time (4)
Model
Avg.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Gen
Reference
-7.45
-7.26
0:48
0.47
0.73
0.74
25.0%
IGAN
-6.33
-6.20
21.1%
11.1%
0.39
0.39
0.50
0.57
0.67
3.9%
GraphHP
.4.80
.4.70
14.2%
6.7%
0.43
0.49
0.48
0.79
0.78
0.1%%
AR
92.95%
-6.75
-6.62
37.9%
31.0%
0.51
0.63
0.63
0.70
0.70
7.1%
19659
Podeet2Mol
98.31%%
-7.15
-6.79
48. 4%
0.56
0.57
0.74
0.75
0.09
0.71
24.4%
2504
TargetDiff
90.35%
-7.80
-7.91
58.19 58.1%
59.1%
0.48
0.48
0.58
0.58
0.72
0.71
10.5%
3/28
DIISBDD
85.01%
-8.03
-7.75
55.3%
56.6%
0.17
0.47
0.55
0.56
0.76
0.76
6.0%
160
DitompDiiff
71.96%
-8.39
.8.43
64.4%
71.0%
0.45
0.43
0.61
0.60
0.68
0.68
24.5%
6189
TACoGEN (Ours)
100%
-8.24
-8.44
67.5%
92.0%
0.67
0.67
0.79
0.79
0.53
0.53
56.0%
et al. 2023bb- For consistency with existing baselines, molecules are docked using QVina this problem setting (Trott & Olson 2010 Alhossary et al. 2015). For details on the docking protocol for the generative setting, please see Appendix D.2
2) Optimization methods are able to leverage docking on the target pocket. Unlike the generative setting, these methods typically iteratively optimize the candidate pool and select the top 100 molecules. We compare against the following methods: RGA (Fu et al.) 2022) and EvoSBDD (Reidenbach 2024) use a black-box algorithms to conduct rounds of the optimization process, with molecule fitness based on docking score to the target pocket. DecompOpt and Tff (Zhou al. 2024) optimizes molecules generated by pre-trained SBDD models via rounds of optimization process involving re-docking to the target pocket. TacoG fine-tunes pre-trained TAcoGFN to tailor the model to the target pocket using docking as the reward.
5.4 Experimental results
Table compares TACOGFN against other baselines in the generative problem setting for SBDD. We ex- amine the docking score performances for individual pockets in Figure 4 and 5 We also show examples of generated molecules in Figure 3 and analyze their average physical properties in Table [ We then compare TACOGEN+FT against, existing methods for the optimization problem setting in Table 3 and 4 where docking is used in optimization rounds, Lastly, we conduct ablation studies to show the benefits of using a larger docking score datasot Table and validate the utility of the pocket conditioning in Table 6
Generative setting. Table highlights the strong performance of TAcoGFN in the generative problem setting for the DookeD test set pockets. Notably, TACoGFN boasts a significant improvement in success rate at 56.0%, more than doubling the previous best of 24.5% achieved by DecompDiff. This improvement is due to TAcoGEN's ability to generate high-affinily molecules that simultaneously satisfy the drug-likeness and case-of-synthesis requirements. In fact, TACOGFN records the best QED, SA, and High Affinity si- multaneously among all generative methods. In Figure 3 we show examples of molecules generated by TACoGFN and show its ability to generate molecules with significantly improved docking scores compared to native ligands.
It is difficult to discover molecules that simultaneously exhibit better QED and Vina Dock compared to known binders and molecules from existing baselines due to the trade-off between Vina Dock and QED. Molecules with higher molecular weight are more likely to have strong Vina Dock because of the presence of more interacting atoms, but they tend to have worse drug-like properties (QED). 4 In Figure ! we show the performance breakdown for the 100 test protein pockets. TAcoGFN achieves better average Top-10 Vina Dock than Diff in 57% of the test pockets. Notably, the top molecules generated by TACOGFN
This trade-off clearly observed with Pocket2Mol and DecompDiff (see Table Dock but lower QED, while Moc exhibits the opposite trend.
where DecompDiff has strong Vina
9

---

Published in Transactions on Machine Learning Research (09/2024)
Outs (dive))
Reference (diwe)
Vina -99 Nov: 0.52
Vire 991 Nov: 0.56
Vina 104 Nov 049
Vira 10.1 Nov: 0.51
Vina -90 Nov: 0.51 Vira: 10.7 Nov: 0.48
Vira -8.7
QED: 0.83
QED: SA: 0.81
QED: SA: 0.78
QED: 0.68 SA: 0.81
QED: 0.73
QED: 0.74 0.72
QED: 0.8
Ous (dg(tt))
Refarence (4:8k)
Vina-833 Nov: 0.56
Vine -7.5 Nov: 0.36
Vina -7.1 0.58 Vina 0.56 Vina
Nov: 0.88 Vina Nov:
Vina -.11
QED: 061 SA: 081
QED: 0.62
SA: 0.73
QED: 0.72 SA: 080
QED: 0,73 SA: 0.80
QED: 0.61 SA: 0.65
QED: 0.70 SA: 0.82
QED: 0.76 SA: 0.89
Our method is focused on de novo hit discovery finding novel and diverse high-scoring hits for a protein target. Our method does not optimize based on given reference soed compound. The goal of reward-based sampling is to sample diverse high-scoring molecules. To provide fair overview of the model's performance, selected protein pockets 4ing and 4980, which are at the 25th and 75th percentiles, respectively, based on their docking scores with their native ligands. We show compare the molecules generated by TACOGFN against the native ligand with their QED, SA, Novelty, and Docking score (Vina).
Figure 3:
also consistently demonstrate higher QED values. As shown in Appendix Figure 5 in the pockets where DecompDiff achieves a lower Top-10 Vina Dock, the molecules often exhibit molecular mass larger than 500 daltons. These heavicr molecules violate the idcal properties of orally active drugs according to the Rulc of 5 @Lipinski el al. 1997). In addition, heavy molecules with high docking scores are more likely to be false positives (Pan et al. 2002). TACoGFN able to find molecular spaces that improve both QED and Vina Dock requirements not only by modelling the reward distribution but also by exploring a broader chemical space. This is achieved through learning from generated examples using online policy, rather than being limited to the training data.
We note exploring millions of molecules online could not be easily achieved with existing SBDD baseline. This is because the generation process of TACOGFN is a few orders of magnitude faster than existing autoregressive or ciitusion-baseyy generative methods. Additionally, TACOGFN achieves 100% in validity and uniqueness, demonstrating the efficiency of our fragment-based 2D generation framework. As shown Table TAcoGFN achieves significant improvement in both time and validity over previous methods.
Molecules generated from TACOGFN have more ideal molecular weight and drug-likeness properties, in addition to achieving better Vina Dock; Therefore, they are more suitable as drug candidates. TACoGFN samples al reward lemperature 8 of 64 at inference time, resulting in the modelling of the probability p(z|c) & ((zzc))z This policy focuses more on the modes reward function. Therefore the molecules sampled have higher quality but lower diversity than those generated by the methods that do not attempt to satisfy multiple objectives. By changing or reward function we can trade off rewards with diversity shown in Appendix DD4.2
10

---

Published in Transactions on Machine Learning Research (09/2024)
Averagee of Top. 10 Vina Dock
-8-
-10
-12
target
target 20
target
target J0
target
targes 20
target
target 70
carget UO
targe: 90
DecompDiff (lowest in 43%)
TacoGFN (lowest in 57%)
The average of the top-10 Vina Dock of molecules generated for individual CrossDocked test pockets (target) by DecompDiff and TACOGFN. Targets sorted by the average of the top-10 docking score of TACoGFN generated molecules. A lower docking score means higher estimated binding affinity, Color is used to denote the average QED value of molecules in the Top-10 set. A higher QED indicates the molecule is more drug-like.
Figure 4:
Evaluation of physical properties. We evaluate the following physical properties important for small molecule drugs in Table 2 (1) Ideal Mol. Wt. is the percentage of molecules satisfying molecular weight
within 160 480 daltons the typical acceptable molecular weight range for small molecule drugs (Ghose
et al. 1999)- (2) Mol. Wt. is the molecular weight of molecules in Daltons. (3) Num Heavy Atom is the heavy atom (any atom that is not hydrogen) count of the molecules generated. (4) Strain Energy is the difference between the ro-docked poses (meaning generated molecules are docked with RDKit's ETKDG conformation inilialization), versus the strain energy of their unstrained ETKDG poses. We consider the sum of torsional, steric and electrostatic strains. the strain energy of the ligand's docked pose is very high,
the physical compatibility for such binding becomes poor (Perola & Charifson 2004 Gu et al. 2021
For baseline, compare against. the top three generative methods ranked by success rate: DecompDiff Guan et al. (2023b) Pocket2Mol (Peng et al. 2022) and Target Guan al. (023a Additionally, sample random set of molecules from the ZINC virtual library for comparison.
Table 2:
Comparison on the number heavy atoms, strain energies
Ideal Mol. Wt. (t)
Mol.
Wt.
Num Heavy Atom
Strain Energy (1)
Model
Avg.
Avg.
Med.
Avg.
Med.
Avg.
Med.
ZINC molecules
98.73%
337.63
337.34
23.53
23.50
377.77
375.17
Poc
64.55%
248.65
230.99
18.29
16.86
254.21
218.52
Target Diff
78.37%
323.64
327.31
22.78
23.15
600.31
547.86
DecompDiff
51.18%
494.05
487.51
34.93
34.00
834.72
781.34
TacoGFN (ours)
99.76%
402.56
402.65
30.47
30.52
360.51
357.33
As shown in Table 2 99.76% of molecules generated by TAcoGFN have molecular weights within the 160 to 480 dalton range- the ideal range for small molecule drugs. This ratio is much higher than those of existing methods such as DecompDiff (51.18%), Pocket2Mol (64.55%) and Tifget (78.37%). For Pocket2Mol, molecules fall outside of the idcal molccular weight range because they are too small (below 160 Daltons); While for DecompDiff, most molecules are outside of range because they are too heavy (greater than 480 Daltons). We note there are indeed drug candidates with molecular weights or drug- like-lis outside of the typical acceptable range. However, they may be regarded as an exception rather than the norm (from a specialized drug class) have elevated risks for poor absorption and bioavailability (Ritchie & Macdonald. 2014).
11

---

Published in Transactions on Machine Learning Research (09/2024)
Table 3: Comparison of the properties of molecules generated in the optimization problem setting for the CrossDocked test set pockets. The prior method results are taken from their publication. The number of fine-tuning steps is 300 unless herwiee specified.
Valdity (t)
Vina Dock (1)
High Affiity (1)
QED (1)
SA (f)
Diversity (1) Med.
Success Rate (
Time (4)
Model
Aug.
Avg.
Med.
Avg.
Med.
Avy.
Med.
Avg.
Merl.
Avg.
Avg.
Gers Score.
Refermece
100%
-7.45
-7.96
0.47
0.74
25.0%
RGA
-8.01
-8.17
64.45%
59.3%
0.57
0.57
0.71
0.73
0.41
0.41
48.2%
+ Ops
-8.30
-8.15
T1.0%
96.955
0.66
0.68
0.68
0.67
0.31
0..00
25.8%
>-3726
DecompOpt
73.5%
53.35
0.45
0.65
0.60
EvSBDD (a - 13, 100R)
100%
10.27
10.35
96.55%
100%
0.53
0.52
0.75
0.77
0.63
0.63
78.85
6300
ExSSBD (a 0- a =1 140R)
100%
-10.14
-10.27
94.4%
100%
0150
0.59
0.77
0.77
0.02
0.82
80,45
63000
TooGFN FT Boep D. 0.6)
100%
-10.78
97.1%
100%
0.47
0.60
0.02
0.62
87.8%
Too (ieE 155. 0.8)
100%
-10.33
-10.39
97.0%
100%
0.58
0.58
0.82
0.82
0.61
0.61
88.8%
6230
DooGFN, FT Pgep 0.6,ts 08)
100%
-10.14
-10.21
96,255
100%
0.66
0.65
0.82
0.82
0.30
0.70
86.3%
61440
PDBbind [Liu al. 2014), the parent database of the CrossDocked set (Francoeur al., 2020), contains large number of common biomolecules such ATP (molecular weight: 508 daltons) sulphate ion (molecular weight: 96 daltons). This inherent bias in the training dataset hinders data distribution learning- based generative models from generating molecules that would be acceptable as real drugs. In contrast, TACOGFN learns the actual physical property distribution of drugs, rosulting in an average molccular weight (402.56 daltons) of the generated molecules that closely matches that of FDA-aaproved drugs (~ 430 daltons).
With regards to strain energy, TACOGFN attains far lower strain energy than DecompDiff and Target- Dilf. The strain energy of TACoGFN is comparable with that of ZINC molecules, shown in Table 2 Pocket2Mol achieves lower strain than ZINC and all other methods, due to the small molecular weight they generate. In summary, this analysis supports that molecules generated by TACOGFN show appropriate physical properties and compatibility with its binding pocket.
Optimization setting. Here we study the optimization problem setting, where methods can conduct op- timization rounds leveraging docking program on the target pocket. We evaluate TACOGFN+FT, where we fine-tune a pre-trained TACOGFN, using UniDock (Yu et al. 2023 on the target pocket as the affinity reward for generated molecules. For each protein pocket, we chose lo finetune TACoGFN for 300 steps (unless otherwise specified) to match the time for TACoGEN+FT (toED 0.5. 0.8) with EvoS- BDD's (Reidenbach) 2024). We report the time taken for fine-tuning and docking molecules for our method, measured on single A4000 GPU a GPU with less performance than the one used by EvoSBDD (A6000 (A100 GPU). The details of the fine-tuning process and UniDock
GPU), or OpsteeDDfff-O//Doooooo can be found in Appendix
TFC is able to simultaneously optimize a set of potentially conflicting objectives - consisting of Vina Dock, QED and SA. The TAcOGFN+FT with balanced objective (LyED = 0.5, tSa = 0.8) simul- taneously achieves state-of-art results in Vina Dock, high affinity, synthetic accessibility and success Rate, Since TACOGFN samples molecules during fine-tuning with lower reward temperature 3 (sampled uni- formly between 0 and 64), the diversity of TAcoGFN+FT is higher than TACOGFN and comparable with EvoSEDD. Our method variant (+QED = 0.4, = 0.6) focusing on affinity attains the best average Vina Dock of -10.78. Our method variant focusing on QED achieves a comparable QED to the best baseline TargetDiff+OOp (0.65 vs 0.66), but a much better Vina Dock (-10.14 vs -8.30) and SA (0.82 vs 0.68). Please see Appendix Table 12 for full results.
Overall, our proposed fine-tuning pipeline is highly efficient, as TACOGFN surpasses state-of-art perfor- mance while taking less time. In comparison to EvoSBDD (Reidenbach) 2024h, which does not take pocket structure as input, TACOGFN is already trained to generate drug-like and synthesizable molecules with binding conditions to protein pocket structure. Therefore, finetuning TACOGFN generates molecular set with a better success rate and high-affinity rate for a new protein pocket. Compared to DecompOpt [Zhou] et al. 2024), our method does not need to traverse the 500-1000 denoising steps to generate a molecule.
"Uni-Dock is recently proposed GPU-accelerated molecular docking program that achieves more than 2000-fold speed-up compared with the AutoDock Vina running in single CPU core.
12

---

Published in Transactions on Machine Learning Research (09/2024)
This means we could search for more molecules and discover higher reward candidates in shorter amount of time.
Table 4 compares the performance of TACOGEN+FT by simply generating an equal number of molecules with base TACoGFN and ranking the top 100 molecules without finetuning (TAcoGFN + +RAnk); Here we show, although TAcoGFN+ RANK already attains competitive results, our proposed fine-tuning meaning- fully further improves Vina Dock and Success Rate.
Table 4: Comparison of performance between TACOGFN with Finetuning (TAcOGEN+FT) with simply
ranking the same number of generated molecules from TACoGFN by their reward T RANK).
Valdity (1)
Vina Dock (1)
High Affiity (1)
QED (f)
SA (f)
Diverstty (1) Med.
Success Rate (f)
Time (4)
Model
Aug.
Med.
Mel.
Med.
Med.
Gen
TN FT Pogu 0.5.aa 08)
100%
-10.33
97.0%
100%
0.58
0.58
0.82
0.61
0.61
88.8%
TooGEN- Ronks
100%
-10.00
-10.10
97.0%
100%
0.59
0.59
0.82
0.82
0.60
0.61
86.0%
5183
5.5 Ablation studies
We conduct additional ablation studies using the base TACoGFN under the generative setting to examine the effect of docking score prediction accuracy and pocket conditioning on our method.
Effects of using higher quality docking score predictor. Here, we study the effect of using a more accurate docking score predictor, which is trained on a larger dataset, for reward. Since training a docking score predictor docs not require high-quality protein-ligand structural data such as the CrossDock-100k set, we can introduce second, larger dataset for docking score prediction called ZINCDock-15M It. consists of about 15M docking simulation data from docking 1,000 random ZINC20 (Irwin et al 2020 molecules into each of the 15,207 unique pockets from CrossDock-100k training split using QVina. We then train our pharmacophore-basee based docking score predictor on this larger dataset. Please see Appendix Table for a comparison of docking score accuracy between using CrossDock-100k and ZINCoock-15M. As shown in Table 5 TAcoGFN using the docking score predictor trained on the larger ZINCDock-15.. dataset demonstrates improvements in average Vina Dock, high-affinity rate and success rate. This confirms is possible leverage the easily generated large-scale docking score data generate more novel and higher affinity molecules.
Table 5: Evaluation of de-novo drug design performance of TACOGFN using docking score predictors trained on two different docking score datasets.
Docking score dataset
Vina Dock (4) Avg.
High Affinity(t) Avg.
Success Rate(t) Avg.
Model
Med.
Med.
Med.
TACoGFN
CrossDock- 100k
-8.24
-8.44
67.5%
92.0%
56.0%
61.5%
TACOGFN
ZINCDock-15M
-8.35
-8.53
69.5%
94.5%
58.3%
67.5%
Effects of pocket conditioning. To examine the effect of the proposed pocket conditioning for GFlowNet, we train molecular generation policy unconditioned on pocket information. The docking score predictor is unchanged meaning it still predicts docking score for a molecule with pocket information. The results are shown in Table [ We observe that the pocket-conditioned GFlowNet achieves higher docking scores compared to the GFlowNet without pocket conditioning. We further measure the number of non-covalent interactions of a molccule to respective pocket, by obtaining the binding pose using QVina (See Appendix D. We demonstrate generated molecules by pockel-conditioned GFlowNet result in more non-covalent interactions of for all categories (Hydrophobic interactions, Van der Waals contacts, Hydrogen binding) in Table D More non-covalent interactions are indicative of the molocule having better specificity to the protein target. This ablation validates that our method is indeed leveraging pocket conditioning learn a family of molecular distribution across different pocket structures.
13

---

Published in Transactions on Machine Learning Research (09/2024)
Table 6: Effectiveness of pocket structure conditioning evaluated using Vina Dock.
Docking score dataset
Vina Dock (4)
Top-10
Vina Dock (L)
Model
Avg.
Med.
Avg.
Med.
TACoGFN
11N
-8.35
-8.53
-9.97
-10.22
w/o pocket conditioning
ZINCDock-15M
-8.04
-8.18
-9.65
-9.81
6 Conclusion
In this paper, we have invostigated the problem of structure-base, drug design. To address the limitations of methods based on distribution learning, we have framed pocket-conditioned molecule generation as learning a multi-objective reward distribution using RT. To this end, we propose TAcoGFN, a pocket structure conditioned GelowNet which generate drug-like and high-affinity molecules with respect to any 3D pocket structure. To the best of our knowledge, TACoGFN is the first RL model address the challenging task of modelling family of reward functions induced from all pocket structures. Our model effectively explores the greater chemical space, through generating millions of molocules using the online policy during training. We have further introduced novel pharnaacophore-bssel affinily predictor, where coarse-graining to the protein pocket is shown to achieve more accurate and robust predictions than existing architectures and protein representations. We finally introduce TACOGFN+FT which fine-tunes the generic TACoGFN for a given test pocket.
Our experiments on the CrossDocked2020 benchmark have demonstrated that TACOGFN and TACoGFN+FT outperform the stato-of-tho-art methods in terms of Vina Dock, high affinity and succoss rate. This demonstrates the potential of TACoGFN as powerful tool for structure-based drug discovery. In future work, we plan to validate the top generated ligands for some clinically relevant protein pockets in-vitro, i.e in wet-lab experiments.
Broader Impact Statement
This paper presents work whose goal is advance machine learning methods for drug discovery. Such methods increasingly being employed in the pharmaceutical industry since they promise to greatly speed-up the lengthy process of drug discovery and to significantly reduce its huge cost. If that promise holds, these machine-learning methods will benefit patients through better care and our society through a reduction the economic burden of drug development.
References
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):2214-2216 2015.
Emnanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation, 2021.
G. Richard Bickerton, Gaia V. Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L. Hopkins. Quantifying the chemical beauty of drugs. Nature Chemistry, 4(2):90 98, January 2012. ISSN 1755-4349. doi: 10. 1038 1198/nchem.111 1243. URL http:///xx doi. oooi.org//0... 1ttp//...138//annn 1038//cceee.... 1243
Thomas Blaschke, Josep Arús-Pous, Hongming Chen, Christian Margreitter, Christian Tyrchan, Ola Engkvist, Kostas Papadopoulos, and Atanas Patronov. REINVENT 2.0: An AI tool for de novo drug design. Journal of Chemical Information and Modeling, 60(12):5918 5922, October 2020. doi: 0c00915. URL https:://oi..rrg wttp///..///.../..s..oooooo
wet.:1011acc.ccooo htto///ccc.c......00000... 0c00915
Michael Brocidiacono, Paul Francoeur, Rishal Aggarwal, Konstantin Popov, David Koes, and Alexander Tropsha. Bigbind: Learning from nonstructural data for structure-based virtual screening. November 2022. doi: 0..26434/chemmxxx-222222
URL http:hh:p//d.....orr////////
doi. http:///ww.......oooo//////////e
3qc9t
14

---

Published in Transactions on Machine Learning Research (09/2024)
Lucian Chan, Marcel Verdonk, and Carl Poelking. Embracing assay heterogencity with neural processes for markedly improved bioactivity predictions. arXiv preprinl or Xin:2300 09086, 2023.
Connor W. Coley. Defining and exploring chemical spaces. Trends in Chemistry, 3(2):133 145, 2021. ISSN 2589-5974. doi: http::/ddb..rgg/11.00t////tttr http: URL https://www. https://ww..cieencddreee sciencedirect. com/ ooceac///arrccc////////555555555 Special Issuc: Machine Learning for Molecules and Materi- als.
Jörg Degen, Christof Wegscheid- Gerlach, Andrea Zaliani, and Matthias Rarey. On the art of compiling and using "drug-like" chemical fragment. spaces. ChemMMedCeem, 3(10):1503- 1507, October 2008. ISSN 1860-7187. doi: 10.1002/cmd::220000017 URL
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score drug-like molecules based on molccular complexity and fragment contributions. Journal of Cheminformaticc, 1(1), June 2009. ISSN 1-8. URL http://dx doi. org http:///tt.dal..rr//11111/////// 1186/117 -9111888-99------
1758-2946. doi: 10.1186
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Paul G. Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B. Iovanisci, Ian Snyder, and David R. Koes. Three-dimensional convolutional neural networks and a cross-docked data set for structure- based drug design. Journal of Chemical Information and Modeling, 60(9):4200 4215, August 2020. ISSN 1549-960X. doi: 10.1021/acs, baossjcci..c00441. URL //dx: dwi. org/10. 1021/acs. jcim 0c00411
Tianfan Fu, Wenhao Gao, Connor W. Coley, and Jimeng Sun. Reinforced genetic algorithm for structure- based drug design. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrerier: http://ppemrrriieeeeeefffooooooo net/forum?it [X6UPWOr9LF]
Francesco Gentile, Vibudh Agrawal, Michael Hsing, Anh-Tien Ton, Fuqiang Ban, Ulf Norinder, Martin E Gleave, and Artem Cherkasov. Deep docking: deep learning platform for augmentation of structure based drug discovery. ACS central science, 6(6):939-99) 149999999,4-4 2020.
Arup K Ghose, Vellarkad N Viswanadhan, and John Wendoloski. A knowledge-basse based approach in de- signing combinatorial medicinal chemistry libraries for drug discovery. 1. qualitative and quantitative characterization of known drug databases. Journal of combinatorial chemistry, 1(1):55 68, 1999.
Ross Girshick. Fast r-enn. In Proceedings of the IEFE international conference on computer vision, pp. 1440-1448, 2015.
Manan Goel, Shampa Raghunathan, Siddhartha Laghuvarapu, and U. Deva Priyakumar. Molegu- lar: Molecule generation using reinforcement learning with alternating rewards. July 2021a. doi: 2212643//chemxxi-2222
URL lttt:://t/// http//xxx.di..r//////// 26434/ cttp//heeeeeeeeeeeeeee chenrxxv-20211-199
Manan Goel, Shampa Raghunathan, Siddhartha Laghuvarapu, and U. Deva Priyakumar. MoleGuLAR:
Molecule generation using reinforcement learning with alternating rewards. Journal of Chemical In- formation and Modeling, (1())):5815-56 Decomber 2021b. doi: 10.1021 hatp//.c.c.mmmm........ URL
https://doi.org. http:/10.022//aa 1021/acs...... 1c013411
Shuo Gu, Matthew S. Smith, Ying Yang, John J. Irwin, and Brian K. Shoichet. Ligand strain energy in large library docking. Journal of Chemical Information and Modeling, 61(())4331-43)) September 2021. ISSN 1549-960X. doi: 10.1021/aae /acssjcim.m
mcc.06......006...... 1000368. URL http://dx orgg 1021/022.csccco jccmm//.cc....m......... jeim. 1c00368
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In ICLR, 2023a.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang. Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: Diffusion models with decomposed priors for structuro-base drug design. TCML, 2023b.
15

---

Published in Transactions on Machine Learning Research (09/2024)
Atia Hamidizadeh, Tony Shen, and Martin Ester. Semi-supervised junction tree variational autoencoder for molecular property prediction, 2023.
Charles Harris, Kieran Didi, Arian R. Jamasb, Chaitanya K. Joshi, Simon V. Mathis, Pietro Lio, and Tom Blundell. Benchmarking generated poses: How rational is structure-based drug design with generative models?, 2023.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. ar preprint arXic:1905. 12265, 2019.
Ruth Huey, Garrett M Morris, and Stefano Forli. Using autodock 4 and autodock vina with autodocktools: a tutorial. The Scripps Research Institule Molecular Graphics Laboratory, 10550(92037) 2012.
John J Irwin, Khanh G Tang, Jennifer Young, Chinzorig Dandarchulunn, Benjamin R Wong, Munkhzul Khurelbaatar, Yurii S Moroz, John Mayfield, and Roger A Sayle. Zinc20 a free ultralarge-scale chemical database for ligand discovery. Journal of chemical information and modeling, 60(12):6065-66)) 6073, 2020.
Moksh Jain, Sharath Chandra Raparthy, Alex Hernandez-Garcia, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets, 2023.
Woosung Jeon and Dongsup Kim. Autonomous molecule generation using reinforcement learning and docking to develop potential novel inhibitors. Scientific Reports, 10(1), December 2020, doi: 10.1038/ s41598-000--8553722 URL https://doo org/10. 10. 00006//4158-0000-78533-3
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Junction tree variational autoencoder for molecular graph generation. CoRR, abs/110020042 2018. URL http://arxiv.. org/ absp//ass////eee/ 04364
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael J. L. Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons, 2021.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunkkool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 536(7873))583--553)) -89, 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Maria Korshunova, Niles Huang, Stephen Capuzzi, Dmytro S. Radchenko, Olena Savych, Yuriy S. Mo- roz, Carrow 1. Wells, Timothy M. Willson, Alexander Tropsha, and Olexandr Isayev. Generative and reinforcement learning approaches for the automated de novo design of bioactive compounds. Com- municalions Chemistry, 5(1), October 2022. ISSN 2399-3669. doi: 10.108//22000-002-00733- URL
http://dx. doi. oooii.ogg/1... 1t10:/000000//22000-022------000
Greg Landrum et al. Rdkit: Open-source cheminformatics, 2006.
Elaine Lau, Nikhil Murali Vemgal, Doina Precup, and Emmanuel Bengio.
DGFN: Double genera- tive flow networks. In NeurIPS 2023 Generative AT and Biology (GenBio) Workshop, 2023. URL
https:://ppenreview wtt.:n/eooooomo/dddaaaaaaaaaa
Seul Lee, Jachyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of-iisttibuttin generation. In International Conference on Machine Learning, pp. 18872 18892. PMLR, 2023.
Shuya Li, Fangping Wan, Hantao Shu, Tao Jiang, Dan Zhao, and Jianyang Zeng Monn: a multi-objective neural network for predicting cunpound-proted interactions and affinities. Cell Systems, 10(4):308 322, 2020.
Christopher A. Lipinski, Franco Lombardo, Beryl W. Dominy, and Paul J. Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development scttings. Advanced Drug Delivery Reviews, 23(1):3-25, 1997. ISSN 0169-409X. doi: https://dol (1111000006//0016--000000//////1
URL htt///ssciinccdireetttt. www.s//wwcddeeee...eeeeeeeeeeeea htt.////ciaaacc/artc.c////////// $0169409X960042211 Tn Vitro Models for Selection of Development Candidates.
16

---

Published in Transactions on Machine Learning Research (09/2024)
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. In TCML, 2022.
Zhihai Liu, Yan Li, Han, Jie Li, Jie Liu, Zhixiong Zhao, Wei Nie, Yuchen Liu, and Renxiao Wang. Pdb- wide collection of binding data: current status of the pdbbind database. Bioinformatics, 31(3):405 412, October 2014. ISSN 1367-4803. doi: 10.1093/bioiie bio.norrmatics/////o/ioooo 2atnnformaaaiic/tb22666 btu626. URL http://ttx..i..o//1..o//
bioinformaticcs/buuss.
Siyu Long, Yi Zhou, Xinyu Dai, and Hao Zhou. Zero-shot 3d drug design by sketching and generating, 2022.
Ily Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference Learning Represenlations, 2018.
Wei Lu, Qifeng Wu, Jixian Zhang, Jiahua Rao, Chengtao Li, and Shuangjia Zheng. Tankbind: Trigonometry- aware neural networks for trug-protein binding structure prediction. Advances in Neural Information Processing Systems, 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3D generative model for structure-based drug design. In NewrIPS, 2021.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Im- proved credit assigument. in gllownets, 2023.
Tomohide Masuda, Matthew Ragoza, and David Ryan Koes. Generating 3d molecular structures conditional on a receptor binding site with deep generative models. arXiv preprint arXim:2000144442 2020.
Noel M O' O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geoffrey R Hutchi- son. Open babel: An open chemical toolbox. Journal of cheminformatics, 3(1):1 14, 2011.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo de-
sign through deep reinforcement learning. Journal of Cheminformatiis, 9(1), September 2017. doi: 11.1186/113321--17-02222
URL https://doi: org/ 1e/g///0..1
Yongping Pan, Niu Huang, Sam Cho, and Alexander D. MacKerell. Consideration of molecular weight during compound selection virtual target- database screening. Journal of Chemical Information and Computer Sciences, 43(1):267 272, November 2002, ISSN 1520-5142. doi: 10.1021/ci00005f.. URL http://dx..o doi www.o.oogg/.... /0210///00//
Mohit Pandey, Mariia Radaeva, Hazem Mslati, Olivia Garland, Michael Fernandez, Martin Ester, and Artem Cherkasov. Ligand binding prediction using protein structure graphs and residual graph attention networks. Molecules, 27(16), 2022. ISSN 1420-3049. doi: (0.33900/molecule2277
URL https:
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, gegh-performanc deep lcarning library. In Advonces in Newral Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molccular sampling based on 3d protein pockets. In ICML, pp.
PMLRR 2022.
Emanuele Perola and Paul S. Charifson. Conformational analysis of drug-like molecules bound to pro- teins: An extensive study of ligand reorganization upon binding. Journal of Medicinal Chemistry, 47(10): 2499-2510, April 2004. ISSN 1520-4804. doi: 10.1021/jm030563v. URL http: http:///xx.r doi http://tt.........ooo./oo//o jm030563w
Trang Pham, Truyen Tran, Hoa Dam, and Svetha Venkatesh. Graph classification via deep learning with virtual nodes, 2017.
17

---

Published in Transactions on Machine Learning Research (09/2024)
Alexander S. Powers, Helen H. Yu, Patricia Suriana, Rohan V. Koodli, Tianyu Lu, Joseph M. Paggi, and Ron O. Dror. Geometric deep learning for structure-based ligand design. ACS Central Science, November 2023. ISSN 2374-7951. doi:
3c00572
Sudeep Pushpakom, Francesco lorio, Patrick A. Eyers, K. Jane Escott, Shirley Hopper, Andrew Wells, Andrew Doig, Tim Guilliams, Joanna Latimer, Christine McNamee, Alan Norris, Philippe Sanseau, David Cavalla, and Munir Pirmohamed. Drug repurposing: progress, challenges and recommendations. Nature Reviews Drug Discovery, 18(1):41-5) 58, oct 2018. doi: 20.1088/nrd.2015 1038/nrd. 168
URL https://doi. org/10.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chem. Sci., 13:2701-2713, 2022. doi: (0.1089/DISCO597AA
URL http://dxddii..rrr11111101131330
Danny Reidenbach. EvoSBDD: Latent evolution for accurate and efficient structure- drug design. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024. URL https://openrevieew
net/forrum?dd=sshhUhh00u....
Joseph M. Replogle, Reuben A. Saunders, Angela N. Pogson, Jeffrey A. Hussmann, Alexander Lenail, Alina Guna, Lauren Mascibroda, Eric J. Wagner, Karen Adelman, Gila Lithwick-Yann Nika Iremadze, Florian Oberstrass, Doron Lipson, Jessica L. Bonnar, Marco Jost,, Thomas M. Norman, and Jonathan S. Weissman. Mapping information-rice genotype-phene penotype-phenotyee landscapes with genome-scale perturb-seq. Cell, 185(14); 2575.2275.c28, July 2022. ISSN 0092-8674. doi: 10.1016/j.coll coll..c2222222 2022.05.013 10. 10.1016/j.com cell.elll......
URL http::/dxx. doi. org/
Timothy J. Ritchie and Simon J.F. Macdonald. How drug-like are 'ugly' drugs: do drug-likeness metries predict adme behaviour humans? Drug Discovery Today, 19(4):489 495, 2014. ISSN 1359-6446. doi: https://doi.orgg/In 10.1////0.00001111111 aerricllepp1//135554466664000
URL hatpps:/wwe www.seedieeeect.....m... com/science/
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, 1lia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia. Structure-based drug design with equivariant diffusion models, 2023.
Seonghwan Seo and Woo Youn Kim. Pharmaconet: Accelerating large-scale virtual screening by deep pharmacophore modeling, 2023.
Seonghwan Seo, Jaechang Lim, and Woo Youn Kim. Molecular generative model via retrosynthetically prepared chemical building block assembly. Advanced Science, 10(8):2206674, 2023.
Chao Shen, Xujun Zhang, Yafeng Deng, Junbo Gao, Dong Wang, Lei Xu, Peichen Pan, Tingjun Hou, and Yu Kang. Boosting pigtein-ligand binding pose prediction and virtual screening based on residue-atom distance likelihood potential and graph transformer. Journal of Medicinal Chemistry, 65(1))110691 10706, 2022.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature Biotechnology, 35(11):1026-10)) 1028, October 2017. ISSN 1546-1696. doi: 10.1038 10.1038/nbt.3305 39888 URL http:://xx doi http://x...o..o//......o/. 10tg//bb...///tt..... 3988
Oleg Trott and Arthur J Olson. AutoDock Vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2): 461, 2010.
Izhar Wallach and Abraham Heifets. Most ligand-based classification benchmarks reward memorization rather than generalization. Journal of chemicul informalion and modeling, 58(5):916 (365)9916,93,, 2018.
Canille-Goorges Wermuth, CR Ganellin, Per Lindberg, and LA Mitscher. Glossary of terms used in medicinal chemistry (iupac recommendations 1998). Pure and applied Chemistry, 70(5):112 11641113,,,13,,
1143, 1998.
18

---

Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N. Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific Reports, 9(1), jul 2019. doi: 00.1088/s11588-001-471144444 URL 000/://0000000011111111000001110 Published in Transactions on Machine Learning Research (09/2024)
Sheng- Yong Yang. Pharmacophore modeling and applications in drug discovery: challenges and recent advances. Drug discovery today, 15(11-12):444-550 2010.
Yuejiang Yu, Chun Cai, Jiayue Wang, Zonghua Bo, Zhengdan Zhu, and Hang Zheng. Uni-dock: Gpu- accelerated docking enables ultralarge virtual screening. Journal of Chemical Theory and Computation. 39(11) 3345, April 2023. ISSN 1549-9626. doi: 10.1021 (a1002/acc.jctc.220 jcc.2111 URL http://dx.... http::/xxxxdoo..oo org/ 10. 10.1021/acs 2c01145
Seongjun Yun, Minbyul Jeong, Rachyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. Graph transformer networks, 2020.
Haiping Zhang, Konda Mani Saravanan, and John ZH Zhang. Deepbindgen: Integrating molecular vec- tor representation with graph convolutional neural networks for protein ligand interaction prediction. Molecules, 28(12):4691, 2023a.
Zaixi Zhang and Qi Liu. Learning subpocket prototypes for generalizable structure-based drug design. ICML, 2023.
Zaixi Zhang, Yaosen Min, Shuxin Zheng, and Qi Liu. Molecule generation for larget protein binding with structural motifs. In The Eleventh ICLR, 2023b.
Alex Zhavoronkov, Yan A. Ivanenkov, Alex Aliper, Mark S. Veselov, Vladimir A. Aladinskiy, Anas- tasiya V. Aladinskaya, Victor A. Terentiev, Daniil A. Polykovskiy, Maksim D. Kuznetsov, Arip Asadu- laev, Yury Volkov, Artem Zholus, Rim R. Shayakhmetov, Alexander Zhebrak, Lidiya I. Minaeva, Bog- dan A. Zagribelnyy, Lennart Н. Lee, Richard Soll, David Madge, Li Xing, Tao Guo, and Alán Aspuru- Guzik. Deep Icarning enables rapid identification of potent ddrl kinase inhibitors. Nature Biotech- nology, 37(9) 1040, September 2019. ISSN 1546-1696. doi: 10.0038/411557--09--02244 URL http://dx. doi 10t.////0.0000///551--0000000000
Xiangxin Zhou, Xiwei Cheng, Yuwei Yang, Yu Bao, Liang Wang, and Quanquan Gu. Decompopt: Control- lable and decomposed diffusion models for structure-based molecular optimization, 2024.
A Softwares
In this study, we used the open-sourced code for GFlowNet (Bengio et al. 2021 PharmacoNet (Seo & Kim 2023) and GVP-GNN (Jing et al. 2021). Our models were implemented using the Pytorch (aaskke ct al. 2019 and PyTorrh Geometric (Fey & Lenssen 2019 librarics, which enabled efficient training and evaluation. We utilized RDKit. (Landrum et al. 20066), widely-used chem-informatics library, to handle the molecular structures and compute chemical properties. We employed the QuickVina 2.1 (QVina) (Alhossary
ct al.
2015) and UniDock (Yu et al.] 2023) for docking, and used Openbabel (O'Boyle et al. 2011 and
AuckDock Tools (Huey et al. 2012 to generate ready-to-dock files.
B Problem definition compared to existing RL baselines
TAcoGFN addresses the problem of structure-based drug design (SBDD), which aims to have one model generate high-affinity molecules conditioned on any unseen protein structure. We train this one model to generate molecules conditional to different protein target structures in the CrossDocked training set, using predicted affinity, Synthetic Accessibility (SA), and drug-likeness (QED) as the reward. Then we evaluate the trained model's performance on the 100 unseen protein target structures in the CrossDocked test set. Molecules are generated for unseen protein structures during evaluation to measure whether SBDD models have learned generalized protein-ligand interaction patterns during training.
19

---

Published in Transactions on Machine Learning Research (09/2024)
On the other hand, many existing optimization-based RL methods [Bengio et al. 2021 Zhavoronkov et al, 2019 Korshunova et al. 2022 Jeon & Kim 2020 Goel et al. 2021a Lee et al. 2023 Reidenbach 2024) focus
on the target-free problem setting meaning the target protein structure is not used as input conditioning. RGA Fu et al. (2022). DecompOpt Zhou et al. (2024) and TACOGEN+FT use protein structure as input, however, they still require docking oracle calls to the test protein pocket during evaluation generale a molecule set. Therefore, they also fall within the optimization-basee problem setting. These models are trained optimized to generate molecules which optimize for predicted affinity on one protein target only and then evaluate the same target. The goal of this evaluation is to measure the effectiveness of the algorithm in terms of generating objects which match a reward distribution. Although both methods use the commonly used metrics such as affinity, Synthetic Accessibility (SA), and drug- drrug-ikenees (QED) as the reward, the goal of the evaluation and the problem difficulty is different compared to the SBDD setting.
Because protein-ligand interaction is highly specific, the distribution of molecules with high affinity will vary greatly across different possible pocket structures. The SBDD setting, which requires modelling the molecule distribution conditional to any protein pocket. and generalizing to unseen protein pockets, is therefore different and more challenging task than modelling molecule distribution for single protein pocket.
C Method Details
C.1 Additional details of pocket conditioned GFlowNet
C.1.1 Molecular generation actions
We follow action used in previous works on fragment- based molecular generation (Jin et al. 2018 Bengio et al., 2021 Hamidizadeh al., 2023) construct molecules fragment by fragment. We present the three types of actions available to TACOGFN below:
At each step, for each fragment node of in the molecular graph, apply the same MLP over its node embedding (,(()) which produces logits over the fragment vocabulary. Each logit represents the unnormalized score for attaching fragment node uf to a particular new fragment node (uf) from the vocabulary. These logits correspond to the PragmendAddditinn action type. While the Fragment.dd dition specifies whether fragment node uf connects to fragment node off it does not specify how they are connected (i.e. which atom on fragment of forms bond with which atom on fragment v/))
(1) TragmentAddition
add MLP h (NN
(2) AttachmoutSccification determines how the fragment pairs are connected. At cach step, for cach directional edge in the molecular graph which connects fragment of to of we produce a logit over the aloms of fragment of. Each logit represents the unnormalized score for fragment i connecting to fragment uf via single bond from particular atom fragment of, based edge embedding e, L(N). The molecule only be completed when all attachment edges are specified.
Attach = MLP (eL(N))
(3) StopConstruction a graph action that marks the finish a molecule. The logit is produced from a single MLP output based on the final graph embedding (((N), All logits are concatenated and scaled into probabilities using the softmax function, and an action sampled from the distribution. The same process is repeated for each time step until the stop construction action is sampled.
astop MLP g (g(())
20

---

Published in Transactions on Machine Learning Research (09/2024)
After all the scores for all possible actions computed, final action is sampled based the equations follows:
Pa hgr) softmar (Concat (d.dd, atttach, astep))
a a~ Pa (algf hgr)
C.1.2 Reward function.
Our reward function consists of properties highly relevant for drug candidate: Vina. Docking Score (DS); Drug Likeliness (QED) and Synthetic Accessibility (SA). Unless otherwise mentioned, tps -800, IQED 0.7 and Asa 0.8 for all models.
Drug Likeliness (QED) and Synthetic Accessibility (SA): We first normalize the raw SA score using the formula. 10-8A to obtain a reward between 0 and 1. While QED and SA need to meet a certain threshold to make good drug molecule, optimizing these values beyond the threshold does not. bring additional utility Coley 2021]. Therefore, we clip reward for QED or SA to 1 when they achieve their respective threshold L. In other words, our model will not be incentivised to optimize QED/SA beyond their threshold value; Therefore more priority will be placed on optimizing the Vina Docking Score when these thresholds are reached.
QED L@ED
TOED = min
SA tts
TSA = min
,1
Vina Docking Score (DS): Here, we define the docking score threshold tps to -8.0 kcal/mol corre- sponding to luM an important requirement for a drug candidate. Since molecules will not be as useful if they do not surpass this affinity requirement, we scaled down the component of docking score not surpassing threshold tps by 0.2. This has the effect of reducing rewards for molecules not surpassing this docking score threshold. Lastly, Pan al. (2002) notes screening based on docking score is biased toward the selection of high molecular weight, compound size may unfairly contribute to the energy score. We follow their suggestion of normalizing reward by the cube root of heavy atom count (HAC) to reduce the false positives resulting from the molecular weight bias. Lastly, multiply the whole term by -1 as our goal is to minimize the Vina Dock score.
(DS- (ps) 0.2 max (ps)
RDS
AHAC
We obtain the final reward by multiplying the normalized rewards together:
r rps TQED TSA
Model Details. We use Double GFN [Lau et al. 2023 to improve exploration in sparse reward domains and high-dimensional states. Our model is trained via the gradient descent method Adam (Kingma & Ba 2017 We list hyperparameters used in Table 7 and compare the training time of our method in Table 10
C.2
Additional details of docking score predictor
Motivation. When developing a docking score prediction model, two essential requirements are its speed of processing and its applicability to varicty of protcins and ligands. Since the binding poses are computa- tionally or experimentally expensive, previous affinity prediction models or docking score prediction models 7Zhang et al. 2023al predict energy by integrating the 1D representation vectors z" of the protein P and
21

---

Published in Transactions on Machine Learning Research (09/2024)
Table 7: Hyperparameters used for target conditional Gel
Hyperparameterss
Values
Num of training steps
30, 000
Learning rate
10-4
Weight decay
10-8
Momentum
0.9
Adam eps
10-8
Sampling т
0.99
Learning rate esstimatoor
10-3
Max nodes
9
Random action prob
0.01
Batch size
8
Training reward temp Inference reward temp Pocket cond dim
Uni 64)
64
128
Transformer hidden dim
256
Num of transformer layers QED threshold LQED SA threshold tsa
2
0.7
0.8
Table 8: Comparison of training time.
Model
Total Steps
Batch Size
Total Time (hrs)
Hardware
Mocket2Mol
475k
8
72.6
GTX A100 80GB
DifgetDiff
300k
4
25.0
GTX A100 80GB
DecompDiff
300k
4
41.7
GTX A100 80GB
TACoGFN
30k
8
17.7
RTX 3090 24GB
xf of ligand L, respectively:
Zp = Qp(P)
(6)
2, cc
(7)
E 62 (Concat(Zp. 2:)
(8)
However, this approach not consider the atom pairwise interactions between ligands and proteins due to global pooling, it shows less generalizability to unseen ligands or proteins. Compared to previous methods, MONN (Li et al. (0200) proposed pairwise interaction map between protein amino acid embeddings (h?"
and ligand node embeddings [hf]:
== (h o (hf}
(9)
E Sum (ool())
(10)
However, the use of full amino acid sequences is computationally expensive for large proteins. Furthermore, in target-basod drug design tasks that prioritize binding pocket information, affinity prediction over the entire protein can sometimes lead to incorrect energy calculations for different pockets. Therefore, our study adopts the use of pharmacophores within the binding pocket entire protein sequence. This approach not only effectively captures the unique features of the protein pocket, but also simplifies its topology, thereby addressing the computational burden and the generalization issues. Also, concept ually our docking score predictor determines whether a ligand alom corresponds to each pharmacophore node rather than whether it forms non-covalent interactions with each amino acid or atom.
alternative considering the
In the table, we find TACoGFN (C-alpha) outperforms BigBind and DeepBindGCN, indicating that our model architecture (see equation 5 of our paper) which integrates both pairwise interactions and global
22

---

Published in Transactions on Machine Learning Research (09/2024)
pooling information is more effective than the other architectures. Moreover, also observed that the pharmacophore representation significantly improved the performance over the residue graph representa- tion. Notably, TACOGFN (Pharmacophore) trained with Cross shows better performance on ZINCDock-test compared to the other models trained with ZINCDoc- Given that the main challenge of non-struchuree-based bioactivity predictors is poor generalization performance to ligands not used in the training set Chan et al. (2023), this result demonstrates that the pharmacophore representation the pocket greatly improves both accuracy and generalization compared to the residue graph representation.
Training details For model training, we use 15% of pocket the training set for the validation set. We used AdamW (Loshehilov & Hutter 2018) optimizer with betas of (0.9, 0.999) and a weight decay of 0.05, and learning rate 0.0001 with a decaying factor of 0.1 per 20,000 iterations. Each iteration has 32 randomly sampled pockets and 128 randomly selected ligands for each pocket during training on ZINCDock-15M For CrossDock- -00k,, we used up to 4 ligands per pocket. We optimize model with SmoothL (Girshick, 2015) for 100,000 iterations and select the best model weights with the lowest validation loss. The training process takes about a day on NVIDIA RTX A4000 GPUs.
Additional experiment details
D.1 Dataset
CrokeD (Francoeur et al.
2020 is a dataset containing 22.5 million poses of ligands docked into multiple
For CrossDocked-100k set, we use an identical processing strategy 2021 Peng et al., 2022): First, data points with binding pose RMSD greater than 1Â are filtered. Then, mmseq2 [Steinegger & Söding, 2017) is to cluster data at 30% sequence identity. After splitting, 100,000 protein-ligand pairs are randomly drawn for the training set. 100 test pockets are drawn from the remaining pocket clusters. 15,307 unique protein pockets remain in the training set.
similar binding pockets from P PDBBind
to previous works (Luo et al.
D.2 Docking protocols
For our protocol under the generative setting, we first convert all generated molecules into SMILES and calculate their ETKDG conformers (sEETKDGv3) using RDKit. Then, we prepare ready-to-dood dock files of ligands and proteins with Openbabel and AutoDockTools (O'Boyle et al.
2011
Huey et al. 2012). Finally, 2015). For
we dock these conformers using Quick VINA 2.1 (QVina) (Trott & Olson
2010
Alhossary
QVina, we box size of 20 Á and exhaustiveness of 8. Note for the optimization setting, we adopt
UniDock instead following previous works [Reidenbach 2024 (Please see details in Appendix
E.1
D.3 Baseline evaluations
For our baseline generation, we selected Pocket2Mol (Peng et al. 2022). Tirget (Guan et al. 2023a DecompDiff (Guan et al. 2023b). We adhered to the default hyperparameter settings for all models. De- compDiff incorporates three prior modes: subpocket, reference, and beta. Of these, we selected the beta mode for our analysis, as it demonstrated the highest docking score, aligning with findings reported in its original publication.
We generated 100 molecules for each of the 100 protein pockets from the Crossdocked2020 Francoeur al, 2020) test set. Following generation, we applied a filtering process to ensure that all molccules were unique and met validity criteria. Molecules were deemed invalid and discarded, if they had reconstruction errors, were duplicates, or were disconnected. The remaining SMILES obtained from these models were compiled establish the bascline. We then proceeded perform the docking protocol on each of the SMILES obtain their respective docking scores. Our results were in line with those reported in the original studies of the respective models.
23

---

Published in Transactions on Machine Learning Research (09/2024)
D.4
Additional Results
D.4.1 Additional ablation on effects of pocket conditioning
To further demonstrate the effectiveness of pocket conditioning, compare the numbers of non-covalent interactions achieved by the generated molecules from pocket-conditioned TacoGFN with TacoGFN without pocket conditioning. The non-covalent interactions between protein and ligand fall into the following cate- gories: (1) Hydrophobic Interactions are interactions between nonpolar regions of the protein and the ligand. (2) Van der Waals Contacts are weak forces arising from induced electrical interactions between closely positioned atoms of the protein and ligand. (3) Hydrogen Bonding refers strong dipole-dipole interactions between an electronegative atom and (typically) hydrogen atom. (4) Total Interactions
refers to the total number of interactions between the protein and the ligand, including all types of non-
covalent interactions. The generated molecules are docked using QVina (Trott & Olson 2010 Alhossary
ett al. 2015) to generate a binding conformation. We use PoseCheck (Harris el al. 2023 to compute the number of interactions.
In table D we found that pocket conditioning increased all types of non-covalent interactions between protein and generated molecules. this result suggests that the pocket structural information can provide guidance in selecting functional groups that can form better supramolecular interactions with the target pocket.
Table 9: Effectiveness of pocket structure conditioning measured using the average numbers of non-covalent interactions (NCI) achieved after docking the generated molecules into the target pocket using QV Qiiaa. Higher numbers of interactions indicate a stronger binding interaction.
Model
Docking datuset
Hydrophabic
Van der Warls contacts
Hydregen honding
Tosal interactions
TacoGFN
ZINCDock 15M
5.67
7.23
1.15
14.05
w/o pocket conditioning
ZINCDock-1aM
5.19
6.83
1.11
13.11
D.4.2 Discussion on diversity
In our experiments with GFlowNet, we find trado-off between diversity and the goal of optimizing for a higher average affinity of the generated candidate set. As TAcoGFN seeks to generate molecules with strong affinity to specific pocket structure while optimizing for QED and SA, the resulting solution set is inevitably smaller, lcading to reduced diversity.
We note that the molecular diversity of TACOGFN, if desired, can be increased in various ways. Firstly, the reward is currently exponentiated by reward temperture 3. As 3 increases, the reward density is concentrated close to the modes with high reward, and subsequently, there's a reduction in diversity (Jain al. 2023) Diversity can be improved by lowering the B.
Table 10: Comparison of average Vina Dock and diversity of TAcoGFN trained under different reward settings.
Model
TACOGEN Vina Dock (4)
Diversity (f)
-8.24
0.53
TACOGF Ntenient
-7.91
0.73
In addition, we have experimented with various reward function settings. In the reward scenario adopted in the paper the strict setting (see Appendix C.1.2) reduce the reward given to the component docking score that is below -8.0 kcal/mol. We have found that the stricter reward produces better affinity metrics at the expense of lower diversity. In the lemient setting, where we simply reward docking scores based on their normalized raw values, TAcoGFNiemsen still outperforms Pocket2Mol and Tifget in terms of average docking score (-7.91) while having higher diversity (0.73).
In summary, if the key concern is candidate diversity and novelty, TACOGFN can be trained generate generally high reward molecules under the lenient setting and/or low B. If the goal is to generate the best
24

---

Published in Transactions on Machine Learning Research (09/2024)
Average of Top-10 Vina Dock
700
Molect Weights (Da)
650
-0
550
-10
500
00000000000
400
-16
350
Earget
target
target
target
target
target
tanget
target
target
target
DecompDiff (lowest in 43%)
TacoGFN (lowest in 57%)
The average of the top-10 Vina Dock of molecules generated for individual CrossDocked test pockets (larget) by DecompDiff and TACOGFN. Color is used to denole the average molecular weight, of molecules in the Top-10 set. Molecular mass of an orally active drug should be less than 500 daltons] Lipinski et al. 1997]; Heavy molecules with high docking scores are more likely to be false positives (Pan et al. 2002
Figure 5:
Overall TACOGFN consistently achieves ideal molecular weight and strong Vina Dock.
compound set with high average affinity and top affinity with decent diversity, and satisfying QED and SA, TAcoGFN can be rewarded under the strict setting and/or high B.
D.4.3 Performance of docking score prediction model
While many existing dffinity-predictinn architectures either work for one target only (Bengio et al. 2021 Gentile et al. 2020), or require binding complex structures (Shen et al.] 2022), we cannot adopt these model architectures as we require affinity prediction across various protein targets without using their binding com- plex structure. Thus,
compare our pharmacophore-basel affinity prediction architecture with BigBind [Brocidiacono et al. (022)) and DeepBindGCN (Zhang et al. 2023a],, two recently proposed non-complex- based methods which achieved seate-of-the-art performance in virtual screening and affinity prediction, re- coarse-grained encoding
spectively. In addition, we examine the effect of introducing pharmacophore-basee of the protein pocket, by creating an ablation where we use residue graphs with C-alpha as nodes (as typically done other metho Lu et al. (2022) instead of pharmacophore graphs to represent our protein pocket. We call this ablation model TAcoGFN (C-alpha). We re-train these models using the same CrokeDocko02020 and ZINCDock- datasets and evaluate them both test sets.
Table 11: The evaluation of docking score prediction performance predictor trained on two different docking score datasets. Model
Training set
Cross sorckked-OOkktteee 100k-test
SineDock-15M-tte
RMSE
MAE
RMSE
MAE
CrossDockeed-000
1.252
0.991
1.516
1.175
BigBind
2INCDock-15M
1.561
1.227
1.233
0.955
CrossDockkc4100kk
1.739
1.366
1.912
1.531
DeepBindGCN
ZINCDock-15M
1.877
1.470
1.409
1.066
CrossDockedd-000:
1.091
0.821
1.517
1.157
TAcoGFN (C-alpha)
ZINCDock-15M
1.483
1.156
1.170
0.872
CrossDockocd1000k
0.881
0.652
1.121
0.791
TAcoGFN (Pharmacophore)
15
1.143
0.880
0.862
0.574
To evaluate the performance of our docking score predictor, we use two test sets: Cross test and ZiNCDock-test. ProssDoocedd-100k- is the test set of CrossDocked. 100k dataset used in
25

---

Published in Transactions on Machine Learning Research (09/2024)
Luo et al. 2021 and contains one ligand per each pocket. We re-docked each test ligand to perform the evaluation in the same environment as the generation process. ZINCDock-test is the docking simulation data of randomly selected 100 ZINC20 molecules for each pocket in Bross-ocked-100kkee The size of the test sets is 100 pocket-ligand pairs for the CrossDockod-100k-teee and 10,000 pocket-ligand pairs for the ZINCDock-test.
E
Finetuning of TacoGFN
Table 12: TACOGEN+FT ablations: Here, we vary the reward function parameters.
Validiy (1)
Vino
Tigh Avg.
(1)
QED (*)
(1)
Diversity Ang.
Sucoess (?)
Model
Avg.
Avg
Med.
Med.
Avp.
Med.
Ang
Med.
Avg.
Gen + Scorn.
Rederence
100%
-7.45
-7.26
0.48
0.47
0.77
0.74
20.0%
RGA
-5.01
-5.17
61.4%
89. 39.355
0.57
0.57
0.71
0.73
0.41
01.41
66.2%
Opt
-8.30
-8.15
T1.35
90.55
0.06
0.68
0.68
11.77
0.31
11.30
25.85
0:3725
Deconzppt
9.01
T3.35C
90.35
0.45
0.63
0.81
52.55
12241
BvoSDD
BvoSBDD 0,0 1,
100%
-10.16
-10.27
01:45
100%%
0.59
0.50
0.77
0.02
0.62
96.45L
(3000
ToaGGN--TT (xoxo (.80..sa 1,,0 241)
100%
-10.63
-30.78
97.
100%
0.48
0:47
0.72
0.71
0.62
11.02
86.67
(2200)
TocoGEN-1T (toso - -.500 0.80 240)
100%
10.24
10.01
97.05%
100%
11.58
0.58
0.82
11.82
0.61
0..11
57.716
0080
TocoGEN-IT 11:00-0.00 fea 0.75. 240)
100%
10.31
06.25C
100%
0.58
0.57
0.62
0.81
56.055
5210
TwaGFN-FT (tgoo
0.90, 240)
IooGFN-IT (tgeo fo00.faa 10,00.n 240)
300.1
4910
TwaGFN-FT (tazo 0.10, 0.00. 330)
100%
-10.78
-10.93
97.15
1001%
0.47
0:47
0.70
0.62
0.62
57.85L
7750
TeuGFN-FT (5geD -0.50, fex 0..00. 300)
100%
-10.32
-10.39
97.0%%
1005
0.58
0.58
0.82
0.52
0.01
0.61
88.8%6
0930
BmnGFN-FT (igso -0.00, (au (1.75
300)
100%
-10.11
-30.41
96.2%
100%
11.77
0.57
0.80
01.50
0.62
0.02
87.7%
6520
TomGFN-TT (tozo 0.55,
a00)
100%
300.35
97.0%%
100%
87.0%
0270
TocGFN-11T (toso 0.00.ss 0,80 300)
30.21
6240
E.1 Finetuning settings.
For finetuning TACOGFN, we adopt the same reward function as described in appendix section C112 However, instead of using the previously mentioned docking score predictor for affinity reward, we use the docking program directly. Following the same setting as EvoSEDD Reidenbach (2024), we adopt UniDock (Yu et al., 2023)- a GPU-accelerated docking program to compute docking score between the generated molecule and the protein target during the fine-tuning stage. We use UniDock's default balanced mode, which has exhaustiveness of 384 and max step of 40. We fine-tune using a batch size of 64 to leverage the speedup from UniDock's parallelization for computing docking scores.
For fair comparison with the previous methods, we re-scored the UniDock's docking pose with Vina (in a scorc-only mode without any structural changes) and obtained the same values within the margin of error (+ 0.005 kcll/mol). The top 100 molecules with the highest rewards from the fine-tuning stage are used for evaluation.
E.2 Results.
For ablations studies, we vary our reward function by adjusting IoED and tSA. Once the threshold for EQED or tSΛ is reached for molccules, the model will not be incentivised to optimize its QED or SA further. With lower threshold standard (tQED and tsa), our model will prioritize optimizing for Vina Dock instead of optimizing for QED SA. (See details in Appendix C.1.2). We observe such trade-off between QED and SA with Vina Dock in our ablation studies. We also experiment with various fine-tuning steps We show that Vina Dock metric will continue to improve with more fine-tuning steps.
26


Reviews:
[{"review_id": "f9u6bwd6LN", "reviewer": "Reviewer_3Yyf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\nAs the first application of GFNs to achieve both pocket conditioning and reward-based sampling for ligand generation, the method is interesting and overall is presented clearly.\n\nThe experiments are thorough, given the constraints of the proxy and surrogate functions available for evaluation with CrossDock. \nTable 6 shows evidence that pocket-conditioning does improve Vina score, and improvements are not simply coming from generation of more inherently “dockable” molecular graphs.\n\n\n\n\nWeaknesses\nOne point of confusion, the authors state “Since a molecule’s reward (representing its desirability as a real-world drug) should be the same regardless of its predicted 3D conformation, we represent ligands as 2D graphs here.” And then model the reward in terms of docking scores, which are completely sensitive to 3D conformation. Docking score is also only a surrogate for actual binding affinity, which depends on the accessible conformers of the ligand. What do the authors mean by saying that the reward should be invariant to 3D ligand conformation?\n\nI appreciate the direct visualization of molecules in Fig 3, but it makes the evaluation a bit unclear. Is the goal hit expansion (finding high scoring molecules in a lead series related to the ground truth reference), which the method of including the reward exponentiation and reward-based sampling would suggest? Or is it hit diversification, finding high-scoring, high diversity samples?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors introduce TacoGFN, the first application of GFlowNets to protein pocket conditioned ligand generation. The method follows the GFN framework of learning a policy to generate combinatorial objects with probability proportional to a reward function. The authors consider a composite objective of normalized proxy scores for druglikeness and synthetic accessibility, and a surrogate function trained on docking data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Related to Table 6, the claims related to pocket-conditioning are interesting and it would be informative to see more evidence that the distributions of samples are similarly conditionally dependent on the pocket.\n\nA brief description of Double GFN, the architecture, and training procedure for the policy, in the main text would be helpful.\n\nAs a general statement, the gap between modeling the reward distribution of docking score predictions and ranking “suitable drug candidates” is significant. Wherever possible, focusing on the technical contributions of developing GFNs for pocket-conditioned generation (which are clear) is appreciated.", "broader_impact_concerns": "Broader impact statement is sufficient.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yUZQnw0GIH", "reviewer": "Reviewer_VTSJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall speaking, I’m not very familiar with drug design, so I make my comments mostly from the general deep learning perspective.\n\nStrengths:\n\n1.\tThe paper is clearly written. The background of optimization-based and structure-based drug design methods is comprehensive. The motivation of using a flow-based network is clearly stated.\n2.\tMany design choices are justified by referring to previous established works or by providing explanations and ablation studies, e.g., the use of geometric vector perceptrons (GVP) in Section 4.1.\n3.\tThe performance of TacoGFN shows a good improvement over previous structure-based methods, especially in the success rate. At the same time, TacoGFN shows a significant reduction in generation time.\n\nWeaknesses:\n\n1.\tThe evaluation metrics partially overlap with the reward function. The reward function includes three metrics including QED, SA, and Vina Docking Score (DS). In this way, the generation of molecules is inherently biased towards higher QED, SA, and DS, since the probability of each action (e.g., attaching a new fragment) is proportional to the reward function. Therefore, it may be not surprising that the proposed method shows higher QED, SA, and DS scores in the evaluation, compared to other methods. I would like to see some further discussion or clarification from the authors regarding this issue.\n2.\tAbout the sampling of molecular generation actions in Section B.1.1. In the final step, the probability of actions is determined by $\\pi(a|G_t^L, h_{G^P}) = softmax(Concat(a_{Add}, a_{Attach}, a_{Stop}))$, where the logits $a_{Add}, a_{Attach},a_{Stop}$ are obtained from three different MLPs. My concern is that if the scale of the weights in one MLP (e.g., in the MLP for the “Add” action) is significantly larger than that of the other two MLPs at some timepoint during training, then it is likely that the policy $\\pi(a|G_t^L, h_{G^P})$ will always sample one type of action (e.g., “Add”). Such possibility of imbalanced weights may hamper further training.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel method named TacoGFN for structure-based drug design. TacoGFN leverages a conditional flow-based network to generate a drug-like molecule fragment by fragment, with probabilities proportional to the reward of each action. This method partially overcomes the limited dataset size for structure-based drug design, and the conditioning on pocket structure still enables the method to be applied to different pocket structures. Experiments show that the proposed method outperforms previous structure-based and (some) optimization-based methods while being more efficient.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tWhat is the intuition for multiplying instead of summing up the normalized QED, SA, and Vina Dock scores in the design of the reward function? I think this particular design choice needs more explanation for it to make sense.\n2.\tI would appreciate some discussion on why in Table 3, the QED score of TacoGFN+Finetuning is consistently lower than that of previous methods, while the Vina Dock and SA score both improves.\n3.\tMinor. There are two seemingly conflicting notation of $\\pi$. In the second paragraph of Section 4, $\\pi(L|P,\\beta)$ means the probability of constructing molecules L given protein structure P and the exponent $\\beta$. On the other hand, in Equation (2), $\\pi(a|G_t^L, h_{G^P})$ means a policy from which we sample different actions. The authors are encouraged to clarify whether these notations conflict with each other.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "WYK0IwSVXe", "reviewer": "Reviewer_dEZb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper provides a thorough case study on drug discovery with the novelty of incorporating pocket structures. Detailed explanations of how pocket structures are generated were provided in the paper. (I am not an expert on the molecules and hence cannot comment on the validity and novelty of the procedures.)\n\nThe paper lacks methodological contributions in GFlowNet preliminaries and can benefit from further clarification. Please see the detailed comments on the requested changes. \n\nOverall, the paper feels like reapplying the existing method to a common application. Although the authors named their proposed method TacoGFN, it closely resembles those proposed previously (see citations below). Additionally, GFlowNet and its variants had been extensively applied to molecule discovery previously as evidenced in\nJain et al. (2023) Multi-objective gflownets, ICML.\nBengio et al. (2021) Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation, NeurIPS 2021.\nBengio et al. (2023) GFlowNet Foundations, JMLR.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper builds on prior work on conditional GFlowNet and multi-objective GFlowNets and applies the framework to the application molecule generation in drug discovery. The authors propose to model the reward distribution conditioned on pocket structure to promote learning across protein targets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In the conclusion of the paper, the authors claim that \"we have framed pocket-conditioned molecule generation as learning a multi-objective reward distribution.\" However, according to the writing of Section 3, it is unclear why this is a multi-objective problem:\n- In Jain et al (2023), the variable w is a d-dimensional vector, however, the parameter c here in the paper seems to represent only one pocket structure: on Page 3 first paragraph under Section 3, the authors write \" each conditional information, denoted as c \\in C, induces a unique reward function R(x|c).\"\n- Similarly, for the problem to be a multi-objective problem, c in Equation (1) needs to be a high-dimensional binary vector.\n2. The authors often use the word \"policy\" when they really mean probabilities. Conventionally, a policy refers to a mapping from state to action, so the policy would correspond to the function rather than the value of the function at a point. E.g.,  on page 4 under Equation (1), the authors write \"our goal is to learn a policy \\pi(x|c,\\beta)...\"\n- additionally, the authors could change the phrase \"forward policy\" to \"forward probability function\" (in the first sentence of the paragraph containing Eq(1)) to avoid confusion between the conventional mapping that refers to from state to action.\n\nSection 4 seems to be authors' main contributions. However, it does seem that these methods have all been developed previously from other papers. I am willing to change my decision if the authors can convince me how the approaches taken in Section 4 are novel.\n\nHow do the results in Section 5 differ from those in Bengio et al. 2023? It does feel that most of the metrics utilized here were covered by Bengio et al. as well. A detailed contrast between the experimental setups could be beneficial. \n\nFor the paper to get published, the authors need to focus on the novelty of the paper and highlight the differences between the proposed experiments and those of the prior work.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper was reviewed by three reviewers. After rebuttal, all three reviewers were positive (2 accept and 1 leaning accept). 

Initial concerns included lack of clarity in presentation about reward modeling and evaluation, lack of background description on Gflownet in the main text, partial overlap between evaluation metric and reward function, imbalanced weights of the trained networks, and lack of novelty. These concerns were addressed successfully by the rebuttal. 

In summary, the paper makes a useful contribution to structure-based drug design.
Decision: Accept (poster)