RGFN: Synthesizable Molecular Generation Using GFlowNets
Authors: Michał Koziarski
Andrei Rekesh
Dmytro Shevchuk
Almer M. van der Sloot
Piotr Gaiński
Yoshua Bengio
Cheng-Hao Liu
Mike Tyers
Robert A. Batey
Venue: NeurIPS 2024 poster (2024)
Keywords: drug discovery
generative models
GFlowNets
synthesizability
Primary area: machine_learning_for_healthcare
TL;DR: Molecular generation with GFlowNets in the chemical reaction space, ensuring synthesizability out-of-the-box

Abstract: Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.

RGFN: Synthesizable Molecular Generation Using GFlowNets
Michal Koziarski Andrei Rekesh Dmytro Shevchuk Almer van der Sloot Piotr Gainski*.12 Yoshua Bengio'2 Cheng-Hao Liu Mike Tyers Robert A. Batey Mila Québec AI Institute, Université de Montréal, University of Toronto, Jagiellonian University, McGill University, The Hospital for Sick Children Research Institute, Acceleration Consortium, Equal contribution mi aichal. kozzaaski@mmi amila...mmiia.......m..........e queaec. (a. rekesh, dmytro. shevchuk) 0mail, utoronto.co
Abstract
Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction- Peaction-GFlowNer GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across range oracle models, including pretrained proxy models and GPU-accelerated docking.
Introduction
Traditionally, machine learning has been applied to drug discovery for screening existing libraries of compounds, whether actual physical collections or pre-configured in silico collections readily synthesizable compounds, in supervised fashion. However, supervised screening of the whole drug- like space, often estimated contain approximately 10 41 different compounds, infeasible in practice. Generative methods offer the potential to circumvent this issue by sampling directly from distribution over desirable chemical properties without the need to evaluate every possible molecular structure. Despite these advances, existing generative approaches tend not to explicitly enforce synthesizabilityy (21), generating samples that might be either very costly or altogether impossible to chemically synthesize. Ensuring that generative methods operate in the space of synthesizable compounds, yet at a much larger and more diverse scale than existing chemical libraries, remains an open challenge.
In this paper, we propose Seaction-GFlowNe (RGFN), an extension of the GFlowNet framework S that generates molecules by combining basic chemical fragments using a chain of reactions. We propose relatively small collection of cheap and accessible chemical building blocks (reactants), as well as established high-yield chemical transformations, that together still produce search space orders of magnitude larger than existing chemical libraries. We additionally propose several domain-specifer extensions of the GFlowNet framework for action representation and scaling to larger space of possible actions.
Source code available at http::/:/..tttbbocccc///oaaaaaaa
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

We experimentally evaluate RGFN on set of diverse screening tasks, including docking score approximation with a trained proxy model for soluble epoxide hydrolase (sEH), GPU-accelerated direct docking score calculations for multiple protein targets (Mpro, CIpP. TBLRI I and sFH), and biological activity estimation with a trained proxy model for dopamine receptor type 2 (DRD2) receptor activity and senolytic activity [74]. We demonstrate that RGFN produces similar optimization quality and diversity to existing fragment-best approaches while ensuring straightforward synthetic routes for predicted hit compounds.
2 Related work
Generative models for molecular discovery. plethora of methods have been developed for
molecular generation [88 7 using machine learning. These methods can be categorized depending on the molecular representation used, including textual representations such as SMILES [34 [9]]
molecular graphs [3
46 [54] or 3D atom coordinate representations [52] as well as the underlying
methodology, for example, variational autoencoders [33] 46), reinforcement learning [54]]]] 37 or
diffusion models (60] Recently, Generative Flow Networks (GFlowNets) 4 50 59
64
72
19
have emerged as a promising paradigm for molecular generation due to their ability to sample large and diverse candidate small molecule space, which is crucial in the drug discovery process. Traditionally, GFlowNets for molecular generation operated on the graph representation level, and candidate molecules were generated as sequence of actions in which either individual atoms small molecular fragments were combined to form a final molecule. While using graph representations, opposed to textual or 3D representations, allows the enforcement of the validity of the generated molecules, it does not guarantee straightforward route for chemical synthesis. Here, we expand on the GFlowNet framework by modifying the space of actions consist of choosing molecular fragments and executing compatible chemical reactions/rannfoomatioos, in turn guaranteeing both physical-chemical validity and synthesizability.
Synthesizability in generative models. One approach to ensuring the synthesizability of generated molecules is by using scoring function, either utilizing it as of the optimization criteria (37), or as processonn step for filtering generated molecules. Multiple scoring approaches, both heuristic 18 23 and ML-based [4), exist in the literature. Another branch of research focuses using reaction models and traversing predicted synthesis graph [8] 12
38 55 However, reaction and synthesizability estimation is difficult in practice, and can fail to generalize anttof-distribution in the case of ML models. Furthermore, theoretical synthesizability does not necessarily account for the cost of synthesis. Because of this, preferable approach might be to constrain the space of possible molecules to those easily synthesized by operating in a predefined space of chemical reactions and
fragments. Several recent strategies employ this approach [22 49 677, including reinforcement
yearning-based methods [24] [7] and a concurrent work utilizing GFlowNets [15). We extend this line
of investigation not only by translating the concept to the GFlowNet framework but also by proposing
a curated set of robust chemical reactions and fragments that ensure efficient synthesis at low total costs.
3 Method
3.1 Generative Flow Networks
GFlowNets are amortized variational inference algorithms that are trained sample from an unnor- malized target distribution over compositional objects. GFlowNets aim to sample objects from a set of terminal states X' proportionally pointed directed acyclic graph (DAG), G = A), where:
reward function R X R GFlowNets are defined
E S are the nodes, referred to as states in our setting, with the special starting state so being the only state with no incoming edges, and the terminal states X' have no outgoing edges, s' A are the edges, referred to actions in setting, and correspond to applying an action while in a state s and landing in state s'.
We can define non-negative flow function on the edges F(s - s') and on the states F(s) of the DAG such that Vo (FF(x) R(x)). perfectly trained GFlowNet should satisfy the following flow-mate matching constraint:

---

E S F(s)= =
Σ F(s" 75) +))= -8)A
Σ F(s + s'). (++8
(1)
A state sequence т = (su for all i, is called a complete trajectory. We denote the set of trajectories T.
z), with Sn = E d' and (s; + sit1) E A
Another way to rephrase the flow-matching constraints is to learn a forward policy Pr(84t1 (8:) such that trajectories starting at So and taking actions sampled by Pp terminate at x X proportional to the reward.
Trajectory balance. Several training losses have been explored to train GFlowNets. Among these, trajectory balance 45] has been shown to improve credit assignment. In addition to learning forward policy Pr, also learn backward policy Pg and scalar Ze, such that, for every trajectory ($u + $1 7 + Sn = 2), they satisfy:
Zo II
(p( = R(x) II PB (s((s-1(8s)
(2)
3.2 Sraction-GFlowNe
generates molecules by combining basic chemical fragments using a chain of reactions. The generation process comprises the following steps (illustrated Figure
1. Select initial building block (reactant or surrogate reactant; Appendix for more details about surtogate reactants).
2. Select the reaction template (a graph transformation describing the reaction).
3. Select another reactant.
4. Perform the in silico reaction and select one of the resulting molecules.
5. Repeat steps 2-4 until the stop action selected.
In the rest of this section, we describe the design of the Reaction -Feeooee in detail.
Reagert CrEaaa
Reaction
Product selection
Reaction selection
selection
selection
selection
or reward
selection
Not selected
Reward
cerating
Only one
SyAr
Only one
CHID-
Figure Illustration of RGFN sampling process. At the beginning, the RGFN selects an initial molecular building block. In the next two steps. reaction and proper reactant are chosen. Then the in silico reaction simulated with RDKit's 't's RunReactants functionality and one of the resulting molecules is selected. The process is repeated until the stop action is chosen. The obtained molecule then evaluated using the reward function.
Preliminaries. Reaction-GFlow GelooNet uses predefined set reaction patterns and molecules intro- duced in Section 3.3 We denote these as R and M respectively. As backbone for our forward policy Pr. we use a graph transformer model f from [77). The graph transformer takes as an input molecular graph m and outputs the embedding f (m) E R where D is the embedding dimension.

---

In particular, f can embed an empty graph 0. It can additionally be conditioned on the reaction r E R which we denote as (m, r). The reaction in this context is represented as its index in the reaction set R.
Select an initial building block. At the beginning of each trajectory, Deaction-GFlowNNe selects an initial fragment from the collection of building blocks M. The probability of choosing i-th fragment my is equal to:
pmm, (0) = o (s):, s; MLP (f)())s
where MLP M: RDD - RIMI is a multi-layer perceptron (MIP). The a² a standard softmax over the logits vector E TR= of the length k:
o (s): = exp(s:)
Σ exp(s;)
Select the reaction template. The next step is select a reaction that can be applied to the molecule m. The probability of choosing i-th reaction from R is described as:
p(r:|m) = a RR +1 (S); 8, MMMPMPA (m)))
where MLPR R R + [ is an MLP that outputs logits for reactions from R and an additional stop action with index [R] + Choosing the stop action in this phase ends the generation process. Note that not all the reactions may be applied to the molecule m. We appropriately filter such reactions and assume that the score 8; for non- non-feasible reactions is equal to -00
Select another reactant. We want to find molecule m, E M that will react with vn in the reaction The probability for selecting m, is defined as:
p(m.(mm)) r) Il (s); s. MLP (f),,)(
(5)
where MLPM M is shared with the initial fragment selection phase. As in the previous phase, not all the fragments can be used with the reaction r, so we filter these out.
Perform the reaction and select one of the resulting molecules. In this step, we apply the reaction to the two fragment molecules chosen in previous steps. As the reaction pattern can be matched to multiple parts of the molecules, the result of this operation is a set of possible outcomes M'. We choose the molecule m² M' by sampling from the following distribution:
p(m))= = a IM (s);, 8; = MLP Mr (f(m)))
(6)
where MLP My :RD + R scores the embedded m² molecule.
Backward Policy. A backward policy in RGFN only non-deterministic in states corresponding to molecule m which is a result of performing some reaction r E R on molecule m' and reactant m" E R. We denote the set of such tuples (r, m', mm") that may result in m as T. We override the indexing and let (r,, m4, mt²) be the i-th tuple from T. The probability of choosing the i-th tuple is:
(7) where MLP R. + R and f is backbone transformer model similar to the one used in the forward policy. To properly define T, we need implicitly keep track of the number of reactions performed to obtain m (denoted as k). Only those tuples (r, m', m²) are contained in the T for which we can recursively obtain m' in k reactions.
p((r,, ((r m (m) = (s) s; MLP B (f((m,
Action Embedding. While the MLP MM used to predict the probabilities of selecting a molecule mi E . works well for our predefined M, it underperforms when the size of possible chemical building block library is increased. Such an MLP likely struggles to reconstruct the relationship between the molecules. Intuitively, when a molecule my is chosen in some trajectory, the training signal from the loss function should also influence the probability of choosing structurally similar my. However, the MLP M disregards the structural similarity by construction and intertwines the probabilities of choosing m; and m, only with the softmax function. To incorporate the relationship between molecules into the model, we embed the molecular building blocks with simple machine learning model and reformulate the probability of choosing particular building block m,:
p(m m, r) (s)), st @(W r)) 9
(8)

---

where o is some activation function (we use GELU) and W E RR is a learnable linear layer. Note that if we define g(m,) as an index embedding function that simply returns distinct embedding for every mh, we will obtain a formulation equivalent to Equation (5) To leverage the structure
of molecules during the training. we use ! that linearly embeds MACSS fingerprint [40] of an
input molecule my along with the index i. Note that this approach does not add any additional
computational costs during the inference as the embeddings g(m;) can be cached. In Section 4.3 we
show that this method greatly improves the performance when scaling to larger sets of fragments.
3.3 Chemical language
We select seventeen reactions and 350 building blocks for our model. These include amide bond formation, nucleophilic aromatic substitution, Michael addition, isocyanate based urea synthesis, sulfur fluoride exchange (SuFEx), sulfonyl chloride substitution, alkyne-azide and nitrile-azide azide cycloadditions, esterification reactions, urea synthesis using carbonyl surrogates, Suzuki- Miyaura, Buchwald-Hartwin and Sonogashira cross-couplings, amide reduction, and peptide terminal thiourea cyclization reactions to produce iminohydantoins and tetrazoles. The chosen reactions are known to be typically quite robust [|] and are often high-yielding (75-100%). thus enforcing reliable synthesis pathways when sampling molecules from our model. To simulate couplings in Python, reactions are encoded as SMARTS templates. To ensure compatible building blocks yielding specific, chemically valid products and to enable parent state computation, we introduce multiple variants corresponding to differing reagent types for most of the proposed reactions. In some cases SMARTS templates encode reactions where one of the reagents is not specified. We describe these transformations as implicit reactions (Appendix EE. Additionally, we introduce surrogate reactions where the SMARTS templates and SMILES strings encode for alternate building blocks (see Appendix C.2 for more details). Finally, once again for the sake of specificity, reactions are duplicated by swapping the order of the building block reactants. In total, 132 different SMARTS templates are used.
During the construction of the curated building block database, only affordable reagents (i.e., building blocks) are considered. For the purposes of this study we define affordable reagents to be those priced less than or equal to $200 per gram. The mean cost per gram of reagents selected for this study is $22.52, the lowest cost $0.023 per gram, and the highest cost $190 per gram (see Appendix N for more details on cost estimation).
A crucial consideration when choosing the set of reactions and fragments used is the state space size (the number of possible molecules that can be generated using our framework). This is difficult to compute precisely since a different set of reactions or building blocks is valid for every state in given trajectory. We estimate this based on 1,000 random trajectories instead (details can be found in Appendix[A) (A). ). In addition to our 350 low-cost fragments, we also perform this analysis with 8,000 additional random Enamine building blocks. Comparison for different numbers of maximum reactions is presented in Figure 2 We demonstrate that even with curated low-cost reactants and limiting the number a maximum of four reactions, state space size is an order of magnitude greater than the number of molecules contained in Enamine REAL [17]. This size can increase significantly with the addition of more fragments and/or an increase in the maximum number of reactions. Additional discussions regarding scaling can be found in Section 4.3
4 Experimental study
In the conducted experiments we compare oracle scores and synthesizability scores of RGFN with several state-of-the-arr reference methods. Secondly, we examine the capabilities of RGFN to scale to larger fragment libraries, in particular when using the proposed action embedding mechanism. Finally, we perform an in in-depth examination of generated ligands across several biologically relevant targets.
4.1 Set-up
Throughout the course of the conducted computational experimental study, we aim to evaluate the performance of the proposed approach across several diverse biological oracles of interest, This includes proxy models (machine learning oracles, pretrained on the existing data and used for higher computational efficiency); first, the commonly used sEH proxy as described in 4 Second, graph
5

---

do not share code or curated neural network trained on the biological activity classification task of senolytic 74] recognition. Third, the Dopamine Receptor D2 (DRD2) oracle [53] from Therapeuties Data Commons 28 Proxy
model details are provided in Appendix B
Per the GFlowNet training algorithm, the reward is calculated for a batch of dozens to hundreds of molecules at each training step, rendering traditional computational docking score algo- rithms like AutoDock Vina 770 infeasible for very large training runs. As a result, previous applications of GFlowNets to biological design 1 64 employed fast pre- pre-trained proxy model trained on docking scores instead. These prox- ies, while lightweight, present potential issues should the GFlowNet generate molecules out- side their training data distributions and require receptor-specific datasets. To circumvent this, we use the GPU-accelerated Vina-GPU 2.1 [68] implementation of the QuickVina 2 [2] docking algorithm to calculate docking scores directly in
Enamine REA
(550)
10
(8350)
Estmated state saace $26
1000
10a
101
Number of reuctions used
the training loop of RGFN. This approach allows for drastically increased ficxibility in protein tar- get selection while eliminating proxy general- ization failure. We selected X-ray crystal struc- tures of human soluble epoxy hydrolase (sEH), ATP-dependent Clp protease proteolytic subunit (CIpP), SARS-CoV-2 main protease (Mpro), and transducin 3-like-related protein as targets for evaluating RGFN using a docking reward (detailed motivation for specific target selection is provided in Appendix
Figure Estimation of the state space size of RGFN as function of the maximum number of allowed reactions. RGFN (350) indicates a vari- ant using 350 hand-picked inexpensive building blocks, while RGFN (8350) also uses 8,000 ran- domly selected Enamine building blocks. Enamine REAL (6.5B compounds) is shown as a reference.
4.2 Comparison with existing methods
We begin experimental evaluation with a comparison to several state-of-the-att methods for molecular discovery. Specifically, we consider genetic algorithm operating on molecular graphs (GraphGA) 332 as implemented in [00, which has been demonstrated to be very strong baseline for molecular discovery (11, Monte Carlo tree search-based SyntheMol [67]. cascade variational autoencoder as implemented in (57). For FGFN, we additionally considered its variant that had a SAScore one of the reward terms (FGFN+SA). Training details can be found in Appendix B.2 It is worth noting that besides SyntheMol, which also operates in the space of chemical reactions and building blocks derived from the Enamine database, and cas VAE, which used the set of reaction trees obtained from the USPTO database (3), our remaining benchmarks do not explicitly enforce synthesizabilityy when generating molecules. Because of this, in this section, we will examine not only the quality of generated molecules in terms of optimized properties but also their synthesizability. We consider only two reaction based approaches, other existing methods employing this paradigm 27 24 reactions and building blocks, making reproduction difficult.
(cas VAE)
49 and fragment-based GFlowNet (FGFN)
We first examine the distributions of rewards found by each method across four different oracles used for training: sEH proxy, senolytic proxy, DRD2 proxy, and GPU-accelerated docking for CIpP. The results are presented in Figure 3 As can be seen, while RGFN underperforms in terms of average reward when compared to the method not enforcing synthesizability (GraphGA), outperforms SyntheMol's and cas VAE's reaction- based sampling. Interestingly, when compared to standard FGFN, RGFN either performs similarly (CIpP docking) or achieves higher average rewards. This is most striking in the case of the challenging senolytic discovery task, in which proxy is trained on a severely imbalanced dataset with less than 100 actives, resulting in a sparse reward function. We suspect that this, possibly combined with lack of compatibility between the FGFN fragments and known senolyties, led to the failure to discover any high-reward molecules. However, RGFN succeeds in the task and finds wide range of senolytic candidates. Finally, the gap in performance is

---

even larger between RGFN and FGFN+SA, indicating that introducing synthesizabilltty constraints reduces the ability of FGFN discover high-reward molecules.
Andeet AN FORTAA
1996 rostiay
(a) sEH (proxy)
(b) senolytics (proxy)
DRD2 (proxy)
(d) ClpP (docking)
Figure 3: Distributions of rewards across different tasks.
Secondly, we examine the number of discovered modes for each method, with a mode defined as molecule with computed reward above a threshold (sEH: 7, senolytics: 50, DRI>2 0.95, ClpP docking: 10), and Tanimoto similarity to every other mode < 0.5. We use Leader algorithm for mode computation. The number of discovered modes across tasks as a function of normalized iterations presented in Figure 4 Note that in the case of GraphGA, FGFN, FGFN+SA, and RGFN this simply translates to the number of oracle calls, but for SyntheMol and cas VAE, due large computational overhead, we impose maximum number of oracle calls such that training time was comparable to RGFN (see Appendix for details). Note that the case of cas VAE, this resulted in very small number of molecules being visited in the allotted time. As can be seen, despite slightly worse average rewards, FGFN still outperforms other methods terms of the number of discovered modes (with the exception of senolytic discovery task, where it fails to discover any high-reward molecules). This includes FGFN+SA, despite its generally worse performance than FGFN. This suggests that RGFN samples are less diverse, possibly due to the relatively small number of fragments and reactions used. However, RGFN still outperforms remaining methods across all tasks, suggesting that preserves some of the benefits of the foocuse GelowNet framework.
(a) sEH (proxy)
(b) senolytics (proxy)
(c) DRD2 (proxy)
(d) ClpP (docking)
Figure 4: Number of discovered modes as function of normalized iterations. Log scale used.
Finally, we evaluate the synthesizability of the generated compounds as a key output. We present average values of several synthesizability-rdll
metrics, computed over top-k modes generated for each method, in Table [ We include measures indicating average molecular weight and drug-likeness (QED) to gauge the size of generated compounds. Furthermore, for completeness, we also include SAScores 18], but note that they are only a rough approximation of ease of synthesis. For better estimate of synthesizabilityy we perform retrosynthesis using AiZynthFinder (23] and count the average number of molecules for which valid retrosynthesis pathway was found. However, is important to note that both SAScores and AiZynthFinderr scores are inherently noisy metrics. While we evaluate all methods using them for the sake of rigorousness, ultimately molecules generated by RGFN (as well SyntheMol and cas VAE) are guarantced to be highly likely synthesizable. Note that to reduce variance, we compute molecular weight, QED, and SAScores over the top-500 modes, but due to high computational cost, AiZynthFinder scores are computed only over top-100 modes. As can be seen, while there is some variance across tasks, RGFN performs similarly to SyntheMol and cas VAE terms of both synthesizability scores, and significantly outperforms GraphGA and FGFN, Crucially, including SAScore as reward does improve the performance of FGFN in terms of that metric, but does not drastically change the AiZynthFinder scores, demonstrating that insufficient to guarantee synthesizability. All RGFN modes were additionally inspected manually by an expert chemist and confirmed as synthesizable, which indicates that AiZynth scores are likely underestimated.
7

---

Table 1: Average values of synthesirability-rrlted metrics for top-k modes.
Task
Method
Mol. weight +
QED
SAScore +
AiZyyth f
sEH
GraphGA
528.6 + 42.3
0.21 + 0.06
3.87 = 0.24
0,04
SyntheMol
411.1 + 66.7
0.57 = 0.18
2.85 0,55
0.80
casVAE
421.6 103.4
2.41 + 0.47
0.82
FGFN
473.4 58.9
0.39
3.43 +
0.14
FGFN+SA
473.7 62.2
0.36
3.01 + 0.50
0.27
RGFN
495.2
0.29
3.09 0.39
0.56
Scno.
GraphGA
#85.7
0.09 + 0.05
2.92 = 0.26
0.05
SyntheMol
4 83.5
0.48
2.77 = 0.40
0.53
casVAE
431.5 # 100.9
0.19
2.82 0.46
0.65
FGFN
468.9 + 47.7
0.42
3.55 + 0.52
0.02
FGFN+SA
+ 54.5
0.32
2.83 + 0.44
0.13
RGFN
0.21
3.24 0.32
0.58
ClpP
GraphGA
21.0031.8
0.32
4.14 0.07
0.00
SyntheMol
458.2 6077
0.16
2.86 0.56
0.56
casVAE
423.0
0.17
2.44 0.41
0.84
FGFN
548.6
0.02
2.94 0.54
0.25
FGFN+SA
+
0.244
2.61 + 0.49
0.33
RGFN
526.2
0.23
2.83 0.22
0.65
DRD2
GraphGA
4
0.42 +4220.11
2.50 0.23
0.41
SyntheMol
0.72
2.78 = 0.43
0.66
casVAE
83.5
0.29
2.42
0.87
FGFN
38..5
0.13
2.58 + 0.54
0.76
FGEN+SA
081.1
0.14
2.37 + 0.37
0.78
RGFN
447.1 45.7
0.41
2.79 + 0.34
0.87
4.3 Scaling to larger sets of fragments
Next investigate the influence of fragment embedding scheme proposed in Section 3.2 In the standard implementation of the GFlowNet policy, actions are represented as independent embeddings in the MLP. These encode actions as indices, effectively disregarding their respective structures and all information contained therein. The model must thus select from a library of reagents without any knowledge their chemical makeup properties. While finding similarities between actions may be a relatively easy task for small action spaces, it becomes more difficult when the size of the action space increases. To scale RGFN larger size of the building block library, we proposed encode building block selection actions using molecular fingerprints, allowing the model to leverage their internal structures without any additional computational overhead during inference. In Figure [5] we observe that our fingerprint embedding scheme allows for drastically faster convergence compared to the standard independent action embedding, especially for large library sizes. The details on how the larger fragment libraries were created can be found in Appendix
4.4 Examination of the produced ligands
In the final stage of experiments we examine the capabilities of RGFN to produce high quality ligands across multiple diverse docking targets (see Appendix for more details). The aim is evaluate whether 1) the chemical language used is expressive enough to produce structurally diverse molecules for different targets. and 2) whether the generated ligands form realistic poses in the binding pockets. We first demonstrate the diversity of ligands across targets on a UMAP plot of extended-coonectivve fingerprints (Figure 6. Ligands assigned specific targets form very distinct clusters, showcasing their diversity. Interestingly, we also observe structural differences between sEH proxy and sEH docking, possibly indicating poor approximation of docking scores by the proxy model. Secondly, we examine the docking poses of the highest scoring generated ligands (Figure 77) As can be seen, generated molecules produce realistic docking poses, closely resembling the poses known ligands (Appendix K), despite being diverse in terms of structural similarity (Appendix M). We further conduct cost analysis and synthesis planning for top modes Appendices and Overall, this demonstrates the usefulness of the proposed RGFN approach in the docking- baseed screens.

---

Number Bscevared scafleiss with JEH (proyy +
discovered
(prosy > intepeede embestin
Iegegtn unbaling
200k
26 26 B6 TiE 20
580
26. H EC TE
(a) sEH (proxy) >
(b) sEH (proxy)
Figure 5: The number discovered Murcko scaffolds with sEH proxy value above (a) and 8 (b) as function of fragment library size. We compare standard independent embeddings of fragment selection actions (blue) with fingerprint-based embeddings (orange) that account for the fragments' chemical structure. The number of scaffolds is reported after 2k training iterations for random seeds (the solid line the median, while the shaded area spans from minimum to maximum values). We observe that our approach greatly outperforms independent embedding when scaling to larger action space.
(a) sEH
(b) ClpP
(c) Mpro
Figure 7: Top docked RGFN ligands after filtering steps (blue) overlaid with the PDB-derived ligand (purple) for each of sEH, CIpP, and Mpro.
5 Limitations
The current proof-of-principle implementation of RGFN uses only 17 reaction types and 350 building blocks. Although these limited inputs already generate a vast chemical space, this rep- resents only a small fraction of possible drug- like space, which in turn limits the quality and potency of the generated molecular structures. The scaling experiment demonstrates that the number of building blocks can readily be in- creased, and increasing the number and diver- sity of building blocks is straightforward way to enhance and survey the accessible chemical space.
Target
Cire
Mpro
TBLK1
Senclytic
sent sarcay))
SEH (docklng))
The current set of building blocks and reac- tions tends generate linear and flat-shaped molecules. Adding small set of cycliza- tion reactions (such as peptide macrocycliza- tion and ring-closing metathesis), along with more complex-shaped scaffold building blocks, as well as reactions that introduce sp' hybridized atoms and stereochemical complexity will there- fore allow for greater shape diversity and the
35
Figure 6: UMAP plot of chemical structures of top-500 modes generated for each target. RGFN generates sufficient chemical diversity to produce distinct clusters of compounds. See Appendix G for description of each target protein.

---

generation of more potent molecules [5T 20 It is also important to recognize that RGFN does not explicitly generate synthetic routes to the molecules, at least not in strict chemical sense, which in addition to a sequence of reactions transforming sets of reactants into products (which RGFN does provide), would also include choice of reaction conditions, external reagents, catalysts, protection group strategies, etc.
Another significant limitation in the quality of the generated molecular structures the reliance on molecular docking as a scoring oracle. Although molecular docking has been successfully used in large-scale virtual screening efforts 44 61 351, it has well-known shortcomings in its predictive power. First, docking scores correlate strongly with molecular weight (MW) [13] and do not account for drug-likenee requirements like optimal MW or ClogP. In this work, molecule size was constrained only by the number of reaction steps, encouraging RGFN to generate large molecules within the building block limit. This can be somewhat rectified by augmenting reward with a drug-likeness or ligand efficiency term. Second, its binding affinity predictions and rankings often correlate weakly with experimental values and are highly dependent on the nature of the target protein's 's binding site This is further illustrated by the fact that known ligands are not necessarily characterized by highest possible docking scores (Appendix D) This limitation impacts the learning of the chemical structure-activity relationship space, leading to the generation of sub-optimal molecules. One solution to this limitation is to incorporate more accurate but computationally expensive methods (such as ensemble docking. MM-PBSA, and FEP) within multi- multi-fidelity framework (26). However, since we focus on robust, affordable, and facile synthesis methods, we ultimately aim to extend our approach beyond computational scoring methods by directly conducting experimental evaluation of synthesized compound batches within an active learning loop.
73 71.
6 Conclusions
In this paper, we present RGFN, an extension of the GFlowNet framework that operates in the action space of chemical reactions. We propose a curated set of high-yield chemical reactions and low-cost molecular building blocks that can be used with the method. We demonstrate that even with a small set of reactions and building blocks, the proposed approach produces a state space with a size orders of magnitude larger than typical experimental screening libraries while ensuring high synthesizability of the generated compounds. We also show that the size of the search space can be further increased by including additional building blocks and that the proposed action embedding mechanism improves scalability to very large building block spaces.
In the course of our experiments, we show that RGFN achieves roughly comparable average rewards to statc-of-the-art methods, and it outperforms another approach operating directly in the space of chemical reactions and, crucially, standard fragment-basee significantly improves the synthesizability of generated compounds when compared to fragment- based GelowNet. Analysis of ligands produced across the set of diverse tasks demonstrates sufficient diversity of proposed chemical space to generalize to various targets. While not yet demonstrated experimentally, ease of synthesis (due to the small stock of cheap fragments and high-yield chemical reactions used) combined with reasonably high optimization quality of bespoke ligands offer particular, it can be
GFlowNets. At the same time,
promising alternative to standard thhoughuuu screening applications. beneficial for active learning-bac -based pipelines with significant wet lab component, reducing the reliance on inaccurate docking oracles. Facilitating the drug discovery process through the generation of novel small molecules can eventually lead to the discovery of novel medications leading to significant societal benefits.
Acknowledgments and Disclosure of Funding
This work was supported by funding from CQDM Fonds d' Accélération des Collaborations en Santé (FACS) / Acuité Québec and the National Research Council (NRC) Canada, the Cana- dian Institutes for Health Research (CIHR), grant no. FDN-167277, Samsung and Microsoft. D. Shevehuk is grateful for support from the Mitacs Globalink Graduate Fellowship, grant no. FR121 120 The research of P. Gaiński was supported by the National Science Centre (Poland), grant no. 2022/45/B/ST6/01 117. Additional thanks for funding provided to the University of Toronto's Acceleration Consortium from the Canada First Research Excellence Fund, grant no. CFREF-2022-00042. Computational resources were provided by the Digital Research Alliance of
10

---

Canada (https://all) allanccccan.cccc ca) edge Poland's pereforerrrmacee Infrastructure PLGni (ACK Cyfronet Athena, HPC) for providing computer facilities and support within computational grant no PL.G/2023/015550
ca) and Mila (https://mil) quebec). We gratefully acknowl-
References
[1] Beatrice lexander-Howdeen Li Zhang, Almer M van der Sloot, Sylvain Tollis, Daniel J St-Cyr, Frank Sicheri, Adrian P Bird, Mike Tyers, and Matthew J Lyst. screen for MeCP2-TBLI interaction inhibitors using luminescence-based assay. Sci. Rep., 13(1):3868, March 2023.
[2] Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with Vina 2. Bioinformatics, 31(13):2214-2216 02 2015. ISSN 1367-4803. doi: 10. 1009//iiinfoomatiissrr882. URL
[3] Josep Arús-Pous, Atanas Patronov, Esben Jannik Bjerrum, Christian Tyrchan, Jean-Louis Reymond, Hongming Chen, and Ola Engkvist. SMILES-based deep generative scaffold decorator for de-novo drug design. Journal of cheminformatiic, 12:1-18, 2020.
[4] Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, -293399412 2021.
[5] Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward Hu, Mo Tiwari, and Emmanuel Bengio. GFlowNet foundations. Journal of Machine Learning Research, 24(210)11555, 2023.
[6] Vaibhav Bhandari, Keith Wong, Jin Lin Zhou, Mark F Mabanglo, Robert A Batey, and Walid A Houry. The role of CIpP protease in bacterial pathogenesis and human diseases. ACS Chem. Biol.,
June 2018
[7] Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Gener- ative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5) 2022.
[8] John Bradshaw, Brooks Paige. Matt Kusner, Marwin Segler. and José Miguel Hernández- Lobato. A model to search for synthesizable molecules. Advances in Neural Information Processing Systems, 32, 2019.
[9] Dean G. Brown and Jonas Boström. Analysis of past and present synthetic methodologies on medicinal chemistry: Where have all the new reactions gone? Miniperspective. Journal of Medicinal Chemistry, 59(10):4443-4458 2016.
[10] Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain Vaucher. GuacaMol: bench- marking models for de novo molecular design. Journal of chemical information and modeling, 1993):1096-1108 110,, 2019.
[11] Martin Buttenschoen, Garrett M. Morris, and Charlotte M. Deane. PoseBusters: Al-based docking methods fail to generate physically valid poses or generalise to novel sequences, 2023.
[12] Alexander Button, Daniel Merk, Jan A Hiss, and Gisbert Schneider. Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis. Nature machine intelligence, 1(7):307-315, 2019.
[13] Giorgio Carta, Andrew JS Knox, and David G Lloyd. Unbiasing scoring functions: new normalization and rescoring strategy. Journal of chemical information and modeling, 47(4): 1564,1571, 2007.
[14] COVID Moonshot Consortium, Hagit Achdout, Anthony Aimon, Dominic S Alonzi, Robert Arbon, Elad Bar- Haim Barr, Amir Ben-Shmuel, James Bennett, Vitaliy A Bilenko, et al. Open science discovery potent non-covalent SARS-CoV-2 main protease inhibitors. BioRxiv, pages 2020-10, 2020.

---

[15] Miruna Cretu, Charles Harris, Julien Roy. Emmanuel Bengio, and Pietro Lio. SynFlowNet: Towards molecule design with guaranteed synthesis pathways. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024.
[16] Thomas Duflot, Clothilde Roche, Fabien Lamoureux, Dominique Guerrot, and Jeremy Bellien. Design and discovery of soluble epoxide hydrolase inhibitors for the treatment of cardiovascular diseases. Expert Opin. Drug Discov., (3):229-2433
March 2014.
[17]
Enamine. Enamine REAL database. http://enane real - wtwl-cooooooddrreaadddedeeee-eee
net/ pete/copppnn-colleettiooo collections/ 2024. [Online; Accessed 22 May 2024].
[18] Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1:1-11, 2009.
[19] Piotr Gaiński, Michat Koziarski, Krzysztof Maziarz, Marwin Segler, Jacek Tabor, and Marek Šmieja. RetroGFN: Diverse and feasible retrosynthesis using GFlowNets. arXiv preprint WYXv:22406.18739, 2024.
[20] Warren R.I.D. Galloway, Albert Isidro-Llobet, and David R. Spring. Diversity-oriented synthesis as tool for the discovery of novel biologically active small molecules. Nature Communications, 1(1):80, Sep 2010. ISSN 2041-1723. doi: 10. 10.1008//comm// 1081. Utl. 10. 10011088/ncomms1003
[21] Wenhao Gao and Connor W Coley. The synthesizability molecules proposed by generative models. Journal of chemical information and modeling. 60(12):5714-577 5723, 2020.
[22] Wenhao Gao, Rocío Mercado, and Connor W Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. arXiv preprint wrXiv:2110.06389 2021.
[23] Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Esben Bjerrum. AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning. Journal of cheminformatics, 12(1):70, 2020.
[24] Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. International conference muchine learning, pages 3668-3679. PMLR, 2020.
[25] Paul R Graves, Lucas Aponte-Collazo, Emily M J Fennell, Adam Graves, Andrew E Hale, Nedyalka Dicheva, Laura E Herring, Thomas S K Gilbert, Michael P East, Ian M McDonald, Matthew R Lockett, Hani Ashamalla, Nathaniel J Moorman, Donald Karanewsky, Edwin J Iwanowiez, Ekhson Holmuhamedov, and Lee M Graves. Mitochondrial protease ClpP is target for the anticancer compounds ONC201 and related analogues. ACS Chem. Biol., 14(5): 1020-1029, May 2019.
[26] Alex Hernandez-Garriaa Nikita Saxena, Moksh Jain, Cheng-Hao Liu, and Yoshua Bengio. Multi-fidelity active learning with GelowNets. arXiv preprint arXiv:2366.117/5 2023.
[27] Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS omega, (551)329844322999 2020.
[28] Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learn- ing datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548, 2021.
[29] John J. Irwin, Da Duan, Hayarpi Torosyan, Allison K. Doak, Kristin Ziebart, Teague Sterling, Gurgen Tumanian, and Brian K. Shoichet. An aggregation advisor for ligand discovery. Journal of Medicinal Chemistry, 58 58(17)7076-7088 2015. doi: 10. 1021/acs. 10e///cc.mmeeeemmmm. jnedchem.5tbl 5051 URL
https://doi. org/10. 1021/aca jmedchem. 5b01105
PMID: 26295373.
12

---

[30]
Jo Ishizawa, Sarah F Zarabi. R Eric Davis, Ondrej Halgas, Takenobu Nii, Yulia Jitkova, Ran Zhao, Jonathan St-Germain, Lauren Heese, Grace Egan, Vivian R Ruvolo, Samir H Barghout, Yuki Nishida, Rose Hurren, Wencai Ma, Marcela Gronda, Todd Link, Keith Wong, Mark Mabanglo, Kensuke Kojima, Gautam Borthakur, Neil Macl.ean Man Chun John Ma, Andrew Leber, Mark D Minden, Walid Houry, Hagop Kantarjian, Martin Stogniew, Brian Raught, Emil F Pai, Aaron D Schimmer, and Michael Andreeff. Mitochondrial ClpP-mediater proteolysis induces selective cancer cell lethality. Cancer Cell, 35(5):211-737 .9, May 2019.
[31] Samuel Jacques, Almer M van der Sloot, Caroline C Huard, Jasmin Coulombe- Coulombe-Huntingtonn Huntington, Sarah Tsao, Sylvain Tollis, Thierry Bertomeu, Elizabeth J Culp, Daniel Pallant, Michael A Cook, Eric Bonneil, Pierre Thibault, Gerard D Wright, and Mike Tyers. Imipridone anticancer compounds ectopically activate the ClpP protease and represent a new scaffold for antibiotic development. Genetics, 214(4): 100111031111 1120, April 2020.
[32] Jan H Jensen. A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration chemical space. Chemical science, 10(12):356-33572, 2019.
[33] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference machine learning, pages 2323 2332. PMLR, 2018.
[34] Seokho Kang and Kyunghyun Cho. Conditional molecular design with deep generative models. Journal of chemical information and modeling, 59(1):43 -2, 2018.
[35] Anat Levit Kaplan, Danielle N. Confair, Kuglae Kim, Ximena Barros-Alvarez Ramona M. Rodriguiz, Ying Yang, Oh Sang Kweon, Tao Che, John D. McCorvy, David N. Kamber, James P. Phelan, Luan Carvalho Martins, Vladimir M. Pogorelov, Jeffrey F. DiBerto, Samuel T. Slocum, Xi-Ping Huang, Jain Manish Kumar, Michael J. Robertson, Ouliana Panova, Alpay B. Seven, Autumn Q. Wetsel, William C. Wetsel, John J. Irwin, Georgios Skiniotis, Brian K. Shoichet, Bryan L. Roth, and Jonathan A. Ellman. Bespoke library docking for 5-HT2A receptor agonists with antidepressant activity. Nature, 610(7932):582-599 591, Oct 2022. ISSN 1476-4687 doi: 10.
URL UtR.htts:::/ //tp:p:::/do..oo org
orw.ooogr/100 10. 1038/ s22554402----00000
[36] Daniel W Kneller, Gwyndalyn Phillips, Hugh M O'Neill, Robert Jedrzejezak, Lucy Stols, Paul Langan, Andrzej Joachimiak, Leighton Coates, and Andrey Kovalevsky. Structural plasticity of SARS-CoV-2 3CI mpro active site cavity revealed by room temperature x-ray crystallography. Nat. Commun., 11(1)3320, June 2020.
[37] Maksym Korablyov, Cheng-Hao Liu, Moksh Jain, Almer M van der Sloot, Eric Jolicoeur, Edward Ruediger, Andrei Cristian Nica, Emmanuel Bengio, Kostiantyn Lapchevskyi, Daniel St-Cyr, et al. Generative active learning for the search of small-molecule protein binders. arXiv preprint arXiv:2405.011 016666, 2024.
[38] Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric Xing. ChemBO: Bayesian optimization of small organic molecules with synthesizable recommendations. In International Conference on Artificial Intelligence and Statistics, pages 3393-3403, PMLR, 2020.
[39] Panagiotis- Chriistss Kotsias, Josep Arús-Pous Hongming Chen, Ola Engkvist, Christian Tyr- chan, and Esben Jannik Bjerrum. Direct steering of de novo molecular generation with descriptor conditional recurrent neural networks. Nature Machine Intelligence, 2(5))254 265, 2020.
[40] Hiroyuki Kuwahara and Xin Gao. Analysis of the effects of related fingerprints on molecular similarity using an eigenvalue entropy approach. Journal of Cheminformatics, 13:1-12, 2021.
[41] Christopher Lipinski, Franco Lombardo, Beryl Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced drug delivery reviews, 23(1-3):3 1997.
[42] Cheng-Hao Liu, Maksym Korablyov, Stanislaw Jastrzebski, Pawel Włodarezyk Sataarezykk-uuzysskii Yoshua Bengio, and Marwin Segler. RetroGNN: fast estimation of synthesizability for virtual screening and de novo design by learning from slow retrosynthesis software. Journal of Chemical Information and Modeling, 22103)222330 2022.
13

---

[43] Daniel Mark Lowe. Extraction of chemical structures and reactions from the literature. PhD thesis, 2012.
[44] Jiankun Lyu, Sheng Wang. Trent E. Balius, Isha Singh, Anat Levit. Yurii S. Moroz, Matthew J. O' O'Meara, Tao Che, Enkhjargal Algaa, Kateryna Tolmachova, Andrey A. Tolmachev, Brian K. Shoichet, Bryan L. Roth, and John J. Irwin. Ultra-large library docking for discovering new chemotypes. Nature, 566(7773) 229,, Feb 2019. ISSN 1476-4687. doi: 10.1038/ s41 $41586-019-0917.9, https://:// org/10, 10. 10w1///110/0-0-01 019--0999---999999--99-999999-99
[45] Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment in GFlowNets. Advances in Neural Information Processing Systems, 35:5955-5966 2022.
[46] Lukasz Maziarka, Agnieszka Pocha, Jan Kaczmarczyk, Krzysztof Rataj, Tomasz Danel, and Michal Warchot. Mol-CycleGAN: generative model for molecular optimization. Journal of Cheminformatics, 12(1):1-18, 2020.
[47] Kevin McCloskey, Eric A Sigel, Steven Kearnes, Ling Xue, Xia Tian, Dennis Moccia, Diana Gikunju, Sana Bazzaz, Betty Chan, Matthew A Clark, John W Cuozzo, Maric-Aude Guié, John Guilinger, Christelle Huguet, Christopher D Hupp. Anthony D Keefe, Christopher J Mulhern, Ying Zhang, and Patrick Riley. Machine learning on DNA-encoded libraries: A new paradigm for hit finding. J. Med. Chem., 63(16))8857 8866, August 2020.
[48] Joshua Meyers, Benedek Fabian, and Nathan Brown. De novo molecular design and generative models. Drug Discovery Today, 26(11):707--2115, 2021.
[49] Dai Hai Nguyen and Koji Tsuda. Generating reaction trees with cascaded variational autoen- coders. The Journal of Chemical Physics, 156(4), 2022.
[50] Andrei Cristian Nica, Moksh Jain, Emmanuel Bengio, Cheng- Liu, Maksym Korablyov, Michael M Bronstein, and Yoshua Bengio. Evaluating generalization in GFlowNets for molecule design. In ICLR2022 Machine Learning for Drug Discovery, 2022.
[51] Anthony Nicholls, Georgia B. McGaughey, Robert P. Sheridan, Andrew C. Good, Gregory Warren, Magali Mathieu, Steven W. Muchmore, Scott P. Brown, J. Andrew Grant, James A. Haigh, Neysa Nevins, Ajay N. Jain, and Brian Kelley. Molecular shape and medicinal chemistry: A perspective. Journal of Medicinal Chemistry, 53(10):382--388 2010. doi: 10.1021/ jm9008188 18008818.. URL URL htt////ii.oo//////////////// 1021
PMID: 20158188.
[52] Pedro O O Pinheiro, Joshua Rackers, Joseph Kleinhenz, Michael Maser, Omar Mahmood, Andrew Watkins, Stephen Ra, Vishnu Sresht, and Saeed Saremi. 3D molecule generation by denoising voxel grids. Advances in Neural Information Processing Systems, 36, 2024.
[53] Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de- novo design through deep reinforcement learning. Journal of cheminformatics, 241114, 2017.
[54] Aryan Pedawi, Pawel Gniewek, Chaoyi Chang, Brandon Anderson, and Henry van den Bedem. An efficient graph generative model for navigating ultra-large combinatorial synthesis libraries. Advances in Neural Information Processing Systems, 35:8731 15:8731-8745, 2022.
[55] Bo Qiang, Yiran Zhou, Yuheng Ding, Ningfeng Liu, Song Song. Liangren Zhang, Bo Huang, and Zhenming Liu. Bridging the gap between chemical reaction pretraining and conditional molecule generation with a unified model. Nature Machine Intelligence, 5(12):1476-1485 2023.
[56] A. K. Rappe, C. J. Casewit, K. Colwell, W. A. Goddard, and W. M. Skiff. Uff, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American Chemical Sociery, 1 1144255:100244-0033 1992. doi: 10.1021/ja0005 httpst htttt::tt:::tttt 10. /ttp:///...11//30001000000
[57] Recursion, Recursion GFlowNet repository. gflownet. 2024. [Online; Accessed 22 May 2024].
14

---

[58] Sereina Riniker and Gregory A. Landrum. Better informed distance geometry: Using what we know improve conformation generation. Journal of Chemical Information and Modeling, 55 (12):2562-28) 2574, 2015. doi: 10. (t00l//cc.ccimmb00000000
URI. ftttps://doo.orgg//// 1021/
acs. acs.jeim. jcs.jeimm5b00654 PMID: 26575315.
[59] Julien Roy, Pierre-Luc Bacon, Christopher Pal, and Emmanuel Bengio. Goal-conditioned GFlowNets for controllable multi-objective molecular design. arXiv preprint arXivv2336604466 2023.
[60] Nicholas T Runcie and Antonia SIS Mey. SILVR: Guided diffusion for molecule generation. Journal of Chemical Information and Modeling, 63(19):5996-6005, 2023.
[61] Arman A. Sadybekov, Anastasiia V. Sadyhekov, Yongfeng Liu, Christos llipoulos-Tssuutsuuvee Xi-Ping Huang, Julie Pickett, Blake Houser, Nilkanth Patel, Ngan K. Tran, Fei Tong, Nikolai Zvonok, Manish K. Jain, Olena Savych, Dmytro S. Radchenko, Spyros P. Nikas, Nicos A.
Petasis, Yurii S. Moroz, Bryan L. Roth, Alexandros Makriyannis, and Vsevolod Katritch. Synthon-based ligand discovery in virtual libraries of over 11 billion compounds. Nature, 601(7893):452-459, Jan 2022. ISSN 1476-4687, doi: 10.1038/41556-002--02222 https://doi.. org/oogg/// 11ogg/10.... 10. (1888/44186--00 022-002--------2--999
URL
[62] Gabriele Scalia, Steven T Rutherford, Ziqing Lu, Kerry R Buchholz, Nicholas Skelton, Kangway Chuang. Nathaniel Diamant, Ch--Chisttian Huetter, Jerome Luescher, Ahn Miu, al. A high- throughput phenotypic screen combined with an ultra-large-scale deep learning-based virtual screening reveals novel scaffolds of antibacterial compounds. bioRxiv, pages 2024-09, 2024.
[63] Ruth R Shah and Adrian P Bird. MeCP2 mutations: progress towards understanding and treating rett syndrome. Genome Med., 9(1):17, February 2017.
[64] Tony Shen, Mohit Pandey, and Martin Ester. TacoGFN: Target conditioned gflownet for structure-based drug design. arXiv preprint arXiv:3310003223, 2023.
[65] Vanessa Smer-Barreto, Andrea Quintanilla, Richard JR Elliott, John C Dawson, Jiugeng Sun, Víctor M Campa, Álvaro porente-Macias Asier Pneiti-Broceta, Neil O Carragher, Juan Carlos Acosta, et Discovery of senolytics using machine learning. Nature communications, 14(1): 3445, 2023.
[66] Teague Sterling and John J Irwin. ZINC 15-ligand discovery for everyone. Journal of chemical information and modeling, 15:2234 2337, 2015.
[67] Kyle Swanson, Gary Liu, Denise B Catacutan, Autumn Amold, James Zou, and Jonathan M Stokes. Generative AI for designing and validating easily synthesizable and structurally novel antibiotics. Nature Machine Intelligence, 6(3):338-353, 2024.
[68] Shidi Tang, Ji Ding, Xiangyu Zhu, Zheng Wang, Haitao Zhao, and Jiansheng Wu. Vina- GPU 2.1: towards further optimizing docking speed and precision of AutoDock Vina and its derivatives. bioRxiv, 2023. doi: 10. 101/2023 11 1...0..55... 04.565429. 11. .4.4..4... 2...044.55650 565429
httttp::/www.bborrtt
[69] Paul C. Trippier and Christopher McGuigan, Boronic acids in medicinal chemistry: anticancer, antibacterial and antiviral applications. Med. Chem. Commun., 1(3):183- 198, 2010.
[70] Oleg Trott and Arthur J Olson. AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Comput. Chem., 31 (2):455-461, January 2010.
[71] Tiziano Tuccinardi, Giulio Poli, Veronica Romboli, Antonio Giordano, and Adriano Martinelli. Extensive consensus docking evaluation for ligand pose prediction and virtual screening studies. Journal of Chemical Information and Modeling, 54(10))2980-299) 2986, 2014. doi: 10.1021/ ci500424n. URL httpps:////oii org/ 1021/ ci1004244
PMID: 25211541.
[72] Alexandra Volokhova, Michal Koziarski, Alex Hernández-Garehe Cheng-Hao Liu, Santiago Miret, Pablo Lemos, Luca Thiede, Zichao Yan, Alán Aspuru-Guzik, and Yoshua Bengio. Towards equilibrium molecular conformation generation with GFlowNets. Digital Discovery, 2024.
15

---

[73] Zhe Wang, Huiyong Sun, Xiaojun Yao, Dan Li, Lei Xu, Youyong Li, Sheng Tian, and Tingjun Hou. Comprehensive evaluation of ten docking programs on diverse set of -attind complexes: the prediction accuracy of sampling power and scoring power. Phys. Chem. Chem. Phys., 18: 129755 2016. doi: 10. 10.1039.6CPP11555. URL http://dx....m./// doi orgg
[74] Felix Wong, Satotaka Omori, Nina M Donghia, Erica Zheng, and James J Collins. Discovering small-molecule senolytics with deep neural networks. Nature Aging, 3(6):734-750, 2023.
[75] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprini arXiv:1810.008266 2018.
[76] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In International conference on machine learning, pages 5453 PMLR, 2018.
[77] Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph transformer networks. Advances in neural information processing systems, 32, 2019.
[78] Xin-Min Zhang, Qing Chang, Lin Zeng, Judy Gu. Stuart Brown, and Ross S Basch. TBLR1 regulates the expression of nuclear hormone receptor co-repressors. BMC Cell Biol., 7(1):11 August 2006.
16

---

A State space size estimation
We estimate the state space size by first sampling 1,000 random trajectories, masking out the end-of- sequence action unless the maximum trajectory length mar is reached. Then, for every i-th reaction or fragment in the trajectory. we count the average number of valid fragments frag, and reactions react, from given state in the trajectory, as well as the average number of unique trajectories trajs into which state can be decomposed using the backward policy. We estimate the state space size as
mar
( i=
max react;)
==0
(9)
trojmax
Experimentally derived average values of these parameters can be found Table Note that in the second setting we randomly picked 8,000 fragments from the Enamine stock (with the same balancing procedure as Section 4.3). which after merging with our own fragments, canonization and duplicate removal yielded total of 8,317 fragments.
Table 2: Experimentally derived average values of valid fragments, valid reactions, and possible trajectories.
350 fragments
8350 fragments
# reactions
frag.
react,
traj:
frag:
react;
traj:
0 350.0
1.0
8317.0
1.0
37.5
11.8
3.5
835.8
12.0
4.2
2
39.9
16.5
16.8
822.1
15.9
17.0
3
40.7
15.4
76.7
832.6
17.0
75.2
4
40.0
15.8
349.3
814.0
18.0
480.8
5
42.1
16.8
1825.8
857.6
18.9
3058.1
B Training details
B.1 Proxy models
The sEH proxy is described in 4.. is an MPNN trained on a normalized docking score data. We utilize the exact same model checkpoint as provided in [57].
The senolytic classification model is a graph neural network trained on the biological activity classification task of senolytic recognition [7] Specifically, it was trained on two combined, publicly
available senolytic datasets 74| 65). Reward given by the predicted probability of a compound being senolytic. It is worth noting that due to the low amount of data and high imbalance (< 100 active compounds, a high proportion which contained macrocycles and were infeasible to construct with fragment-basee based generative models), this is expected to be difficult task with sparse reward.
The senolytic proxy model GNEprop 62 which consisted of GIN layers [75] with hidden dimensionality of 500, utilized Jumping Knowledge shortcuts (6). and had a single output MLP layer. Pretraining was done in an unsupervised fashion on the ZINC15 dataset [66]. The training was done for 30 epochs using the Adam optimizer with learning rate of 5 x 10 and batch size of 50.
The DRD2 proxy is described in [53 28
It support vector machine classifier with Gaussian kernel using ECFP6 fingerprints as a feature representation.
B.2 Generative models
Both RGFN and FGFN were trained with trajectory balance loss 45 using Adam optimizer with a learning rate of x 10 logZ learning rate of x 10-1 and batch size of 100. The training lasted 4,000 steps. random action probability of 0.05 was used, and RGFN used a replay buffer of 20 samples per batch. Both methods use graph transformer policy with 5 layers, 4 heads, and 64 hidden dimensions. Exponentiated reward R(x)) = exp(B [ score(x)) was used, with 3 dependent on the task: 8 for sEH proxy, 0.5 for senolytic proxy, 48 for DRD2 proxy, and 4 for all docking runs. Note
17

---

that due to different ranges of score values, this resulted in a roughly comparable range of reward values.
For FGFN+SA, we used a modified reward function R(x) e * (0.5 (10 SAASSAAsoooo [AA_score)/10) with B adjusted per proxy match the original reward range.
prozy(a)) //aa//aaorrryyyyooa +
All sampling algorithms were outfitted with the Vina GPU-2.1 docking, senolytic proxy, sEH proxy, and DRD2 proxy scoring functions. While model architecture hyperparameters and batch sizes were kept consistent between FGFN and RGFN, we allowed FGFN a maximum fragment count of 6 as opposed to RGFN's 5 due to RGFN's larger average building block sizes.
GuacaMol's 's Graph GA model was trained with a population size of 100, offspring size of 200, and a mutation rate of 0.01 for 2000 generations for a total of 400,000 visited molecules.
For SyntheMol experiments, we used the default building block library of 132,479 compatible molecules and pre-computed docking, senolytic, and sEH proxy scores for all prior to executing rollouts to follow the established methodology. Due to CPU constraints, sampling 500,000 molecules with SyntheMol was impractical. Instead, we executed 100,000 rollouts over approximately 72 hours to match the RGFN training time with docking, yielding 111,964 unique molecules. Additionally, we performed 50,000 rollouts each (approximately 24 hours) for sEH, senolytic, and DRD2 proxies, resulting in 73,941, 69,652, and 62,320 unique molecules, respectively.
cas VAE was trained with Bayesian optimization (BO) using default parameters consisting of a hidden size of 200, latent size of 50, and message passing depth of 2. Again, due to time constraints imposed by BO latent space updates, we instead opted to approximately match RGFN training times, training for 72, 24, 24, and 24 hours the docking, sEH, senolytic, and DRD2 tasks respectively for a total of 135, 41, 38, and 40 rollouts and 7708, 2097, 1521, and 2165 total generated molecules, respectively.
C GPU-accelerated docking
Our docking oracle first accepts canonized SMILES strings as input. These are then converted to RDKit Molecules, protonated, and a low-energy conformer is generated and minimized with the ETKDG (88] conformer generation method and UFF force field (56), respectively. For computational efficiency, we generate one initial conformer per ligand. Each conformer is converted to a pdhqq file and docked against a target with Vina-GPU 2.1 using model defaults: exhaustiveness (denoted by "thread" in the implementation) of 8000 and a heuristically determined search depth d given by
d = max (1, 00.36 x Natom + 0.44 x Nrot 5.11J),
(10)
where Natom and Nrot are the number of atoms and the number of rotatable bonds, respectively. in the generated molecule. Box sizes were determined individually to encompass each target binding site and centroids were calculated be the average position of ligand atoms in the receptor template PDB structure file. A negative score is calculated and returned as a reward.
C.1 Target preprocessing
Each target was prepared by removing its complexed inhibitor and atoms of other solvent or so- lute molecules. We selectively prepared the ClpP 7UVU protein structure by retaining only two monomeric units to ensure the presence of a single active site available for ligand binding and similarly prepared the Mpro 6W63 protein structure by retaining only one monomeric unit.
C.2 Boron substitution for docking
Due to the lack of force field parameters for boron atoms, Quick is unable process ligands with boronic acid or ester groups. Therefore, we chose to substitute the boronic acid group and its derivatives in the building blocks/reactants with a carboxylic acid, where the carbonyl carbon is a C13 isotope. This was done for two reasons: firstly, carboxylic acids are considered to be bioisosteric to boronic acids[69], and secondly, the C3 tag would allow us to distinguish actual carboxylic acids from boronic acid surrogates for SMARTS encoding purposes. This allowed us to dock all possible final products while maintaining structural similarity to the original group and specificity to compatible reaction SMARTS.
18

---

D
Computational resources used
Evaluating fragment scaling took approximately 800 GPU hours GeForce RTX 4090 in total. Remaining experiments took roughly 24 GPU hours per run on Quadro RTX 8000 for sEH, DRD2 and senolytic proxies, and roughly 72 GPU hours per A100 for backing-baseed proxies.
E Implicit reactions
In this work, we define implicit reactions as SMART'S reactions, products of which contain atoms that are not included in the reactants and come from "implicit reagents". One example is urea synthesis using carbonyl surrogates: the SMARTS template uses two amines, but the product has more atoms than specified in the reactants. These come from the "implicit reagent", in our case any phosgene surrogate (e.g., CDI) (Figure 8
X
NH2
NH2
R1
R2
R
R₂
Implicit reagent
1eactant1
Reactant
Figure 8: An example of an implicit reaction: urea synthesis reaction encoded in SMARTS.
In addition to urea formation, other implicit reactions encoded in SMARTS include azide-alkyne cycloaddition, azide-nitrile cycloadditions, and tetrazole synthesis using peptide terminal thiourea cyclizations.
F Sampling large fragment set from Enamine building blocks
In order evaluate our approach scale, our building block set had be balanced prevent introducing learning biases via building block selection. This was achieved by grouping reagents into twelve fragment classes, for which a specific weight was assigned based on the frequency of appearance of these structures in different reactions. For example, carboxylic acid appearing in two distinct reactions was assigned weight coefficient of two, which would later be used calculate the normalized amount of building blocks per group for specific database size using the following formula:
Weight coef.
Amount of BBs Sum of weights
database size
(11)
The only exception is Michael acceptors, group 3, which has weight of 0.25 due the low availability of these reagents in most public molecular databases and possible overlap with structures that might fit the corresponding SMARTS code (for example, 2-cyano or 2-carbonyl aromatic compounds).
The final set of fragments constructed by randomly sampling fragments from publicly available Enamine building blocks in a way that roughly preserved the above grouping. This was done in greedy round-robin fashion, randomly selecting fragment at time from the remaining fragments belonging to a given group, and iterating through all groups until the specified number fragments selected.
G Target selection
Our targets (sEH, Mpro, ClpP, and TBLRI) were chosen with diverse functions and binding sites in mind as test cases for RGFN. ClpP is a highly conserved compartmentalized protease that degrades
19

---

substrates in dependent fashion and is a promising target in cancer and infectious disease [6]. A number of structurally unrelated small molecules that allosterically activate CIpP, thereby deregulating its protease activity, inhibit the growth of cancer cells and bacterial pathogens 31 25
30
The main protease of SARS-CoV-2, Mpro (also called 3CLpro), is required for processing of the virally-encoded polyprotein and thus for viral replication, and well-validated and intensively studied anti-SARS-Cov.) target [4]. The Mpro catalytic pocket contains 4 distinct sub-pockets that recognize amino acid motifs in substrate sites and is a challenging target due to its structural
plasticity [36].
Soluble epoxide hydrolase (sEH) catalyzes key step in the biosynthesis of cicosanoid inflammation mediators and is being pursued as potential target cardiovascular and other diseases [16] sEH has been used benchmark substrate for machine learning-basse drug discovery and particularly
amenable to computational methods because of its deep hydrophobic pocket [47].
Finally, TBLI and its paralog TBLRI are components of the NCoR transcriptional repressor complex and contain WD40 domains that participate protein interactions (78). TBLI/TBLRI mediates the transcriptional repression function of the MeCP2 methylCpG-bindin in the neurodevelopmental disorder Rett syndrome. Loss of function mutations in Rett syndrome frequently disrupt the interaction of a disordered region in McCP2 that binds the WD40 domain of TBLI/TBLRI; conversely, MeCP2 gain-of-funed function by copy number mediated overexpression leads to X-linked intellectual disability (63).
protein, which is mutated
Well-validated experimental assays are available for each of these different targets
H Ligand post-processing
To ensure the diversity, specificity, and conformer validity of top-generated molecules for each target, we initially categorized our molecules into distinct modes, each representing any SMILES string with Tanimoto similarity of 0.5 lower with all other modes. Subsequently, selected the top 100 modes based on their Vina-GPU 2.1 scores and filtered their docked poses using PoseBusters 11 in "mol" mode, where any pose failing any PoseBusters check excluded from consideration. As final precaution, we selected only modes with Tanimoto coefficients to known aggregators of 0.4 lower using UCSF's Aggregation Advisor (29 dataset. This process resulted in 35, 68, 31, and 15 top modes for sEH, CIpP, Mpro, and TBLR1 binders, respectively Appendix J Comparative analyses of docked top RGFN modes and confirmed sEH, CipP, and Mpro ligand poses can be found in Appendices K to M
I Posebusters, Aggregation Advisor analysis of top 100 modes
Table 3: Proportion of top- 100-100g generated sEH, ClpP, Mpro, and TBLRI modes satisfying each of PoseBuster's s Mol-mode checks, as well as the Aggregation Advisor Tanimoto similarity threshold of 0.4. Molecules were assessed in PoseBuster according to their ability to be loaded and sanitized in RDKit, reasonableness of bond lengths and angles, lack of steric clashes in the pose, aromatic ring and double bond flatness, and internal energy.
Condition
Torget
Load
Sunttize
Connected
Bend Lengits
Bond Angles
Steric Cladies
Aromute Ring Hattess
Double Bond Himness
Intemal Energy
App. Sim. <004
1.0
1.0
1.0
1.0)
1.0
1.0
0.9
0.3%
Ctp?
1.0
1,0
1.0
1.01
1.0
[195
0.94
0.72
Mpro
1.0
1,0
1.0
1.01
1.0
0189
0.62
TELRI
1.0
10
20

---

Top filtered molecules for all targets
Senolytics
ororoo
O
11-16/12
Ke 1AW6
Ho 3404
K+ 16/00
He 720l
OuD
11440
R- 11
n+ MA
51506
91595
$1/00
9120
1000
Figure Top 25 senolytic compound modes generated by RGFN.
SEH
p ara gapdd
pricoo
24074
2434542
2400
34507
1- 24225
8
r
3477
a 5400
1430
8: 94300
a a
bb
Co
Pig
duo
14322
24333
Figure 10: Top filtered binders sEH drawn from top 100 RGFN modes.
21

---

ClpP
R 23346
Rx 12320
R- 22188
Re 13204
Re 13804
1284
1202
1-72.66
Rx 12399
12
Re 12797
Ne 22744
Figure 11: Top 25 filtered binders to CIpP drawn from top 100 RGFN modes.
Mpro
E+ 12233
x- 21377
R+ 12139
Re 1209
1+ 2104
11003
1067
.006
16400
CO.
1000
8-32354
18553
Ri 10A06
a+ :0327
6 12324
1+ 30316
Figure 12: Top 25 filtered binders Mpro drawn from top 100 RGFN modes.
22

---

TBLR1
8
000
C5
35000
Rx 12.955
2250
12
12X2
80O 00
00
12915
R292
22006
22879
12857
Figure 13: All 15 filtered binders to TBLR1 drawn from top 100 RGFN modes.
23

---

K
Docked poses of top generated molecules
Ours (sEH, 4JNC)
Vina-GPU 2.1 Score: -1.997
Vina-GPU 2.1 Score: -14.94
Vina-GPU 2.1 Score: -4.99
Reference (sEH, 4JNC)
Vina-GPU 2.1 Score: -1..3
Vina-GPP 2.1 Score: -14.97
Figure 14: Top left to right: Top 3 generated ligand scaffolds for sEH (blue). Bottom left: Reference ligand pose (purple, PDB ID: 1LF). Bottom right: Reference ligand (purple) overlaid with top-scoring ligand (blue).
24

---

Ours (CIpP, 7UVU)
Vina-GPU 2.1 Score: -13.35
Vina-GPU 2.1 Score: -13.32
Vina-GPU 2.1 Score: -13.19
Reference (CIpP, 7UVU)
Vina-GPU 2.1 Score: -10.31
Vina-GPU 2.1 Score: -13.35
Figure 15: Top left to right: Top generated ligand scaffolds for ClpP (blue). Bottom left: Reference ligand pose (purple, PDB ID: OY9). Bottom right: Reference ligand (purple) overlaid with top-scoring ligand (blue).
25

---

Ours (Mpro, 6W63)
Rep
Vina-GPU 2.1 Score: -11.22
Vina-GPU 2.1 Score: -11.18
Vina-GPP 2.1 Score: -11.14
Reference (Mpro, 6W63)
Vina-GPU 2.1 Score: -8.53
Vina-GPU 2.1 Score: -11.22
Figure 16: Top left to right: Top generated ligand scaffolds for Mpro (blue). Bottom left: Reference ligand pose (purple, PDB ID: X77). Bottom right: Reference ligand (purple) overlaid with top-scoring ligand (blue).
26

---

Ours (TBLRI, SNAF)
Vina-GPU 2.1 Score: -13.40
Vina-GPU 2.1 Score: -13.27
Vina-GPU 2.1 Score: -13.26
Figure 17: Left to right: Top 3 generated ligand scaffolds for the TBLRI WD40 domain. TBLRI has no known small molecule ligands.
L Docking analysis of known ligands
Table 4: Vina-GPU 2.1 docking scores of 10 PDB-available ligands per target.
sEH
ClpP
Mpro
Ligand
PDB ID
Score
PDB ID
Score
PDB ID
Score
5IV
-11.79
OX0
-1.100
J70
-10.13
2
XDZ
-11.64
PJF
-10.67
KAE
-9.45
WJ5
-11.30
P4I
-10.40
7YY
-9.33
4
E3N
-11.14
P30
-10.39
7XB
-8.6
ILF
-1.133
OY9
-0.31
XYV
-8.6
6
8S9
-1.07
ZLL
10.188
X77
-8.54
TK9
-9.64
7SR
-10177
XFI
-8.40
8
G3W
-9.60
OSR
-9.49
OEN
-8.19
9
G3Q
-9.14
ONC
-9.47
J7R
-8.00
10
JOU
-8.75
9DF
-9.39
4NO
-7.33
We calculate and report the Vina-GPU 2.1 docking scores of 10 true ligands per target assessed the paper. Each PDB was prepared described in Appendix C.I
27

---

M Tanimoto similarity to known ligands
Average Maximum moni Similarity Distribution Target
SEH
Clpp
0.35
0.30
Moxmmum
0.2
0.20
0.15
SEH
Target
Figure 18: Average Maximum Tanimoto Similarity known ligands. For each target, we plot the highest Tanimoto similarity score across all generated molecules to the ten corresponding PDB- derived ligands.
28

---

Ligands cost analysis
To demonstrate the synthesizability and cost-effectiveness of the ligands produced by RGFN, we conducted comprehensive cost analysis and proposed synthesis plan for the top 10 scored ClpP hits generated by RGFN and SyntheMol (see Appendix O). We compared these two methods due to their similarity in chemical approach. For the cost calculation, building blocks, which were directly sourced from EnamineStore for the case of Synthemol ligands. All prices are represented in US dollars per 0.1 mmol of product (Figures I9 and 20). Notably, for very common reagents only available bulk, costs per gram were estimated by taking the cheapest smallest available alternative and dividing its cost by its mass. All prices for individual building blocks were sourced from their respective vendors, which included Millipore Sigma, Oakwood Chemicals, Combi-Blocks TCI, and AngeneSci.
only considered the price of
Position
Ligand
per mucol,
Position
Ligand
Cost 0,1 mumol,
2.07
1.72
1.90
3,93
3.
1.76
8
1.82
1:37
9
2.35
5
1
10
Figure 19: Synthesis cost for top-10 scoring ClpP ligands produced by RGFN.
Position
Ligand
par manol S
Position
Ligand
Cost muzol, S
185.79
112.81
2
17.68
7
253.95
3
260.33
8
78.62
4
233.44
9
35.44
5
NIA
10
185.02
Figure 20: Synthesis cost for top- 10 scoring CipP ligands produced by Synthemol.
29

---

As expected, ligands generated by RGFN are significantly cheaper to produce compared to SyntheMol, despite having more synthesis steps and lower overall theoretical yields, with an average of 55%70% after 4 steps in the case of RGFN compared to 90-95% average yield after step for SyntheMol. Noticeably, SyntheMol ligand 5 wasn't found to be synthesizable by using reactions proposed in their work. After conducting further analysis, it was found that such a compound could have been produced by Reaction via nucleophilic substitution of the fluorine atom in the trifluoromethyl group; however, this particular reaction is not likely to be performed with sufficient yields, and therefore such a product cannot be considered synthesizable.
Examples of plausible synthetic routes to molecules
Molecular Weight: 153.9
Molecular Weight: 176.0
Br
suggested yield:
B(HH)2
$10/g
$10/g
1.2 eq; 6.81 mmol (1.05
(5.68 mmol) $10
$10
5.11 mmml
Molecular Weight: 139.9
Molecular Weight: 172.0
NH2
Br
B(OH))
NHz
suggested yield: 90%
$9/g
$10/g
1.2 eq; mmol (1.43g) g)
8.5 mmol (1.465 g)
$13
$15
7.67 mmol
NH2
suggested yield: 75%
5.11 mmol
7.67 mmml
3.83 mmol
Molecular Weight: 153.1
NH2
suggested yield: 90%
$33/g
HN
3.83 mmol
1.2 eq.; 4.6 mmcl (0.704
$23
3.45 mmol; $71
Figure 21: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand
30

---

Molecular Weight: 153.9
Moleculaa Weight: 178,0
Br
suggested yleld:
B(OH)2
$10/g
$10/g
1.2 eq; 6.81 mmol (1.05g)
(5.68 mmol)
$10
$10
5.11 mmol
Molecular Weight: 157.9
Molecular Weight: 172.0
NHz
Br
NHz
suggested yield: 90%
((HH)z
et
$10/g
1.2 eq; 10.2 mmol (1.614 g)
8.5 mmal (1.465
$16
$15
7.67 mmol
NHz
suggested yleld: ~ 75%
5.11 mmmol
7.87 mmol
3,83 mmol
Molecular Weight: 152,6
suggested yield: 75%
105gg
3.83 mmol
NH
1.2 eq.; 4.6 mmml (0.702 g) $7
3.06 mmol; $58
Figure 22: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 2.
31

---

Molecular Weight: 139.9
Molecular Weight: 176.0
suggested yield: 90%
B(OH)2
59/g
S10/g
1.2 eq; 6.81 mmol (0.953 g)
1 g (5.68 mmol)
5. mmol
Molecular Weight: 139.9
Molecular Waight: 172.0
NHz
NH2
suggested yield: 90%
B(OH)2
$9/g
$10/g
eq; 10.2 mmol g)
8.5 mmol (1.465 g)
$13
$15
7.67 mmol
NH₂
suggested yiald: 75%
5.11 mmol
7.67 mmol
3.83 mmol
Molecular Weight: 152.6
suggested yield: 80%
Cl
NH
105/g
3.83 mmml
1.2 eq.: 4.6 mmol (0.702 g)
$7
3.07 mmol; $54
Figure 23: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 3.
32

---

Molecular Weight: 153.9
Molecular Weight: 176.0
Br
suggested yield: 90%
B(HH)22
$10/g
$10/g
1.2 6.81 mmol (1.05
(5.68 mmol) $10
$10
5.11 mmml
Molecular Weight: 136.9
(HO),B
NH2
suggested yield: 75%
B(HH);
$10/g
5.11 mmol
1.5 eq.: 7.67 mmol (1.073 g) $11
3.83 mmol
Molecular Weight: 169.2
N=C=
B(OH)z
B(OH)2
suggested yield: 95%
$0.60/g eq;; 3.83 mmol; 0.65 ~ 50.4
HN
3.83 mmol
3.64 mmol
Molecular Weight: 176.0
B(OH)2
suggested yiekd: 85%
HN
$10/g
1.2 eq; 4.37 mmol; 0.769 $8
HN
3.64 mmol
3.1 mimol, $42
Figure 24: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 4.
33

---

Molecular Weight: 153.9
Molecular Weight: 178.0
Br
suggested yield: 90%
B(OH)z
$10/g
S10/g
1.2 6.81 mmol (1.05 $10
(5.68 mimo)
$10
5.11 mmol
Molecular Weight: 157.9
Molecular Weight: 187.0
NHz
B
B(OH)2
NHz
suggested yield: 90%
$10/g
$1.36/g
1.2 eq; 10.2 mmol (1.614 g)
8.5 mmol (1.593
F
16$
$2
7. 7.67 mmol
NHz
suggested yield: - 75%
5.11 mmml
7. 7.67 mmol
3.83 mmol
Molecular Weight: 153.1
NHz
suggested yield: 90%
33/g
HN
3.83 mmol
1.2 sq.; 4.6 mmml (0.704
$23
mmol; $62
Figure 25: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 5.
34

---

Molecular Weight: 153.9
Molecular Weight: 176.0
Br
suggested yield: 90%
B(HH)2
S10/g
S10/g
1.2 eq; 6.81 mmal (1.55g) )
g (5 mmol) $10
$10
5.11 mmmll
Molecular Weight: 136.9
(HO),B
B(HH);
NH2
suggested yield: ~ 75%
$10/g
5.11 mmol
1.5 eq.; 7.67 mmol (1.073 g)
3.83 mmol
S11
Molecular Weight: 147.2
B(OH)2
B(OH)2
suggested yield: - 75%
$5 60/g
3.83 mmml
1.5 eq,; 5.75 mmol; 0.846 $5
2.85 mmol
Molecular Weight: 176.0
B(OH)2
suggested yield: - 85%
105/g
1.2 eq: 3.45 mmol; 0.607
2.88 mmol
2.44 mmol, ~ $42
Figure 26: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 6.
35

---

Molecular Weight: 153.9
Molecular Weight: 178.0
suggested yield: 90%
B(OH)/
$10/g
$10/g
1.2 eq; 8.81 mmmll (1.05
(5.68 mmol) ~ $10
$10
5.11 mmol
Motecular Weight: 220.1
Bpin
suggested yield: ~ 75%
NHz
5.11 mmol
$39/g
3.83 mmol
1.5 eq.: 7.67 mmol (1,667
$65
Molecular Weight: 144.2
Bpin
Bpin
suggested yield: 90%
N
NH₂
HN
$40/g
3.83 mmml
eq.; 3.83 mmol; 0.552 322
3.45 mmol
Molecular Weight: 176.0
Bpin
suggested yield: 85%
$ $10/g
1.2 4. 14 mmol; 0.728 $7
3.45 mmol
2.93 mmol, $115
Figure 27: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 7.
36

---

Molecular Weight: 157.9
Molecular Weight: 178.0
suggested yield: 90%
B(HH)2
$101g
$10/g
1.2 eq; 6.81 mmol (1.076 g)
(5.68 mmol)
$11
$10
5.11 mmol
Molecular Weight: 136.9
OH
B
HO
OH
NH2
suggosted yield: ~ 75%
OH
$10/g
5.11 mmol
1.5 eq.; 7.67 mmol (1.073 g)
3.83 mmol
Molecular Weight: 152.6
OH
"O
suggested yleld: 75%
$10/g
3.83 mmol
1.2 eq.; 4.6 mmol (0.702 g)
B-OH
S7
HO
2.88 mmol
Molecular Weight. 176.0
Br
suggested yield:
S10/g
B OH
HO
1.2 eq; 3.45 mmol (0.607 g)
$8
2.88 mmol
2.44 mmol, $44
Figure 28: Plausible synthesis plan and estimated precursor cost for RGFN- ClpP ligand 8.
37

---

Molecular Weight: 153.9
Molecular Weight: 176.0
Br
suggested yield: 90%
B(OH))
$10/g
$10/g
eq, 6.81 mmol (1.05 9)
(5.68 mmol)
N
$10
$10
5.11 mmol
Molecular Weight: 139.9
Moleqular Weight: 172.0
NHz
B
NHz
suggested yield: 90%
B(OH),
$9ig
$10/g
eq; 10.2 mmol (1.43
mmol (1.465 g)
F
513
$15
7.87 mmol
NH2
suggested yield: 75%
N
5.11 mmmll
7.67 mmol
3.83 mmol
Molecular Weight: 137.1
suggested yield: - 80%
$38ig
3.83 mmol
1.2 eq.: 4.6 mmml (0.63 g)
3,06 mmol; $72
Figure 29: Plausible synthesis plan and estimated precursor cost for RGFN-produced ClpP ligand 9.
38

---

Molecular Weight: 139.9
Molecular Weight: 176.0
Br
suggested yield: ~ 90%
B(OH)/
$9/g
S10/g
1.2 eq; 6.81 mmol (0.953
(5. 68 mimo) $10
$10
5.11 mmal
Molecular Weight: 137.9
Molecular Weight: 172.0
NH2
OH
NH2
suggested yield: 90%
B(OH)z
$10/g
$10/g
1.2 10.2 mmol (1.41 g)
8.5 mmol (1.465 g) ~ $15
OH
$14
7.67 mmol
NHz
suggested yield: 75%
OH
5.1 mmol
7.67 mmml
3.83 mmol
Molecular Waight: 152.6
suggested yield: - 80%
OH
Cl
OH
$10/g
3.83 mmol
1.2 eq.: 4.6 mmol (0.702 g) $7
3.06 mmol; $56
Figure Plausible synthesis plan and estimated precursor cost for RGFN-produced CipP ligand
39

---

Molecular Weight: 225.3
Molecular Weight: 198.2
suggested yield: ~ 95%
HO
NH2
0. mmol; $353
0.22 mmol,
0.2 mmol, $239/0.00 05 g: 0.0396 g $189
$162/0.05 g: 0.0505 $163
Molecular Weight: 226.3
Molecular Weight: 193.2
suggested yield: ~ 95%
NHz
HO
0.22 mmol: 0.0498 g: $19/0. g: S19
0.2 mmol: 0.0386 $19/0.08 g. - $15
0.19 mmol; $34
Molecular Weight:
Molecular Weight: 209.2
OH
suggested yield: ~ 95%
HN
mmol; 0.0528 gi
0.22 mmol; 0.046 g: $344/0.08 05 g; $317
0. mmol: $495
$337/0.1 $178
Molecular Weight: 258.2
Molecular Weight: 225.2
suggested yield: 90%
NHa
NH
NH
NFN
NH
0.2 mmol: 0.0512 g: $272/0.1 g: $139
0.2 mmol, 0.045 $3 $312/C.08 g: $281
0.1 mmol: $420
feasible pathways using the reactions provided by SyntheMol and that correspond known reactions
Figure 31: Plausible retrosynthesis plan and estimated precursor cost for Synthe ClpP
ligands 1-5.
40

---

Molecular Weight: 176.2
Molecular Weight: 259.3
suggested yield: 95%
OH
HN
HgN
0.2 mimol: $63/0.1 1.10 g: 0.0352 gi ~ $44
0.22 mmol; $298/0.1 1 g: 0.057 g: S170
0.19 mmol: $214
Molecular Weight:
Molecular Weight: 183.3
suggested yleld: 95%
233.2 2
NHz
H
0.18 mmol: $475
0.2 mmol, $624/0 05 g. 0.0366 g. $457
0.2 mmol; $19/0.05 0 3.0466 g: - $18
Molecular Weight: 229.2
Molecular Weight: 284
suggested yiald: 95%
HO
NH
0.19 mmol: $149
0.2 mmol; $19/0.05 1.448 g: ~ $17
mmol; $232/0.1 gi 0.0569 g: $132
Molecular Weight: 177.2
Molecular Weight: 139.2
Molecular Weight: 301.3
OH
suggested yield: 90%
NH
NHz
mmol: $19/0.05 050 g: 0.0354 g: $13
0.2 mmol: $19/0.05 0.0278 g: $11
mmol;
$66/0.05 g: 0,0602 $40
0.18 mmol; S64
suggested yield: ~ 95%
Molecular Weight: 176.2
Molecular Weight: 215.2
HN
H&N
OH
0. mmol;
0.2 mmol; $212/0.05 g: 0.0352 gi ~ $149
0.2 mimol: $235/0.05 0.05 0.043 $202
Figure 32: Plausible retrosynthesis plan and estimated precursor cost for Synthe:Mol-producee ClpP
ligands 6- 6-10.
41

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We provide extensively results various tasks and compare against other state -of-the-art methods.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the practicality of the generation process being limited by types reactions used, oracles employed, and auxillary pharmacological constraints that are not considered.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
42

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: For each task, provide detailed information experimental setup. Ap- pendix provides further details which allow for full reproducibility of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
43

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All baselines are either derived from openly available repositories or described in sufficient detail to allow reproduction. Code related to the current manuscript is publicly accessible.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
mublic/guides Codeeuumissioooooooo......
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips wttpccccccwccpcdeeeeeeeeeeepeeee wwweeoomiieiioooooooooooo iwtobbmmssioooooolomooomo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification; We provide training details including hyperparameters and optimizers, and other details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Multiple seeds were run for ablation experiment, and the variance in these experiments to estimate and report error bars. For molecular docking, because of the high compute cost, only run experiment per algorithm. Errors bars are also included when applicable for metric computation across generated samples.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
44

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide details in the appendix for compute resources required.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://eeur//: wttpheewwwrp...cpppoopeeeeeeeeee httpp/ubbl..bhiicccuiueeeeeeeeea
Answer: [Yes]
Justification: The research adheres strictly to the NeurIPS Code of Ethics. There are no human subjects participants involved, and we carefully considered the safety and security aspects this research.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the societal impact of the work in conclusions.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
45

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do foresee this paper to pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All previous basclines and tasks are properly referenced and licenses used within the appropriate terms.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
46

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification: Code of this manuscript is open sourced and contains documentation describing how to set and run the code.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
crowdsourcing or human subjects are involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No human subjects are involved.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
47

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
48


Reviews:
[{"review_id": "JK8DmTmdJ0", "reviewer": "Reviewer_FKST", "summary": "This work proposes a workflow to synthesize molecules with reaction templates starting from some building blocks. Through this pipeline, the synthesizability of the generated molecules can be improved based on some experimental evidence of molecular drug design tasks.", "questions": "(1) This reviewer is concerned about the pre-trained oracle models. It seems that almost every step of the pipeline relies on a single pre-trained model. So how to derive these pre-trained models independently?\n\n(2) This reviewer is concerned about the ability to generate truly novel molecules. It seems the proposed approach starts with a selected set of fragments and deduces products through reaction templates. However, since reaction templates are extracted from known reactions, it seems nearly impossible to generate purely novel molecules through this pipeline. How do the authors address and evaluate this issue?", "limitations": "(1) It seems the whole pipeline of RGFN heavily relies on many different pre-trained models. Each stage can affect the overall process drastically. \n\n(2) The proposed approach is based on reaction templates. The reaction templates must be updated frequently to cover more types of reactions. However, the template-based method has poor generalization to unseen reactions (new combinations of reactants).", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "(1) This reviewer agrees with the importance of researching the synthesizability of generated molecules;\n\n(2) The presentation of the proposed approach is very clear. This reviewer can quickly understand the major idea of the whole workflow; \n\n(3) The idea of generating molecules with chemical reactions is very valuable since this reviewer also thinks the most important thing is generating synthesizable molecules.", "weaknesses": "(1) The related work discussion in this work is insufficient. This reviewer just searches a bit on the web and find three highly related articles, which are \"Bridging the gap between chemical reaction pretraining and conditional molecule generation with a unified model\", \"A generative model for molecule generation based on chemical reaction trees\" and \"Generating molecules via chemical reactions\". This reviewer believes other relevant articles are not covered in this article. A more comprehensive and systematic related work section is required;\n\n(2) The proposed method has many limitations, including reliance on reaction templates and powerful pre-trained models. We state these limitations in the following section;\n\n(3) The evaluation of the proposed approach is not comprehensive enough. It does not include the conventional molecular generation benchmarks like novelty, validity, and diversity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Weakness 1 Response:\nThank you for pointing out the omission of the Qiang et al., 2023 manuscript (“Bridging the gap..”) and the work by Nguyen et al. (“A generative model … reaction trees”) , we added the references to the revised version. In the related work section we cited a review by Meyers et al., 2021 (reference #42 in the original manuscript) that covers many other relevant works. The manuscript “Generating molecules via chemical reactions\" by Bradshaw et al., was actually cited by us (reference #8). The cited manuscript (“A Model to Search for Synthesizable Molecules”) refers to the NeurIPS version, while the version pointed out by the reviewer (“Generating molecules via chemical reactions”) refers to an earlier ICLR workshop version. We also include several references mentioned by another Reviewer.\n# Weakness 2 Response:\nPlease refer to Question 1 Response for the answer to the stated limitation being reliance on pre-trained models, and Limitation 2 Response to the stated limitation being reliance on reaction templates.  \n\nReaction templates are necessary to avoid post facto evaluation of synthesizability for analogs generated solely based on physico-chemical parameters without regard to the realities of chemical synthesis. Synthetic intractability has in fact been a major issue with previous generative methods. The use of reaction templates solves this problem without imposing detrimental constraints on chemical diversity. Reaction templates are thus inherent to our approach as the only reliable means to avoid infeasible structures and/or to enforce focussing on robust chemical reactions (or those available at a preferred chemical supplier, like Enamine).  \n\nWe have added additional text to the manuscript to clarify this important point on the challenges of real-world synthetic chemistry. We thank the reviewer for highlighting this issue and hope the reviewer concurs with our arguments.\n# Weakness 3 Response:\nWe apologize for the lack of clarity on this point. By the definition of our search space, all molecules generated are inherently valid (e.g., correct valence, reasonable size, etc.) because all compounds are generated by bona fide synthetic reactions. We also demonstrate diversity of generated compounds in Figure 4, which shows RGFN finds more modes (i.e., molecules dissimilar to each other) compared to GraphGA and SyntheMol. We note that because the GFN model is trained from scratch, all generated molecules are “new” because the model is not trained on any known molecules. They are also highly different from known ligands, as demonstrated in Appendix L. We also note that the current largest space of synthesizable molecules at Enamine (~40 billion) is not systematic in terms of reaction combinations and is more than an order of magnitude smaller than even the limited RGFN space we present here. We have revised the text of our manuscript to better reflect these points.\n# Question 1 Response:\nWe apologize for a lack of clarity on this key issue. While some of the considered oracles are indeed pre-trained ML models (sEH proxy and senolytic proxy), we specifically include the GPU-accelerated docking as an alternative reward function to alleviate this issue. Our approach is agnostic to the reward function, and could even use something like a QED score as the reward. We emphasize that all of the considered baselines also rely on the exact same oracle models (including the pre-trained proxies). We believe that a strength of our approach is its adaptability to all possible reward functions. \n# Question 2 Response:\nAlthough having a limited set of known reactions in principle does restrict diversity, the space of synthesizable molecules is still vast and greatly exceeds all synthesized molecules to date. A huge space of compounds remains to be queried for biological activity based only on established chemical reactions, due to the massive number of related building blocks that can be plugged into any given reaction. Thus, there are approximately 219M substances known to be produced and characterized in the literature [1] and the evaluated size of produced chemical space in RGFN is already larger, despite using only a modest number of input reactions and building blocks. The total chemical space encompassed by these reactions and associated reactants has barely been explored. \n# Limitation 1 Response:\nAs above, we emphasize  that RGFN does not heavily rely on pre-trained models. Please refer to Question 1 Response for the answer to this perceived limitation. \n# Limitation 2 Response:\nWe would like to clarify the difference between reactions and reactants. The number of feasible reactions is limited, the number of reactants that can be joined together by these reactions is very large, and hence the number of combinations is extremely large. To give a simple example, naturally occurring peptides are generated by a single reaction type (amide coupling) with only 20 reactants (the L amino acids) yet can linearly generate tremendous chemical diversity. The reaction templates need not be updated as more reactants are added - the diversity of possible reactants that can be accommodated alone enables a combinatorial explosion. In practical terms, it is also possible to add any reaction to the dataset using the SMARTS language, and those that we present in the paper are known to be generally very robust and reliable. While this method obviously doesn't allow us to discover ligands that could be produced from yet unknown reactions, it is not an aim of this work to devise new synthetic reaction methods. On the contrary, our stated goal is to ensure \"out-of-the-box\" synthesizability by using selected reactions that are more likely to deliver the product and thus shorten the discovery process, while at the same time ensuring massive chemical diversity.  \n\n[1] https://www.cas.org/cas-data/cas-registry", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1WbmPGaMXX", "reviewer": "Reviewer_rpKJ", "summary": "This work proposes a GFlowNet-based framework for synthesizable molecule design, which comprises of building block selection stages and reaction selection stages. The GFlowNet-based model uses chemical oracle functions as reward and the goal is to generate synthesizable molecules with the desired property quantified by the scoring function.", "questions": "- Molecules generated by building blocks and reactions are guaranteed to be highly likely synthesizable. Is it still necessary to evaluate empirical synthetic accessibility scores of these molecules? as these scores are generally inaccurate and they cannot tell much when a molecule already has synthesizability guarantee.\n- Can this framework be applied to bottom-up synthesis planning?", "limitations": "See Weaknesses", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This work tackles the synthesizability challenge in molecular design, which is crucial to experimental validation but has been long overlooked by previous work on molecular generation.\n- Molecular synthesis pathways have been represented by directed acyclic graphs, which fit well the structure of GFlowNet sampling process. Therefore, this work choosing GFlowNet as the framework is proper and well-motivated.\n- The experimental study is comprehensive. It demonstrates that RGFN can produce synthesizable molecules with higher docking score. It also examines the diversity and docking structures of the generated molecules", "weaknesses": "- As acknowledged in the paper, the major limitation is scalability to a larger chemical space. It becomes much more difficult to discover hit scaffolds when the chemical space gets larger. In addition, the capability of this framework is limited by oracle functions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Weakness 1 Response:\nRegarding limited library size, please refer to the Overall Response. We agree with the oracles being a limiting factor, but as the Reviewer states, this is not a unique problem to RGFN.\n# Question 1 Response:\nThe reviewer raises a good point. We agree that the SA scores are inaccurate, and in our opinion using them is not really necessary. However, we have tried as best as possible to demonstrate the synthesizability aspect in a rigorous way, and including the SA scores was one data point (in addition to the AiZynthFinder and manually showing the synthesis paths of top-10 molecules). Huge differences in SA scores can thus serve as a rough guideline. We also anticipated that not providing SA scores might lead to reviewers requesting the scores be included in our analysis, since the application of such scores is  fairly common. We acknowledge that demonstrating synthesizability is difficult in general, and more work into this problem  is required (but is outside the scope of this paper). We have added some text to highlight the caveats with current synthesizability assessment methods.\n# Question 2 Response:\nAt this juncture the platform does not automatically recommend specific synthetic routes.  However, a trained synthetic chemist can readily propose reasonable synthetic routes for any proposed structure from RGFN, particularly given that the reaction templates used are well known. Synthetic routes may also be proposed by well-developed stand alone retrosynthetic software packages.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "biYvLSSidg", "reviewer": "Reviewer_5soV", "summary": "This paper presents a model for molecule design called Reaction-GFlowNet (RGFN). RGFN is an extension of the GFlowNet framework [4] (which has previously been used to generate molecules by building them out of small fragments) to generate molecules through virtual chemical reactions, ensuring that the molecules RGFN generates are more likely to be synthesizable. The paper evaluates RGFN against GraphGA (a graph genetic algorithm), FGFN (the original fragment-based GFlowNet), and SyntheMol (a recent synthesis-based generative model of molecules proposed in [54] based around Monte Carlo tree search). The paper shows RGFN finds competitively scoring molecules on two proxy* and one docking task (as expected the unconstrained GraphGA does best), while also ensuring the molecules it suggests do better on important synthesizability metrics.\n\n* proxy meaning the task is to optimize against a graph neural network (GNN) oracle trained on a small amount of target data.\n\n(Edited Aug 14 to update the score: see comments below).", "questions": "1. What is the distribution of the number of reaction steps typically used when finding useful molecules?\n\n\n\n2. I don't completely follow how the size of the state space was calculated in Figure 2. Perhaps a cartoon would be useful in Appendix A. Is the state space size different to the total number of unique molecules that can be generated?\n\n\n\n3. I'm confused as to why RGFN seems to often outperform FGFN in the docking/proxy scores of the molecules it finds. I would have thought the latter model is more flexible and so should be able to find better scoring molecules, even if they did not end up being synthesizable. The paper suggests that the library of fragments available for RGFN might be less suitable than those found in the building blocks picked for RGFN. Have different fragments been tried or other experiments carried out to investigate this hypothesis?\n\n\n\n4. RGFN seems to produce molecules with lower QED scores than SyntheMol and FGFN: is there any intuition for why this might be the case?\n\n\n\n5. Line 243 says:\n> \"All RGFN modes were additionally inspected manually by a chemist and confirmed as synthesizable, which indicates that AiZynth scores are likely underestimated.\"  \n   \nDoes this mean that a chemist took the proposed molecule and synthesized it in the lab? Were the proposed synthetic routes from RGFN used or were alternatives derived?\n \n\n\n6. Line 282 states:\n>  \"It is also important to recognize that RGFN does not explicitly generate synthetic routes to the molecules.\"    \n\nI did not understand what was meant by this. I thought the whole point of RGFN is that it did explicitly generate synthetic routes to the final molecule through the sampled trajectory? \n\n\n\n7. Equation 4 versus 7:\na. My understanding is that the architecture expressed in Equation 4 is used instead of that of Equation 7 (apart from in Section 4.3)?\nb. The architecture expressed in Equation 7 seems to do better in Figure 5 so why not always use this?\nc. Could also a similar principle be used to scale up the number of reaction templates available?\n\n\n\n8. Line 221 discusses a procedure for picking molecular modes from the list of proposed molecules. Given modes have to be a certain Tanimoto distance apart, I assume this procedure is run greedily and is somewhat dependent on how the molecules are ordered?", "limitations": "Two limitations are discussed in Section 5: (i) that the model is currently limited to a very small reaction and building block library, and (ii) the issues inherent with docking oracles. Regarding the first of these, the authors discuss and evaluate a modification of the architecture to try to enable the method to scale (see Section 4.3), but performance still degrades as the initial fragment library increases and so this seems to remain an outstanding limitation (see also the weaknesses section above). The second limitation (docking oracles are not perfect) is not limited to RGFN and is a problem among similar models more generally. The authors discuss incorporating additional, more-accurate oracles in a multi-fidelity framework to resolve this which seems reasonable.\n\nAnother limitation which is not discussed is the sample efficiency of the proposed approach (which is again also a limitation for the baselines). The number of molecules the non-GFlowNet baselines visit during optimization for the different tasks is described in Section 4. These are high and range from 70k-400k, and I would be interested in how this compares to the two GFlowNet models.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "## Summary of review\nThis paper addresses one of the limitations of GFlowNets by explicitly incorporating synthesis plans into the molecular generative process. However, this seems to be a fairly straightforward extension of the GFlowNets framework with synthesis-based generative models of molecules proposed elsewhere. The experiments in this paper compare the proposed method against only one of these existing models.\n\n## Originality/Significance\nRFGN is an extension of GFlowNets [4] to build molecules through chemical reactions rather than through combining small molecular fragments. This addresses a key limitation with the original GFlowNet, which the experiments show often generates molecules for which no synthetic route can be found.\n\nHaving said that, there have been many synthesis-based generative models of molecules proposed (e.g., [8,17,54,19,22] and others not cited -- see below), and RGFN seems a fairly straightforward combination of these ideas with GFlowNets.\n\n> Korovina, K., Xu, S., Kandasamy, K., Neiswanger, W., Poczos, B., Schneider, J. and Xing, E. (2020) ‘ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations’, in International Conference on Artificial Intelligence and Statistics. AISTATS 2020    \n\n> Nguyen, D.H. and Tsuda, K. (2022) ‘Generating reaction trees with cascaded variational autoencoders’, The Journal of chemical physics, 156(4), p. 044117.   \n\n> Vinkers, H.M., de Jonge, M.R., Daeyaert, F.F.D., Heeres, J., Koymans, L.M.H., van Lenthe, J.H., Lewi, P.J., Timmerman, H., Van Aken, K. and Janssen, P.A.J. (2003) ‘SYNOPSIS: SYNthesize and OPtimize System in Silico’, Journal of medicinal chemistry, 46(13), pp. 2765–2773.  \n\n\n> Button, A., Merk, D., Hiss, J.A. and Schneider, G. (2019) ‘Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis’, Nature Machine Intelligence, 1(7), pp. 307–315.\n​\n\n## Quality\nThe optimization tasks done to evaluate RGFN seem interesting and it's nice to see the molecules suggested also evaluated in terms of their diversity and synthesizability (using a retrosynthesis planner). While I think it is great that the tasks were more complicated than some of the simple ones often used elsewhere, it might have been nice to include some more commonly-used (yet still challenging) benchmarks too (such as the Therapeutics Data Commons benchmark -- citation below). This would make comparisons with existing models easier.\n\n> Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik.   Therapeutics data commons:  machine learning datasets and tasks for therapeutics. arXiv preprint arXiv:2102.09548, 2021.\n\n\n## Clarity\nOverall I thought the paper was well-written and easy to follow. In particular, I thought Section 3.2 was helpful in giving an overview of the approach (along with the informative Figure 1), and Equations 2–7 were helpful in explaining the parameterizations of the various networks involved. The experiments also seemed to be clearly presented. Readers unfamiliar with GFlowNets [4] would likely have to read about them elsewhere (lines 70–82 provide a short, although limited introduction). However, this seems reasonable due to the overall space available.", "weaknesses": "## Originality\nAs mentioned under the originality/significance heading in the section above, RGFN seems a fairly straightforward combination of the ideas of GFlowNets with recent synthesis-based generative models of molecules. It would have been helpful if there had been more focus on why synthesis-based GFlowNets were a better approach to take than the currently proposed alternatives to better promote the paper's significance. (Hence why I have gone with a low contribution score).\n\n## Only one other synthesis-based generative model of molecules is compared against \nThe only other synthesis-based generative model for molecules that is compared against in the experiments is SyntheMol, and this baseline has been limited here in the amount of compute it uses. I can understand no comparison is done against [22,19] due to the lack of open-sourced code, but it would have been nice to have had a comparison against some of the other ones cited, e.g., [17] with its open sourced code available on GitHub (https://github.com/wenhao-gao/SynNet). It would also be nice to have a comparison to the other mentioned methods for encouraging synthesizability, e.g. via using scoring methods [34].\n\n(This along with the fact that no code is available -- see below -- is why I have gone with a lower soundness score).\n\n## No code is shared\nAs far as I'm aware there is no code currently available for RGFN? It does not seem to be provided as part of the submission (despite this box being ticked in the paper checklist)? I would find it impossible to reproduce the results of the paper using the details provided in Appendices B-D alone (e.g., I'm unsure of what building blocks and reaction templates to use or even how the molecules fed into the GNNs were featurized). One of the proxies was initially trained in an unsupervised manner on the ZINC dataset (Appendix B), but there are scant details on how this was done.\n\nOn a similar note, the compute resources used to train the models also seems to be missing. This is again marked as included in the paper checklist, so would appreciate being pointed towards this if I have made a mistake and missed this?    \n\n\n## Method is currently limited to a restricted synthetic space\nRGFN uses only 350 building blocks and 17 reactions. This is low compared to the ~150k building blocks and 49 reactions used in [19], or the 5000 building blocks and 90 reactions used in [22]. The experiments done in Section 4.3 suggests the RGFN scales badly to larger building block libraries even with architecture modifications and larger reaction libraries do not seem to have been tried. This, combined with the fact that RFGN can only explore linear synthetic trees (i.e., multiple complex intermediates cannot be created and then combined) and bimolecular reactions restricts the molecules that the approach could explore. This might be an issue in more challenging optimization tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Significance\nWe thank the Reviewer for pointing out additional references that were included in the revised version. While the idea of extending GFNs to operate in the reaction space might seem straightforward, significant work was needed to implement the approach, not emphasized enough in the original manuscript. These include: \n- Handling multiple possible products of a given reaction, which results in a de facto non-deterministic environment. This is not an issue in FGFN, as operating on graphs allows us to specify the place of inserting a new fragment. We deal with this by introducing a product selection step (Eq. 5).\n- Since GFNs require computation of parent states and masking invalid actions, this required implementing efficient recursive decomposition of molecules into BBs and crafting specific SMARTS templates for reactions that would limit the possible number of parents.\n- Crafting a specific set of high-yield reactions and low-cost fragments that would enable synthesis.\n- Improving scalability by introducing fingerprint-based action embeddings for fragment selection.\n\nAdditional discussion can be found in Overall Response 2.\n# Quality\nWe included the DRD2 benchmark from TDC. We attach the results. The trends are comparable to other oracles.\n# W1\nPlease refer to the answer above regarding the originality of the approach. The comment regarding needing to emphasize more the differences between our and existing methods, was addressed in the Overall Answer.\n# W2\nWhen implementing SynNet, we were unable to reproduce a decoder that reliably generated valid SMILES strings. As the initial trained weights provided did not match the most recent version of the US Enamine BB stock and generated nonsensical SMILES, we retrained the model’s four MLPs on the recent stock, which still did not result in reliable SMILES outputs. Due to time constraints, we instead opt for RXNGenerator, a VAE model designed for generating synthetic trees. For a general response regarding other synthesizable molecule generation methods, see Overall Response 2.\n# W3\nWe indicated in the paper checklist that the code would be provided upon acceptance and hope it wasn’t misleading to mark it as such in the checklist. We provide the code as is in the AC comment.  \n\nWe did mistakenly omit the computational resources in the original manuscript, we apologize for that. Evaluating fragment scaling took approximately 800 RTX4090 hours in total. Remaining experiments took roughly 24 hours on Quadro RTX 8000 for sEH, DRD2 and senolytic proxies, and roughly 72 hours on an A100 for docking-based proxies.\n# W4\nSee Overall Response 1.\n# Q1\nVast majority of the molecules were generated using 4 reactions (maximum allowed number). We applied this constraint because larger molecules tend to achieve higher docking scores, and GFNs generate samples with the probability proportional to the reward, leading to a higher likelihood of generating larger molecules. Achieving higher size diversity would be possible by penalizing larger molecules.\n# Q2\nThe state space size is precisely the number of unique molecules that can be generated from the listed reactions and reactants. We have tried to explain this more clearly in the revised text.\n# Q3\nUnfortunately, we were unable to explore using different FGFN fragments in time for the rebuttal. Other factors we suspect were 1) larger state space size of FGFN (which makes convergence more difficult), and 2) implementation differences (we based FGFN on [1], while RGFN was implemented from scratch).\n# Q4\nOur hypothesis is that the QED differences stem mostly from different average molecular weights. While we tried to achieve comparable weight ranges for different methods, it’s still task dependent (Table 1). SyntheMol uses shallow synthesis trees which lead to smaller molecules. We note that QED could easily be improved upon by adding it directly as a reward term. \n# Q5\nSynthetic routes from RGFN were used. Costs of synthesis were estimated according to literature reaction yields, building block availability, and unit price. The molecules generated in the paper were not synthesized, but we are in the process of doing that for another batch.\n# Q6\nIt is unlikely actual synthetic routes would fully follow those generated by RGFN due to the linear nature of RGFN compared to a retrosynthetic analysis that would aim to maximize the yield RGFN is not intended to optimize the synthetic route but rather to generate synthesizable compounds.  \n# Q7\n- Yes, we use the architecture expressed in Eq. 4 rather than Eq. 7, which outperforms with larger fragment libraries. However, for the fragment library outside Section 4.3 which contains 350 molecules, both architectures perform on-pair, as illustrated in Figure 5. We opt for Eq. 4 only because it is simpler. We note this in the text.\n- The embedding scheme proposed in Eq. 7 can be used for reaction templates. We did experiment with using RxnRep [2] embeddings in the early stage of the project but observed only a negligible performance improvement.\n# Q8\nWe employed the standard techniques of the Leader algorithm as implemented in RDKit. While in theory this process is dependent on the order of the molecules, in practice the molecules are always shuffled and the standard deviation for the number of modes across 10 shuffled runs is less than 0.5%. We now add a note to this in the manuscript.\n# L1\nRegarding limited library size, please refer to the Overall Response. We agree with the oracles being a limiting factor, but as the Reviewer states, this is not a unique problem to RGFN.\n# L2\nThe GFlowNet models each visited 400k molecules, the budget set for baselines, with the exception of SyntheMol. However, our models were able to converge much earlier, and allowing RGFN/FGFN to run for the remaining time was meant to allow exploration. We attach convergence plots for RGFN to illustrate this.  \n\n[1] github.com/recursionpharma/gflownet  \n[2] doi.org/10.1039/d1sc06515g", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZP1hZL5P8l", "reviewer": "Reviewer_sx7b", "summary": "This study modifies prior GFlowNet-based frameworks to produce molecules satisfying synthesizability. While previous GFlowNet-based molecular generation completes molecules by adding fragments or atoms, the proposed method repeats following stems to generate molecules: (1) selects reaction templates, (2) selects reactants, and (3) performs reactions and selects one of the resulting molecules. In the experiments, the proposed method shows competitive performance while preserving synthesizability.", "questions": "See weaknesses.", "limitations": "The authors have acknowledged the limitations of their approach.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The proposed method first incorporates a reaction-based generation framework with GFlowNet for synthesizability.\n- A chemical language for synthesizability seems to offer cost advantages in real-world applications, demonstrating that it acts as a promising inductive bias to generate promising molecules.\n- The proposed method shows competitive performance compared to (1) FGFN in terms of synthesizability and (2) SyntheMol in terms of average reward and mode discovery. Although it finds fewer modes compared to FGFN, this seems to stem from constraints in the generation space.", "weaknesses": "- The preliminary is insufficient for understanding the components of the method, particularly mentioning the flow matching condition while the method section actually describes forward and backward policy.\n- To improve synthesizability-related metrics, one may consider including them in the rewards. Can authors provide a comparison with FGFN including the SA score in the reward? (e.g., multiobjective GFN)\n- The lower number of modes in RGFN (compared to FGFN) is due to constraints in the generation space (ensure high SA scores). Therefore, it would be interesting to compare the number of modes filtered by SA scores.\n- Can authors compare the cost of generation time? Additionally, I am curious about time costs for (finding a synthesizability path + generation with FGFN) vs. (generation with RGFN).\n\n**Minor:**\n- Error in line 114 $m'i$", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Weakness 1 Response:\nThank you for this comment, we agree that the preliminary section present in the original manuscript was insufficient for understanding the method. We revised it to include the following information, which hopefully will address Reviewer’s remark:\n> Another way to rephrase the flow-matching constraints is to learn a forward policy $P_F(s_{i+1}|s_i)$ such that trajectories starting at $s_0$ and taking actions sampled by $P_F$ terminate at $x \\in \\mathcal{X}$ proportional to the reward.\n\n> Trajectory balance. Several training losses have been explored to train GFlowNets. Among these, trajectory balance [citation] has been shown to improve credit assignment. In addition to learning a forward policy $P_F$, we also learn a backward policy $P_B$ and a scalar $Z_\\theta$, such that, for every trajectory $\\tau = (s_0 \\rightarrow s_1 \\rightarrow \\dots \\rightarrow s_n = x)$, they satisfy:\n\\begin{equation}\nZ_\\theta \\prod_{t=1}^{n} P_F(s_t|s_{t-1}) = R(x) \\prod_{t=1}^{n} P_B(s_{t-1}|s_t)\n\\end{equation}  \n\nHowever, please note that we are still fairly constrained in terms of space, hence the description has to remain brief.\n\n# Weakness 2 Response:\nThank you for the suggestion. We included a fragment-based GFN that used the average of proxy value and SA score as the reward as another baseline (specifically, we used $R(x) = exp(\\beta * (0.5 * proxy(x) / max_{proxy} + 0.5 * (10 - SA_{score}) / 10)$, with $\\beta$ adjusted per proxy to match the original reward range. We attached the results in the PDF with the main answer to the reviews. As can be seen, using the SA score as a part of the reward slightly reduces the obtained proxy values, while improving the SA scores (which is expected). However, as can be seen, it does not translate to AiZynthFinder scores on par with RGFN, which highlights the issue of poor reliability of SA scores that was discussed in the manuscript.\n\n# Weakness 3 Response:\nThank you for the suggestion. As requested, we investigated computing the modes based on SA score-thresholded molecules only (with SA <3, which should correspond to easy-to-medium synthesis difficulty; lower values yielded similar trends). We conducted this experiment specifically for ClpP, with the results attached. However, this did not turn out to change the results significantly, since as discussed in the original manuscript, SA scores are a poor approximation of synthesizability. We could not conduct a similar experiment with AiZynthFinder in time due to much larger computational cost. We agree with the Reviewer that selecting only synthesizable modes should significantly improve RGFN’s performance, but the issue is once again evaluating the synthesizability.\n\n# Weakness 4 Response:\nOn a single RTX Quadro 8000 GPU, generating 1000 molecules from a trained FGFN model took ~7 seconds, while evaluating the top 100 molecules with AiZynthFinder took 2206 seconds. On the other hand, generating 1000 samples using RGFN took ~14 seconds. It’s worth noting that FGFN used a heavily optimized implementation [1], while our approach didn’t leverage multiprocessing for sampling, which might have produced additional overhead. Still, this illustrates that when taking into account the cost of synthesis, using RGFN becomes much more efficient.\n\n# Minor\n\nThank you for spotting this, it was revised!\n\n[1] https://github.com/recursionpharma/gflownet", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you very much for your detailed feedback on our manuscript. We understand and have carefully considered your concerns with our work, and aim to provide a summarized response to the most common points here.  
# Small Fragment Library
We agree with the Reviewers that the search space is indeed constrained as it stands. However, we wish to note that this constraint was purposeful to ensure practical fast synthesis for the generated molecules. That is, the end goal is to allow a typical chemical laboratory to quickly synthesize de novo molecules and test their properties experimentally. It is reasonable to expect such a laboratory to own hundreds of building blocks and conduct a handful of very reliable organic reactions, whereas it is impractical to employ >1000s building blocks and >20s reactions, as many building blocks/reactions require specialized synthetic expertise. We agree with the Reviewers that the trade-off is a limited search space, but our method nevertheless yields billions to trillions of synthesizable molecules, exceeding all current experimental techniques (e.g., DNA encoded libraries) which already yield strong drug candidate molecules for many targets. We do plan to incorporate non-linear synthetic trees and multi-component reactions but these aspects need careful experimental validation to ensure true synthesizability and robustness. 
# Originality / Similar Works
We thank the Reviewers for highlighting additional relevant papers that we overlooked. We note that the majority (UniRXN, RXNGenerator, SynNet, ChemBO) rely on pre-trained neural networks to predict synthetic trees and therefore require the curation of large datasets of druglike molecules and generating ground-truth synthesis routes. This is in part an additional drawback of fragment and reaction datasets too large to serve as discrete action spaces. Moreover, while template-free methods like UniRXN, ChemBO, and MoleculeChef are capable of accurately predicting the products of reactions given reactants, they do not specify exactly the reaction used at each step - valuable information for chemists to prioritize easy-to-synthesize molecules out of a large pool of generated candidates. We present a simple alternative to these methods that avoids both the time cost of pre-training and provides template-based synthetic routes.
# Scalability to Larger Fragment Libraries
As stated in the manuscript, we recognize that one of the limitations of our approach is degrading performance with very large fragment libraries. However, as explained before, it is not necessarily our goal to support very large libraries in the first place, as they can be impractical if one wants to achieve rapid in-lab synthesis. Still, we recognize that their support can be useful in some settings, and provide a mechanism for improving scalability (using fingerprint-based action embeddings). We additionally would like to point out that 1) we believe decreasing performance is somehow expected, since by adding more fragments we increase the state space size exponentially, and it takes the model longer to converge, 2) still, even in the considered setting, the number of discovered modes remained large, suggesting the feasibility of using large libraries to increase chemical diversity (while preserving a high number of discovered modes). First point was illustrated in another experiment we conducted (attached with the figures), in which we count the number of discovered high-reward scaffolds in the last 10k samples throughout training. There we can see that as the model trains, the gap between small and large fragment sets decreases.
# Additional Results
As requested by the Reviewers, we included FGFN that used SA score as one of the reward terms as one of the baselines. We considered one more task, DRD2 oracle from TDC repository, as requested by another Reviewer (with results for SyntheMol still computing). We also included an additional baseline, RxnGenerator [2], but due to a very slow generation speed at this time only obtained results for the DRD2 task with a small number of molecules generated. While it achieves similar synthesizability, it discovers significantly less modes. Finally, we’d like to indicate that there was an error with the original results presented for FGFN in the senolytic generation task, in which we didn’t scale the proxy values properly. We attach the updated plots, but would like to point out that it did not change the overall conclusions, as FGFN still discovers only a very small number of high reward molecules. Still, we apologize for that mistake.

[1] Huang, Kexin, et al. "Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development." arXiv preprint arXiv:2102.09548 (2021).    
[2] Nguyen, Dai Hai, and Koji Tsuda. "A generative model for molecule generation based on chemical reaction trees." arXiv preprint arXiv:2106.03394 (2021).

Meta-review: This paper studies generative models for molecular discovery, focusing on incorporating synthesizability by generating molecules within the chemical reaction space. The proposed method, RGFN, performs molecular generation explicitly in this space.

Given that this paper is on the borderline, we had a detailed AC-reviewer discussion to carefully assess pros and cons of the manuscript. The reviewers mainly identified the following concerns:
- **Evaluation on the AiZynth Score:** The evaluation is not entirely convincing. Specifically, it is important to clarify whether the reaction templates used in the GFlowNet framework overlap with those in AiZynthFinder and whether the baseline models are appropriately tuned, as these factors could significantly influence the scores.
- **Novelty:** The novelty of the approach is somewhat limited, as it is a relatively straightforward extension of the existing GFlowNet method.
- **Discussion of Related Work:** The initial submission lacked a sufficient discussion of related work, although this was adequately addressed in the authors' rebuttal.

In my opinion, these issues are not prohibitive for publication. For a new method, especially in empirical research, comprehensive evaluation is crucial, while it is not mandatory for the proposed method to outperform all existing approaches. Rather, it is essential that the method is reliable, which I believe is an important contribution from the ML side, and this paper satisfies it.

Considering these points, I recommend accepting the paper. Please ensure that the final version includes revisions based on the reviewers' suggestions and incorporates the additional empirical results provided in the authors' rebuttal, particularly those in the PDF.
Decision: Accept (poster)