SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraints
Authors: Miruna Cretu
Charles Harris
Ilia Igashov
Arne Schneuing
Marwin Segler
Bruno Correia
Julien Roy
Emmanuel Bengio
Pietro Lio
Venue: ICLR 2025 Spotlight (2025)
Keywords: GFlowNets
de novo molecular generation
synthesizable molecular design
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A GFlowNet with an action space of chemical reactions and a trained backward policy.

Abstract: Generative models see increasing use in computer-aided drug design. However, while performing well at capturing distributions of molecular motifs, they often produce synthetically inaccessible molecules. To address this, we introduce SynFlowNet, a GFlowNet model whose action space uses chemical reactions and buyable reactants to sequentially build new molecules. By incorporating forward synthesis as an explicit constraint of the generative mechanism, we aim at bridging the gap between in silico molecular generation and real world synthesis capabilities. We evaluate our approach using synthetic accessibility scores and an independent retrosynthesis tool to assess the synthesizability of our compounds, and motivate the choice of GFlowNets through considerable improvement in sample diversity compared to baselines. Additionally, we identify challenges with reaction encodings that can complicate traversal of the MDP in the backward direction. To address this, we introduce various strategies for learning the GFlowNet backward policy and thus demonstrate how additional constraints can be integrated into the GFlowNet MDP framework. This approach enables our model to successfully identify synthesis pathways for previously unseen molecules.

Published as a conference paper at ICLR 2025
SYNFLOWNET: DESIGN OF DIVERSE AND NOVEL MOLECULES WITH SYNTHESIS CONSTRAINTS
Miruna Cretu', Charles Harris', Ilia Igashov Arne Schneuing', Marwin Segler', Bruno Correia', Julien Roy", Emmanuel Bengio', Pietro Liò 'University of Cambridge, "EPFL, 'Microsoft Research, *Valence Labs (mtc49, cch57, pi219 @cam ac uk,
emmanuel www.eeeeeeeeeeeeeeeeoeoeeeeeeeee
com
ABSTRACT
Generative models increasing in computer-aided drug design. However, while performing well at capturing distributions of molecular motifs, they often produce synthetically inaccessible molecules. To address this, introduce Syn- FlowNet, a GFlowNet model whose action space uses chemical reactions and purchasable reactants to sequentially build new molecules. By incorporating for- ward synthesis as an explicit constraint of the generative mechanism, we aim at bridging the gap between in silico molecular generation and real world synthesis capabilities. We evaluate our approach using synthetic accessibility scores and an independent retrosynthesis tool to assess the synthesizability of our compounds, and motivate the choice of GFlowNets through considerable improvement in sam- ple diversity compared to baselines. Additionally, we identify challenges with reaction encodings that can complicate traversal of the MDP in the backward direc- tion. To address this, we introduce various strategies for learning the GFlowNet backward policy and thus demonstrate how additional constraints into the GelowNet MDP framework. This approach enables our model to success- fully identify synthesis pathways for previously unseen molecules. Source code is available at https: / //ggttttuhu/// com mi wwwm//iiuancrt//o/ooocooooo.ooee www.uaacrttt/nnoonett..t..mm
be integrated
INTRODUCTION
Designing molecules with targeted biochemical properties is a critical challenge in drug discovery, where computational models could play a significant role to increase efficiency and effectiveness. Recently, generative models have lead to a renaissance in de novo molecular design (Stanley & Segler, 2023; Du et al., 2024). However, most current de novo design models do not explicitly account for synthetic accessibility. Many approaches operate SMILES strings (Segler et al., 2017; Gupta et al., 2018) or assemble molecules by composing atoms or molecular fragments into a graph (Lewis & Dean, 1989; Jensen, 2019; Jin et al., 2018), leaving guarantee that the sampled molecules be synthesized (Stanley & Segler, 2023; Gao & Coley, 2020).
To address this limitation, synthetic complexity scores (Ertl Schuffenhauer, 2009; Coley et al., 2018; Liu et al., 2022) have been proposed as a method to assess molecules and complement generative models with knowledge on synthetic accessibility, however these heuristies and learned metrics are often oversimplified. Alternatively, computer-aide aided synthesis planning can can be performed as subsequent step to molecule generation (Segler et al., 2018; Schwaller et al., 2020). While synthesis planning can provide principled routes, it can take seconds to minutes to propose pathways, which hampers its integration as reward function in the generation process (Liu et al., 2022). Finally, synthetically accessible chemical spaces can be assembled from combinations of reactions involving readily available reactants (Klarich et al., 2024), but virtual screening on these vast datasets is already impractical today (screening only fraction of such libraries amounts to thousands of years of compute on a single CPU; Sadybekov et al., 2022) and these spaces are still growing exponentially.
Given the challenges of synthesizability in atom- and tigmentt-based generative models, propose formulating molecule generation in an action space of chemical reactions which naturally enhances synthesizability. However, simply employing synthesizable space is not sufficient, To be applicable

---

(C) Finally, Published as a conference paper at ICLR 2025
(A) SynFlowNet reaction DAG
Forward Flow
Backward Flow
Reward
Doua
Binding Affinity
MIP
AddF i rstReactant
Graph Transformer
$0
MLLP
ReactLnn
(MLP
React8t
MLP
Adckesctant
MMIP)
Stop
(MIP)
(MLP)
BekReac tünn
MLP
1c1
(B) Building block sampling
(C) Learned backward policy
without finge- ceeppiintt
Logits
Prebabiitios
Sk
Buitdin Mlod 181
-
So
Binary Morgan Ançerooins
with Enng Inggrpprint prinnts
noca B8
Sk
MIP
Loghts
Protutilttes
Figure SynFlowNet allows for synthesis-aware molecule generation. (A) The state space is induced by combining purchasable building blocks (BBs) and chemical reactions. Every final molecule (rectangle box) associated with a reward. Training trajectories are constructed by sampling the model forward. Our policy (r(als) is parameterized as graph transformer which at each timestep processes the current molecular state 8; and outputs a shared embedding which is passed to separate MLP heads to predict the action logits for different action types (5 forward and backward action types). An action a, is then sampled from this hierarchical distribution to transition to the next state via reaction. (B) To allow handling large sets of BBs (up 200k), represent them using Morgan fingerprints and compute the probability of sampling a particular BB from the normalised dot product between this representation and the MLP output for the current state. The state representation is concatenated with the one-hot encoding of the selected reaction.
when traversing the MDP backwards, to reduce the probability of exiting the MDP defined by our set of reactions and BBs, we train the backward policy to avoid paths that do not terminate in Su-
drug discovery programs, such model needs maximize desired properties and retrieve molecules from distinct modes of the available synthetic space. Generative Flow Networks (GFlowNets; Bengio framework capable of generating diverse samples and requiring
al., 2021) have emerged fewer evaluations of the reward function compared to alternatives like Markov Chain Monte Carlo or Proximal Policy Optimization (Schulman al., 2017).
In this work, we introduce SynFlowNet, a GFlowNet specifically trained to generate molecules from documented chemical reactions and purchasable starting materials, thereby constraining exploration to a synthetically accessible chemical space and sampling not only target compounds but also the synthetic routes leading to them. Our main contributions
We train a GFlowNet using an action space defined by documented chemical reactions and purchasable starting materials generate synthesizable molecules.
We show the advantage of using reaction-based environment over fragment- one in terms of synthesisability, and the benefits of GFlowNets over Reinforcement Learning (RL) in terms of sample diversity several targets. Additionally, when comparing to other generative methods with synthetically-aceest generates more novel candidates w.r.t known drug-like molecules.
outputs, we show that SynFlowNet
2

---

Published as a conference paper at ICLR 2025
We evaluate different action representation alternatives to allow efficient scaling of the action space to up to 200k chemical building blocks.
We identify inherent problem of employing reaction- based Markov Decision Process (MDP) with GFlowNets, stemming from the lack of guarantee that beckkwrrd-cossrrucee trajectories can return the initial state 8g- To resolve this issue, propose one of the first attempts at training the backward policy in a GelowNet with a separate objective from the forward policy, which show correct backward-gent wawwdddeeeerttedd trajectories in the MDP while retaining the ability to discover diverse and high-reward modes.
We show that our proposed framework can be integrated with target-specific experimental data to inform selection of building blocks, which improves the efficiency of our model.
2 BACKGROUND AND RELATED WORK
2.1 GFLOWNRTS
GFlowNets (Bengio al., 2021) are a class of probabilistic models that learn a stochastic policy to generate objects through sequence of actions, with probability proportional to a reward R(xr). The sequential construction of objects x can be described as a trajectory E in a directed acyelic graph (DAG) G' = (S, E), starting from an initial state so and using actions to transition from a state to the next: s + s'. A GFlowNet uses a forward policy (=(-)s), which is a distribution over the children of state s, sample sequence of actions based the current states. Similarly, backward policy (1s) is the distribution over the parents of state s, and can be used to calculate probabilities of backward actions, leading from terminal initial states. The training objective which we adopt in this paper is trajectory balance (Malkin et al., 2022):
Zo IT Pr( (c;
(T) log
(1)
R(x:) IT ( 18;;
used to learn the forward and backward policies Pp(-1s; 0) and PB(-Is; 0) parameterized by 0 and to estimate the partition function Zg RJ F(50) = = F(T).
Bengio et al. (2021) have used GFlowNets to generate molecules with high binding affinity to protein target by linking fragments to form a junction tree (Jin et al., 2019). For generative chemistry, the framework was extended to multi-objective optimisation (Jain et al., 2023; Roy et al., 2023), where the model was trained to simultaneously optimise for binding affinity to the protein target, Synthetic Accessibility (SA), drug likeness (QED) and molecular weight.
2.2 GELOWNETS WITH PARAMETERIZED BACKWARD POLICY
GFlowNets train a forward policy Pr to match the backward policy Pn according to Eq. 1. The choice of PA therefore impacts the training of GelowNets and sample quality. Despite this relationship, the choice of backward policy in GFlowNets has attracted limited attention but for few works discussed here. Malkin et al. (2022) proposed parameterizing the backward policy and training PB and PF simultancously using the trajectory balance objective (Eq. 1). They also proposed fixing the backward policy to uniform distribution when modeling the distribution over parents states proves difficult. Mohammadpour et al. (2024) compare maximum-entropy GFlowNet GelowNets with uniform backward policy. Closer to our work, Jang et al. (2024) propose Pessimistic GelowNets, which use maximum likelihood over observed trajectories train PB. This ensures that the flow induced by the backward policy is concentrated around observed (training) states, making the model pessimistic about unobserved intermediate states having flow. In this work, we address the idea of ensuring that backward backward-generuted generated trajectories belong to the GFlowNet MDP. In doing so, we propose a data-driven solution to a MDP design challenge.
2.3 GENERATIVE MODELS FOR MOLECULE DESIGN
A large number of works have been proposed for generative molecular design (Du et al., 2024). Early methods employed techniques such as variational autoencoders (VAEs) 2018), deep reinforcement learning (Segler al., 2017; Olivecrona al., 2017b), and generative
et al.,
3

---

Published as a conference paper at ICLR 2025
adversarial networks (GANs) (Guimaraes al., 2018; Cao & Kipf, 2022). (Zang & Wang, 2020) Despite these advances, challenges remain, particularly in ensuring that generated molecules adhere to physical and chemical constraints (Stanley & Segler, 2023; Harris et al., 2023).
2.4 SYNTHESIS- MOLECULE GENERATION
The idea of tackling molecule generation and synthesis simultancously has been investigated early by Vinkers et al. (2003), who introduced SYNOPSIS, which generates molecules from a starting dataset of available compounds, relying applying chemical modifications to functional groups and assessing the value of the product with a fitness function. The works of Bradshaw et al. (2019) and Korovina et al. (2020) provided early neural models for one-step synthetic pathways. Bradshaw et al. (2020) generalized this idea as a generative model for synthesis DAGs, which can be optimized in VAE or RL setup. Gottipati et al. (2020b) used reinforcement learning generate compounds from reactions and commercially available reactants. Gao et al. (2021) formulate an MDP to model the generation of synthesis trees, which can be optimized with respect to the desired properties of a product molecule. Luo et al. (2024) proposed model that can project unsynthesizable molecules from existing generative models to synthesizable chemical space by utilizing postfix notations to represent synthesis pathways. Guo & Schwaller (2024) showed that retrosynthesis models can be treated as an oracle in goal-directed molecule generation. Concurrently to our work, Koziarski et al. (2024) propose a similar GFlowNet-basse framework for synthesizable molecular generation but using a different set of reaction templates and making use of a different backward policy.
While our work resembles an RL setting for synthesis-aware molecular generation (Gottipati et al., 2020b; Horwood & Noutahi, 2020), the key difference lies in the sampling distribution of the learned model. Contrary RI., the GFlowNet objective is not generate the single highest-return sequence of actions, but rather to maximise both performance and diversity by sampling terminal states proportionally their reward. This especially useful in the context of molecule generation, where we want to explore different modes of the distribution of interest.
METHODS
In this work, we present framework train GFlowNets on a Markov Decision Process (MDP) made of molecules obtained from sequences of chemical reactions. Below, we describe how this compositional space of synthesizable molecules assembled (3.1), we present a method making use of backward policies to palliate imperfect information contained in reaction templates (3.2) and present the model used to navigate that state space and learn the target distribution (3.3).
3.1
MDP OF CHEMICALLY ACCESSIBLE SPACE
Problem definition We model synthetic pathways as trajectories in a GFlowNet, starting from purchasable compounds and ending with molecules that are optimized for some desired properties, via a set of permissible reaction templates. At each timestep I, the state & represents the current molecule and stepping forward in the environment consists in building up the molecule by applying new pairs of reactions and reactants until either a termination action is chosen or the path reaches a maximum length. We encode reaction templates using RDKit reaction SMARTS (see Figure A.1).
Forward Action Space We define five types of forward actions: Stop, AddF Reacttaann. React Uni, ReactBi, and AddReact ant. ReactUni and ReactBi represent uni uni-moleuular and bi-molecular reactions. The AddRea atant action, which is available only after Rea ci action, represents the choice of reactant for the bi-molecular reaction. In more detail, each trajec- tory starts from an empty molecular graph which is followed by a building block sampled from AddF i restReactand action type is Stop, we reach a terminal state and end the trajectory; (b) if ReactUni action is sampled, apply the uni-molecular reaction template to the molecule in state to yield the product in state s': (c) if the action type is ReactBi. the sampled reaction used as input to an additional ML., together with the state embedding, to sample subsequent action of type AddRea ctant.
We then continue based on the sampled action type as follows: (a) if the
4

---

Published as a conference paper at ICLR 2025
Backward Action Space The GFlowNet framework requires us to model travelling backward along trajectories in the state-space. To unfold a reverse trajectory we proceed similarly: (a) if the action-type is a BcKReactUni, the action yields the reactant molecule directly; (b) if the action type is BckReactBi, we obtain two reactants, and the molecule that is not a building block becomes the next state (or previous state in the DAG). If the two resulting reactants are both building blocks (which happens at the beginning of the forward trajectory), the molecule that populates the next state picked with p = 1/2 from the two building blocks. The last action in backward trajectory is Bc BckRemoveF irst warssteaactant leading to the empty molecular graph So (initial state).
Masking Prior to sampling actions in both forward and backward direction, we ensure that the reactions and building blocks to be sampled are compatible with the current state using masks, obtained by checking for substructure match between the reaction template and the reactant (for forward actions) or product (for backward actions) molecules. We also enforce through masking that at least one of the resulting reactants when running backward reaction is building block.
3.2 CHALLENGES WITH GENERATING BACKWARD PATHS
Given the synthesis pointed DAG G = (A), one needs to define a forward probability function Pe and backward probability function Py both consistent with G (see Eq. 1). Contrary to previous fragment-based or atom-based action can lead to so (removing nodes and edges sequentially will lead to an empty graph; Bengio et al., 2021), I) defining Pn in reaction-basee environment is non-trivial. This is because not every parent state (obtained by applying reaction template backwards) will ensure that there exists sequence of actions that leads all the way back to building block, and therefore So. Note that the masking described in Section 3.1 is insufficient account for this, as it does not ensure that the state obtained is further decomposable into building blocks. Consequently, to maintain a pointed DAG, no flow should be assigned such transition. A uniform backward policy, which is a standard choice in GFlowNet literature (Malkin et al., 2022), will fail at achieving this it will assign positive flow to every backward action, including those leading to states that are not attainable from forward trajectories initialised in su (see Figure 1-C). To address this issue, explore few training schemes for a parameterized Pg that force
polecule-generattie
GFlowNet environments, where any backward
trajectories to end in Sg.
Training the backward policy We first explore training scheme for Py which makes of forward-gent -ewardd-enerrted trajectories. Similarly to Jang et al. (2024), we train P3 using the maximum likelihood objective over trajectories generated from Pf:
(0n) = E-- log (T(T;On))
In that setting, we (1) generate trajectories using Pr. (2) update Pr according to the trajectory balance objective in Equation I and (3) update PB using these same trajectories according to Equation 2 (see
Algorithm in Appendix A.4).
While the maximum-likelihon approach presented above sufficient to limit PA in allocating flow to paths that do not connect back to su. it restricts the model's exploration by encouraging Pe to collapse on a single path for each terminal molecule. To allow Pg to ban erroneous paths while retaining a higher entropy, also explore the use of policy gradient methods. Specifically, explore maximising an expected backwards reward via REINFORCE (Williams, 1992), which is suitable for short trajectory environments like our reaction-based MDP:
(B(BB) = ET [R[(T]] -
Here, H(Pp) -Ry-pg (log(Pn(7)) is the entropy term and the reward Rs is set to 1 for trajectory that ends 8o and -1 otherwise. In this setting we train the backward policy not only on the trajectories generated by the forward policy, but also on newly generated backward trajectories sampled directly from PB (see Algorithm 2, Appendix A.4).
Interestingly, training the backward policy navigate back 50 is analogous to the retrosynthesis problem (Corey, 1989; Segler et al., 2018). We employ this approach of training P3 against a different objective than Pp to palliate to design challenge of the MDP, but similar strategy could also be employed to fold additional preferences over different synthesis routes leading to the same terminal state into the system, for example to take into account the synthesis costs of particular path.
5

---

Published as a conference paper at ICLR 2025
3.3 SCALING OF THE REACTANT SPACE TO LARGE NUMBER OF BUILDING BLOCKS
Estimating the size of the state space We leverage the properties of GFlowNets (Bengio et al., 2021) estimate the size of the state space induced by our ac- tion space. Specifically, noting that GFlowNets learn
Estimated Size of State Space
10P1
SynflowNet
SynflowNet I-4
10
Eramine REAL FragGIN
log = log Ex R(z), we train model with R 1 for all terminal states to estimate their total count. We so for different numbers of building blocks and different max- imum trajectory lengths (L), and find that SynFlowNett using L 3 and 10k building blocks matches the size of the Enamine REAL space (Enamine), and that the size of the space quickly increases with the number of building blocks. We use our full set of 105 reactions. Note how caction-constrained models considerably limit the explo- ration of the chemical space, with a fragments GFlowNet exploring space ~ 10 orders of magnitude larger.
10-1
10
100
10.1
Fraction of Enamine Building Block Set
Figure 2: Estimated size of state spaces. The full building block set contains 221,181 molecules. is the maximum trajectory length in the GFlowNet.
Scaling method To ensure synthetic accessibility of all samples, our model is inherently constrained by the initial set of available building blocks (BBs). To cover a large chemical space, it is thus crucial to use extensive BB collection. The model must demonstrate scalability to accommodate larger sets of BBs, both in terms of training efficiency and overall performance. To do so, we follow Dulac-Arnold et al. (2015); Gottipati et al. (2020a) and change the representation of the BBs and their selection mechanism, as shown in Figure IB. Instead of the weight matrix of the mapping from hidden units logits associated with BBs be randomly initialized, is fixed be the matrix of binary Morgan fingerprints (Rogers & Hahn, 2010).
4 RESULTS
Our results support number of claims: (i) a reaction-basee MDP greatly improves the synthesisabilitty of generated molecules (Sec. 4.1). (ii) employing GFlowNets enables much more diverse molecule sampling over RL (Sec. 4.5), (iii) learning backward policy results in higher-reward molecules and enables finding retrosynthetic pathways for molecules belonging to our state-space (Sec. 4.4), and (iv) our method of sampling molecules based on chemical fingerprints allows for efficient scaling to large chemical spaces (Sec. 4.5). Finally, we show that this approach also allows to improve results for particular programs by curating the set of building blocks based on target-specific experimental data (Sec 4.6). Unless otherwise specified, all experiments below use a backward policy trained with maximum likelihood, and Morgan fingerprint embeddings for our action space, with a library of 10,000 Enamine building blocks and 105 reaction templates.
4.1 REACTION-BASER MDP DESIGN
SeH proxy
Diversity
SA
AiyythFinnder
67.0%
0.6
0.6-
0.4
0.0
0.0%
Synteett:: SerPoritt Pradens PaacOhs
Syefleyte:: SyrPoreet
PraCOF
Syetleyte:: SyrPloetet PueCON
Nucofm
SynPooee Svetlerke
Figure 3: Comparison across MDPs. We evaluate four GFlowNet models: SynFlowNet trained with an action space of chemical reactions and maximum trajectory lengths (L) of 3 and 4, FragGFN and FragGFN SA are trained with an action space of fragments, however the latter also optimises for synthetic accessibility (SA), besides the sEH binding proxy reward. SynFlowNet molecules are achieving higher binding scores and better synthesizability.
We compare GFlowNets using two action spaces- fragments vs. reactions and report reward, diversity and synthesizability of the generated samples. We retrain the GFlowNet model proposed by

---

Published as a conference paper at ICLR 2025
Bengio et al. (2021) with fragments derived from our building blocks set. This ensures that similar chemical spaces and that we get as close as possible to a fair comparison between the two MDPs. In the fragment space, it is common practice optimise for synthetic accessibility scores to improve synthesizability from the model. We therefore train two versions of the FragGFNN model: one using the sEH binding proxy as reward function, and another that optimizes for both sEH binding and synthetic accessibility (SA) score. SynFlowNet was only trained with the sEH proxy as reward.
The results in Fig. 3 show that our MDP design achieves great improvement in terms of synthesiz- ability, while preserving high rewards and diversity. While one might expect the more expressive FragGFN model to result in higher rewards, we hypothesize that the larger exploration space in the fragments environment (see Figure 2) hinders the model's 's efficiency. We see that within the reaction environment, MDP design choices can further influence sample quality: a maximum trajectory length of 3 achieves better SA scores and AiZynthFinder (Genheden et al., 2020) retrosynthesis success (62%) compared to a maximum trajectory length of (40%). Note that synthesisability metrics correlated with molecule size (Skoraczyííski et al., 2023), and that molecules assembled from longer synthesis routes are naturally larger (see Table A.3). Both FragGFN models score 0% with AiZynthFinder. Diversity, measured as average Tanimoto distances between molecular fingerprints, is preserved from a fragments to a reaction environment, albeit less so for shorter trajectories.
4.2
GELOWNETS AS SAMPLERS OF CHEMICAL SPACE
SEH
GSK33
0.8
0.95
0.9
Reward
0.90
0.7
0.8
0.85
0.6
0.7
0.80
0.5
# unique Murcke scaffolds
SynFlowNee
Solt Q-Learning
SynFlowNet
Soft Q Q-Leerning
# unique scaffolds
SynFlowNet
Soft Q-Learnnng
10a
10
10*
with 2,8
with 0.6
102-
10a
10'
100
SynFlowNet
10a
SynFlowWiet.
10a
SynFlowNet
Learning
Learning
Q Learning
of eva luateded molecules per thread
# of evaluated meleculesle4 per thread
evaluated molecules per thread
Figure 4: Comparison between GFlowNet and RL. SynFlowNet (GFlowNet with synthesis actions) discovers more modes compared to entropy regularree RL trained on the same state action space.
While the works of Gottipati al. (2020b) and Horwood & Noutahi (2020) are closest to our method, where RL is paired with a synthesis action space, their code is not publicly available. We reproduce their setting by pairing an RL algorithm with our MDP. We train our model with a soft Q-learning (Haarnoja et al., 2017) objective and compare to SynFlowNet on three different reward functions: sEH, GSK3, and DRD2 (see Appendix A.
In Fig. 4, we show the reward distribution of generated samples after equal numbers of training steps. We notice that soft Q-learning collapses sampling narrower distribution of high-reward molecules for all targets. This is also shown in the number of Bemis-Murcko scaffolds counted for molecules with rewards above a certain threshold, where soft Q-learning quickly collapses.
4.3 FURTHER COMPARISON TO BASELINES
We continue by comparing SynFlowNet to strong baselines from the literature. First, REINVENT uses policy-gradient method to tune an RNN pre-trained to generate SMILES strings (Blaschke et al., 2020; Loeffler et al., 2024). REINVENT has been shown to outperform many models in terms of sample efficiency (Gao et al., 2022a) and proposing realistic 3D molecules upon docking (Cieplinski et al., 2023). Second, SyntheMol shares similar MDP design with SynFlowNet, but is a search-based method using Monte Carlo Tree Search (Swanson al., 2024). Contrary to our model,
7

---

Published as a conference paper at ICLR 2025
SeH proxy 1
Diversity
Reward/Novelty trade-off
1.
1.0
1.0
nowevt
Sirianity CIENEL
0.8
Santnalll
Vsea
0.6
0.8
8.8
0.4
0.4
0.0
Syntiowfeet REINVENT Synthello!
Synflowhet RERAVNT SymtheNol
Synilomkeer REINENN SyrtheMol
0.0
0.1 0.0
Roward
Figure 5: SynFlowNet is competitive against other popular models from the literature. Syn- FlowNet achieves good balance between reward optimisation, diversity, synthesizability and novelty (assessed by maximum similarity to ChEMBL molecules), REINVENT stays close to its pretraining distribution, harming the novelty of the proposed molecules. SynFlowNet is closest to ideal.
synthesis pathways in SyntheMol contain one reaction, and its estimated state space has 30 billion molecules, reached from the Enamine building block set and 13 reactions (Swanson et al., 2024).
The comparison is summarized in Figure 5. SynFlowNet achieves comparable rewards REINVENT. and better sample diversity. Although REINVENT does not use explicit knowledge of synthesizabilityy in its generation process, implicitly optimizes for by being likelihood model overfit on curated ChEMBL dataset (Zdrazil et al., 2023). This is reflected in our last sub-plot, where we look at the maximum similarity to ChEMBL molecules, as proxy for novelty. This proxy is strong enough, as ChEMBL contains one of the largest collection of diverse drug-like molecules to date (Zdrazil et al., 2023). While REINVENT seems to be competitive in terms of synthesizability and reward, generates molecules with poor novelty, staying close to its pretraining distribution. Notably, SynFlowNet achieves high novelty for high-scoring molecules, even when trained using ChEMBL.. derived building blocks (see App. Fig. A.5). Overall, SyntheMol scores comparably to SynFlowNet, however SynFlowNet achieves a better reward/diversity trade-off and SynFlowNet's capabilities extended to explore larger state spaces (see Fig. 2). We repeat the comparison on other targets, and again find that SynFlowNet outperforms SyntheMol (see App. B. Further experiments showcasing SynFlowNet's sample efficiency and performance when optimizing for direct docking score calculations, are discussed in Appendix B. We again discover that SynFlowNet is competitive to other models and that improves sample efficiency over the fragments GFlowNet.
4.3.1 REDISCOVERY TASKS
We investigate whether SynFlowNet can discover known synthesizable actives by training it with rediscovery rewards (Huang al., 2021). The results are shown in Table where report top-k similarity and SA scores. When running the SpaceLight software (BioSolvelt, 2024) to determine whether the molecules are present in the REAL space (i.e. reachable through our set of building blocks), we found that aripiprazole is present and that celecoxib is not, which explains the two rates.
Table 1: SynFlowNet performance rediscovery tasks. We report mean and standard deviation of top-10 10 discovered molecules from running three models with different seeds.
Task
Reward (1)
SA (L)
Aripiprazolc rediscovery
0.90 + 0.00
2.19 + 0.00
Celecoxib rediscovery
0.48 + 0.01
2.47 + 0.04
4.4 IMPROVED MDP CONSISTENCY THROUGH TRAINED BACKWARD POLICY
We evaluate the effectiveness of our proposed training schemes for the backward policy P8 by measuring (1) whether Py can ensure that backwardd-oonstrueted constructed trajectories reliably find a trajectory back to the initial state 8p and (2) whether brings any benefit to the forward policy. In Table 2 we compare a fixed, uniform backward policy to three versions of parameterized backward policy: free policy, updated w.r.t. the trajectory balance loss, (see Eq. 1), a policy trained with maximum likelihood on the forward-generated trajectories and policy that is allowed to explore the backward action space and trained with REINFORCE to find paths leading back to 8y. We observe that both the maximum likelihood and REINFORCE policies succeed in ensuring that backward flow is not lost outside of the MDP, as they manage to construct trajectories that start from terminal states sampled from Pr all the way so- We refer to such routes solved routes (Irain) in Table 2, as the terminal
8

---

Published as a conference paper at ICLR 2025
states have been visited by the GFlowNet during training. We also test the ability of the trained policies to retrieve synthesis routes for molecules which have not been visited during training (test). These were obtained from a random sampler using set of reactions and building blocks. While we see small differences in the number of high-reward modes discovered for different backward policies, see that free policy consistently fails to be competitive with the rest. More ablation studies in App. A.6 show that an retropy-regularized REINFORCE policy excels at discovering high reward modes. Overall, the maximum likelihood and REINFORCE policies prove effective in guiding Pp to high reward modes, while ensuring MDP consistency, which is crucial for sampling from the Boltzmann distribution defined by the reward function (Bengio et al., 2023).
PB policy
of solved routes (train)
% of solved routes (test)
of high reward modes from Pr
Uniform Free
11.0 3.7%
11.0 + 4.1%
47, + 11, 261.7
67.3 - 3.7%
1.0 = 0.8%
6, 754.3 + 4, 9990. 37, 708.2 + 13, 992.6 55, 387 6 + 28, 886.3
MaxLikelihood
99.3 = 0.5%
32.3 + 7.3%
REINFORCE
100.0 = 0.0%
44.3 + 2.6%
Table 2: Effect of different training paradigms for PB. Training the GFN PB ensures that beckward-consrructed trajectories belong to our MDP and can marginally improve Pr. % of solved routes refers to the ability of Pn to retrieve synthesis routes for on-policy (train) and off-policy (test) molecules reachable through our state space. Test molecules have not been seen during training. # of high reward (R >> 0.9) modes from Pp is reported out of ~ 500, 000 samples seen during training.
4.5
SCALING THE ENTIRE ENAMINE SET
A
Sampled reward
B
Utilisation of building blocks C
Runtime
0.80
wib Enguptrrs
30
8 12500
Engepents
W Prrgeeeets (ochilammeci)
Auwrage reward
0.75
Loed
10000
wiEngepenti 1- 2.
Whltrgepent nocel Joward TE31
7500
0.70
5000
0.65
Sugeraliis
2500
0.2
0.4
0.6
0.8
1.0
0.25
0.50
0.75
1.00
0.2
0.4
0.6
0.8
1.0
Fraction of the "ull 88 set
Froction of the full 8B set
Fraction of the full BB
Set
D
Training efficiency
E
Cluster exploration
F
Cluster exploitation
5000
Engrepions
whb tngerepents
0.8
urique
cluptrr uied
4000
AUS or lifls
10000 pur clusnar
Booser
2000
2000
0.4
w/l Engepeitt Regageenn
3000
0
2000
4000
6000
8000
2000
4000
6000 8000
2000
4000 6000
8000
mingg stop
Training stop
Training stop
Figure 6: Ablation over action-selection mechanism. Average reward of samples (A), utilization of building blocks (B), and training times (C) for both models on different subsets of building blocks. Embeddings enable more efficient learning (D), they help the agent navigate the set of available building blocks and tend to exploit more relevant BB clusters (F) rather than explore new clusters (E).
For each experiment, three models trained with different random seeds were used.
In this section, we study the ability of SynFlowNet scale to large sets of building blocks using the method proposed in Section 3.3 as opposed to using a learnable embedding for each building block. First, we compare the performance of our models using the softmax action-selectio the fingeprints--asee based softmax on the full Enamine set of 221, 181 building blocks and randomly sampled subsets containing 75%, 50% 25%, 10%, and 3% the full set. As shown in Figure 6A, increasing the number of building blocks while using the same number of training steps negatively affects the reward for the model without fingerprints. However, the model with fingerprints consis- tently outperforms the former and makes the reward degradation effect almost negligible. Next, study whether models are able to use the blocks they are given. As shown in Figures 6B, the model with fingerprints consistently uses more unique building blocks than the baseline model. Notably, the
mechanism and
9

---

Published as a conference paper at ICLR 2025
baseline model is only able to small subset of building blocks produce high-quality samples (R > 0.8). contrast, fingerprints enable the use of 5-fold larger set of blocks for the same quality level, thus significantly surpassing the baseline samples in terms of diversity. Finally, measure the average time of the forward pass and average time of forward pass with auxiliary RDKit operations like template matching in Figure 6C.
As shown in Figure 6D, fingerprints enable more efficient learning on a large building block set. Indeed, instead of memorising all available BBs, the model learns navigate in the space of embeddings and to maximise dot product with the relevant building blocks. To illustrate this, we clustered all available building blocks based on Tanimoto similarity. Next, we studied how both models explore new building blocks within and across these clusters. As shown in Figures 6E-F, the baseline model tends explore more clusters while discovering low-reward molecules, thus becoming less efficient. The model with fingerprints, on the contrary, is able to focus on clusters that maximise the reward, leveraging the structure of the chemical space embedded in fingerprints.
4.6 GUIDING SYNFLOWNET WITH EXPERIMENTAL FRAGMENT SCREENS
A
B
Tarpet
Frugrant ScHan
Exporieetally va kd0td Fognons
Wildition
Designed
0.55
6.70
Pandam
at a-rap
0:00
cryda Bagappy
High dimllortty
Enamine
Syn FlowNet
0.46
+ Docking
0.10
Bukding tlock
Targel speatfo building block oot
Shep
Figure 7: Effect of curating the building block set. (A) We adapt SynFlowNet's building block library based on experimentally validated fragments for a given target. (B) The curated set improves Mpro rewards over random building blocks.
One advantage of our framework is that building block sets can be used to specialize the model for a particular target. Fragment-based Drug Discovery (FBDD) (Thomas et al., 2019) is major strategy to increase the efficiency of drug discovery campaigns. FBDD, instead of screening large chemical spaces, relatively low number of small compounds (called fragments) experimentally validated to bind, and are then linked or merged in silico to make full molecules.
screened and
We hypothesise that experimental data from x-ray fragment screens (Murray & Rees, 2009) guide and enhance SynFlowNet's capabilities. As a proof-offconcepe we focus on the SARS-CoV-2 main protease (Mpro), leveraging strueural data from the COVID Moonshot project (Boby et al., 2023). We compile a building block set from Enamine with high similarity to fragments confirmed by x-ray crystallography (see Fig. 7A). We find that biasing the building block library towards fragments with known protein-ligany complementarity increases the reward over randomly selected molecules (see Fig. 7B). Further details on methods and results are given in App. Section B.4.
5 CONCLUSION
In this work, discuss the application of GFlowNets de novo molecular design paired with forward synthesis. We demonstrate that an action space of chemical reactions an effective way of enforcing synthesisability, and that pairing it with GFlowNets excels in terms of diversity. Comparisons to state-of-the-att baselines emphasised that SynFlowNet explores novel regions of the chemical space. We also proposed novel paradigm for training the backward policy in the GFlowNet and in doing so we improved and validated the correctness of our MDP design. Furthermore, we studied the building block exploration and exploitation mechanisms of SynFlowNet, showing efficient scaling to using hundreds of thousands of building blocks. Finally, as proof of concept for the adaptability of SynFlowNed to real drug discovery programs, we showed that the framework can specialised to target-specift molecule generation by making use of experimental fragment screens.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We thank Enamine for providing the building blocks and for useful conversations. We thank Daniel Cutting, Austin Tripp and Jeff Guo for discussions regarding baselines. We thank Doo-Hyun Kwon (Recursion) for running the SpaceLight software. Miruna Cretu is supported by the SynTech CDT at the University of Cambridge and part of the work was completed during an internship at Valence Labs.
REFERENCES
A Alhossary, SD Handoko, Y Mu, and CK Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):2214-222 2211,,, 2015. doi: 10.1033iionfommiis/br000...
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. arXiv preprint arXiv:2106.04399 2021.
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward Hu, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. Journal of Machine Learning Research, 24(100)1-55, 2023.
BioSolvelt. SightLight. chemical-spaces/ 2024.
h/wp:./wwswevttd//epolgggheegepp -hw-ppptlliiitt------a--llooooo hhap -uutter-or-
Thomas Blaschke, Josep Arús-Pous, Hongming Chen, Christian Margreitter, Christian Tyrchan, Ola Engkvist, Kostas Papadopoulos, and Atanas Patronov. REINVENT 2.0: An ai tool for de novo drug design. Journal of Chemical Information and Modeling, 60(12):5918-59) 2020. doi: htt2l//cc.cci..oo/ll
Melissa L Boby, Daren Fearon, Matteo Ferla, Mihajlo Filep, Lizbé Koekemoer, Matthew C Robinson, COVID Moonshot Consortium John D Chodera, Alpha A Lee, Nir London, et al. Open science discovery of potent noncovalent BARS-CoV.L main protease inhibitors. Science, 382(6671)::
eabo7201, 2023.
John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, and José Miguel Hernández- Lobato. A model to search for synthesizable molecules. CoRR, abs/1900.005 05022, 2019.
John Bradshaw, Brooks Paige, Matt Kusner, Marwin Segler, and José Miguel Hernández-Lobato. Barking up the right tree: approach to search over molecule synthesis dags. Advances in neural information processing systems, 33:6852-68666 2020.
Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In International Conference on Learning Representations, 2018.
Martin Buttenschoen, Garrett M Morris, and Charlotte M Deane. Posebusters: Al-based docking methods fail to generate physically valid poses or generalise to novel sequences. Chemical Science, 15(9):3130-3139, 2024.
Alexander Button, Daniel Merk, Jan A Hiss, and Schneider Gisbert. Automated de molecular design by hybrid machine intelligence and rule-driven chemical synthesis. Nature Machine Intelligence, 1(7):307315,, 2019.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. 1805. /9000///93.... 1973, 2022.
arXiv preprint arXiv:
Tobiasz Ciepliáski, Tomasz Danel, Sabina Podlewska, and Stanistaw Jastrzbski. Generative models should at least be able to design molecules that dock well: A new benchmark. Journal of Chemical Information and Modeling, 63(11):3238-3247, 2023. doi: 10. 10.100//as.jiim.2200
Connor W. Coley, Luke Rogers, William H. Green, and Klavs F. Jensen. Sescore: Synthetic complexity learned from a reaction corpus. Journal of Chemical Information and Modeling, 58(2): 252 261, 2018. doi: 10. 10.102/aacs.jimm./
Cheng X.-M. Corey, E. J. The Logic of Chemical Synthesis. New York: John Wiley and Sons, 1989.
11

---

Published as a conference paper at ICLR 2025
Yuanqi Du, Arian R Jamasb, Jeff Guo, Tianfan Fu, Charles Harris, Yingheng Wang, Chenru Duan, Pietro Liò, Philippe Schwaller, and Tom Blundell. Machine learning- aided generative molecular design. Nature Machine Intelligence, pp. 1-16, 2024.
Gabriel Dulac-Aroodd, Richard Evans, Hado Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin. Deep reinforcement learning in large discrete action spaces. arXiv preprini 1512.0:511207777 2015.
Enamine. Enamine.
Accessed: 28.1 1.2023.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics, (1):1-11, 2009.
Wenhao Gao and Connor W. Coley. The synthesizability of molecules proposed by generative models. Journal of Chemical Information and Modeling, 60(12):5714-577) 5723, 2020. doi: 10.1021/acx.jcim. Oc00174.
Wenhao Gao, Rocío Mercado, and Connor W. Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. CoRR, abs/2110.06389, 2021.
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor W. Coley. Sample efficiency matters: A benchmark for practical molecular optimization, 2022a. URL https::/arxii..orgg abs/ 2206. 12411.
Wenhao Gao, Rocío Mercado, and Connor W. Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design, 2022b. Rel.htttss:/arriv..oo abs/ 2210/2110.0 06389.
S. Genheden, A. Thakkar, V. Chadimová, J. L. Reymond, O Engkvist, and Esben Bjerrum. Aizyn- thfinder: a fast, robust and flexible open-source software for retrosynthetic planning. J Cheminform, 12:70, 2020. doi: 10. 032101133310000000777
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 1263-1272, 2017.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In International conference on machine learning. pp. 3668 3668-3679. PMLR, 2020a.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, Sarath Chandar, and Yoshua Bengio. Learning navigate the synthetically accessible chemical space using reinforcement learning. In Proceedings of the 37th International Conference Machine Learning, volume 1 119, pp. 3668-3679, 2020b.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik, Objective-reinforceed generative adversarial networks (organ) for sequence generation models. arXiv preprint
2018.
Jeff Guo and Philippe Schwaller: Directly optimizing for synthesizability in generative molecular design using retrosynthesis models, 2024. URL https: /atp://///aivv.i/aae
Anvita Gupta, Alex T Müller, Berend JH Huisman, Jens A Fuchs, Petra Schneider, and Gisbert Schneider. Generative recurrent networks for de novo drug design. Molecular informatics, 37(12)); 1700111, 2018.
Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Ben- jamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera- Iparrauuip, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven continuous representation of molecules. ACS Central Science, 4(2):268-22 2766. 2018. doi: 10. 10.1021/acscennsci./b00000
12

---

Published as a conference paper at ICLR 2025
Tuomas Haarmoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies, 2017. URL https::tttaaatt httppp////aa.llooo////////////// 0816..
Charles Harris, Kieran Didi, Arian Jamasb, Chaitanya Joshi, Simon Mathis, Pietro Lio, and Tom Blundell. Posecheck: Generative models for 3d structure- drug design produce unrealistic poses. In NeurIPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Walter M Rupp M Reisen Proschak E Weggen S Stark H Schneider G. Hartenfeller M, Zettl H. DOGS: reaction-driven de novo design of bioactive. PLOS Computational Biology, 8(2), 2012. doi: 10.137 (..1371/jourmal.po pcbi. 1002380.
Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS Omega, 5(51):32984 -2993994 2020. doi: 10.1021/acsomeg 0c04153.
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W. Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development, 2021. URL https: /ttp:///xxxivoo/oo/ abs/ 22ss/2102 09548.
John D. Imig and Bruce D. Hammock. Soluble epoxide hydrolase as a therapeutic target for cardiovascular diseases. Nat Rev Drug Discov, 8(2):794-805, 2009. doi: 10. 10.1038/mrd2875.
Moksh Jain, Sharath Chandra Raparthy, Alex Hermandez-Garcia, Jarrid Rector-Brooks. Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective GFlowNets. arXiv preprint arXiv:2210. 12765, 2023.
Hyosoon Jang, Yunhui Jang, Minsu Kim, Jinkyoo Park, and Sungsoo Ahn. Pessimistic backward policy for gflownets. arXiv preprint urXiv:2405.16012. 2024.
Jan H Jensen. A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration chemical space. Chemical science, 10(12):3567-3577 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pp. 2323-2332. PMLR, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction Tree Variational Autoencoder for Molecular Graph Generation. arXiv preprint arXiv:/802.04364, 2019.
Vicky Jung, Kenneth Lopez Perez, Lexin Chen, Kate Huddleston, and Ramon Alain Miranda Quintana. Efficient clustering of large molecular libraries. bioRxiv, pp. 2024-08, 2024.
Dongsung Kim, Lorenz Herdeis, and Dorothea et al. Rudolph. Pan-kras inhibitor disables oncogenic signalling and tumour growth. Nature, 619(79668:160-1166 2023. doi: 10.1038/ $411886-033-06123-3.
Kathryn Klarich, Brian Goldman, Trevor Kramer, Patrick Riley, and W. Patrick Walters. Thompson sampling-an efficient method for searching ultralarge synthesis demand databases. Journal of Chemical Information and Modeling, 64(4):1158-1171, 2024. doi: 10. 10.1021/acs,pp jcmll//csccim..c111990
Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric Xing. Chembo: Bayesian optimization of small organic molecules with synthesizable recommendations. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108, pp- 3393-3403, 2020.
Michat Koziarski, Andrei Rekesh, Dmytro Shevchuk, Almer van der Sloot, Piotr Gaiński, Yoshua Bengio, Cheng-Hao Liu, Mike Tyers, and Robert Batey. RGFN: Synthesizable molccular generation using GelowNets. arXiv preprint arXiv:2406. 0700660506. 2024.
Richard A Lewis and PM Dean. Automated site-directed drug design: the concept of spacer skeletons for primary structure generation. Proceedings of the Royal Society of London. B. Biological Sciences, 236( (36(1280 183): 1989.
13

---

Published as a conference paper at ICLR 2025
Yibo Li, Liangren Zhang, and Zhenming Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10:1-24, 2018.
Cheng-Hao Liu, Maksym Korablyov, Stanislaw Jastrzebski, Pawel Wlodarczyk-Pruusynnski Yoshua Bengio, and Marwin Segler. RetroGNN: fast estimation synthesizability for virtual screening and de novo design by learning from slow retrosynthesis software. Journal of Chemical Information and Modeling, 62(10))2293-2300 2022.
Hannes H. Loeffler, Jiazhen He, Alessandro Tibo, Jon Paul Janet, Alexey Voronov, Lewis H. Mervin, and Ola Engkvist. Reinvent 4: Modern Al-driven generative molecule design. Journal of Cheminformatics, 16, 2024.
Shitong Luo, Wenhao Gao, Zuofan Wu, Jian Peng, Connor W. Colcy, and Jianzhu Ma. Projecting molecules into synthesizable chemical spaces, 2024. URL 04628.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment gflownets. CoRR, abs/2201. 13259, 2022.
AD Mesecar al. A taxonomically- driven approach to development of potent, broad-spect broad-spectrum inhibitors of coronavirus main protease including SARS-CoV- (COVID-19). Be Publ, 10, 2020.
Sobhan Mohammadpour, Emmanuel Bengio, Emma Frejinger, and Pierre-Luc Bacon. Maximum entropy GFlowNets with soft Q-learning, 2024. URL https: 14331.
Christopher W Murray and David C Rees. The rise of fragment- drug discovery. Nature chemistry, 13)):187-12, 2009.
Christopher W Murray, Maria G Carr, Owen Callaghan, Gianni Chessari, Miles Congreve, Suzanna Cowan, Joseph E Coyle, Robert Downham, Eva Figueroa, Martyn Frederickson, et al. Fragment- based drug discovery applied to Hsp90. Discovery of two lead series with high ligand efficiency. Journal of medicinal chemistry, 53(16):5942-598 59555, 2010.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular do-novo design through deep reinforcement learning, Journal of cheminformaties, 9:1-14, 2017a.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics, 9(48), 2017b. doi: 10.11866413321--17-0223
David Rogers and Mathew Hahn. Extended-comecctivve fingerprints. Journal of Chemical Information and Modeling, 50(5):742-754, 2010. doi: 10.1021/ci100050. URL https:
//doi. orooi.oogg10... 1et1/ PMID: 20426451.
Julien Roy, Pierre-Luc Bacon, Christopher Pal, and Emmanuel Bengio. Goal-conditioned GFlowNets for controllable multi-objective molecular design. arXiv preprint arXiv:2306.0400 2023.
Arman A. Sadybekov, Anastasiia V. Sadybekov, Yongfeng Liu, Christos Sappoulos-Tsoussouuue Xi-Ping Huang, Julie Pickett, Blake Houser, Nilkanth Patel, Ngan K. Tran, Fei Tong, Nikolai Zvonok, Manish K. Jain, Olena Savych, Dmytro S. Radchenko, Spyros P. Nikas, Nicos A. Petasis, Yurii S. Moroz, Bryan L. Roth, Alexandros Makriyannis, and Vsevolod Katritch. Synthon-based ligand discovery in virtual libraries of over 11 billion compounds. Nature, 601(7893):452-49) 2022. doi: 10. 1038/44 586-021-042200
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 2017.
Philippe Schwaller, Riccardo Petraglia, Valerio Zullo, Vishmu H. Nair, Rico Andreas Haeuselmann, Riccardo Pisoni, Costas Bekas, Anna Iuliano, and Teodoro Laino. Predicting retrosynthetic pathways using transformer- based models and hyper-grapp exploration strategy. Chem. Sci., 11: 3316-3325, 2020. doi: 10. 00.139/99S005704H.
14

---

Published as a conference paper at ICLR 2025
Marwin H. S. Segler, Thierry Kogej, Christian Tyrchan, and Mark P. Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Science, 4(1): 120-131, 2017. doi: 10. 10.0002//cscentsccc.00055533
Marwin HS Segler, Mike Preuss, and Mark Waller. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604- 610, 2018.
Grzegorz Skoraczyáski, Mateusz Kitlas, Blažej Miasojedow, and Anna Gambin. Critical assessment of synthetic accessibility scores in computer-assistd synthesis planning. Journal of Cheminfor- matics, 15, 2023.
Megan Stanley and Marwin Segler. Fake it until you make it? Generative de novo design and virtual screening of synthesizable molecules. Current Opinion in Structural Biology, 82:102658, 2023.
Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(1 (5(11)0006-1108 2017.
Kyle Swanson, Gary Liu, Denise B Catacutan, Autumn Arnold, James Zou, and Jonathan M Stokes. Generative ai for designing and validating easily synthesizable and structurally novel antibiotics. Nature Machine Intelligence, 6(3):338-35 3003333--33, 2024.
Shidi Tang, Ji Ding, Xiangyu Zhu, Zheng Wang, Haitao Zhao, and Jiansheng Wu. Vina-gpu 2.1: to- wards further optimizing docking speed and precision of autodock vina and its derivatives. bioRxiv, 2023. doi: 10.1101/2023.11 1.64.665429. URL https: //wps:://wwww./ http///wwwwww....o//////////tttt biorxiv. wwwrriii...//.e.eeeeoe.eeeooeeee wwtp://tggg/////////o//o///////t content/ early/2023/ /11222311/// 0001000//0000 11. 04.00..5555.. 565429.
Sherine E Thomas, Patrick Collins, Rory Hennell James, Vitor Mendes, Sitthivut Charoensut- thivarakul, Chris Radoux, Chris Abell, Anthony G Coyne, R Andres Floto, Frank Von Delft, et al. Structure-guided fragment-based drug discovery at the synchrotron: screening binding sites and correlations with hotspot mapping. Philosophical Transactions of the Royal Society A, 377(2147): 20180422, 2019.
Oleg Trott and Arthur J. Olson. Autodock Vina: improving the speed and accuracy of docking with new scoring func- tion, efficient optimization, and multithreading. Journal of Computational Chemistry, 31(2):455-461. 2010.
doi: 10.1002/jcc.213333
H. Maarten Vinkers, Marc R. de Jonge, Frederik F. D. Daeyaert, Jan Heeres, Lucien M. H. Koymans, Joop H. van Lenthe, Paul J. Lewi, Henk Timmerman, Koen Van Aken, and Paul A. J. Janssen. Synopsis: SYNthesize and OPtimize System in Silico. Journal of Medicinal Chemistry, 46(13): 2765-2773, 2003, doi: 10. 10.1021/jm000809.
Ronald J. Williams. Simple statistical meadient-folloiing algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992. doi: 10.10077BF00992666
Andrew J Woodhead, Hayley Angove, Maria G Carr, Gianni Chessari, Miles Congreve, Joseph E Coyle, Jose Cosme, Brent Graham, Philip Day, Robert Downham, et al. Discovery of (2, wwtpr/drrooo--ssoooppphhhoie (4--mmm------mmtttmlllppprraaaa
1-mmethhyl)-1, 3-dihydroosooiddol-.4.
methanone (3113387), a novel inhibitor of the molecular chaperone Hsp90 by fragment based drug design. Journal of medicinal chemistry, 53(16):5556-996) 2010.
Moksh Jain Yoshua Bengio, Kolya Malkin. Flow Network based Generative Models for Nte-Ittee Diverse Candidate Generation. https://gitee wttp://cc../..o...........c ffw.a//flll.oo.ooooooooomom 2024. Accessed: 14.09.2024.
Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo Kim. Graph transformer networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- -uu, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Chengxi Zang and Fei Wang. Moflow: An invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20 pp- 617-626, 2020. doi: 10.1 10.1145/3394486.3403300
15

---

Published as a conference paper at ICLR 2025
Barbara Zdrazil, Eloy Felix, Fiona Hunter, Emma J Manners, James Blackshaw, Sybilla Corbett, Marleen de Veij, Harris Ioannidis, David Mendez Lopez, Juan F Mosquera, Maria Paula Magarinos, Nicolas Bosc. Ricardo Arcila, Tevfik Kizilören, Anna Gaulton, A Patricia Bento, Melissa F Adasme, Peter Monecke, Gregory A Landrum, and Andrew R Leach. The ChEMBL Database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Research, 52 ((DD))DD1800DD
2023. doi: 10. 1003///rrgga
Tian Zhang. Raghu Ramakrishnan, and Miron Livny. BIRCH: An efficient data clustering method for very large databases. ACM sigmod record, 25(2):103-114, 14, 1996.
16

---

Published as a conference paper at ICLR 2025
A EXTENDED METHODS
A.1 1 REWARD FUNCTIONS
We train SynFlowNet for number of reward functions/targets. In some cases (e.g. when benchmark- ing against the Fragment-base GFlowNet), we also perform multiple objective optimisation with SA score.
sEH Our main reward function is defined as the normalized negative binding energy as predicted by a pretrained proxy model, available from Bengio et al. (2021) and trained on molecules docked with MhoDockVinna (Trott & Olson, 2010) for the sEH (soluble epoxide hydrolase) protein target, well studied protein which plays part in respiratory and heart disease (Imig & Hammock, 2009). The proxy model, which utilizes the weights from Bengio et al. (2021), was trained using message-passing neural network (MPNN) (Gilmer et al., 2017) that processes atom graphs as input. Details of the model architecture are available in Bengio al. (2021). was trained on a dataset of 300,000 randomly generated molecules, achieving a test mean squared error (MSE) of 0.6. Note that the reward scale in our results differs from the original GFlowNet publication, with rewards adjusted by a factor of 1/8 in our analysis.
GSK33 and DRD2 We also employ two oracle functions from the PMO Gao al. (2022a) benchmark, which provide machine learning proxies trained fit to experimental data to predict the bioactivities against their corresponding disease targets. The two targets et al. (2018) and dopamine receptor D2 (DRD2) (Olivecrona et al., 2017a).
here are GSK38 Li
Easy adoption to other targets using GPU-accelerated Vina docking Finally, we wish for users rapidly be able to adapt SynFlowNet to learn binding for their target interest without having to retrain a new proxy relying on slow docking simulations. We accomplish this using the new GPU-accelerated Vina-GPU 2.1 docking algorithm (Tang al., 2023; Alhossary et al., 2015). For our experiments, we use the PDB: 6W63 (Mesecar et al., 2020) structure for the SARS-CoV-2 main protease (Mpro) target, PDB: : 2XJX (Murray et al., 2010) for Heat Shock Protein 90 (HSP90) target and PD3: 8AZR for KRAS (Kim et al., 2023). Receptors are prepared for docking with cecraareeeeepptorroo py and the center of the docking defined as the center of mass for the ligand with a size of 25 Á accordance with previous work (Buttenschoen al., 2024). In order to prevent the model from optimising for large molecules, we add reward penalty of -0.4 for molecules with a number heavy atoms larger than that of the reference ligand plus an allowance of 8 additional heavy atoms. The Vina scores are scaled between 0 and I according to:
offinities + reward_scale_nnee R reward_scmle_nee
1,
(4)
mma
where seraard_scal_paee and -10.0 O by default.
are tunable and set to -1 O and respectively
A.2 DATA
We use commercially available building blocks (BBs) from Enamine, which are small fragments of
molecules prepared in bulk to be readily synthesised into candidate molecules. Reaction templates are obtained from two publicly available template libraries (Button et al., 2019; Hartenfeller M, 2012). After preprocessing, we obtain a total of 105 templates: uni- and 92 bi-molecular reactions respectively (see App. 1.2.1). We also use 12 Enamine REAL reactions in small number of experiments (see Section 4.6 and App. A.2.1).
A. CHEMICAL REACTIONS
Reaction pro-psscssi and action masking Reaction templates pre- pre-processing necessary to ensure that the templates could be run backwards. All templates containing wildcards (*) in the reactant SMARTS were duplicated with replacements for all atoms that the wildcard substituted for.
17

---

Published as a conference paper at ICLR 2025
NaN
41
Figure A.1: Example of reaction templates (Hartenfeller M. 2012). The templates act as rules, which match any molecule that left side (before the arrow) as subgraph. The matching part is then transformed into the right hand side of the rule. "R" represents any group, "A" represents an aromatic atom. Note the implicit reagent used in reaction #2.
When running experiments with a REINFORCE backward policy (which requires to sample trajec- tories backward on-policy), we enforced additional masking that forward reactions are sampled if and only if they are reversible (i.e. when applying the template backward on the product, the same reactants obtained as the ones utilised by the template in the forward direction). Note that reaction templates can be uni-molecular (employing one reactant, Fig. A.1 #2) or bi-molecular (employing two reactants, Fig. A.1 #1).
Enamine REAL Space Reactions For the case study Section 4.6, we used reactions from Enamine to stay close to an experimental setup where the molecules generated would be readily purchasable from Enamine. Enamine assembled a molecular space called the REAL space, which vast catalogue of 48B purchasable compounds". We use subset of Enamine reactions available from Swanson et al. (2024) which produces 93.9% of the REAL space'.
A.3 MODEL AND TRAINING
A graph neural network based on graph transformer architecture (Yun et al., 2019) is used to parameterize the forward and backward policies. The model's action space is defined using separate MLPs for each action type (see Figure 1). Our model is trained in online fashion, meaning that it learns exclusively from trajectories sampled from the GFlowNet policy, without relying on an external dataset of trajectories set of target molecules. Note however that this framework is compatible with offline training, which makes use of such datasets as starting point for exploring the molecular space.
SynFlowNet Training We adapt the framework from Bengio et al. (2021) to train a GelowNet sampler over space of synthesisable molecules, which are assembled from an action space of chemical reactions and reactants. A graph neural network with a graph transformer architecture (Yun et al., 2019) is used to produce state-conditional distribution over the actions. A state is represented as a molecular graph in which nodes contain atom features. Edge attributes are bond type and the indices of the atoms which are its attachment points. This representation is augmented with a fully-connected virtual node, which is an embedding of the conditional encoding of the desired sampling temperature, obtained using MLP. The sampling temperature is controlled by a temperature parameter B, which also plays a role in reward modulation, allowing for exponential scaling of the rewards (by making rewards received during training equal R&). We experimented with sampling 3 from multiple distributions, and use a constant distribution in the reported results in this paper. We used a thermometer encoding of the temperature (Buckman et al., 2018).
The model is trained using the trajectory balance objective (Malkin et al., 2022) and thus is parame- terized by forward and backward action distributions Pp and PB and an estimation of the partition function Z ΣtET F(r).
For the state space estimation experiment, we used R = 1, 10 000 training steps and varying trajectory lengths. For the rest of the SynFlowNet experiments with SeH proxy as reward, we used / 32, 5000 training steps and varying trajectory lengths (see main text). For training with the DRD2 and GSK33 targets as reward, we maximum trajectory length of 4 and a reward exponent 3 = 16. For the backward policy training, see App. A.4. The rest of the hyperparameters are fixed and are
hhtps://enamine.nct
Note: this dataset contains single tri-molecular reaction which has heen removed here for simplicity
18

---

Published as a conference paper at ICLR 2025
presented in Table A. For the multi-objective optimization experiments in Sections 4.1 and 4.2, we multiplied the different rewards.
FragGFN training We obtained fragments and their attachment points from Enamine building blocks by following the protocol provided by Jin et al. (Jin et al., 2019). The model trained with the sEH reward was trained with the default implementation in Yoshua Bengio (2024), and an optimised reward exponent 3 = 64. The model optimising for synthetic accessibility, well sEH binding, was trained with a reward obtained by multiplying the two scores.
Soft Q-lcarning training We implemented version of Soft Q-learning (SQL) (Haarnoja et al., 2017), an energy-based policy learning method, that operates on SynFlowNet's MDP. We optimised the training procedure by performing both manual and grid searches across several values of entropy regularisation parameter and reward scaling parameter 3. For 3, contrarily to GFlowNets, since the method does not only learn a policy but tries to estimate the Q-values of each actions directly, we found that using large values of reward scaling such as 3 = 64, which are common for GFlowNets, would destabilise the algorithm and had to be lowered to 3 = 4 or 3 = 2. For a, we tried several values that would strike the best tradeoff between allowing the model to find high-performing molecules while maximising diversity and avoiding to collapse the agent's 's distribution on only a few modes, with our best model using a 0.01.
REINVENT training We benchmarked our approach against REINVENT4 (Loeffler et al., 2024)². Following the setup described in their methodology, we fine-tuned the REINVENT prior model using reinforcement learning. We change little else other than the reward function used train the model. The training was conducted with the default batch size of 100 over 3,000 training steps, resulting in a total of 300,000 oracle calls, which is consistent with the number of oracle calls used in SynFlowNet experiments.
Mon generation We run Molth (Swanson et al., 2024) Monte Carlo Tree Search (MCTS) algorithm using their standard sets of 139 493 building blocks and 13 chemical reactions from the Enamine REAL space, but replace the provided bioactivity prediction models with the reward functions used in this study (Sec. A.1). We use the publicly available code repository' and perform 50 000 rollouts. Other hyperparameters were kept at default settings, including a maximum of 1 reaction and exploration parameter c 10. Before the search, building block scores were pre-computed using the target reward functions. For the final selection, we sample 1000 molecules randomly from all returned molecules.
Hyperparameters
Values
Batch size
64
Number of GNN layers
4
GNN node embedding size
128
Graph transformer heads
2
Learning rate (Pr)
10-4
1.carning rate (Pp)
10-4
Learning (2)
10-3
Table A.1: Hyperparameters used our SynFlowNet training pipelines.
A.4 BACKWARD POLICY TRAINING ALGORITHMM
For the maximum likelihood backward policy, at each training iteration, P3 is updated to minimize the maximum likelihood loss over trajectories generated in that batch from Pp. Similarly for REINFORCE Pg, we train Pp as above, but maintain a replay buffer. To train PB. we sample terminal states from the buffer and sample trajectories backwards from Pp. which are used in REINFORCE update. To improve training stability, we also used trajectories generated from Pr (in 1:1 ratio) update/ Pp-
We used the code available at https::////t.. ////t/tttb///eee cow/Molleeeooooooooocooooooooooo cooleculle arAI 1 /RE NVENT4 httpp:: / gi thub. com/ hwt///wwnnonooooooo///// 4/ /SSytthMo0
19

---

Published as a conference paper at ICLR 2025
Contrary to the rest of the models (see App. A.3), the backward policy models were trained for 8000 steps, with reward exponent = 64 and max_maxleeee 5. The results Section 4.4 are reported for a REINFORCE loss with an entropy multiplier term 1.0.
Algorithm I Training of Maximum Likelihood Backward Policy for GFlowNets
1: Initialize the forward policy Pr, backward policy PB, and Zo-
2: repeat
3: Sample a batch of trajectories {-(") J from Pp.
4: Update PF and Zo to minimize LTB using (r(1)
Jn=1.
5: Update Pg to minimize Lg over {r(1) Jn=I-
6: until convergence
Algorithm 2 Training of REINFORCE Backward Policy for GFlowNets
1: Initialize the replay buffer B, forward policy Pr, backward policy PB, and Za.
2: repeat
3: Sample batch of trajectories (n) ) from Pr.
4: Update B 4- B U {TF
(8) J
5: Sample random trajectories from B and extract their final states s to sample backward
trajectories {TB Jkm from PB.
6: Update Ppr and Zo to minimize LTB using (Tp (
Ju=l.
7: Update PB to minimize JB over (TB U (TF (m) }.
8: until convergence
A.5 SYNFLOWNET SCALING
Here, provide technical details of the experiments described in Section 4.5. For each building block subset, we trained 3 models using fingerprints, and 3 models without fingerprints, which amounts 36 models in total. All models had exactly the same hyper-parameter configuration with SeH proxy as reward, 8000 training steps, batch size 8 and B 32. Other parameters were the same as in Table ??
For evaluation, we sampled 5000 trajectories with each model and considered only trajectories with valid final molecules. In Figure 6C, we report average runtime per batch over the training process. model. forward () corresponds to the average time required to perform the matrix operations (PyTorch back-end, single GPU NVIDIA H100) during the forward pass of the model and aims to highlight the performance overhead introduced by the dot product with molecular fingerprints. model. forward () RDKit t additionally accounts for the costly template matching operations CPU after the model's forward pass (to shortlist the set of allowed building blocks for the chosen reaction). These values demonstrate the effect of the increased building block set. To characterize the structural diversity of the molecules generated by different models, additionally provide the average pairwise Tanimoto dissimilarity of the molecules sampled by different models (using different building block subsets) in Figure A.2.
For Figures 6D-F, the entire building block set was used. To compute clusters of building blocks for Figures 6E-F, used BiuBIRCH (Jung al., 2024), a recent adaptation of Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH) algorithm (Zhang et al., 1996), recently proposed for efficient clustering of large molecular libraries.
B EXTENDED RESULTS
B. COMPARISON TO BASELINES
Table A.3 contains metrics for all the models and baselines we have run. We additionally show results from training SynFlowNet with a reward measuring binding affinity to KRAS using GPU-accelerated VINA docking (Tang et al., 2023), and comparison to baselines. Figure A.4 contains results for
20

---

Published as a conference paper at ICLR 2025
Structural diversity
0.84
Tanimoto dissimilarity
0.82
0.80
w/o fingerprints w/ fingerprints
0.78
w/o fingerprints, w/ fingerprints, R 0.8
0.2
0.4
0.6
0.8 1.0
Fraction of the full BB set
Figure A.2: Diversity of the molecules produced by the baseline and fingerprint models trained with different building block subsets.
Figure A.3: Example of synthesis pathway generated by SynFlowNet. The reward here is the sEH binding affinity proxy. Right shows the Vina docked pose and molecule metrics. More examples are shown Figure A.7.
molecules generated with SyntheMol for the DRD2 target. compared to SynFlowNet. SynFlowNet superior finding high-reward molecules. We tried various hyperparameters for SyntheMol to optimise for the GSK33 target and were unsuccessful - no nonzero rewards were found.
Method
sEH proxy (1)
Diversiyy
SA)())
AiZymih
QED
Mol. weight (1)
ChEMBL Similarit
REINVENT
0.61
FruGFFN
0.77
0:00
6.28 0.02
0.00
0.30 0.01
724.62 32.50
0:21: 0.09
FrgGFN SA
0:01
0.8 0.01
5.45 + 0.05
0.00
0.29 + 0.01
683.31 = 59.92
0.21 + 0.01
SyntheMoi
D.64 0 0.01
0.06
3.08 + 0.001
0.82
0.63 + 0.01
412.24 2-20--1-00 0.98
0.49 + 0.01
Soft Q-Laarnin
0.80 0.07
0.02
2.63
0.96
0.39 0.02
0.52 0.02
SyaFlowNer
2,92
0.50
365
0.43
SyaFlowNet
0.94 2: 0.01
0.75
0.02
2.67 4: 0.00
0.90
0.08 +: 0.0L
338.27 3.552
0.48 = 0.01
SyaFlewNet ChEMBL 3)
0.91 1: 0.02
0.80 0.0.1
2,68 1: 0.18
0.67
[.668 1: 0.03
312.67 8.00
0.19 1 0.01
SyaFlowNet (L =
0.88 0.01
0.82.10 172.1.0000 0.01
3.51 1. 0.03
0.40
0.27 1: 0.01
357 1.8
0.38 1 0.001
Table A.2: Comparison baselines. Results obtained by averaging over 1000 random molecules sampled from the trained models. Standard errors obtained from training using 3 seeds. SynFlowNet ChEMBL refers model trained with building blocks derived from ChEMBL molecules. Due to high computational cost, AiZynthFinder scores are computed over 100 random samples.
Method
KRAS Vina
Diversity (t)
(1)
(t)
Mol. weight
ChEMBL Similarity
REINVENT
0.76 0.03
0.85 0.01
2.51 + 0.05
0.53 + 0.03
359.52 + 6.51
0.57 0.01
FragGEN
0.86 = 0.06
0.62 0.11
3.54
0.43 + 0.03
318. 09 34.42
0.42 0.04
Soft Q-Learning
0.06 0.01
0.68 0.02
3.10
370.97
0.41 0.02
SynFlowNet (L
0.73 0.09
0.87 0.01
3.05 4 0.10 0.56 4: 0.08
389.91 23.27
0.42 0.01
Table A.3: KRAS binding optimization with GPU-accelerated Vina docking, Results obtained by averaging over 1000 random molecules sampled from the trained models. Standard errors obtained from training using 3 seeds. Vina scores are scaled between 0 and using Eq.
21

---

Published as a conference paper at ICLR 2025
DRD2 T
1.0
0.6
0.4
0.0
SynFlowNet
SyntheMol
Method
Figure A.4: Comparison between SynFlowNer and SyntheMol with DRD2 as reward. SyntheMol
struggles to find high-reward molecules. We also run SyntheMol with GSK38 and were not able to optimise Results for SynFlowNet with both DRD2 and GSK33 are reported in Figure 4.
We further compare SynFlowNet's performance to SynNet's (Gao et al., 2022b), which models the containing multi-step synthesis pathways as Markov decision process.
generation of synthetic The approach differs from SynFlowNet and Gottipati et al. (2020a); Horwood & Noutahi (2020) in that relies on a trained model optimized using a genetic algorithm, instead learning the target distribution solely from the reward function, based on an RL objective. In Table A.4 we report t0p-100 performance on optimization tasks performed in SynNet.
Table A.4: Comparison to SynNet. We report average of top- 100 scores. Results for SynNet are reported from Gao et al. (2022b).
Method
QED
GSK3,3
DRD2
JNK3
SyaNet
0.947
0.815
0.998
0.719
SynFlowNet
0.947
0.862
0.999
0.710
Novelty For a more strict evaluation of the novelty of soaellooWet-proposee designs, we investigate whether SynFlowNet can still generate low-similarity molecules to ChEMBL when relying on building blocks derived from ChEMBL actives. In this way, we reach comparable vocabularies between REINVENT's prior and SynFlowNet's action space. For this, select 70 000 random ChEMBL molecules from Zdrazil et al. (2023) and run AiZynthFinder retrosynthesis (Genheden al., 2020) to decompose the molecules into building blocks. This results in 8527 unique building blocks which were used to train SynFlowNeet We report comparative results from this experiment in Figure A.5 and Table A.3 and note that SynFlowNet preserves high novelty compared REINVENT, which emphasizes the advantages of the framework.
SeH proxy 7
Diversity
SA
ChEMBL Similarity
Reward/Novelty trade-off
1.
0/
Spiehandd
Smilartt to
0g
0.8
64
80
033
Sperkwdee Mhrii
SpeleM
05
Figure A.5: SynFlowNet trained with ChEMBL-derrveed building blocks remains competitive against other popular models from the literature. SynFlowNet achieves the best novelty/reward and diver- sity/reward trade-offs.
B.1.1 SAMPLE EFFICIENCY
Following Gao et al. (2022a), we benchmark SynFlowNet against other generative models for molecular design in terms of sample efficiency. We focus the reward functions (DRD2 and
22

---

Published as a conference paper at ICLR 2025
GSK33) that included in the benchmark and report the under the curve (AUC) of the top-10 average performance versus oracle calls. Following Gao et al. (2022a), we limit the maximum number of oracle calls 10 000 and report the results in Table A.5.
Table A.5: Sample efficiency results. We report mean and standard deviation of the AUC of top- 10 average scores versus the number of oracle calls from 5 independent runs. Results for all other methods are reported from Gao et al. (2022a).
Task
Method
AUC
GSK33
REINVENT
0.865 = 0.043
SynNet
0.789 = 0.032
FragGFN
0.651 + 0.026
SynFlowNet
0.691 + 0.034
DRD2
REINVENT
0.943 = 0.005
SynNet
0.969 = 0.004
FragGFN
0.590 = 0.070
SynFlowNet
0.885 = 0.027
le4
Uniform
1.00
unique Murcko Scaftolds with
Maximum likelihood
0.75
REINFORCE =
Mean backward Reward
REINFORCE
0.50
REINFORCE =
0.6
Free
REINFORCE
REINFORCE =
0.00
0.4
Uniform
-0.25
0.2
-0.50
0.0
-0.75
0
2
4
evaluated molecules (for one thread)
1e4
-1.00
Iterations
1e3
Figure A.6: (LHS) Effects of different parameterizations of the backward policy. For REINFORCE.
(a) represents the multiplier to the entropy term. (RHS) REINFORCE is trained on on-policy and the mean rewards of the sampled backward trajectories are plotted. Note that the backward reward is the woncdard-costrrueeed trajectory ends in so and -1 otherwise. A baseline for Uniform policy is shown, obtained by sampling 100 random trajectories backward from terminal objects the DAG. Results averaged over 8 seeds.
B.2 OUTLOOK
While SynFlowNet shows promising results for generating high-quality synthesizable molecules, there remain challenges to be addressed in future work. First, SynFlowNer does not explore non- linear synthetic pathways. i.e. considering intermediates in the DAG second reactants. At the moment the choice of second reactants is limited to the building blocks library. SynFlowNet also does handle reactions with more than reactants. We also emphasize that reaction selectivity is not accounted for, and that including reaction feasibility in the design process would strengthen real-world synthesizability results. Another inherent limitation of the reaction templates employed in this work is the lack of consideration of reaction conditions and stereochemistry. We envision that the backward policy SynFlowNet can be an innovative way accounting for additional constraints in the MDP design, and our preliminary results in Section 3.2 training the backward policy with separate reward from the forward policy support this. One could further bake in constraints such as synthesis cost, reaction yield and selectivity the backward reward Rg. Additionally, more careful curation of the reaction set is encouraged in future work, which would enhance the coverage of the chemical space via diverse transformations: cyclizations, side-chain extensions, functional group
23

---

Published as a conference paper at ICLR 2025
interconversions etc. Lastly, the rediscovery rate of SynFlowNet is not perfect, meaning that certain regions of the chemical space may remain inaccessible to the model under the current datasets.
B.3 EXAMPLE TRAJECTORIES
Figure A.7 shows examples of molecules and synthesis path ways generated from NetF
Vinw: 13.0 DED: 0.87 4.14.10
Vin:: 18.0
035 SA: 2:44
Vina:-10.4 QED: S4:3.27
12.0 OED: 0:48 $4:2.75
Figure A.7: SynFllowWee--geneeate molecules and synthesis pathways for SeH.
24

---

Published as a conference paper at ICLR 2025
B.4 CURATED BUILDING BLOCK SETS GIVEN TARGET DATA
Here we provide further details on the methods and rationale for the fragment-based building block curation strategy outlined in 4.6. To enable high molecule purchaseability in real world drug discovery campaign, we use the Enamine reactions in all these experiments. GPU-accelerated Vina docking was used for reward computation (see App. A.1).
Experimental fragment extration pipeline Fragment- based drug discovery (FBDD) simplifies the search space by focusing on smaller molecular fragments rather than designing or screening full molecules, enabling a more efficient exploration potential binding interactions (Murray Rees, 2009; Thomas et al., 2019). To identify molecules bound within the same pocket across different protein structures, we performed a sequence-bar based search using MMSeqs2 (Steinegger Söding, 2017) across the Protein Data Bank (PDB). We retained only hits with a sequence identity of 90% or greater and an alignment overlap exceeding 80% to ensure high similarity to the reference target. The resulting PDB structures were structurally aligned according to the ligand- bound chain with the reference structure. Any ligand with at least one atom within 2 Á of the reference ligand in the binding site was selected for further analysis.
# of atoms in HSP90 ligand structures
x10
of atoms in REAL building blocks
20
Molecules Fragments
3
15
Count
Count
2
10
1
5
0
0
Number
Number Atoms
Figure A.8: Curation of HSP90 fragments and building blocks. Red line shows threshold at which denote molecule 'fragment" and was choosen the upper limit of the typical BB size.
Selection of small molecule fragments To isolate small molecular fragments suitable for FBDD, we filtered out ligands containing more than 25 atoms, as this atom count aligns with typical building block molecule sizes. This is done for reasons; (i) we can pretend that we are in drug discovery campaign where a fragment screen has just been conducted and there is no leaked information from 'full' molecules and (ii) this threshold aligns well with the typical Enamine building block in terms of size (Figure A.10).
Curation of Enamine Building Blocks For every experimentally validated fragment, we selected the top 100 closest building block molecules from the Enamine library based on molecular similarity. After removing duplicate SMILES entries, this process usually resulted in thousands of curated building blocks for each target.
Example fragment screens We study two targets. The first is Heat Shock Protein 90 (HSP90), for which there are a large number of ligand bound structures, mostly thanks to FBDD campaigns conducted industry (Murray al., 2010; Woodhead et al., 2010). The second the -ARS-CoV-2 Main Protease (Mpro), for which there is large amount of structural data and in particular fragments from a fragment screen performed by the COVID Moonshot open-science initiative (Boby al., 2023). In the case of HSP90, using PDB entry 2XJX (Woodhead et al., 2010) as the reference structure, ligand extraction pipeline identified 92 molecules. Filtering for small molecular fragments of fewer
We the super www.sppshhoooomccm ogs function from biotite: b i te-python.org
25

---

Published as a conference paper at ICLR 2025
than 25 atoms further refined this set for analysis. The methodology repeated for Mpro using PDB entry 7GAW, where we extracted 138 molecules.
Examples of molecules designed for Mpro using our curated building block set are shown in Figure A.9.
Alignment of SynFlow Net designs with experiments Murray al. (2010) performed fragment screen of HSP90 and identified two distinct lead classes of binding modes: (i) an aminopyrimidine class that formed hydrogen bonds with an asparagine residue and several conserved water molecules within the pocket and (ii) phenolic class that primarily binds through water-mediated hydrogen bonds networks. These complicated deper-dependent interaction networks are extremely challenging to model computationally and is clear limitation of Vina docking (which does not take into account waters). Figure A. 10 shows that by guiding SynFlowNet in this way, we can design molecules and poses that fit with these experimentally validated interactions.
SynFlownet (curated building block set)
Reference (6W63)
Vina: Sim: 52
Vins: -8.80 Sim: 0.52 QED: 0.65
V7na--700 Sm: 0:37
Vha -8.0 Sim 0.42
Vina: -870 Sim: 0.35 QED:0.71 2A:2.22
Vin::7.1
QED:0.0 3A:3.45
SA: 4.30
QED:0.53 SA:
QED: SA:3.14
QED:0.66 SA: 3.18
Figure A.9: Example molecules generated by SynFlowNet the Mpro case study. Green molecules those generated by SynFlowNet. Magenta shows the molecule from the reference structure. 'Sim' is the Tanimoto similarity between a designed molecule and the reference structure.
A
B
SynFlowNet PDB: 5M4H
SynFlowNet PDB: 2YEE
Figure A. 1010: SynFlowNet designs molecules that align with real world fragment experiments. Murray et al. (2010) performed a fragment screen for the HSP90 target and identified two classes of binding modes for the target: those built around (A) phenolic compounds (e.g. PDB:5M4H) and (B) aminopyrimidine compounds (e.g. PDB:2YEE). SynFlowNet trained on a target-curated building block set was able consistently design compounds based these experimentally validated binding modes, meaning we can be more certain that they are likely to bind well reality. Green shows SynFlowNet--esiggnn molecule and magenta fragments from x-ray crystallography experiments.
26


Reviews:
[{"review_id": "coVA6I4MJC", "reviewer": "Reviewer_f18i", "summary": "It was a pleasure to review this paper on synthesizability-constrained molecular generation using GFlowNets. Overall, I felt the paper was strong and focuses on an important application area of generative models, it was well-written, and generally builds well on previous work in the field and introduces some novelty. In my review below, I focus on aspects and unresolved questions that, if addressed, could enhance the paper. I summarize my main concerns as: (1) in its current state, the work is not reproducible, and (2) the evaluation metrics and comparisons to prior work used in evaluating SynFlowNet are not always the most relevant for molecular design, and do not tell us much about how this method would actually perform in a realistic setting. There is room for improvement here. I think this paper has the potential to be a really strong paper if it addresses the concerns below.", "questions": "* I could not find how $s_0$ is initialized, nor how important this is (i.e., how different initializations here impact the performance of SynFlowNet). Can the authors clarify? Does it correspond to a molecular fingerprint/embedding of the target molecule, is it random noise, or something else?\n* One of the contributions of this paper is stated as \"we propose one of the first attempts at training the backward policy in a GFlowNet with a separate objective from for the forward policy, which we show to correct backward-generated trajectories and improve sample quality and diversity\" but I am not sure I got that. Which experiment was it exactly that demonstrated the new backward policy led to improved sample quality/diversity? Are these results in the paper? If so this needs to be clarified.\n* How sample efficient is SynFlowNet? I would have liked to see an analysis of this, which is really important for molecular generation tasks as we are almost always operating in the low data regime and data/sample efficiency is key.\n* How does SynFlowNet perform when optimizing for targets that are highly symmetric, e.g., molecules with a Cn rotation axis, where n$\\ge$2? Has this been evaluated? This is something that is a known weakness of other synthesizability-constrained molecular generation frameworks (e.g., SynNet). \n* What are the failure modes of SynFlowNet? This should be more thoughtfully analyzed, as all models have failure modes and it is important to understand them to help future research build on the results of SynFlowNet.\n* How challenging would it be to expand SynFlowNet beyond bi-molecular reactions? I am not suggesting this should have been done here, but asking more from curiousity and wondering if this is a future direction the authors are aiming to explore in future work.\n\nMinor:\n* Are the points in Figure 6 (last panel) shuffled for plotting? It was not clear to me if there are red points under the blue ones or vice-versa. This should be clarified as the results from this plot are informing the conclusions.\n* The references are poorly formatted, this should be fixed.\n* gsk3$\\beta$ should be GSK3$\\beta$ (weird capitalization in the Appendix).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The background and related work section was strong, and covered the previous work in this domain well.\n* The paper, including the appendix, was well-structured and easy to follow.\n* There are more strengths, but I mention them below amongst some of the specific questions I had, to put the questions in context.", "weaknesses": "* I searched throughout the paper and in the submission, and could not find any code to accompany the paper. Given that the authors lamented the lack of publicly-available code for two previous works in their paper, I would have expected a greater emphasis on publishing the code here, as it would also help with the reproducibility of this work. To me this is unacceptable for a computer science conference submission in 2024, so please let me know if I simply missed it, and I can raise my score (I ranked it below the acceptance threshold because of this, and would update my score if the code was made available as it was overall a good paper).\n* Throughout the paper, the authors repeat that one of the key advantages of SynFlowNet is its ability to generate more diverse candidates than other methods (namely, REINVENT and SyntheMol). However, I argue that diversity is only a meaningful metric in molecular generative models if the types of molecules being generated are from the accessible molecular space; if the generated molecules are diverse but not accessible, then the model is not very practically useful. That being said, it was good the the authors evaluated synthesizability using AiZynthFinder, but an \"easy\" experiment that the authors could have also done that would add a lot of value is to consider molecular rediscovery benchmarks. That is, can the model also find known, previously-synthesized binders; celecoxib rediscovery is a widely-used standard benchmark for this, but there are also more challenging targets. Such rediscovery metrics are a nice alternative to experimental validation which is not necessarily possible on the review timeline. To summarize, I would like to know that SynFlowNet can generate diverse and *known* synthesizable molecules, not just diverse *predicted* synthesizable molecules, because at the end of the day CASP tools are not a replacement for experiment.\n* I found the lack of comparison to other methods for synthesizability-constrained molecular generation a weakness. The authors mentioned that the models of Gottipati and Horwood & Noutahi were not publicly available, and thus focused on comparison to SyntheMol and REINVENT. While these comparisons were interesting and certainly well-motivated, I would have also liked to know how SynFlowNet compares to other synthesizability-constrained molecular generation models which *do* have publicly available codebases, like MoleculeChef or SynNet. As it stands, we do not know which model is best, because SyntheMol has not done any comparisons to those models either.\n* In the Appendix, the authors state that 300K oracle calls were used both to train REINVENT and SynFlowNet, however, this is an extreme number of oracle calls; can the authors justify this? REINVENT demonstrates good performance after only 10K oracle calls for most objectives, so are 300K oracle calls simply what is needed for good performance in SynFlowNet? I would like to see the analogous experiments and comparisons/results with only 10K oracle calls, which is the standard number of oracle calls used for comparing molecular generative models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2BhTeq34oj", "reviewer": "Reviewer_6JHQ", "summary": "The authors proposed a molecular generative model that explicitly considers the synthesizability of generated molecules. Specifically, they formulated a molecular design problem into a Markov decision process (MDP) that successively adds a purchasable building block (BB) with a selected reaction to the previous state. A GFlowNet has been adopted to tackle the MDP. Namely, SynFlowNet shows better performance in improving the diversity and high reward modes simultaneously, whereas conventional optimization approaches like reinforcement learning only focus on the highest reward modes. To handle more BBs, they are represented with binary Morgan fingerprints, and the probability of sampling a particular BB is computed as a function of its fingerprint and the embedding vector of the current state. In addition,  new backward policies have been tested to ensure that the backward trajectories can return to existing building blocks. They also performed a case study that leverages experimental data for more efficient drug design as a practical application. Overall, the manuscript is well-written, but several issues need to be addressed.", "questions": "1. In Table 1, while the two backward policies, the Maxlikelihood and REINFORCE, show relatively high ratios of the solved routes in both the training and test sets, their number of high reward modes is similar to that of the Uniform Policy. This result is rather disappointing, considering the objective of the generative model is to sample more high-reward modes. In this context, the authors need to discuss the effect of the proposed backward policies on the performance of the forward policy in more detail.\n2. In Figure 4, lower rewards achieved by FragGFN and FragGFN SA seem questionable because FragGFN covers a much larger chemical space, as seen from Figure 2, and so is more likely to find molecules with higher rewards. Has the poor performance of FragGFN originated from insufficient training? Or is it because of the use of SeH proxy as a reward, which may have a severe generalization problem for unseen molecules? This should be justified in the revision. \n3. In Section 4.1, what is the rewarding strategy used for multi-objective optimization (FragGFN SA, SynFlowNet SA, SynFlowNet QED)? The possible reward structures could be Multi-objective GFlowNet (MOGFN) or multiplication-based rewards.\n4. Regarding the metric Reward/Novelty trade-off, it is possible that improving the novelty for SynFlowNet may be simply due to the use of the Enamine building blocks, while REINVENT has been trained with ChEMBL. Thus, the high similarity of the molecules generated by REINVENT to ChEMBL is not unexpected and would be reduced if training it with the Enamine REAL. The authors need to address this issue.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The use of the GFlowNet improves both diversity and high reward modes compared to other optimization methods such as reinforcement learning.\n2. The proposed backward policy ensured that backward-constructed trajectories belong to the GFlowNet MDP. \n3. Various benchmark and ablation studies were performed to ensure the strategies of the proposed framework.", "weaknesses": "1. The conclusions of some experiments are arguable (see the questions below), which need to be justfied probably with additional experiments for clarity.  \n2. Increasing the number of BBs slows down training linearly that may cause a computational burden when dealing with a large amount of BBs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "agbQDdOE9w", "reviewer": "Reviewer_cHNB", "summary": "The paper presents SynFlowNet, a model designed to improve the generation of novel drug-like molecules while ensuring their synthetic accessibility. SynFlowNet constructs molecules sequentially using commercially available reactants and documented chemical reactions, effectively constraining the generation process within a synthesizable chemical space. The model integrates a forward synthesis constraint, evaluates synthetic accessibility through external retrosynthesis tools, and optimizes diversity and reward (e.g., binding affinity) without relying on reinforcement learning, which tends to sample fewer, high-reward molecules. SynFlowNet introduces a backward policy for navigating reaction trajectories, improving synthetic feasibility and sampling accuracy. By utilizing molecular fingerprints and scalable building blocks, SynFlowNet can explore vast chemical spaces efficiently, making it adaptable for specific drug targets when provided with experimental fragment screening data, such as in SARS-CoV-2 protease inhibitors.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Focus on Synthetic Accessibility: SynFlowNet adopts a framework that effectively addresses a key limitation in generative models for drug design by ensuring that generated molecules are synthetically feasible, which is crucial for practical drug development applications.\n- Diverse and Optimized Molecule Sampling: Leveraging GFlowNets, SynFlowNet provides a balanced sampling process that maximizes both reward and structural diversity, effectively mitigating the typical mode collapse encountered in reinforcement learning-based models.\n- Validation through Synthetic Accessibility Metrics: In silico experiments demonstrate SynFlowNet’s advantage over fragment-based approaches, particularly in terms of synthesizability, underscoring the practical benefits of synthesis-based molecule generation for drug discovery.", "weaknesses": "Limited Testing on Optimization Tasks: While SynFlowNet showcases improvements in SeH proxy, its optimization abilities are not fully tested against established molecular optimization benchmarks. Incorporating PMO benchmark tasks and directly comparing results with other generative baselines, such as those in Gao et al. (2022) [1], could provide a more comprehensive evaluation of SynFlowNet’s optimization performance.\n\n### Reference\n[1] Gao, Wenhao, et al. \"Sample efficiency matters: a benchmark for practical molecular optimization.\" Advances in neural information processing systems 35 (2022): 21342-21357.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rx6FMrm8q6", "reviewer": "Reviewer_AADH", "summary": "This paper presents SynFlowNet, a practical approach for drug design that incorporates reaction-based Markov Decision Processes (MDPs) into a Generative Flow Network (GFlowNet) to generate diverse molecules with guaranteed synthetic pathways.\nNotably, authors propose a novel backward policy training strategy to ensure the presence of backward trajectories.\nFurthermore, the authors investigate the potential of SynFlowNet to be integrated with wet experiments such as fragment-based crystallography.", "questions": "1. **Page 2, Figure 1B:**\n    - As I understand it, the building block selection probability depends on not only the state embedding but also the selected reaction. (Page 4 Line 213). However, the Figure 1B indicates that the selection probability is calculated only from the state information $s_k$. I suggest revising the figure and including the series of equations for the neural network to clarify this process.\n    - Also, I was wondering if dot product operation between Morgan fingerprint (binary vector) and state embedding can satisfy the GFlowNet Objective. When the 4-bit fingerprints for building blocks $b_1$, $b_2$, and $b_3$ are $[1,0,0,0]$, $[0,1,0,0]$, and $[1,1,0,0]$, respectively, the probability of $b_3$ can be derived from those of $b_1$ and $b_2$:\n      $$\\log P(b_3|s) = \\frac{\\log P(b_1|s) + \\log P(b_2|s)}{\\sqrt{2}}$$\n      I think that this relationship can introduce bias in the forward transition probability.\n3. **Page 7, Footnote:**\nIs it typo?: '… which are different to the **reactants** used by SynFlowNet …'. As I know, AiZynthFinder allows user to use different reactants. I think 'reactants' should be replaced to 'reactions'.\n4. **Page 9, Figure 7C:**\nI suggest changing the title \"Performance\" to \"Runtime\" or a similar term.\n5. **Page 19, Table A.1 (Hyperparameters)**:\nCould you clarify the meaning of *max_nodes*? In the standard GFlowNet implementation, it refers to the maximum number of heavy atoms in molecules, but the generated molecules illustrated in the paper have more than 9 heavy atoms.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Novel Backward Policy of GFlowNet training: SynFlowNet proposes a novel learning strategy for backward policy to ensure the directed acyclic graph and facilitate synthesizable molecules.\n2. Connection to Wet-Lab Experiments: The case study in Section 4.6 provides a clear demonstration of how to connect wet-lab experiments and *in silico* drug design, highlighting the potential real-world applicability of the method.\n3. Comprehensive Ablation Studies: The paper provides ablation studies for many design choices.", "weaknesses": "**Overall Comment**\n\nI acknowledge the novelty of this work in successfully adapting the GFlowNet framework for synthesizable molecular design, but there are several critical aspects of the current manuscript that need to be addressed.\nI think this manuscript may be suitable for acceptance after revision, and I’m positive about changing the scores during the rebuttal period.\n\n*\\* The issues are sorted in the order they appeared in the manuscript.*\n\n**Issue 1: Lack of novelty / appropriate citations for the major contribution.** Page 2, Line 103, Contribution 3\n\nOne major concern is that the manuscript does not contain appropriate citations for this major contribution.\nTo use 200k building blocks, the authors used action embedding [1], which has already been employed in existing synthesis-oriented molecular generative models [2-4].\nIn particular, Koziarski et al. [4] introduced action embedding into GFlowNet and investigated its effectiveness in large action spaces.\nWhile the authors use a different fingerprint representation (Morgan fingerprints instead of MACCS fingerprints), Gottipati et al. [2] have already evaluated various representations, including both fingerprints.\nAuthors should highlight the novel differences from existing approaches or provide appropriate citations.\n\n**Issue 2: Lack of experimental details.** Section 4.1 and Section 4.5\n- In Section 4.1, what is the rewarding strategy used for multi-objective optimization (FragGFN SA, SynFlowNet SA, SynFlowNet QED)? The possible reward structures could be Multi-objective GFlowNet (MOGFN) [5] or multiplication-based rewards [6].\n- In Section 4.5, I couldn't find the experimental details. The related section is on Page 19, Section A.5, but it does not provide crucial information about this experiment. I suggest to include experimental details and mention it like Section 4.6.\n  - Reward Function: What reward function was used in the ablation studies? Is it sEH?\n  - Average Times: Clarify what *average times* refers to–e.g., runtime per batch or runtime per molecule.\n  - For Figure 7 D-F, entire building block set is used?\n\n**Issue 3: Fairness of evaluation metric.** Page 8, Figure 6\n\nThe improvement in the reward/diversity trade-off is impressive, but I have questions about the metric, **Reward/Novelty trade-off**.\nRegarding the Reward/Novelty trade-off, it is possible that the improvement in novelty is due to the use of the Enamine building blocks rather than the effectiveness of well-structured generative model architecture.\nIn this case, the REINVENT's high similarity to ChEMBL would be greatly reduced by simply retraining it on Enamine REAL or ZINC.\n\nFor a fair comparison in terms of novelty, the comparison should be performed on the same exploration space:\n- Train REINVENT on Enamine REAL.\n- Use ChEMBL-like building blocks, such as blocks obtained from the retrosynthetic decomposition [3] of ChEMBL molecules.\n\n**Issue 4: Limited comparative study.** Sections 4.2 and 4.3\n\nThe comparison between the proposed method and baselines was performed only on proxy models.\nI recognize that proxy models such as sEH [7] are widely used to evaluate the effectiveness of optimization strategies in the optimization community [8], so I think this is a minor issue.\nHowever, I have concerns about the appropriateness of evaluating and comparing the performance of discovering novel molecules based on proxy models trained on a restricted and known chemical space.\n\n**Issue 5: Restricted scalability to a large building block set.** section 4.5\n\nI definitely agree with the argument for employing an extensive building block (BB) set to explore the vast chemical space.\nI believe that it can mitigate the reward-diversity trade-off by expanding the sample space.\nHowever, I concern that the current experimental results do not seem to provide sufficient evidence about this argument and the scalability of SynFlowNet.\n\nThe authors demonstrated that a larger BB set yielded a higher selection of unique BBs (Figure 7B), but this has a negative impact on the average reward (Figure 7A) and speed (Figure 7C).\nConsequently, the current findings seem to emphasize the negative effects of the usage of extensive BB sets on the quality and throughput for generation.\nTo justify the significance of utilization of unique BBs, I suggest including structural diversity metrics for generated samples, such as the number of unique Bemis-Murcko scaffolds or the average pairwise Tanimoto distance.\n\n---\n**Reference**\n1. Dulac-Arnold, Gabriel, et al. \"Deep reinforcement learning in large discrete action spaces.\" arXiv preprint arXiv:1512.07679 (2015).\n2. Gottipati, Sai Krishna, et al. \"Learning to navigate the synthetically accessible chemical space using reinforcement learning.\" International conference on machine learning. PMLR, 2020.\n3. Seo, Seonghwan, Jaechang Lim, and Woo Youn Kim. \"Molecular generative model via retrosynthetically prepared chemical building block assembly.\" Advanced Science 10.8 (2023): 2206674.\n4. Koziarski, Michał, et al. \"RGFN: Synthesizable Molecular Generation Using GFlowNets.\" ICML'24 Workshop ML for Life and Material Science: From Theory to Industry Applications (2024).\n5. Jain, Moksh, et al. \"Multi-objective gflownets.\" International conference on machine learning. PMLR, 2023.\n6. Lee, Seul, Jaehyeong Jo, and Sung Ju Hwang. \"Exploring chemical space with score-based out-of-distribution generation.\" International Conference on Machine Learning. PMLR, 2023.\n7. Bengio, Emmanuel, et al. \"Flow network based generative models for non-iterative diverse candidate generation.\" Advances in Neural Information Processing Systems 34 (2021): 27381-27394.\n8. Gao, Wenhao, et al. \"Sample efficiency matters: a benchmark for practical molecular optimization.\" Advances in neural information processing systems 35 (2022): 21342-21357.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)