Generative Flows on Synthetic Pathway for Drug Design
Authors: Seonghwan Seo
Minsu Kim
Tony Shen
Martin Ester
Jinkyoo Park
Sungsoo Ahn
Woo Youn Kim
Venue: ICLR 2025 Poster (2025)
Keywords: GFlowNet
synthesizability
structure-based drug design
molecule optimization
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a synthesis-oriented generative framework that aims to discover diverse drug candidates through a large action space and generative flow network objective.

Abstract: Generative models in drug discovery have recently gained attention as efficient alternatives to brute-force virtual screening. However, most existing models do not account for synthesizability, limiting their practical use in real-world scenarios. In this paper, we propose RxnFlow, which sequentially assembles molecules using predefined molecular building blocks and chemical reaction templates to constrain the synthetic chemical pathway. We then train on this sequential generating process with the objective of generative flow networks (GFlowNets) to generate both highly rewarded and diverse molecules. To mitigate the large action space of synthetic pathways in GFlowNets, we implement a novel action space subsampling method. This enables RxnFlow to learn generative flows over extensive action spaces comprising combinations of 1.2 million building blocks and 71 reaction templates without significant computational overhead. Additionally, RxnFlow can employ modified or expanded action spaces for generation without retraining, allowing for the introduction of additional objectives or the incorporation of newly discovered building blocks. We experimentally demonstrate that RxnFlow outperforms existing reaction-based and fragment-based models in pocket-specific optimization across various target pockets. Furthermore, RxnFlow achieves state-of-the-art performance on CrossDocked2020 for pocket-conditional generation, with an average Vina score of –8.85 kcal/mol and 34.8% synthesizability. Code is available at https://github.com/SeonghwanSeo/RxnFlow.

Published as a conference paper at ICLR 2025
GENERATIVE FLOWS ON SYNTHETIC PATHWAY FOR DRUG DESIGN
Seonghwan Seo'; Minsu Kim Tony Shen?, Martin Ester?, Jinkyoo Park 1.3
Sungsoo Ahn', Woo Youn Kim 1,5-
1 KAIST Simon Fraser University
OMELET
4POSTECH
"HITS
ABSTRACT
Generative models in drug discovery have recently gained attention as efficient al- ternatives to brute-force virtual screening. However, most existing models do not account for synthesizability. limiting their practical use in real-world scenarios. In this paper, we propose RXNFLOW, which sequentially assembles molecules using predefined molecular building blocks and chemical reaction templates to constrain the synthetic chemical pathway. We then train on this sequential generating pro- cess with the objective of generative flow networks (GFlowNets) generate both highly rewarded and diverse molecules. To mitigate the large action space of syn- thetic pathways GelowNets, implement novel action space subsampling method. This enables RXNFLOW to learn generative flows over extensive action spaces comprising combinations of 1.2 million building blocks and 71 reaction templates without significant computational overhead. Additionally, RXNFLOW can employ modified or expanded action spaces for generation without retraining, allowing for the introduction of additional objectives or the incorporation of newly discovered building blocks. We experimentally demonstrate that RXNFLOW out- performs existing reaction-based and fragment-based models in pocket-specific optimization across various target pockets Furthermore, RXNFLOW achieves sate-of-the oftt--ofthheassd ft-th--rtt art performance CrossDocked2020 for ponket-conditionall gener- ation, with an average Vina score of -8.85 kcal/mol and 34.8% synthesizability.
INTRODUCTION
Structure-based drug discovery (SBDD) has emerged as pivotal paradigm for early drug discov- ery, facilitated by the increasing accessibility of protein structure prediction tools (Jumper et al., 2021) and high-resolution crystallography (Liu et al., 2015). However, traditional brute-force vir- tual screening is computationally expensive (Graff et al., 2021), prompting the development of deep generative models that can bypass this inefficiency. In this context, various approaches such as deep reinforcement learning (Zhavoronkov et al., 2019), variational autoencoders (Zhung et al., 2024) generative adversarial network (Ragoza et al., 2022), and diffusion models (Guan et al., 2023a;b) have been proposed to directly sample candidate molecules against given protein structure.
While generative models have shown success in molecular discovery with desirable biological prop- erties, most overlook synthesizability which is crucial factor for wet-lab validation (Gao & Coley 2020). One line to improve synthesizability is multi-objective optimization using cheap functions to estimate the synthesizability (Ertl & Schuffenhauer, 2009), but this is too simplified to reflect complex synthetic principles (Cretu et al., 2024). Other efforts aim to project molecules from gener- ative models into synthesizable space (Gao al., 2022b; Luo al., 2024; Gao et al., 2024b), but chemical modifications in this process can degrade the optimized properties.
To address this issue, recent works formulate the generation of synthetic pathways as Markov decision process (MDP) for molecular design (Gottipati et al., 2020). These approaches return a synthesizable molecule by assembling purchasable building blocks and reaction templates accord- ing to the generated synthetic pathway. Notably, the emergence of virtual libraries created by combinatorially enumerating building blocks and reaction templates, such as Enamine REAL (Gry- gorenko et al., (220)aallows the generated molecules to be readily synthesizable on demand with
"Correspondence (shwan01 06, wooyoun) @kai istttttt... ar.kr

---

Published as a conference paper at ICLR 2025
(a) Action Space
Continuous Action Space A
(b) Pollcy Estimation
Fo(sr, a)
(10M tomplato-tioc pairs, sedion 3:2)
Action Space
Building
Blocks (1.2M)
Embedding
Action
DOCAA f
Template A
Template
Subsampling
(Sedion
02
81
Raaction Templates (71)
Tempiate
Px fo
Partial Space
One-hot
Encoding
...
Importance Sampling
(c) Generation
Reaction
Generative Policy my
Train GFlowNet objective (Bedion 5.1
R(a))
Starting Block
Og Mela 8zj A2)
90
Overview of RXNFL.OW. (a) Synthetic action space which represented in a continuous action space. Each colored box corresponds to a reaction template and the molecules in the box are reactant blocks. (b) Policy estimation using the action space subsampling in a manner of importance sampling. (c) Molecular generation process and model training.
Figure 1:
their synthetic pathways. Recent studies (Cretu et al., 2024; Koziarski et al., 2024) advanced this approach by training the decision-making policy using the objective of generative flow networks (GFlowNets; Bengio et al., 2021). This objective encourages the policy to sample in proportion to the reward function, enabling the retrieval of samples from diverse range of modes.
Unlike atom-based or fragment- based models, the synthetic action spaces are massive and composed of millions of building blocks and tens of reaction templates. While the large action spaces offer opportunities to discover novel hit candidates by expanding an explorable chemical space (Sady- bekov al., 2022), it incurs significant computational overhead. Thus, prior works have restricted the action spaces to trade off the size of the search space for efficiency. However, reducing the search space leads to a decrease in diversity and synthetic complexity. While one could compensate for the reduced number of building block candidates by adding more reaction steps, this leads to the increase in synthetic complexity, negatively influencing synthesizability, yield, and cost (Coley et al., 2018; Kim et al., 2023).
In response to this challenge, we propose RXNFLOW, synthesis orientted generative framework that allows training generative flows over large action space to generate synthetic pathways for drug design. The distinctive features of this method are as follows. First, we introduce an action space subsampling (Figure 1) to handle massive action spaces without significant memory overhead, enabling us to explore a broader chemical space with fewer reaction steps than cxisting models. Then, we train the generative policy with GFlowNet objective to sample both diverse and potent molecules from the expanded search space. We demonstrate that RXNFLOW effectively generates drug candidates, outperforming existing faction-based atom-based, and fragment-basee based baselines across various SBDD tasks, while ensuring the synthesizability of generated drug candidates. We also achieve new state-of-the-art Vina score, drug-likeness, and synthesizability on the Cross- Docked2020 pocket-conditiona generation benchmark (Luo et al., 2021).
Furthermore, we formulate adaptable MDP (Figure 2) for consistent flow estimation modified building block libraries, which can be highly practical in real-world applications. By combining the proposed MDP with action embedding (Dulac-Arnold al., 2015), which represents actions in continuous space instead of discrete space, RXNFLOW can achieve further objectives or incorpo- rate newly discovered building blocks without retraining. We experimentally show that RXNFLOW can achieve an additional solubility objective and behave appropriately for unseen building blocks. This capability makes RXNFLOW highly adaptable to real-world drug discovery pipeline, where new objectives frequently arise (Fink et al., 2022) and building block libraries continuously expanding (Grygorenko et al., 2020).
RELATED WORKS
Structure-bard based drug discovery. The first type of SBDD involves pocket-specific optimization methods to enhance docking scores against single pocket, including evolutionary algorithms (Rei-
2

---

Published as a conference paper at ICLR 2025
denbach, 2024). reinforcement learning (RL) (Zhavoronkoy al., 2019), and GFlowNets (Bengio et al., 2021). However, these require individual optimizations for each pocket, limiting scalability. The second type is based on pocket-conditional generation, which generates molecules against arbitrary given pockets without additional training. This can be achieved by distribution-based gen- erative models (Ragoza al., 2022; Peng et al., 2022; Guan al., 2023b; Schneuing et al., 2023; Qu et al., 2024) trained on protein-ligand complex datasets to model ligand distributions for given pock- ets. On the other hand, Shen et al. (2023) formulated a pondet-con policy for GelowNets to generate samples from reward-biasee distributions in a zero-shot manner.
Syntheis-oriented generative models. To ensure the synthesizability of generated molecules, synthesis-oriented de novo design approaches incorporate combinatorial chemistry principles into generative models. Bradshaw et al. (2019) represented the synthetic pathways as directed acyclic graphs (DAG) for generative modeling. Horwood & Noutahi (2020) formulated the synthetic path- way generation as an MDP and optimized the molecules with RL. Similarly, Gao et al. (2022b) employed a genetic algorithm to optimize synthesis trees to generate molecules with the desired properties. Seo et al. (2023) proposed a conditional generative model to directly sample molecules with desired properties without optimization. Recently, Cretu et al. (2024); Koziarski et al. (2024) proposed the reaction- based GFlowNet to generate diverse and potent molecules.
Action embedding for large action spaces. To handle large action spaces in the mnthesis-orienter generation, Gottipati et al. (2020) employed action embedding (Dulac-Arnol) et al., 2015) that rep- resents building blocks in continuous action space with their chemical information. Later, Seo et al. (2023); Koziarski et al. (2024) experimentally demonstrated that it can enhance the model training and generative performance. The continuous action space provides the benefit of reducing the computational complexity for sampling from large space of actions and the memory requirement for parameterizing the categorical distribution over the large action space.
Generative Flow Networks. GFlowNets are learning framework for a stochastic generative policy that constructs an object through a series of decisions. where the probability of generating each object is proportional to a given reward associated with that object (Bengio et al., 2021). Unlike other optimization methods that maximize rewards and often converge to a single solution, GetowNets aim to sample diverse set of high-rewarded modes, which is vital for novel drug design (Shen et al., 2023; Jain et al., 2022). To this end, the generative policy is trained using objectives such as flow matching (Bengio et al., 2021), detailed balance (Bengio et al., 2023), and trajectory balance (Malkin et al., 2022). Extending the GFlowNets to various applications is an active area of research, e.g., GFlowNets have been applied to designing crystal structures (Nguyen et al., 2023b), phylogenetic inference (Zhou et al., 2023), finetuning diffusion models (Venkatraman et al., 2024), and causal inference (Nguyen al., 2023a).
3 METHOD
3.1 GFLOW PRELIMINARIES
GFlowNets (Bengio et al., 2021) are the class of generative models that learn to sample objects E X' proportional to a given reward function, i.e., p(x) x R(x). This is achieved by sequentially constructing a compositional object æ through a series of state transitions s - s'. forming a trajec- Sn z) E T. The set of all complete trajectories from the initial state So can be represented as a directed acyclic graph G = (S,A) with a reachable state space and an action space A. Each action induces transition from the state s to the state s', expressed as s' = T(s, a) and represented as s + s'. Then, we define the trajectory flow F(T), which flows along the trajectory T (so + .... + Sn. = xr), as the reward of the terminal state, R(x)). The edge flow F(s + s'), or equivalently F(s, a). is defined as the total flow along the edge a 8 + 8':
tory (so
F(s + s') F(s,a)=
Σ
F(T).
TET SL (a (S-JE
The state flow F(s) for the intermediate state is defined as the total flow through the state
F(x)= = Σ
F(T)=
Σ
F(6" +) = Σ
(2)
TET SET
(s"-))EAA
(S-
Intermediate foR matching condition
3

---

Published as a conference paper at ICLR 2025
In addition to the flow matching condition for intermediate states, there are two boundary conditions for the states. First, the flow of a terminal state I must equal the reward of the objective: F(z) = R(z). Second, the partition function, Z, is equivalent to the sum of all trajectory flows and the sum of all rewards: Z = F(so) = E-ET F(T) = Ezex R(x). These three conditions-one for intermediate states and two for boundary states are known the flow matching conditions and ensure that GFlowNets generate objectíves proportional to their rewards.
To convert the flow network into usable policy, we define the forward policy as the forward transi- tion probability (r(s'|s) and the backward policy as the backward transition probability Pa (s(s'))
F(s ((s-+')) F(s)
PB(s|s)) = P(s +'1s's F(s ++8
Pf(6'|s) = P(s + s'(ss
(3)
F(s")
3.2 ACTION SPACE FOR SYNTHETIC PATHWAY GENERATION
Following Cretu al. (2024), we treated chemical reaction as forward transition and synthetic pathway as trajectory for molecular generation. For the initial state sg. the model always chooses AddFi ist Rea addFirstReactant at to sample building block b from the entire building block set B as starting molecule. For the later states s, the model samples actions among ReactUni, ReactBi, or Stop. When the action type is ReactUni, the model performs in silico uni-molecular reactions with an assigned reaction template r R2. When the action type is ReactBi, the model performs bi- molecular reactions with a reaction template r E R2 and a reactant block b in the possible reactant set for the reaction template r: B, B. If Stop is sampled, the trajectory is terminated. To sum up, the allowable action space A(s) for the state 8 is:
if s = 80 otherwise
A(s) = [sstop] UR U b(r E R2, E B,}
(4)
where unavailable reaction templates to the molecule of the state are masked.
3.3 FLOW NETWORK ON ACTION SPACE SUBSAMPLING
We propose a novel memory-efficiett technique called the action space subsampling, that estimates the state flow (s(s) (s) from a subset of the outgoing edge flows Fa - s') for forward policy esti- mation. First, we implement an auxiliary policy, termed subsampling policy ((A)) which samples a subset of the action space A" C A. This reduces both the memory footprint and the computational complexity from (2)) to O[
with the controllable size |B* We then estimate the forward policy by importance sampling. In contrast to the parameterized forward policy, we formu- late a fixed backward policy since it is hard to force invariance to molccule isomorphism (Malkin al., 2022). Theoretical backgrounds provided in Sec. A.
Subsampling policy. Subsampling policy P(AA performs uniform sampling for the initial state and importance sampling for the later states. For the initial state, the allowable action space A(su) = B is homogeneous since all of them are AddF i www.rrtteeaa..ccm ant actions. For the later state, the action space is comprised of one Stop, tens of ReactUni actions, and millions of ReactBi actions. To capture rare-type actions in the inhomogeneous space, we use all Stop and ReactUni actions. The partial action space A (s(s) ~ P(A(s)) comprises the uniform subset B C or B; B,:
(² [B" HSsoo) URI U((r,))r R2
if 80 otherwise
(5)
Forward policy. To estimate the forward policy PF(s'(s) F(s s')/F(s) from the partial
action space AA,, we estimate the state flow F(s) with a subset of outgoing edge flows F's - s').
Since we introduce importance sampling for action types, we weight the edge flow F(s + s') of edge a s > according to the subsampling ratio:
[ /B-1
if E B
Wo ((++)'))
|B.1/1811
if is (r;b) where r E Ra if is Stop or a E R,
(6)
4

---

Published as a conference paper at ICLR 2025
(a) Hierarchical MDP (existing)
(b) Non-hierarchical MDP (ours)
Added
15
60
20
20
20
Modified Library
Modified Library
60
60
60
60
60
80
160
20
160
160
20
20.
60
80
Removed
Flood
Reaction template
Building Blocks
Figure 2: Comparison of using modified building block library for the generation: (a) a hier- archical MDP, and (b) non- non-hierarchical MDP. More details are in Figure 8.
By weighting edge flows, we can estimate the state flow Fo(s; Ar) as:
Ao(s;AA)=
Σ
((s Fo(s + 5'):
(7)
(a-tE)EA*(s)
which the estimated forward policy is PF(s'|s (s; Ar;0) Fols
Action embedding. In the standard implementation (Bengio et al., 2021) of the flow function Fg and its neural network Φa. the edge flow of the edge a : s is computed with the corresponding action-specifie parameter 0a: Fa(s + s') Fo(s, a) og (Ø (s)).
However, large action spaces require numerous parameters which increase model complexity. To address this, embeds the building block into a continuous action space with its structural information, molecular fingerprints (see Sec. B.3):
additional network ablock for AddFi i rrsttt Reactant and ReactBi, which
Fo b) de (# sisate (s), 08
block (b)), Fols, (r, b)) = 08 (00
flow sstate (s), 0()), block (b))
(8)
where 8(r) is the one-hot encoding for bi-molecular reaction template
GFlowNet training. this work, we use the trajectory balance (TB; Malkin et al., 2022) as the training objective of GFlowNets from Eq. (9) and train models following Sec. B.4. The action space subsampling performed for each transition s - S1+1: A ~ P(A).
(1og Zo I1 PF 81 11 R(xx) I. PB PB(St-1(st)
(TB(T)
log
(9)
For online training, we use the sampling policy proportional to Pf(-l-;A'; 0), given by:
W(s- Fo - +s')
To(5'1s;
(10)
(5-1")) W(o-to") Fals + 5")
3.4 JOINT SELECTION OF TEMPLATES AND BLOCKS
For bi-molecular reactions, existing synthesis-oriented methods (Gao et al., 2022b; Cretu et al., 2024; Koziarski al., 2024) formulated hierarchical MDP which selects reaction template first and then the corresponding reactant block b sequentially from Eq. (11). However, as shown in Figure 2, the probability of selecting each reaction template is fixed after training in hierarchical MDP, and this rigidity can lead to incorrect policy estimates in modified block libraries. Therefore, we formulate a non-hierarchical MDP that jointly selects reaction templates and reactant blocks (r, b) at once, as given by Eq. (12), resulting in more consistent estimates of forward policy Pr.
Fo(s,))
Fr(s,(r,b)
PT(T(s, b)]ss; = EEERROR Fo(s,r') EVEB. Fols, (r,
(11)
Fols. (r,b))
(r,T(s, EVERIU(EOO)) Fols,r') Er EveB, Fols, (r,,))
(12)
5

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
Overview, We validate the effectiveness of RXNFLOW in two common SBDD tasks: pocket-specific optimization (Sec. 4.1) and pocket-conditional generation (Sec. 4.2). To the best of our knowledge, this is the first synthesis-orienter approach for pocket-conditionn the applicability of RXNFLOW in real-world drug discovery pipelines where new further objectives may be introduced (Sec. 4.3) and the building block libraries are constantly expanded (Sec. 4.4). Lastly, we conduct an ablation study in Sec. 4.5 and a theoretical analysis in Sec. D.8. Source code available at https: / / github.. com/ www.oooooww/aaaaeeeeeeoo SeonghwanSeo RRxFFlos
generation. We also investigate
Setup. We use the reaction template set constructed by Cretu et al. (2024) including 13 uni- and 58 bi-molecular reaction templates. For the building blocks, we use 1.2M blocks from the Enamine comprehensive catalog. We use up to 3 reaction steps for generation following Enamine REAL Space (Grygorenko et al., 2020), while SynFlowNet and RGEN allow 4 steps. For the subsampling policy, we set sampling ratio of 1%. The experimental details are provided in Sec.
Synthesizabilityy estimation. To assess the synthesizability of the generated compounds, we used the computationally intensive retrosynthetic analysis tool AiZynthFinder (Genheden et al., 2020) with the Enamine building block library. We note that the molecule is identified as synthesizable only if can be synthesized using the USPTO reactions (Lowe, 2017) and given building blocks.
4.1 POCKET-SPECIPC OPTIMIZATION WITH GPU-ACCELERATED DOCKING
Setup. Since GFlowNets sample a large number of molecules for online training, we employed GPU-accelerated UniDock (Yu al., 2023) with Vina scoring (Trot & Olson, 2010). It is well known that docking can be hacked by increasing molecule size (Pan et al., 2003), so the appropriate constraints are required. We select QED (Bickerton et al., 2012) as comprehensive molecular prop- erty constraint, QED50.5, and set the reward function as R(x) = w1 QED(x) + w2 Vina(æ) where W1, W2 are used as the input of multi-objective GFlowNets (Jain et al., 2023) for all GFlowNets and are set to 0.5 for GeowGet baselines. Vina is a normalized docking score (Eq. (32)).
Each method generates up to 64,000 molecules for each of the 15 proteins in the LIT-PCBA dataset (Tran-Nguyen et al., 2020). We then filter the molecules with the property constraint and select the top 100 diverse candidates based on the docking score, using Tanimoto distance threshold of 0.5 to ensure structural diversity. The selected molecules are evaluated with the following metries: Hit ratio (%) measures the fraction of hits, defined as the molecules that are identified as synthesizable by AiZynthFinder and having better docking scores than known reference ligands (Lee et al., 2023). Vina (kcal/mol) measures the average docking score. Synthesizability (%) is the fraction of syn- thesizable molecules identified by AiZymthFinder. Synthetic complexity, which is highly correlated to yield and cost, is evaluated the average number of synthesis steps (Coley et al., 2018).
Baselines. We perform comparisons to various synthetic- oriented approaches: genetic algorithm (SynNet) (Gao et al., 2022b), conditional generative model (BBAR) (Seo al., 2023), and GFlowNets (SynFlowNet, RGFN) (Cretu et al., 2024; Koziarski et al., 2024). For SynFlowNet and RGFN, we used 6,000 and 350 blocks, respectively, and the maximum reaction step following the original papers. Moreover, we consider fragment-based GFlowNets (FragGFN) to analyze the effects of synthetic constraints the performance. For FragGFN, we also consider additional synthesizability objectives with commonly-used synthetic accessibility score (SA; Ertl & Schuffenhauer, 2009) (FragGFN+SA).
Results. The results for the first five targets are shown in Tables and 2, and additional results for the 10 remaining targets are reported in Sec. D.I. The property distribution for each target are re- ported in Sec. D.2. RXNFLOWW outperforms the baselines across all test proteins, demonstrating that the expanded sample space with the large action space enabled the model to generate more potent and diverse molecules. Additionally, as shown in Tables 3 and 4, RXNFLOW ensures the synthesiz- ability of the generated molecules more effectively than the other synthesis-orienter methods and GFlowNets which employ the same reactions as ours. These results support our primary assertion that existing synthesis-oriente approaches using more synthetic steps with smaller building block libraries can increase overall synthesis complexity and reduce synthesizability. Furthermore, Frag-

---

Published as a conference paper at ICLR 2025
Table 1: Hit ratio. Mean and standard deviation over 4 runs. The best results are in bold.
Hit Ratio (%, f)
Category
Method
ADRB2
ALDHI
ESR_ago _ago
_antaoo
FEN1
FragGFN
4.00 (#: 3.54)
3.75 (#: 192)
0.25 (+: 0.43)
0.25 (#: 0.43)
0.25 (#: 0.43)
Fragment
FragG
5.75 1.48)
4.00 (#: 1.58)
0.25 (E: 0.43)
0.00 (:: 0.00)
0.00 (: 0.00)
SynNet
45.83 7.22)
25.00 (:: 25.00)
0.00 0.00)
0.00 (:: 0.00)
50.00 (#: 0.00)
BBAR
21.25 (4: 5.36)
18.25 (#: 192)
3,50 ( 1.12)
2.25 (:: 1.09)
11.75 (4: 2.59)
Reaction
SynFlowNet
52.75 (#: 1.09)
57.00 (#: 6.04)
30.75 ( 10.03)
11.25 (#: 148)
53.00 (#: 8.92)
RGFN
46.75 (# 6.87)
39.75 (# 8.17)
4.50 (# 1.66)
1.25 (# 0.43)
19.75 (# 4.32)
RXNFLOW
60.25 3.77)
63.25 (#: 3.10) 71.25 (# 4.15) 46.00 (#: 7.00)
65.50 (+: 4.09)
Table 2: Vina. Mean and standard deviation over runs. The best results in bold.
Average Vina Docking Score (kcal/mol, 4
Category
Method
ADRB2
ALDHI
_ao
ESR_antago
FENI
FragGFN
-10.19 (.L. 0.33)
-10.43 (1 0.29)
-9.81 (1 0.09)
-9.85 (1 0.13)
-7.67 (LL 0.71)
Fragment
FragGFN+SA
-9.70 (# 0.61)
-9.83 (# 0.65)
-9.27 (# 0.95)
-10.06 (# 0.30)
-7.26 (# 0.10)
SynNet
-8.03 (+: 0.26)
-8.81 (# 0.21)
-8.88 (#: 0.13)
-8.52 (4: 0.16)
-6,36 (1: 0.09)
BBAR
-9.95 (#: 0.04)
-10.06 (# 0.14)
-9.97 (1: 0.03)
-9.92 0.05)
-6.84 (1: 0.07)
Reaction
SynFlowNet
-10.85 (#: 0.10)
-10.69 (# 0.09)
-10.44 (# 0.05)
-10.27 0.04)
-7.47 (#:
RGFN
-9.84 (#: 0.21)
.9.93 (# 0.11)
-9.99 (# 0.11)
-9.72 0.14)
-6.92 (#: 0.06)
RXNFLOW
-11.45 (#: 0.05)
-11.26 (#: 0.07)
-11.15 (# 0.02)
-10.77 (#: 0.04)
-7.66 (#: 0.02)
Table 3: Synthesizability. Mean and standard deviation over 4 runs. The best results are in bold.
AiZynthFinder Success Rate (%. f)
Category
Method
ADRB2
ALDHI
ESR ago
ESR antago
FEN1
FragGFN
4,00 3,54)
3.75 (# 192)
1,00 (= 1.0) 4.00 (= 2.24)
3.75 (#: 1.92)
0.25 (# 0.43)
Fragment
FragGFN+SA
5.75 (:: 1.48)
6.00 (# 2.55)
1.00 0.00)
0.00 (# 0.00)
SynNet
54.17 7.22)
50.00 (# 0.00)
50.00 (=: 0.00)
25.00 (:: 25.00)
50,00 (#: 0.00)
BBAR
21.25 5.36)
19.50 (# 3.20)
17.50 (= 1.50)
19.50
3.64)
20.00 (# 2.12)
Reaction
SynFlowNet
52.75 (#: 1.09)
57.00 (# 6.04)
53.75 (= 9.52)
56.50
2.29)
53.00 (# 8.92)
RGFN
46.75 (+: 6.86)
47.50 (# 4.06)
50.25 (= 2.17)
49.25 (# 4.38)
48.50 (#: 6.58)
RXNFLOW
60.25 (1: 3.77)
63.25 (1: 3.11)
71.25 (:) 4.15)
66.50 (1) 4.03)
65.50 (1: 4.09)
Table 4: Synthetic complexity. Mean and standard deviation over 4 runs. The best results are in
bold.
Average Number Synthesis Steps (4)
Category
Method
ADRB2
ALDHI
ESR_ago
ESR_antago
FEN1
FragGFN
3.60 (1L 0.10)
3.83 (1 0.08)
3.76 (1 0.20)
3.76 (d 0.16)
3.34 (1 0.18)
Fragment
FragGFN+SA
3.73 (t: 0.09)
3.66 0.04)
3.66 (#: 0.07)
3.67 (# 0.21)
3.79 0.19)
SynNet
3.29 (1: 0.36)
3.50 0.00)
3.00 (:: 0.00)
4.13 (d: 0.89)
3.50 0.00)
BBAR
3.60 0.17)
3.62 0.19)
3.76 (:: 0.04)
3.72 (1: 0.11)
3.59 0.14)
Reaction
SynFlowNet
2.64 (1: 0.07)
2.48 0.07)
2,60 (:: 0.25)
2.45 (#: 0.09)
2.56 0.291
RGFN
2,88 (1: 0.21)
2,65 (#: 0.09)
2.78 (+: 0.19)
2,91 (#: 0.23)
2.76 (:: 0.17)
RXNFLOW
2.42 (2: 0.23)
2.19 (4: 0.12)
1.95 (+: 0.20)
2.15 (# 0.18)
2.23 (1: 0.18)
GFN+SA does not show meaningful improvement in synthesizability, implying that optimization of a cheap synthesizability estimation function is suboptimal.
Furthermore, it is noteworthy that RxNFL.OW outperformed FragGFN which does not consider syn- thesizability. This improvement can be attributed to two key factors. First, the Enamine building block library is specifically curated for drug discovery, limiting the search space X' to drug-like chemical space and simplifying the optimization complexity. Second, RXNFLOW needs shorter tra- jectories compared FragGFNs since it assembles molecules with large building blocks instead of atoms or small fragments. This is beneficial for trajectory balance objectives where stochastic gradient variance tends to increase over longer trajectories (Malkin al., 2022).

---

Published as a conference paper at ICLR 2025
Table 5: CrossDocked2220 benchmark. We report the average and median values over the average properties for each test pocket. The best results are bold and the second ones are underlined. We denote the Generation Success Succ., Synthesizability Reference means the known binding active molecules in the test set. MolCRAFT-large (Qu et al., 2024) is the result when generating with more atoms than the reference ligands.
Synthesiz.. and Diversity Div.
Succ. (1)
Vina (4)
QED (1)
Synthesiz. (1)
Div. (1)
Time
Category
Model
Avg.
Avg.
Med.
Avg.
Med.
Avg.
Med.
Avg.
Avg.
Reference
-7.71
-7.80
0.48
0.47
36.1%
Pocket2Mol
98.
-7.60
-7.16
0.57
0,58
29.1%
22.0%
0.83
2504
TargetDiff
91.5%
-7.37
-7.56
0.49
0.49
9.9%
3.2%
0.87
3428
DecompDiff
66.0%
-8.35
-8.25
0.37
0.35
0,9%
0.0%
0.84
6189
Atom
DiftSBDD
76.0%
-6.95
-7.10
0.47
0.48
2.9%
2.0%
0.88
135
MolCRAFT
96.7%
-8.05
-8.05
0.50
0.50
16.5%
9.1%
141
MolCRAFTTlarge
70.8%
-9.25
-9,24
0.45
0.44
3.9%
0.0%
0.82
-41
Fragmeent
TacoGFN
100.0%
-8.24
-8.44
0.67
0.67
1.3%
1.0%
0.67
Reaction
RXOFLOW
100.0%
-8.85
-9.03
0.67
0.67
34.8%
34.5%
0.81
4
(a) Generated Molecules
(b) Reference Ligands
Vina: 9.3, QED: 0.49
Vina: 9.2 QED: 0.53
Vina: 8.7, QED: 0.44
Vina: 8.0. OED: 0.88
(c) Synthetic Pathway Heck Reaction
SeAR
1,2,4 1.i.4-Tiiazole
Figure 3:
Visualization of generated molecules zero-shot manner. (a-b) Docking results of generated molecules and known reference ligands of TBKI (PDB Id: IFV, SU6). (c) Generative trajectory, which is the generated synthetic pathway of the left molecule in (a).
4.2 ZERO-SHOP SAMPLING VIA POCKET CONDITIONING
Setup. We extend our works to pocket-conditional generation problem to design binders for arbi- trary pockets without additional training oracles (Ragoza et al., 2022; Liu al., 2022; Peng et al., 2022: Guan et al., 2023a;b; Schneuing et al., 2023). To address this challenge, we follow TacoGFN (Shen et al., 2023), which is fragment- based GFlowNet for pocket-conditional generation. Since requires more training oracles to learn pocket-conditional policies than target-specific genera- tion, TacoGFN used pre-trained proxies that predict docking scores against arbitrary pockets using pharmacophore representation (Sco & Kim. 2024), Since RXNFLOW explicitly considers synthesiz- ability, we exclude the SA score from the TacoGFN's reward function described in Sec. C.2.
We generate 100 molecules for each of the 100 test pockets in the CrossDocked2020 benchmark (Francoeur al., 2020) and evaluate them with the following metrics: Vina (kcal/mol) measures the average docking score from Quick (Alhossary et al., 2015). QED measures the average drug- likeness of molecules. Synthesizability (%) is the fraction of synthesizable molecules. Diversity measures an average pairwise Tanimoto distance of ECFP4 fingerprints. Moreover, we report the Generation Success (%) which is the percentage of unique RDKit-readable molecules without disconnected parts, and Time (sec.) which is the average sampling time to generate 100 molccules.
Baselines. We compare RXNFLOW with state-of-the-art distribution learning on synthesizable drug set: an autoregressive model (Pocket2Mol (Peng et al., 2022)), diffusion models (TargetDiff (Guan et al., 2023a), DifiSBDD (Schneuing et al., 2023), DecompDiff' (Guan et al., 2023b)), and bayesian flow network (MolCRAFT (Qu al., 2024)). We also perform comparison with an optimization aaased TacoGFN (Shen et al., 2023). For a fair comparison with the distribution learning- approaches, used the docking proxy trained on CrossDocked20200.
models trained
8

---

Published as a conference paper at ICLR 2025
B
30
1.0
Vina (kcal/mol)
150
0.8
-6
TPST
0.6
-8
0.4
0.2
0.0
all low
low
all
seen unseen all seen unseen all
Figure Property distribution of sampled molecules with "all" building blocks and "low"- TPSA building blocks. Vina score was calculated against the KRAS-GI2C target.
QED reward distribution of generated molecules for each of the "seen". "unseen", and "all" blocks. Ad- ditional results are in Figure 14.
Figure 5:
Result. As shown Table 5, RXNFLOW achieves significant improvements in drug-related prop- erties. In particular, RXNFLOW outperforms the docking score for TacoGFN and attains high drug- likeness while showing a competitive docking score with the state-of-the-art model, MolCRAFT: large. Moreover, RXNFLOW ensures the synthesizability comparable outperforming the fragment-based TacoGFN trained on the SA score objective and the distribu- tional learning-based models trained synthesizable drug molecules. Figure 3 illustrates generated molecules against TANK bindding kinase (TBKI) which is not included in the training set.
known active ligands,
An important finding is that RXNFLOW maintains high structural diversity (0.81) despite the typ- ical trade-off between optimization power and diversity (Gao et al., 2022a). This is a significant improvement over the fragment- TacoGFN, which scored 0.67 and is comparable to the distri- butional learning- based models that range from 0.83 to 0.87. We attribute this enhancement to our action space, which contains chemically diverse building blocks, contrast to small and lim- ited fragment sets used in fragment-based GFlowNets. This suggests that our model can effectively balance the potency and diversity of generated molecules.
4.3 INTRODUCING ADDITIONAL OBJECTIVE WITHOUT RETRAINING
In drug discovery, new objectives often arise during the research process, such enhancing solubil- ity, reducing toxicity, or improving selectivity (Fink et al., 2022; Joshi et al., 2021). These additional objectives typically only require retraining models but also increase the optimization complex- ity. In this context, RXNFLOW can achieve some additional objectives by simply introducing con- straints MDP without retraining thanks to the non- non-hierarchical action space (Sec. 3.4). As shown in Figure 4, we explore the scenario of adding a solubility objective to pre-trained GFlowNet in Sec. 4.2. Specifically, we target the generation of hydrophobic molecules by restricting the build- ing blocks with topological polar surface area (TPSA) in the bottom 15% ("low") and sampled 500 molccules for the KRAS-GI2E mutant (PDB Id: 6oim). While there are slight differences in the QED distributions due to the correlation between TPSA and QED, the generated molecules are more hydrophobic and retain similar overall reward distributions. We also performed the ablation study of non-hierarchical MDP in Sec. D.6.
4.4 SCALING ACTION SPACE WITHOUT RETRAINING
The building block libraries for drug discovery continue to grow, from 60,000 in 2020 to over 1.2 million blocks today, to enhance chemical diversity and novelty (Grygorenko al., 2020). However, existing generative models require retraining to accommodate newly discovered building blocks, limiting their scalability and adaptability. On the other hand, RXNFLOW can integrate new building blocks without retraining by understanding the chemical context of actions through action embed- ding. We first divide the IM-sized block library ("all") into two sets: 500,000 blocks for training ("seen") and the remaining blocks ("unseen"). After training with the QED objective various reward exponent settings (R*), we generate 5,000 molecules from each set ("seen", "unseen". and "all"). Figure 5 shows that the reward distributions of samples are nearly identical, demonstrating that RXNFLOW performs robustly with unseen building blocks. This result highlights the general- ization ability and scalability of RXNFLOW, significant advantage for real-worle applications.

---

Published as a conference paper at ICLR 2025
(+) -11.1
(b)
Structural Diversity E
(d),
1.0
Scaffoid Diversity
Vina (kcalmoll
-11.2
0.8
0.79
Time (sec)
0.78
-11.3
0.6
0.77
-11.4-
0.4
-11.5
10a 10 10*
10 10
10 10a
10a 10" 10
10a 102 10 10 10*
10" 10 10" 10" 10
Num Building Blocks
Numn Building Blocks
Nurn Building Blacks
Numn Building Blocks
Figure 6: Optimization power, diversity, and generation time according to building block library size. (a-c) Average and standard deviation of properties of the top-1000 high-affinity molecules over 4 runs pocket-specific generation. (a) Average docking score. (b) The uniqueness of Bemis- Murcko scaffolds. (c) Average Tanimoto distance.
(d) Average runtime to generate 100 molecules over the CrossDocked2020 test pockets in zero-shot manner.
4.5 ABLATION STUDY: THE EFFECT OF BUILDING BLOCK LIBRARY SIZE
Expanding the size of building block libraries provides an opportunity to discover more diverse and potent drug candidates (Grygorenko et al., 2020). In Sec. 4.1, RXNFL.Ow outperforms Syn- FlowNet and RGFN which use smaller block libraries, but differences in model architectures may have contributed to these results. To isolate the effect of the building block library size, we conduct an ablation study using partial libraries with pocket-specific optimization task against the kappa- opioid receptor (PDB Id: 6b73), as illustrated in Figure 6(a-c). The results indicate that increasing the library size enhances both optimization power, in terms of docking scores, and diversity, in terms of a higher number of unique Tanimoto diversity of the generated molecules. Additionally, as shown in Figure 6(d), the genera- tion time only doubles on the 100-fold larger action space, highlighting the efficiency of RxNFLOW. These results demonstrate the forte of RXNFLOW navigating broader chemical space discover novel drug candidates by overcoming the computational limitations for expanding the action space. We also investigated the scaling laws with other reastion-based GFlowNets in Sec. D.3.
scaffolds (Bemis & Murcko, 1996) and an increased
4.6 TOWARDS FURTHER DEVELOPMENT
Top 100 Vina (kcal/mol)
-11
Our framework has room for improvement regarding more efficient learning and sampling. First, the 3D interaction modeling can en- hance the generative performance. Given the high correlation be- tween binding affinity and conformation, such spatial relationships
2D
3D
-12
could provide meaningful information for our generative model
make a reasonable decision. In Figure 7, we observed that the 3D interaction modeling using docking conformations boosts the dis- covery of candidate molecules against the beda-e-adren recep- tor. Second, the current action space subsampling method forces exploration due to uniform sampling to minimize bias. We can en- hance the exploitation by prioritizing the building blocks of action space subsampling instead of uniform subsampling.
-13
500
1000
Training Step
Figure 7: Affect of 3D inter- action modeling. Mean and standard error over 3 seeds
5 CONCLUSION
In this work, we introduce RXNFLOW, synthesis-oriented generative framework designed to ex- plore broader chemical spaces, thereby enhancing both diversity and potency for drug discovery. Our framework efficiently handles massive action spaces to expand the search space without signifi- cant computational or memory overhead by employing a novel action space subsampling technique. effectively identify diverse drug candidates with desired properties and synthetic feasibility by learning the objective of generative flow networks on synthetic pathways. Addition- hon-hierarchical MDP, RXNFLOW can model generative flows modified action spaces, allowing it to achieve additional objectives and incorporate newly discovered build- ing blocks without retraining. These results highlight the potential of RXNFLOW practical and versatile solution for world drug discovery.
RxNFL.
ally, by formulating
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was supported by Basic Science Research Programs through the National Re- search Foundation of Korea (NRF), grant-funded by the Ministry of Science and ICT (NRF- 2023R I IA2C200336, was supported by a grant Korean ARPA-H Project through the Korea Health Industry Develop- ment Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (RS-2024- 00512498).
11-2022MM3JA106330022
RS-2023-00257479, In addition, this research
REFERENCES
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):221--2216 2015.
Guy W Bemis and Mark A Mureko. The properties of known drugs. 1. molecular frameworks. Journal of medicinal chemistry, 39(15):2887-289) 1996.
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381-27394, 2021.
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J Hu, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. The Journal of Machine Learning Research, 24(1):10006-1000) 10060, 2023.
Richard Bickerton, Gaia Paolini, Jérémy Besnard, Sorel Muresan, and Andrew Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
John Bradshaw, Brooks Paige, Matt Kusner, Marwin Segler, and José Miguel Hernández-Lobato. A model to search for synthesizable molecules. Advances in Neural Information Processing Systems, 32, 2019.
Alexander Button, Daniel Merk, Jan A Hiss, and Gisbert Schneider. Automated de novo molecu- lar design by hybrid machine intelligence and rule-driven chemical synthesis. Nature machine intelligence, 1(7):307-315, 2019.
Connor W Coley, Luke Rogers, William H Green, and Klavs F Jensen. Sescore: synthetic com- plexity learned from a reaction corpus. Journal of chemical information and modeling, 58(2): 252-261, 2018
Miruna Cretu, Charles Harris, Julien Roy, Emmanuel Bengio. and Pietro Lio. Synflownet: Towards molecule design with guaranteed synthesis pathways. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024. URL
Gabriel Dulac-Arnodd Richard Evans. Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin. Deep rein- forcement learning in large discrete action spaces. arXiv preprint arXiv: 1512.07679, 2015.
Joseph L Durant, Burton A Leland, Douglas R Henry, and James G Nourse. Reoptimization of mdl keys for use in drug discovery. Journal of chemical information and computer sciences, 42(6): 1773-1280, 2002.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1:1-14 11,11,,200 2009.
Elissa A Fink, Jun Xu, Harald Hübner, Joao M Braz, Philipp Seemann, Charlotte Avet, Veronica Craik, Dorothee Weikert, Maximilian F Schmidt, Chase M Webb, et al. Structure- dis- covery nonopioid analgesics acting through the a2a-adrenc receptor. Science, 377(6614): eabn7065, 2022.
11

---

Published as a conference paper at ICLR 2025
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. Three-dimensional convolutional neural networks and a cross-docked data set for structure- based drug design. Journal of chemical information and modeling, 60(9))4200- 4215, 2020.
Bowen Gao, Minsi Ren. Yuyan Ni, Yanwen Huang, Bo Qiang, Zhi-Ming Ma, Wei- Ying Ma, and Yanyan Lan. Rethinking specificity in SBDD: Leveraging delta score and energy-guided dif- fusion. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller. Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Con- ference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 14811-14825, PMLR, 21-27 Jul 2024a. URL https: https::/ppoceeeeeeee /ptpp/eeeoccceg.eeee/ee/e/c/..ee wwt..mmlr....................me. press/ http:/aaa44..ht...
Wenhao Gao and Connor W Coley. The synthesizability of molecules proposed by generative mod- els. Journal of chemical information and modeling, 6011):5714-5723, 2020.
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. Sample efficiency matters: benchmark for practical molecular optimization. Advances in neural information processing systems, 35: 21342-21357, 2022a.
Wenhao Gao, Rocío Mercado, and Connor W. Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. In International Conference on Learning Representations, 2022b. URL https: opwnreviewwweeeeeeeeeeeeee.ee..e
Wenhao Gao, Shitong Luo, and Connor W Coley. Generative artificial intelligence for navigating synthesizable chemical space. arXiv preprint anXiv:2410003494,200000 2024b.
Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Esben Bjerrum. Aizynthfinder: fast, robust and flexible open-source software for retrosynthetic planning. Journal of cheminformaties, 12(1):70, 2020.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In International conference on machine learning, pp. 3668-3679, PMLR, 2020.
David E Graff, Eugene Shakhnovich, and Connor W Coley. Accelerating high-throughput virtual screening through molecular pool-based active learning. Chemical science, 12(22):7866-788) 2021.
Oleksandr O Grygorenko, Dmytro S Radchenko, Igor Dziuba, Alexander Chuprina, Kateryna E Gu- bina, and Yurii Moroz. Generating multibillion chemical space of readily accessible screening compounds. Iscience, 23(11), 2020.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffuuson for target-aware molecule generation and affinity prediction. In The Eleventh Interna- tional Conference on Learning Representations, 2023a. URL https: / http:///ppeeeeeeeeeeeeeeeeeeeeee n Corum?id=kJqhe XMM...
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang. Yu Bao, Jian Peng. Jianzhu Ma. Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: Diffusson models with decomposed priors for structure-based drug design. ICML, 2023b.
Markus Hartenfeller, Heiko Zettl, Miriam Walter, Matthias Rupp, Felix Reisen, Ewgenij Proschak, Sascha Weggen, Holger Stark, and Gisbert Schneider. Dogs: reaction-driven de novo design of bioactive compounds. PLoS computational biology, 802)xe 2012.
Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS omega, 5(51) (151):329443399999 2020.
Ruth Huey, Garrett M Morris, Stefano Forli, al. Using autodock 4 and autodock vina with autodocktools: a tutorial. The Scripps Research Institute Molecular Graphics Laboratory, 10550 (92037):1000, 2012.
12

---

Published as a conference paper at ICLR 2025
Moksh Jain, Emmanuel Bengio, Alex Grmmandez-Gariii Jarrid Rector- Dector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp. 9786-9801. PMLR, 2022.
Moksh Jain, Sharath Chandra Raparthy, Alex Hernández- Jarrid Rector-Brooks, Yoshua Ben- gio, Santiago Miret, and Emmanuel Bengio. Multi-objectivve gflownets. In International confer- ence on machine learning. pp.
PMLR, 2023.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. International Conference on Learning Representations, 2020.
Tanuja Joshi, Priyanka Sharma, Tushar Joshi, Hemlata Pundir, Shalini Mathpal, and Subhash Chan- dra. Structure-based screening of novel lichen compounds against sars coronavirus main protease (mpro) as potentials inhibitors of covid-19, Molecular Diversity, 25:1665-16777
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool. Russ Bates, Augustin Zidek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-555 2021.
Hycongwoo Kim, Kyunghoon Lee, Chansu Kim, Jaechang Lim, and Woo Youn Kim. Dfrscore: deep learning-based scoring of synthetic complexity with frrusfo retrosynthetic analysis for high-throughput virtual screening. Journal of Chemical Information and Modeling, 64(7): 2432 2444, 2023.
Michal Koziarski, Andrei Rekesh, Dmytro Shevehuk, Almer van der Sloot, Piotr Gainski, Yoshua Bengio, Cheng-Hai -ao Liu, Mike Tyers, and Robert Batey, Rgfn: Synthesizable molecular gener- ation using gflownets. arXiv preprint arXiv:2406.088. 2024.
Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8(31.10):2811, 2013.
Seul Lee, Jachyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of- distribution generation. In International Conference Machine Learning, pp. 18872-18892. PMLR, 2023.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding- arXiv preprint arXiv:2204.089 22902294440000 2022.
Zhihai Liu, Yan Li, Li Han, Jie Li, Jie Liu, Zhixiong Zhao, Wei Nie, Yuchen Liu, and Renxiao Wang. Pdb-wide collection of binding data: current status of the pdbbind database. Bioinformaties, 31 (3):405-412, 2015.
Daniel Lowe. Chemical reactions from US patents (1976-8ep2016) 2017. URL
https: http::///iishaeeeeeee com articles /ataaatt ceteee///tceaccleeetttiooooo US
wwacteeacinnee.. from
/5114887.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229-6239, 2021.
Shitong Luo, Wenhao Gao, Zuofan Wu, Jian Peng, Connor W. Coley, and Jianzhu Ma. Projecting molecules into synthesizable chemical spaces. In Forty-first International Conference on Machine Learning, 2024. URL https: hat:::///ppeeeee
wpwpr/veeeeeeeeeeeeeeeeeeeeoe.ee net forum? ?id=scF 1Ftdd111..........
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955-5967, 2022.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed representa- tions of words and phrases and their compositionality, 2013.
13

---

Published as a conference paper at ICLR 2025
Harry L Morgan. The generation of unique machine description for chemical structures-a tech- nique developed at chemical abstracts service. Journal of chemical documentation, 5(2):107-113, 1965.
Trang Nguyen, Alexander Tong, Kanika Madan, Yoshua Bengio, and Dianbo Liu. Causal inference in gene regulatory networks with gflownet: Towards scalability in large systems. arXiv preprint arXiv:2310.03572 00.0977, 2023a.
Tri Minh Nguyen, Sherif Abdulkader Tawfik, Truyen Tran, Sunil Gupta, Santu Rana, and Svetha Venkatesh. Hierarchical gflownet for crystal structure generation. AI for Accelerated Materials Design-NeurIPS 2023 Workshop, 2023b.
Noel M O' O'Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geof- frey R Hutchison. Open babel: An open chemical toolbox. Journal of cheminformaties, 3:1-14, 2011.
Yongping Pan, Niu Huang, Sam Cho, and Alexander D Mackerell. Consideration of molecular weight during compound selection in virtual target-based database screening. Journal of chemical information and computer sciences, 43(1):267-272, 2003.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Effi- cient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning. pp. 17644-17655 PMLR, 2022.
Yanru Qu, Keyue Qiu, Yuxuan Song, Jingjing Gong, Jiawei Han, Mingyue Zheng, Hao Zhou, and Wei-Ying Ma. MoICRAFT: Structure-based drug design in continuous parameter space. In Ruslan Salakhutdinov. Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learn- ing, volume 235 Proceedings of Machine Learning Research, pp. Jul 2024. URL www.thtpt/::tom...tm http://prroeeeee i ngs. m1 press v235/ qu24//uu24: html.
PMLR, 21-27
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 13(9):2701-2713, 2022.
Danny Reidenbach. Evosbdd: Latent evolution for accurate and efficient structure-based drug de- sign. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024.
Arman A Sadybekov, Anastasiia V Sadybekov, Yongfeng Liu, Christos Iliopoulos- lliopoulos-ssouusouvvaa Xi- Ping Huang, Julie Pickett, Blake Houser, Nilkanth Patel, Ngan Tran, Fei Tong, et al. Synthon- based ligand discovery in virtual libraries of over 11 billion compounds. Nature, 601(7)93): 452-459, 2022.
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pictro Lió. Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia. Structure-bass drug design with equivariant diffusion models, 2023.
Seonghwan Seo and Woo Youn Kim. PharmacoNet: deep learning- pharmacophore modeling for ultra- virtual sereening. Chemical Science, 15(46):19473-1944) 2024. ISSN 2041- 6520. doi: 10. 10.0039/44c04854g. PharmacoNet.
Seonghwan Seo, Jaechang Lim, and Woo Youn Kim. Molecular generative model via retrosynthet- ically prepared chemical building block assembly. Advanced Science, 10(8):2206674, 2023.
Tony Shen, Sconghwan Sco, Grayson Lec, Mohit Pandey, Jason R Smith, Artem Cherkasov, Woo Youn Kim, and Martin Ester. TacoGFN: Target-conditioned GFlowNet for Structure-based TacoGFN.
Drug Design. arXiv, 2023. doi: 10.485500/mxiv.231000332
Viet-Khoa Tran-Nguyen, Célien Jacquemard, and Didier Rognan. Lit-peba: unbiased data set for machine learning and virtual screening. Journal of chemical information and modeling, 60(9): 4263 2020.
Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):455-4611 2010.
14

---

Published as a conference paper at ICLR 2025
Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, et al. Amortizing intractable in- ference in diffusion models for vision, language. and control. arXiv preprint arXiv:2405.20971, 2024.
Shuzhe Wang, Jagna Witek, Gregory A Landrum, and Sereina Riniker. Improving conformer genera- tion for small rings and macrocycles based on distance geometry and experimental torsional-angle preferences. Journal of chemical information and modeling. 60(4):2044- 2058, 2020.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning. 8:229-256, 1992.
Yuejiang Yu, Chun Cai, Jiayue Wang, Zonghua Bo, Zhengdan Zhu, and Hang Zheng. Uni-dock: Gpu-accelerated docking enables ultralarge virtual screening. Journal of chemical theory and computation, 19(11):33 19(11)333363335, 2023.
Seongjun Yun, Minbyul Jeong, Sungdong Yoo, Seunghun Lee, S Yi Sean, Rachyun Kim, Jaewoo Kang, and Hyunwoo Kim. Graph transformer networks: Learning meta-path graphs improve gnns. Neural Nehworks, 153:104-1199 2022.
Alex Zhavoronkov, Yan A Ivanenkov, Alex Aliper. Mark S Veselov, Vladimir A Aladinskiy, Anas- tasiya Aladinskaya, Victor Terentiev, Daniil A Polykovskiy, Maksim Kuznetsov, Arip Asadulaev, et al. Deep learning enables rapid identification of potent ddr1 kinase inhibitors. Na- 2019.
ture biotechnology,
Mingyang Zhou, Zichao Yan, Elliot Layne, Nikolay Malkin, Dinghuai Zhang, Moksh Jain, Mathieu Blanchette, and Yoshua Bengio. Phylogfn: Phylogenetic inference with generative flow networks. arXiv preprint arXiv:2310.08774, 2023.
Wonho Zhung. Hyeongwoo Kim, and Woo Youn Kim. 3d molecular generative framework for interaction-guided drug design. Nature Communications, 15(1):2688, 2024.
15

---

Published as a conference paper at ICLR 2025
A THEORETICAL ANALYSIS
In this section, we provide the theoretical background action subsampling. define as uniform subsampling policy, i.e., B Uniform( (B" C B 8 k)).
Bias of log forward policy. In this section, we prove that the log forward policy estimation is unbiased (Eq. (7)) with the following weights:
|B//181| if E B if is (r, 6) and r R₂ if Stop E Ri
|B,1/1851
For readability, we express the edge flow and forward policy Pe where the action a is + s as follows:
Fo O PF(s'|s;0)
a;
Pr(als;0)
Then, the forward policy PF and the estimated policy PF (Eq. (7)) can be rewritten as follows:
Pf(s'|s;0) PF(als; 0) Fo(s,a)
Fe(s,a)
Fo(s)
Emee((s) Fo(s,a")
(0; = Ar; = Fo(s,a).
Po(s,a)
Po(s;
Eelest(s) We Fo
The expectation of the estimated initial state flow Fo(so;A²)
EoEs Fo a) is given by
BAw-PiaaPP000: Ar)] = EB-~u((B)
Fo(so, b)
Σ
Po(so, Fo(so). (13)
1
BEB-
BEB
For a later state s + s0, the expectation of the state flow A") is given by
BB
Σ
Fels, a) + Σ EBB
Σ (r.s,(r,b))
B
(ERoU(StoP)
rERg
BEB;
Σ
Fo(s,a) + 3
Σ
Fo(s, (r, b))
(ELOU(
rER, bES.
PF(s'|s;0)
(14)
Variance of log forward policy. We define the standard deviation of the probability Pr(-Is; (o(--s), For the initial state so, the variance of log Pr(s' (80; A"; 0) is given by
Var Var [log 100 A3]
IBY JB8-
Fo (so, b) ]
Varg
log Egto 48) log
Σ
DEB
-Varg=>1(B))
log Σ Pr(blss;
(by normalizing with (oo)))
DEB
|B| 1B*1)
a -180)
(by Eq. (24))
(15)
[B 1)
16

---

Published as a conference paper at ICLR 2025
For the later state + so, the variance of log Ar: is:
(-PA)))((A log Pr(s'ss; Ar]
B+
PPAA)-P((A)
log Σ
Pr(als:0) + Σ
Pr (r((r,b)(s;;)
B
rERa
BEB;
VaAA-~P(A))
E PF(a(s;0) LE
EoeB: PF((r, (p((r,b)(s:))
[8,1
BA-AP(A)
EnERiL((Soo) Pr(als; ErERa
|8 [871
Every Pr((r; b)(s; 0)
B
Var(-~((A(+))
Σ
Σ Pr(((,,)]ss;0
1
Σ EuSAto PF(a|s:0)
ER
bEB;
(IB 13
|B; (1B, 1) 00 (0 -)88) (by Eq. (22))
(16)
PER
Finally, we get the variance of log PF(T; 0) where T = (so + - Sn = x):
VarA*~P(A)) log (r;
( [B')) 1:
n-1
[B, - (3,1) 18 -1) (o((r, -)))))
1 B 1) ge(-150) +
(17)
t=1 FER
Expectation of trajectory balance loss. Due to the variance of forward policy, there is a bias of the trajectory balance loss equal to the variance of log (T):
(ewpP(a)(TaTT))
=EA-NP(A) log Zo (T:00
?
ZePr ZoPa(T:0) log R(x)PB(T(T)
VarA-)P(A)
PIx))
[TB(T) + Var PP(A) log (F(
(18)
LTT(T)+ + [B| 13 [B+[([]] - 1) (-(=-10))
n-1
[I |Bt1) [B - 1)
(r: ():-)))s)²
(19)
reRa
Therefore, the gradient of trajectory balance loss is:
E AAaA((A)
VoL(B(F)]
( 1B*)) V Vooa
B, ((B,
[3 B: (BB 1)
~ VTLTB(T)+ +
(-1o))
+ 1=1
(r)(((r-))ss))
18 (
TER2
(20)
Given that o-(-), which is the standard deviation of probability select actions, is inversely pro- portional to [5] or [B,], the bias highly dependent on the subsampling size (e.g. (B*) rather than the subsampling ratio (e.g. [5*T 1/B). Moreover, flow estimation, and the bias is reversely proportional to the number of samples k. However, in the same computational cost (proportional to O(k Σ, [B,)),, using the larger partial action spaces without MC sampling is more precise than using smaller partial action spaces with multiple MC samples. In Sec. D.8, we expermentally show that the bias is relatively small to trajectory balance loss with the toy experiment.
decrease the bias by MC sampling for state
17

---

Published as a conference paper at ICLR 2025
A.1 THEORETICAL BACKGROUNDS
Variance of uniformly sampled subset For the set X' with a size of n, we define its uniformly sampled subset with size m X" P(X). For the function F(X) Exex (x), we define the unbiased estimation:
F(X)
n - F(X")
71
Σ f(x)
m
m ZEX"
When the mean and standard deviation of f(x) is μy and o, variance of ((x) is:
F(X)
(P(x)
-Ex-p(X2)
[((x)]] Ex)xp(2)
[(x2)]
2
Σ
m Σ 1(x)
FF(2)2
nOm XEP((X
f(b:)) nn Cm-2 ((00)100))) - -(I
1 n²
n- Cm-1
"C m
==1
i=1 fol
TE
n
f(x,)2+ + 2n(m- - 1) fzx)f(z)) Σ f(x))² + > Σ f()))((s)))
m(n 1) i= joc
n - m
(n- Σ fxx))² - (x))(aa))
m(n - 1)
i=1
538
n n2
Σ (f(x;) - f(x,))2
m(nn - 1)
53i
n²(n m))
m(n 1) oj
(21)
For MC sampling with samples, the variance is:
Varx=~((2))
[(()]]
n n2(n m)
kmn(n 1) 07
(22)
The variance of log F(X) is:
Varx>~(()) F(x2)
[og [X)) ~
(n - m) km 1) (o5/
Varx=~p(X)
(23)
Ex [(x)]
2
When the F(X) = 1, the variance is:
Varx* [log M(23]
m²(n m)
km(n 0j
(24)
18

---

Published as a conference paper at ICLR 2025
B RXNFLOW ARCHITECTURE
B.1 FORWARD POLICY
We use the model architecture inspired from Cretu et al. (2024); Koziarski et al. (2024). We used graph transformer (Yun et al., 2022) as the backbone fo following Bengio et al. (2021) and a multi- layer perceptron (MLP) for action embedding 90- The graph embedding dimension is d1, and the building block embedding dimension is d2. The molecular graph for a state 8, and the GFlowNet condition vector is с which includes a reward exponent and multi-objective optimization weights (Jain et al., 2023). alb means the concatenation of two feature vectors a and b.
Initial block selection. For the first action, the model always selects AddFi Reaccttantt tion which selects b E B for the starting molecule with MLP M..PaaarrseecRocccan: Rdi+d -> IR.
(s b, c) MLP rstReactnn (Jo(so. c))lgo(b))
(25)
Reaction selection. For the later states + Sp. the model calculates the logits for the Stop action, ReactUni actions 11 E R1, and ReactBi actions (r2, b) E R2 x B.
The logit for Stop is calculated by MLP st.p R R: Fo(s,
MLPstep (fo(s, c)).
(26)
The logit for React action T1 E R1 is calculated by MLP deactuns :R4 - R:
(,(s,rie)) MIP" Reacturs (Jo(s,c))
(27)
Finally, the logit for a ReactBi action (r2, is calculated by the one-hot embedding of reaction template 6(r2) {0, 1) I) and MI.PReactes RRdt+/Ral+de + R:: Fo(s, (r2, b), c) = MLPseactes (fo(s, c)ll
(28)
Pocket conditioning. For pocket-conditional generation, the model uses K-NN pocket residual graph go and encodes GVP-GNN (Jing et al., 2020) according to Shen al. (2023). The pocket conditions are included in the GFlowNet condition vector
3D interaction modeling. Instead of using 2D molecular graph of the ligand, the model can utilize 3D binding complex graph to represent the state as the input of the graph transformer backbone fo- In this graph, each ligand atom connects to its K₁ nearest protein atoms as well as to its neighhoring ligand atoms, while each protein atom connects to its K₂ nearest protein atoms. The edges encode spatial relationship information between the connected nodes.
Furthermore, we lightweight the MDP formulation to implement computationally intensive 3D in- teraction modeling with modified layers MLP MLP AddPics:Reactent Rdd Id and MLP Idd++Ral R2l. + d2 as follows:
ReactBi :
Fa b, c) = MLPRedrirstReacee (Jo(s) c)) o (a)), Fo (T2, b) MLP t (f(ss))))())) (b)),
(29)
(30)
B.2 BACKWARD POLICY.
Malkin al. (2022) introduced uniform backward policy for small-scale drug discovery. However, directed acyelic graph (DAG) on synthetic pathways, where each state has numerous outgoing edges but few incoming ones, there is significant imbalance in the number of trajectories along each incoming edge, depending on the distance from the initial state. In a uniform backward policy, where the flow of all incoming edges is equal, this imbalance diminishes the flow of trajectories that reach a state via shorter routes, i.e., those with fewer reaction steps. To facilitate shorter synthetic pathways as trajectories, we set the backward transition probability proportional the expected number of trajectories along each incoming edge of the state, a :" +
ErET:r
((ss)
Ps(s"|s) P((s" E T()
(31)
EreT A(s) N-1T
where N is the maximum length of trajectories.
19

---

Published as a conference paper at ICLR 2025
B.3 BUILDING BLOCK REPRESENTATION FOR ACTION EMBEDDING.
To represent building blocks, we used both physicochemical properties, chemical structural prop- erties, and topological properties. For physicochemical properties, we used molecular features: molecular weight, the number of atoms, the number of H-bond acceptors/donors (HBA, HBD), the number of aromatic/non-arrmate rings, LogP, and TPSA. For chemical properties, we used the MACCS fingerprint (Durant et al., 2002), which represents the composition of the chemical func- tional groups in the molecule. For topological information, we used the Morgan ECFP4 fingerprint (Morgan, 1965) with a dimension of 1024, which is widely used in fingerprint-based deep learning researches. All properties are calculated with RDKit (Landrum et al., 2013).
B.4 GFLOWNET TRAINING.
The training algorithm with the trajectory balance objective is described at Algorithm 1. The nota- tions are Sec. 3.3.
Algorithm 1 Training GFlowNets with action space subsampling
1: input Entire action space A, Maximum trajectory length N
2: repeat
3: Sample partial action spaces Aj. Ai. .... A from subsampling policy P(AA)
4: Sample trajectory т from sampling policy No( Ao(-1-AA(-))
5: Update model <- 0 (T)))
6:
until model converges
B.5 COMPARISON BETWEEN THREE GFLOWNET METHODS
Table 6: Comparison with SynFlowNet and RGFN. If the method includes each technique, it is denoted by an O otherwise by an x. RGFN investigates scaling up to 64,000 building blocks, but their experimental validation and -ernn-cp-priiipleeeee implementations use only 350.
Methods
Hierarchical Action Embedding ReactUni
Num Blocks
Max Reaction Steps
Fnamine REAL
>M
RGFN
xxo0
350-64.000
SynFlowNer(y2024.55
6,000
SynFHowNNe(yy224410) RXNFLOW
10,000-220.000
3/4
IM
While RXNFLOW shares similar rules with SynFlowNet (Cretu et al., 2024) and RGFN (Koziarski et al., 2024), there are several major differences as described in Table 6. Since there are two versions of SynFlowNet, the version presented at the ICLR 2024 GEM workshop and the version updated on October 16, 2024, we refer to the updated version SynFlow/Ne(v20224.0) For the benchmark studies, we use the workshop version of SynFlowNe(y2002..55)
Backward Trajectory. In contrast to the simple fragment-based or atom-based GFlowNet, some backward transitions do not reach the initial state 8p- While RGFN and SynFlowNet do not con- sider the invalid transitions, SynFlowNer(uppd) and RXNFLOW address this issue. To prevent the invalid backward transition, SynFlowNe((upd) introduces additional maximum likelihood and RE- INFORCE (Williams, 1992) training objectives for parameterized backward policy to prefer the backward transitions which can reach the initial state. In contrast, we implement explicit retrosyn- thetic analysis within maximum reaction steps for each state to collect valid backward trajectories.
Flow Network Framework. In contrast to SynFlowNet which uses discrete action space, RGFN, SynFlowNet(upd), and RXNFLOW include the action embedding (Dulac-Arnold et al., 2015) for chemical building blocks. While every building block must be sampled and trained at least once to prevent unsafe actions in a discrete action space, action embedding expresses actions based on structural information-domall knowledge. In particular, we formulate the non-hierarchical MDP instead of the hierarchical MDP for bi-molecular reactions. is adaptable to modified building block libraries described in Sec. B.6 but increases the computational cost and memory require- ment dramatically. Thanks to action space subsampling, RxNFLOW can use non non-hienenn hicrarchical MDP without computational burden problem.
20

---

Published as a conference paper at ICLR 2025
Sampling algorithm. For each forward transition step, RGFN and SynFlowNet (both versions) model the flow for the entire action space to estimate the forward transition probability. Therefore, the computational cost and memory requirement is proportional to the number of using building blocks B, i.e., (( ). Instead of modeling all actions, we propose an action space subsampling technique, which is similar to the negative sampling technique (Mikolov al., 2013) in natural language processing, to estimate the forward transition probability from the subset of action space. It makes a cost-variance trade-off to decrease the computational cost complexity to (r)))), where r is subsampling ratio. As a result, RXNFLOW can handle millions of building blocks on training and inference with the computational cost of handling 10,000 building blocks.
B.6 NON-IHERARCIICAA MARKOV DECISION PROCESS
To describe the difference, we supposed the situation that the toxicity is observed in molecules containing a functional group -NR3, thereby excluding blocks with the function group from the building block library. In trained hierarchical MDP, the modification of the block library cannot change the probability of reaction templates, leading to the overestimation or underestimation of edge flows (the red color of Figure 8(a)). However, in the non-hierarchical MDP, the exclusion of some reactants does not affect actions that share the same reaction template (Figure 8(b)).
(a)
Stop
20
Stop
20
ReactLini Acad lommlion
20
ReactLini Agd
R
80
femation
Disallow -NR,
Hy- Mgg
Fig Ng1g
200
OH
R
OH
200
20
200
200
ReactBi
30
R2
ReactBi
H
70
Re
Ry
RR
(b)
Stop
Stop
20
20
ReactUni Asii Chronida Formation
80
ReactUni Ae Chiir du Formation
R
R
20
ReactBi Arrication Rg- NPx
20
Disallow -NR.
0
20
ReactBi Arridttinn Rg-NH
30
ReactBi Amicction Fg NN's
ReactBi Amiiatin Ry N0g
30
70
800
Reactii Amicasion p. Re
ReactBi Amidsson ng Pe
fl
70
ReactBl Amiicttion
ReactBl
80
R
Arridatonn
R
R.
Figure 8: Illustration of a situation where an additional objective is introduced: excluding building blocks (reactants) containing the functional group -NR3- The Gray dashed lines mean masked actions. (a) GelowNet on hierarchical MDP. (b) GFlowNet on non- non-hierarchical MDP.
21

---

Published as a conference paper at ICLR 2025
C EXPERIMENTAL DETAILS
C.I ACTION SPACE DETAILS
Reaction templates. In this work, used the reaction template set constructed by Cretu et al. (2024) from two public collections Hartenfeller et al. (2012); Button et al. (2019). The entire reaction template set includes 71 reaction templates, 13 for uni-molecular reactions and 58 for bi-molecular reactions. In in silico reactions with bi-molecular reaction templates, the products depend on the order of the input reactants. To ensure the consistency of the action, we consider two templates for each bi-molecular template according to the order of reactants, i.e. [Ri] 13, RR2 = 116. We note that our template set does not contain templates that have the same first and second reactant patterns.
Building blocks. We used the Enamine comprehensive catalog with 1,309,385 building blocks released on 2024.06.10 10 (Grygorenko et al., 2020). We filtered out building blocks that are not RDKit-readable, have possible reactions, or contain unallowable atoms', resulting in 1,193,871 remaining blocks.
C.2 GFLOWNET TRAINING DETAILS
To minimize optimization performance influencing, factors, we mostly followed the standard GFlowNet's model architecture and hyperparameters* except for some parameters in Table 7. All experiments were performed on a single NVIDIA RTX A4000 GPU.
Table 7: Default hyperparameters used in RXNFLOW training.
Hyperparameters
Values
Minimum trajectory length Maximum trajectory length
2 (minimum reaction steps: 1)
(maximum reaction steps: 3)
GFN temperature B
Uniform(O, 64)
Train random action probability Action space subsampling ratio
0.05 (5%)
1%
Building block embedding size
For action space subsampling, we randomly subsample 1% actions for AddF sellfirstReee Resttatt and each bi-molecular reaction template E R2. However, for bi-molecular reactions with small pos- sible reactant block sets B, E B, the memory benefit from the action space subsampling is small while a variance penalty is large. Therefore, we set the minimum subsampling size to 100 for each bi-molecular reaction, and the action space subsampling is not performed when the number of ac- tions smaller than 100.
The number of actions for each action type is imbalanced, and the number of reactant blocks (B,) for each bi-molecular reaction template r is also imbalanced. This can make some rare action categories not being sampled during training. We empirically found that React action were only sampled during 20,000 iterations (1.28M samples) in a toy experiment that uses one bi-molecular reaction template and 10,000 building blocks in some random seeds. Therefore, we set the random action probability as the default of 5%, and the model uniformly samples each action category in the random action sampling. This prevents incorrect predictions by ensuring that the model experiences trajectories including rare actions. We note that this random selection is only performed during model training.
C.3 EXPERIMENTAL DETAILS
Pocket-specific optimization. For pocket-specific optimization with GPU-accelerated UniDock, we normalize the Vina and SA scores for multi-objective optimization:
Vina(x) ()) 0.1 maxx(Vina(z)) 0),
SA(x) = (10 1110-SAA SS(=))//
(32)
'The allowable atom types are B, C, N, 0, O, CI, Br,
Default hyperparameters in https://git thubb. com wwccrrsrrerssioeo www.:uusssiioooaamaaaaaaaaaa http://fowooot///ooooooo ob/ trunk/src/gen wwwet///ss//t/ee///////s//
22

---

Published as a conference paper at ICLR 2025
For FragGFN, set the maximum trajectory length to 9, and for SynFlowNet and RGFN, used the same hyperparameters as our framework except for the maximum trajectory length and building set the maximum trajectory length
block library size. According to their default setting, rather than 4, and we randomly sampled 350 building blocks for RGEN and 6000 building blocks for SynFlowNet. Moreover, we do not perform action subsampling for SynFlowNet and RGFN.
Pocket-conditional generation in a zero-shot manner. We used the modified version of the TacoGFN's reward function and training set. Since we don't need to optimize SA (Ertl & Schuffen- hauer, 2009), we excluded the SA term from the reward functions:
if v Proxy(x)
0.04 X Proxy(x)
8 < Proxy S
T Fallinity (1)
0.22 x Proxy(x)) 1.28
- 13 v (roxy(x)= I -8
1.32
if Proxy(x) S 13
(EE((s)/07 if QED(2) S 0.7
IQED (2)
[1
otherwise
SA(x)/0.8 if s 0.8
TSA (3))
1
otherwise
Taffinity (r) x (QED (z)
TacoGFN-Reward(s)
(33)
(HeavyAtomCounnt(())
Taftinit (x) x (o)
RxnFlow-Reward(22
(34)
V
For hyperparameters, we set the pocket embedding dimension to 128 and the training GFN tem- perature to Uniform(O, 64) which are used in TacoGFN. We trained the model with 40,000 oracles whereas TacoGFN is trained for 50,000 oracles.
Introducing further objectives without retraining. For the restricted block library (TPSA<30), and ReactBi,
we set the action space subsampling ratio as 10% for both and we set that as 1% for an entire library.
Scaling action space without retraining. For action space subsampling, we set the subsampling ratio 2% for both AddFirrs isstteactantt and ReactBi for the "seen" and "unseen" libraries. For the "all" library ("seen" + "unseen"), we set the subsampling ratio as 1 1%.
Ablation study. We set different subsampling ratios according to the building block library size. For 100-sized, kk-sized, and 10k-sized libraries, we do not perform the action space subsampling. We set action space subsampling ratio of 10% for 100k-sized one and 1% for MM-sized one.
Further improvement with 3D interaction modeling, To better analyze the effects interaction modeling, limit the maximum reaction step to 1. Since the initial state is empty graph, can analyze the impact of interaction modeling on a single decision process for selecting reaction to perform. To obtain the 3D binding conformation, 2023) under the fast search mode. We used the same neural network structure for both 2D-based generation and 3D-based generation. For the reward setting, we used the Vina docking score reward function and filtered out molecules according to the Lipinski Rule.
used GPU-accelerated UniDock (Yu al.,
C.4 SOFTWARES
Molecular docking software. For a fair comparison with the baseline model, we used UniDock (Yu et al., 2023) for target-specific generation and Quick 2.1 (Alhossary et al., 2015) for SBDD. The initial ligand conformer is generated with srETKDG3 (Wang et al., 2020) in RDKit (Landrum al., 2013). For Quick Vina, we converted the molecule format to pdbqt with OpenBabel (O'Boyl Boyle et al., 2011) and AutoDock Tools (Huey et al., 2012). To set up exhaustive search, set the search mode to balance for UniDock and the exhaustiveness to 8 for Quick Vina. We kept the seed fixed at throughout the ETKDG and the whole docking process.
23

---

Published as a conference paper at ICLR 2025
Docking proxy. We used the Quick Vina proxy proposed by Shen et al. (2023) which is implemented in PharmacoNet (Seo & Kim, 2024). We used a proxy model trained on the CrossDocked2020 training set rather than the model trained on the 15M training set.
Synthetic accessibility estimation. To evaluate the synthetic accessibility of molecules, we used the retrosynthesis planning tool AiZynthFinder (Genheden al., 2020). AiZynthFinder uses MCTS to find synthesis paths and estimate the number of steps, search time, success rate, and synthetically accessible score as metrics to indicate synthesis complexity or synthesizability.
C.5 BASELINES
SynNet. For SynNet (Gao et al., 2022b), we perform multi-objective optimization with the following reward function:
R(x)= 0.5QED(x) + 0.5Vina(zz.
(35)
According to the standard setting for optimization, we set the number of offspring to 512 and the number oracles to 125. To use pre-trained models, we used SynNet's template set (91 templates) instead of template set (71 templates) used in our work.
RGFN. Since the code of RGEN is not released, we reimplement the RGFN.
BBAR. Since BBAR (Seo et al., 2023) allows multi-conditional generation, we directly used QED and docking scores without any processing. We split 64,000 ZINC20 molecules according to the reported splitting of BBAR: 90% for the training set, 8% for the validation set, and 2% for the test set (in our case, the number sampling molecules). We performed UniDock for training and validation set to prepare the label for the molecules. Since BBAR requires the desired property value, we used the average docking score of the top 100 diverse modes from our model.
Pocket2Mol, TifgetDiff, DecompDiff, TacoGFN. We followed the reported generative setting to generate 100 molecules for each CrossDocked test pocket. We set the center of the pockets with the reference ligands in the CrossDocked2020 database. We reuse reported runtime in Shen et al. (2023), which is measured on NVIDIA A100 for Pocket2Mool, TargetDiff, and DecompDiff, and NVIDIA RTX3090 for TacoGFN.
DiftSBDD, MolCRAFT. We used the generated samples from their official GitHub repository.
C.6 LIT-PCBA POCKETS
Table 8 describes the protein information used in spcket-specific optimization with UniDock, which is performed on Sec. 4.1.
Table 8: The basic target information of the LIT-PCBA dataset and PDB entry used in this work.
Target
PDB Id
Target name
ADRB2
4ldo
Beta2 adrenoceptor
ALDHI
512m
Aldehyde dehydrogenase
ESR_ago
2p15
Estrogen receptor a with agonist
ESR antago
2iok
Estrogen receptor a with antagonist
FENI
5fv7
FLAP Endonuclease
GBA
2v3d
Acid Beta-Glluceeerebrooidaae
IDHI
Isocitrate dehydrogenase 1 4umx
KAT2A
5h86
Histone acetyltransferase KAT2A
MAPKI
4zzn
Mitogen-activated protein kinase
MTORCI
4dri
PPlase domain of FKBP51, Rapamycin
OPRKI
6b73
Kappa opioid receptor
PKM2
4jpg
Pyruvate kinase muscle isoform M1/M2
PPARG
5y2t
Peroxisome proliferator-activaee receptor 7
TP53
3zme
Cellular tumor antigen p53
VDR
3a2i
Vitamin D receptor
24

---

Published as a conference paper at ICLR 2025
D ADDITIONAL RESULTS
D.1 ADDITIONAL RESULTS FOR POCKET-SPECIFI GENERATION TASK
We reported the additional results of Sec. 4.1 for the remaining 10 pockets on the LIT-PCBA bench- mark.
Table 9: Hit ratio (%). Average and standard deviation for 4 runs. The best results in bold.
Hit ratio (%,, f)
Category
Method
GBA
IDHI
KAT2A
MAPK1
MTORC1
FragGFN
5,00 4.24)
4.50 (#: 1.66)
1,25 0.83)
0.75 (#: 0.83)
0.00 (#: 0.00)
Fragment
FragGFN+SA
3.00 (#: 1.00)
4.50 (d: 4,97)
1,50 (:: 0.50)
2.00 (:: 1.73)
0.00 (1: 0.00)
SynNet
50.00 (#: 0.00)
50,00 0.00)
29.17 18.16)
37.50 (#: 21.65)
0.00 (:: 0.00)
BBAR
17.75 (#: 2.28)
19.50 (#: 1.50)
18.75 (#t: 1.92)
16,25 (#: 3.49)
0.00 (#: 0.00)
Reaction
SynFlowNet
58.00 (#: 4.64)
59.00 (+:: 4.06)
55.50 (#: 10.23)
47.25 (#: 6.61)
0.00 (+: 0.00)
RGFN
48.00 (#: 1.23)
43.00 2.74)
49.00 (1: 1.22)
38.00 4.12)
0.00 (:: 0.00)
RXNFLOW
66.00 1.58)
64.00 5.08)
66.50 (d: 2.06)
63.00 4.64)
0.00 (:: 0.00)
OPRK1
РKM2
PPARG
TP53
VDR
FragGFN
0.50 (#: 0.50)
7.25 (d: 1,92)
0.75 (:: 0.43)
4.25 (d: 1.64)
0,00 (+: 0.00)
Fragment
FragGFN+SA
0.50 0.87)
4.50 (#: 1.50)
1.00 0.71)
2.25 (#: 1.92)
0.00 (#: 0.00)
SynNet
0.00 0.00)
0.00 (LL 0.00)
33.33 (:L: 20.41)
8.33 (1L 14.43)
0.00 (L 0.00)
BBAR
2.50 1.12)
20.00 0.71)
10.50 2.69)
14.00 3.94)
0.00 0.00)
Reaction
SynFlowNet
23.50 (+: 5.94)
50.75 1.09)
53.50 5.68)
55.50 9.94)
0.00 (:: 0.00)
RGFN
2.50 (#: 2.06)
34.75 (#: 6.57)
29.00 (::: 6.52)
37,00 (#: 6.60)
0.00 (+: 0.00)
RXNFLOW
72.25 (# 2.05)
62.00 ( 3.24)
65.50 (+: 4.03)
67.50 (#: 2.96)
1.75 (+: 0.83)
Table 10: Vina. Average and standard deviation for 4 runs. The best results are in bold.
Average Vina Docking Score (kcal/mol, 4)
Category
Method
GBA
IDHI
KAT2A
MAPKI
MTORCI
Fragment
FragGFN
-8.76 (#: 0.46)
.9.91 (# 0.32)
0.20)
-8.93 (#: 0.18)
-1051 (# 0.31)
FragGFN+SA
-8,92 (+: 0.27)
-9.76 (#: 0.64)
-9.14 (:: 0.43)
-8.28 (#: 0.40)
-00.14 (# 0.30)
SynNet
-7.60 (+: 0.09)
-8.74 (# 0.08)
-7.64 0.38)
-7.33 (#: 0.14)
-9.30 (:: 0.45)
BBAR
-8.70 (t: 0.05)
-9.84 (#: 0.09)
-8.54 (:: 0.06)
-8.49 (1: 0.07)
-10.07 (:: 0.16)
Reaction
SynFlowNet
-9.27 (4: 0.06)
-10.40 0.08)
-9.41 0.04)
-8.92 (1 0.05)
-10.84 0.03)
RGFN
-8.48 (#: 0.06)
-9.49 (# 0.13)
-8.53 0.11)
-8.22 (#0.15)
-9.89 0.06)
RXNFLOW
-9.62 (t 0.04)
-10.95 (# 0.05)
-9.73 0.03)
-9.30
0.00
-11.39
0.09)
OPRK1
PKM2
PPARG
TP53
VDR
Fragment
FragGEN
-10.28 (L 0.15)
-11.24 (1 0.27)
-9.54
0.12)
-7.90 (1 0.02)
-10.96 (.: 0.06)
FragGFN+SA
-9.58 (1 0.44)
-10.83 (1: 0.34)
-9.19 0.29)
-7.61 (1 0.27)
-10.66 0.61)
SynNet
-8.70 0.36)
-9.55 (1: 0.14)
-7.47
0.34)
-5.34 (1 (1:)23)
-10.98 (1: 0.57)
BBAR
-9.84 (:: 0.10)
-11.39 (#: 0.08)
-8.69 0.10)
-7.05 (1: 0.09)
-11.07 (1: 0.04)
Reaction
SynFlowNet
-10.34 (+:: 0.07)
-11.98 0.12)
-9.40 (1: 0.05)
-7.90 (1: 0.10)
-11.62 (1: (2.033)
RGFN
-9.61 (4: 0.11)
-10.96 0.18)
-8.53 (4: 0.07)
-7.07 (#: 0.06)
-10.86 (-: 0.11)
RXNFLOW
-10.84 (#: 0.03)
-12.53 (4 0.02)
-9.73 0.02)
-8.09 ( 0.06)
-12.30 (=: 0.07)
25

---

Published as a conference paper at ICLR 2025
Table 11: Syuthesizability. Average and standard deviation for runs. The best results are in bold.
AiZynthFinder Success Rate (%,
Category
Method
GBA
IDHI
KAT2A
MAPKI
MTORCI
FragGFN
5.00 (1: 4.24)
4.50 (L: 166)
1.25 (1L 0.83)
0.75 (1 0.83)
2.75 (1: 1,30)
Fragment
FragGFN+SA
3.00 (d: 1.00)
4.50 (1 4.97)
1,50 (L 0.50)
3.25 (1 1.48)
3.50 (dL 2.50)
SynNet
50.00 (# 0.00)
50.00 (#: 0.00)
45.83 (#: 27.32)
50.00 (# 0.00)
54.17 (#: 7.22)
BBAR
17.75 2.28)
19.50 (:: 1,50)
18.75 1.92)
16.25 (d: 3.49)
18.75 3.90)
Reaction
SynFlowNet
58.00 4.64)
59.00 (d: 4.06)
55.50 (:: 10.23)
47.25 (d: 6.61)
57.00 7.58)
RGFN
48.00 (#: 1.22)
43.00
(:: 2.74)
49.00 (#: 1.22)
42.00 (:: 3.00)
44.50 (:: 4.03)
RXNFLOW
66.00 (# 1.58)
64.00
(#: 5.05)
66.50 (#: 2.06)
63.00 (#: 4.64)
70,50 (#: 2,87)
OPRK1
PKM2
PPARG
TP53
VDR
FragGFN
2.50 (# 2.29)
8.75 (+: 3.11)
0.75 (#: 0.43)
4.25 (+: 1.64)
3.50 (:: 2.18)
Fragment
FragGEN+SA
3.25 (#: 1.79)
9.75 (#: 2.28)
1.25 (#: 1.09)
2.25 (#: 1.92)
3.75 (#: 2.77)
SynNet
54.17 (d: 7.22)
50.00 (:: 0.00)
54.17 7.22)
29.17 (1: 18.16)
45.83 7.22)
BBAR
13.75 (#: 3.11)
20.00 (# 0.71)
15,50 (#: 2.29)
18,50 (+: 3.28)
12.25 (#: 3,34)
Reaction
SynFlowNet
56.50 (# 7.63)
50,75 (#: 1.09)
53,50 5.68)
55,50 (#: 9,94)
53,50 (# 1.80)
RGFN
48.00 2.55)
48.50 (#: 3.20)
47.00
53.25 (#: 3.63)
46.50
RxNFLOW
72.25 2.05)
62.00 (d: 3.24)
65.50 4.03)
67.50 2.96)
66.75 2.28)
Table 12: Synthetic complexity. Average and standard deviation for 4 runs. The best results are in
bold.
Average Number of Synthesis Steps (J)
Category
Method
GBA
IDHI
KAT2A
MAPKI
MTORCI
FragGFN
3.94 (#: 0.11)
3.74 (# 0.10)
3.78 (#: 0.09)
3.72 (#: 0.18)
3.84 (#: 0.18)
Fragment
FragGFN+SA
3.94 (#: 0.15)
3.84 (# 0.23)
3.66 (#: 0.18)
3.69 (#: 0.21)
3,94 (#: 0.88)
SynNet
3.38 (#: 0.22)
3.38 (# 0.22)
3,46 (+: 0.95)
3,50 (#: 0.00)
3.29 (#: 0.36)
BBAR
3.71 0.12)
3.68 0.02)
3.63 (#: 0.05)
3,73 (+: 0.05)
3.77 (#: 0.09)
Reaction
SynFlowNet
2.48
2.61 0.13)
2.45 0.37)
2.81 (1. 0.24)
2.44 (L
RGFN
2.77 (1: 0.20)
2.97 (1 0.15)
2.78 (1: 0.10)
2.86 (1L 0.19)
(1 0.06)
RXNFLOW
2.10 (1: 0.08)
2.16 (1L 0.11)
2.29 (1: 0.05)
2.29 (L 0.11)
2.05 (1L 0.09)
OPRK1
PKM2
PPARG
TP53
VDR
Fragment
FragGFN
3.82 0.13)
3.71 0.12)
3.73 (#: 0.24)
3,73 (+: 0.23)
(:: 0.06)
FragGFN+SA
3.62 (1 0.12)
3.84 (#: 0.21)
3.71 (t: 0.04)
3.66 (:: 0.05)
3.67 (::: 0.25)
SynNet
3,29 0.36)
3.50 0.00)
3.29 (+: 0.361
3.67 (+: 0.91)
(#: 0.22)
BBAR
3.70 0:117
3.61 (# 0.05)
3.72 (+ 0.13)
3.65 (:: 0.05)
3.77 (# 0.16)
Reaction
SynFlowNet
2,49 (1: 0.33)
2.62 (# 0.10)
2.56 0.12)
2,51 (#: 0.27)
2.55 (# 0.09)
RGEN
2.81
2.82 0.10)
2.82 ( 0.18)
2.64 (+: 0.10)
2.84 (# 0.18)
RXNFLOW
2.00 0.09)
2.34 0.19)
2.21 ( 0.06)
2.12 (+: 0.12)
2.12 (#: 0.12)
26

---

Published as a conference paper at ICLR 2025
D.2 PROPERTY DISTRIBUTION FOR POCKET-SPECIRTS GENERATION TASK
reported the property distribution of the generated molecules for FragGFN, SynFlowNet, RGFN, and RXNFLOW for each of 15 LIT-PCBA targets.
Vina Score (kcal/ool)
Synthesis Steps
SA Score
-10
-12
FreGFN SynflowNet
RGFN
Renfllow
FregGFN Synflowhie:
RGFN
Renflow
FregGFN SynFlowieet
RGFN
Ranifoow
-10-
-12
FragGFN/ Synflowttet RGFNN
RxnFlow
FragGFN SynFlowNer
RGGNN
RknFllow low
FragiFN synFlowtket
RGGN
Roniflow
ESRt ago
-10
FraGFNN SynFlowNet
RaaGFN SynFlowNet
RragGFN SynFiowtict
ESA Lenaagoo
-12-
FreGGNN SynflowNel
RGFN
RenFlow
FragGFN SynFlowNet
RGPN
RonFlow
FragGFN SynFlowthet
RGPN
Raniflow
TregGFN SynflowNett
RGFN
RenFllow
FregGFN SynilowNeet
RGFN
RonFlow
FragGFN lowllet
RGFN
Romflow
-8-
-10
-12
IragGFN Synflowteet
RGFN
RenFllw
FragGFN SyntlowNet
RGFN
RknFlow
FragGFFN Synt lowhet
RGiN
RonFlow
10113
-10
-12
FragGFN SynFlowNett
RGFN
RxnFlow
FragGFN SynFlowNet
RGFN
RunFlow
FragGRN SynFioutket
RGFN
RonFllw
KAT2A
-10
FragGFN SynFlowNet
RonFllw
RaaGFN SynFlowNat
RagGFN SynFlowttet
RanFllw
-8
MAPKL
-10
FragGFN SynffowWet
RGFN
RenFlow
FregOFN SynFlowNet
RGFN
FragGFN SynFlowteet
RGFN
Runflow
MTORC1
-12
TragGFN Synflowftet
RGFF
RenFllw
FragGFN SynilowNee:
RGFN
RxnFlow
PragGFN Synt
RGFF
Ronfllw
Figure The property distribution of the generated samples for the first LIT-PCBA targets.
27

---

Published as a conference paper at ICLR 2025
Synthesis
-10
FragGNN SynFlowNet RGFN
RxnFlow
FogGFN SynflowNet
RGFN
RxnFlow
RagGFN SynFiouNett
RGFN
RonFlow
PH02
-12
-14
FrogGFNi SynFlowNeet
RGPN
RenFlow
RagGFN SynFlowNet
RGFN
RanFlow
FragGFN SynFiouket
RGFN
Ranillow
-8
HAMPG
FroGFN SynFlowNet
RGFN
RxnFlow
FroGFFN SynFlowNet
RGPN
RxnFlow
RragFFN SynFioutict
RGFN
RonFlaw
FraGFFN SynFlowNet
RGFN
RxnFlow
FragGFN SynFlowNet
RGFN
RonFlow
FragiFN SynFiouteet
RGFN
RonFlbw
VOR
-12
-14
FragGFN SynFlowNet
RGFN
RxnFlow
FragGFN SynFlowNet
RGFN
RxnFlow
FragSFN SynFloutett
RGFN
RonFlow
Figure 10: The property distribution of the generated samples for the last 5 LIT-PCBA targets.
D.3 SCALING LAWS WITH BASELINE GFLOWNETS
In this section, investigate the scaling laws of our model and the baseline GFlowNets, Syn- FlowNet and RGFN, focusing performance (Figure 11) and computational cost (Figure 12). Our action space subsampling method reduces the computational cost and memory consumption via cost-variance trade-off and memory-variance trade-off.
a)
-10.0
Vina (kcalmool)
-10.5
-11.0
-11.5
10a
10a
10a
10
b)
Structural Diversity Scaffold Uniqueress
1.00
0.95
0.85
10a
10a
105
C)
0.82
0.80
-
0.76
SynFlowNet
RxnFllw
10a
10a
10a
Num Building Blocks
Figure 11: Optimization power and diversity. Average of standard deviation over the 4 runs. (a) Average docking score. (b) The uniqueness of Bemis-Murcko scaffolds. (c) Average Tanimoto distance.
28

---

Published as a conference paper at ICLR 2025
(a)
(b)
Training Time
Sampling Time
4.0
4.0
RGFN
SynFlowtiet
3.5
RxnFlow(1k)
3.0
RxnFlow(10k)
3.0
2.5
2.5
2.0
Time per
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
10k
200k
400k
600k
800k
1M
10k
200k
400k
600k
800k
1M
Number of Building Blocks
Number of Building Blocks
Figure 12: Runtime according to the building block library size. Average of standard deviation over the 100 batches. (a) The training runtime. (b) The sampling runtime without model training.
Performance. We conducted optimization for the kappa-opioid receptor using RXNFLOW and baseline GFlowNets. To prevent the hacking of docking by increasing the molecular size, we performed Vina-QED multi-objective using the multi-objective GFlowNet framework (Jain et al., 2023). As shown Figure 11, all three models exhibit similar trends docking score optimiza- tion. However, conventional GFlowNets face memory resource constraints that scale with the action space size. As a result, SynFlowNet and RGFN were restricted to library sizes up to 100,000 and 500,000 sizes, respectively, due to memory limitations. In contrast, RxNFLOW can accommodate larger action spaces leveraging the memory-variane, trade-off.
Speed. While SynFlowNet and RGFN consider actions in the massive action space to estimate the forward transition probability, architecture estimate Our architecture estimates forward tran- sition probabilities over a subset of the action space, unlike SynFlowNet and RGFN, which compute all flows for the entire action space. This design allows RXNFLOW to handle larger action spaces without encountering computational bottlenecks. To evaluate the cost-efficiency measured runtime during both training and generation. To unify the generation environments across different methods, we used constant reward function R(x)) = 1 and maximum reaction step of 1. Training times were measured on random trajectories, and sampling times were measured with initialized models, according to the building block library size. Due to memory limitations, SynFlowNet was restricted to library sizes up to 400,000.
our technique,
For RXNFLOW, we tested two configurations: RXNFLOW (1k) and RXNFLOW (10k), which sample up to 1,000 and 10,000 building blocks, respectively. As demonstrated in Figure 12, RxNFLOW achieves significantly better cost efficiency in both training and generation compared to the base- line models. Additionally, computational costs can be easily reduced by adjusting the action-space subsampling ratio.
29

---

Published as a conference paper at ICLR 2025
D.4 STATISTICAL INFORMATION FOR POCKET-CONDITIONAL GENERATION TASK
Table 13:
Statistical Information for pocket-conditionll generative models. Mean and standard
deviation for sample sets.
Vina ())
QED (t)
Category
Model
Avg.
Std.
Avg.
Std.
Pocket2Mol
-7.603
0.087
0.567
0.007
TargetDiff
-7.367
0.028
0.487
0.006
Atom
DiftSBDD
-6.949
0.079
0.467
0.002
DecompDiff
-8.350
0.033
0.368
0.004
MoICRAFT
-8.053
0.033
0.500
0.004
MolCRAFT-large
-9.302
0.033
0.448
0.002
Fragment
TacoGFN
-8.237
0.268
0.671
0.002
Reaction
RXNFL.
-8.851
0.031
0.666
0.001
D.5 TARGET SPECIFICITY OF GENERATED SAMPLES
To investigate the target specificity of pocket-conditional generation, we measured delta score (Gao al., 2024a) for the top- molecules for each pocket. The delta score evaluates the pocket speci- ficity of a proposed molecule by comparing the docking scores difference in how well each molecule binds other proteins compared to the target protein.
Table 14: Delta Score for each methods
Category
Model
Delta Score
Atom Atom
Pocket2Mol
-1.74
DecompDiff
-1.29
Atom
DiftSBDD
-1.02
Atom
MolCRAFT
-2.08
Fragment
TacoGFN
-1.13
Reaction
RXNFLOW
-1.13
D.6 ABLATION STUDY FOR NON-HIERARCHICAL MARKOV DECISION PROCESS STRUCTURE
(a) Non-hlerarchical MDP
(b) Hierarchical MDP
Vina (kcal/mol)
6
-8-
-10-
-12
General
Nonaromatic
General
Nonaromatic
1.0
0.8
QED
0.6
0.4
0.2
General Nonaromatic
General
Nonaromatic
Figure 13: Reward distribution of generated molecules under different building block libraries: randomly sampled building block (General) and nonaromatic building block set (Nonaromatic). (a) The non- mi--hiearchhcal MDP. (b) The hierarchical MDP.
30

---

Published as a conference paper at ICLR 2025
To investigate the effectiveness of the non-hierarchical MDP structure, performed ablation study. We randomly selected 100,000 general building blocks from the Enamine building block library for model training (general). To simulate scenario where specific functional groups unallowed, we filtered out all aromatic building blocks to create a nonaromatic building block set. We trained the GFlowNets under Vina-QED multi-objective settings (Jain et al., 2023) against the beta-2 adrenergic receptor for 1,000 training oracles. After training, we generated 100 molecules using both the general block and the nonaromatic block set without additional training.
As shown in Figure 13, the proposed non-hierarchical MDPs closely align with the identified reward distributions for both objectives. Vina and QED, on the general and nonaromatic building block sets. In contrast, hierarchical MDPs, as utilized in existing methodologies, demonstrate shift in the reward distribution when the building block set restricted. This indicates that non-hierarchical MDPs are more robust in changes in the building block set compared to hierarchical MDPs.
D.7 ADDITIONAL RESULTS FOR SCALING ACTION SPACE WITHOUT RETRAINING
We reported the additional results for additional reward exponent settings (R
B 1
B
10
B 20
B 30
1.0
0.8
0.6
QED
0.4
0.2
0.0
seen unseen all
seen unseen all seen unseen all seen unseen all
Figure 14: QED reward distribution generated molecules.
Moreover, we investigated the generalization ability of our method to structurally different building blocks under the Vina-QED objectives against the beta-2 adrenergic receptor, For model training, we randomly selected 100,000 building blocks from the entire library (seen). Additionally, we selected 100,000 building blocks with a Tanimoto similarity of less than 0.5 to all training building blocks (unseen). The model was trained using the seen blocks first, and then the trained model subsequently generate molecules with seen blocks, unseen blocks, and a combination (all), respectively. As illustrated in Figure 15, the model also generate samples with similar reward distributions from building blocks with different distributions than the ones used for training.
both sets
QED
Vina
1.0
0
0.8
0.6
-5
0.4
0.2
-10
0.0
seen unseen all
seen unseen all
Figure 15:
Reward distribution of generated molecules.
31

---

Published as a conference paper at ICLR 2025
D.8 THEORETICAL ANALYSIS
To assess the impact of action space subsampling in GelowNet training, we conduct a toy experiment using a simplified setup with 10,000 blocks, uni-molecular reaction template, one bi-molecular reaction template, and the QED objective. We used the Hell- Vol--olllaadeelissyy reaction as a uni- molecular reaction and the Amide reaction as bi-molecular reaction, which are illustrated in Fig. 16. We used minimum trajectory length of I, max trajectory length 2. constant GFN temperature of 1.0, and learning rate decay of 3,000 for Pr and log Z. For the GFlowNet sampler, we used the same weights of the proxy model, i.e. EMA factor of O. We performed optimization for 30,000 oracles with batch size of 64.
As shown in Figures 17(a) and 17(b), we compare baseline GFlowNet trained without subsam- pling ("base") to models using various subsampling ratios and Monte Carlo (MC) sampling. The differences in log Ze relatively small (<0.005) across all settings, and increasing MC samples for state flow estimation F9 further reduced the bias. In Figures 17(c) and 17(d), we also evaluate the bias in the trajectory balance loss (Lrg) and its gradient norm (I) (e I]) during training, find- ing negligible differences compared the true values. These results indicate that our importance sampling reweighting approach effectively mitigates bias from action space subsampling, enabling efficient and accurate policy estimation.
(a)
(b)
R
Ri
F2
R
OH,
R
Rg: C(sp))
R6
C(sp?, aromatic), Rg. Rg: Cisc", aromatic)
X: OH, O CI
Figure 16: Reaction templates employed toy experiments. (a) Hell- Zelinsky reaction. (b) Amide reaction.
(a) 16.50
Trajectory balance loss (b)
(c)
0.10
Cyn
Perrll
16.49
0.08
Gradient Norm
16.48
4-
0.04
16.47
0.02
MNN
16.46
0.00
1/9
1/5 base
2
20k
40k
20k
Sampling Ratio
Num MC Samples
Step
Step
(a) log Za according to the action space subsampling ratio (left) and the number of MC samples where the subsampling ratio 1/9 (right). (b) The trajectory balance
Figure 17: Bias estimation.
loss (CTB, LTι) where the subsampling ratio is 1/9 under 4 MC samples. (c) The loss gradient norms
V Il. V ||) where the subsampling ratio 1/9 under 4 MC samples.
32


Reviews:
[{"review_id": "wmf38eSgtw", "reviewer": "Reviewer_91bL", "summary": "This paper introduces RXNFLOW, a generative framework that integrates synthesizability considerations into molecular generation for drug design. Following previous work, RXNFLOW addresses synthesizability by generating molecules using molecular building blocks, chemical reaction templates and a GFlowNet model. RXNFLOW aims to efficiently learn to sample from a large and complex action space, employing an action space subsampling technique to handle the combinatorial explosion of possible synthetic pathways. The authors claim that RXNFLOW achieves strong performance in pocket-specific and pocket-conditional generation tasks, with high synthesizability and diversity scores compared to previous models.", "questions": "- Table 5: Values in time column have ‘a’ and ‘b’ superscripts. What do these mean?\n- Section 4.2: Can the authors clarify what is ‘zero-shot’ about this method?\n- Do you use the same reward functions for methods when comparing to previous GFlowNets?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- RxnFlow performs well across all tasks/targets and the comparison of a reaction-based GFlowNet to SBDD models is welcome.\n- The subspace sampling technique substantially reduces memory and computational complexity.\n- The use of non-hierarchical action base combining reactions and build blocks is novel and well motivated.\n- Article is generally very well written and the quality of figures is high.", "weaknesses": "**Weakness/points to be worked on:**\n\n- The main contribution of the works is the non-hierarchical and continuous action space. There are many theoretical benefits to this, but the benefit of their method is not concretely controlled with respect to the building block data used and compute budgets.\n    - I would recommend Figure 6 be amended with scaling laws for the SynFlowNet [1], RGFN [2] and SyntheMol [3] methods.\n- Another fundamental claim is that the model can generalise to ‘’unseen” building blocks. While in theory their method should allow this, it has not been justified with experimental evidence.\n    - As far as I can tell, this is only benchmarked in Figure 5 which looks at QED. This is a highly uninformative metric given how well generative models do on QED. Benchmarking the seen and unseen molecules on one of the targets for instance would be a better example.\n    - Furthermore, a random split of the building blocks are used, given the high degree of redundancy in the Enamine Building Block set it is not surprising that performance does not change as these sets are highly similar. How does the model do on highly novel building blocks?\n- While technically impressive, I see no practical utility gained in the case on not retraining on new building block sets as (i) these libraries at not updated regularly and (ii) the costs of training these models are so small one would prefer to retrain a model anyways.\n    - The y-axis scales in Figure 6 are extremely narrow for many metrics and does not represent a meaningful change in values for Vina and Structural Diversity. For Figure 6b there is only a meaningful reduction in scaffold diversity when number of available building blocks is 10^2, an extremely smaller number and other works do not use.\n    - Furthermore, RGFN [2] introduces a similar method to scale to large build block spaces and a proper comparison is not conducted.\n- Some aspects of the evaluation could be improved:\n    - What is the with per run variance for the docking scores for molecules in Tables 1-4? Is there really a significant difference between the methods? I would recommend including per run box plots of Vina scores in the appendix.\n    - There is next to no discussion on the limitations of their model and possible future work\n    - L458: The authors do not include statistical tests or error bars/standard deviation values in Table 5 yet claim ”RXNFLow achieves significant improvements in drug-related properties” in the CrossDocked experiments.\n    - No effort is made to disentangle the technical changes of the authors model v.s. building block library size when comparing to baselines.\n- The article is generally very well written but parts are not clear, for instance:\n    - The fact that the continuous embedding space is based on chemical fingerprints is not mentioned in the main text.\n    - Table 6:  No legend/keys are provided to indicate what the symbols mean. \n- L429: “Since RXNFLOW explicitly considers synthesiz- ability, we exclude the SA score from the TacoGFN’s reward function as described in Sec. C.2.”.  I Could not find reference to SA score in Section C.2. Am I right in thinking the authors removed the SA reward from TacoGFN? In which case they ablated another model they are comparing to. Or have they removed it when training their model? Please clarify.\n- No code is provided.\n\n**Minor points:**\n- L122: “...trained models with the GFlowNet objective”, is it not more accurate to say these works propose reaction-based GFlowNet models?\n- I struggle to see the added value in Algorithm 1 in the main text. It does not aid in understanding the method. I recommend removing.\n- Table 6 seems wrong? \n- In Table 3, calling this metric “Percentage of Synthesizable Molecules” is misleading, as these are predicted not ground truth. I would prefer AIZynFinder success as done in the previous literature [1].\n- I appreciate the use of the LIT-PCBA targets, but is there a reason the SEH Proxy used in the rest of the GFlowNet literature [1,2] was not used?\n- Section B.1: The formulation sees to be highly inspired by SynFlowNet, if so please cite accordingly.\n\n\n\n\n1. Cretu, Miruna, et al. \"Synflownet: Towards molecule design with guaranteed synthesis pathways.\" arXiv preprint arXiv:2405.01155 (2024).\n2. Koziarski, Michał, et al. \"RGFN: Synthesizable Molecular Generation Using GFlowNets.\" arXiv preprint arXiv:2406.08506 (2024).\n3. Swanson, Kyle, et al. \"Generative AI for designing and validating easily synthesizable and structurally novel antibiotics.\" Nature Machine Intelligence 6.3 (2024): 338-353.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0DSlx3JMtn", "reviewer": "Reviewer_dTeg", "summary": "This paper introduces a new gflownets-based method RxnFlow for conducting the synthesizability-aware generation. To this end, the authors proposed to redefine the action space with Reaction templates and building blocks. To make the training procedure flexible, the authors involve a subsampling method for training RxnFlow in huge spaces.  The experiments over the SBDD benchmark have demonstrated the effectiveness of the RxnFlow for generating molecules with both good binding affinity and synthesizability.", "questions": "Refer to above", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The motivation of the paper is clear. I believe that apart from the generative approaches that formulate the problem as a constraint generation/projection, the proposed methods focus on an alternative perspective, i,e,  explicitly limiting the action space of gflownets, which should also be explored.\n\n2. The paper introduces a simple yet effective approach, which is referred to as subspace sampling. The method takes a very simple formulation while it enables the reduction of complexity and enables feasible training of the algorithm. \n\n3. The empirical study of the proposed framework is extensive, with both pocket conditioned/ specific ligand generation being considered.", "weaknesses": "1. Though I appreciate the methods with simplicity and effectiveness, I believe that a more systematic investigation and overview of the proposed methods is needed.  Based on the bias/variance discussion in the appendix, does the proposed approach conduct a variance/efficiency tradeoff, i.e. large variance for high efficiency? This is a little counterintuitive for me, could the authors discuss this further. From the ablation in Fig. 6, with sufficient steps, a larger subspace shows better performance. Does the key benefit of the proposed method is accelerating the optimization?\n\n2. Some important baselines are suggested to be included in Table 5, for example [1] for atom-level SBDD. And the paper mentioned the recent relevant work [2], so why not include it? \n\n3. I would like to suggest the author include the SA score as another metric for evaluating the synthesizability. I am curious to see whether SA is aligned with the used metrics in the paper. \n\nI would like to adjust my score based on the response.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UpUa5ew1Hn", "reviewer": "Reviewer_sR3E", "summary": "This paper proposes  a GFlowNets based method 'RXNFLOW' for drug discovery. By employing space subsampling technique, RXNFLOW can expand the search space and handle massive action spaces. This method can be adopted for pocket-specific optimization task as well as pocket-conditional generation task.", "questions": "1. What limitations do you perceive in the current method, and how might it be improved in the future?\n2. Regarding the pocket-conditional generation task, I am curious about the method's ability to synthesize molecules with specific binding characteristics, such as calculating the 'delta score' metric propsed in Paper [1].\n\n\n[1]. Gao, Bowen, et al. \"Rethinking Specificity in SBDD: Leveraging Delta Score and Energy-Guided Diffusion.\" ICML 2024.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method enables the generation of synthetic pathways for molecules, allowing for the sampling of highly synthesizable compounds while maintaining a significant level of diversity, which is meaningful for drug discovery.\n2. With the enhancement of building blocks, the method demonstrates good scalability.\n3. The experiments were conducted thoroughly, and the presentation is relatively clear.", "weaknesses": "1. Regarding the pocket-conditional generation task, to my knowledge, the more advanced methods [1] have not been compared.  This has somewhat affected the persuasive power of the experiments.\n2. Compared to other SBDD methods, it seems that direct generation of conformations combined with the pocket is not achievable.\n3. Regarding the pocket-specific optimization task, I notice that the reward function consits of Vina Score which is also used for evaluation. I have concerns about whether the method may be overfitting to Vina, which could result in inflated evaluation metrics.\n\n\n[1]. Qu, Yanru, et al. \"MolCRAFT: Structure-Based Drug Design in Continuous Parameter Space.\" ICML 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "loYi9YZxhH", "reviewer": "Reviewer_ovVP", "summary": "In this manuscript, the authors adopted the GFlowNet to generate synthesizable molecules based on the protein pocket. They were able to demonstrate that RXNFLOW outperforms existing reaction-based and fragment-based models in pocket-specific synthetic pathways, and their method achieves state-of-the-art performance on CrossDocked2020 for pocket-conditional generation.", "questions": "1. For conventional SBDD methods, they are generally trained on molecules that can bind to protein pockets, resulting in generated small molecules with a specific distribution. I am curious about how the molecules generated in this space of synthesizable reaction-based small molecules differ from those generated by SBDD methods, such as pocket2mol. Because drug-likeness is not solely determined by synthesizability and vina score alone. For instance, the bond angle, number of rings. These facts are hard to encode into the reward function.\n2. Please compare the speed of different methods. \t\t\t\t\t\n3. Why use 3 reaction steps for your method and 4 steps for SynFlowNet and RGFN?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, this paper is interesting. The GFlowNet model is suitable to be adopted to solve the synthesizability problem of small molecules. The problem with GFlowNet is that it lacks a method to construct a graph that covers the entire space of small molecules. This paper cleverly utilizes chemical synthesis reactions to construct this graph, while also adopting a down-sampling approach to enhance the effectiveness of the method.", "weaknesses": "The main issue is the insufficient characterization of 3D interactions; the model does not explicitly model the interactions between atoms in 3D space and only uses the generated molecule's vina score with the pocket as the reward. In addition, a careful discussion of the differences from SynFlowNet is needed. Currently, the authors have avoided mentioning SynFlowNet in the introduction. According to my understanding, the method proposed in this paper has no structural difference from SynFlowNet in terms of the model framework, with some improvements in sampling techniques. I hope the authors can discuss the differences from SynFlowNet in detail from the perspectives of graph construction for the flow, reward settings, and the sampling algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ugDjPYnstC", "reviewer": "Reviewer_Q8DB", "summary": "The author proposed a GFlowNet for on synthetic pathway, aiming to leverage the large action space for drug synthesis while retaining computational efficiency. The author proposed an action embedding which embeds the template and building blocks to parameterize the policy network. They further reduces the computational complexity by introducing a subsampling technique on the action space via importance sampling. The author validate their approach on LIT-PCBA, CrossDocked2020 dataset and outperformed baselines.", "questions": "1. Table 5, what is a and b?\n2. line 45, what does it mean by 'chemical modification can degrade in the optimized propeties', can you give some examples?\n3. It's unclear to me how you introduce the additional objective without retraining, can you clarity that?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed methods outperforms baseline in standard benchmark.\n2. The methods exhibit nice scaling behavior on unseen blocks.", "weaknesses": "1. **Conceptual Novelty:** Utilizing reinforcement learning (RL) to determine reaction pathways is not a novel approach. The authors have simply applied this existing methodology to GFlowNets, which does not represent a significant conceptual advancement.\n\n2. **Technical Novelty:** The proposed action embedding and importance sampling techniques do not appear to introduce new concepts. Essentially, the authors are reparameterizing the policy network with template and building block embeddings— a standard practice in RL when addressing large action spaces.\n\n3. **Figure Clarity:** Figure 2 is currently confusing and requires further refinement. The use of diamonds to represent templates and squares for reactions may initially cause some misunderstanding. In contrast, Figure 7 is clearer and more effectively conveys the intended information.\n\n4. **Efficiency Discussion:** The paper's main claim centers on reducing the action space to enhance computational tractability. However, there is a need for a more in-depth discussion and comparison of the training and inference times between the proposed network and existing baselines to substantiate this claim.\n\n5. **Ablation Study:** The manuscript lacks an ablation study comparing Hierarchical Markov Decision Processes (MDPs) with non-hierarchical MDPs. Such a study is essential to evaluate the effectiveness and advantages of the non-hierarchical approach presented.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)