Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Authors: Mohit Pandey
Gopeshh Subbaraj
Artem Cherkasov
Martin Ester
Emmanuel Bengio
Venue: ICML 2025 poster (2025)
Keywords: Generative Flow Networks
Foundation model
Drug Discovery
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: We introduce Atomic GFlowNets, a foundational generative model leveraging individual atoms as building blocks to explore drug-like chemical space more comprehensively.

Abstract: Generative Flow Networks (GFlowNets) have recently emerged as a suitable framework for generating diverse and high-quality molecular structures by learning from rewards treated as unnormalized distributions. Previous works in this framework often restrict exploration by using predefined molecular fragments as building blocks, limiting the chemical space that can be accessed. In this work, we introduce Atomic GFlowNets (A-GFNs), a foundational generative model leveraging individual atoms as building blocks to explore drug-like chemical space more comprehensively. We propose an unsupervised pre-training approach using drug-like molecule datasets, which teaches A-GFNs about inexpensive yet informative molecular descriptors such as drug-likeliness, topological polar surface area, and synthetic accessibility scores. These properties serve as proxy rewards, guiding A-GFNs towards regions of chemical space that exhibit desirable pharmacological properties. We further implement a goal-conditioned finetuning process, which adapts A-GFNs to optimize for specific target properties. In this work, we pretrain A-GFN on a subset of ZINC dataset, and by employing robust evaluation metrics we show the effectiveness of our approach when compared to other relevant baseline methods for a wide range of drug design tasks.  The code is accessible at https://github.com/diamondspark/AGFN.

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Mohit Pandey'
Gopeshh Subbaraj² Artem Cherkasov
Martin Ester
Emmanuel Bengio* +
Abstract
1. Introduction
GFlowNets are amortized samplers that learn stochastic policies to sequentially generate compositional objects from given unnormalized distribution, i.e. a reward signal. They can generate diverse sets of high-reward objects, which has a demonstrated utility in small molecules drug discovery tasks (Bengio al., 2021). Traditionally, GFlow used for molecule generation have focused on fragment-based drug discovery, i.e. their action space comprises of some prede- termined fragments building blocks for molecules. While producing diverse candidates, fragment-based approaches limit the pockets of accessible chemical space (Zhu et al., 2024; Bengio et al., 2021; Jain et al., 2023; Shen et al., 2024). A truly explorative generative policy, tapping into the full po- tential of GFlowNet would be realizable when using atoms instead of fragments the action space (Mouchlis al., 2021; Meyers et al., 2021; Nishibata & Itai, 1991). On the other hand, the vastness of this accessible state space makes training atom-based GFlowNets susceptible to collapse. Ear- lier atom-based GFlowNets attempt to overcome this issue by limiting models to small trajectories (Jain et al., 2023). However, since most commercially available drugs have molecular weights ranging from 200 600 daltons(Bos & Mcinardi, 2000; Bickerton et al., 2012), the molecules generated from these small trajectories are unlikely to pos- sess the drug-like characteristics necessary for therapeutic efficacy. In this work, we propose to scale the trajectory length of atom-based GFNs by pretraining them with expert demonstrations, coming in the form of offline trajectories generated from a dataset of drug-like like molecules. Our main contributions in this paper follow:
Generative Flow Networks (GFlowNets) have recently emerged as suitable framework for generating diverse and high-quality molecular structures by learning from rewards treated as unnormalized distributions. Previous works in this framework often restrict exploration by us- ing predefined molecular fragments as building blocks, limiting the chemical space that can be accessed. In this work, we introduce Atomic GFlowNets (A-GFNs), a foundational generative model leveraging individual atoms building blocks to explore drug-like chemical space more comprehensively. We propose an unsupervised pre-training approach using drug-like molecule datasets, which teaches A-GFNs about inexpen- sive yet informative molecular descriptors such as dtrug-likelinee topological polar surface area, and synthetic accessibility scores. These prop- erties serve as proxy rewards, guiding A-GFNs towards regions of chemical space that exhibit desirable pharmacological properties. We further implement a goal-conditioned finetuning process, which adapts A-GFNs to optimize for specific tar- get properties. In this work, we pretrain A-GFN on subset of ZINC dataset, and by employing ro- bust evaluation metrics show the effectiveness of our approach when compared to other rele- vant baseline methods for wide range of drug design tasks. The code is accessible at https: / //git ggithuu.... com/ di dimoodssaar//AGR
We introduce A-GFN, an atom-based GelowNet for sampling molecules in proportion to rewards governed by inexpensive molecular properties.
'Vancouver Prostate Centre, The University of British Columbia "Mila- Quebec Al Institute. Université de Montréal
"Simon Fraser University 'Recursion Pharmaceuticals. Correspon- dence Mohit Pandey <mkpandey suudent.ubc.c..... Gopeshh Subbaraj hoopehh..subbance @ mula.que
We propose novel strategy for unsupervised pretrain- ing of A-GFNs by leveraging drug-like molecules us- ing offline, off-policy training. This pretraining enables broader exploration of chemical space while maintain- ing diversity and novelty relative to existing drug-like like molecule datasets.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
We demonstrate that goal-conditioned finetuning of A-GFNs for sampling molecules with desired proper- ties offers significant computational advantages over training A-GFNs from scratch.
2024), along with the benefits of scalability to large ac- tion spaces and credit assignment makes them well-suited for molecular generation tasks; the primary focus of our work. We consider molecules as their topological graphs G = G F,, X') such that X E RRad define d-dimensional atomic features for the n nodes in G, E E {0, 1] _ is the edge-adjacency tensor for b types of edges connecting n nodes, and finally A E {0. (0.1) is the adjacency matrix for nodes. A trajectory = (so, a.)......8n an) can be generated de novo from so by sampling actions a from a conditional policy Pr(- | <; 0), or by "deconstructing" a molecular graph G = Sn, via P'(( c) States s; are partially constructed subgraphs, albeit we ensure that they remain valid molecular graphs throughout.
Finally, we demonstrate the practical utility of A-GFN finetuning for optimization of structual and pharma- cological properties of drugs, de novo generation of high-affinity binders, and for lead optimization.
2. Related Works
GFlowNets (Bengio et al., 2021) have emerged as power- ful framework for sampling diverse candidates proportional to given reward, distinguishing them from traditional re- inforcement learning (RL) approaches. They have been applied to molecular design (Jain et al., 2022), causal dis- covery (Deleu et al., 2022), and scheduling (Zhang et al., 2023). Unsupervised pretraining has been explored in RL to enhance sample efficiency (Liu & Abbeel, 2021; Kim et al., 2024) and in GFlowNets through outcome-condition ing (Pan et al., 2023), which enables reward- -wward-free pretraining for adaptive downstream learning. Recent GFlowNet exten- sions incorporate goal-conditioning (Jain et al., 2022; Roy et al., 2023), allowing for controlled generative modeling, and multi-objective formulations (Jain et al., 2023), enabling trade-offs between competing molecular properties. These advances improve sample efficiency and targeted molecule generation.
Our primary objective is to learn a model Go (Pr(: Pb: (:)(:)), f)), such that the generated G are chemically valid molecular graphs satisfying some molecular property conditional ranges c, and exhibiting sufficient diversity. We express this through a conditional reward R(G]c) measuring how well G satisfies c. We also want to be able to finetune these models to achieve drug discovery tasks with certain molecular property constraints, and establish the benefits of finetuning GFlowNets. Trajectory balance (TB; Malkin et al., 2022), being standard GFlowNet objective, is used as the training method in this work. TB requires learning an additional parameter log Za which estimates the partition function. The loss is:
train-
Zo T [=1 Pr(s: 91 st 1:0) .00
LIB(T) (log
RL-based molecular design has leveraged auto-regressive SMILES generation (Olivecrona et al., 2017: Popova et al., 2018) and graph-based approaches (You al., 2018), while GFlowNets provide an alternative with improved diversity and exploration. For a comprehensive review, see (Bilodeau et al., 2022). A more detailed discussion of related works is provided in Appendix B.
R(x) I (B((s_11s
where trajectories T = (sp, 81, 8n) are such that 8n = Ø is fully constructed object.
4. Pretraining with Inexpensive Rewards
To construct molecular graphs via Pp, we design an ac- tion space with 5 action types: (addNade, addiigge, addNodeAtt
3. Preliminaries
addEdgeAttribute stop). The agent can add node (heavy atom) to the graph, add an edge between nodes, set node's properties (e.g. its chirality), set a bond's properties (i.e. its bond order), stop the trajectory. To deconstruct them via PB. we use the deleterious actions: (deleteNode, de deleteEage, removeNodeAtt ribuee, and Removeeddgetttribbtee which undo their constructive counterpart.
GFlowNets (Bengio et al., 2021) are generative models designed sample structured objects from a state space S in proportion to reward function R (sT) assigned to terminal states ST. A model generates trajectories T = (so, .o...., by transitioning between states through ac- tions, guided by a forward policy Pr (s' (). The key idea in GelowNets is that they are trained to preserve a allow quan- tity, such that the total flow quantity entering a terminal state equal to its unnormalized probability. This is achieved by aligning forward and backward policies Pr and PB, which can be understood as distributing flow forward and back- wards through the state space. This induces a policy Pr which at convergence yields Pr-P (ST) x H(ST).
We use a graph neural network (Veličković et al., 2017). see SC.1, to parameterize the policies, using the GNN's invariances to produce per-node and per-edge logits. We are also careful to mask these logits such that the produced molecules always have valid valences (i.c. by design all states are convertible to RDKit molecules and SMILES). Since our method generates molecules atom by atom, we refer to the approach as Atomic GFlowNet (A-GFN).
Existing extensions of GFlowNets that handle multiple, po- tentially conflicting objectives (Jain al., 2023; Zhu et al.,
2

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
CONDETIIONAS
TASK
Foffline
Follline
QED, &Rings, 7PS4, SAS
QED, Aitings, 7F'A, SAS
lng!
THERMONBTER ENCODING
PROPERTY REWARD
THERMINETH ENGHORNG
PROPERTY REWARD
OFFLINE DATASET ZINC15
OFFLINE DATASET (TASK)
Toffline
A-OFN
SCALARIZATION
Toffline
A-OFN
SCALARIZATION
Tonline
"online
(a) Pretraining A-GFN
(b) Finetuning A-GFN
Figure 1. Illustration of the AGEN workflow, showing the overall process divided into two main parts: (a) Pretraining phase and (b)
Finetuning phase.
4.1. Inexpensive Molecular Rewards
here, À controls the decay rate, [x] max x). Clow and Chigh are the lower and upper bounds for property and dp E R (lypically dp E {-1,0,13) represents the "preference direction" hyperparameter, indicating whether lower or higher property values within the range cp are preferred (for further details, see appendix D.E). Note that during training, we sample these ranges that the model learns to be robust inference-time queries (sec 4.1.2).
In the context of pre-training A-GFN models for molecular design, we utilize inexpensive molecular rewards-such as Topological Polar Surface Area (TPSA), Quantitative Esti- mate of Drug-likeness (QED), synthetic accessibility (SAS), and the number of five or six mimmeeeeee rings in a molecule. These rewards are computationally cheap to evaluate and serve as proxies for more complex properties. We will later ($5) discuss finetuning models more expensive and computationally intensive tasks, such predicting binding affinity or toxicity (e.g., LD50), which are crucial for drug discovery but require significant computational resources or experimental data to assess accurately.
Drug discovery is an inherently multi-objective optimization (MOO) problem where the drug candidates are expected to simultaneously have several desired properties, such as a high drug-likelines (QED), high SAS, and reasonably low TPSA, among other criteria. To implement this, we choose this standard multiplicative scalarization (Jain al., 2023) of the multi-objective reward over the property set P:
Table 1. Properties used as Pretraining conditionals in Figure along with the probability distributions they are sampled from. Property
R(xlep...... = II Rp(2lcp)
(3)
Distribution
PEP
QED
N(Xqed, : 0.65, 0.80)
(0.65, 0.80)
#Rings TPSA
M(X 12
31.3)
4.1.2. REWARD CONDITIONING A-GFN
N(X tpsa: ;²: 60, 100)
U660, 100)
SAS
N(X 2; 11,3)
(1,3)
To train our conditional model, we sample for every property Cp (Ciow, Chigh) for each new training trajectory, using reasonable lower and upper bounds Cmin and Cmax of the molecular property ranges we care about, as well as extrema values " Cmax of possible values we can imagine the property taking. This ensures that the model learns from a diversity of chemical regions relevant to drug discovery.
4. 1. REWARD FUNCTION
To prait a goal-conditional A-GEN for learning molecu- lar properties p E P, we define property-specific conditional ranges Cp (Clow: Chigh) which we want samples belong to. We use the following goal-conditioned reward function for property and molecule
We use two types of training trajectories, online trajectories sampled from Pp, and offline trajectories whose terminal state is drawn from dataset and then sampled backwards from PB. For online trajectories. we draw Clow Chigh ~ U(mmin Cmax) uniformly (maintaining Ctow < Chigh), ) but with some probability e instead draw from U(fiin fmazk) to teach the model about unlikely but possible value ranges.
[4,)]
2 exp 9um
if Ps. < Clow>
Rx(x))p))= =
2
del
exp
Chip Pa x
if Pv > Chigh
2 |d
+ dp Pr Pam
otherwise.
2
2 Suigh Uew
3

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
For offline trajectories ending in some state x, since we know each property P='s 's value, we center our sampling around them. We draw Clow Chigh ~ (p=: op; Cmin: Cmax) from T a truncated normal distribution whose op is a hyperparameter (again maintaining Clow < Chigh). We also provide negative (< O reward) examples with some small probability e, for which with probability 1/2 we set Clow Cmin and draw Chigh U (Cmin: Pr), and otherwise set Chigh = Cmax and draw Clow U (p=: Cmax).
and molecular property rewards in $4.1, we optimize the A-GFN (Go) by minimizing Eq. (4) until convergence.
5. Finetuning
We investigate using pretrained A-GFN models (Ga) and adaptating them for downstream drug-discovery tasks, char- acterized by more challenging rewards. As usual, we initial- ize a new model with @ := 0.
Finally, we encode Cp via two thermometer encod- ings (Buckman et al., 2018) with bounds 'iin mmax.
We finetune G6 by using a new task-specific reward Renti
Ree(rlep (pip)) (x) II pEP
Rp(x)
(5)
4.2. Pretraining GFN with expert offiine trajectories
Bengio et al. (2021) employ replay buffer-based off-policy training for GFlowNets to generate novel molecules. How- ever, such online off-policy methods can suffer from high variance (Fedus et al., 2020; Vemgal et al., 2023) and a lock-in of suboptimal trajectories. This risk is high in sparse reward settings where the agent struggles explore rare, high-reward regions of the state space, re- sulting in slow convergence and suboptimal performance. To mitigate these challenges, we propose leveraging the vast amounts of readily available inexpensive and unla- belled molecular data to perform a hybrid online-offline off-policy pretraining of A-GEN. In this work we use the ZINC dataset (Sterling & Irwin, 2015) containing over 200M molecules from which we derive expert trajectories.
where (2( denotes the previous property-based rewards used during pretraining. Such a reward formulation ensures that the GFlowNet receives a high reward for generating molecules that simultaneously follow the desired molecular properties and are highly suitable for the downstream task. It should be noted that Zo in Eq. (1) is the normalization constant R= Emex R(x). While we do initialize our fine- tuned model with this value, we also recalibrate it to account for the updated reward Rys. by adding some some (0, o/) noise to Z common strategy in finetuning (Yuan et al., 2023; Tong et al., 2022).
adequately
Finally, we finetune our models using the Relative Trajec- tory Balance objective (RTB; Venkatraman et al., 2024). RTB enables us to treat the density over objects po(x)c) induced by Go's 's PF as a prior, and to learn a posterior poat (arle) such that p (xle) x Ro(()cc))((xxec). The RTB objective is:
4.3. Balancing Exploration and Synthesis Feasibility
The primary benefit of molecular generative models with large explorative capacities such as A-GFN is their ability to navigate the almost arbitrarily large possibilities of chem- ical structures. On the other hand, ensuring that generated molecules lie close to the chemical space feasible for synthe- sis, particularly within the constraints of maae--nn-emman (MOD) libraries such as ZINC, is crucial for synthesis and in vitro validation in drug discovery. To balance these two seemingly conflicting goals, we introduce regularization term in the pretraining objective. Specifically, we combine the exploration objective with a Maximum Likelihood Esti- mation (MLE) loss over the offline dataset, leading to the following regularized loss function:
[log] =(10g
Zo IT= Pr Pr ) [[(x) II= PF (2
CXIB(T; ())
where & is optimized and 0 is fixed.
6. Experiments
We evaluate A-GFN's effectiveness during both pretraining and finetuning using comprehensive metrics such as novelty, diversity (Polykovskiy et al., 2020), uniqueness, validity, LI- distance, #modes, #scaffolds, #circles (Xie et al., 2021b), hit-ratio, and novel hit ratio (Guo & Schwaller, 2024b). Validity for A-GFNs is trivially 1 by construction and is not reported. #Modes: Measures the number of distinct, high-reward molecular clusters. A mode is defined as any molecule with reward > 0.5 that has Tanimoto similarity 0.5 to all previously counted modes.
[(T) >))TT(TT + 22LMLE(T)
(4)
where, LMLE(T) = II log Pr(sols1-1,
LTn encourages exploration of the chemical space, while LMLE encourages proximity to MOD libraries, and Xı, d2 are hyperparameters controlling the trade-off between ex- ploration and adherence to the MOD space (see Tab. 7).
RW#C, RW#S, RWTD: weighte circle count (RW#C), scaffold count (RW#S), and Tanimoto diversity (RWTD) capture structural diversity among high-reward molecules. Each computed as the product of a diversity
With the online and offline trajectories described in $4.2

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
metric and the top-K reward average: RW Melric Reopk x Metric, typically with K = 100
eral tasks from the TDC benchmark (Huang et al., 2022). To demonstrate the practical utility of A-GFN, benchmark it against contemporary works in de novo binder generation, and lead optimization.
Normalized L1-dist: This measures how far generated moleculc's properties deviate from their target ranges. For each property, the L.1 distance is calculated by taking the absolute difference between the generated property value and the 10th percentile value in desired range. and then normalized by the respective property range to ensure com- parability across properties.
Following the setup in You et al. (2018), we focus on the following finetuning objectives:
Property Optimization: In this single objective problem, the goal is generate molecules that maximize or minimize a specified physicochemical, structural, binding property while ensuring diversity among the generated molecules. In this unconstrained optimization setting, we compare A-GFN finetuned with TB and RTB to the task-trained A-GFN Property Targeting: In this multi-objectivve problem, we generate molecules that satisfy property constraints different from those used during pretraining, while ensuring structural distinction from the pretraining set.
Success Percent: For a set of N generated graphs (G) and conditionals Crask for given task, we define success percentage as
1 N
Σ Io(Gi) x 100
Susk
Craskl
cECuat
Property Constrained Optimization: This problem requires generating molecules that simultaneously meet property optimization and property targeting criteria, i.e. molecules must lie within the specified property ranges while also maximizing (or minimizing) the targeted property.
where, I.) counts the molecules within desired condi- tional ranges and defined as
if dp < 0 [[]c(G) Cowl S 0.1 |Cowl)
Ta(G)=
I[[c(G) Chigh| 2 0.1 |(ighh))
if dz > 0 if dp =
T [(ga S c(G) I (high):
To demonstrate the practical applicability of A- we benchmark it against state-of-the of-the-art models for de novo molecular generation, and perform lead opti- mization. In de novo generation, we evaluate A-GFN on its ability to generate high-affinity molecules for specified protein targets. For lead optimization, we assess its capa- bility to refine initial molecular scaffolds and improve their binding affinity, while adhering to synthetic accessibility and other physicochemica constraints.
Binding Tasks:
where, for any considered task, c(G) is the the value of the conditional property e for the molecular graph G, Ctow and Chigh are the lower and upper bounds of c, dy indicates whether lower values (< 0), higher values (> 0), or values within a range (= 0) are preferred for that conditional.
Quantifies the percentage of molecules that meet the following criteria: docking score better than the median of known actives, QED 0.5, and SA <5 (Guo & Schwaller, 2024b).
Hit Ratio:
6.2. Baselines
Novel Hit Ratio: Restricts the hit ratio to molecules that are both unique and novel i.e., with maximum Tanimoto similarity 0.4 to any training molecule (Lee et al., 2023b).
We evaluate A-GFN against diverse set of molecular gen- erative state-of-the-art baselines. To be consistent with the atomic resolution of A-GFN, we include MolDQN (Zhou et al., 2019) and GraphMCTS (Jensen, 2019) (both of which construct molecular graphs atom-by-atom), and REIN- VENT 4 (Loeffler et al., 2024), a widely adopted baseline in the pharmaceutical industry known for its multi-objective molecular optimization capabilities and seamless integration with in-silico screening tools (which uses a character-level RNN for SMILES, i.e. also atom-level). To assess the benefits of A-GFN over fragment- approaches, we use an unconditional fragment- trained from scratch on the optimization task. For de novo molecular generation, we compare against the baselines con- sidered in Guo & Schwaller (2024b), which leverage diverse generative paradigms model chemical space.
Full details of other standard metrics are provided in $E.
6.1. Tasks and Evaluation Criteria
We will first benchmark the pretraining phase of our pro- posed model, then benchmark its finetuning capabilities on well-understood exemplar tasks. Finally, we will evaluate our model on challenging practical tasks.
GFlowNet baseline,
Pretraining: We compare the performance of A-GFN against fragment-based GFlowNets, conditioned on the same inexpensive molecular properties (sec 4.1.. Our results show that A-GFN significantly outperforms fragment-GFN in exploring drug-like chemical space across multiple objec- tives.
6.3. Pretraining
Finetuning: We benchmark the pretrained A-GFN against A-GEN trained from scratch (task-trained A-GFN) on sev-
To create robust initialization for downstream molecular generation tasks, we pretrain A-GFN using hybrid online-
5

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
offline off-policy learning strategy, where each minibatch consists of half offline trajectories sampled from expert dataset and half on-policy trajectories. The reward is con- ditioned on molecular properties-TPee QED, SAS, and ring count-constrainee within predefined target ranges. To benchmark fragment-basee GFlowNets, we adapt the purely online training paradigm of (Malkin et al., 2022). Both methods are conditioned on the same properties and ranges.
models on QED (Jeon & Kim, 2020; Zhou et al., 2019; You et al., 3018), we exclude QED from our evaluation, as our pretrained A-GFN is already optimized for it. Instead, in 3 independent tasks, we focus on molecular weight (M. W.), logP, and toxicity respectively. We will consider harder tasks and additional baselines in later sections.
As expected, for the simple tasks of M.W. and logP opti- mization, the finetuned A-GFN has a higher success rate than the task-trained A-GFN, indicated by higher top-k rewards and an increased number of distinct modes. How- ever, for more complex tasks such as LD50 prediction, both finetuning and task training struggle (Table 3), suggesting that a more sophisticated reward scheme beyond direct task rewards is necessary.
Empirical results indicate that A-GFN learns broad dis- tribution over valid and synthetically accessible molecules while significantly improving scaffold diversity, covering nearly twice as many distinct scaffolds as the fragment- based GFlowNet (Tab. 2). Furthermore, molecules sampled from the pretrained A-GFN are chemically valid, novel, and adhere to specified property constraints without directly replicating structures from ZINC250K (Fig. 3). Complete details on molecular constraints, training dynamics, and benchmarking are provided in Appendix C.
Interestingly, in these relatively simple single-objective op- timization tasks, finetuning A-GFN using the TB objective outperforms using the RTB objective (Table 10). This can be attributed to the inherent constraints imposed by the RTB objective, which anchors the policy to the pretrained prior designed for drug- likensss While beneficial for main- taining chemically valid and synthesizable molecules, this constraint limits exploration of non-drug-like solutions that might satisfy the optimization objective more easily.
Table Comparison of pretraining A-GFN and fragment-based GFlowNet on the same molecular property conditionals.
#scaffold
Method
#Modes
Div
Uniq
Nov
Valid
(n=200)
Frag. GFN
0
113
0.002
0.001
0.97
0.99
A-GFN
1252
196
0.88
0.96
1.0
1.0
Table 3. Comparison of finetuned A A-GFN with A-GFN trained from scratch for logP. molecular weight, and toxicity optimization tasks. Detailed comparaision in table 10
6.4. Finetuning
We investigate number of tasks: property optimization, property targeting, and property constrained optimization. Each task corresponds to a set of conditionals and a reward function, detailed in subsequent sections. For tasks where some small labeled dataset is accessible, we investigate the benefits of finetuming A- A-GFNs off- ofolpo with that data. Fol- lowing You et al. (2018 8), we consider two structural tasks, molecular weight (M.W.), and logP (partition coefficient for drugs' wraer-to-octant We also consider some standard drug-discovery tasks where rewards are based on empirical models such as the ADME-T (absorption, distribution, metabolism, excretion, and toxic- ity) tasks of TDC (Huang et al., 2021). Using these, we aim to show that finetuning a pretrained A-GFN achieves the task's 's objectives more quickly than training new A-GFN from scratch (Task Trained A-GFN). Unless stated other- wise, all models are finctuned for 24 hours using the RTB training objective on a single V100 GPU. Results are based on 1000 samples for all tasks except binding, where 3000 samples are used for consistency with prior work (Guo & Schwaller, 2024b).
Rew
Task
Method
#Modes
Dlv?
L1-Dit
Novt
(lop-100)
A-GPN Task trained
0.12
4.56
<10
A-GFN Finetumed
992
0.91
0.88
1.0
0.98
logp
A.GFN Task trained
0.16
2.65
A-GEN Finetuned
994
0.84
0.06
1.0
0.98
Taxicity
A-GFN Task trained
0.02
1.56
1.0
< 10
A-GPN Finctuned
0.16
0.75
1.0
0.18
concentration; Sangster, 1997).
6.4.2. PROPERTY TARGETING
Here, we the goal is to generate molecules that satisfy an altered range for a specific molecular property constraint used during the pretraining phase. This demonstrates the model's capacity for finetuning while adapting to modified property ranges. We call this adaptation Dynamic Range Ad- justment (DRA). Compared to task- training, finetuning an A-GFN leads to significantly faster convergence, even when property ranges are altered. We report results on modifying the TPSA property while keeping the constraints for other molecular properties as pretraining. To assess generaliza- tion to new property constraints, we shift the TPSA range from [60, 100] (pretraining) to [40, 60) and [100, 120). Fine- tuned A-GFN consistently surpasses training from scratch in both convergence speed and performance, excelling in high-reward mode discovery (#modes), diversity, and suc- cess rate, with notable gains under the RTB objective (Tab. 4, 11).
6.4.1 1. PROPERTY OPTIMIZATION
In this task, the goal is to generate novel molecules that min- imize (or maximize) specific physicochemical properties. While previous works have commonly benchmarked their
6

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
The disparity in performance between the TB and RTB objectives becomes increasingly pronounced in this MOO problem (Tab. 11). Finetuning without the RTB objective exhibits catastrophic forgetting, wherein the model loses the ability to generate chemically valid, drug-like molecules learned during pretraining. Although in some cases, the model eventually relearns these skills, that process inef- ficient and slow. In other words, while extended training improves the TB objective by progressively aligning the learned policy with the reward signal, overtraining the RTB objective degrades its performance.
Space
Methal
Comrage
Uivardlt
Li-dit
(Tep-100
Rewaad
taxio
KWTD
BV
TPSA
iRigg
SAS
QED
Task
MeITON
Ratrreet
6:555
03335
10332
B.1KK
0259
Prg OffonNei
31
5.611
0.800
0.33
2106
1..34
01335
0,396
Penureed
Grago MCTS
AGEN Takkoiaed
6/00
000001
0.661
0.99)
3860
1.000
AGEN Festeel
6/400
0.67
0.066
0.27
2067
3422
0.11
0.044)
0.755
0.501
0.001
0417
2,12
0.356
0.67
0.354
AGRN F?
738
6:708
0.996
0.01
3009
01:34
0.176
4.9804
ADEN Pt
6.716
0.963
0.006
0.962
0.29
0.833
Rm dats
MeltiN
Xetrrcai
6:578
0.859
0032
0,23
0.411
0.776
In the RTB formulation, the updated posterior distribution is proportional to the product of the prior and the reward function, i.e., (o( x po(x)) R(x), where (a())) the pre- trained prior distribution and R(x) is the reward function. As training progresses, this posterior collapses onto high- reward regions while maintaining proximity to the prior, thereby constraining the effective solution space. Conse- quently, longer training under the RTB objective produces high-reward molecules but at the cost of reduced diversity and uniqueness in the generated samples (Fig. 10). Thus, a trade-off between diversity and optimization efficiency must be carefully balanced as a function of training duration when using the RTB objective.
Flag Grttnher
6.534
0001
3000
01:58
040
AGFN Tukknanal
6,000
0.001
0.351
1,235
3.221
0.554
1225
0/000
AGEN Resteed
6/07
0.54
0.181
0.961
0/099
0.555
0.060
0432
2.82
0.21
0.20
0.7%6
wid
AGEIN ore
479
8:783
0.597
0.009
0.134
3/00
0.:92
02215
0999
AGEN PT
0.566
0.986
0.145
0.208
11356
0-405
RTT
MelligN
Rekercei
0.403
0.748
0.077
0.31
6.01
0.574
Flgg GitrhWer
6.217
0.003
0.206
0/06
D.NK
0.775
High V95A
MITN Taikmsiand
C100)
4.000
1.760
1.64
3002
1.201
AGIN Fisdenal
6/01
0.44
6011
2122
3.11-02
AGFN FT
w/daa
AGEM Dr
430
6.541
0.544
0.137
0.206
3081
0.399
0.55
0996
The ability of the finetuned A-GFN to adapt to modified property constraints highlights the robustness of the pre- trained Go model, which has not merely memorized specific property ranges but has learned about the state space's 's struc- ture. This udaptability is crucial in real-world drug discovery applications, where molecular property requirements often shift during the drug discovery process.
KrT
ADRN PC
280
6/677
0.594
0.468
0335
2.73
0316
0.02
0.7%
RTB dait
Table 5. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines for property- constrained optimization objective for a representative logP task. For more extensive comparision, see $F44
tasks from TDC (Cell Effective Permeability, Lipophilicity, Solubility, Plasma Protein Binding Rate, Hepatocyte Clear- ance, Microsomal Clearance and Acute Toxicity) as shown in Tables 12 through 20.
DRA High TPSA
Method
MModes
HV1
RWID:
Suceess
Nov?
(lup-100)
Rew!
A-GFN Task Trained
0.
0.16
0.07
0
0.24
0.15
A-GEN FT RTB
147
1.0
0.68
94.5
0.65
1.0
Low
A-GEN Task Truined
0.03
0.03
In the conditionals preeerred finetuning setup, we retain the same conditional property ranges (cp) used during pretrain- ing, setting the task's de in accordance to table 9, which directs the model to minimize the target property. The de- sired ranges for these task properties are set between the 25th percentile of the ZINC dataset and a predetermined maximum threshold (see Tab. 8 and 9 for specifics).
TPSA
A-GEN w/ RTB
79
1.0
0.56
94.6
0,80
1.0
Table 4. Comparing the offectiveness of finetuned A-GFN over A-GFN trained from scratch for TPSA adjusted property targeting objective. Detailed comparision in table 11
6.4.3. PROPERTY CONSTRAINED OPTIMIZATION
The finetuned A-GFN significantly outperforms the A-GFN trained from seratch, achieving superior results within the same computational budget. In the dynamic range adjust- ment scenario, we modify one of the pretraining condition- als (TPSA) by shifting its range from 60 < TPSA < 100 to both lower (40 TPSA 60) and higher (100- < TPSA < 120) ranges. In this case, the finetuned A-GFN again surpasses its non- non-pretrained counterpart, demonstrat- ing faster convergence and higher success rates (Tab.5). We also explore the hybrid online-offlin finetuning approach, with and without the RTB objective, where A-GFN lever-
For this comprehensive benchmarking task, the aim is to generate molecules that optimize specific target prop- as M.W., logP, or toxicity, while preserv- ing essential drug-like characteristics learned during pre- training. We evaluate two key finetuning paradigms: conditionals- prreeervd finetuning and Dynamic Range Ad- justment (DRA), demonstrating that finetuning with RTB consistently improves performance across all ADME-T
7

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
ages offline task-specific data, with the desired Cp similar to pretraining. In some tasks, hybrid finetuning with RTB outperforms other approaches for the metrics shown in the tables referenced above. However, this performance gain is accompanied by an inductive bias toward the data distri- bution of the offline fine-tuning dataset, particularly when its support is limited. Even though these are vast chemi- cal spaces to be explored, evident through the number of modes metric, we also observe that a finetured A-GFN with RTB significantly outperforms all baselines, by consistently achieving greater chemical diversity (RWTD) metric).
7.1. De novo Generation
Following prior works (Lee et al., 2023b;a), we gener- ate molecules for five protein targets, optimizing dock- ing score (DS), QED, and SAS. Similar to Saturn (Guo & Schwaller, 2024b), benchmarking is conducted with Quick Vina2 (Alhossary et al., 2015), and we report hit ratio and novel hit ratio ($E). Finetuning employs DRA- -easedded property-constrainedd optimization with modified constraints on # r rings and dp =1 1.
We evaluate two A-GFN priors: (i) trained solely on offline data (Enamine 6.75M, 18 days, 4 A100-40GB GPUs) and (ii) a hybrid online-offline prior (ZINC250K, 12 days, same hardware). With both priors, a high inverse-temperature finetuned A-GFN (3 >> 1) matches or surpasses Saturn in hit ratio (Tab. 21). However, under a strict novelty constraint (Tanimoto similarity S 0.4), the offline-only prior underper- forms in novel hit ratio. This arises because RTB finetuning keeps the policy anchored to its prior, biasing generation toward known scaffolds.
For example, Table 13 shows results from a challenging molecular generation task where the goal is to generate molecules with 1ogP=1.55 while maintaining the conditional properties outlined in Tab. 9. The inherent difficulty of this task is underscored by the fact that only 0.002% of molecules in the ZINC250K dataset meet these stringent criteria. Despite the challenge, A-GFN finetuned with the RTB objective achieves better metrics than other baselines. This underscores the potential of hybrid online-offiine fine- tuning to unlock otherwise inaccessible regions of the chem- ical landscape, offering a promising strategy for tackling difficultto-samplle molecular spaces.
For applications prioritizing structural novelty, finetuning the hybrid prior significantly improves exploration while maintaining competitive hit rates, outperforming all base- lines (Tab. 6, 22). This highlights the benefit of integrating offline expertise with online exploration to enhance molec- ular diversity while preserving biological relevance. Full experimental details are provided in Appendix F.5.
Results from the experiments in the DRA paradigm, where we modify one of the conditional properties like TPSA, re- inforce the fact that A-GFN with RTB enables faster adapt- ability when transitioning to new property distributions. All our results establish RTB as an effective strategy for constrained molecular property optimization, consistently reaching greater diversity, and improved adaptability when compared to other previous state-of-the-arr baselines such as MolDQN, REINVENT, and Graph MCTS.
Table 6. Percentage improvement by A-GFN finetuning of the hy- brid prior over the best- performing method reported in (Guo & Schwaller, 2024b). Detailed benchmarking results are provided in Tables 21 and 22.
Target
Percent Improvement
Hit Ratio
Novel Hit Ratio
parp fa7
31,72
37.66
7. Case Study
148.35
34.73
Shtlb
43.59
35,86
braf
63.00
188.29
In this section, we explore the generation of drug-like molecules by finetuning the pretrained A-GFN prior under property constrained optimization setting for optimizing docking scores-an established in silico drug discovery proxy for protein-ligand binding. We use 5 targets widely adopted in recent studies, namely fa7, barf. 5hIb, parpl, and jak2 (Guo & Schwaller, 2024b). Generating molecules suitable for docking is a challenging task, as valid 3D conformations must be realizable for accurate scoring. We find that models trained from scratch struggle to generate molecules with viable conformers that can be successfully docked. As such, we focus exclusively on finetuning the pretrained A-GFN prior, rather than training a model from scratch for each target.
jak2
60.98
54.29
7.2. Lead Optimization
In lead optimization, we modify a well-performing com- pound to enhance its properties. Here we consider improve- ment of docking score (DS) as example. This score, while common metric. is biased toward compounds with high molecular weight and lipophilicity (Carta et al., 2007; Garefa- Garefa-Ortegón Ortegón et al., 2022), which often results in large, hydrophobic molecules with poor ADME-T properties and off- off-target effects. To address this, we optimize docking scores alongside ligand efficiency (Hopkins et al., 2014), de- fined as - -DS/ # H Alomss, while ensuring structural similarity to the lead compounds to preserve experimentally validated properties.
We use the Demis-Murcke scaffold of the lead (B) as the seed during finetuning (so = B3) with RTB. We do not
8

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
change the property ranges. Through atom-wise decoration of B we improve the docking score and ligand efficiency for each of the 5 targets considered in Sec. 7.1. The best- docked compound from Zinc250K for each target is selected as the lead. Top-2 optimized results per target are shown in Fig. 11.
Impact Statement
This work introduces A-GINs, novel framework for atom- level de novo molecular generation using GFlowNets. From broader perspective, A-GFNs contribute the develop- ment of scalable and flexible molecular design tools, po- tentially expediting early-stage drug discovery and lead op- timization. The ability to generate diverse and pharmaco- logically relevant molecules can aid in identifying candi- date compounds for underexplored or neglected diseases. However, any advances molecular generation technology, including A-GFNs, must be coupled with careful ethical oversight to ensure responsible use in pharmaceutical re- search.
8. Conclusion
In this work, we introduced Atomic GFlowNets (A-GFN) an application of the GFlowNet framework that leverages atoms as fundamental building blocks to explore molec- ular space. By shifting the action space from predefined molecular fragments to individual atoms, A-GFN able to explore a much larger chemical space than its fragment- based counterpart, enabling the discovery of more diverse and pharmacologically relevant molecules. To address the challenges posed by the vastness of this atomic action space, we propose pretraining strategy using datasets of drug-like like molecules. This off-policy pretraining approach conditions A-GFN on informative but computationally inexpensive molecular properties such as drrugliiknnes, topological po- lar surface area, and synthetic accessibility, allowing it to effectively explore regions of chemical space that are more likely to yield viable drug candidates.
Acknowledgments
This research was enabled by funds provided by a Cana- dian Institutes of Health Research (CIHR) doctoral award (FRN: FBDD-185593) to MP. MP would like to further ac- knowledge numerous helpful discussions with Dr. Hong Wang. GS would like to acknowledge the support from the Canada CIFAR AI Chair Program and from the Canada Ex- cellence Research Chairs (CERC) Program. AC would like to acknowledge Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant (RGPIN- 2024-04153). We also thank the Mila cluster, Recursion Pharma compute cluster, and Compute Canada for providing computational resources.
In comparison with models trained from scratch, we show that models pretrained with both expert trajectories and self- generated online samples lead to improved diversity and novelty, as well as lower amortized computational cost.
However, akin to challenges in finetuning RL models (Wol- czyk et al., 2024), A-GFN is prone to catastrophic forgetting of its pretraining- -hase knowledge when extensively fine- tuned. This highlights the utility of methods like relative trajectory balance (Venkatraman et al., 2024) that regular- ize the finetuning process, ensuring that the policy stays anchored to a priort while still adapting to task-specific goals.
References
Ahn, S., Kim, J., Lee, H., and Shin, J. Guiding deep molec- ular optimization with genetic exploration. Advances in neural information processing systems, 13:12008-12021, 2020.
Alhossary, A., Handoko, S. D., Mu, Y., and Kwoh, C.- K. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatiic, 31(13)22214-2D 2216, 2015.
While pretraining has been the hallmark of other generative frameworks (Brown, 2020), we believe that this work is the first to demonstrate a large scale, useful application of pretraining within the GFlowNet framework. Some obvi- ous next steps thus include deeper analysis of the scaling behaviors of this pretraining phase (Kaplan et al., 2020), in terms of amounts of expert data (of which there are poten- tially vast quantitics in chemistry). of the size of models, and of the overall length of the pretraining phase. Indeed, thanks to its online/on-policy pretraining component, the model has, in principle, access to its entire state space. This means such models could in principle learn about the (sim- ple) properties of all accessible compounds, given enough time, and become powerful priors to draw upon.
Atanackovic, L., Tong, A., Wang. B., Lee, L. J., Bengio, Y., and Hartford, J. Dyngfn: Towards bayesian inference of gene regulatory networks with gflownets. 2023. URL https:/ http///axvvv.../// org abs 22022 04178.
Atance, S. R., Diez, J. V., Engkvist, O., Olsson, S., and Mer- cado, R. De novo drug design using reinforcement Iearn- ing with graph- deep generative models. Journal of chemical information and modeling, 62(20)44833-487. 2022.
Bengio, E., Jain, M., Korablyov, M., Precup, D.,, and Ben- gio, Y. Flow network based generative models for non- iterative diverse candidate generation. Advances in Neu-

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
ral Information Processing Systems, 34:27381-27394 2021.
Garcfa-Ortegón, M., Simm, G. N., Tripp, A. J., Hernández- Lobato, J. M., Bender, A., and Bacallado, S. Dockstring: easy molecular docking yields better benchmarks for lig- and design. Journal of chemical information and model- ing, 62(15))3486-3502 2022.
Bengio, Y. Lahlou, S., Deleu, T., Hu, J., Tiwari, M., and Bengio, E. Gflownet foundations. Journal of Machine Learning Research, 24(210)1-555, 2023. URL http: //jmmr.org wwtprrr..ggppp.//////e/// wttpppapppspa//// 0364244-344.. html.
Goel, M., Raghunathan, S., I Laghuvarapu, S., and Priyaku- mar, U. D. Molegular: molecule generation using rein- forcement learning with alternating rewards. Journal of Chemical Information and Modeling, 61(12):5815-5826. 2021.
Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S.,
and Hopkins, I. I.. Quantifying the chemical beauty of drugs. Nature chemistry. 4(2):00 98, 2012.
Bilodeau, C., Jin, W., Jaakkola, T., Barzilay, R., and Jensen, K. F. Generative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5):e 168, 2022.
Guo, J. and Schwaller, Augmented memory: Sample- efficient generative molecular design with reinforcement learning. Jacs Au, 2024a.
Guo, J. and Schwaller, P. Saturn: Sample-efficient genera- tive molecular design using memory manipulation. arXiv preprint arXiv: 17066, 2024b.
Bos, J. D. and Meinardi, M. M. The 500 dalton rule for the skin penetration of chemical compounds and drugs. Experimental Dermatology: Viewpoint, 9()):165-169, 2000.
He, H., Chang, C., Xu, H., and Pan, L. Looking backward: Retrospective backward synthesis for goal-conditioned gflownets, 2024. URL https: /ttxxvivvvvvv//////// org/ / 2406. 01150.
Brown, N., Fiscato, M., Segler, M. H., and Vaucher, A. C. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 19(3):199-1108, 2019.
Hopkins, A. L., Keserü, G. M., Leeson, P. D., Rees, D. C., and Reynolds, C. H. The role ligand efficiency metrics in drug discovery. Nature reviews Drug discovery, 13(2): 105-121, 2014.
Brown, T. B. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165,,,00.,,66,,, 2020.
Buckman, J., Roy, A., Raffel, C., and Goodfellow, I. Ther- mometer encoding: One hot way to resist adversarial examples. In International conference on learning repre- sentations, 2018.
Hu, E. J.. Malkin, N., Jain, M., Everett, K., Graikos, A., and Bengio, Y. Gflownet-em for learning compositional latent variable models, 2023. URL https: // /arxiv. org/ abs 2202 06576.
Carta, G., Knox, A. J., and Lloyd, D. G. Unbiasing scoring functions: normalization and rescoring strategy. Journal of chemical information and modeling, 47(4): 1571,157,220 2007.
Huang, K., Fu, T., Gao, W., Zhao, Y., Roohani, Y., Leskovec, J., Coley, C. W., Xiao, C., Sun, J., and Zitnik, M. Ther- apeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint 09548, 2021.
Deleu, T, Góis, A., Emezue, C., Rankawat, M., Lacoste- Julien, S., Bauer, S., and Bengio, Y. Bayesian structure learning with generative flow networks. In Cussens, J. and Zhang, K. (eds.), Proceedings of the Thirtt-Eighth Conference on Uncertainty in Artificial Intelligence, vol- ume 180 of Proceedings of Machine Learning Research, pp.
Huang, K., Fu, T., Gao, W., Zhao, Y., Roohani, Y., Leskovec, J.. Coley, C. W., Xiao, C., Sun, J., and Zimik, M. Artificial intelligence foundation for therapeutic science. Nature chemical biology, 18(10):0033-1033 2022.
PMLR, 2022.
Jain, M., Bengio, E., Hernandez-Garcia, A., Rector- -Boook, J., Dossou, B. F. P., Ekhote, C. A., Fu, J., Zhang, T., Kil- gour, M., Zhang, D., Simine, L., Das, P., and Bengio, Y. Proceed- Biological sequence design with GFlowNets. ings of the 39th International Conference on Machine Learning, pp. 9786-9801, PMLR, 2022.
Eckmann, P. Sun, K., Zhao, B., Feng, M., Gilson, M. K., and Yu, R. Limo: Latent inceptionism for targeted molecule generation. Proceedings of machine learning research, 162:5777, 2022.
Fedus, W., Ramachandran, P., Agarwal, R., Bengio, Y., Larochelle, H., Rowland, M., and Dabney, W. Revisit- ing fundamentals experience replay. In International conference on machine learning, pp. 3061-3071. PMLR, 2020.
Jain, M., Raparthy, S. C., Hernández-Garue A., Rector- Brooks, J., Bengio, Y., Miret, S., and Bengio, E. Multi- objective gflownets. In International conference on ma- chine learning, pp. 14631-14655 PMLR, 2023.
10

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Jensen, J. H. A graph-based genetic algorithm and gener- ative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):35773357, 2019.
Li, G., Xiong, C., Thabet, A., and Ghanem, B. Deep- ergen: All you need to train deeper gens. arXiv preprint urXiv:2006.07739 2020.
Liu, H. and Abbeel, P. Behavior from the void: Un- supervised active pre-training, 2021. URL https:
Jeon, W. and Kim, D. Autonomous molecule generation using reinforcement learning and docking to develop po- tential novel inhibitors. Scientific reports, 10(1):22104, 2020.
/
abtp:/bs/221000 04551.
Loeffler, H. H., He, J., Tibo, A., Janet, J. P., Voronov, A., Mervin, L. H., and Engkvist, O. Reinvent 4: Modern ai-driven generative molecule design. Journal of Chem- informatics, 16(1):20, 2024.
Jin, W., Barzilay, R., and Jaakkola, T. Junction tree vari- ational autoencoder for molecular graph generation. In International conference on machine learning, pp. 2323- 2332. PMLR, 2018
Luo, Y., Yan, K., and Ji, S. Graphdf: A discrete flow model for molecular graph generation. In International conference on machine learning, pp. 7192-7203. PMLR, 2021.
Jin, W., Barzilay, R., and Jaakkola, T. Hierarchical gen- eration of molecular graphs using structural motifs. In International conference machine learning, pp. 4839- 4848. PMLR, 2020a.
Malkin, N., Jain, M., Bengio, E., Sun, C., and Bengio, Y. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955-5967, 2022.
Jin, W., Barzilay, R., and Jaakkola, T. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pp. 4849- 4859. PMLR, 2020b.
Malkin, N., Lahlou, S., Deleu, T., Ji, X., Hu, E., Everett, K., Zhang, D., and Bengio, Y. Gflownets and variational inference, 2023. URL https: https/////xxvvv.rr///o 2210 00580.
Jo, J., Lee, S., and Hwang, S. J. Score-based generative modeling of graphs via the system of stochastic differen- tial equations. In International conference on machine learning, pp. 10362-1083 10383. PMLR, 2022.
Meyers, J., Fabian, B., and Brown, N. De novo molecular design and generative models. Drug discovery today, 26 (11):2707-2715, 2021.
Kalapos, A. and Gyires-Tónee B. Cun-jepa: Self- supe-super pretraining convolutional neural networks using joint embedding predictive architecture. arXiv preprint S5ti:22408.055/444 2024.
Mouchlis, V. D., Afantitis, A., Serra, A., Fratello, M., Papa- diamantis, A. G., Aidinis, V., Lynch, I., Greco, D., and Melagraki, G. Advances in de novo drug design: from conventional to machine learning methods. International journal of molecular sciences, 22(():1676, 2021.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
Mutti, M., Mancassola, M., and Restelli, M. Unsupervised reinforcement learning in multiple environments, 2021. URL https:tps//.. //attps://axiv.... arwiv. org/ abs / /21/1111.11. 08746.
Kim, Park, S., and Levine. S. Unsupervised-to-ollm reinforcement learning, 2024. URL htt.t:t//////ititt arg/abs/2400 14785.
Nigam, A., Friederich, P., Krenn, M., and Aspuru Guzik, A. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. arXiv preprint urXiv:1909. 1655, 2019.
Kong, X., Huang, W., Tan, Z., and Liu, Y. Molecule gen- eration by principal subgraph mining and assembling. Advances in Neural Information Processing Systems, 35: 2550-2563, 2022.
Nishibata, Y. and Itai, A. Automatic creation of drug candi- date structures based on receptor structure. starting point for artificial lead generation. Tetrahedron, 87(43):8985 8990, 1991.
Lee, S., Jo, J., and Hwang, S. J. Exploring chemical space with score- based distri--ittiittoonn generation. In Inter- national Conference on Machine Learning, pp. 18872- 18892. PMLR, 2023a.
Notwell, H. and Wood, M. W. Admet property prediction through combinations of molecular fingerprints. arXiv preprint arXiv:2310.00/77 2023.
Lee, S., Lee, S., Kawaguchi, K., and Hwang. S. J. Drug discovery with dynamic goal-aware fragments. arXiv preprint arXiv:2310 1..00040 2023b.
Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9:1-14, 2017.

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Pan, L., Jain, M., Madan, K., and Bengio, Y. Pre-training and fine-tuning generative flow networks. arXiv preprint arXiv:2310.03419, 2023.
Tong, S., Dong, Q., Dai, D., Song, Y., Liu, T., Chang, B., and Sui, Z. Robust fine-tuning via perturbation and interpolation from in-batch instances. In Raedt, L. D. (ed.). Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, 4397- 4403, International Joint Conferences on Artificial Intel- ligence Organization, 2022. doi: 10.24963/ijcai 2022/
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling B., Golo- vanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Ar- tamonov, A., Aladinskiy, V., Veselov, M., Kadurin, A., Johansson, S., Chen, H., Nikolenko, S., Aspuru-Guzik, A., and Zhavoronkov, A. Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Mod- els. Frontiers in Pharmacology. 2020.
610. URL https: / /d//iii... pooi.rrp orgg 10 20962/49933333 2022 / 610. Main Track.
Veličkovié, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. Graph attention networks. arXiv preprint 1tt:::1710.199000 2017.
Popova, M., Isayev, O., and Tropsha, Deep reinforcement learning for de novo drug design. Science advances, 4(7): eaap7885, 2018
Vemgal, N., Lau, E., and Precup. D. An empirical study the effectiveness of using a replay buffer on mode discovery in gflownets. arXiv preprinl urXiv:230.006674 2023.
Roy, J., Bacon, P.-L., Pal, C., and Bengio, E. Goal- conditioned gflownets for controllable multi-objectivv molecular design, 2023. http::/:: /ttp:://axvvvvvooo abs/2306 2306. 620.
Venkatraman, S., Jain, M., Seimeca, L.,, Kim, M., Sendera, M., Hasan, M., Rowe, L., Mittal, S., Lemos, P., Bengio, E., et al. Amortizing intractable inference in diffusion models for vision, language, and control. arXiv preprint arXiv:2405.20977 2024.
Sangster, J. M. Octanol-water partition coefficients: funda- mentals and physical chemistry, volume John Wiley & Sons, 1997.
Schaul, T., Horgan, D., Gregor, K., and Silver, D. Universal value function approximators. In International conference on machine learning, pp. 1312-1320. PMLR, 2015.
Wellnitz, J., Martin, H.-J., Anwar Hossain, M., Rath, M., Fox, C., Popov, K. I., Willson, T. M., Muratov, E. N., and Tropsha, Stoplight: hit scoring calculator. Journal of Chemical Information and Modeling, 2024.
Shen, T., Seo, S., Lee, G., Pandey, M., Smith, J. R., Cherkasov, A., Kim, W. Y., and Ester, M. TacoGFN: Target-conditioned GFlownet for structure-based drug de- sign. Transactions Machine Learning Research, 2024. ISSN 2835-8856. URL https: net / iatoorumiddNBePPP http::://ppeneeeeeee /pperrevie..
Wolezyk, M., Cupiat, B., Ostaszewski, M., Bortkiewicz, M., Zajac, M., Pascanu, R., Kuciñski, E., and Miloš, P. Fine-tuning reinforcement learning models is se- cretly a forgetting mitigation problem. In Salakhut- dinov, R., Kolter, 7.,, Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceed- ings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learn- ing Research, pp. 53039-53078. PMLR, 21-27 Jul 2024. URL hetph:ttpss..m http:::::/pprcceeeee http://pooceeeeeeee///ee mlr wlr.press/////// v235 1235/wolld htt5//ool2yk2222 html.
Shi, C., Xu, M., Zhu, Z., Zhang, W., Zhang, M., and Tang, J. Graphaf: a flow-based autoregressive model for molec- ular graph generation. arXiv preprint wrXiv:200100938, 2020a.
Shi, Y., Huang, Z., Feng. S., Zhong, H., Wang. W., and Sun, Y. Masked label prediction: Unified message passing model for supi-supe classification. arXiv preprint 00Xi::2009.03509, 2020b.
Xic, Y., Shi. C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103. 10432, 2021a.
Sterling, T. and Irwin, J. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324-2333 2015.
Xie, Y., Xu, Z., Ma. J., and Mei, Q. How much space has been explored? measuring the chemical space covered by databases and machine-generated molecules. arXiv preprint ar 12542, 2021b.
Tingle, B. I., Tang, K. G., Castanon, M., Gutierrez, J. Khurelbaatar, M., Dandarchuluun, C., Moroz, Y. S., and Irwin, J. J. Zinc-22-a free multi-billion-scale database of tangible compounds for ligand discovery. Journal of chemical information and modeling, 1364111166 2023.
Yang, S., Hwang, D., Lee, S., Ryu, S., and Hwang, S. J. Hit and lead discovery with explorative rl and fragment- molecule generation. Advances in Neural Information Processing Systems, 34:7924-7936, 2021.
12

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
You, Liu, B., Ying, Z.., Pande, V., and Leskovec, J. Graph convolutional policy network for goal-directed molecu- lar graph generation. Advances in neural information processing systems, 31, 2018.
Yuan, H., Yuan, Z., Tan, C., Huang, F., and Huang, S. HyPe: Better pre-trained language model fine-tuning with hid- den representation perturbation. In Rogers, A., Boyd- Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Compula- tional Linguistics (Volume 1: Long Papers), pp. 3246- 3264, Toronto, Canada, July 2023. Association for Com- putational Linguistics. doi: 10. 10.18665//1122233ac/oogzz 182. URL httpt::/t//e...... //tp:://ccllloo www.llaaattolloooooooo..... acl 1al-long.o 182.
Zhang, D., Malkin, N., Liu, Z., Volokhova, A., Courville, A., and Bengio, Y. Generative flow networks for discrete probabilistic modeling. In Chaudhuri, K., Jegelka, S., Song. L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Ma- chine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 26412-26428. PMIR, 17-23 Jul 2022.
Zhang, D. W., Rainone, C., Peschl, M., and Bondesan, R. Robust scheduling with gflownets, 2023. URL https: //arxiv.orge abs/ 2302. 05446.
Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P. Op- timization of molecules via deep reinforcement learning. Scientific reports, 91):10752, 2019.
Zhu, Y., Wu, J., Hu, C., Yan, J., Hou, T. Wu, J., et al. Sample-efficient multi-objective molecular optimization with gflownets. Advances in Neural Information Process- ing Systems, 36, 2024.
13

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Appendix
A. Markoy Decision Process
Reward
Reward
Ituprofen
Reward
Logits Masking
Naprowon
CH4
Reward
Valence Violation
Katoprofen
2Hg
irturnonstaanssutloaridd
Figure 2. MDP governing the construction of molecules with A-GFN.
B. Related Works
B.1. Gel
GFlowNets were introduced by Bengio et al. (2021) as framework for training energy-based generative models that learn to sample diverse candidates in proportion to given reward function. Unlike traditional Reinforcement Learning (RL) methods, which focus on maximizing rewards through sequence of actions, GelowNets aim to generate samples with probabilities proportional their associated rewards. This distinction allows GFlowNets to explore a broader solution space, facilitating the discovery of novel, high-quality, and diverse objects across various domains. Recent works have been conducted along multitude of directions, some focusing theoretical aspects, such connections variational methods (Zhang et al., 2022; Malkin et al., 2023), and some focused on improved training methods for better credit assignment and sample efficiency (Malkin et al., 2022; Bengio et al., 2023). Due to their flexibility, GFlowNets have also been successfully applied to different settings such as biological sequences (Jain et al., 2022), causal discovery (Deleu et al., 2022; Atanackovic et al., 2023), discrete latent variable modeling (Hu al., 2023), and computational graph scheduling (Zhang et al., 2023).
B.2. Unsupervised pretraining in RL and GFlowNets
Pretraining models machine learning has virtually become the default way to obtain powerful models (Brown, 2020; Kalapos Toites-Tóth, 2024). Specifically, unsupervised pretraining reinforcement learning (RL) has emerged as a promising strategy to enhance data efficiency and improve agent performance across various tasks. Recent works have explored different methodologies leverage unsupervised interactions with the environment before finetuning on specific objectives. For instance, Liu & Abbeel (2021) introduced Active Pre-Training reward-free pre-training method that maximizes particle-based entropy in contrastive representation space, achieving human-level performance several Atari games and significantly enhancing data efficiency in the DMControl suite. Similarly, Mutti et al. (2021) addressed unsupervised RL. multiple environments, proposing framework that allows for pre- pre-training across diverse scenarios to improve the agent's adaptability. Additionally, the Unsupervised- to -oaiedd-t-OOOieee RL framework (Kim et al., 2024) was developed, which replaces domain speciffc offline RL. with unsupervised pre-training, demonstrating that a single pre-trained model
14

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
can be effectively reused for multiple downstream tasks, often outperforming traditional methods. Recent advancements in GFlowNets have also introduced unsupervised pre-training strategies, such as the autcome-conditiones GFlowNet (Pan et al., 2023), which enables reward-free pre-training by framing the task as self-supervised problem. This approach allows GFlowNets to learn to explore the candidate space and adapt efficiently to downstream tasks, showcasing the potential of unsupervised pre-training in enhancing the performance of generative models in molecular design.
B.3. Goal-Conditioned and Multi-Objectiin Gflownets
GFlowNets have been extended to conl-condi and multi-objective frameworks, enhancing their applicability in complex generative tasks. Goal-conditioned GFlowNets enable the generation of diverse outputs tailored to specific objectives, improving sample efficiency and generalization across different goals, as demonstrated by Jain et al. (2022) and further refined through methods like Retrospective Backward Synthesis (He et al., 2024) to address sparse reward challenges. Roy et al. (2023), impose hard constraints through focus regions as goal-design strategy, making it comparable to goal- conditioned reinforcement learning (Schaul al., 2015). Additionally, the development of multi-objective GFlowNets (Jain et al., 2023) allows for simultaneous optimization of multiple criteria, providing practitioners with greater control over the generative process and the ability to explore the Pareto fronts of trade-offs between competing objectives.
B.4. Molecule Generation
To contextualize our approach, we provide brief review of prior research using atom-based vocabularies in molecular generative modeling. Reinforcement learning (RL) studies optimal decision-making methodologies to maximize cumulative rewards. Given the shared notion of object-constructing Markov decision policies between GFlowNets and RL, we focus on the latter literature for molecule generation. For a comprehensive review of methods in molecule generation, we point the readers to the work of Bilodeau et al. (2022). RL is frequently employed in de novo design tasks due to its capability to explore chemical spaces beyond the compounds present in existing datasets. Moreover, RL allows for targeted molecule generation for constrained property optimization. Early works focused on auto-regressive generation of SMILES within an RL-loop for molecular property optimization (Olivecrona et al., 2017; Popova et al., 2018; Goel et al., 2021). MolDQN (Zhou et al., 2019) employs deep Q-Networks and multi-objective molecular properties for scalarization of rewards to generate 100% valid molecules without pretraining on dataset. You et al. (2018) and Atance et al. (2022) employed graph neural networks to generate molecular graphs. These are trained on offline molecular datasets as well as through policy-gradient updates to ensure that the generated molecules adhere to specified property profiles.
C. Pretraining Setup
For our pretraining, we utilize ZINC250K, curated subset of the ZINC database, which consists of 250,000 commercially accessible drug-like compounds drawn from over 37 billion molecules available in ZINC (Tingle et al., 2023).
The primary goal during pretraining is to optimize the A-GFN for generic yet critical drug-like properties: TPSA, QED, SAS, and the number of rings. The desired ranges for these properties are enumerated in Tab. These properties are chosen based on their established relevance in guiding molecular design towards compounds with desirable pharmacokinetic and pharmacodynamie profiles, following the framework of (Wellnitz et al., 2024). By optimizing for these properties, we aim to equip A-GFN with the capability to generate molecules that strike balance between drug-likened and structural diversity.
We restrict atom types to a core set commonly found in drug-like molecules: C,S, P, N, O, F, Cl, Br, I, and implicit hydrogen (H). This ensures that the generated molecules are synthetically relevant and pharmacologically plausible, avoiding rare or exotic atom types that are less likely to lead to viable drug candidates.
For benchmarking fragment- based GelowNets, we adapt the purely online training setup proposed in Jain et al. (2023), conditioning it on the same molecular properties as A-GFN. To align with our A-GFN setup, we generate a fragment vocabulary from the BRICS decomposition of ZINC250K, selecting the 73 (following (Bengio al., 2021)) most common fragments. This equips the fragment-based GFlowNet with a diverse and representative set of building blocks for molecular generation.
Through pretraining, we observe that A-GFN effectively adapts to the specified property ranges while maintaining high molecular diversity and uniqueness. All molecules generated by A-GFN are valid, adhering to chemical rules, and novel, as they do not replicate any molecules in the ZINC250K dataset (Fig.3).
15

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
For the same of property conditionals, A-GFN demonstrates superior chemical scaffold exploration compared the tragment-based GFlowNet, covering nearly twice many distinct scaffolds (Tab.2). While the fragment-based method has higher success rate and better control over specific molecular properties, A-GFN excels in uniqueness and novelty, making it more powerful tool for exploring uncharted chemical space in drug discovery. This highlights the model's capacity to explore novel regions of chemical space while adhering to fundamental molecular design principles.
Pretraining was done on Nvidia A100-40G GPUs for 12 days.
C.1. Architecture
We use graph neural network that is mainly a graph attention network (Veličković et al., 2017), using specifically the variant proposed by Shi et al. (2020b). To help the model with counting-like behavior, we also include convolutional layers with additive aggregation, following Li et al. (2020), whose output is concatenated with the input of each graph attention layer.
16

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Table 7. Pretraining and finetuning setups and corresponding hyperparameters
Pretraining
FT(+ RTB)
FT (# RTB) w/data
Hyperparameter
Values
Values
Values
max_num_iter iter
500,000
bootstrap_owe owwneerar__o_eewaarrd
FALSE
random_seed
1,428,570
beta
96
64
OOB @or_percent
0.1
zinc_rad_scale (A)
1
gin_batch_st
FALSE
reward reward_aggree aggreeation
mul
sampling batch size
2048
1024
training batch size
64
learning_ rate
1.00c-04
online mnline_offline mix_ _atio
0.5
num_ workers
2
gfn_lose eff__lss_ooe loss_coefff (A1)
0.04
0.04
MIFF_coeff (A2)
20
20
num_emb
128
num_layers
8
num_mip_layen
4
num_heads
2
i2h_wiie width
illegal_action action_tiiooo logreward
-512
reward reward_loss_mune moss_multiplier
weight_
1.00e-08
mun__aaa__loder_worre
8
momentum
0.9
adam_eps
1.00e-08
dr_decay
20,000
Zr dr_decay
20,000
siip_grad_yy _tpp
norm
clip_ clip_grad__ Leagggrrrrrreeeg
param
10
aandom_atiion_o
0.001
Landom_stop_prott
0.001
num num_back, Bar___acc_ssts____ee_e
stsps_tepaaaa max
25
max_ max_traj_len
40
max _od_n
45
max_edges
50
tb j_.p_biis_prramtterize
TRUE
mon_thermmomttr_dimm
16
sample_temp
1
checkpoint_every
1,000
500
learning rate
le-3
17

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
QED: 0.67, TPSA: 73.9,
QED: 0.65, TPSA: 76.1,
QED: 0.79, TPSA: 77.0,
QED: 0.65, TPSA: 90.5,
Rinas: 3. SAS: 2.79
Rinas: 2. SAS: 3.65
Rinas: 2. SAS: 3.13
Rinas: 2. SAS: 2.76
QED: 0.65, TPSA: 84.2,
QED: 0.54, TPSA: 70.7.
QED: 0.38, TPSA: 72.8.
QED: 0.76, TPSA: 61.9.
Rings: 1, SAS: 1.96
Rings: 2, SAS: 3.20
Rings: 1, SAS: 2.86
Rings: 3, SAS: 3.40
QED: 0.79, TPSA: 91.0,
QED: 0.63, TPSA: 67.6, Rinas: 2. SAS: 2.45
QED: 0.67, TPSA: 69.6, Rinas: 3, SAS: 2.87
QED: 0.46, TPSA: 72.2. Rinas: 2. SAS: 2.54
Rinas: 3, SAS: 2.67
QED: 0.70, TPSA: 62.2,
QED: 0.57, TPSA: 87.4,
QED: 0.83, TPSA: 84.4, Rings: 3, SAS: 3.04
QED: 0.66, TPSA: 54.0, Rings: 3, SAS: 2.62
Rings: 3, SAS: 4.53
Rings: SAS: 3.09
18

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Nr
QED: 0.71, TPSA: 78.5,
QED: 0.61, TPSA: 67,4,
QED: 0.67, TPSA: 12.0, Rinas: 1. SAS: 1.29
QED: 0.69, TPSA: 93.9,
Rinas: 3. SAS: 2.80
Rinas: 2. SAS: 2.68
Rinas: 3. SAS: 3.41
QED: 0.69, TPSA: 78.9,
QED: 0.39, TPSA: 90.9,
QED: 0.60, TPSA: 84.7.
QED: 0.52, TPSA: 76.7.
Rings: 2, SAS: 2.99
Rings: 2, SAS: 3.36
Rings: 2, SAS: 3.24
Rings: 3, SAS: 3.17
N4
QED: 0.74, TPSA: 88.5,
QED: 0.59, TPSA: 41.3, Rinas: 1. SAS: 2.62
QED: 0.78, TPSA: 57.0,
QED: 0.72, TPSA: 70.0, Rinas: 2. SAS: 2.72
Rinas: SAS: 4.30
Rinas: 2. SAS: 2.67
QED: 0.62, TPSA: 73.5,
QED: 70.6. Rings: 2, SAS: 3.01
QED: 0.57, TPSA: 69.8, Rings: 1, SAS: 1.95
QED: 0.81, 84.2,
Rings: 2, SAS: 2.97
Rings: 3, SAS: 3.38
Figure 3. Some randomly chosen successful molecules from the pretrained A-GFN.
D. Conditionals and Rewards
When higher values of molecular property p task, within desired range Cp (Clow Chigh) preferred, preference direction is set to d' > 0 and the corresponding reward as defined in eq.2. Similarly, when lower values are preferred,
derreccedirectton
19

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
(Cow (Cow Pui
exp
if Ps. < Clow
Rp d <0) reward<pp Cp. d <
0.5 * exp
(pr- (Peggn) A -p. -low! (Chigh -low]
if Ps > Chigh otherwise
+
When there is preference i.e. d = 0, the reward is defined
exp
(Siow p=) X (hiihh)
if ps < Clow
Rp(æ|cp, d= 0) reward(ps I dp. 0) = exp
Ps > Chigh
(8)
otherwise
The rate parameter is set for each property individually-
AgED=Asas=ANamttin = =1, and PSA=22..
preference direction senference_directionn
preference_dircction direction <0
1.0
desired range
1.0
desired range
Cuw
Cose
Chigo
Chigh 800
0.8
0.8
0.6
0.6
Rix c2)
R(x
0.2
0.2
200
1000
400
600
preference_diresee direction
1.0
0.8
0.6
desired range
0.2
Cow 100
Chigh 800
0.0
200
400
800
1000
Figure 4, Visual representation of reward as function of preferenco_diveetion
20

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Table 8. TPSA Ranges for Different Finetuning Objectives
Finetuning Objective
TPSA Range
Property Optimization
[60,100.0]
Preserved Property Constrained Optimization
[60,100,0]
DRA Property Constrained Optimization
[100,120.0]/ (40,60,,)
Task
Finetuning Objective
Task Range
Property Optimization
[100,800,-1]
Mol. Wt.
Preserved Property Constrained Optimization
[302,800,-1]
DRA Property Constrained Optimization
1,008000-1]]/330..000,,
Property Optimization
[-5,6,-1]
logP
Preserved Property Constrained Optimization
[-1.65,5,-1]
DRA Property Constrained Optimization
[1.5,5,-1] [2.4,5,1
Property Optimization
[2,6.1]
LD50
Preserved Property Constrained Optimization
[2.3435,5.541.1]
DRA Property Constrained Optimization
[2.12,4.88.1] |
Property Optimization
N/A
Hep. Clearance
Preserved Property Constrained Optimization [3.0,13.18.11]
DRA Property Constrained Optimization
[30,2000-11/133033333-111
Property Optimization
N/A
Mic. Clearance
Preserved Property Constrained Optimization
[5.0.15.0.0]
DRA Property Constrained Optimization
3.0015.0001/55...5500000
Property Optimization
N/A
PPBR
Preserved Property Constrained Optimization [10.09.95.77.-1]
DRA Property Constrained Optimization
12.4.96.7211]1111.1..99444
Property Optimization
N/A DRA Property Constrained Optimization
Caco2
Preserved Property Constrained Optimization |-4.7545,4406.1]
15.44.4.95.1//|445997.
Property Optimization
N/A
Aq.Solv.
Preserved Property Constrained Optimization [-2.798,06623.1]
DRA Property Constrained Optimization
12.710550021111330508817111
Property Optimization
N/A
Lipophil.
Preserved Property Constrained Optimization
[-1.45,4.48.0]
DRA Property Constrained Optimization
1112.377.00|(-1.3.4.29909
Table Task desired property ranges for different finetuning objectives. The QED, SAS, and Num Rings conditional ranges [0.65, 0.8, 0}, [1,3,0], and [1,3,1] respectively across all experiments where the ordering Chight
21

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
E. Evaluation Metrics
In line with earlier works (Brown et al., 2019; You et al., 2018), we employ following set of metrics to ensure a robust comparison across different generative methods.
Validity: A molecule is considered valid if it successfully passes RDKit's sanitization checks. Note that in the proposed method, masking makes this trivially 1.
Diversity: For a set of generated molecules, diversity is defined based on the Tanimoto similarity of Morgan Fingerprint representation of molecules.
Diversity = N(N 1)
2
Σ ISSSISN
IM M, [ U M,
The ratio of distinct canonicalized SMILES strings (without stereachemistry) to the total number of generated molecules, after filtering out duplicates and invalid structures.
Uniqueness:
Novelty: The ratio of unique generated molecules, that are not present in the pretraining ZINC dataset, to the total number of generated molecules.
Time: For each task, a fixed time budget is allotted to A-GFN for finetuning and fask-training. Other baseline methods are allowed to run for time needed to run their default configuration.
#Modes: quantifies the number of distinct, high-reward molecular modes identified by A-GFN model. We define mode as molecule whose reward exceeds threshold, typically set at 0.5. For molecule to be counted as mode, its Tanimoto similarity to any previously identified mode must be less than specified threshold (typically 0.5), ensuring that only sufficiently diverse molecules are counted.
RW#C, RW#S, RWTD: Reward weightt #circles (RW#C), number of scaffolds (RW#S), and Tanimoto diversity (RWTD) are composite metrics that evaluate molecular diversity while prioritizing high-quality candidates. Each metric is computed as the product of the top-K K (typically 100) reward score (Rtopk) )) and the respective molecular diversity measure
This measures how far generated molecule's properties deviate from their target ranges. For each property, the LI distance is calculated by taking the absolute difference between the generated property value and the 10th percentile value in desired range, and then normalized by the respective property range to ensure comparability across properties. This allows us to assess how well the generated molecules meet multi-objective constraints a unified metric.
Normalized Li-dist:
Suecess Percent: For a set of N generated graphs { and conditionals Crask for given task, we define success percentage
Stask
Σ
Σ Le(G.) x 100
N
Crask
eECus
where, L.) counts the molecules within desired conditional ranges and defined as
I [[c(G) Clowl I 0.1 (Coo l),
if dp < 0 if dp > 0 if dp
I.(G)
[(c(G) - Chigh < 0.1 ((nghh)
[Clow I c(G) < Chigh)
where, for any considered task, c(G) is the the value of the conditional property e for the molecular graph G, Ctow and Chigh are the lower and upper bounds of c, dc(c) indicates whether lower values (<c 0), higher values (> 0), or values within range (= 0) are preferred for that conditional.
Hit Ratio: quantifies the percentage of molecules that meet the following criteria: docking score better than the median of known actives, QED 0.5, and SA~ <5 (Guo & Schwaller, 2024b).
Novel Hit Ratio: quantifies the percentage of generated molecules that are both unique and novel (Lee et al., 2023b). Novel molecules are defined as those with a maximum Tanimoto similarity below 0.4 to any molecule in the training set. Hits are defined as molecules meeting the following criteria: DS < the median DS of known actives, QED 0.5, and SA < 5.
22

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F. Experiments
F.1. Ablation Studies
F.1.1. EFFECT OF OFFLINE DATASET SIZE ON PRETRAINING
0.55
En6.75M
Zinc250K
Average Reward
0.50
ZincIM
0.45
0.40
0.35
0.30
0.25
0.20
0
5000
10000
15000
20000
25000
Steps
Figure Smoothed Reward Steps for Drug-like like Datasets. Three different oflline datasets varying sizes such as Zine 250k, Zine 1M, and Enamine 6.75M samples considered
Figure 5 above shows how the average reward evolves during pre-training for three different offline datasets: En6. 75M, Zinc250K, and M.. Smaller datasets (Zinc250K) lead to faster initial learning due to reduced diversity, but plateaus quickly as the model converges quickly to a local optimum. On the other hand, larger datasets (En6.75M) require more steps to converge due to higher diversity, but ultimately achieve better generalization, as seen in the final rewards.
0.6
0.5
Average Reward
0.4
0.3
0.2
Offline Reward (En6.75M)
Online Reward (En6.75M)
Offline Reward (Zinc250K)
0.1
Online Reward (Zinc250K)
Offline Reward (ZincIM)
Online Reward (Zinc1M)
0.0
0
5000
10000
15000
20000
25000
Steps
Figure Offline and Online Rewards Steps for Different Datascts. Three different datascts of varying sizes such Zine 250k, Zinc IM, and Enamine 6,7 6.75M samples are considered
23

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Figure shows the average rewards for offline and online pretraining different dataset sizes. Offline rewards plateau for all datasets, indicating that offline pretraining alone cannot fully explore and works best in combination with the adaptability of online training. Larger datasets like En6.75M enable higher final rewards, which showcases their ability to avoid overfitting.
1.2. EFFECT OF TEMPERATURE (3)
0.5
0.4
Reward
0.3
0.2
0.1
0.0
B = 8
B = 32
B 64 Beta Values
B = 9
Figure Effect of Temperature Parameter /3 on Reward Scale
Figure shows the influence of temperature 3 reward performance, highlighting the trade-off between exploration and exploitation in GFlowNet. Lower values of encourage greater exploration, allowing the model to traverse state space more freely. However, given the vast molecular space, this setting may significantly slow down convergence, as it requires extensive searching before identifying high-reward molecular modes. Higher 3 values result in diversity-speed trade-off as they promote exploitation, enabling the model to focus on high-reward molecules it encounters early in the training. While this improves convergence speed, risks early convergence by overexploiting specific regions of the manifold and failing to discover all possible high-reward modes. Excessively high / values also introduce instability in learning.
F.1.3. EFFECT OF OFFLINE DATA ON PRETRAINING
In Figure 8 we compare the average online reward achieved by an A-GFN model under three different pretraining schemes: Purely Online, Purely Offline and Online Offline. Models with offline pretraining (Purely Offline and Online Offline) achieve higher rewards compared to the purely online scheme. This shows that offline pretraining helps the model learn a good initial representation, reducing the burden on online training. Moreover, the plateau the Purely Offline curve suggests that while offline data provides a good starting point, it lacks the adaptability and exploration benefits provided by online learning. Purely online training collapses under the weight of large state space.
24

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Average Online Reward
0.6
Purely Online
Purely Offline
0.5
Online + Offline
0.4
0.3
0.2
0.1
0.0
0
5000
10000
15000 Steps
20000
25000
Figure 8. Rewards Vs Steps for A-GFN Sampled Trajectories for Different Pretraining Configurations Online, Offline and Online +
Offline
F.1.4. EFFECT OF PARAMETERS SCALING
0.5
5M
0.4
Reward
0.3
0.2
0.1
0.0
6 GTx layers
8 GTx layers Layer Configuration
10 GTx layers
Figure 9. Number of A-GFN Parameters Vs Rewards. Number of parameters written over the bars
Figure 9 demonstrates the benefit of parameter scaling for A-GFN, where increasing the number of Graph Transformer layers leads to higher average rewards. This aligns with the intuition that deeper models provide better representations and this improvement reward suggests that the GFlowNet becomes more efficient at sampling diverse, high-reward trajectories with the increase in number of parameters
25

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.2. Property Optimization
Chem. Space
Task
Method
Diversity
Reward
Coverage
#Madess
RWIC
RwiS
RWTD
L1-diit
Novelty
Uniqueness
Success %
Top-1
Top-10
Top-100
AGEN Task Trained
20.14
2.43
0.12
4.5624
1
0.0009
0.0008
1.00F-D5
AGEN Finetuned
992
79.26
692.97
0.91
0,0683
1.7
0.9999
0.9956
9.80E-01
Mol.Wi Wr.
AGFN Finetuned
489
118.12
240.25
0.83
0.0803
0.835
0.835
0.9841
0.982
9.67F-01
AGEN Task Trained
11.54
5,24
0.16
2.6541
1
0
0.0009
0.0009
1.00E-05
AGEN Finetuned
439.91
0,0855
0.9983
9.86E-01
logP
AGPN Finetuned
185.89
936.42
0.1536
0,998
0.9963
0.9914
9.64E-01
RTB
AGEN Task Trained
6.84
3,94
0.02
1.5617
0.0008
0.0008
1.00E-05
Toxicity
AGEN Finetuned
154.46
6.91
0.16
0.7504
1
0.2688
0.2364
1.87E-01
(LD50)
AGFN Finetuned
43.4
108.90
0.09
0.8574
0.1525
0.1383
1.09E-01
w/ RTB
Table Comparing the effectiveness finetuned alomic-GFlowNett over atomic-GFlowNer trained from scratch tasks with property optimized objective.
F.3. Property Targeting
Chem. Space
MOO metrics
Diversity
Reward
DRA
Method
Coverage
LI-dist J
Success
#Modes
RWa
RWaS
RWTD
Nov.
Uniq.
TPSA
#Rings
SAS
QRD
Top-1
Top-10
Top-100
AGEN Tasktrained
0
0.037
0.037
0.02
0.02
0
0.0373
0
1.5
0.62
2.40
0.03
0.03
0.03
AGEN Finetuned
0
0.0096
0.0064
0,00007
0.037
0.037
0
0.1228
16.243
1.5
0.7209
4.0357
0.1196
0.0318
0.0032
AGEN FT
79
12
62
0.567
0.799
0.8
94.6
0.025
0.028
0.00036
0.0371
1
w/ RTB
AGEN Tasktrained
0.147
0.298
0.0773
0.240
0.240
0.1689
0.002
0.004
1.35
0.163
0.163
0.149
High
AGEN Finetuned
0.0252
0.0168
0.0092
0.006
0.006
0.01444
2.5189
0.1144
2,3395
0.0109
0.0086
0.0084
TPSA
AGEN FT
0.0456
0.0099
RTB
Table 11. Comparing the effectiveness of finetuned A-GFN A-GFN trained from scratch for TPSA property largeting objective.
E.4. Property Constrained Optimization
We present the results of extensive benchmarking across relevant drug discovery tasks. These tasks encompass all regression benchmarks included in the Therapeutics Data Commons (TDC) ADME-T group. The benchmark rewards were generated using a gradient-boosted decision tree CatBoost (Notwell & Wood, 2023) trained on the respective datasets provided by TDC. addition to these, we evaluated two tasks with deterministic rewards computed using RDKit: molecular weight and logP. For each noperty-constraing optimization variant Preserved and DRA (low TPSA and high TPSA) the desired ranges for each task are summarized in Table
26

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
1.0
Average Reward
0.8
0.6
HITPSA FT LoTPSA FT
HITPSA RTB
0.4
LoTPSA RTB
0.2
0.0
4
8
10
12
Log Steps
(a) A-GFN Finctuning: TB RTB objectives
Percent Unique in Batch
1.0
0.8
0.6
0.4
HITPSA FT
0.2
LoTPSA FT
HITPSA RTB
LOTPSA RTB
0.0
2
4
6
8
10
12
Log Steps
(b) Percent Unique in Batch vs Steps
Figure 10. AGEN Finetuning with without RTB objective the effect of overtraining
27

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F,4.1. TASK: MOLECULAR WEIGHT
Chem. Space
MOO metrics
Diversity
Reward
Coveraage
Li-dist
Method
Amodes
RWAC
RWeS
RWTD
Nov
Uniq
HV
TPSA
#Rings
SAS
Task
Top-1
Top-10
Top-100
%
MollXQN
4,992
5.568
0.14
D.0
0.070
0.006
0.79
1.495
2.243
0.051
0,03
0,026
0.016
Reinvent
94
368
0.532
0.998
23.5
0.861
0.237
0.819
0.808
0.781
Frag GelowNet
24
76.725
322.710
0.396
22.0
0.751
0.000
0.035
0.006
0.144
0.094
0.680
0.564
0.465
Preveved
Graph MCTS
100-480
179.294
0.276
1.6
0.645
0
0.371
0.357
0.508
0.06
0.53
0.451
0.314
AGEN Tasktrained
0.001
0.000
0.000
0.1
0.109
5.792
1,500
1.862
2.493
0.1.3
<10%-4
<10^-4
<10%44
AGEN
AGEEN FT
532
314.885
824.023
0.766
2,0
0.986
0.033
0.49%
0.174
0.279
0.0001
0.969
0.943
0.887
data
RTB
AGEN FT
466
349.040
846.422
0.758
0.3
0.990
0.025
0.398
0.259
0.26
0.136
0.957
0.924
0.873
RTB
Reievent
204
19.775
253.120
0.589
0.998
37.2
0.883
0
0.038
0
0.091
0.089
0.847
0.832
0.791
Prag GFlow/et
38
88.528
482.377
0.442
11.2
0.811
0.001
0.174
0.216
0.279
0.003
0.785
0.682
0.503
Low TPSA
Graph
0.690
0.49
0.477
AGEN Tusktrained
0.001
0.056
0.000
0.0
0.002
1.623
1,500
2.065
0.995
0.234
<10%
<10%44
<10^44
AGEN Finctuned
0.060
0.160
0.001
0
0
0.0
0.106
1.9944
1.497
0.002
2.913
1X000
0.081
0.059
0.020
AGEN
data
AGEN FT
786
80.070
897.726
0.772
0.994
0.994
10.0
0.990
0.072
0.061
0.036
0.260
0.0092
0.973
0.967
0.942
ITB
505
0.184
RTB data
MeIDQN
1.944
2.034
0.005
1
0.0
0.029
0.37
0.843
1.513
2.978
0.039
0.015
0.012
0.006
Reinvent
129
11.628
269.496
0.503
37.3
0.758
0.026
0.107
0.088
0.704
0.699
0.584
Frag Geoowerr
61.500
235.500
0.219
2.4
0.592
0.002
0.217
0.394
0.856
0.51
0.412
0.25
Hlggh TPSA
Graph MCTS
44.451
61.776
0.088
1
0.0
0.444
0.005
0.579
0.938
1.871
0.074
0.256
0.213
0.099
AGEN Tasktrained
0.000
0.002
0.000
0.0
0.001
2.653
1.500
1.620
1.423
0.365
<10%<4
<10^44
<10%44
AGEN
AGEN FT
516
3334.83
870.720
0.785
1
8.5
0.998
0.204
0.400
0.188
0.227
0.000
0.975
0.953
0.907
data
0.165
w/ RTB
AGEN FT
467
332.742
130.319
0.771
1
5.5
0.946
0.234
0.364
0.18
0.294
0.122
0.997
0.963
0.891
RTB daiia
Table 12. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property optimization objective for Mol. Wt. task.
28

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.2. TASK: LOGP
Chem. Space
MOO metrics
Diversily
Reward
Coverage
L.Idiit
Method
Buodes
#Kingg
Top-1
Tog-10
Top-100
%
MoiDQN
5.729
30.330
0.255
2.1
0.582
0.353
0.14
0.096
0.175
0.16
0.431
0.337
Reinvent
0.233
0.798
0.787
0.759
Frng GiFlowNet
46.09
366.088
0.461
11.3
0,820
0
0.13
0.056
0.204
0.135
0.711
0.632
0.536
Graph MCTS
0.713
Preservel
AGPN Taskrminedd
0,000
0.004
0.000
0.0
0.0001
0.0K5
0.956
2.862
1.000
0.446
1.00F-04
1.00F:04
<10%4
AGEN Finctuned
43.703
42:468
0.040
0.677
0.086
0.827
2.047
3,622
0.311
0.6267
0.2412
0.0441
AGEN FT
419
29W.290
809,401
0.735
4.0
0.991
0.031
0.417
0.172
0.266
0.167
0.9995
0,779
0.8447
w/ data
71.569
0.793
0.997
0.997
0.996
0.069
0.029
0.9989
0.9981
0.9804
w/ RTB
AGEN
KTB data
MolDQN
6.314
15,860
0.189
0,517
0
0.514
0.292
0.106
0.301
0.302
0.287
0.21
Frng NitlowNet
106
24,880
299.4666
0.534
27.3
0.888
0.093
0.005
0.168
0.169
0.741
0.712
0.622
Low TPSA
Graph MCTS
75.240
164.160
0.392
0.2
0.79
0.493
0.036
0.16
0.234
0.592
0.536
0.156
AGEN Tasktrained
0.000
0.0001
0.351
1.238
0.321
0.654
1.235
0.005
0.002
0.000
AGIEN Finetuned
29.282
28.077
0.027
0.D
0.534
0.181
0.961
1.996
2.918
0.304
0.395
0.146
0.029
0.706
w/ data
AGEN FT
w/ RTB
AGEN FT
112
251.687
633.039
0.556
0.999
0.999
0.986
0.145
0.331
0.288
0.358
0.485
0.970
0.917
0.645
MolDQN
6.383
5.417
0.012
0.116
0.901
1.098
2.114
0.252
0.053
0.03
0.013
Reinvent
102
6.066
324.194
0.463
0.748
0.007
0.131
0.091
0.729
0.699
0.674
GillwWet
High TPSA
Graph MCTS
50.841
85.617
0.165
0.550
0
0.554
0.286
1.011
0.176
0.349
0.264
0.189
0.001
AGEN Finetuned
33.998
32.668
0.031
D.D
0.454
0.288
0.911
2.122
3.941
0.275
0.385
0.198
3.41F-02
AGEN FT
233.989
629.417
0.99%
0.787
0.419
0.953
0.883
0.654
wi data
AGEN FT
490
55.200
898.176
0.741
0.991
0.991
7.6
0.994
0.137
0.206
0.031
0.389
0.153
0.998
0.990
0.936
AGEN FT
260
256.113
744.280
0.667
2.8
0.994
0.468
0.355
0.173
0.316
0.202
0.995
0.941
0.776
IRBB
Table 13. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines
optimization objective for logP task.
29

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.3. TASK: ABSORPTION CELL EFFECTIVE PERMEABILITY
Chemn Space
MOO metries
Diversity
Reward
Coveraage
Li-dist
Method
Amodes
RW#C
RWID
Uniq
#Rings
SAS
Top-1
Top-10
Top-100
McIDQN
0
0.316
0.272
0.001
0.0
0.002
0.000
0.919
0.500
1.167
0.649
0.003
0.001
0.001
Reinvent
1.079
0.005
0.004
0,230
0.004
0.207
0.243
0.008
0.007
0.006
Frag Gel
0.435
0.005
0.006
Preserved
Graph MCTS
0.735
1.584
0.004
4.7
0.004
0.000
0.465
0.047
0.149
0.380
0.006
0.005
0.004
AGEN Tasktrained
0.050
0.100
0.001
0.005
0.200
0.500
0.400
2,000
1,500
0.0044
0.002
0.001
Finetuned
0.786
1.667
AGEN HT
4.822
13.054
0.015
2.2
0.035
0.045
0,350
0.131
0.372
1.149
0.009
0.026
0.017
w/ data
56.242
RTB
AGFN FT
174.124
381.866
0.376
1.0
0.578
0.018
0.340
0.141
0.461
1.204
0.494
0.488
0.431
MoIDON
0.286
Reiment
0.988
5.649
0.006
43.7
0.004
0.000
0.229
0.002
0.305
0.198
0.007
0.007
0.007
Frag GElowNet
0.556
3,931
0.005
32.2
0.004
0.001
0.267
0.058
0.184
0.270
0.007
0.006
0.005
Law TPSA
Graph
0.648
0.004
AGEN Tasktruined
0.080
0,220
0.001
0.006
0.220
0.550
0.420
2.200
1.600
0.005
0.003
0.002
AGEN Finetuned
1.229
0.761
0.001
0.0
0,012
0.161
1.232
1.093
3.527
1.706
0.005
0.003
0.002
FT
3.596
11,280
0.012
0.027
0.301
0.122
0.359
1,286
0.020
0.014
w/ data
AGEN FT
26.970
423.150
0.359
0.0
0.510
0.070
0.092
0.019
0.670
1,227
0.500
0.492
0.465
RTB
143.856
421.356
0.385
0.512
0.275
0.140
1.256
0.495
0.444
w/ RTB data
MoiDQN
0.106
0.097
0.000
0.0
0.000
0.000
0.829
0.848
1.902
0.342
0.000
0.000
0.000
Reinvent
0.427
1.115
0.001
0.002
0.000
0.392
0.175
0.525
0.829
0.002
0.002
0.001
Frag GFlow/et
0.163
1.144
0.001
0.001
0.001
0.186
0.118
0.314
0.668
0.003
0.002
0.002
High TPSA
Graph MCTS
0
0.226
0.428
0.001
0.7
0.001
0.001
0.638
0.181
0.964
0.348
0.002
0.002
0.001
AGEN Tasktrained
0.090
0.360
0.002
40.0
0.007
0.250
0.600
0.450
2.400
1.800
0.006
0.004
0.003
Finetuned
0.299
0.000
0.000
AGEN FT
0.816
4.243
0.007
14.0
0.023
0.351
0.400
0.134
0.368
0.525
0.014
0.011
0.010
w/ data
RTB
AGEN FT
142
111.313
586.348
0.595
10.3
0.993
0.447
0.315
0.129
0.498
0.384
0.966
0.915
0.709
w/ RTB data
Table Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property.constrained optimization objective for Cell Effective Permcability (Caco-2) task.
30

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F,4.4. TASK: ABSORPTION LIPOPHILICITY
Cbemn Space
MOO metrics
Diversity
Reward
Coverage
1.1diist
Success
Nov
Uniq
#Rings
QED
Top-1
TTop-00
Top-100
MoIDQN
1,720
14.190
0.133
0.0
0.423
0.000
0.352
0.248
0.134
0.646
0.302
0.263
0.215
Reinvnnt
119
6.050
271.750
0.103
0.998
0.0
0.723
0.000
0
0
0.058
0,707
0.569
0.563
0.55
Frig
0.335
0.401
0.464
Preserve
MCTS
105.270
0.273
0.504
0.500
0.366
AGEN Tasktrained
0.399
0.238
0.006
0.0
0.058
0.252
1.390
1.362
3,014
0.000
0.021
0.016
0.007
AGEN Finetuned
22.487
17.153
0.021
0.0
0.263
0.082
1.163
1.948
3,929
0.000
0.118
0.086
0.023
1.000
data
AGEN FT
821
277,000
986.00
0.862
48.7
1.000
0.026
0.199
0.074
0.273
0.000
1.000
1.000
1.000
AGENN
0,879
0.310
1.000
RTB ditta
MolDQN
14,190
16.020
0.080
0.0
0.355
0.001
L.085
0.193
0.898
0.31
0.166
0.125
0.09
Reinvent
123
4.004
25-15400
0.394
0,999
0.0
0.728
0.000
0.005
0.796
0.587
0.583
0.572
Prng GelowNer
19.352
0.391
0.723
0.243
0.566
0.517
0.472
Low TPSA
Graph MCTS
48 600
120.600
0.309
0.0
0.6696
0.000
0.469
0.022
0.144
0.625
0.458
0.409
0.36
AGEN Tasktrained
0.384
0.128
0.066
0.0
0.167
0.055
1.500
0.020
1.888
0.000
0.161
0.159
0.128
AGPN
566
318.000
823.000
0.873
30.0
1,000
0.133
0.271
0.113
0.379
0.000
1,000
1.000
1.000
w/ data
AGEN
986.923
0.853
0.275
1.000
AGEEN FT
567
29900000
793.0001
0,879
42.8
1.000
0.117
0.313
0.106
0.297
0.000
1,0000
1 1.000
1.000
w/ RTB data
MoIDQN
18.615
18.462
0.046
0.0
0.213
0.001
0.763
0.642
1.645
0.299
0.08
0.071
0.051
Prng Gilowwer
25.705
165.625
0.229
0.1
0.616
0.001
0.271
0.141
0.334
0.308
0,416
0.362
0.265
High TPSA
Graph MCTS
36.080
60.024
0.142
0.0
0.553
0.001
D.665
0.192
0.954
0.415
0.288
0.257
0.164
Tasktruined
AGEN Finetuned
19.860
12.736
0.018
0.0
0.259
0.543
1.233
2.091
4.212
0.000
0.133
0.073
0.020
AGEN FT
393
296.888
832.667
0.871
30.1
1.000
0.380
0.259
0.162
0.325
0.000
1.000
1.000
1.000
data
AGEN
RTB
AGEN FT
697
20030000
9W9.00
0.845
41.9
1.000
0.283
0.166
0.075
0.341
0.000
1.000
1.000
1.0000
RTB data
Table 15. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property
optimization objective for Lipophilicity task.

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F,4.5. TASK: ABSORPTION SOLUBILITY
Chem. Space
MOO metries
Diversity
Reward
Coverage
LI-dist
Success
Method
Amodes
RW#C
RWES
RWID
Nov
Uniq
HV
TPSA
#Rings
SAS
QED
Task
Top-1
Top-10
Top-100
MoiD)N
5.075
0,390
0.332
0.106
0.026
0.643
0.564
Reinvent
147
8,809
235,435
0.593
0.438
0.000
0.000
0.000
0.146
0.436
0.825
0.816
0.801
Frag GFlowNet
71
21.986
319.588
0.509
11.3
0.426
0.000
0.120
0.014
0.208
0.232
0.711
0.688
0.610
Graph MCTS
14
73.586
155.140
0.401
0.2
0.388
0.000
0.485
0.052
0.153
0.265
0.640
0.553
0.469
Preserved
2.100
Finetuned
25.395
AGEN FT
234
270.200
570,906
0.623
0.978
0.045
0.439
0.162
0.399
1,375
0.956
0.874
0.709
w/ data
AGEN HI
110
20.536
552.660
0.464
0.829
0.029
0.305
0.026
0.454
0.427
0.707
0.669
0.604
116.424
0.261
0.163
0.297
RTB datta
MolDQN
6.497
39.603
0.229
0.0
0.309
0.0000
0.565
0.193
0.158
0.215
0.473
0.410
0.309
Reinvent
121
6,490
456.706
0.567
0.7
0.477
0.000
0.000
0.000
0.005
0.365
0.832
0.827
0.811
Frag GFlowNet
47.068
422.422
0.518
0,455
0.001
0.335
0.059
0.144
0.302
0.785
0.706
0.596
Low TPSA
Graph MCTS
AGEN Tusktrained
2.160
1.100
0.017
0.006
1.133
1.300
1.700
3.500
1,200
0,200
0.080
0.020
AGEN Finctuned
48.913
36,000
0.050
0,473
0.2006
1.199
1.3002
3.122
1.077
0.355
0.148
0.056
AGEN FT
139
188.292
462.885
0.527
0.0
0.964
0.1005
0.399
0.144
0.413
1,414
0.863
0.733
0.604
w/ data
RTB
AGEN FI
78.861
196.746
0.236
0.0
0.693
0.112
0.501
0.092
0.365
1,208
0.457
0.387
0.271
w/ RTH dltta
MelDQN
19.936
20.527
0.036
0.111
0.001
0.724
0.832
1.794
0.352
0.127
0.087
0.039
Reinvent
1.986
83.399
0.435
0.362
0.000
0.000)
0.084
0.452
0.688
0.662
Frag Gelow/Net
34,934
0.312
0.132
0.364
High TPSA
Graph MCTS
46.759
88.056
0.189
0.510
0.001
0.596
0.196
0.941
0.251
0.404
0.343
0.219
AGEN Tasktriined
1.050
1.320
0.013
0.0
0.006
0.997
1.100
1.900
3.700
1.100
0.2.0
0.090
0.015
AGEN Finetuned
25.572
23.104
0.023
0.0
0.463
0.371
1.006
1.970
3.878
1.059
0.385
0.110
0.026
data
AGEN FT
48.158
386.458
0.327
0.566
0.166
0.172
0.044
0.439
1.194
0.500
0.477
0.398
w/ RTB
AGEN ITT
75.816
214.488
0.277
0.493
0.275
0.453
0.090
0.332
1.096
0.453
0.420
0.324
w/ RTB data
Table 16. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property.constrrinee
optimization objective for Aqucous Solubility task.
32

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.6. TASK: DISTRIBUTION PLASMA PROTEIN BINDING RATE
Chem. Space
MOO metrics
Diversity
Reward
Coverage
LI-dist
Success
Method
Amodes
RWaC
RWeS
RWTD
Nov
Uniq
HV
TPSA
#Rings
SAS
QED
Task
Top-1
Top-10
Top-100
%
MolDQN
0.476
10-4
10
110-
Reinvent
0.013
0.062
0.000
0.0
0.000
0.000
0.203
0.006
0.221
0.896
< =10-4
110
110-
Frag Net
0.792
< (10
<.
<
Preseved
Graph MCTS
0.007
00.017
0.000
1,000
0.000
0.448
0.088
0.214
0.841
100
<00-
1000
AGEN Tasktrained
0.000
0.000
0.000
0.0
0.000
0.438
1.155
1.635
2.319
0.648
0.000
0.000
0.000
AGEN Finetuned
0.001
0.000
0.0
0.000
1.037
2.205
4.007
0.753
0.000
0.000
0.000
AGEN FT
0.023
0.061
0.000
0.0
0.000
0.059
0.305
0.124
0.324
0.923
0.000
0.000
0.000
AGEN FT
136
29.210
602.615
0.488
0.0
0.788
0.019
0.145
0.049
0.235
0.454
0.752
0.703
0.635
AGEN FT
132,683
218.800
0.244
0.9
01.993
0,055
0.569
0.200
0.473
0.797
0,411
0.379
0.237
RTB data
MolDQN
10-
-100
Reinvent
0.006
0.019
0.000
0.996
0
0.000
0.000612
0.3195
0.0178
0
0.8828
< 110
< 10
< 110
Frag GillowNer
0.000
<
<. 10
<
Law TPSA
Graph MCTS
0.003
00007
0.000
0.000
0.000109
0.4505
0.0433
0.182
0.8686
100
10
100
AGEN Tasktrained
0.000
0.000
0.000
0.0
0.000
0.000
1.500
0.036
2.383
0.399
0.000
0.000
0.000
AGEN Finetuned
0.000
0.000
0.0
0.237
1.869
3.739
0.783
0.000
0.000
AGRN FT
0.007
0.020
0.000
0.0
0.000
0.161
0.305
0.105
0.414
0.971
0.000
0.000
0.000
AGEN FT
154
26.732
611.576
0.505
0.0
0.818
0.081
0.166
0.062
0.163
0.416
0.742
0.708
0.652
AGEN FT
136.697
234.668
0.257
0.0
0.671
0.188
0.523
0.179
0.394
0.837
0.494
0.409
0.289
RTB dasta
MolDQN
1010
101
1000
Reinvent
0.000
0.000
0.000
0.998
0.1
0.000
0.158375
0.384
0.1572
0.59
0.6318
< 100
< <010
< 100
Frag
110
10-
100
High High TPSA
Graph MCTS
0.000
0.000
0.000
0.000
0.000939
0.5615
0.2006
0.869
0.4183
< 10
10-
10
AGEN Tasktrained
0.000
0.000
0.000
0.0
0.000
1,451
1.270
2.019
2.534
0.301
0.000
0.000
0.000
AGEN Finetuned
0.000
0.000
0.000
0.0
0.000
0.456
1.129
2.134
4.211
0.364
0.000
0.000
0,000
AGEN FT
0.000
0.000
0.000
0.1
0.000
0.524
0.281
0.162
0.377
0.715
0.000
0.000
0.000
AGEN FT
403
64.158
818.448
0
13.1
0.989
0.173
0.147
0.045
0.331
0.302
0.994
0.966
0.867
AGEN FT
12
200.992
325.248
0.310
1.0
0.893
0.395
0.456
0.467
0.637
0.604
0.772
0.648
0.352
RTB
Table Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property-constrained optimization objective for Plasma Protein Binding Rate task.
33

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.7. TASK: EXCRETION HEPATOCYTE CLEARANCE
Chem. Syace
MOO metrics
Diversity
Reward
Coveragee
LI-dist
Method
imodes
RWIC
RWAS
RWTD
Nov
Uniq
IV
TPSA
#Rings
SAS
QED
Task
Top-10
Tup-100
MolDQN
0.806
0.697
0.002
0.022
10
0.952
0.514
1.193
2.798
0.006
0.004
0.002
Prgg GellowNet
1,210
15.245
0.016
10
0.223
2.917
0.023
0.019
Preserved
Graph MCTS
2.291
4,961
0,013
0,059
10
0.479
0.044
0.165
4.798
0.020
0.019
0.016
AGEN Tasktrained
0.006
0.004
0.000
0.0
0.000
0.865
1.165
1.980
2.994
2,405
0.001
0.000
0.000
AGEN FT
9,455
23.994
0.027
0,073
0.071
0,280
0,141
0.326
2.335
0,075
0.050
0.031
w/ data
AGFN
26
9,425
608.275
0.468
0.993
0.063
0.092
0.077
2,026
0.123
0.942
0.894
0.725
AGIEN FT
79.728
152.944
0.154
0,335
0.064
0,380
0,244
0,430
2.202
0,772
0.681
0.176
w/ RTB + data
MolDQN
0.706
0.575
0.002
0.020
10
0.952
0.514
1.193
2.798
0.006
0.004
0.002
Frag GFlowNet
1,862
14.515
0.017
0,060
10
0.110
0.003
0.223
2.917
0,025
0.023
0.019
Low TPSA
Graph MCTS
2.275
5.009
0.014
0.071
10
0.479
0.043
0.166
4.798
0.020
0.019
0.016
AGEN Tasktrained
0.003
0.003
0,000
0.003
0.000
1,500
2.383
0.081
0.003
0.003
0.003
AGEN
0.664
AGINN FT
11.837
31.318
0.036
2.4
0.103
0.181
0.274
0.126
0.503
0.417
0.104
0.062
0.041
w/ data
0.058
0.117
0.266
RTB
AGEN FT
124
254.100
$94.00
0.574
1.2
0.980
0.175
0.298
0.284
1.193
0.292
0.949
0.854
0.660
RTB data
MolDQN
Reinvent
0.566
1.165
0.010
0.000
0.267
2.259
0.017
0.012
Prag GillowNet
0.319
1.027
0,010
0.0
0.045
0.000
0.1.5
0.022
0.07%
1.436
0.018
0.016
0.012
High TPSA
Graph MCTS
0.228
0.396
0.006
0.046
0.000
0.440
0.067
0.265
2.601
0.015
0.011
0.007
Tasktruined
AGEN Finetuned
0.031
0.031
0.002
0.006
0.337
1,500
0.255
3.933
2.463
0.005
0.004
0.004
AGEN FT
3.792
11.439
0.025
3.9
0.048
0.580
0.180
0.090
0.303
0.886
0.045
0.042
0.032
data
AGEN
13,813
710.938
0.581
0.205
0.723
0.363
0.813
RTB
AGEN FT
37
237.442
391
.022
014400
1.0
0.981
0.548
0.426
0.431
0.656
0.817
0.964
0.836
0.454
RTB data
Table 18. Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property.constraineed
optimization objective for Hepatocyte Clearance task.
34

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.8. TASK: EXCRETION MICROSOMAL CLEARANCE
Chem.
MOO metrics
Diversity
Reward
Coverage
I.1-iist
Success
Method
#moods
RW#C
RW#S
RWTD
Nov
Uniq
TPSA
#Rings
SAS
QED
Tusk
Top-1
Tup-10
Top-100
MoiDQN
0.116
0.102
0,000
0.003
0.000
0.928
0.538
1,202
1.022
0.001
0.001
0.000
Reinvent
0
0.468
2.285
0.002
0.998
1
0
0.007
0.000
0.240
0.008
0.222
2.094
0.003
0.003
0.003
Frag GFlowNet
0.176
1.727
0.002
0.2
0.007
0.000
0.194
0.027
0.172
1.495
0.003
0.003
0.002
Preserved
Graph MCTS
AGEN Tasktrained
0
0.004
0.002
0,900
0.0
0.0012
0.532
1,275
1,735
1.885
1.330
0.001
0.000
0.000
AGEN Finctuned
0
0.399
0.348
0.000
0.0
0.000
0.119
1.045
2.155
4.189
3.976
0.004
0.002
0.000
FT
w/ data
AGEN FT
0
2.758
10.704
0.009
1
0.3
0.019
0.021
0.190
0.076
0.252
2,551
0.017
0.014
0.011
RTB
w/ RTB data
MoIDQN
0.102
0.095
0.000
0.003
0.000
1.063
0.430
1.076
0.849
0.001
0.001
0.000
Reinvent
0.504
2.309
0.002
0.007
0.000
0.285
0.006
0.305
0.003
0.003
0.003
Frag GFlowNet
0.250
Law TPSA
Graph MCTS
0
0.290
0.748
0,902
0.007
0.000
0,459
0.021
0.150
2.204
0.003
0.002
0.002
AGEN Tasktrained
0
0.006
0.003
0.000
0.0
0.002
0.820
1,470
0.451
0.445
0.728
0.001
0.001
0.000
Finetuned
0.299
0.269
0,900
0.0
0.005
0.259
0.965
2.210
4.191
0.003
0.001
0.000
AGEN FT
1.659
4.673
0.005
5.6
0.014
0.160
0.268
0.132
0.670
1.017
0.009
0.008
0.006
w/ data
AGEN FT
2.496
10.254
0.009
0.1
0.018
0.155
0.233
0.118
0.253
2.822
0.017
0.015
0.010
AGEN FT
0
2.114
5.124
0.906
6.4
0.013
0.169
0.334
0.126
0.359
0.996
0.014
0.010
0.007
w/ RTB data
MolDQN
0.107
0.900
0.002
0.001
0.761
0.892
1.849
1.393
0.001
0.000
0.000
Reinvant
Frag GFlowNet
0.174
1.213
0.001
0.4
0.005
0.001
0.192
0.135
0.320
1.272
0.002
0.002
0.002
High TPSA
Graph MCTS
0
0.197
0.353
0.001
0.005
0.001
0.652
0.171
0.862
1.692
0.002
0.002
0.001
AGEN Tasktrained
0.001
0.000
0,900
0.0
0.000
2.904
1.355
1.755
2.301
0.000
0.000
AGEN Finetuned
0.029
0.054
0.001
0.0
0.003
0.366
1.473
0.564
4.097
4.453
0.001
0.001
0.001
AGEN FT
0
1.159
3.931
0.005
4.4
0.017
0.548
0.194
0.204
0.723
1.498
0.013
0.009
0.006
w/ data.
RTB
AGEN FT
0
2,208
4.397
0.004
1.1
0.011
0.720
0.482
0.377
0.478
1.412
0.010
0.007
0.005
RTB data
Table 19, Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property-constrained
optimization objective for Microsomal Clcarance task.
35

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.4.9. TASK: TOXICITY ACUTE TOXICITY
Chem. Space
MOO metries
Diversity
Reward
Coveraage
Li-dist
Method
Amodes
RW#C
RWID
Uniq
#Rings
SAS
Top-1
Top-10
Top-100
McIDQN
0
2.160
1.938
0.006
0.0
0,043
0.000
0.950
0.507
1.224
0.077
0.013
0.010
0.006
Reinvent
0.531
23.954
0.043
48.6
0.133
0.000
0.000)
0.149
0.061
0.061
0.059
Frag Gel
2,904
0.038
0.054
0.044
Preserved
Graph MCTS
0
5,236
11.424
0.029
4.6
0.000
0.000
0.478
0.045
0.145
0.050
0.039
0.038
0.004
AGEN Tasktrained
1.232
1,980
0.037
0.000
0.000
0.000
0.010
0.199
0.040
0.039
0.038
0.044
Finetuned
43.824
AGEN HT
23.681
50.932
0.052
0.0
0.124
0.041
0,409
0.216
0.492
0.929
0.089
0.081
0.000
w/ data
RTB
AGEN FT
143.416
288.288
0.319
0.0
0.485
0.014
0.472
0.166
0.511
0.946
0.493
0.415
0.364
MeIDON
0.003
Reiment
2.277
50.853
0.053
13.2
0.124
0.000
0.015
0.000
0.131
0.180
0.074
0.073
0.069
Frag GElowNet
4.180
26.448
0.033
3.7
0.120
0.001
0.262
0.152
0.359
0.147
0.052
0.050
0.038
Law TPSA
Graph
AGEN Tasktruined
0.004
0.004
0.000
0.004
0.000
1.500
0.036
2.383
1,102
0.004
0.004
0.004
AGEN Finetuned
3.976
3,768
0.004
0.0
0.081
0.284
0.926
2.082
3.551
0.802
0.004
0.004
0.004
FT
22.827
51.504
0.057
0.168
0.374
0.175
0.517
0.928
0.159
0.064
w/ data
AGEN FT
12.989
392
0.322
0.0
0.500
0.084
0.046
0.021
0.904
1.092
0.498
0.471
0.419
RTB
106.260
349.480
0.336
0,516
0.061
0.336
1,000
0.385
w/ RTB data
MoiDQN
1.680
1,490
0.005
0.0
0.031
0.000
1.067
0.430
1.113
0.104
0.010
0.008
0.005
Reinvent
0.864
19.170
0.041
79.8
0.162
0.000
0.000
0.000
0.159
0.051
0.056
0.056
0.054
Frag GFlow/et
3.403
30.832
0.036
34.3
0.121
0.246
0.000
0.169
0.081
0.051
0.041
High TPSA
Graph MCTS
0
5.148
11.451
0.028
4.1
0.117
0.000
0.492
0.031
0.155
0.052
0.039
0.037
0.033
AGEN Tasktrained
0.000
0.000
0.000
0.023
1.328
1.250
1.981
2.975
0.625
0.016
0.003
0.000
Finetuned
2.695
0.002
1.146
0.005
AGEN FT
16.212
45.394
0.050
0.0
0.118
0.367
0.408
0.172
0.475
0.883
0.092
0.076
0.058
w/ data
20.145
0.395
RTB
AGEN FT
11,9799
29.304
0.043
0.0
0.113
0.426
0.491
0.142
0.397
0.907
0.065
0.062
0.050
w/ RTB data
Table 20, Comparing the effectiveness of finetuned A-GFN over A-GFN trained from scratch and other baselines property.constraineed
optimization objective for Acute Toxicity task.
36

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.5. De novo Generation
Analogous contemporary works (Lee et al., 2023b;a), we generate novel molecules for the aforementioned five targets, optimizing for high docking score (DS), QED, and SAS. As in Saturn (Guo & Schwaller, 2024b), benchmarking is conducted using QuickVini (Alhossary et al., 2015) for docking, and hit ratio and novel hit ratio are reported $E. Finetuning is performed with DRA property constrained optimization with # trings modified to [1, 5), and dp =
We consider 2 settings for the A-GFN prior - (i) pretrained with solely an offiin data, (ii) pretrained with the proposed hybrid online-offline strategy (fig.6). The offline prior is trained on Enamine 6.75M dataset on 4 Nvidia A100-40GB for 18 days, while the hybrid prior is trained on the same setup for 12 days on Zinc250K dataset. With either prior, we observe that an exploitative A-GFN finetuned with high inverse temperature for sampling (high 3) matches or outperforms Saturn and other baselines for the hit ratio metric (Ta However, when structural novelty is enforced via Tanimoto similarity (T.S.) threshold of 0.4, the offline prior underperforms in novel hit ratio compared to the best Saturn model. This is expected as RTB training objective keeps the finetuned policy anchored to the pretrained policy, the generated molecules retain structural similarities to the training set, particularly when the pretrained model is derived exclusively from offline data. While a T.S. > 0.4 threshold is often used to assess novelty, we argue that it may not always align with practical drug discovery objectives. In many real-world applications, compounds with known scaffolds and well-characterized chemistry are preferable to highly novel structures. Thus, rigid novelty constraints might not always be desirable.
That said, if high structural novelty is essential for a project, finetuning the hybrid prior (offline + online) leads to broader exploration of the chemical space, yielding a significantly higher novel hit ratio while still maintaining strong hit rates, and comprchensively outperforming all baseline methods (table 6 and 22). This again reinforces that incorporating both offline expertise and online experience during pretraining enables the model to generate more diverse yet biologically relevant compounds.
For protein targets, we finetune the pretrained A- A-GFN model with RTB objective in purely online fashion. Benchmarking results are taken from Saturn (Guo & Schwaller, 2024b). A-GFN is run across 3 seeds. The variability in QuickVina's docking results is accounted by considering the best docking score for each protein-ligamt pair across three independent docking runs. The offline prior used is trained on Enamine 6.75M dataset on 4 Nvidia A 1100-40GB for 18 days, while the hybrid prior is trained on the same setup for 12 days on Zinc250K dataset.
Method
parp1
fa7
5htlb
braf
jak2
Datasets
ZINC 250k (Sterling & Trwin, 2015)
3.993 + 0.355
1.097 + 0.1
24.260 +: 0.622
1.020 + 0.193
1.133 + 0.344
ChEMBL 33 (Gaulton et al., 2012)
6.077 = 0.453
1.830 = 0.240
24.163 + 0.715
2.073 + 0.181
9.013 = 0.562
Generative Models REINVENT (Olivecrona et al., 2017))
4.693 1: 1.776
1.967 1 0.661
26.047 +: 2.497
2,207 1 0.800
5.667 1 1.067
JT-VAE (Jin et al., 2018)
3.200 + 0.348
0.933 + 0.152
18.044 + 0.747
0.644 + 0.157
5.856 0.204
GraphAF (Shi al., 2020a)
0.822 0.113
0.011 + 0.016
6.978 + 0.952
1.422 + 0.556
1.233 0.284
MORLD (Jeon Kim, 2020)
0,047 0.050
0,U07 0.013
0.893 0.738
0,047 0.040
0,227 = 0.118
HierVAR (Tin et al., 2020a)
1.180 0.182
0.033 + 0.030
0.740 + 0.371
0.367 0.187
0.487 0.183
GraphDF (Tuo et al., 2021)
0.044 0.031
0.000 0.000
0.000 + 0.000
0.016
0.011 0.016
FREED (Yang ct al., 2021)
4.860 + 1.415
1
14.227 + 5.116
2.707 + 0.721
6.067 = 0.790
FREED-QS (Yang et al., 2021)
5.960 1 0.902
L687 1 0.177
23.140 1 2.422
3.880 1 0.623
7.653 1.373
LIMO (Eckmann et al., 2022)
0.456 + 0.057
0.443 0.016
1.200 = 0.178
0.278 0.134
0.711 0.329
GDSS (Jo et al., 2022)
2.367 0.316
0.467 0.112
6.267 0.287
0.300 0.198
1.367 0.258
MOOD (Lee et al., 2023a)
7.260 0.764
0.787 1 0.122
21.427 1 0.502
0.784
10.367 1 0.616
Aug. Mem. (Guo & Schwaller, 2024a)
16.966 + 3.224
2.637 + 0.860
52.016 2.302
8.307 + 1.714
1.714
21.548 + 4.938
GEAM (Lec et al., 2023b)
45.158 + 2.408
20.552 + 2.357
47.664 + 1.198
30.444 + 1.610
46. 19 + 2.073
Saturn (Guo& & Schwaller, 2024b)
57.981 + 18.537
14.527 + 9.961
68,185 + 3,400
38.999 + 10.114
60.827 = 11.502
A-GEN Finetuning
8= = 96 (offline prior)
42,900 1: 2,300
20.167 1 3.200
59.200 2,500
35.334 1 3.100
63.434 1 2.700
8 = 96 w/ uniq. filter (offline prior)
72.507 + 3,400
34.181 + 2,800
98. + 3,200
60.468 + 2,900
85,566 + 2,500
96 (offline online prior)
45.800 + 2.500
9.200 + 4.000
73.400 + 2.200
13.500 = 3,300
73,400 + 2,400
96 w/ uniq. filter (offline online prior)
76.375 1 3.200
36.078 2.400
97.910 3,500
63.579 1 2.600
97.910 1 3.400
Table 21. Hit Ratio (%): Finetuning is done with RTB objective in all A-GFN experiments. Unique Filter refers sampling n unique trajectories (n2 = 3000).
37

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Method
parp1
5htlb
braf
jak2
REINVENT (Olivecrona al., 2017)
0,480 + 0.344
0.213 = 0.081
2.453 = 0.561
0.127 0.088
0.613 = 0.167
GCPN (You al., 2018)
0.056 0.016
0.444
0.444 0.150
0.033 0.027
0.256 0.087
JT-VAE (Jin et al., 2018)
0.856 0.211
0.289
4.656 1.406
0.144 0.068
0.815 = 0.044
GraphAF (Shi et al., 2020a)
0.689 1: 0.166
0.016
3.178 1: 0.393
0.956 0.319
0.767 = 0.098
GraphGAA (Jensen, 2019)
4.811 +: 1.661
0,422 1.193
7.011 +: 2.732
3.767 1.498
5.311 = 1.667
MORLD (Jeon & Kim, 2020)
0.047 0.050
0.007 0.013
0.880 0.735
0.047 0.040
0.227 = 0.118
HierVAE (Jin al., 2020a)
0.353 0.214
0.013
0.507
0.207 0.220
0.227 = 0.127
RationaleRL (Jin et al., 2020b)
4.267 0.450
0.096
2.967 + 0.307
0.000
2.967 = 0.196
GA+D (Nigam et al., 2019)
0,044 +: 0.042
0.016
1,544 0.273
0.800
0.756 = 0.204
MARS (Xie et al., a)
1.178 + 0.299
0.072
6.833 7.706
0.478 +
2.178 0.545
GEGL.
2020)
0.789 0.150
0.256 0.083
3.167 0.260
0.244
0.933 0.072
GraphDF (Luo et al,, 2021)
0.044 0.031
+.000
0.000 0.000
0.011
0.011 + 0.016
FREED (Yang al., 2021)
4.627 + 0.727
1.332 0.1113
16.767 + 0.897
2.940 0.3359
5.800 + 0.295
LIMO (Fckmann al., 2022)
0.455 +: 0.057
0.016
1.189 0.181
0.278 0.114
0.689 0.319
GDSS (Jo ct al. 2022)
1.933 + 0.208
0.100
4.667 0.306
0.167
1.167 0.281
PS-VAE (Kong et al., 2022)
1.644 0.389
0.418
12.622 1.437
0.367 0.047
4.178 = 0.933
MOOD (Loc ct al., 2023a)
7.017 + 0.428
0.143
18.673 + 0.423
0.285
9.200 = 0.521
GEAM (Lee et al., 2023b)
39.159 + 2.790
19.540
40.123 + 1.611
27.467 1.374
41.765 3.412
Saturn (Guo & Schwaller, 2024b)
3.839 3.316
0.272
5.731 6.166
3.652 3.777
6.129 5.449
Satum-Tanimont (Guo Schwaller, 2024b)
50.552 + 9.530
20.181 + 5.598
54.260 6.722
19.820 + 10.120
47.785 = 14.01
A-GFN Finetuning
8 96 (offline prior)
18.400 2.300
8.233 3.200
25.834 2.500
14,400 3.100
36.266 2.700
B 96 w/ uniq. filter (oftline prior)
31.098 + 3.400
13.955 2.800
42.841
3.200
24.640 2,900
48.921 2.500
B 96 (offline + online prior)
41.734 + 2,500
6.934 + 4.000
55.267 + 2,200
12.134 = 3.300
55.267 = 2.400
B 96 w/ uniy. filter (offline online prior)
69.594 + 3.400
27.190 + 2.800
73.721 + 3.200
57.143 = 2.900
73.721 2.500
Table 22. Novel Hit Ratio (%)
38

---

Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
F.6. Lead Optimization
DS: -13
DS: -13
DS: -12
LE:
LE:
LE:
5ht1b
DS:-124 -12.4 LE 0.36
DS: -13.7
DS: -12,4 0.37
LE: 0.41
braf
DS: -10.2 1E:0.3
DS: -11.2
DS: -1..1 LE: 0 32
LE: 0.30 30
fa7
D8: -13.1 1
DS: -13.44
DS. -13.2 2
LE: 0.4
LE: 0.41
LE: 0.42
jak2
DS: 7 LE: 0.36
DS. -14.5 LE: 0.37
DS: -13.6 LE: 0.36
parp1
Figure I. Dacking comparison of the lead compound and the two best lead-optimized compounds their respective targets five targets. First col. shows the chosen lead compound and col. are optimized for docking score and ligand efficiency.
39


Reviews:
[{"review_id": "RMP3KYpFST", "reviewer": "Reviewer_tSwN", "summary": "This paper introduces Atomic GFlowNets (or A-GFNs), an atom-based generative framework for molecular design based on GFlowNets, proposing a more general-purpose exploration of the chemical space. The authors propose pre-training A-GFNs on inexpensive molecular properties that act as rewards for training the underlying GFlowNet policy, and then fine-tune for use on a variety of downstream tasks. The authors show the effectiveness of the proposed method on a variety of drug design tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Why does TB sometimes outperform RTB in single-objective tasks (Table 3)? Is it due to over-regularization in RTB?\n\nThank you for raising this important question. The observed performance difference stems from fundamental differences in how TB and RTB balance optimization objectives:\nYes, RTB's design introduces over-regularization in single-objective tasks due to its explicit anchoring to the pretrained prior. While this prior (e.g., a chemical validity model) helps maintain desirable properties in multi-objective settings, it creates conflicting constraints when optimizing for a single objective (which does not capture these desirable properties). We made initial efforts to address this trade-off in our current draft (lines 297-303), where we discuss how RTB’s prior anchoring leads to its weaker performance when compared to TB. We would like to point out that this effect would diminish when the prior already overlaps with high-reward solutions, and/or if tasks require balancing multiple constraints (e.g., both chemistry rules and potency). We appreciate this opportunity to clarify and will emphasize this trade-off more explicitly in our revision.\n\n2. How sensitive is the method to the choice of inexpensive reward descriptors?\n\nThis is an excellent question, and we appreciate the reviewer's interest in understanding the sensitivity of our method to the choice of inexpensive reward descriptors. The selection of pretraining properties plays a crucial role in shaping the learned policy, and our choices were guided by well-established principles in drug discovery. Specifically, we selected QED (quantitative estimate of drug-likeness), SAS (synthetic accessibility score), TPSA (topological polar surface area), and the number of rings, as these properties are widely used in molecular generation and optimization tasks due to their strong correlation with drug-likeness and pharmacokinetic properties [1].\n\nEmpirically, we found that these four descriptors provided a well-balanced pretraining signal that improved sampling efficiency while preserving chemical diversity. Importantly, we observed that adding additional constraints led to premature convergence and mode collapse, as the optimization problem became overly restrictive, significantly reducing the number of valid and diverse molecules that could be generated. This aligns with findings in other large-scale generative modeling approaches, where excessive constraints during pretraining can limit the exploration capacity of the model and degrade its generalization ability in downstream fine-tuning.\n\n3. How would a fragment-structured GFN compare to A-GFN?\n\nWe thank the reviewer for this question. The finetuned A-GFN generally outperforms fragment based GFN across all tasks considered in terms of diversity of molecules as well as the success percentage, which quantifies the ratio of molecules simultaneously satisfying all the enforced objectives. We have detailed these results in Table 12- 20 in Appendix F. \n\n[1] Wellnitz, James, et al. \"STOPLIGHT: a hit scoring calculator.\" Journal of Chemical Information and Modeling 64.11 (2024): 4387-4391.", "claims_and_evidence": "Claims are well-supported by clear and convincing experiments.", "methods_and_evaluation_criteria": "The atom-based action space makes sense for exploring novel scaffolds, albeit potentially increasing state space complexity. The hybrid online-offline approach and RTB-finetuning are well-motivated and empirically validated in experiments. The use of inexpensive rewards as proxies for general properties is pragmatic and promising for scaling. The evaluation and baselines are comprehensive and properly demonstrate the effectiveness of the proposed method.", "theoretical_claims": "None to be discussed.", "experimental_designs_or_analyses": "Yes, the experiment design is sound and extensive overall.", "relation_to_broader_scientific_literature": "The main contributions are very relevant for designing general-purpose molecular generation models that can be fine-tuned for a variety of useful downstream tasks.", "essential_references_not_discussed": "None to the best of my knowledge.", "other_strengths_and_weaknesses": "None to be particularly mentioned here.", "other_comments_or_suggestions": "None to be particularly mentioned here.", "questions_for_authors": "- Why does TB sometimes outperform RTB in single-objective tasks (Table 3)? Is it due to over-regularization in RTB? \n- How sensitive is the method to the choice of inexpensive reward descriptors? \n- How would a fragment-structured GFN compare to A-GFN?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NKGrIi62HC", "reviewer": "Reviewer_sFAt", "summary": "This paper proposes a training strategy to improve GFlowNet-based molecular generation. First, it uses atom-based policy rather than fragment-based policy to enable access to a larger chemical space. Second, this work proposes using expert trajectories constructed from ZINC to pretrain the network, which improves drug-likeness and sampling efficiency. Third, this work also proposes pre-training the policy network using inexpensive rewards (including QED, NumRings, TPSA, SAS, which can be computed instantly with RDKit). Such pretraining improves the sampling performance for more complex oracle scores.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful and constructive review of our paper. We are pleased to see that you find our work well-motivated, comprehensive in evaluation, and a notable contribution to the application of GFlowNets in molecular design. Additionally, we are grateful for your recognition that our claims are well-supported by experimental evidence, with no significant concerns raised regarding our methodology, theoretical foundation, or experimental design.  Based on the feedback from other reviewers, we have further improved our paper to enhance clarity, strengthen key arguments,\n\nGiven that no major weaknesses have been identified in your review and that our contributions are acknowledged as valuable to the field, we kindly ask you to reconsider and raise your current score. Your support would help ensure that this contribution reaches the broader community and advances research at the intersection of GFlowNets and molecular generation.", "claims_and_evidence": "- The first claim of this work is the atom-based model enables exploration of larger chemical space. Table 2 supports this claim, with more modes discovered by A-GFN and higher diversity compared to fragment-based models.\n- The second claim of this work is the unsupervised pretraining strategy \"*enables broader exploration of chemical space while maintaining diversity and novelty relative to existing drug-like molecule datasets*\". This claim has been supported by results presented in Table 3, Table 4, etc, which shows that pretraining the model using trajectories derived from ZINC and inexpensive property conditioning can improve sampling.\n- The last claim is the fine-tuning strategy and its effectiveness in multiple applications, which has also been well-supported by experimental results presented in Section 6.4 and Section 7.\n- Overall, the main claims of this work have been well-supported by experimental evidence.", "methods_and_evaluation_criteria": "- The proposed method is well-motivated by insights into molecular generation and it is a notable contribution to the area of GFlowNet's application in molecular design. Notably, it makes sense that \"*these rewards are computationally cheap to evaluate and serve as proxies for more complex properties*\", as complex properties have the foundation of physico-chemical properties which are usually inexpensive to compute.\n- The overall framework is well-motivated, but the components are mostly based on existing techniques. \n- The evaluation is comprehensive and supports the main claims of this work. Justification of each component and demonstration of application are both provided.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "No remarks.", "relation_to_broader_scientific_literature": "This work is a contribution to both GFlowNet application and molecular generation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AOyyj5S5GL", "reviewer": "Reviewer_uZjw", "summary": "The paper introduces Atomic GFlowNets (A-GFN), a novel generative model for molecular graph generation that leverages individual atoms as building blocks to explore drug-like chemical spaces more comprehensively. It adopts a pretraining mechanism using ZINC dataset, where A-GFN learns from inexpensive yet informative molecular descriptors such as drug-likeness, and presents a goal-conditioned finetuning process to adapt A-GFN for downstream optimization tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive feedback. Our response and proposed revisions for the concerns raised by the reviewer are as follows  \n# 1\nWe will ensure that figure sizes remain consistent throughout the appendix, particularly improving the font readability of Figure 3. Thank you for pointing this out. We have separated the molecules into separate high-resolution images for better legibility:  https://imgur.com/4pAWr4Q; https://imgur.com/a/jGxFnhY; https://imgur.com/a/uroAmCe; https://imgur.com/a/0biixYL. We’ll update the figure in the revised version of the paper. These molecules were sampled from the same pretrained model as before.  \n# 2 \nOur approach does not involve post-hoc RDKit filtering or penalization. Instead, A-GFN ensures atomic valency correctness at every generation step, preventing invalid states.\nThis is crucial for atom-based GFlowNets, which operate in a vast combinatorial space, unlike fragment-based GFN, which is limited to predefined fragments. Without valency constraints, the model would frequently generate infeasible structures, making exploration intractable.\nMolecular validity is just one of many evaluation criteria. The key metric #Modes measures how many diverse molecules satisfy all four pretraining conditionals (TPSA, Num Rings, QED, SAS). A-GFN significantly outperforms fragment-based GFN in this regard, demonstrating superior exploration—not merely benefiting from valency constraints.\nThus, valency enforcement is an inherent design choice, not a post-hoc filtering step, and does not confer an artificial advantage.\n# 3\nAll baselines (Fragment-GFN, REINVENT, MolDQN, GraphMCTS) were fine-tuned under identical compute constraints: 24 hours on a single V100 GPU, ensuring a fair comparison.\nA-GFN pretraining lasted \\~12 days on 4 A100 GPUs (250K steps), but a 100K-step checkpoint (\\~5 days) was sufficient for fine-tuning, as reward plateaued (see https://imgur.com/a/hLITsk5 ; to be included in the revised paper).\nWhile pretraining incurs a one-time cost, it benefits multiple downstream tasks. Once pretrained, A-GFN fine-tuning takes just 24 hours on a V100, making it comparable to molecular optimization baselines. This aligns with established pretraining and finetuning practices seen in ChemBERTa and MoLFormer.\n\n# 4.1 \nWe would like to clarify that our primary contribution is not that atomic-level design enhances pretraining, but rather the other way, i.e., pretraining is essential for A-GFN to function effectively. Without pretraining, A-GFN fails to generate viable molecules. Our experiments confirm that training A-GFN from scratch (Task train AGFN) consistently fails to satisfy task constraints.\n# 4.2 \nUsing ZINC for pretraining could introduce overlap with evaluation tasks like logP optimization, but it is common in pretraining large models  to have some solutions for the downstream task to be contained in large scale pretraining data as a broadly representative pretraining data distribution is desired.  Zero-shot OOD molecule generation is non-trivial, even for experts.\nTo ensure benchmarking remains challenging, we chose property thresholds so that <25% of ZINC molecules meet the optimization criteria. Additionally, our Novelty metric measures unique molecules not present in ZINC, confirming that solutions stem from exploration rather than memorization.\n# 4.3\nSome baselines (Fragment-GFN, REINVENT) were pretrained using their respective methods, while others (MolDQN, GraphMCTS) were trained from scratch, per the PMO benchmark (https://github.com/wenhao-gao/mol_opt/tree/main) . We used publicly available pretrained models where applicable, ensuring fair comparisons. A-GFN’s superior performance is due to better exploration, not an unfair pretraining advantage.\n\n# 5\nWe acknowledge that our paper does not include detailed descriptions of the baseline implementations, and we appreciate you bringing this to our attention. All baseline methods used in our work were directly adopted from the publicly available GitHub repository: https://github.com/wenhao-gao/mol_opt/tree/main. This repository is associated with the preprint https://arxiv.org/pdf/2206.12411, which includes some relevant implementation details, such as whether a method involves pretraining (check Table 7). However, specific aspects such as pretraining compute or further fine-tuning configurations are not comprehensively documented there either. We will add these details in the revised version of the paper for better transparency and reproducibility.", "claims_and_evidence": "The authors provide detailed experimental results that support the claim of proposed A-GFN framework in tasks including property optimization, property targeting, and property-constrained optimization.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are well-aligned with the molecular generation task.", "theoretical_claims": "The paper applies existing GFlowNet theory.", "experimental_designs_or_analyses": "The authors provide detailed descriptions of their experiments on the ZINC dataset pretraining and 3 downstream optimization tasks, comparing their approach to fragment-based GFlowNets, REINVENT, and GraphGA. The ablation studies and sensitivity analyses provide valuable insights into the impact of different design choices", "relation_to_broader_scientific_literature": "This paper extends fragment-based GFlowNets to atomic action spaces. Its pretraining aligns with unsupervised RL pretraining but adapts to molecular design via hybrid online-offline learning.", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Strengths:\n1. The paper introduces Atomic GFlowNets (A-GFN), a novel generative model that leverages individual atoms as building blocks, allowing for a more comprehensive exploration of drug-like chemical spaces compared to traditional fragment-based methods.\n2. The authors provide a thorough evaluation of A-GFN across various tasks, including property optimization, property targeting, and property-constrained optimization, showing superior performance compared to other methods.\n\nWeaknesses:\n1. Figures (e.g., Figure 3,5,6,7,8,9, 10) are blurry or inconsistently scaled, hindering interpretation of molecular structures and training dynamics.\n2. Post-hoc validity corrections (e.g., RDKit filtering) during finetuning muddy the contribution of atomic-level modeling. This undermines claims about the standalone effectiveness of the pretrained policy. \nFor instance, the fragment-based GFlowNet baseline might not explicitly enforce validity via RDKit during generation, while A-GFN inherently guarantees validity by design. This creates an unfair advantage for A-GFN.\n3. The paper does not provide computational costs comparisons between A-GFN and other methods (fragment-based GFlowNets, REINVENT), preventing a clear assessment of performance-to-resource tradeoffs. Given the substantial pretraining investment (4 A100 GPUs for 12-18 days), explicit efficiency comparisons would help understand if the atomic-level modeling advantages justify these increased computational demands.\n4. The paper constructs an extensive framework that somewhat obscures its core contributions. The structure needs to better emphasize how the atomic-level design specifically enhances pretraining capabilities, rather than focusing excessively on comparisons with non-pretrained methods. The introduction of more data through pretraining potentially creates unfair comparisons (For instance, pretraining data from ZINC may partially overlap with evaluation tasks like logP optimization), leading to somewhat trivial conclusions with non-pretraining methods. Additionally, the pretraining comparisons are affected by the design concerns mentioned in 2. regarding validity enforcement. Based on these observations, I believe this work requires further revision to better highlight and clarify its actual contributions.\n5. The paper seems not provide detailed descriptions of the implementation of the baseline methods. The baselines, such as fragment-based GFlowNets, MolDQN, Reinvent, and others, are mentioned in the context of comparison, but the paper does not delve into the specifics of how these baseline methods were implemented or fine-tuned. This makes the experimental results difficult to interpret, as differences in implementation details could significantly impact performance comparisons.", "other_comments_or_suggestions": "Please refer to weakness", "questions_for_authors": "Please refer to weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proppose  Atomic GFlowNets (A-GFNs), a foundational generative model leveraging individual atoms as building blocks to explore drug-like chemical space.  The authors pre-train A-GFN on inexpensive molecular properties as rewards, and then further fine-tune on  downstream tasks. The authors show the effectiveness of the proposed method on a variety of drug design tasks.

Strengths of the paper: 
- The proposed method is well-motivated by insights into molecular generation. 
- The rewards are computationally efficient. 
- The paper provides a thorough evaluation of A-GFN across various tasks, including property optimization, property targeting, and property-constrained optimization, showing superior performance compared to other methods.

Weaknesses of the paper: 
- The overall framework's components are based on existing techniques. 
- Post-hoc validity corrections (e.g., RDKit filtering) during finetuning undermine the claimed effectiveness of the pretrained policy.
Decision: Accept (poster)