Title: RGFN: Synthesizable Molecular Generation Using GFlowNets
Venue: NeurIPS 2024
Accepted: yes
Keywords: drug discovery

generative models

GFlowNets

synthesizability
Abstract: Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.

RGFN: Synthesizable Molecular Generation Using GFlowNets

Abstract

Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.

1 Introduction

Traditionally, machine learning has been applied to drug discovery for screening existing libraries of compounds, whether actual physical collections or pre-configured in silico collections of readily synthesizable compounds, in a supervised fashion. However, supervised screening of the whole druglike space, often estimated to contain approximately 10 60 [41] different compounds, is infeasible in practice. Generative methods offer the potential to circumvent this issue by sampling directly from a distribution over desirable chemical properties without the need to evaluate every possible molecular structure. Despite these advances, existing generative approaches tend not to explicitly enforce synthesizability [21], generating samples that might be either very costly or altogether impossible to chemically synthesize. Ensuring that generative methods operate in the space of synthesizable compounds, yet at a much larger and more diverse scale than existing chemical libraries, remains an open challenge.

In this paper, we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework [5] that generates molecules by combining basic chemical fragments using a chain of reactions. We propose a relatively small collection of cheap and accessible chemical building blocks (reactants), as well as established high-yield chemical transformations, that together can still produce a search space orders of magnitude larger than existing chemical libraries. We additionally propose several domain-specific extensions of the GFlowNet framework for action representation and scaling to a larger space of possible actions.

Source code available at https://github.com/koziarskilab/RGFN .

We experimentally evaluate RGFN on a set of diverse screening tasks, including docking score approximation with a trained proxy model for soluble epoxide hydrolase (sEH), GPU-accelerated direct docking score calculations for multiple protein targets (Mpro, ClpP, TBLR1 and sEH), and biological activity estimation with a trained proxy model for dopamine receptor type 2 (DRD2) receptor activity and senolytic activity [74]. We demonstrate that RGFN produces similar optimization quality and diversity to existing fragment-based approaches while ensuring straightforward synthetic routes for predicted hit compounds.

2 Related work

Generative models for molecular discovery. A plethora of methods have been developed for molecular generation [48, 7] using machine learning. These methods can be categorized depending on the molecular representation used, including textual representations such as SMILES [34, 3, 39], molecular graphs [33, 46, 54] or 3D atom coordinate representations [52], as well as the underlying methodology, for example, variational autoencoders [33, 46], reinforcement learning [54, 37] or diffusion models [60]. Recently, Generative Flow Networks (GFlowNets) [4, 50, 59, 64, 72, 19] have emerged as a promising paradigm for molecular generation due to their ability to sample large and diverse candidate small molecule space, which is crucial in the drug discovery process. Traditionally, GFlowNets for molecular generation operated on the graph representation level, and candidate molecules were generated as a sequence of actions in which either individual atoms or small molecular fragments were combined to form a final molecule. While using graph representations, as opposed to textual or 3D representations, allows the enforcement of the validity of the generated molecules, it does not guarantee a straightforward route for chemical synthesis. Here, we expand on the GFlowNet framework by modifying the space of actions to consist of choosing molecular fragments and executing compatible chemical reactions/transformations, in turn guaranteeing both physical-chemical validity and synthesizability.

Synthesizability in generative models. One approach to ensuring the synthesizability of generated molecules is by using a scoring function, either utilizing it as one of the optimization criteria [37], or as a post-processing step for filtering generated molecules. Multiple scoring approaches, both heuristic [18, 23] and ML-based [42], exist in the literature. Another branch of research focuses on using reaction models and traversing predicted synthesis graph [8, 12, 38, 55]. However, reaction and synthesizability estimation is difficult in practice, and can fail to generalize out-of-distribution in the case of ML models. Furthermore, theoretical synthesizability does not necessarily account for the cost of synthesis. Because of this, a preferable approach might be to constrain the space of possible molecules to those easily synthesized by operating in a predefined space of chemical reactions and fragments. Several recent strategies employ this approach [22, 49, 67], including reinforcement learning-based methods [24, 27] and a concurrent work utilizing GFlowNets [15]. We extend this line of investigation not only by translating the concept to the GFlowNet framework but also by proposing a curated set of robust chemical reactions and fragments that ensure efficient synthesis at low total costs.

3 Method

3.1 Generative Flow Networks

GFlowNets are amortized variational inference algorithms that are trained to sample from an unnormalized target distribution over compositional objects. GFlowNets aim to sample objects from a set of terminal states X proportionally to a reward function R : X → R + . GFlowNets are defined on a pointed directed acyclic graph ( DAG ), G = ( S, A ) , where:

s ∈ S are the nodes, referred to as states in our setting, with the special starting state s 0 being the only state with no incoming edges, and the terminal states X have no outgoing edges,

a = s → s ′ ∈ A are the edges, referred to as actions in our setting, and correspond to applying an action while in a state s and landing in state s ′ .

We can define a non-negative flow function on the edges F ( s → s ′ ) and on the states F ( s ) of the DAG such that ∀ x ∈ X F ( x ) = R ( x ) . A perfectly trained GFlowNet should satisfy the following flow-matching constraint:

A state sequence τ = ( s 0 → s 1 → . . . → s n = x ) , with s n = x ∈ X and a i = ( s i → s i +1 ) ∈ A for all i , is called a complete trajectory. We denote the set of trajectories as T .

Another way to rephrase the flow-matching constraints is to learn a forward policy P F ( s i +1 | s i ) such that trajectories starting at s 0 and taking actions sampled by P F terminate at x ∈ X proportional to the reward.

Trajectory balance. Several training losses have been explored to train GFlowNets. Among these, trajectory balance [45] has been shown to improve credit assignment. In addition to learning a forward policy P F , we also learn a backward policy P B and a scalar Z θ , such that, for every trajectory τ = ( s 0 → s 1 → . . . → s n = x ) , they satisfy:

3.2 Reaction-GFlowNet

Reaction-GFlowNet generates molecules by combining basic chemical fragments using a chain of reactions. The generation process comprises the following steps (illustrated in Figure 1):

Select an initial building block (reactant or surrogate reactant; see Appendix C.2 for more details about surrogate reactants).

Select the reaction template (a graph transformation describing the reaction).

Select another reactant.

Perform the in silico reaction and select one of the resulting molecules.

Repeat steps 2-4 until the stop action is selected.

In the rest of this section, we describe the design of the Reaction-GFlowNet in detail.

cross-

coupling

SNAr

Not

selected

Reward

Only one possible

product

Only one possible

product

SNAr

Amide

coupling

cross-

coupling

Reaction templates

Reagent database

Reaction

selection

Next state

or reward

Reactant

selection

Product

selection

Reaction

selection

Reactant

selection

[    ]

Initial BB

selection

[   ]

…

[   ]

…

[   ]

…

[   ]

…

…

Preliminaries. Reaction-GFlowNet uses a predefined set of reaction patterns and molecules introduced in Section 3.3. We denote these as R and M respectively. As a backbone for our forward policy P F , we use a graph transformer model f from [77]. The graph transformer takes as an input a molecular graph m and outputs the embedding f ( m ) ∈ R D , where D is the embedding dimension.

In particular, f can embed an empty graph ∅ . It can additionally be conditioned on the reaction r ∈ R which we denote as f ( m,r ) . The reaction in this context is represented as its index in the reaction set R .

Select an initial building block. At the beginning of each trajectory, Reaction-GFlowNet selects an initial fragment from the collection of building blocks M . The probability of choosing i -th fragment m i is equal to:

where MLP M : R D → R | M | is a multi-layer perceptron (MLP). The σ k is a standard softmax over the logits vector s ∈ R k of the length k :

Select the reaction template. The next step is to select a reaction that can be applied to the molecule m . The probability of choosing i -th reaction from R is described as:

where MLP R : R D → R | R | +1 is an MLP that outputs logits for reactions from R and an additional stop action with index | R | +1 . Choosing the stop action in this phase ends the generation process. Note that not all the reactions may be applied to the molecule m . We appropriately filter such reactions and assume that the score s i for non-feasible reactions is equal to -∞ .

Select another reactant. We want to find a molecule m i ∈ M that will react with m in the reaction r . The probability for selecting m i is defined as:

where MLP M is shared with the initial fragment selection phase. As in the previous phase, not all the fragments can be used with the reaction r , so we filter these out.

Perform the reaction and select one of the resulting molecules. In this step, we apply the reaction r to the two fragment molecules chosen in previous steps. As the reaction pattern can be matched to multiple parts of the molecules, the result of this operation is a set of possible outcomes M ′ . We choose the molecule m ′ i ∈ M ′ by sampling from the following distribution:

where MLP M ′ : R D → R scores the embedded m ′ i molecule.

Backward Policy. A backward policy in RGFN is only non-deterministic in states corresponding to a molecule m which is a result of performing some reaction r ∈ R on molecule m ′ and reactant m ′′ ∈ R . We denote the set of such tuples ( r, m ′ , m ′′ ) that may result in m as T . We override the indexing and let ( r i , m ′ i , m ′′ i ) be the i -th tuple from T . The probability of choosing the i -th tuple is:

where MLP B : R D → R and f is a backbone transformer model similar to the one used in the forward policy. To properly define T , we need to implicitly keep track of the number of reactions performed to obtain m (denoted as k ). Only those tuples ( r, m ′ , m ′′ ) are contained in the T for which we can recursively obtain m ′ in k -1 reactions.

Action Embedding. While the MLP M used to predict the probabilities of selecting a molecule m i ∈ M works well for our predefined M , it underperforms when the size of possible chemical building block library is increased. Such an MLP M likely struggles to reconstruct the relationship between the molecules. Intuitively, when a molecule m i is chosen in some trajectory, the training signal from the loss function should also influence the probability of choosing a structurally similar m j . However, the MLP M disregards the structural similarity by construction and it intertwines the probabilities of choosing m i and m j only with the softmax function. To incorporate the relationship between molecules into the model, we embed the molecular building blocks with a simple machine learning model g and reformulate the probability of choosing a particular building block m i :

where ϕ is some activation function (we use GELU) and W ∈ R D × D is a learnable linear layer. Note that if we define g ( m i ) as an index embedding function that simply returns a distinct embedding for every m i , we will obtain a formulation equivalent to Equation (5). To leverage the structure of molecules during the training, we use g that linearly embeds a MACSS fingerprint [40] of an input molecule m i along with the index i . Note that this approach does not add any additional computational costs during the inference as the embeddings g ( m i ) can be cached. In Section 4.3, we show that this method greatly improves the performance when scaling to larger sets of fragments.

3.3 Chemical language

We select seventeen reactions and 350 building blocks for our model. These include amide bond formation, nucleophilic aromatic substitution, Michael addition, isocyanate-based urea synthesis, sulfur fluoride exchange (SuFEx), sulfonyl chloride substitution, alkyne-azide and nitrile-azide cycloadditions, esterification reactions, urea synthesis using carbonyl surrogates, Suzuki-Miyaura, Buchwald-Hartwig, and Sonogashira cross-couplings, amide reduction, and peptide terminal thiourea cyclization reactions to produce iminohydantoins and tetrazoles. The chosen reactions are known to be typically quite robust [9] and are often high-yielding (75-100%), thus enforcing reliable synthesis pathways when sampling molecules from our model. To simulate couplings in Python, reactions are encoded as SMARTS templates. To ensure compatible building blocks yielding specific, chemically valid products and to enable parent state computation, we introduce multiple variants corresponding to differing reagent types for most of the proposed reactions. In some cases SMARTS templates encode reactions where one of the reagents is not specified. We describe these transformations as implicit reactions (Appendix E). Additionally, we introduce surrogate reactions where the SMARTS templates and SMILES strings encode for alternate building blocks (see Appendix C.2 for more details). Finally, once again for the sake of specificity, reactions are duplicated by swapping the order of the building block reactants. In total, 132 different SMARTS templates are used.

During the construction of the curated building block database, only affordable reagents (i.e., building blocks) are considered. For the purposes of this study we define affordable reagents to be those priced at less than or equal to $200 per gram. The mean cost per gram of reagents selected for this study is $22.52, the lowest cost $0.023 per gram, and the highest cost $190 per gram (see Appendix N for more details on cost estimation).

A crucial consideration when choosing the set of reactions and fragments used is the state space size (the number of possible molecules that can be generated using our framework). This is difficult to compute precisely since a different set of reactions or building blocks is valid for every state in a given trajectory. We estimate this based on 1,000 random trajectories instead (details can be found in Appendix A). In addition to our 350 low-cost fragments, we also perform this analysis with 8,000 additional random Enamine building blocks. Comparison for different numbers of maximum reactions is presented in Figure 2. We demonstrate that even with curated low-cost reactants and limiting the number to a maximum of four reactions, state space size is an order of magnitude greater than the number of molecules contained in Enamine REAL [17]. This size can increase significantly with the addition of more fragments and/or an increase in the maximum number of reactions. Additional discussions regarding scaling can be found in Section 4.3.

4 Experimental study

In the conducted experiments we compare oracle scores and synthesizability scores of RGFN with several state-of-the-art reference methods. Secondly, we examine the capabilities of RGFN to scale to larger fragment libraries, in particular when using the proposed action embedding mechanism. Finally, we perform an in-depth examination of generated ligands across several biologically relevant targets.

4.1 Set-up

Throughout the course of the conducted computational experimental study, we aim to evaluate the performance of the proposed approach across several diverse biological oracles of interest. This includes proxy models (machine learning oracles, pretrained on the existing data and used for higher computational efficiency): first, the commonly used sEH proxy as described in [4]. Second, a graph

neural network trained on the biological activity classification task of senolytic [74] recognition. Third, the Dopamine Receptor D2 (DRD2) oracle [53] from Therapeutics Data Commons [28]. Proxy model details are provided in Appendix B.1.

Per the GFlowNet training algorithm, the reward is calculated for a batch of dozens to hundreds of molecules at each training step, rendering traditional computational docking score algorithms like AutoDock Vina [70] infeasible for very large training runs. As a result, previous applications of GFlowNets to biological design [4, 64] employed a fast pre-trained proxy model trained on docking scores instead. These proxies, while lightweight, present potential issues should the GFlowNet generate molecules outside their training data distributions and require receptor-specific datasets. To circumvent this, we use the GPU-accelerated Vina-GPU 2.1 [68] implementation of the QuickVina 2 [2] docking algorithm to calculate docking scores directly in the training loop of RGFN. This approach allows for drastically increased flexibility in protein target selection while eliminating proxy generalization failure. We selected X-ray crystal structures of human soluble epoxy hydrolase (sEH), ATP-dependent Clp protease proteolytic subunit (ClpP), SARS-CoV-2 main protease (Mpro), and transducin β -like-related protein 1 as targets for

0

1

2

3

4

5

Number of reactions used

10 4

10 7

10 10

10 13

10 16

10 19

10 22

Estimated state space size

Enamine REAL

RGFN (350)

RGFN (8350)

evaluating RGFN using a docking reward (detailed motivation for specific target selection is provided in Appendix G).

4.2 Comparison with existing methods

We begin experimental evaluation with a comparison to several state-of-the-art methods for molecular discovery. Specifically, we consider a genetic algorithm operating on molecular graphs (GraphGA) [32] as implemented in [10], which has been demonstrated to be a very strong baseline for molecular discovery [21], Monte Carlo tree search-based SyntheMol [67], cascade variational autoencoder (casVAE) [49], and a fragment-based GFlowNet (FGFN) [4] as implemented in [57]. For FGFN, we additionally considered its variant that had a SAScore as one of the reward terms (FGFN+SA). Training details can be found in Appendix B.2. It is worth noting that besides SyntheMol, which also operates in the space of chemical reactions and building blocks derived from the Enamine database, and casVAE, which used the set of reaction trees obtained from the USPTO database [43], our remaining benchmarks do not explicitly enforce synthesizability when generating molecules. Because of this, in this section, we will examine not only the quality of generated molecules in terms of optimized properties but also their synthesizability. We consider only two reaction-based approaches, as other existing methods employing this paradigm [27, 24] do not share code or curated reactions and building blocks, making reproduction difficult.

We first examine the distributions of rewards found by each method across four different oracles used for training: sEH proxy, senolytic proxy, DRD2 proxy, and GPU-accelerated docking for ClpP. The results are presented in Figure 3. As can be seen, while RGFN underperforms in terms of average reward when compared to the method not enforcing synthesizability (GraphGA), it outperforms SyntheMol's and casVAE's reaction-based sampling. Interestingly, when compared to standard FGFN, RGFN either performs similarly (ClpP docking) or achieves higher average rewards. This is most striking in the case of the challenging senolytic discovery task, in which a proxy is trained on a severely imbalanced dataset with less than 100 actives, resulting in a sparse reward function. We suspect that this, possibly combined with a lack of compatibility between the FGFN fragments and known senolytics, led to the failure to discover any high-reward molecules. However, RGFN succeeds in the task and finds a wide range of senolytic candidates. Finally, the gap in performance is

even larger between RGFN and FGFN+SA, indicating that introducing synthesizability constraints reduces the ability of FGFN to discover high-reward molecules.

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

Method

0

2

4

6

8

Reward

(a) sEH (proxy)

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

Method

0

20

40

60

80

100

Reward

(b) senolytics (proxy)

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

Method

0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Reward

(c) DRD2 (proxy)

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

Method

0.0

2.5

5.0

7.5

10.0

12.5

15.0

17.5

Reward

(d) ClpP (docking)

Secondly, we examine the number of discovered modes for each method, with a mode defined as a molecule with computed reward above a threshold (sEH: 7, senolytics: 50, DRD2: 0.95, ClpP docking: 10), and Tanimoto similarity to every other mode < 0 . 5 . We use Leader algorithm for mode computation. The number of discovered modes across tasks as a function of normalized iterations is presented in Figure 4. Note that in the case of GraphGA, FGFN, FGFN+SA, and RGFN this simply translates to the number of oracle calls, but for SyntheMol and casV AE, due to large computational overhead, we impose a maximum number of oracle calls such that training time was comparable to RGFN (see Appendix B.2 for details). Note that in the case of casVAE, this resulted in a very small number of molecules being visited in the allotted time. As can be seen, despite slightly worse average rewards, FGFN still outperforms other methods in terms of the number of discovered modes (with the exception of senolytic discovery task, where it fails to discover any high-reward molecules). This includes FGFN+SA, despite its generally worse performance than FGFN. This suggests that RGFN samples are less diverse, possibly due to the relatively small number of fragments and reactions used. However, RGFN still outperforms remaining methods across all tasks, suggesting that it preserves some of the benefits of the diversity-focused GFlowNet framework.

100000

150000

200000

250000

300000

350000

400000

# normalized iterations

10 1

10 2

10 3

10 4

Discovered modes

Method

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

(a) sEH (proxy)

100000

150000

200000

250000

300000

350000

400000

# normalized iterations

10 1

10 2

10 3

Discovered modes

Method

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

(b) senolytics (proxy)

100000

150000

200000

250000

300000

350000

400000

# normalized iterations

10 2

10 3

10 4

Discovered modes

Method

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

(c) DRD2 (proxy)

100000

150000

200000

250000

300000

350000

400000

# normalized iterations

10 2

10 3

10 4

Discovered modes

Method

GraphGA

SyntheMol

casVAE

FGFN

FGFN (+SA)

RGFN

(d) ClpP (docking)

Finally, we evaluate the synthesizability of the generated compounds as a key output. We present average values of several synthesizability-related metrics, computed over top-k modes generated for each method, in Table 1. We include measures indicating average molecular weight and drug-likeness (QED) to gauge the size of generated compounds. Furthermore, for completeness, we also include SAScores [18], but note that they are only a rough approximation of ease of synthesis. For a better estimate of synthesizability we perform retrosynthesis using AiZynthFinder [23] and count the average number of molecules for which a valid retrosynthesis pathway was found. However, it is important to note that both SAScores and AiZynthFinder scores are inherently noisy metrics. While we evaluate all methods using them for the sake of rigorousness, ultimately molecules generated by RGFN (as well as SyntheMol and casVAE) are guaranteed to be highly likely synthesizable. Note that to reduce variance, we compute molecular weight, QED, and SAScores over the top-500 modes, but due to high computational cost, AiZynthFinder scores are computed only over top-100 modes. As can be seen, while there is some variance across tasks, RGFN performs similarly to SyntheMol and casVAE in terms of both synthesizability scores, and significantly outperforms GraphGA and FGFN. Crucially, including SAScore as a reward does improve the performance of FGFN in terms of that metric, but does not drastically change the AiZynthFinder scores, demonstrating that it is insufficient to guarantee synthesizability. All RGFN modes were additionally inspected manually by an expert chemist and confirmed as synthesizable, which indicates that AiZynth scores are likely underestimated.

4.3 Scaling to larger sets of fragments

Next we investigate the influence of a fragment embedding scheme proposed in Section 3.2. In the standard implementation of the GFlowNet policy, actions are represented as independent embeddings in the MLP. These encode actions as indices, effectively disregarding their respective structures and all information contained therein. The model must thus select from a library of reagents without any knowledge as to their chemical makeup or properties. While finding similarities between actions may be a relatively easy task for small action spaces, it becomes more difficult when the size of the action space increases. To scale RGFN to a larger size of the building block library, we proposed to encode building block selection actions using molecular fingerprints, allowing the model to leverage their internal structures without any additional computational overhead during inference. In Figure 5, we observe that our fingerprint embedding scheme allows for drastically faster convergence compared to the standard independent action embedding, especially for large library sizes. The details on how the larger fragment libraries were created can be found in Appendix F.

4.4 Examination of the produced ligands

In the final stage of experiments we examine the capabilities of RGFN to produce high quality ligands across multiple diverse docking targets (see Appendix G for more details). The aim is to evaluate whether 1) the chemical language used is expressive enough to produce structurally diverse molecules for different targets, and 2) whether the generated ligands form realistic poses in the binding pockets. We first demonstrate the diversity of ligands across targets on a UMAP plot of extended-connectivity fingerprints (Figure 6). Ligands assigned to specific targets form very distinct clusters, showcasing their diversity. Interestingly, we also observe structural differences between sEH proxy and sEH docking, possibly indicating poor approximation of docking scores by the proxy model. Secondly, we examine the docking poses of the highest scoring generated ligands (Figure 7). As can be seen, the generated molecules produce realistic docking poses, closely resembling the poses of known ligands (Appendix K), despite being diverse in terms of structural similarity (Appendix M). We further conduct a cost analysis and synthesis planning for top modes in Appendices N and O. Overall, this demonstrates the usefulness of the proposed RGFN approach in the docking-based screens.

1k

2k

4k

8k

16k

32k

64k

Number of chemical fragments in the library

70k

80k

90k

100k

110k

Number of discovered scaffolds

Number of discovered scaffolds with sEH (proxy) > 7

independent embedding

fingerprint embedding

(a) sEH (proxy)

>

7

1k

2k

4k

8k

16k

32k

64k

Number of chemical fragments in the library

3k

4k

6k

10k

Number of discovered scaffolds

Number of discovered scaffolds with sEH (proxy) > 8

independent embedding

fingerprint embedding

(b) sEH (proxy)

>

8

(a) sEH

(b) ClpP

(c) Mpro

5 Limitations

The current proof-of-principle implementation of RGFN uses only 17 reaction types and 350 building blocks. Although these limited inputs already generate a vast chemical space, this represents only a small fraction of possible druglike space, which in turn limits the quality and potency of the generated molecular structures. The scaling experiment demonstrates that the number of building blocks can readily be increased, and increasing the number and diversity of building blocks is a straightforward way to enhance and survey the accessible chemical space.

The current set of building blocks and reactions tends to generate linear and flat-shaped molecules. Adding a small set of cyclization reactions (such as peptide macrocyclization and ring-closing metathesis), along with more complex-shaped scaffold building blocks, as well as reactions that introduce sp 3 hybridized atoms and stereochemical complexity will therefore allow for greater shape diversity and the

y

5

0

5

10

15

x

5

0

5

10

15

Target

ClpP

Mpro

TBLR1

Senolytic

sEH (proxy)

sEH (docking)

generation of more potent molecules [51, 20]. It is also important to recognize that RGFN does not explicitly generate synthetic routes to the molecules, at least not in a strict chemical sense, which in addition to a sequence of reactions transforming sets of reactants into products (which RGFN does provide), would also include choice of reaction conditions, external reagents, catalysts, protection group strategies, etc.

Another significant limitation in the quality of the generated molecular structures is the reliance on molecular docking as a scoring oracle. Although molecular docking has been successfully used in large-scale virtual screening efforts [44, 61, 35], it has well-known shortcomings in its predictive power. First, docking scores correlate strongly with molecular weight (MW) [13] and do not account for drug-likeness requirements like optimal MW or ClogP. In this work, molecule size was constrained only by the number of reaction steps, encouraging RGFN to generate large molecules within the building block limit. This can be somewhat rectified by augmenting reward with a drug-likeness or ligand efficiency term. Second, its binding affinity predictions and rankings often correlate weakly with experimental values and are highly dependent on the nature of the target protein's binding site [73, 71]. This is further illustrated by the fact that known ligands are not necessarily characterized by highest possible docking scores (Appendix L). This limitation impacts the learning of the chemical structure-activity relationship space, leading to the generation of sub-optimal molecules. One solution to this limitation is to incorporate more accurate but computationally expensive methods (such as ensemble docking, MM-PBSA, and FEP) within a multi-fidelity framework [26]. However, since we focus on robust, affordable, and facile synthesis methods, we ultimately aim to extend our approach beyond computational scoring methods by directly conducting experimental evaluation of synthesized compound batches within an active learning loop.

6 Conclusions

In this paper, we present RGFN, an extension of the GFlowNet framework that operates in the action space of chemical reactions. We propose a curated set of high-yield chemical reactions and low-cost molecular building blocks that can be used with the method. We demonstrate that even with a small set of reactions and building blocks, the proposed approach produces a state space with a size orders of magnitude larger than typical experimental screening libraries while ensuring high synthesizability of the generated compounds. We also show that the size of the search space can be further increased by including additional building blocks and that the proposed action embedding mechanism improves scalability to very large building block spaces.

In the course of our experiments, we show that RGFN achieves roughly comparable average rewards to state-of-the-art methods, and it outperforms another approach operating directly in the space of chemical reactions and, crucially, standard fragment-based GFlowNets. At the same time, it significantly improves the synthesizability of generated compounds when compared to a fragmentbased GFlowNet. Analysis of ligands produced across the set of diverse tasks demonstrates sufficient diversity of proposed chemical space to generalize to various targets. While not yet demonstrated experimentally, ease of synthesis (due to the small stock of cheap fragments and high-yield chemical reactions used) combined with reasonably high optimization quality of bespoke ligands offer a promising alternative to standard high-throughput screening applications. In particular, it can be beneficial for active learning-based pipelines with significant wet lab component, reducing the reliance on inaccurate docking oracles. Facilitating the drug discovery process through the generation of novel small molecules can eventually lead to the discovery of novel medications leading to significant societal benefits.

Acknowledgments and Disclosure of Funding

This work was supported by funding from CQDM Fonds d'Accélération des Collaborations en Santé (FACS) / Acuité Québec and the National Research Council (NRC) Canada, the Canadian Institutes for Health Research (CIHR), grant no. FDN-167277, Samsung and Microsoft. D. Shevchuk is grateful for support from the Mitacs Globalink Graduate Fellowship, grant no. FR121160/FR121161. The research of P. Gai´ nski was supported by the National Science Centre (Poland), grant no. 2022/45/B/ST6/01117. Additional thanks for funding provided to the University of Toronto's Acceleration Consortium from the Canada First Research Excellence Fund, grant no. CFREF-2022-00042. Computational resources were provided by the Digital Research Alliance of

Canada ( https://alliancecan.ca ) and Mila ( https://mila.quebec ). We gratefully acknowledge Poland's high-performance Infrastructure PLGrid (ACK Cyfronet Athena, HPC) for providing computer facilities and support within computational grant no PLG/2023/016550.

References

Beatrice Alexander-Howden, Li Zhang, Almer M van der Sloot, Sylvain Tollis, Daniel J St-Cyr, Frank Sicheri, Adrian P Bird, Mike Tyers, and Matthew J Lyst. A screen for MeCP2-TBL1 interaction inhibitors using a luminescence-based assay. Sci. Rep. , 13(1):3868, March 2023.

Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with QuickVina 2. Bioinformatics , 31(13):2214-2216, 02 2015. ISSN 1367-4803. doi: 10.1093/bioinformatics/btv082. URL https://doi.org/10. 1093/bioinformatics/btv082 .

Josep Arús-Pous, Atanas Patronov, Esben Jannik Bjerrum, Christian Tyrchan, Jean-Louis Reymond, Hongming Chen, and Ola Engkvist. SMILES-based deep generative scaffold decorator for de-novo drug design. Journal of cheminformatics , 12:1-18, 2020.

Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems , 34:27381-27394, 2021.

Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J Hu, Mo Tiwari, and Emmanuel Bengio. GFlowNet foundations. Journal of Machine Learning Research , 24(210):1-55, 2023.

Vaibhav Bhandari, Keith S Wong, Jin Lin Zhou, Mark F Mabanglo, Robert A Batey, and Walid A Houry. The role of ClpP protease in bacterial pathogenesis and human diseases. ACS Chem. Biol. , 13(6):1413-1425, June 2018.

Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Generative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science , 12(5):e1608, 2022.

John Bradshaw, Brooks Paige, Matt J Kusner, Marwin Segler, and José Miguel HernándezLobato. A model to search for synthesizable molecules. Advances in Neural Information Processing Systems , 32, 2019.

Dean G. Brown and Jonas Boström. Analysis of past and present synthetic methodologies on medicinal chemistry: Where have all the new reactions gone? Miniperspective. Journal of Medicinal Chemistry , 59(10):4443-4458, 2016.

Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. GuacaMol: benchmarking models for de novo molecular design. Journal of chemical information and modeling , 59(3):1096-1108, 2019.

Martin Buttenschoen, Garrett M. Morris, and Charlotte M. Deane. PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences, 2023.

Alexander Button, Daniel Merk, Jan A Hiss, and Gisbert Schneider. Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis. Nature machine intelligence , 1(7):307-315, 2019.

Giorgio Carta, Andrew JS Knox, and David G Lloyd. Unbiasing scoring functions: a new normalization and rescoring strategy. Journal of chemical information and modeling , 47(4): 1564-1571, 2007.

COVID Moonshot Consortium, Hagit Achdout, Anthony Aimon, Dominic S Alonzi, Robert Arbon, Elad Bar-David, Haim Barr, Amir Ben-Shmuel, James Bennett, Vitaliy A Bilenko, et al. Open science discovery of potent non-covalent SARS-CoV-2 main protease inhibitors. BioRxiv , pages 2020-10, 2020.

Miruna Cretu, Charles Harris, Julien Roy, Emmanuel Bengio, and Pietro Lio. SynFlowNet: Towards molecule design with guaranteed synthesis pathways. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design , 2024.

Thomas Duflot, Clothilde Roche, Fabien Lamoureux, Dominique Guerrot, and Jeremy Bellien. Design and discovery of soluble epoxide hydrolase inhibitors for the treatment of cardiovascular diseases. Expert Opin. Drug Discov. , 9(3):229-243, March 2014.

Enamine. Enamine REAL database. http://enamine.net/compound-collections/ real-compounds/real-database , 2024. [Online; Accessed 22 May 2024].

Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics , 1:1-11, 2009.

Piotr Gai´ nski, Michał Koziarski, Krzysztof Maziarz, Marwin Segler, Jacek Tabor, and Marek ´ Smieja. RetroGFN: Diverse and feasible retrosynthesis using GFlowNets. arXiv preprint arXiv:2406.18739 , 2024.

Warren R.J.D. Galloway, Albert Isidro-Llobet, and David R. Spring. Diversity-oriented synthesis as a tool for the discovery of novel biologically active small molecules. Nature Communications , 1(1):80, Sep 2010. ISSN 2041-1723. doi: 10.1038/ncomms1081. URL https://doi.org/ 10.1038/ncomms1081 .

Wenhao Gao and Connor W Coley. The synthesizability of molecules proposed by generative models. Journal of chemical information and modeling , 60(12):5714-5723, 2020.

Wenhao Gao, Rocío Mercado, and Connor W Coley. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. arXiv preprint arXiv:2110.06389 , 2021.

Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Esben Bjerrum. AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning. Journal of cheminformatics , 12(1):70, 2020.

Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In International conference on machine learning , pages 3668-3679. PMLR, 2020.

Paul R Graves, Lucas J Aponte-Collazo, Emily M J Fennell, Adam C Graves, Andrew E Hale, Nedyalka Dicheva, Laura E Herring, Thomas S K Gilbert, Michael P East, Ian M McDonald, Matthew R Lockett, Hani Ashamalla, Nathaniel J Moorman, Donald S Karanewsky, Edwin J Iwanowicz, Ekhson Holmuhamedov, and Lee M Graves. Mitochondrial protease ClpP is a target for the anticancer compounds ONC201 and related analogues. ACS Chem. Biol. , 14(5): 1020-1029, May 2019.

Alex Hernandez-Garcia, Nikita Saxena, Moksh Jain, Cheng-Hao Liu, and Yoshua Bengio. Multi-fidelity active learning with GFlowNets. arXiv preprint arXiv:2306.11715 , 2023.

Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS omega , 5(51):32984-32994, 2020.

Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548 , 2021.

John J. Irwin, Da Duan, Hayarpi Torosyan, Allison K. Doak, Kristin T. Ziebart, Teague Sterling, Gurgen Tumanian, and Brian K. Shoichet. An aggregation advisor for ligand discovery. Journal of Medicinal Chemistry , 58(17):7076-7087, 2015. doi: 10.1021/acs.jmedchem.5b01105. URL https://doi.org/10.1021/acs.jmedchem.5b01105 . PMID: 26295373.

Jo Ishizawa, Sarah F Zarabi, R Eric Davis, Ondrej Halgas, Takenobu Nii, Yulia Jitkova, Ran Zhao, Jonathan St-Germain, Lauren E Heese, Grace Egan, Vivian R Ruvolo, Samir H Barghout, Yuki Nishida, Rose Hurren, Wencai Ma, Marcela Gronda, Todd Link, Keith Wong, Mark Mabanglo, Kensuke Kojima, Gautam Borthakur, Neil MacLean, Man Chun John Ma, Andrew B Leber, Mark D Minden, Walid Houry, Hagop Kantarjian, Martin Stogniew, Brian Raught, Emil F Pai, Aaron D Schimmer, and Michael Andreeff. Mitochondrial ClpP-mediated proteolysis induces selective cancer cell lethality. Cancer Cell , 35(5):721-737.e9, May 2019.

Samuel Jacques, Almer M van der Sloot, Caroline C Huard, Jasmin Coulombe-Huntington, Sarah Tsao, Sylvain Tollis, Thierry Bertomeu, Elizabeth J Culp, Daniel Pallant, Michael A Cook, Eric Bonneil, Pierre Thibault, Gerard D Wright, and Mike Tyers. Imipridone anticancer compounds ectopically activate the ClpP protease and represent a new scaffold for antibiotic development. Genetics , 214(4):1103-1120, April 2020.

Jan H Jensen. A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration of chemical space. Chemical science , 10(12):3567-3572, 2019.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning , pages 2323-2332. PMLR, 2018.

Seokho Kang and Kyunghyun Cho. Conditional molecular design with deep generative models. Journal of chemical information and modeling , 59(1):43-52, 2018.

Anat Levit Kaplan, Danielle N. Confair, Kuglae Kim, Ximena Barros-Álvarez, Ramona M. Rodriguiz, Ying Yang, Oh Sang Kweon, Tao Che, John D. McCorvy, David N. Kamber, James P. Phelan, Luan Carvalho Martins, Vladimir M. Pogorelov, Jeffrey F. DiBerto, Samuel T. Slocum, Xi-Ping Huang, Jain Manish Kumar, Michael J. Robertson, Ouliana Panova, Alpay B. Seven, Autumn Q. Wetsel, William C. Wetsel, John J. Irwin, Georgios Skiniotis, Brian K. Shoichet, Bryan L. Roth, and Jonathan A. Ellman. Bespoke library docking for 5-HT2A receptor agonists with antidepressant activity. Nature , 610(7932):582-591, Oct 2022. ISSN 1476-4687. doi: 10.1038/s41586-022-05258-z. URL https://doi.org/10.1038/s41586-022-05258-z .

Daniel W Kneller, Gwyndalyn Phillips, Hugh M O'Neill, Robert Jedrzejczak, Lucy Stols, Paul Langan, Andrzej Joachimiak, Leighton Coates, and Andrey Kovalevsky. Structural plasticity of SARS-CoV-2 3CL mpro active site cavity revealed by room temperature x-ray crystallography. Nat. Commun. , 11(1):3202, June 2020.

Maksym Korablyov, Cheng-Hao Liu, Moksh Jain, Almer M van der Sloot, Eric Jolicoeur, Edward Ruediger, Andrei Cristian Nica, Emmanuel Bengio, Kostiantyn Lapchevskyi, Daniel St-Cyr, et al. Generative active learning for the search of small-molecule protein binders. arXiv preprint arXiv:2405.01616 , 2024.

Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric Xing. ChemBO: Bayesian optimization of small organic molecules with synthesizable recommendations. In International Conference on Artificial Intelligence and Statistics , pages 3393-3403. PMLR, 2020.

Panagiotis-Christos Kotsias, Josep Arús-Pous, Hongming Chen, Ola Engkvist, Christian Tyrchan, and Esben Jannik Bjerrum. Direct steering of de novo molecular generation with descriptor conditional recurrent neural networks. Nature Machine Intelligence , 2(5):254-265, 2020.

Hiroyuki Kuwahara and Xin Gao. Analysis of the effects of related fingerprints on molecular similarity using an eigenvalue entropy approach. Journal of Cheminformatics , 13:1-12, 2021.

Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced drug delivery reviews , 23(1-3):3-25, 1997.

Cheng-Hao Liu, Maksym Korablyov, Stanisław Jastrzebski, Paweł Włodarczyk-Pruszynski, Yoshua Bengio, and Marwin Segler. RetroGNN: fast estimation of synthesizability for virtual screening and de novo design by learning from slow retrosynthesis software. Journal of Chemical Information and Modeling , 62(10):2293-2300, 2022.

Daniel Mark Lowe. Extraction of chemical structures and reactions from the literature . PhD thesis, 2012.

Jiankun Lyu, Sheng Wang, Trent E. Balius, Isha Singh, Anat Levit, Yurii S. Moroz, Matthew J. O'Meara, Tao Che, Enkhjargal Algaa, Kateryna Tolmachova, Andrey A. Tolmachev, Brian K. Shoichet, Bryan L. Roth, and John J. Irwin. Ultra-large library docking for discovering new chemotypes. Nature , 566(7743):224-229, Feb 2019. ISSN 1476-4687. doi: 10.1038/ s41586-019-0917-9. URL https://doi.org/10.1038/s41586-019-0917-9 .

Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment in GFlowNets. Advances in Neural Information Processing Systems , 35:5955-5967, 2022.

Łukasz Maziarka, Agnieszka Pocha, Jan Kaczmarczyk, Krzysztof Rataj, Tomasz Danel, and Michał Warchoł. Mol-CycleGAN: a generative model for molecular optimization. Journal of Cheminformatics , 12(1):1-18, 2020.

Kevin McCloskey, Eric A Sigel, Steven Kearnes, Ling Xue, Xia Tian, Dennis Moccia, Diana Gikunju, Sana Bazzaz, Betty Chan, Matthew A Clark, John W Cuozzo, Marie-Aude Guié, John P Guilinger, Christelle Huguet, Christopher D Hupp, Anthony D Keefe, Christopher J Mulhern, Ying Zhang, and Patrick Riley. Machine learning on DNA-encoded libraries: A new paradigm for hit finding. J. Med. Chem. , 63(16):8857-8866, August 2020.

Joshua Meyers, Benedek Fabian, and Nathan Brown. De novo molecular design and generative models. Drug Discovery Today , 26(11):2707-2715, 2021.

Dai Hai Nguyen and Koji Tsuda. Generating reaction trees with cascaded variational autoencoders. The Journal of Chemical Physics , 156(4), 2022.

Andrei Cristian Nica, Moksh Jain, Emmanuel Bengio, Cheng-Hao Liu, Maksym Korablyov, Michael M Bronstein, and Yoshua Bengio. Evaluating generalization in GFlowNets for molecule design. In ICLR2022 Machine Learning for Drug Discovery , 2022.

Anthony Nicholls, Georgia B. McGaughey, Robert P. Sheridan, Andrew C. Good, Gregory Warren, Magali Mathieu, Steven W. Muchmore, Scott P. Brown, J. Andrew Grant, James A. Haigh, Neysa Nevins, Ajay N. Jain, and Brian Kelley. Molecular shape and medicinal chemistry: A perspective. Journal of Medicinal Chemistry , 53(10):3862-3886, 2010. doi: 10.1021/ jm900818s. URL https://doi.org/10.1021/jm900818s . PMID: 20158188.

Pedro O O Pinheiro, Joshua Rackers, Joseph Kleinhenz, Michael Maser, Omar Mahmood, Andrew Watkins, Stephen Ra, Vishnu Sresht, and Saeed Saremi. 3D molecule generation by denoising voxel grids. Advances in Neural Information Processing Systems , 36, 2024.

Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics , 9:1-14, 2017.

Aryan Pedawi, Pawel Gniewek, Chaoyi Chang, Brandon Anderson, and Henry van den Bedem. An efficient graph generative model for navigating ultra-large combinatorial synthesis libraries. Advances in Neural Information Processing Systems , 35:8731-8745, 2022.

Bo Qiang, Yiran Zhou, Yuheng Ding, Ningfeng Liu, Song Song, Liangren Zhang, Bo Huang, and Zhenming Liu. Bridging the gap between chemical reaction pretraining and conditional molecule generation with a unified model. Nature Machine Intelligence , 5(12):1476-1485, 2023.

A. K. Rappe, C. J. Casewit, K. S. Colwell, W. A. III Goddard, and W. M. Skiff. Uff, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American Chemical Society , 114(25):10024-10035, 1992. doi: 10.1021/ja00051a040. URL https://doi.org/10.1021/ja00051a040 .

Recursion. Recursion GFlowNet repository. https://github.com/recursionpharma/ gflownet , 2024. [Online; Accessed 22 May 2024].

Sereina Riniker and Gregory A. Landrum. Better informed distance geometry: Using what we know to improve conformation generation. Journal of Chemical Information and Modeling , 55 (12):2562-2574, 2015. doi: 10.1021/acs.jcim.5b00654. URL https://doi.org/10.1021/ acs.jcim.5b00654 . PMID: 26575315.

Julien Roy, Pierre-Luc Bacon, Christopher Pal, and Emmanuel Bengio. Goal-conditioned GFlowNets for controllable multi-objective molecular design. arXiv preprint arXiv:2306.04620 , 2023.

Nicholas T Runcie and Antonia SJS Mey. SILVR: Guided diffusion for molecule generation. Journal of Chemical Information and Modeling , 63(19):5996-6005, 2023.

Arman A. Sadybekov, Anastasiia V. Sadybekov, Yongfeng Liu, Christos Iliopoulos-Tsoutsouvas, Xi-Ping Huang, Julie Pickett, Blake Houser, Nilkanth Patel, Ngan K. Tran, Fei Tong, Nikolai Zvonok, Manish K. Jain, Olena Savych, Dmytro S. Radchenko, Spyros P. Nikas, Nicos A. Petasis, Yurii S. Moroz, Bryan L. Roth, Alexandros Makriyannis, and Vsevolod Katritch. Synthon-based ligand discovery in virtual libraries of over 11 billion compounds. Nature , 601(7893):452-459, Jan 2022. ISSN 1476-4687. doi: 10.1038/s41586-021-04220-9. URL https://doi.org/10.1038/s41586-021-04220-9 .

Gabriele Scalia, Steven T Rutherford, Ziqing Lu, Kerry R Buchholz, Nicholas Skelton, Kangway Chuang, Nathaniel Diamant, Jan-Christian Huetter, Jerome Luescher, Ahn Miu, et al. A highthroughput phenotypic screen combined with an ultra-large-scale deep learning-based virtual screening reveals novel scaffolds of antibacterial compounds. bioRxiv , pages 2024-09, 2024.

Ruth R Shah and Adrian P Bird. MeCP2 mutations: progress towards understanding and treating rett syndrome. Genome Med. , 9(1):17, February 2017.

Tony Shen, Mohit Pandey, and Martin Ester. TacoGFN: Target conditioned gflownet for structure-based drug design. arXiv preprint arXiv:2310.03223 , 2023.

Vanessa Smer-Barreto, Andrea Quintanilla, Richard JR Elliott, John C Dawson, Jiugeng Sun, Víctor M Campa, Álvaro Lorente-Macías, Asier Unciti-Broceta, Neil O Carragher, Juan Carlos Acosta, et al. Discovery of senolytics using machine learning. Nature communications , 14(1): 3445, 2023.

Teague Sterling and John J Irwin. ZINC 15-ligand discovery for everyone. Journal of chemical information and modeling , 55(11):2324-2337, 2015.

Kyle Swanson, Gary Liu, Denise B Catacutan, Autumn Arnold, James Zou, and Jonathan M Stokes. Generative AI for designing and validating easily synthesizable and structurally novel antibiotics. Nature Machine Intelligence , 6(3):338-353, 2024.

Shidi Tang, Ji Ding, Xiangyu Zhu, Zheng Wang, Haitao Zhao, and Jiansheng Wu. VinaGPU 2.1: towards further optimizing docking speed and precision of AutoDock Vina and its derivatives. bioRxiv , 2023. doi: 10.1101/2023.11.04.565429. URL https://www.biorxiv. org/content/early/2023/11/05/2023.11.04.565429 .

Paul C. Trippier and Christopher McGuigan. Boronic acids in medicinal chemistry: anticancer, antibacterial and antiviral applications. Med. Chem. Commun. , 1(3):183-198, 2010.

Oleg Trott and Arthur J Olson. AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J. Comput. Chem. , 31 (2):455-461, January 2010.

Tiziano Tuccinardi, Giulio Poli, Veronica Romboli, Antonio Giordano, and Adriano Martinelli. Extensive consensus docking evaluation for ligand pose prediction and virtual screening studies. Journal of Chemical Information and Modeling , 54(10):2980-2986, 2014. doi: 10.1021/ ci500424n. URL https://doi.org/10.1021/ci500424n . PMID: 25211541.

Alexandra Volokhova, Michał Koziarski, Alex Hernández-García, Cheng-Hao Liu, Santiago Miret, Pablo Lemos, Luca Thiede, Zichao Yan, Alán Aspuru-Guzik, and Yoshua Bengio. Towards equilibrium molecular conformation generation with GFlowNets. Digital Discovery , 2024.

Zhe Wang, Huiyong Sun, Xiaojun Yao, Dan Li, Lei Xu, Youyong Li, Sheng Tian, and Tingjun Hou. Comprehensive evaluation of ten docking programs on a diverse set of protein-ligand complexes: the prediction accuracy of sampling power and scoring power. Phys. Chem. Chem. Phys. , 18:12964-12975, 2016. doi: 10.1039/C6CP01555G. URL http://dx.doi.org/10. 1039/C6CP01555G .

Felix Wong, Satotaka Omori, Nina M Donghia, Erica J Zheng, and James J Collins. Discovering small-molecule senolytics with deep neural networks. Nature Aging , 3(6):734-750, 2023.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 , 2018.

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In International conference on machine learning , pages 5453-5462. PMLR, 2018.

Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph transformer networks. Advances in neural information processing systems , 32, 2019.

Xin-Min Zhang, Qing Chang, Lin Zeng, Judy Gu, Stuart Brown, and Ross S Basch. TBLR1 regulates the expression of nuclear hormone receptor co-repressors. BMC Cell Biol. , 7(1):31, August 2006.

A State space size estimation

We estimate the state space size by first sampling 1,000 random trajectories, masking out the end-ofsequence action unless the maximum trajectory length max is reached. Then, for every i -th reaction or fragment in the trajectory, we count the average number of valid fragments frag i and reactions react i from a given state in the trajectory, as well as the average number of unique trajectories traj i into which a state can be decomposed using the backward policy. We estimate the state space size as

Experimentally derived average values of these parameters can be found in Table 2. Note that in the second setting we randomly picked 8,000 fragments from the Enamine stock (with the same balancing procedure as in Section 4.3), which after merging with our own fragments, canonization and duplicate removal yielded a total of 8,317 fragments.

B Training details

B.1 Proxy models

The sEH proxy is described in [4]. It is an MPNN trained on a normalized docking score data. We utilize the exact same model checkpoint as provided in [57].

The senolytic classification model is a graph neural network trained on the biological activity classification task of senolytic recognition [74]. Specifically, it was trained on two combined, publicly available senolytic datasets [74, 65]. Reward is given by the predicted probability of a compound being a senolytic. It is worth noting that due to the low amount of data and high imbalance ( < 100 active compounds, a high proportion of which contained macrocycles and were infeasible to construct with fragment-based generative models), this is expected to be a difficult task with sparse reward.

The senolytic proxy model is GNEprop [62], which consisted of 5 GIN layers [75] with hidden dimensionality of 500, utilized Jumping Knowledge shortcuts [76], and had a single output MLP layer. Pretraining was done in an unsupervised fashion on the ZINC15 dataset [66]. The training was done for 30 epochs using the Adam optimizer with a learning rate of 5 × 10 -5 and batch size of 50.

The DRD2 proxy is described in [53, 28]. It is a support vector machine classifier with a Gaussian kernel using ECFP6 fingerprints as a feature representation.

B.2 Generative models

Both RGFN and FGFN were trained with trajectory balance loss [45] using Adam optimizer with a learning rate of 1 × 10 -3 , logZ learning rate of 1 × 10 -1 , and batch size of 100. The training lasted 4,000 steps. A random action probability of 0.05 was used, and RGFN used a replay buffer of 20 samples per batch. Both methods use a graph transformer policy with 5 layers, 4 heads, and 64 hidden dimensions. Exponentiated reward R ( x ) = exp ( β ∗ score ( x )) was used, with β dependent on the task: 8 for sEH proxy, 0.5 for senolytic proxy, 48 for DRD2 proxy, and 4 for all docking runs. Note

that due to different ranges of score values, this resulted in a roughly comparable range of reward values.

For FGFN+SA, we used a modified reward function R ( x ) = exp ( β ∗ (0 . 5 ∗ proxy ( x ) /max _ proxy + (10 -SA _ score ) / 10) , with β adjusted per proxy to match the original reward range.

All sampling algorithms were outfitted with the Vina GPU-2.1 docking, senolytic proxy, sEH proxy, and DRD2 proxy scoring functions. While model architecture hyperparameters and batch sizes were kept consistent between FGFN and RGFN, we allowed FGFN a maximum fragment count of 6 as opposed to RGFN's 5 due to RGFN's larger average building block sizes.

GuacaMol's Graph GA model was trained with a population size of 100, offspring size of 200, and a mutation rate of 0.01 for 2000 generations for a total of 400,000 visited molecules.

For SyntheMol experiments, we used the default building block library of 132,479 compatible molecules and pre-computed docking, senolytic, and sEH proxy scores for all prior to executing rollouts to follow the established methodology. Due to CPU constraints, sampling 500,000 molecules with SyntheMol was impractical. Instead, we executed 100,000 rollouts over approximately 72 hours to match the RGFN training time with docking, yielding 111,964 unique molecules. Additionally, we performed 50,000 rollouts each (approximately 24 hours) for sEH, senolytic, and DRD2 proxies, resulting in 73,941, 69,652, and 62,320 unique molecules, respectively.

casVAE was trained with Bayesian optimization (BO) using default parameters consisting of a hidden size of 200, latent size of 50, and message passing depth of 2. Again, due to time constraints imposed by BO latent space updates, we instead opted to approximately match RGFN training times, training for 72, 24, 24, and 24 hours on the docking, sEH, senolytic, and DRD2 tasks respectively for a total of 135, 41, 38, and 40 rollouts and 7708, 2097, 1521, and 2165 total generated molecules, respectively.

C GPU-accelerated docking

Our docking oracle first accepts canonized SMILES strings as input. These are then converted to RDKit Molecules, protonated, and a low-energy conformer is generated and minimized with the ETKDG [58] conformer generation method and UFF force field [56], respectively. For computational efficiency, we generate one initial conformer per ligand. Each conformer is converted to a pdbqt file and docked against a target with Vina-GPU 2.1 using model defaults: exhaustiveness (denoted by "thread" in the implementation) of 8000 and a heuristically determined search depth d given by

where N atom and N rot are the number of atoms and the number of rotatable bonds, respectively, in the generated molecule. Box sizes were determined individually to encompass each target binding site and centroids were calculated to be the average position of ligand atoms in the receptor template PDB structure file. A negative score is calculated and returned as a reward.

C.1 Target preprocessing

Each target was prepared by removing its complexed inhibitor and atoms of other solvent or solute molecules. We selectively prepared the ClpP 7UVU protein structure by retaining only two monomeric units to ensure the presence of a single active site available for ligand binding and similarly prepared the Mpro 6W63 protein structure by retaining only one monomeric unit.

C.2 Boron substitution for docking

Due to the lack of force field parameters for boron atoms, QuickVina2 is unable to process ligands with boronic acid or ester groups. Therefore, we chose to substitute the boronic acid group and its derivatives in the building blocks/reactants with a carboxylic acid, where the carbonyl carbon is a C13 isotope. This was done for two reasons: firstly, carboxylic acids are considered to be bioisosteric to boronic acids[69], and secondly, the C 13 tag would allow us to distinguish actual carboxylic acids from boronic acid surrogates for SMARTS encoding purposes. This allowed us to dock all possible final products while maintaining structural similarity to the original group and specificity to compatible reaction SMARTS.

D Computational resources used

Evaluating fragment scaling took approximately 800 GPU hours on GeForce RTX 4090 in total. Remaining experiments took roughly 24 GPU hours per run on Quadro RTX 8000 for sEH, DRD2 and senolytic proxies, and roughly 72 GPU hours per run on an A100 for docking-based proxies.

E Implicit reactions

In this work, we define implicit reactions as SMARTS-encoded reactions, products of which contain atoms that are not included in the reactants and come from "implicit reagents". One example is urea synthesis using carbonyl surrogates: the SMARTS template uses two amines, but the product has more atoms than specified in the reactants. These come from the "implicit reagent", in our case any phosgene surrogate (e.g., CDI) (Figure 8).

X

"HN-"

+R2

²HN-

R1-1

R2

R1

N

N

H

Implicit

Reactant1

Reactant2

reagent

In addition to urea formation, other implicit reactions encoded in SMARTS include azide-alkyne cycloaddition, azide-nitrile cycloadditions, and tetrazole synthesis using peptide terminal thiourea cyclizations.

F Sampling large fragment set from Enamine building blocks

In order to evaluate our approach at scale, our building block set had to be balanced to prevent introducing learning biases via building block selection. This was achieved by grouping reagents into twelve fragment classes, for which a specific weight was assigned based on the frequency of appearance of these structures in different reactions. For example, a carboxylic acid appearing in two distinct reactions was assigned a weight coefficient of two, which would later be used to calculate the normalized amount of building blocks per group for a specific database size using the following formula:

The only exception is Michael acceptors, or group 3, which has a weight of 0.25 due to the low availability of these reagents in most public molecular databases and possible overlap with structures that might fit the corresponding SMARTS code (for example, 2-cyano or 2-carbonyl aromatic compounds).

The final set of fragments was constructed by randomly sampling fragments from publicly available Enamine building blocks in a way that roughly preserved the above grouping. This was done in a greedy round-robin fashion, randomly selecting one fragment at a time from the remaining fragments belonging to a given group, and iterating through all groups until the specified number of fragments was selected.

G Target selection

Our targets (sEH, Mpro, ClpP, and TBLR1) were chosen with diverse functions and binding sites in mind as test cases for RGFN. ClpP is a highly conserved compartmentalized protease that degrades

substrates in a signal-dependent fashion and is a promising target in cancer and infectious disease [6]. A number of structurally unrelated small molecules that allosterically activate ClpP, thereby deregulating its protease activity, inhibit the growth of cancer cells and bacterial pathogens [31, 25, 30].

The main protease of SARS-CoV-2, Mpro (also called 3CLpro), is required for processing of the virally-encoded polyprotein and thus for viral replication, and is a well-validated and intensively studied anti-SARS-CoV-2 target [14]. The Mpro catalytic pocket contains 4 distinct sub-pockets that recognize amino acid motifs in substrate sites and is a challenging target due to its structural plasticity [36].

Soluble epoxide hydrolase (sEH) catalyzes a key step in the biosynthesis of eicosanoid inflammation mediators and is being pursued as potential target in cardiovascular and other diseases [16]. sEH has been used as a benchmark substrate for machine learning-based drug discovery and is particularly amenable to computational methods because of its deep hydrophobic pocket [47].

Finally, TBL1 and its paralog TBLR1 are components of the NCoR transcriptional repressor complex and contain WD40 domains that participate in protein interactions [78]. TBL1/TBLR1 mediates the transcriptional repression function of the MeCP2 methylCpG-binding protein, which is mutated in the neurodevelopmental disorder Rett syndrome. Loss of function mutations in Rett syndrome frequently disrupt the interaction of a disordered region in MeCP2 that binds to the WD40 domain of TBL1/TBLR1; conversely, MeCP2 gain-of-function by copy number mediated overexpression leads to X-linked intellectual disability [63].

Well-validated experimental assays are available for each of these 4 different targets [31, 14, 47, 1].

H Ligand post-processing

To ensure the diversity, specificity, and conformer validity of top-generated molecules for each target, we initially categorized our molecules into distinct modes, each representing any SMILES string with a Tanimoto similarity of 0.5 or lower with all other modes. Subsequently, we selected the top 100 modes based on their Vina-GPU 2.1 scores and filtered their docked poses using PoseBusters [11] in "mol" mode, where any pose failing any PoseBusters check was excluded from consideration. As a final precaution, we selected only modes with Tanimoto coefficients to known aggregators of 0.4 or lower using UCSF's Aggregation Advisor [29] dataset. This process resulted in 35, 68, 31, and 15 top modes for sEH, ClpP, Mpro, and TBLR1 binders, respectively Appendix J. Comparative analyses of docked top RGFN modes and confirmed sEH, ClpP, and Mpro ligand poses can be found in Appendices K to M.

I Posebusters, Aggregation Advisor analysis of top 100 modes

J Top filtered molecules for all targets

Senolytics

R = 94.712

R = 94.587

R = 94.041

R = 93.701

R = 92.646

口

Y

R = 92.628

R = 92.615

R = 92.602

R = 92.600

R = 92.229

R = 91.904

R = 91.840

R = 91.807

R = 91.789

R = 91.643

R = 91.588

R = 91.566

R = 91.498

R = 91.078

R = 91.024

O

Y

Q

R = 90.944

R = 90.941

R = 90.900

R = 90.877

R = 90.866

sEH

Q

n'o

R = 14.974

R = 14.942

R = 14.936

R = 14.917

R = 14.916

8

中

R = 14.762

R = 14.744

R = 14.740

R = 14.722

R = 14.681

8

Y

R = 14.672

R = 14.668

R = 14.599

R = 14.580

R = 14.530

8

Paorod

R = 14.453

R = 14.445

R = 14.437

R = 14.429

R = 14.421

mrmlrra

Q

R = 14.394

R = 14.392

R = 14.374

R = 14.325

R = 14.323

ClpP

口

OYY

R = 13.348

R = 13.320

R = 13.186

R = 13.104

R = 13.094

YY

YY

Y

Y

YY

口

丫

R = 13.093

R = 13.052

R = 13.009

R = 12.911

R = 12.858

Y

YQ

KY

R = 12.853

R = 12.848

R = 12.838

R = 12.804

R = 12.782

YY

YYY

YY

Q

Y

A

Y

R = 12.766

R = 12.763

R = 12.758

R = 12.757

R = 12.744

Y

Y

Y

R = 12.682

R = 12.675

R = 12.672

R = 12.665

R = 12.652

Mpro

B

8

888

8

BA

8

R = 11.223

R = 11.177

R = 11.136

R = 11.099

R = 11.041

98

8

R = 11.033

R = 10.979

R = 10.956

R = 10.901

R = 10.896

Aa

8

Y

8

Y

8

R = 10.891

R = 10.884

R = 10.868

R = 10.863

R = 10.849

8

R = 10.838

R = 10.828

R = 10.827

R = 10.824

R = 10.816

Qo:

88

8

8

8

8

Y

R = 10.793

R = 10.775

R = 10.747

R = 10.729

R = 10.727

TBLR1

8

R = 13.403

R = 13.274

R = 13.256

R=13.119

R = 13.050

8

8

R = 13.020

R = 12.965

R = 12.927

R = 12.922

R = 12.922

R = 12.915

R = 12.902

R = 12.896

R = 12.879

R = 12.857

K Docked poses of top generated molecules

Ours (sEH, 4JNC)

Reference (sEH, 4JNC)

Vina-GPU 2.1 Score: -11.13

CH3

Vina-GPU 2.1 Score: -14.97

Ours (ClpP, 7UVU)

HC

CH,

Reference (ClpP, 7UVU)

Vina-GPU 2.1 Score: -13.35

Ours (Mpro, 6W63)

Vina-GPU 2.1 Score: -11.18

Reference (Mpro, 6W63)

Vina-GPU 2.1 Score: -8.53

Vina-GPU 2.1 Score: -11.22

1C

CH

Ours (TBLR1, 5NAF)

Vina-GPU 2.1 Score: -13.40

Vina-GPU 2.1 Score: -13.27

Vina-GPU 2.1 Score: -13.26

L Docking analysis of known ligands

We calculate and report the Vina-GPU 2.1 docking scores of 10 true ligands per target assessed in the paper. Each PDB was prepared as described in Appendix C.1.

M Tanimoto similarity to known ligands

AverageMaximumTanimotoSimilarityDistributionbyTarget

SEH

ClpP

0.40

Mpro

0.35

MaximumTanimotoSimilarity

0.30

0.25

0.20

0.15

SEH

ClpP

Mpro

Target

N Ligands cost analysis

To demonstrate the synthesizability and cost-effectiveness of the ligands produced by RGFN, we conducted a comprehensive cost analysis and proposed a synthesis plan for the top 10 scored ClpP hits generated by RGFN and SyntheMol (see Appendix O). We compared these two methods due to their similarity in chemical approach. For the cost calculation, we only considered the price of building blocks, which were directly sourced from EnamineStore for the case of Synthemol ligands. All prices are represented in US dollars per 0.1 mmol of product (Figures 19 and 20). Notably, for very common reagents only available in bulk, costs per gram were estimated by taking the cheapest or smallest available alternative and dividing its cost by its mass. All prices for individual building blocks were sourced from their respective vendors, which included Millipore Sigma, Oakwood Chemicals, Combi-Blocks, TCI, and AngeneSci.

As expected, ligands generated by RGFN are significantly cheaper to produce compared to SyntheMol, despite having more synthesis steps and lower overall theoretical yields, with an average of 55-70% after 4 steps in the case of RGFN compared to 90-95% average yield after 1 step for SyntheMol. Noticeably, SyntheMol ligand 5 wasn't found to be synthesizable by using reactions proposed in their work. After conducting further analysis, it was found that such a compound could have been produced by Reaction 8 via nucleophilic substitution of the fluorine atom in the trifluoromethyl group; however, this particular reaction is not likely to be performed with sufficient yields, and therefore such a product cannot be considered synthesizable.

O Examples of plausible synthetic routes to molecules

Molecular Weight: 153.9

Molecular Weight: 176.0

Br

suggested yield: ~ 90%

B(OH)2

?

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:139.9

MolecularWeight:172.0

F

Br

B(OH)2

+

*NH2

suggested yield:～ 90%

$9/g

$10/g

1.2 eq; 10.2 mmol (1.43 g)

8.5 mmol (1.465 g)

～$13

～$15

7.67 mmol

suggested yield: ~ 75%

F

5.11 mmol

7.67 mmol

3.83 mmol

MolecularWeight:153.1

N

suggested yield:～ 90%

H

$33/g

3.83 mmol

HN

1.2 eq.; 4.6 mmol (0.704 g)

N

～$23

3.45 mmol;～$71

Molecular Weight:153.9

MolecularWeight:176.0

Br

suggested yield: ~ 90%

B(OH)2

?

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:157.9

Molecular Weight:172.0

F

Br

B(OH)2

suggested yield: ~ 90%

et

$10/g

1.2 eq; 10.2 mmol (1.614 g)

8.5 mmol (1.465 g)

~$16

7.67 mmol

F

suggested yield: ~ 75%

5.11 mmol

7.67 mmol

3.83 mmol

MolecularWeight:152.6

N

suggested yield:~ 75%

口

10$/g

3.83 mmol

NH

1.2 eq.; 4.6 mmol (0.702 g)

~$7

3.06 mmol; ～$58

MolecularWeight:139.9

MolecularWeight:176.0

F

Br

suggested yield:~ 90%

B(OH)2

?

$9/g

$10/g

1.2 eq;6.81 mmol (0.953 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:139.9

MolecularWeight:172.0

F

Br

B(OH)2

十

suggested yield: ~ 90%

$9/g

$10/g

1.2 eq; 10.2 mmol (1.43 g)

8.5 mmol (1.465 g)

～$13

～$15

7.67 mmol

suggested yield: ~ 75%

5.11 mmol

7.67 mmol

3.83 mmol

MolecularWeight:152.6

N

suggested yield: ~ 80%

CI

H

NH

10$/g

3.83 mmol

1.2 eq.; 4.6 mmol (0.702 g)

~$7

3.07 mmol;～$54

Molecular Weight: 153.9

MolecularWeight:176.0

Br-

suggested yield: ~ 90%

+

B(OH)2

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

F

MolecularWeight:136.9

(HO)2B

suggested yield:~ 75%

B(OH)2

$10/g

H

5.11 mmol

1.5 eq.;7.67 mmol (1.073 g)

3.83 mmol

~$11

Molecular Weight:169.2

O=5=

B(OH)2

B(OH)2

suggested yield: ~ 95%

?

H

$0.60/g

HN

3.83 mmol

1 eq.; 3.83 mmol; 0.65 g;

～$0.4

3.64 mmol

Molecular Weight:176.0

B(OH)2

Br

suggested yield: ～ 85%

HN

$10/g

1.2 eq; 4.37 mmol; 0.769 g

HN

$～

3.64 mmol

3.1 mmol,~ $42

Molecular Weight: 153.9

Molecular Weight:176.0

Br

suggested yield: ~ 90%

B(OH)2

?

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:157.9

MolecularWeight:187.0

F

Br

B(OH)2

+

suggested yield:~ 90%

$10/g

$1.36/g

1.2 eq; 10.2 mmol (1.614 g)

8.5 mmol (1.593 g)

～16$

～$

7.67 mmol

suggested yield:~ 75%

N

F

F

5.11 mmol

7.67 mmol

3.83 mmol

Molecular Weight:153.1

N

suggested yield: ~ 90%

33/g

3.83 mmol

HN

1.2 eq.;4.6 mmol (0.704 g)

N

～$23

3.45 mmol;～$62

MolecularWeight:153.9

Molecular Weight: 176.0

Br

suggested yield:～ 90%

+

B(OH)2

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05g )

1 g (5.68 mmol)

～$10

~$10

5.11 mmol

MolecularWeight:136.9

(HO)2B-

suggested yield:～75%

B(OH)2

$10/g

H

5.11 mmol

1.5 eq.;7.67 mmol (1.073 g)

3.83 mmol

～$11

F

MolecularWeight: 147.2

B(OH)2

B(OH)2

suggested yield:～75%

+

$5.60/g

3.83 mmol

1.5 eq.; 5.75 mmol; 0.846 g;

～$5

2.88 mmol

Molecular Weight: 176.0

B(OH)2

Br

suggested yield: ~ 85%

10$/g

1.2 eq; 3.45 mmol; 0.607 g

~$6

2.88 mmol

2.44 mmol,～$42

MolecularWeight:153.9

Molecular Weight: 176.0

Br

suggested yield: ~ 90%

+

B(OH)2

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

Molecular Weight:220.1

suggested yield:～ 75%

Bpin

$39/g

5.11 mmol

3.83 mmol

1.5 eq.; 7.67 mmol (1,687 g)

F

～$65

Molecular Weight: 144.2

Bpin

Bpin

suggested yield: ~ 90%

$40/g

HN

3.83 mmol

1 eq.; 3.83 mmol; 0.552 g;

～$22

3.45 mmol

Molecular Weight: 176.0

Bpin

Br

suggested yield:～85%

$10/g

1.2 eq; 4.14 mmol; 0.728 g

~$7

3.45 mmol

2.93 mmol,~$115

Molecular Weight:157.9

Molecular Weight: 176.0

F

Br

suggested yield: ～ 90%

B(OH)2

$10/g

$10/g

1.2 eq; 6.81 mmol (1.076 g)

1 g (5.68 mmol)

~$11

～$10

5.11 mmol

MolecularWeight:136.9

F

OH

B

HO

OH

suggested yield: ~ 75%

OH

$10/g

5.11 mmol

1.5 eq.; 7.67 mmol (1.073 g)

3.83 mmol

~$11

一

Molecular Weight: 152.6

OH

CI

OH

suggested yield:～75%

N

$10/g

3.83 mmol

1.2 eq.; 4.6 mmol (0.702 g)

B-OH

～$7

HO

2.88 mmol

H

Molecular Weight: 176.0

N

日

N

N

Br

suggested yield: ～ 85%

N

$10/g

?

B-OH

HO

1.2 eq; 3.45 mmol (0.607 g)

~$6

2.88 mmol

2.44 mmol, ～ $44

Molecular Weight: 153.9

Molecular Weight: 176.0

一

Br

suggested yield: ~ 90%

+

B(OH)2

$10/g

$10/g

1.2 eq; 6.81 mmol (1.05 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:139.9

Molecular Weight:172.0

F

Br

suggested yield: ~ 90%

B(OH)2

6/6$

$10/g

1.2 eq; 10.2 mmol (1.43 g)

8.5 mmol (1.465 g)

～$13

7.67 mmol

suggested yield: ~ 75%

5.11 mmol

7.67 mmol

3.83 mmol

MolecularWeight:137.1

H

suggested yield:～80%

$38/g

3.83 mmol

1.2 eq.; 4.6 mmol (0.63 g)

～$24

3.06 mmol; ~$72

MolecularWeight:139.9

Molecular Weight: 176.0

3

Br

suggested yield:～ 90%

B(OH)2

6/6$

$10/g

1.2 eq; 6.81 mmol (0.953 g)

1 g (5.68 mmol)

～$10

～$10

5.11 mmol

MolecularWeight:137.9

Molecular Weight:172.0

OH

Br

B(OH)2

十

suggested yield:～ 90%

$10/g

$10/g

1.2 eq;10.2 mmol(1.41 g)

8.5 mmol (1.465 g)

OH

~$14

～$15

7.67 mmol

suggested yield: ~ 75%

OH

HO

H

5.11 mmol

7.67 mmol

3.83 mmol

Molecular Weight:152.6

Z

suggested yield:～ 80%

OH

OH

H

H

$10/g

3.83 mmol

NH

1.2 eq.; 4.6 mmol (0.702 g)

~$7

3.06 mmol;~$56

MolecularWeight:198.2

suggested

HN

yield: ～ 95%

HO

O

O

0.22 mmol,

0.19 mmol; ~ $353

$162/0.05 g; 0.0505 g

0.2 mmol,

$239/0.05 g; 0.0396 g

～$163

～$189

MolecularWeight:193.2

F

suggested

-NH2

HO

yield:～ 95%

NH

0.22 mmol; 0.0498 g;

0.19 mmol;~$34

$19/0.05 g; ~ $19

0.2 mmol; 0.0386 g;

$19/0.05 g; ~ $15

Molecular Weight:209.2

OH

H

suggested

F

yield: ~ 95%

HN

F

F

0.2 mmol; 0.0528 g;

0.22 mmol; 0.046 g;

0.19 mmol;~$495

$337/0.1 g; ~ $178

$344/0.05 g;~ $317

Molecular Weight: 225.2

suggested

yield: ~ 90%

NF

-NH

0.2 mmol; 0.0512 g;

0.2 mmol; 0.045 g;

$272/0.1 g; ~ $139

$312/0.05 g; ~ $281

0.18 mmol;～$420

Nofeasiblepathways using the

reactions provided by SyntheMol and

that correspond to known reactions

H

MolecularWeight:176.2

Molecular Weight: 259.3

suggested

H

yield: ~ 95%

OH

H2N

0.2 mmol; $63/0.1 g;

0.22 mmol; $298/0.1 g; 0.057 g;

0.0352 g;~$44

～$170

0.19 mmol;~ $214

Molecular Weight:

Molecular Weight:183.3

233.2

suggested

F

yield: ~ 95%

0.18 mmol;~$475

0.2 mmol; $624/0.05 g;

0.2 mmol; $19/0.05 g;

0.0366 g; ~ $457

0.0466 g; ~ $18

MolecularWeight:229.2

suggested

yield:～ 95%

HO

一

NH

O

0.19 mmol; ~ $149

0.2 mmol; $19/0.05 g;

0.0458 g; ~ $17

0.2 mmol; $232/0.1 g; 0.0569 g;

～$132

Molecular

Molecular

Molecular Weight: 301.3

Weight: 177.2

Weight: 139.2

O

OH

suggested

yield: ~ 90%

NH

OH

NH

O

0.2 mmol; $19/0.05 g;

0.2 mmol; $19/0.05 g;

0.2 mmol;

0.0354g;~$13

0.0278 g; ~ $11

$66/0.05 g; 0.0602

0.18 mmol;~$64

g;～$40

suggested

Molecular Weight:176.2

yield: ~ 95%

HN

H2N-

OH

0.19 mmol;～$352

0.2 mmol; $212/0.05 g;

0.0352 g;~$149

0.2 mmol; $235/0.05 g;

0.043 g;~ $202

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We provide extensively results in various tasks and compare against other state -of-the-art methods.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the practicality of the generation process being limited by types of reactions used, oracles employed, and auxillary pharmacological constraints that are not considered.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: For each task, we provide detailed information on experimental setup. Appendix provides further details which allow for full reproducibility of the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All baselines are either derived from openly available repositories or described in sufficient detail to allow reproduction. Code related to the current manuscript is publicly accessible.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide training details including hyperparameters and optimizers, and other details in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Multiple seeds were run for ablation experiment, and we use the variance in these experiments to estimate and report error bars. For molecular docking, because of the high compute cost, we only run one experiment per algorithm. Errors bars are also included when applicable for metric computation across generated samples.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details in the appendix for compute resources required.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research adheres strictly to the NeurIPS Code of Ethics. There are no human subjects or participants involved, and we carefully considered the safety and security aspects of this research.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discussed the societal impact of the work in conclusions.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not foresee this paper to pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All previous baselines and tasks are properly referenced and licenses are used within the appropriate terms.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes] ,

Justification: Code of this manuscript is open sourced and contains documentation describing how to set up and run the code.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No crowdsourcing or human subjects are involved.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: No human subjects are involved.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study modifies prior GFlowNet-based frameworks to produce molecules satisfying synthesizability. While previous GFlowNet-based molecular generation completes molecules by adding fragments or atoms, the proposed method repeats following stems to generate molecules: (1) selects reaction templates, (2) selects reactants, and (3) performs reactions and selects one of the resulting molecules. In the experiments, the proposed method shows competitive performance while preserving synthesizability.\n\nSTRENGTHS:\n- The proposed method first incorporates a reaction-based generation framework with GFlowNet for synthesizability.\n- A chemical language for synthesizability seems to offer cost advantages in real-world applications, demonstrating that it acts as a promising inductive bias to generate promising molecules.\n- The proposed method shows competitive performance compared to (1) FGFN in terms of synthesizability and (2) SyntheMol in terms of average reward and mode discovery. Although it finds fewer modes compared to FGFN, this seems to stem from constraints in the generation space.\n\nWEAKNESSES:\n- The preliminary is insufficient for understanding the components of the method, particularly mentioning the flow matching condition while the method section actually describes forward and backward policy.\n- To improve synthesizability-related metrics, one may consider including them in the rewards. Can authors provide a comparison with FGFN including the SA score in the reward? (e.g., multiobjective GFN)\n- The lower number of modes in RGFN (compared to FGFN) is due to constraints in the generation space (ensure high SA scores). Therefore, it would be interesting to compare the number of modes filtered by SA scores.\n- Can authors compare the cost of generation time? Additionally, I am curious about time costs for (finding a synthesizability path + generation with FGFN) vs. (generation with RGFN).\n\n**Minor:**\n- Error in line 114 $m'i$\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nThe authors have acknowledged the limitations of their approach.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a model for molecule design called Reaction-GFlowNet (RGFN). RGFN is an extension of the GFlowNet framework [4] (which has previously been used to generate molecules by building them out of small fragments) to generate molecules through virtual chemical reactions, ensuring that the molecules RGFN generates are more likely to be synthesizable. The paper evaluates RGFN against GraphGA (a graph genetic algorithm), FGFN (the original fragment-based GFlowNet), and SyntheMol (a recent synthesis-based generative model of molecules proposed in [54] based around Monte Carlo tree search). The paper shows RGFN finds competitively scoring molecules on two proxy* and one docking task (as expected the unconstrained GraphGA does best), while also ensuring the molecules it suggests do better on important synthesizability metrics.\n\n* proxy meaning the task is to optimize against a graph neural network (GNN) oracle trained on a small amount of target data.\n\n(Edited Aug 14 to update the score: see comments below).\n\nSTRENGTHS:\n## Summary of review\nThis paper addresses one of the limitations of GFlowNets by explicitly incorporating synthesis plans into the molecular generative process. However, this seems to be a fairly straightforward extension of the GFlowNets framework with synthesis-based generative models of molecules proposed elsewhere. The experiments in this paper compare the proposed method against only one of these existing models.\n\n## Originality/Significance\nRFGN is an extension of GFlowNets [4] to build molecules through chemical reactions rather than through combining small molecular fragments. This addresses a key limitation with the original GFlowNet, which the experiments show often generates molecules for which no synthetic route can be found.\n\nHaving said that, there have been many synthesis-based generative models of molecules proposed (e.g., [8,17,54,19,22] and others not cited -- see below), and RGFN seems a fairly straightforward combination of these ideas with GFlowNets.\n\n> Korovina, K., Xu, S., Kandasamy, K., Neiswanger, W., Poczos, B., Schneider, J. and Xing, E. (2020) ‘ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations’, in International Conference on Artificial Intelligence and Statistics. AISTATS 2020    \n\n> Nguyen, D.H. and Tsuda, K. (2022) ‘Generating reaction trees with cascaded variational autoencoders’, The Journal of chemical physics, 156(4), p. 044117.   \n\n> Vinkers, H.M., de Jonge, M.R., Daeyaert, F.F.D., Heeres, J., Koymans, L.M.H., van Lenthe, J.H., Lewi, P.J., Timmerman, H., Van Aken, K. and Janssen, P.A.J. (2003) ‘SYNOPSIS: SYNthesize and OPtimize System in Silico’, Journal of medicinal chemistry, 46(13), pp. 2765–2773.  \n\n\n> Button, A., Merk, D., Hiss, J.A. and Schneider, G. (2019) ‘Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis’, Nature Machine Intelligence, 1(7), pp. 307–315.\n​\n\n## Quality\nThe optimization tasks done to evaluate RGFN seem interesting and it's nice to see the molecules suggested also evaluated in terms of their diversity and synthesizability (using a retrosynthesis planner). While I think it is great that the tasks were more complicated than some of the simple ones often used elsewhere, it might have been nice to include some more commonly-used (yet still challenging) benchmarks too (such as the Therapeutics Data Commons benchmark -- citation below). This would make comparisons with existing models easier.\n\n> Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik.   Therapeutics data commons:  machine learning datasets and tasks for therapeutics. arXiv preprint arXiv:2102.09548, 2021.\n\n\n## Clarity\nOverall I thought the paper was well-written and easy to follow. In particular, I thought Section 3.2 was helpful in giving an overview of the approach (along with the informative Figure 1), and Equations 2–7 were helpful in explaining the parameterizations of the various networks involved. The experiments also seemed to be clearly presented. Readers unfamiliar with GFlowNets [4] would likely have to read about them elsewhere (lines 70–82 provide a short, although limited introduction). However, this seems reasonable due to the overall space available.\n\nWEAKNESSES:\n## Originality\nAs mentioned under the originality/significance heading in the section above, RGFN seems a fairly straightforward combination of the ideas of GFlowNets with recent synthesis-based generative models of molecules. It would have been helpful if there had been more focus on why synthesis-based GFlowNets were a better approach to take than the currently proposed alternatives to better promote the paper's significance. (Hence why I have gone with a low contribution score).\n\n## Only one other synthesis-based generative model of molecules is compared against \nThe only other synthesis-based generative model for molecules that is compared against in the experiments is SyntheMol, and this baseline has been limited here in the amount of compute it uses. I can understand no comparison is done against [22,19] due to the lack of open-sourced code, but it would have been nice to have had a comparison against some of the other ones cited, e.g., [17] with its open sourced code available on GitHub (https://github.com/wenhao-gao/SynNet). It would also be nice to have a comparison to the other mentioned methods for encouraging synthesizability, e.g. via using scoring methods [34].\n\n(This along with the fact that no code is available -- see below -- is why I have gone with a lower soundness score).\n\n## No code is shared\nAs far as I'm aware there is no code currently available for RGFN? It does not seem to be provided as part of the submission (despite this box being ticked in the paper checklist)? I would find it impossible to reproduce the results of the paper using the details provided in Appendices B-D alone (e.g., I'm unsure of what building blocks and reaction templates to use or even how the molecules fed into the GNNs were featurized). One of the proxies was initially trained in an unsupervised manner on the ZINC dataset (Appendix B), but there are scant details on how this was done.\n\nOn a similar note, the compute resources used to train the models also seems to be missing. This is again marked as included in the paper checklist, so would appreciate being pointed towards this if I have made a mistake and missed this?    \n\n\n## Method is currently limited to a restricted synthetic space\nRGFN uses only 350 building blocks and 17 reactions. This is low compared to the ~150k building blocks and 49 reactions used in [19], or the 5000 building blocks and 90 reactions used in [22]. The experiments done in Section 4.3 suggests the RGFN scales badly to larger building block libraries even with architecture modifications and larger reaction libraries do not seem to have been tried. This, combined with the fact that RFGN can only explore linear synthetic trees (i.e., multiple complex intermediates cannot be created and then combined) and bimolecular reactions restricts the molecules that the approach could explore. This might be an issue in more challenging optimization tasks.\n\nQUESTIONS:\n1. What is the distribution of the number of reaction steps typically used when finding useful molecules?\n\n\n\n2. I don't completely follow how the size of the state space was calculated in Figure 2. Perhaps a cartoon would be useful in Appendix A. Is the state space size different to the total number of unique molecules that can be generated?\n\n\n\n3. I'm confused as to why RGFN seems to often outperform FGFN in the docking/proxy scores of the molecules it finds. I would have thought the latter model is more flexible and so should be able to find better scoring molecules, even if they did not end up being synthesizable. The paper suggests that the library of fragments available for RGFN might be less suitable than those found in the building blocks picked for RGFN. Have different fragments been tried or other experiments carried out to investigate this hypothesis?\n\n\n\n4. RGFN seems to produce molecules with lower QED scores than SyntheMol and FGFN: is there any intuition for why this might be the case?\n\n\n\n5. Line 243 says:\n> \"All RGFN modes were additionally inspected manually by a chemist and confirmed as synthesizable, which indicates that AiZynth scores are likely underestimated.\"  \n   \nDoes this mean that a chemist took the proposed molecule and synthesized it in the lab? Were the proposed synthetic routes from RGFN used or were alternatives derived?\n \n\n\n6. Line 282 states:\n>  \"It is also important to recognize that RGFN does not explicitly generate synthetic routes to the molecules.\"    \n\nI did not understand what was meant by this. I thought the whole point of RGFN is that it did explicitly generate synthetic routes to the final molecule through the sampled trajectory? \n\n\n\n7. Equation 4 versus 7:\na. My understanding is that the architecture expressed in Equation 4 is used instead of that of Equation 7 (apart from in Section 4.3)?\nb. The architecture expressed in Equation 7 seems to do better in Figure 5 so why not always use this?\nc. Could also a similar principle be used to scale up the number of reaction templates available?\n\n\n\n8. Line 221 discusses a procedure for picking molecular modes from the list of proposed molecules. Given modes have to be a certain Tanimoto distance apart, I assume this procedure is run greedily and is somewhat dependent on how the molecules are ordered?\n\nLIMITATIONS:\nTwo limitations are discussed in Section 5: (i) that the model is currently limited to a very small reaction and building block library, and (ii) the issues inherent with docking oracles. Regarding the first of these, the authors discuss and evaluate a modification of the architecture to try to enable the method to scale (see Section 4.3), but performance still degrades as the initial fragment library increases and so this seems to remain an outstanding limitation (see also the weaknesses section above). The second limitation (docking oracles are not perfect) is not limited to RGFN and is a problem among similar models more generally. The authors discuss incorporating additional, more-accurate oracles in a multi-fidelity framework to resolve this which seems reasonable.\n\nAnother limitation which is not discussed is the sample efficiency of the proposed approach (which is again also a limitation for the baselines). The number of molecules the non-GFlowNet baselines visit during optimization for the different tasks is described in Section 4. These are high and range from 70k-400k, and I would be interested in how this compares to the two GFlowNet models.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-05"}

comments: SUMMARY:
This work proposes a GFlowNet-based framework for synthesizable molecule design, which comprises of building block selection stages and reaction selection stages. The GFlowNet-based model uses chemical oracle functions as reward and the goal is to generate synthesizable molecules with the desired property quantified by the scoring function.

STRENGTHS:
- This work tackles the synthesizability challenge in molecular design, which is crucial to experimental validation but has been long overlooked by previous work on molecular generation.
- Molecular synthesis pathways have been represented by directed acyclic graphs, which fit well the structure of GFlowNet sampling process. Therefore, this work choosing GFlowNet as the framework is proper and well-motivated.
- The experimental study is comprehensive. It demonstrates that RGFN can produce synthesizable molecules with higher docking score. It also examines the diversity and docking structures of the generated molecules

WEAKNESSES:
- As acknowledged in the paper, the major limitation is scalability to a larger chemical space. It becomes much more difficult to discover hit scaffolds when the chemical space gets larger. In addition, the capability of this framework is limited by oracle functions.

QUESTIONS:
- Molecules generated by building blocks and reactions are guaranteed to be highly likely synthesizable. Is it still necessary to evaluate empirical synthetic accessibility scores of these molecules? as these scores are generally inaccurate and they cannot tell much when a molecule already has synthesizability guarantee.
- Can this framework be applied to bottom-up synthesis planning?

LIMITATIONS:
See Weaknesses
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a workflow to synthesize molecules with reaction templates starting from some building blocks. Through this pipeline, the synthesizability of the generated molecules can be improved based on some experimental evidence of molecular drug design tasks.\n\nSTRENGTHS:\n(1) This reviewer agrees with the importance of researching the synthesizability of generated molecules;\n\n(2) The presentation of the proposed approach is very clear. This reviewer can quickly understand the major idea of the whole workflow; \n\n(3) The idea of generating molecules with chemical reactions is very valuable since this reviewer also thinks the most important thing is generating synthesizable molecules.\n\nWEAKNESSES:\n(1) The related work discussion in this work is insufficient. This reviewer just searches a bit on the web and find three highly related articles, which are \"Bridging the gap between chemical reaction pretraining and conditional molecule generation with a unified model\", \"A generative model for molecule generation based on chemical reaction trees\" and \"Generating molecules via chemical reactions\". This reviewer believes other relevant articles are not covered in this article. A more comprehensive and systematic related work section is required;\n\n(2) The proposed method has many limitations, including reliance on reaction templates and powerful pre-trained models. We state these limitations in the following section;\n\n(3) The evaluation of the proposed approach is not comprehensive enough. It does not include the conventional molecular generation benchmarks like novelty, validity, and diversity.\n\nQUESTIONS:\n(1) This reviewer is concerned about the pre-trained oracle models. It seems that almost every step of the pipeline relies on a single pre-trained model. So how to derive these pre-trained models independently?\n\n(2) This reviewer is concerned about the ability to generate truly novel molecules. It seems the proposed approach starts with a selected set of fragments and deduces products through reaction templates. However, since reaction templates are extracted from known reactions, it seems nearly impossible to generate purely novel molecules through this pipeline. How do the authors address and evaluate this issue?\n\nLIMITATIONS:\n(1) It seems the whole pipeline of RGFN heavily relies on many different pre-trained models. Each stage can affect the overall process drastically. \n\n(2) The proposed approach is based on reaction templates. The reaction templates must be updated frequently to cover more types of reactions. However, the template-based method has poor generalization to unseen reactions (new combinations of reactants).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}