SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraints

Authors: Miruna Cretu, Charles Harris, Ilia Igashov, Arne Schneuing, Marwin Segler, Bruno Correia, Julien Roy, Emmanuel Bengio, Pietro Lio
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: gflownets, de novo molecular generation, synthesizable molecular design

Abstract:
Generative models see increasing use in computer-aided drug design. However, while performing well at capturing distributions of molecular motifs, they often produce synthetically inaccessible molecules. To address this, we introduce SynFlowNet, a GFlowNet model whose action space uses chemical reactions and buyable reactants to sequentially build new molecules. By incorporating forward synthesis as an explicit constraint of the generative mechanism, we aim at bridging the gap between in silico molecular generation and real world synthesis capabilities. We evaluate our approach using synthetic accessibility scores and an independent retrosynthesis tool to assess the synthesizability of our compounds, and motivate the choice of GFlowNets through considerable improvement in sample diversity compared to baselines. Additionally, we identify challenges with reaction encodings that can complicate traversal of the MDP in the backward direction. To address this, we introduce various strategies for learning the GFlowNet backward policy and thus demonstrate how additional constraints can be integrated into the GFlowNet MDP framework. This approach enables our model to successfully identify synthesis pathways for previously unseen molecules.

Decision: accept (spotlight)
Reviewer scores: 8, 8, 6, 8

Reviews and discussion:
<thread>
creation date:
1729608728477

thread_type:
official_review

<thread_object>
creator: reviewer_aadh

<content>
summary:
This paper presents SynFlowNet, a practical approach for drug design that incorporates reaction-based Markov Decision Processes (MDPs) into a Generative Flow Network (GFlowNet) to generate diverse molecules with guaranteed synthetic pathways.
Notably, authors propose a novel backward policy training strategy to ensure the presence of backward trajectories.
Furthermore, the authors investigate the potential of SynFlowNet to be integrated with wet experiments such as fragment-based crystallography.

strengths:
1. Novel Backward Policy of GFlowNet training: SynFlowNet proposes a novel learning strategy for backward policy to ensure the directed acyclic graph and facilitate synthesizable molecules.
2. Connection to Wet-Lab Experiments: The case study in Section 4.6 provides a clear demonstration of how to connect wet-lab experiments and *in silico* drug design, highlighting the potential real-world applicability of the method.
3. Comprehensive Ablation Studies: The paper provides ablation studies for many design choices.

weaknesses:
**Overall Comment**

I acknowledge the novelty of this work in successfully adapting the GFlowNet framework for synthesizable molecular design, but there are several critical aspects of the current manuscript that need to be addressed.
I think this manuscript may be suitable for acceptance after revision, and I’m positive about changing the scores during the rebuttal period.

*\* The issues are sorted in the order they appeared in the manuscript.*

**Issue 1: Lack of novelty / appropriate citations for the major contribution.** Page 2, Line 103, Contribution 3

One major concern is that the manuscript does not contain appropriate citations for this major contribution.
To use 200k building blocks, the authors used action embedding [1], which has already been employed in existing synthesis-oriented molecular generative models [2-4].
In particular, Koziarski et al. [4] introduced action embedding into GFlowNet and investigated its effectiveness in large action spaces.
While the authors use a different fingerprint representation (Morgan fingerprints instead of MACCS fingerprints), Gottipati et al. [2] have already evaluated various representations, including both fingerprints.
Authors should highlight the novel differences from existing approaches or provide appropriate citations.

**Issue 2: Lack of experimental details.** Section 4.1 and Section 4.5
- In Section 4.1, what is the rewarding strategy used for multi-objective optimization (FragGFN SA, SynFlowNet SA, SynFlowNet QED)? The possible reward structures could be Multi-objective GFlowNet (MOGFN) [5] or multiplication-based rewards [6].
- In Section 4.5, I couldn't find the experimental details. The related section is on Page 19, Section A.5, but it does not provide crucial information about this experiment. I suggest to include experimental details and mention it like Section 4.6.
  - Reward Function: What reward function was used in the ablation studies? Is it sEH?
  - Average Times: Clarify what *average times* refers to–e.g., runtime per batch or runtime per molecule.
  - For Figure 7 D-F, entire building block set is used?

**Issue 3: Fairness of evaluation metric.** Page 8, Figure 6

The improvement in the reward/diversity trade-off is impressive, but I have questions about the metric, **Reward/Novelty trade-off**.
Regarding the Reward/Novelty trade-off, it is possible that the improvement in novelty is due to the use of the Enamine building blocks rather than the effectiveness of well-structured generative model architecture.
In this case, the REINVENT's high similarity to ChEMBL would be greatly reduced by simply retraining it on Enamine REAL or ZINC.

For a fair comparison in terms of novelty, the comparison should be performed on the same exploration space:
- Train REINVENT on Enamine REAL.
- Use ChEMBL-like building blocks, such as blocks obtained from the retrosynthetic decomposition [3] of ChEMBL molecules.

**Issue 4: Limited comparative study.** Sections 4.2 and 4.3

The comparison between the proposed method and baselines was performed only on proxy models.
I recognize that proxy models such as sEH [7] are widely used to evaluate the effectiveness of optimization strategies in the optimization community [8], so I think this is a minor issue.
However, I have concerns about the appropriateness of evaluating and comparing the performance of discovering novel molecules based on proxy models trained on a restricted and known chemical space.

**Issue 5: Restricted scalability to a large building block set.** section 4.5

I definitely agree with the argument for employing an extensive building block (BB) set to explore the vast chemical space.
I believe that it can mitigate the reward-diversity trade-off by expanding the sample space.
However, I concern that the current experimental results do not seem to provide sufficient evidence about this argument and the scalability of SynFlowNet.

The authors demonstrated that a larger BB set yielded a higher selection of unique BBs (Figure 7B), but this has a negative impact on the average reward (Figure 7A) and speed (Figure 7C).
Consequently, the current findings seem to emphasize the negative effects of the usage of extensive BB sets on the quality and throughput for generation.
To justify the significance of utilization of unique BBs, I suggest including structural diversity metrics for generated samples, such as the number of unique Bemis-Murcko scaffolds or the average pairwise Tanimoto distance.

---
**Reference**
1. Dulac-Arnold, Gabriel, et al. "Deep reinforcement learning in large discrete action spaces." arXiv preprint arXiv:1512.07679 (2015).
2. Gottipati, Sai Krishna, et al. "Learning to navigate the synthetically accessible chemical space using reinforcement learning." International conference on machine learning. PMLR, 2020.
3. Seo, Seonghwan, Jaechang Lim, and Woo Youn Kim. "Molecular generative model via retrosynthetically prepared chemical building block assembly." Advanced Science 10.8 (2023): 2206674.
4. Koziarski, Michał, et al. "RGFN: Synthesizable Molecular Generation Using GFlowNets." ICML'24 Workshop ML for Life and Material Science: From Theory to Industry Applications (2024).
5. Jain, Moksh, et al. "Multi-objective gflownets." International conference on machine learning. PMLR, 2023.
6. Lee, Seul, Jaehyeong Jo, and Sung Ju Hwang. "Exploring chemical space with score-based out-of-distribution generation." International Conference on Machine Learning. PMLR, 2023.
7. Bengio, Emmanuel, et al. "Flow network based generative models for non-iterative diverse candidate generation." Advances in Neural Information Processing Systems 34 (2021): 27381-27394.
8. Gao, Wenhao, et al. "Sample efficiency matters: a benchmark for practical molecular optimization." Advances in neural information processing systems 35 (2022): 21342-21357.

questions:
1. **Page 2, Figure 1B:**
    - As I understand it, the building block selection probability depends on not only the state embedding but also the selected reaction. (Page 4 Line 213). However, the Figure 1B indicates that the selection probability is calculated only from the state information $s_k$. I suggest revising the figure and including the series of equations for the neural network to clarify this process.
    - Also, I was wondering if dot product operation between Morgan fingerprint (binary vector) and state embedding can satisfy the GFlowNet Objective. When the 4-bit fingerprints for building blocks $b_1$, $b_2$, and $b_3$ are $[1,0,0,0]$, $[0,1,0,0]$, and $[1,1,0,0]$, respectively, the probability of $b_3$ can be derived from those of $b_1$ and $b_2$:
      $$\log P(b_3|s) = \frac{\log P(b_1|s) + \log P(b_2|s)}{\sqrt{2}}$$
      I think that this relationship can introduce bias in the forward transition probability.
3. **Page 7, Footnote:**
Is it typo?: '… which are different to the **reactants** used by SynFlowNet …'. As I know, AiZynthFinder allows user to use different reactants. I think 'reactants' should be replaced to 'reactions'.
4. **Page 9, Figure 7C:**
I suggest changing the title "Performance" to "Runtime" or a similar term.
5. **Page 19, Table A.1 (Hyperparameters)**:
Could you clarify the meaning of *max_nodes*? In the standard GFlowNet implementation, it refers to the maximum number of heavy atoms in molecules, but the generated molecules illustrated in the paper have more than 9 heavy atoms.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730177756287

thread_type:
official_review

<thread_object>
creator: reviewer_chnb

<content>
summary:
The paper presents SynFlowNet, a model designed to improve the generation of novel drug-like molecules while ensuring their synthetic accessibility. SynFlowNet constructs molecules sequentially using commercially available reactants and documented chemical reactions, effectively constraining the generation process within a synthesizable chemical space. The model integrates a forward synthesis constraint, evaluates synthetic accessibility through external retrosynthesis tools, and optimizes diversity and reward (e.g., binding affinity) without relying on reinforcement learning, which tends to sample fewer, high-reward molecules. SynFlowNet introduces a backward policy for navigating reaction trajectories, improving synthetic feasibility and sampling accuracy. By utilizing molecular fingerprints and scalable building blocks, SynFlowNet can explore vast chemical spaces efficiently, making it adaptable for specific drug targets when provided with experimental fragment screening data, such as in SARS-CoV-2 protease inhibitors.

strengths:
- Focus on Synthetic Accessibility: SynFlowNet adopts a framework that effectively addresses a key limitation in generative models for drug design by ensuring that generated molecules are synthetically feasible, which is crucial for practical drug development applications.
- Diverse and Optimized Molecule Sampling: Leveraging GFlowNets, SynFlowNet provides a balanced sampling process that maximizes both reward and structural diversity, effectively mitigating the typical mode collapse encountered in reinforcement learning-based models.
- Validation through Synthetic Accessibility Metrics: In silico experiments demonstrate SynFlowNet’s advantage over fragment-based approaches, particularly in terms of synthesizability, underscoring the practical benefits of synthesis-based molecule generation for drug discovery.

weaknesses:
Limited Testing on Optimization Tasks: While SynFlowNet showcases improvements in SeH proxy, its optimization abilities are not fully tested against established molecular optimization benchmarks. Incorporating PMO benchmark tasks and directly comparing results with other generative baselines, such as those in Gao et al. (2022) [1], could provide a more comprehensive evaluation of SynFlowNet’s optimization performance.

### Reference
[1] Gao, Wenhao, et al. "Sample efficiency matters: a benchmark for practical molecular optimization." Advances in neural information processing systems 35 (2022): 21342-21357.

questions:
NA
</content>
</thread_object>

</thread>
<thread>
creation date:
1730625102731

thread_type:
official_review

<thread_object>
creator: reviewer_6jhq

<content>
summary:
The authors proposed a molecular generative model that explicitly considers the synthesizability of generated molecules. Specifically, they formulated a molecular design problem into a Markov decision process (MDP) that successively adds a purchasable building block (BB) with a selected reaction to the previous state. A GFlowNet has been adopted to tackle the MDP. Namely, SynFlowNet shows better performance in improving the diversity and high reward modes simultaneously, whereas conventional optimization approaches like reinforcement learning only focus on the highest reward modes. To handle more BBs, they are represented with binary Morgan fingerprints, and the probability of sampling a particular BB is computed as a function of its fingerprint and the embedding vector of the current state. In addition,  new backward policies have been tested to ensure that the backward trajectories can return to existing building blocks. They also performed a case study that leverages experimental data for more efficient drug design as a practical application. Overall, the manuscript is well-written, but several issues need to be addressed.

strengths:
1. The use of the GFlowNet improves both diversity and high reward modes compared to other optimization methods such as reinforcement learning.
2. The proposed backward policy ensured that backward-constructed trajectories belong to the GFlowNet MDP. 
3. Various benchmark and ablation studies were performed to ensure the strategies of the proposed framework.

weaknesses:
1. The conclusions of some experiments are arguable (see the questions below), which need to be justfied probably with additional experiments for clarity.  
2. Increasing the number of BBs slows down training linearly that may cause a computational burden when dealing with a large amount of BBs.

questions:
1. In Table 1, while the two backward policies, the Maxlikelihood and REINFORCE, show relatively high ratios of the solved routes in both the training and test sets, their number of high reward modes is similar to that of the Uniform Policy. This result is rather disappointing, considering the objective of the generative model is to sample more high-reward modes. In this context, the authors need to discuss the effect of the proposed backward policies on the performance of the forward policy in more detail.
2. In Figure 4, lower rewards achieved by FragGFN and FragGFN SA seem questionable because FragGFN covers a much larger chemical space, as seen from Figure 2, and so is more likely to find molecules with higher rewards. Has the poor performance of FragGFN originated from insufficient training? Or is it because of the use of SeH proxy as a reward, which may have a severe generalization problem for unseen molecules? This should be justified in the revision. 
3. In Section 4.1, what is the rewarding strategy used for multi-objective optimization (FragGFN SA, SynFlowNet SA, SynFlowNet QED)? The possible reward structures could be Multi-objective GFlowNet (MOGFN) or multiplication-based rewards.
4. Regarding the metric Reward/Novelty trade-off, it is possible that improving the novelty for SynFlowNet may be simply due to the use of the Enamine building blocks, while REINVENT has been trained with ChEMBL. Thus, the high similarity of the molecules generated by REINVENT to ChEMBL is not unexpected and would be reduced if training it with the Enamine REAL. The authors need to address this issue.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730634795395

thread_type:
official_review

<thread_object>
creator: reviewer_f18i

<content>
summary:
It was a pleasure to review this paper on synthesizability-constrained molecular generation using GFlowNets. Overall, I felt the paper was strong and focuses on an important application area of generative models, it was well-written, and generally builds well on previous work in the field and introduces some novelty. In my review below, I focus on aspects and unresolved questions that, if addressed, could enhance the paper. I summarize my main concerns as: (1) in its current state, the work is not reproducible, and (2) the evaluation metrics and comparisons to prior work used in evaluating SynFlowNet are not always the most relevant for molecular design, and do not tell us much about how this method would actually perform in a realistic setting. There is room for improvement here. I think this paper has the potential to be a really strong paper if it addresses the concerns below.

strengths:
* The background and related work section was strong, and covered the previous work in this domain well.
* The paper, including the appendix, was well-structured and easy to follow.
* There are more strengths, but I mention them below amongst some of the specific questions I had, to put the questions in context.

weaknesses:
* I searched throughout the paper and in the submission, and could not find any code to accompany the paper. Given that the authors lamented the lack of publicly-available code for two previous works in their paper, I would have expected a greater emphasis on publishing the code here, as it would also help with the reproducibility of this work. To me this is unacceptable for a computer science conference submission in 2024, so please let me know if I simply missed it, and I can raise my score (I ranked it below the acceptance threshold because of this, and would update my score if the code was made available as it was overall a good paper).
* Throughout the paper, the authors repeat that one of the key advantages of SynFlowNet is its ability to generate more diverse candidates than other methods (namely, REINVENT and SyntheMol). However, I argue that diversity is only a meaningful metric in molecular generative models if the types of molecules being generated are from the accessible molecular space; if the generated molecules are diverse but not accessible, then the model is not very practically useful. That being said, it was good the the authors evaluated synthesizability using AiZynthFinder, but an "easy" experiment that the authors could have also done that would add a lot of value is to consider molecular rediscovery benchmarks. That is, can the model also find known, previously-synthesized binders; celecoxib rediscovery is a widely-used standard benchmark for this, but there are also more challenging targets. Such rediscovery metrics are a nice alternative to experimental validation which is not necessarily possible on the review timeline. To summarize, I would like to know that SynFlowNet can generate diverse and *known* synthesizable molecules, not just diverse *predicted* synthesizable molecules, because at the end of the day CASP tools are not a replacement for experiment.
* I found the lack of comparison to other methods for synthesizability-constrained molecular generation a weakness. The authors mentioned that the models of Gottipati and Horwood & Noutahi were not publicly available, and thus focused on comparison to SyntheMol and REINVENT. While these comparisons were interesting and certainly well-motivated, I would have also liked to know how SynFlowNet compares to other synthesizability-constrained molecular generation models which *do* have publicly available codebases, like MoleculeChef or SynNet. As it stands, we do not know which model is best, because SyntheMol has not done any comparisons to those models either.
* In the Appendix, the authors state that 300K oracle calls were used both to train REINVENT and SynFlowNet, however, this is an extreme number of oracle calls; can the authors justify this? REINVENT demonstrates good performance after only 10K oracle calls for most objectives, so are 300K oracle calls simply what is needed for good performance in SynFlowNet? I would like to see the analogous experiments and comparisons/results with only 10K oracle calls, which is the standard number of oracle calls used for comparing molecular generative models.

questions:
* I could not find how $s_0$ is initialized, nor how important this is (i.e., how different initializations here impact the performance of SynFlowNet). Can the authors clarify? Does it correspond to a molecular fingerprint/embedding of the target molecule, is it random noise, or something else?
* One of the contributions of this paper is stated as "we propose one of the first attempts at training the backward policy in a GFlowNet with a separate objective from for the forward policy, which we show to correct backward-generated trajectories and improve sample quality and diversity" but I am not sure I got that. Which experiment was it exactly that demonstrated the new backward policy led to improved sample quality/diversity? Are these results in the paper? If so this needs to be clarified.
* How sample efficient is SynFlowNet? I would have liked to see an analysis of this, which is really important for molecular generation tasks as we are almost always operating in the low data regime and data/sample efficiency is key.
* How does SynFlowNet perform when optimizing for targets that are highly symmetric, e.g., molecules with a Cn rotation axis, where n$\ge$2? Has this been evaluated? This is something that is a known weakness of other synthesizability-constrained molecular generation frameworks (e.g., SynNet). 
* What are the failure modes of SynFlowNet? This should be more thoughtfully analyzed, as all models have failure modes and it is important to understand them to help future research build on the results of SynFlowNet.
* How challenging would it be to expand SynFlowNet beyond bi-molecular reactions? I am not suggesting this should have been done here, but asking more from curiousity and wondering if this is a future direction the authors are aiming to explore in future work.

Minor:
* Are the points in Figure 6 (last panel) shuffled for plotting? It was not clear to me if there are red points under the blue ones or vice-versa. This should be clarified as the results from this plot are informing the conclusions.
* The references are poorly formatted, this should be fixed.
* gsk3$\beta$ should be GSK3$\beta$ (weird capitalization in the Appendix).
</content>
</thread_object>

</thread>
<thread>
creation date:
1732059665581

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response

comment:
We thank all reviewers for their time and effort in providing feedback and constructive criticism on our work. In our individual responses to each reviewer we address all comments and questions. In our global response below we highlight the most important changes we've made to our manuscript. All changes are highlighted in blue in the revised version of the paper.

1. We uploaded the code accompanying the paper: [https://anonymous.4open.science/r/synflownet-01A3](https://anonymous.4open.science/r/synflownet-01A3/README.md)
2. Following the suggestions of reviewers f18i and cHNB, we included sample efficiency results on the PMO benchmark and added evaluations of SynFlowNet on rediscovery tasks.
3. Following the suggestions of reviewers 6JHQ and AADH, we support our novelty claims with additional experiments of training SynFlowNet with ChEMBL-derived building blocks, which show that we preserve novelty scores.
4. We clarified statements with regards to the scalability of our approach, the benefits of the proposed backward policies, the limitations of the model and we better explain various experimental details and design choices.

We thank all reviewers again for their feedback and we remain available for further questions!
</content>
</thread_object>

</thread>


Metareview:
metareview:
In this work, authors introduce SynFlowNet, a generative model that combines GFlowNets with reaction-based Markov Decision Processes to generate synthetically accessible drug-like molecules. The key technical innovation is incorporating forward synthesis constraints directly into the generative process by using commercially available reactants and documented chemical reactions. The model introduces novel backward policy training strategies to ensure generated molecules have valid synthetic routes.

The paper demonstrates several important scientific findings. First, SynFlowNet achieves better diversity in generated molecules compared to baseline methods like REINVENT while maintaining high rewards, as shown through extensive experiments with both proxy models and docking scores (Reviewer f18i). Second, the model scales effectively to large building block sets of over 200k compounds, though with expected computational overhead (Reviewer 6JHQ). Third, the backward policy training innovations help ensure generated molecules have valid synthetic routes back to building blocks, addressing a key challenge in synthesis-constrained generation (Reviewers AADH and 6JHQ).
The paper's key strengths include: 1) A well-engineered solution to the critical problem of ensuring synthetic accessibility in molecular generation, validated through multiple metrics and external retrosynthesis tools, 2) Comprehensive empirical evaluation including ablation studies, comparison to strong baselines, and demonstration on practical drug discovery tasks, 3) Clear practical relevance demonstrated through integration with experimental fragment screening data.

The initial submission had some limitations around reproducibility and evaluation completeness, but these were thoroughly addressed during review. The authors added code, conducted additional experiments on molecular rediscovery tasks and sample efficiency (addressing Reviewer f18i's concerns), and validated the novelty claims by testing with ChEMBL-derived building blocks (addressing Reviewers 6JHQ and AADH's concerns). The revised evaluation provides strong evidence for the method's capabilities.

The authors were highly responsive during review, strengthening the evaluation to conclusively demonstrate the benefits of their approach. While there remain some scaling challenges with large building block sets, the current capabilities are sufficient for many practical applications. The combination of technical innovation, thorough evaluation, and clear practical relevance makes this a strong contribution to the field. I recommend acceptance because the paper makes a significant technical contribution to synthesizable molecular generation while maintaining high standards of empirical validation, an important problem that has received lesser attention in the molecule generation community.