Title: Retrosynthesis Planning via Worst-path Policy Optimisation in Tree-structured MDPs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Retrosynthesis planning

reinforcement learning

self-imitation learning
Abstract: Retrosynthesis planning aims to decompose target molecules into available building blocks, forming a synthetic tree where each internal node represents an intermediate compound and each leaf ideally corresponds to a purchasable reactant. However, this tree becomes invalid if any leaf node is not a valid building block, making the planning process vulnerable to the "weakest link" in the synthetic route. Existing methods often optimise for average performance across branches, failing to account for this worst-case sensitivity. 
In this paper, we reframe retrosynthesis as a worst-path optimisation problem within tree-structured Markov Decision Processes (MDPs). We prove that this formulation admits a unique optimal solution and provides monotonic improvement guarantees. Building on this insight, we introduce Interactive Retrosynthesis Planning (InterRetro), a method that interacts with the tree MDP, learns a value function for worst-path outcomes, and improves its policy through self-imitation, preferentially reinforcing past decisions with high estimated advantage. 
Empirically, InterRetro achieves state-of-the-art results — solving 100\% of targets on the Retro*-190 benchmark, shortening synthetic routes by 4.9\%, and achieving promising performance using only 10\% of the training data.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new approach to retrosynthesis planning called InterRetro, which builds synthesis routes for molecules by focusing on the weakest part of the route. \n\nThe key idea is to ensure that all paths in the synthesis tree end in purchasable compounds, since one failed path makes the whole plan invalid. \n\nInterRetro treats the planning process as a worst-path optimisation problem in a tree-structured Markov Decision Process and improves its decision-making using self-imitation learning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper reframes retrosynthesis planning as worst-path optimisation in tree-structured MDPs, which is both novel and well-justified.\n\nIt derives Bellman equations for the worst-path value and proves the existence and uniqueness of the optimal value.\n\nStrong empirical results achieving state-of-the-art performance.\n\nThe code is available, and the provided example is reproducible.\n\nWeaknesses:\n\nThe quality of the figures does not match the quality of the paper. Some figures (e.g., Figures 2 and 4) show rasterization artifacts when the paper is enlarged to a comfortable reading size, and Figure 4 in particular is difficult to interpret when printed on A4 paper.\n\nThe design of Table 1 is confusing: in the text, you state that you compare established retrosynthesis planning methods and list them together, but in the table, they appear in separate columns. I still struggle to understand what is being presented in the table, even after reviewing it several times.\n\nQUESTIONS:\nOut of all the listed contributions, the third one—\"we apply this framework to retrosynthesis through InterRetro, a search-free approach to multi-step planning\"—is not entirely clear to me. I understand the other three contributions, but this one feels more like an application rather than a contribution in itself. I would appreciate an elaboration on what makes this application a distinct contribution.\n\nIn the Tree-structured MDPs section, could you please explain why the domain of the mapping is 2^S?\n\nWhat does 190 refer to in Retro*? I couldn't find any information about this in the citation link you provided—almost everywhere it's simply referred to as Retro*.\n\nAccording to the metrics, your results are quite impressive. Could you elaborate on why you think this is the case and what the consequences might be for the field? Should we expect a significant leap forward in retrosynthesis planning, or do these strong metrics not necessarily translate to major changes in industry applications?\n\n\nAfter addressing my questions and concerns, I would be willing to reconsider my evaluation. \n\nThank you for your submission and have a nice day!\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper formulates retrosynthesis as a worst-path optimization problem within tree-structured Markov Decision Processes, and proposes InterRetro, a method that interacts with the tree MDP, learns a value function for worst-path outcomes, and improves its policy through self-imitation. Experimental results show that InterRetro achieves state-of-the-art performance, solving 100% of the targets on the Retro*-190 benchmark, and also demonstrates strong results on two additional test sets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well-written and presents a comprehensive and complete study.\n2. The idea of worst-path optimization is quite interesting. It offers a new perspective for addressing the retrosynthesis planning problem.\n3. The experimental results are compelling and demonstrate significant improvements compared to prior algorithms.\n\nWeaknesses:\n1. The clarity of some descriptions could be improved. For example, the deterministic transition function is a mapping from (state, action) to to a set of reactant molecules. Each molecule corresponds to a single state, so transitioning from one state to a set of states is not a well-defined formulation.\n\n2. The experimental part needs to be supplemented with more details. For specific points, please refer to the Questions part.\n\n3. The significance of the paper needs to be further elaborated. In retrosynthesis planning, real-time feedback is not typically required, and taking several minutes to find a feasible solution is generally acceptable. The quality of the solution is more critical. Therefore, the claim that time-consuming real-time search limits practical utility in large-scale molecular design scenarios warrants further scrutiny.\n\nQUESTIONS:\n1. The paper mentions hundreds or thousands of model calls per molecule; however, previous retrosynthesis algorithms tested on the Retro*-190 dataset achieve a success rate of 99% with an average number of calls fewer than 50 [1] [2], and significantly reducing the number of model calls can also be achieved by caching the results of previous single-step retrosynthesis model calls to avoid redundant computations.. Please kindly verify this point again.\n[1] Yu,Y.etal.Grasp:Navigating retrosynthetic planning with goal-driven policy. In Advances in Neural Information Processing Systems (NeurIPS 2022)\n[2]  Xie,S.etal.Retrograph:Retrosynthetic planning with graph search.In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2120–2129 (ACM, 2022).\n\n2. $\\gamma^T$ is employed to penalize deeper decompositions; however, if the target molecule requires a relatively long synthesis pathway, could this negatively impact the search efficiency? Specifically, might the search become biased towards shorter paths and spend excessive effort exploring those regions?\n\n3. Graph2Edits serves as the single-step retrosynthesis model in your approach. Could you please explain how it is combined or integrated with the trained policy during the planning or search process?\n\n4. In table 1, success of Retro* with template-based single-step model is $75.26\\%$ when the number of model calls is limited to $500$, which is different with the result displayed in the paper of Retro*. Could you please provide an explanation?\n\n5. What is the average number of model calls for the algorithm? Could this information be included in a table for clearer comparison?\n\n6. This paper optimizes for the shortest path to generate feasible synthesis routes. However, retrosynthesis involves various optimization objectives, such as the cost of chemical reactions. For example, Retro* approximates cost optimization by using the negative log of the policy function. How does the algorithm proposed in this paper incorporate such diverse optimization objectives?\n\nIf the authors can address the question I raised above, I would be willing to increase my score.\n\nLIMITATIONS:\nThe main text of the paper does not discuss limitations or potential negative societal impacts; these aspects are only addressed in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-24"}

comments: SUMMARY:
This paper introduces InterRetro, a novel approach that frames multi-step retrosynthesis planning as a worst-path optimization in tree-structured MDP. Unlike traditional methods that optimize for average or cumulative rewards, InterRetro focuses on maximizing the value of the most challenging path, ensuring all branches terminate successfully.

STRENGTHS AND WEAKNESSES:
Strengths: The proposed worst-path optimization is intuitive while achieving strong empirical results. Besides, The authors provide rigorous theoretical guarantees, including proofs for the existence of a unique optimal solution and assurances of monotonic policy improvement with their weighted self-imitation learning algorithm.

Weaknesses: My main concern regarding this paper is which component contributes to the empirical performance improvement. While InterRetro proposes a novel objective and training methodology, it utilizes Graph2Edits as its underlying single-step model. Graph2Edits is a high-performing single-step retrosynthesis model itself, and its inherent strength might contribute significantly to the overall success. Without an ablation study specifically varying the single-step model while keeping the worst-path objective and self-imitation learning algorithm consistent, it is difficult to fully ascertain whether the improvements are primarily from the novel objective and learning paradigm or from the strong Graph2Edits base.

QUESTIONS:
1. What is the insight for choosing Graph2Edits as the single-step model?
2. Can you ablate your method by replacing the single-step model with the pre-trained MLP from retro*? This can help understand the relative importance of the design made in the paper.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nHere the authors introduce a new method (InterRetro) for multi-step retrosynthesis planning by introducing a worst-path objective that aims to improve the most infeasible path in a synthetic route. Overall I think the idea is strong and novel, and the paper is nice amd well-written and could be well-suited to this venue, but IMO the main limitation holding it back right now is that it is missing comparison to actual SOTA baselines for this task (e.g., AiZynthFinder or ASKCOS) and instead comparing against not the leading models for multi-step retrosynthesis prediction (then claiming SOTA). This would need to be improved for me to recommend the paper for acceptance.\n\nSTRENGTHS AND WEAKNESSES:\n- Code available (and lots of routes provided as supplementary material, a bit strange), which is a strong point, but the documentation on how to use the code, packages/environment, etc is lacking. This could be improved to improve the utitlity of the model\n- Seems comparing to neither AiZynth nor ASKCOS which IMO are the leading models for retrosynthesis prediction, especially when it comes to multi-step retrosynthesis planning.\n- I like the results illustrating the performance under different budgets (e.g., Figure 4a), it would be nice to also know how the number of calls compares to compute (e.g., if this was done on a single GPU, etc), and how this compares to other methods.\n- Limited details on the dataset used for training (and how evaluations were done), this should be improved. It is not sufficient to simply say USPTO-50k, since how the splits were created for training/validation/testing are also important. With this in mind, I would encourage the authors to look into the PaRoutes (https://doi.org/10.1039/D2DD00015F) dataset which has been nicely curated for this task.\n- Similarly, on the molecules selected for evaluation from the three different datasets (e.g., Retro*-190, ChEMBL-1000, GDB17-1000) not much is said, some justification needs to be provided here. For instance, it is not clear to me why GDB17 was selected at all, as these molecules to my knowledge are not particularly drug-like or representative of a chemical space in either drug discovery or materials science, so unclear why they are used to evaluate a synthesis tool (do not get me wrong, I think there is value in GDB17, just not for this task).\n\nQUESTIONS:\n- How do the depths of the predicted synthesis trees compare to the real synthesis trees in the training data?\n- The authors mention that their models are better at generating shorter routes, which is good, but how good is the model at also predicting known routes? This is not something that has been evaluated in the paper and I think it is important (it is also separate from the success rate).\n- How was data selected for the dataset usage experiments (e.g., 1%, 5%, etc.)? Was this done randomly, and does this make sense?\n- How can it be that sometimes the single-step model is better than the multi-step model (e.g., for limited model calls?)? I feel like this could be better explained because to me this is an unexpected result.\n\nLIMITATIONS:\nComparison to SOTA is not well done (see comments above).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}