LLM-Augmented Chemical Synthesis and Design Decision Programs
Authors: Haorui Wang
Jeff Guo
Lingkai Kong
Rampi Ramprasad
Philippe Schwaller
Yuanqi Du
Chao Zhang
Venue: ICML 2025 poster (2025)
Keywords: Large Language models
Retrosynthesis planning
Molecule design
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: Retrosynthesis, the process of breaking down a target molecule into simpler precursors through a series of valid reactions, stands at the core of organic chemistry and drug development. Although recent machine learning (ML) research has advanced single-step retrosynthetic modeling and subsequent route searches, these solutions remain restricted by the extensive combinatorial space of possible pathways. Concurrently, large language models (LLMs) have exhibited remarkable chemical knowledge, hinting at their potential to tackle complex decision-making tasks in chemistry. In this work, we explore whether LLMs can successfully navigate the highly constrained, multi-step retrosynthesis planning problem. We introduce an efficient scheme for encoding reaction pathways and present a new route-level search strategy, moving beyond the conventional step-by-step reactant prediction. Through comprehensive evaluations, we show that our LLM-augmented approach excels at retrosynthesis planning and extends naturally to the broader challenge of synthesizable molecular design.

LLM-Augmented Chemical Synthesis and Design Decision Programs
Haorui Wang' Jeff Guo
Lingkai Kong" Yuanqi Dut
Rampi Ramprasad Chao Zhang?
Philippe Schwaller
Abstract
nentially with the number of reaction steps the depth of the route tree. Consequently, efficient decision-making in retrosynthesis planning, and more broadly, in chemical design, remains a critical challenge.
Retrosynthesis, the process of breaking down a target molecule into simpler precursors through a series of valid reactions, stands at the core of or- ganic chemistry and drug development. Although recent machine learning (ML) research has ad- vanced single-step retrosynthetic modeling and subsequent route searches, these solutions remain restricted by the extensive combinatorial space of possible pathways. Concurrently, large language models (LLMMs) have exhibited remarkable chem- ical knowledge, hinting at their potential to tackle complex decision- making tasks in chemistry. In this work, we explore whether LLMs can suc- cessfully navigate the highly constrained, multi- step retrosynthesis planning problem. We intro- duce efficient scheme for encoding reaction pathways and present new route-level search strategy, moving beyond the conventional step-by- step reactant prediction. Through comprehensive ovaluations, we show that our LLM-augmented approach excels at retrosynthesis planning and extends naturally to the broader challenge of syn- thesizable molecular design.
Recent research has harnessed machine learning to tackle retrosynthesis by modeling reactions with a single-step model which predicts reaction template, i.e., reac- tion coded as a pattern, to synthesize the given target molecule (Segler & Waller, 2017; Coley et al., 2017; Liu et al., 2017), including graph neural networks (Dai et al., 2019; Chen & Jung, 2021), and subsequently reverse the template to obtain the reactants. Another branch of single- step models do not rely on the provided reaction templates and directly predict reactants (Liu et al., 2017; Schwaller et al., 2020; Igashov et al.). After training the single-step models, they are further connected with search algorithm (e.g. Monte Carlo tree search (Segler et al., 2018) or A search (Chen et al., 2020) to perform multi-step retrosyn- thetic analysis, which halts when a path to set of predefined purchasable molecules is found.
Recent studies have shown that large language models (LLMs) implicitly encode substantial chemical knowledge, as evidenced by their remarkable performance in search- ing molecular structures with optimized properties (Wang et al., 2024). In addition, LLMs have been leveraged for reasoning and planning problems, such as automated exper- imentation in chemistry (M. Bran et al., 2024; Boiko et al., 2023). Despite the apparent promise, the extent which LLMs handle tightly constrained decision processes, such as retrosynthesis planning, remains largely unexplored. Unlike open-ended tasks like text generation, retrosynthe- sis imposes rigorous constraints on the sequence of actions (reaction steps). Only certain reaction templates are valid, and only commercially available or otherwise feasible pre- cursors can be used.
1. Introduction
Retrosynthesis (Corey & Wipke, 1969; Corey et al., 1985) concerns with breaking down a target molecular structure into a sequence of simpler or more readily available precur- sor structures and chemical reactions (Boström et al., 2018). It is essential for many chemistry problems that require the realization of proposed molecular structures from organic synthesis to drug discovery (Blakemore et al., 2018). Nev- ertheless, the search space for given target is tremendous as the number of possible synthesis pathways grows expo-
In this paper, we investigate whether the knowledge em- bedded in LLMs can be effectively leveraged for complex sequential decision-makinn tasks in chemistry such as ret- rosynthesis planning. Crucially and by contrast to existing LLM works for retrosynthesis (Nguyen- Van et al., 2024; Yang et al., 2024), we do not tune the base LLM. Instead, by exploring how LLMs perform under heavy constraints,
'Equal contribution 'Georgia Tech *Cool Polytechnique Fédérale de Lausanne (EPFL) NNational Centre of Competence in Research (NCCR) Catalysis "Harvard University CCorell Univer- sity. Correspondence to: Haorui Wang <hwang984 gatech.edus.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
aim to gain insights into their potential to serve as pow- erful decision-making engines, ultimately advancing our understanding of their capabilities in chemistry and beyond. Furthermore, we expand the scope to study the capabil- ity of L.MMs in not only finding synthesis pathway, but also simultaneously optimizing the property of the target molecule, known as synthesizable molecular design (Brad- shaw al., 2019; 2020: Gottipati et al., 2020; Horwood & Noutahi, 2020; Korovina et al., 2020; Gao et al., 2022; 2024; Koziarski al., 2024; Cretu et al., 2024; Seo et al., 2024; Swanson et al., 2024). We summarize our main con-
without altering the underlying framework, ensuring adapt- ability to various chemical spaces.
Given T and C, our goal is to iteratively select backward reactions that construct a valid synthetic route for the target molecule. Each molecule in the synthetic route, including intermediates, is explicitly defined through the application of reaction templates to its reactants. Compared to general molecule generation tasks, retrosynthesis planning intro- duces additional challenges, such enforeing chemical reaction rules and ensuring the use of commercially avail- able building blocks.
tributions are as follows:
2.1. Retrosynthesis Planning
D. We propose an efficient and effective way to encode the sequence of synthesis decisions: (1) a language to describe reactions that LLMs understand and (2) efficient data structures to store the exponential-grove tree-structured synthesis pathways.
Given target molecule Murget. the objective is to identify sequence of reactions {r1.72..... FnJ Fn] such that:
1. Marget can be recursively decomposed into reactants by applying reaction templates from T.
We integrate a sequence-level search strategy into LLM retrosynthesis planning, sampling complete decision sequences (full multi-step pathways) instead of single reaction steps, and apply a smooth reward with partial feedback to evaluate each pathway.
2. The final set of reactants consists exclusively of molecules in C.
3. Each reaction r; E T is chemically valid and adheres to the predefined reaction rules.
Experimentally, we study both the retrosynthesis plan- ning and synthesizable molecular design problems in this unifying paradigm of I.LM-augmented reaction decision program.
At each decision step we select reaction rt E to apply to a molecule Mj the molecule set. This generates its reactants {M, ..., which are then added to the molecule set, replacing M, The task is completed when all terminal nodes in the synthetic pathway correspond to molecules in C.
2. Problem Formulation
We formulate the retrosynthesis planning problem as se- quential decision-making problem. At the core of this task is molecule set, which contains either the molecules we aim to synthesize (target) or purchase directly (permitted commercial building blocks). We initialize the molecule set with only the target molecule and evolve over succes- sive search steps until there no molecule in the set that is non purchasable.
2.2. Synthesizable Molecular Design
In contrast to retrosynthesis planning, we consider the syn- thesizable molecular design problem, where the goal is to find molecules with optimal properties evaluated by an ora- cle function O while simultaneously ensuring that they are synthetically accessible through feasible reaction pathways.
arg max ((m) s.t. V(R(m)) = E
At each step,
backward reaction to decompose molecule in the set (the product) into its reactants. This involves removing the product from the molecule set and adding the corresponding reactants generated by the selected backward reaction. The process terminates when either all molecules remaining the set are purchasable or the maximum budget of attempts is reached.
where Ω is the set of generated molecules, R(()) returns the synthesis path, and V--) checks the validity of the path.
3. Methodology
3.1. Route formatting
A reaction is formally defined by reaction template, which specifies structural transformation pattern in the form of a SMARTS string (Daylight Chemical Informa- tion Systems). We denote the set of feasible reaction tem- plates by and the set of purchasable compounds by C. Both T and are flexible and can be refined or expanded
Traditional machine learning methods directly predict reac- tion classes reactants based on input molecules, which by definition, defines the synthesis route when the full set of reaction classes and molecules are considered (Zhong et al., 2024). However, using LLMs as retrosynthesis route gener- ators necessitates a well-defined textual input-output format,
2

---

L.M--Augmeneed Chemical Synthesis and Design Decision Programs
1. INITIALIZATION
2. EVALUATION
Level Evaluation
Reaction Routes Database
Mating Pool of Routes
Ensure chemical
validity and prevent
Input target molecule 1, 0,
LLM generates routes
Evolutionary search
adverse hallucinations
1. Molecule
Update Mating Pool Keep ne routes by reward
2. Reaction
3. Route
Retrieve routes top most similar molecules
LLM System Prompt
As professional chemist specialized in synthesis analysis, you are tasked with...
(by Tanimoto fingerprint similarity)
Retrieved Route 1
4. MUTATION
3. SELECTION
Step
Retrieved Route 2
Routes
Selected Route
Step 2:
Step
Step
Step
Step
Retrieved Route 3
Keep top n. routes by reword
Step 2:
Selected Route 2
Step
Step
Step 1: <
Mutate reactions and/or molecules Starting
Step
Step
Selected Route
Step 3: < >
INVALID stops
Step
Step
Compute Route Rewards
<INVALID>
m E M (molecules at INVALID step)
Take molecules
Reward Σ SC Score(m)
proposed at this step
Figure 1. Overview of the LLM-SSn-Planner. 1. INITIALIZATION: Based on the target molecule, reaction routes of similar molecules are retrieved and scored by the SC score (Coley et al., 2018). 2. EVALUATION: The LLM generates new routes which are evaluated. 3. SELECTION: Starting from invalid steps the reaction routes, the SC score of the molecules this step are computed and the top n. routes are selected. 4. MUTATION: Starting from these invalid steps, the L.LM proposes mutations to modify the molecules and/or reactions at this step. Repeat until a solution found or budget reached.
as LLMs are highly sensitive to prompt design (Sclar et al., 2024). A critical challenge lies in determining how to repre- sent the retrosynthesis route for LLMs. Prior research have proposed two main representation formats:
tional tree search-based approaches to retrosynthesis plan- ning (Segler et al., 2018). In these approaches, the nodes in the search tree represent synthetic states, and the tree itself contains all molecules required to synthesize the tar- get molecule at the root. A target molecule is considered synthesized when all leaf nodes in the tree correspond to purchasable building blocks. The edges of the tree corre- spond reactions, which specify a chemical transformation between states of connected nodes.
Textual descriptions (Liu al., 2024a): Textual de- scriptions align naturally with the text generation capa- bilities of LLMs and uses descriptive language to detail each reaction step. However, the flexibility and lack of standardization in textual descriptions make chal- lenging to consistently extract essential information, such as reactants, products, and reactions. This am- biguity complicates the evaluation of individual steps and the validation of the overall synthesis route.
Building this framework, we reformulate retrosynthesis planning into a step-by-ster decision-makinn process that is more suitable for LLMs (Figure 2b). Specifically, we repre- sent the synthesis route as a sequence of decisions, where each step involves proposing a reaction from database of re- action results, i.e., reaction templates. The LLM maintains a dynamic molecule set that starts with the target molecule and evolves reactions selected, ending when all molecules in the set are purchasable. To improve the decision-makinn process, we integrate a reasoning component called the "Ra- tional" at each step. This reasoning step encourages the LLM to think before making decisions (Wei et al., 2022). Additionally, we ask the L.M explicitly output the prod- uct and reactants in each step, in order to keep the generated
Tree structures (Chang et al.): Tree structures (Fig- ure 2a) represent synthetic pathways as hierarchical trees, capturing the relationships between reactants and products a structured manner. While tree struc- tures provide a more systematic representation, their complexity increases significantly in multi-step ret- rosynthesis tasks, leading deeply nested structures that can overwhelm the LLM's 's reasoning capabilities.
To address these limitations, we draw inspiration from tradi-
3

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
route more consistent.
bility of choosing template from an LLM. Therefore, we assign pseudo--robabilities
the predicted reactions by employing self-consistency frequency, frequee,,, which is an ensemble approach that samples k independent reactions for the next step, denoted as {r(41 {rit1)}=1 } ~ p(rt+1|m [m) step t. From these samples, we identify the unique reactions and con- sider them as the set of potential next-step reactions. The frequency of each reaction in this set is then used to compute its cumulative score, given by:
<ROUTEX
"rakkoud"" Tieqet Medocuk EVILES, lata pait Prophoed reaction and araiyell "palinen"
Torgst Mileosed.
Stetonal daes snalylt
Tirge Ni sculff
"redkcud"" Precusso 1 SMiLes, "chiira""
"Seadies Poactio
PRoada 1b: Ploadar 1. Raadent LT:
"rodecud"" TPhesasor StriLes
'Poaden RRadent
Stetonel Jas seokel
Thoaeert SSaadie
"medecudd"
"Boadanss rs: (Bub Roudant 1. bub Reccart 2t
temptew""
you Proposed rnation and analyele "deccmm [-]
Sub Moadans FPaaden
1-
p(n) # j () 1+1 =n
REXPLANATIONN Explesstorn www.adereEEPPPPuor.. EXPLEGATION
(a) Tree format
(b) Sequential format
where #(j r( = n denotes the count of samples for reaction n. In essence, this expression computes how many times each reaction appears in k sampled reactions.
Figure 2. Different route formats for retrosynthesis planning
Finally, we integrate the LLM as a single-step predictor with MCTS (Segler al., 2018) Retro* (Chen et al., 2020) search algorithms to explore retrosynthesis pathways.
3.2. LLM as single-step prediction model
Recent studies have demonstrated the potential of utilizing LLMs planners for complex decision-makinn tasks (Song et al., 2023; Huang et al., 2024). A common approach is integrating LLMs with traditional search algorithms such as MCTS (Zhao et al., 2024) and A* search (Zhuang et al., 2023). This integration addresses a key limitation of LLMs: their lack of a systematic mechanism to explore structured solution spaces. Without such mechanisms, LLMs may struggle to effectively navigate complex decision-makee making scenarios. The core idea of these methods is straightforward: treat the LLM as policy that directly generates the next action based on the history of past actions and observations. Meanwhile, search algorithms like MCTS and A* system- atically explore and optimize the solution space, ensuring robustness and completeness.
Algorithm LLM-Syy-Planner Algorithm
Data: The target molecule T; the reward function F; the evaluation function E; the population size ne; the number of retrieval size no; the routes retrieval set O: the maximum number of attempts budget.
Result: Found synthesis routes population IP*
begin
Po -1
while len(Pu) < n. do
sample IP, = {mr from O proportionally to their products' Tanimoto similarity to T
WAAIIAATIONN (T,,
for p Po do Compute Fp) for t E [0, budget] do of offspring = []
Building this, we propose using LLMs single-step retrosynthesis predictors, and operate in the cmmplate-based approach, where we start with pre-defined templates set that represents all the reactions the I.LM could suggest and a reference reactions database based on USPTO. The product molecule in each step serves as the input, and the LLM is queried predict reaction that synthesizes this product molecule. To do this, we first task the LLM with identifying substructures and functional groups in the product molecule. Next, we draw inspiration from Coley et al. (2017) and com- pute the Tanimoto similarity between the substructures and the product molecules in the reference reactions database. The hypothesis that similar product molecules are syn- thesized from similar reactions. Following these steps, the LLM retrieves template from the pre-defined list, which is important as it removes any possibility of hallucinated tem- plates. The template is then applied to the product molecule to obtain a set of predicted reactants.
for cdum_mutations do
sample p from P. proportionally to reward F(p)
evaluate p using the evaluation function F(p)
to get feedback f offspring aappraaggaappeetttttIIIIIITTTT f))
for p P, do Compute Fp))
P+t1 + sorted nd]
Return Prudget
3.3. LLM as a synthesis pathway sampler
Although L.MSs can leverage search algorithms to explore the search space, akin to existing works that pair single-step reaction prediction with search algorithms (Zhong et al., 2024), we are particularly interested in their ability to de- sign synthesis routes directly for a given target molecule.
By contrast to existing single-step prediction models (Maziarz et al., 2023), non-trivial to obtain a proba-
4

---

L.M--Augmentdd Chemical Synthesis and Design Decision Programs
Level
Type
Explunation
searching for exact matches. If exact match is found, we retrieve the top 100 most similar reactions based on reaction fingerprint similarity. These candidates are then filtered by assessing whether the proposed reaction is chemically feasi- ble for the given product molecule, as even if the retrieved route is for a similar molecule, slight differences in the tar- get molecule structure can render the reaction incompatible. The most similar valid reaction is retained as the matched reaction. Finally, we replace the original reaction proposed by the LLM with the identified match, thus removing the possibility of hallucinated reaction that we cannot easily verify the chemical soundness of. If no valid match is found in this process, we label the reaction as non- non-exxitent. At the route level, we evaluate route connectivity by checking whether the 'molecule set' in a given step aligns with the 'updated molecule set' from the previous step and whether the expected 'product' appears in the current step's molecule set. A step is considered valid if all evaluations pass, except for molecule availability.
Validity
Whether the molecule valid (RDKit parsable) Whether the molecule commercially available. i... the building block stock
Molecule
Availability
Existence
Whether the reaction exists datahase
Reaction
Whether the product be synthesized from the proposed reactants
Validity
Route
Connectivity
Whether the route connected
Table 1. Three levels of feedback in the evaluation stage.
To this end, we propose an evolutionary search algorithm named L.LM-Syn-llanner that enables LLMs generate and optimize the whole retrosynthetic pathways directly. We emphasize generate as the LLM is not explicitly retriev- ing a reaction template like in the case of using the LLM as a single-step prediction model and coupling a search al- gorithm. Unlike existing works that follow this paradigm (Zhong et al., 2024), our approach generates the entire multi- step synthesis tree directly. The algorithm operates fol- lows: Given a target molecule, we first generate an ini- tial pool of retrosynthetic routes using INITIALIZATION queries from LLMs, where each route is evaluated using a reward function, F()). Next, a route is sampled with a probability proportional to its reward and edited using MUTATION operator to generate offspring. This muta- tion process is repeated num_ mutation times, after which the newly generated offspring are added to the population. The offspring are then evaluated using F(.), and the ne fittest candidates at each step are selected to pass on to the next generation. This iterative process continues until the maximum number of model calls reached. The overall workflow consists of four key stages: (1) Initialization, (2) Evaluation, (3) Selection, and (4) Mutation. This process is outlined in Algorithm 1.
The selection stage is the foundation of our evo- lutionary framework, ensuring the maintenance and progres- sion of population of candidate routes. In retrosynthesis planning, the success rate is commonly used to evaluate a route's quality. However, within the evolutionary frame- work, most current routes are unsuccessful. Therefore, we introduce a partial reward mechanism based on SC score (Coley et al., 2018) to assess these incomplete routes. Given a route, we traverse it from the first step sequentially to identify the first invalid step. The molecule set at this step. denoted M, is then used compute the reward for the route. The reward function F(-) )) is defined as follows, where C is the set of purchasable compounds:
Selection.
Initialization. In the INITIALIZATION function, we query the LLM to generate initial retrosynthesis routes for the target molecule. To enhance its predictions, we employ retrieval- -ugmeened genera-
F(MI)
Σ mcM
sc_score(m)
a molecular similarity- tion (RAG) approach, providing reference routes for the LLM. Specifically, we use the Morgan molecular finger- print with Tanimoto similarity to identify structurally simi- lar molecules in a database and retrieve their corresponding synthesis routes. We then provide the synthesis routes of the top three most similar molecules as references to the L.M.
The top routes, as ranked by SC score are selected as the population for the next round of evolution.
Mutation. To optimize the current route, we explore the flexibility of LLMs in synthetic route reproduction. Specif- ically. we enable the LLM to analyze and edit the cur- rent route through instructed to modify the existing route or propose an alter- native if deemed necessary, incorporating evaluation results from multiple perspectives as feedback. If the current route contains reaction-level errors, we retrieve reference routes from O, weighted by their products' Tanimoto similarity to the 'product' molecule in this step and provide them to the LLM. Additionally, for mutation queries, we retain the valid steps of the current route and provide the LLM with only the partial route starting from the first invalid step.
comppt-based mutation. The LLM is
Evaluation. We propose a three-level evaluation process to assess the quality of each step in the generated synthetic route: molecule level, reaction level, and route level, as shown in Table 1.
At the molecule level, we validate whether the molecules in the molecule set are both valid and purchasable. At the reaction level, we first perform reaction mapping to verify the reactions proposed by the LLM. This involves grounding and matching them against a reaction database. We begin by
5

---

LLMM-Augmented Chemical Synthesis and Design Decision Programs
3.4. Optimization for synthesizable molecular design
nism to capture non-local effects.
LLM-Syn-Planner can be easily extended design opti- mized molecular structures alongside their corresponding synthesis pathways. A simple approach is to first optimize molecular structure for the desired properties and then determine its synthesis pathway. As proof of concept, we propose Dyy-sseeees,,,,r, which integrates MoILEO (Wang et al., 2024) as the molecular structure optimizer, which leverages LLMs as genetic operators for molecular optimization. Specifically,, we ask the L.LM generate synthesizable molecule and analyze the synthetic route during the optimization process. To ensure synthesizabil- ity. we filter out molecules proposed by LLMs if their SC score exceeds 3.5 at each iteration of the optimization pro- cess. Additionally, in every round of evolutionary search, our framework acts as the synthesis pathway finder for the generated molecules. By combining these components, the integrated framework enables the end-to-end design of syn- thesizable molecules, harnessing the power of LLMs for both molecular optimization and synthesis planning.
Metries. For retrosynthesis planning tasks, we use the success rate as the evaluation metric. For synthesizable molecular design tasks, we measure performance using the top-1 expected property in the designed molecules.
Configuration '. We utilize GPT-4o 2 (Hurst et al., 2024) and DeepSeek- V3 (Guo et al., 2025) as our L.MMs and set the temperature to 0.7 for all queries, ensuring a balanced trade-off between creativity and reliability. To maintain efficiency, we impose maximum search time of 60 minutes per molecule.
4.2. Retrosynthesis Planning
We present the retrosynthesis planning results in Table 2. The LLMM-bsed approaches show clear distinction be- tween using LLMs as single-step predictors within search algorithm and leveraging them to generate complete ret- rosynthetic routes optimized via tree evolutionary algo- rithms (LLM-SYn-Plaamer)) When LLMs are integrated into MCTS or Retro*, their solve rates are significantly lower than those of traditional models, particularly on challenging datasets (e.g., Pistachio Hard, where solve rates are near zero). This suggests that current LLM-based single-step models struggle to produce high-quality reaction predic- tions, leading to suboptimal search performance. Moreover, increasing the number of model calls does not consistently improve results, especially on the USPTO-190 and Pista- chio datasets, highlighting intrinsic limitations in LLMs' single-step reaction prediction capabilities.
4. Experiments
4.1. Experimental Setup
Dataset. We conduct experiments using the USPTO (Schneider et al., 2016; Dai al., 2019) and Pistachio (pis) datasets. For USPTO, we utilize USPTO-190 (Chen et al., 2020) and a simplified subset, USPTO-EASY, which is ran- domly sampled from the test set used in Retro single-step model training. For the Pistachio dataset, we adopt the ver- sion from (Yu et al., 2024) but remove the starting material constraints. The route database is constructed using the training and validation sets from Retro*, while the reaction used in
In contrast, LLM--Sy--lanner performs remarkably well, achieving solve rates comparable to-or even ex- ceeding- single-step model-guided search. No- tably, LLM-SSn-Planner significantly outperforms LLM (MCTS/Retro*) indicating that optimizing full multi-step retrosynthetic routes rather than predicting step-by-step transformations enhances LLM effectiveness. While LLMs may not yet rival expert-designed single-step models in re- action prediction precision, they can generate promising retrosynthetic routes by using their long-term planning ca- pabilities. These findings suggest that the strength of LLMS can be leveraged by reformulating the problem as generating full retrosynthetic pathways that can be optimized through evolutionary techniques. This underscores a potential shift in focus from improving LLMs for single-step retrosyn- thesis to developing methods that exploit their generative capabilities for full-route planning combined with down- stream optimization strategies like EA.. We justify the cost of using LLMs in Appendix A.5 and show the case studies
database is a processed version of USPTO-Ful (Yu et al., 2024). For the building block set, we canonicalize all SMILES strings from the 23 million purchasable build- ing blocks available in eMolecules, following the approach of (Chen et al., 2020). We show the statistics of the datasets in Appendix A.I.
Baseline. We consider three single-step retrosynthesis models in combination with two search algorithms: MCTS (Segler et al., 2017) and Retro* (Chen et al., 2020). The single-step models are as follows: Graph2Edits (Zhong et al., 2023) is a template-free graph generative model that systematically edits the molecular graph of the target product generate valid reactant structures. RootAligned (Zhong et al., 2022) is another template- free approach that enforces a strict one-to-one mapping between product and reactant SMILES strings by aligning them to a shared root atom. LocalRetro (Chen& & Jung, 2021) is a template-basse method that employs local reaction templates involving atom and bond edits, coupled with global attention mecha-
'Our code is available at https: fttps://github.comm zoom- wang 1 12358/1.1. / 9GPT-40-2024-08-06
6

---

L.M--Augmented Chemical Synthesis and Design Decision Programs
USPTO Easy
USPTO-190
Pistachio Reachable
Pistachio Hard
Algorithm
Solve Rate (%)
Solve Rate (%)
Solve Rate (%)
Solve Rate (%)
N=100
300
500
N=100
300
500
N=100
300
500
N=100
300
500
Graph)Ediis(MCCS)
90.0
93.5
96.5
42.7
54.7
63.5
77.3
88.4
94.2
26.0
41.0
62.0
RootAligned(MCCS)
98.0
98.5
98.5
79.4
81.1
81.1
99.3
99.3
99.3
83.0
85.0
85.0
LocAIRetto(MCTS)
92.5
94.5
95.5
44.3
50.9
58.3
86.7
90.0
95.3
52.0
55.0
62.0
(raph)
92.0
95.5
97.5
51.1
59.4
80.0
94.0
95.0
97.5
71.0
74.0
82.0
ReetAl
99.0
99.0
99.0
86,8
88,9
88.9
98.7
98.7
98.7
78.0
82.0
82.0
LacalRetro(Retroo)
95.5
97.5
98.0
51.0
65.8
73.7
97.3
99.3
99.3
63.0
69.0
72.0
LLM(MCTS)
54.5
68.5
75.5
25.8
27.2
31.3
12.7
17.3
20.7
0.0
4.0
5.0
LLM(Rcctr*))
56,0
69.0
75.5
23.2
26.8
30.6
14.7
19.3
13.3
0.0
2.0
5.0
LLM-SSn-Planne (GPT)
91.0
99.5
100.0
64.7
91.1
92.1
93.3
98.0
98.0
72.0
86.0
87.0
L.LM-Syn-Planner (DS)
93.0
99.5
100.0
62.1
92.1
92.6
96.7
99.3
99.3
74.0
84.0
86.0
Table Summary of retrosynthesis planning performance across four datasets. The best model for each experiment setting is bolded and the top three are underlined. All runs were limited to 60 minutes per molecule. N denotes the model call limit. We denote LLM-SYn-Plannee using GPT-4o as LLM-Syn-Planne (GPT) and LLM-SSn-Plammnn using DeepSeck-V2. as SAMM-SSn-Plaanne (DS). t
The RootAligned model does not finish 300 model calls in 60 minutes due high computational cost.
of LLL--MMypp LLM--Syy-llanner in Appendix C.3.
ponentially growing complexity of the synthesis pathway, thus leading much higher success rates. Additionally, we introduce simple baseline named (Textual Extraction) to allow the L.LM to generate in an arbitrary format, followed by subsequent query to extract the route from the returned response. Surprisingly, this approach also yields decent performance, even with an unconstrained format.
4.3. Synthesizable Design
To evaluate the synthesizable design capability of LLM, we first consider common heuristic oracle functions rele- vant to bioactivity and drug discovery. We compare LLM- Syn-Designer with various molecular optimization methods, including Graph-GA (Jensen, 2019), REINVENT (Olive- crona et al., 2017), MolLEO (Wang et al.. 2024), and MARS (Xie et al., 2021), and present the results in Fig- ure 3. Notably, the baseline methods do not enforce synthe- sizability constraints, allowing them to explore broader chemical space and achieve higher scores, albeit with non- synthesizable molecules. The results demonstrate that -LM-Syn-Desiiner effectively balances optimization effi- ciency and synthesizability. In all cases, the best molecules identified by LLMM-Sy-Dessigee superior fitness compared to traditional algorithms and MolLEO,, while ensuring synthesizability. Specifically, for the isomers_ isomers_CoHIonn C9H10N202PF2C1 target, SLLM-SSn-Designne achieves comparable higher scaled fitness values with fewer oracle calls than all other methods. This suggests that integrating synthesizability constraints within the optimiza- tion process does not necessarily compromise efficiency.
Observation 2: Even rough intermediate feedback can be significantly useful for LLMs. To isolate the contri- bution of the Molecule-RAG module in our retrosynthesis planner, we perform two ablations: (i) removing RAG en- tirely and (ii) substituting the retrieved routes with random routes in the INITIALI ZATIONITOON and MUTATION prompts. Surprisingly, random routes, though not related to the syn- thesis of the target molecule, still significantly increase per- formance, indicating that they serve generic in-context exemplars for the LLM. When we employ RAG with routes retrieved via ment even larger. This holds true even though Morgan fingerprints do not directly encode synthetic feasibility and structurally close analogues not always present in the database. These findings demonstrate that LLMs can ex- tract value from rough, intermediate feedback, and that a lightweight RAG component can markedly enhance ret- rosynthetic planning quality.
exhibit competitive or
similarity, the improve-
4.4. Ablation study
Observation 3: Partial reward is crucial for long-horizon sequential decision- sparse as it only evaluates if the entire synthesis pathway is valid. We validate the importance of partial reward by intro- ducing a simple synthesis accessibility evaluator (SC score). With partial reward, the success rate improves considerably across both datasets.
We conducted several ablation studies to evaluate different design choices: route formats, the use of molecule RAG, reward signals, EA parameters, and prompt robustness. The results are shown in Table 3.
The target reward is very
Observation 1: The linear format of synthesis steps sig- nificantly outperforms the tree format. We investigate the influence of route format in Table 3. The results suggest that linear storage of decision steps better reduces the ex-
Observation 4: Reinforcing explainability helps improve LLM performances. We further examined the impac
7

---

LLM L.M--Augmeneed Chemical Synthesis and Design Decision Programs
6SK38
Isomers C9H10N202PF2CI
JNK3
Scaled fitness
2003
1000
5000
8000
12000
2000
4:200
5000
D0D0
10000
2003
4000
6000
8000
10000
Number of oracle calls
Number of oracle calls
Number of oracle calls
Graph GA
Mars
LLM-Syn-Designer
MolLEO
REINVENT
Convergence reached
Figure 3. Fitness score of the best molecule found by each molecule optimization method. Only LLM- LLM-Syn-Designee (GPT) here ensures the syuthesizability of found molecule.
of prompt design. Incorporating an explicit explanation section in the prompt consistently enhanced performance, indicating that exposing the model's intermediate reasoning steps helps the LLM arrive at accurate decisions.
Setting
Variant
USPTO Easy
USPTO-190
Textual + Extraction
84.5
Route Format
Tree Sequential
65.5
13.1
91.0
64.7
w/o example routes
51.0
20.0
5. Related Work
Retrieval
w/ random routes
84.0
52.1
w/ retrieved routes
91.0
64.7
5.1. ML-based Single-step Retrosynthesis Models
w/ only final reward
63.5
15.3
Reward
w/ partial reward
91.0
64.7
Single-step retrosynthesis models predict the outcome a single reaction step, i... given an input molecule, how can decomposed and into which constituents? Early works directly predicted precursors by seq-to-seq translation on SMILES (Weininger, 1988; Liu al., 2017) using finger- prints (Segler & Waller, 2017; Coley et al., 2017; Fortunato et al., 2020). More recently, single-step retrosynthesis mod- els have employed transformers (Vaswani, 2017) and graph neural networks (GNNs). Methods can be broadly catego- rized into template-basees femplate-free, or semi-template methods. Template-based methods use pre-defined chem- ical rules which can be advantageous if they defined with high granularity (Szymkuć et al., 2016; Grzybowski et al., 2018; Segler & Waller, 2017; Dai et al., 2019; Ishida et al., 2019; Seidl et al., 2022; Chen & Jung, 2021; ;; Xie et al., 2023). Template-fie from data and learn a translation (Liu et al., 2017: Zheng al., 2019; Schwaller al., 2020; Zhong et al., 2022). Fi- nally, semi-template methods make intermediate predictions (such synthons) and then predict the precursors based these (Shi et al., 2020; Somnath et al., 2021; Sacha et al., 2021; Zhong et al., 2023).
w/o <Fxplanation>>
79.5
Prompt
w/o 'Rational'
88.5
62.6
Full
91.0
64.7
Table 3. Ablation study of LLM-SSy-PPaanee across different de- sign choices: route format, use of molecule RAG, reward signal, EA parameters and robustness the prompt. We GPT-4o as the LLM and all the experiments are conducted under N 100.
Planning with Dual Value Networks (PDVN) (Liu al., 2023), and recent double-ended search algorithm (Yu et al., 2024). Since retrosynthesis has broad applicabil- ity for molecular discovery, many retrosynthesis platforms exist, encompassing industrial (Szymkué et al., 2016; Grzy- bowski et al., 2018; Genheden et al., 2020; Saigiridharan et al., 2024; Watson et al., 2019; Molecule. one; Schwaller et al., 2020) and open-source (Genheden al., 2020: Sai- giridharan et al., 2024: Coley al., 2019: Tu et al., 2025) solutions. Very recently, works have investigated applying LLMs for retrosynthesis through fine-tuning (Nguyen- Van et al., 2024), instruction-tuning (Yang et al., 2024), plat- form assistants (Zhang et al., 2025), experimental planning agents (Liu al., 2024b), and integration with knowledge graphs for synthesis planning of polymers (Ma et al., 2025).
methods attempt to learn these rules
5.2. Search-directed Retrosynthesis Planning
By coupling search algorithm with single-step retrosyn- thesis models, multi-step retrosynthesis can be performed. Exemplary works include applying Monte Carlo tree search (MCTS) (Segler et al., 2018), R (Chen et al., 2020),
6. Conclusion
In this paper, we studied the retrosynthesis problem with LLMs. Specifically, we experimented with using LLMs as
8

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
single-step reaction prediction models with a search algo- rithm and found LLMs significantly underperformed spe- cialized reaction models. To improve this, we proposed to sample entire multi-step synthetic pathways and introduced an evolutionary process to optimize them. To scale this approach, we leveraged a linear format to store reaction steps and designed partial rewards with retrieved reaction sub-trajectories. In the end, we bridged the performance gap and matched the SOTA performance in retrosynthesis planning. In addition, we demonstrated LLMs can be easily adapted to the synthesizable molecular design problem to find propeizy-optimized molecules that are synthesizable.
Boström, J., Brown, D. G., Young, R. J., and Keserü, G. M. Expanding the medicinal chemistry synthetic toolbox. Nature Reviews Drug Discovery, 17(10):00-727, 2018.
Bradshaw, J., Paige, B., Kusner, M. J., Segler, M., and Hemández--Loatte J. M. A model to search for syn- thesizable molecules. Advances in Neural Information Processing Systems, 32, 2019.
Bradshaw, J., Paige, B., Kusner, M. J., Segler, M., and Hrrnández-Lobato J. M. Barking up the right tree: an ap- proach to search over molecule synthesis dags. Advances in neural information processing systems, 33:6852-6866. 2020.
Limitation and future work: Despite promising results, we observed that LLMs suffered significantly with sparse rewards (e.g. in the shooting setup) while improved signif- icantly with partial rewards and retrieved sub-trajectories. is worth studying how to incorporate a search algorithm into our framework when LLMs struggle to generate any synthesis paths with the desired target molecule. For future work, it is promising to study more flexible design criteria enabled by LLMs such as material-constrained synthesis planning (Guo & Schwaller, 2024b).
CAS. Cas scifinder.
Chang, L., Zheng, W. Yao, H., and Wei, Y. How well can Ilms synthesize molecules? Ilm-powered framework for multi-step retrosynthesis.
Chen, B., Li, C., Dai, H., and Song. L. Retro": learning retrosynthetic planning with neural guided a* search. In International conference on machine learning, pp. 1608- 1616. PMLR, 2020.
Acknowledgments
Chen, S. and Jung, Y. Deep retrosynthetic reaction predic- tion using local reactivity and global attention. JACS Au, 1(10):6112-1600 2021.
We thank Wenhao Gao for helpful discussions retrosyn- thesis planning. We thank the anonymous reviewers for their valuable feedback, in particular suggestions to conduct cases studies for recently commercialized molecules.
Coley, C. W., Rogers, L.,, Green, W. H., and Jensen, K. F. Computer-assisted retrosynthesis based on molecular sim- ilarity. ACS central science, 3(12): 1237 2017.
Impact Statement
Coley, C. W., Rogers, L.,, Green, W. H., and Jensen, K. F. Sescore: synthetic complexity learned from reaction corpus. Journal of chemical information and modeling, 58(2):252-28 261. 2018.
This work investigates how LLMs can be used for retrosyn- thesis planning and synthesizable molecule design. Both use-cases are applicable to therapeutics and materials design. No molecules were synthesized and experimentally tested so there are no specific societal consequences we feel should be highlighted. However, in the future, the framework, if properly experimentally validated, could have positive soci- etal benefits.
Coley, C. W., Thomas III. D. A., Lummiss, J. A., Jaworski, J. N., Breen, C. P., Schultz, V., Hart, T., Fishman, J. S., Rogers, L., Gao, H., al. A robotic platform for flow synthesis of organic compounds informed by ai planning. Science, 365(6453)xcaax 1566, 2019.
Corey, E. J. and Wipke, W. T. Computer-assion assisted design of complex organic syntheses: Pathways for molecular synthesis can be devised with computer and equipment for graphical communication. Science, 166(3902):178; 192, 1969.
References
Pistachio, january 2024.
URL https://www. nextmovesoftware. www.waaw...ooeaaaaoooooooooooooe wtt.aarr..oooppitcccticccc pistachio.htm
Blakemore, D. C., Castro, L., Churcher, I., Rees, D. C., Thomas, A. W., Wilson, D. M., and Wood, A. Organic synthesis provides opportunities transform drug dis- covery. Nature chemistry, 10(4):383-394 2018
Corey, F. J., Long, A. K., and Rubenstein, S. Computer- assisted analysis in organic synthesis. Science, 228(4698): 408 018-418, 1985.
Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. Au- tonomous chemical research with large language models. Nature, 624(7992):570-57) 2023.
Cretu, M., Harris, C., Roy, J., Bengio, E.,, and Liò, P. Syn- flownet: Towards molecule design with guaranteed syn- thesis pathways. arXiv preprint arXiv:2405.00 155, 2024.

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
Dai, H., Li, C., Coley, C., Dai, B., and Song, L. Retrosyn- thesis prediction with conditional graph logic network. Advances in Neural Information Processing Systems, 32, 2019.
Horwood, J. and Noutahi, E. Molecular design in syntheti- cally accessible chemical space via deep reinforcement learning. ACS omega, (111))32994439999 2020.
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y.. Tang, R., and Chen, E. Understanding the planning of Ilm agents: survey. arXiv preprint arXiv:2402.027111 2024.
Daylight Chemical Information Systems, I. 4. smarts language for describing molecular patterns.
Fortunato, M. E., Coley, C. W., Barnes, B. C., and Jensen, K. F. Data augmentation and pretraining for template- based retrosynthetic prediction in computer-aided syn- thesis planning. Journal of chemical information and modeling, 60(7):3398-3407, 2020.
Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Igashov, I., Schneuing, A., Segler, M., Bronstein, M. M., and Correia, B. Retrobridge: Modeling retrosynthesis with markov bridges. In The Twelfth International Conference on Learning Representations.
Gabrielson, S. W. Scifinder. Journal of the Medical Library Association: JMLA, 106(4):588, 2018.
Gao, W., Mercado, R., and Coley, C. W. Amortized tree generation for bottom-up synthesis planning and syn- thesizable molecular design. Proc. 10th International Conference on Learning Representations, 2022.
Ishida, S., Terayama, K., Kojima, R., Takasu, K., and Okuno, Y. Prediction and interpretable visualization of retrosyn- thetic reactions using graph convolutional networks. Jour- nal of chemical information and modeling, 59((2))5026- 5033, 2019.
Gao, W., Luo, S., and Coley, C. W. Generative artificial intelligence for navigating synthesizable chemical space. arXiv preprint ar arXi::2410.0394. 2024.
Jensen, J. H. A graph-based genetic algorithm and gener- ative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):35673352 2019.
Genheden, S., Thakkar, A., Chadimová, V., Reymond, J.-L.,, Engkvist, O., and Bjerrum, E. Aizynthfinder: fast, ro- bust and flexible open-source software for retrosynthetic planning. Journal of cheminformatiis, 12(1):70, 2020.
Jocys, Z. Willems, H. M., and Farrahi, K. Synth- former: Equivariant pharmacophore- generation of molecules for ligand- drug design. arXiv preprint 140:2:2410.02718, 2024.
Gottipati, S. K., Sattarov, B., Niu, S., Pathak, Y., Wei, H., Liu, S., Blackburn, S., Thomas, K., Coley, C., Tang, J., et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In Interna- tional conference on machine learning, pp. 3668-3679 PMLR, 2020.
Korovina, K., Xu, S., Kandasamy, K., Neiswanger, W., Poc- 2os, B., Schneider, J., and Xing. E. Chembo: Bayesian op- timization of small organic molecules with synthesizable recommendations. In International Conference on Artifi- cial Intelligence and Statistics, pp. 3393-3403. PMLR, 2020.
Grzybowski, B. A., Szymkuć, S., Gajewska, E. P., Molga, K., Dittwald, P., Wolos, A., and Klucznik, T. Chemat- ica: story of computer code that started to think like chemist. Chem, 4(3):390-398) -9899999999111 2018.
Koziarski, M., Rekesh, A., Shevchuk, D., van der Sloot, A., Gaiński, P., Bengio, Y., Liu, C.-H., Tyers, M., and Batey, R. A. Rgfn: Synthesizable molecular generation using gflownets. arXiv preprint wrXiv:2406.08550 2024.
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma. S., Wang, P., Bi, X., et al. Deepseck-rl: In- centivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXivv2501.129M 2025.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Guo, J. and Schwaller, P. Directly optimizing for synthesiz- ability in generative molecular design using retrosynthe- sis models. arXiv preprint arXiv:2407. 12 186, 2024a.
Liu, B., Ramsundar, B., Kawthekar, P., Shi, J., Gomes, J., Luu Nguyen. Q., Ho. S., Sloane, J., Wender, P., and Pande, V. Retrosynthetic reaction prediction using neural sequence-to-sqqeene sequence models. ACS central science, (10)1103-1113,, 2017.
Guo, J. and Schwaller, P. It takes two to tango: Directly optimizing for constrained synthesizability in genera- tive molecular design. arXiv preprint arXiv:2410.11527, 2024b.
10

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
Schneider, N., Stiefl, N., and Landrum, G. A. What's what: Liu, G., Xue, D., Xie, S., Xia, Y., Tripp, A., Maziarz, K., Segler, M., Qin, T., Zhang, Z.,, and Liu, T.-Y. Ret- rosynthetic planning with dual value networks. In Inter- national Conference on Machine Learning, pp. 22266- 22276. PMLR, 2023.
The (nearly) definitive guide reaction role assignment. Journal of chemical information and modeling, 56(1)) 2336 2346, 2016.
Schwaller, P., Petraglia, R., Zullo, V., Nair, V. H., Haeusel- mann, R. A., Pisoni, R., Bekas, C., Iuliano, A., and Laino, T. Predicting retrosynthetic pathways using transformer- based models and a ayper-graph exploration strategy. Chemical science, 11(12)3331-3322 2020.
Liu, G., Sun, M., Matusik, W., Jiang, M., and Chen, J. Multimodal large language models for inverse molecu- lar design with retrosynthetic planning. arXiv preprint vrXiv:2410.0422 2024a.
Liu, X., Chiu, S., and Zhao, H. Autonomous faithful ret- rosynthesis with large language models: From synthesis planning experimental procedures. In 2024 AIChE Annual Meeting, AIChE, 2024b.
Sclar, M., Choi, Y., Tsvetkov, Y., and Suhr, A. Quanti- fying language models' sensitivity to spurious features in prompt design or: How learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations, 2024.
Luo, S., Gao, W., Wu, Z., Peng, J., Coley. C. W., and Ma, J. Projecting molecules into synthesizable chemical spaces. Proc. s1 International Conference Machine Learning, 2024.
Segler, M. H. and Waller, M. P. Neural-symbolic ma- chine learning for retrosynthesis and reaction predic- tion. Chemistry-A European Journal, 23(25):596-5971, 2017.
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., and Schwaller, P. Augmenting large language mod- els with chemistry tools. Nature Machine Intelligence, pp. 1-11, 2024.
Segler, M. H., Preuss, M., and Waller, M. P. Learning to plan chemical syntheses. arXiv preprint arXiv: 777.004222. 2017.
Ma, Q. Zhou, Y., and J. Leveraging large language mod- els as knowledge-driver agents for reliable retrosynthesis planning. Xrxiv preprint arXiv:2501.0880) 2025.
Segler, M. H., Preuss, M., and Waller, M. P. Planning chem- ical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604-611 2018
Maziarz, K., Tripp, A., Liu, G., Stanley, M., Xie, S., Gaiński, P., Seidl, P., and Segler, M. Re-evaluating ret- rosynthesis algorithms with syntheseus. arXiv preprint arXiv:2310.19796. 2023.
Seidl, P., Renz, P., Dyubankova, N., Neves, P., Verhoeven, J., Wegner, K., Segler, M., Hochreiter, S., and Klam- bauer, G. Improving few-and zero-shov shot reaction template prediction using modern hopfield networks. Journal of chemical information and modeling, 62(9):2111-2120, 2022.
Molecule.one. The ml platform.
Nguyen- Van, P., Thanh, L. N., Manh, H. H., Thi, H. Nguyen, T., and Nguyen, V. A. Adapting language models for retrosynthesis prediction. 2024.
Seo, S., Kim, M., Shen, T., Ester, M., Park, J., Ahn, S., and Kim. W. Y. Generative flows on synthetic pathway for drug design. arXiv preprint arXiv:2410.04542, 2024.
Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9:1-14, 2017.
Shi, C., Xu, M., Guo, H., Zhang, M., and Tang. J. A graph to graphs framework for retrosynthesis prediction. In International conference on machine learning, pp. 8818- 8827. PMLR, 2020.
Sacha, M., Blaz, M., Byrski, P., Dabrowski Sabbrowsk-Tuumnskii P., Chrominski, M., Loska, R., Sondaaccyy-Pruszyynee P., and Jastrzebski, S. Molecule edit graph attention network: modeling chemical reactions sequences of graph edits. Journal of Chemical Information and Modeling, 61)) 3273-3284, 2021.
Somnath, V. R., Bunne, C., Coley, C., Krause, A., and Barzilay, R. Learning graph models for retrosynthesis prediction. Advances in Neural Information Processing Systems,
2021.
Song, C. H., Wu, J., Washington, C.,, Sadler, B. M., Chao, W.-L., and Su, Y. Lim-planner Few-shot grounded plan- ning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2998-3009, 2023.
Saigiridharan, L., Hassen, A. K., Lai, H., Torren-Peraire, P., Engkvist, O., and Genheden, S. Aizynthfinder 4.0: de- velopments based on learnings from years of industrial application. Journal of Cheminformatics, 16(1):57, 2024.

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
Swanson, K., Liu, G., Catacutan, D. B., Amold, A., Zou, J., and Stokes, J. M. Generative ai for designing and validat- ing easily synthesizable and structurally novel antibiotics. Nature Machine Intelligence, 6(3):338-355) --3388333,, 2024.
Zhang, C., Lin, Q., Zhu, B., Yang, H., Lian, X., Deng, H., Zheng, J., and Liao, K. Synask: unleashing the power of large language models in organic synthesis. Chemical Science, 16(1):43 2025.
Szymkué, S., Gajewska, E. P., Klucznik, T., Molga, K., Dittwald, P., Startek, M., Bajezyk, M., and Grzybowski, B. A. Computer- Computer-assisted assisted synthetic planning: the end of the beginning. Angewandte Chemie International Edition, 55(20):5904 5937, 2016.
Zhao, Z., Lee, W. S., and Hsu, D. Large language models as commonsense knowledge for large-scale task planning. Advances in Neural Information Processing Systems, 36, 2024.
Zheng, S., Rao, J., Zhang, Z., Xu, J., and Yang, Y. Predicting retrosynthetic reactions using self-corrected transformer neural networks. Journal of chemical information and modeling, 60(1):47-5) 2019.
Tu. Z., Choure, S. J., Fong, M. H., Roh, J., Levin, L,, Yu, K., Joung, F., Morgan, N., Li, S.-C., Sun, X., et al. Askcos: open source software suite for synthesis plan- ning. arXiv preprint arXiv:2501.01835 2025.
Zhong, W., Yang. Z., and Chen, C. Y.C.. Retrosynthesis pre- diction using an end-to-end graph generative architecture for molecular graph editing. Nature Communicationss 14 (1):3009, 2023.
Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Wang, H., Skreta, M., Ser, C.-T., Gao, W., Kong, L., Strieth- Kalthoff, F., Duan, C., Zhuang, Y., Yu, Y., Zhu, Y., al. Efficient evolutionary search over chemical space with large language models. arXiv preprint arXiv:2406. 16976, 2024.
Zhong. Z., Song, J., Feng. Z., Liu, T., Jia, L., Yao, S., Wu, M., Hou, T., and Song, M. Root-aligned smiles: a tight representation for chemical reaction prediction. Chemical Science, 13(31):902339034 2022.
Watson, I. A., Wang, J., and Nicolaou, C. A. A retrosyn- thetic analysis algorithm implementation. Journal of cheminformatics, 11:1-12 2019.
Zhong, Z., Song. J., Feng, Z., Liu, T., Jia, L.. Yao, S., Hou, T., and Song, M. Recent advances in deep learning for retrosynthesis. Wiley Interdisciplinary Reviews: Compu- lational Molecular Science, 14(1)=e1694, 2024.
Wei. J., Wang, X., Schuurmans, D... Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thie -o-thhgghhttttttthh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2488 2022.
Zhuang, Y., Chen, X., Yu, T., Mitra, S., Bursztyn, V., Rossi, A., Sarkhel, S., and Zhang, C. Toolchain Efficient action space navigation in large language models with a search, arXiv preprint arXiv: 2300v22310.133 13227, 2023.
Weininger, D. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1),31-36, 1988.
Xie, S., Yan, R., Guo, J., Xia, Y., Wu, L., and Qin, T. Ret- rosynthesis prediction with local template retrieval. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pp. 5330 5338,, 2023.
Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L. Mars: Markov molecular sampling for multi- objective drug discovery. In Proc. 9th International Con- ference Learning Representations, 2021.
Yang, Y., Shi, R., Li, Z., Jiang, S., Lu, B.-L., Yang. Y., and
Zhao, H. Batgpt-chem: A foundation large model for ret- rosynthesis prediction. arXiv preprint arXiv:2408. 10285, 2024.
Yu, K., Roh, J., Li, Z., Gao, W., Wang, R., and Coley, C. W. Double-ended synthesis planning with goal-constrained bidirectional search. arXiv preprint arXiv:2407.06334 2024.
12

---

L.M--Aummetedd Chemical Synthesis and Design Decision Programs
Appendix for LLM-Augmented Chemical Synthesis and Design Decision Programs
A. Extended Descriptions
A.1. Dataset Statistics
We show the dataset statistics Table
Name
No. of Routes
Avg. Route Length
Avg. SA score
Avg. SC score
USPTO Easy
200
3.7
2.8
3.8
USPTO-199 190
190
6.7
3.6
4.0
Pistachio Reachable
150
5.5
3.1
3.9
Pistachio Hard
100
7.5
3.6
3.9
Table 4. Statistics of the dataset used in the experiments.
A.2. MCTS for Retrosynthesis Planning
The single-step model predicts potential sets of reactants for a given product, transforming a target molecule into plausible precursors. However, multiple steps may needed to reach commercially available or easily synthesized materials. This why the single-step reaction model integrated with MCTS: systematically explores these multi-step routes, pruning unlikely paths while focusing the most promising transformations. By striking a balance between exploration and exploitation, MCTS avoids getting stuck in unproductive branches and can uncover synthetic routes that might not be obvious through manual inspection alone.
Under the MCTS procedure, the target molecule is defined as the root node of a search tree, and each edge represents a single-step retrosynthetic transformation predicted by the reaction model. A policy network can be used to rank or filter the most promising disconnection suggestions each step, while a value function provides an estimate of how likely a given partial route is succeed in the long run. The algorithm selects which node to expand next using an Upper Confidence Bound (UCB), which balances the value estimate (exploitation) with the uncertainty in that estimate (exploration). A reward function then quantifies the outcome of each known starting materials. These reward signals are backpropagated to update the value estimates of each node. Finally, iterating selection, expansion, simulation, and backpropagation until we reach termination condition (time limit, enough solutions found).
iiaaasion-ffeer based reaction feasibility, synthetic cost, reaching
A.3. Retro* Algorithm
Retro (Chen et al., 2020) integrates neural networks with a best-first search strategy to solve retrosynthesis problems. It models the problem as an AND-OR tree, where "AND" nodes represent reactions and "OR" nodes correspond to molecules. neural network, trained on prior retrosynthesis experiences, estimates the cost cach node. Using a best-first search, the algorithm prioritizes the most promising pathways based on these predictions. then applies single-step model to expand the selected node, generating an AND-OR subtree. Finally, it updates the pathway costs guide the next selection step.
A.4. Additional Experimental details
For single-step models, we use the checkpoints from syntheseus 3, In the MCTS algorithm, we employ a basic reward function: a state receives a reward of 1.0 if all molecules purchasable (i.e., the state is solved), and 0.0 otherwise. The
13

---

L.M-Augmented Chemical Synthesis and Design Decision Programs
value function is set as constant 0.5. For policy, by temperature of 3.0 and normalized across the total number of reactions.
softmax values derived from the single-step reaction model, scaled
In the Retro algorithm, we follow the retro*-0 variant described in the original paper (Chen et al., 2020). The OrNode cost function assigns a cost of 0 to purchasable molecules and infinity otherwise. The AndNode cost function defines the reaction cost as -log(softmax) of the reaction model output, thresholded at a minimum value. For the search heuristic (value function), we use a constant value of O, consistent with the retro*-0 algorithm.
A.5. Cost of using LLMs
Indeed, large L.MMs such as GPT-4 currently require more computing than traditional models like RootedAligned. However, our motivation to rigorously examine what LLMs uniquely offer in complex, high-level scientific reasoning tasks like multi-step retrosynthesis planning setting where -omein- models often require extensive retraining and are limited in adaptability.
Our results demonstrate that even without any fine-tuning, LLM-Syn-Plannee matches or outperforms specialized models across multiple datasets. This zero-shot capability highlights crucial point: LLMs offer general-purpose purpose reasoning and chemical udaptability out-of-the-bue box, which cannot be achieved by most lightweight models without costly re-engineering when reaction databases or design goals change.
Furthermore, while cost is a valid concern, we believe must be evaluated context:
The retraining and maintenance overhead for specialized models is non-trivial in dynamic research environments.
Our LLM-based system can immediately leverage new knowledge via retrieval, without retraining.
LLM inference costs are rapidly decreasing as optimized deployment (e.g., quantization, distillation, smaller expert LLMs) becomes mainstream.
As open-source L.MMs improve, smaller models will become more capable (this can also be achieved via distillation). These small models can be hosted locally, thus also not requiring large clusters host (for example, using Ollama to host DeepSeek R1).
Lastly, LLM-Syn- Syn-Planner represents a first step toward broader vision of flexible, generalist AI for scientific discov- somethig static models cannot enable. While not yet universally cost-effective, we argue that the emerging capabilities and flexibility of LLMs justify this early-stage investigation.
A.6. Computational Resources
Our experiments utilized the GPT-4o model and the DeepSeek- V3 model. The GPT-4o model refers to the GPT-4o checkpoint from 2024-08-06 4, All GPT-4o checkpoints were hosted on Microsoft Azure
B. Extended Related Work
B.1. Synthesizable Molecular Design
Synthesizable molecular design aims to generate molecules (with optimal properties) that are also synthesizable, as predicted by a retrosynthesis model. While retrosynthesis methods are often described as "top-down" because they decompose a target molecule into purchasable precursors, the most common methods in literature for synthesizable molecular design proceeds "bottom-up", which combine building blocks to construct the final molecule. Therefore, instead predicting the resulting precursors from an input molecule, "bottom-up" approaches require way to predict the product molecule given precursors. To this end, existing approaches either use forward synthesis prediction models (Bradshaw et al., 2019; 2020) define set of templates which dictate how building blocks can be combined (Gao et al., 2022; 2024; Luo et al., 2024; Koziarski et al., 2024; Cretu et al., 2024; Seo et al., 2024; Swanson et al., 2024; Jocys et al., 2024). These methods can be broadly classified as synthesizabilltyycoostaaaa generative models. An alternative approach is to couple retrosynthesis models directly into
* ooppnoii com
14

---

L.MM-Augmented Chemical Synthesis and Design Decision Programs
Method
LLM
USPTO Easy
Pistachio Hard
GPT-4o
4.0
0.0
Direct query
DeepSeek- V3
4.5
1.0
GPT-4o
91.0
72.0
MMMM-SynPPlanner
DeepSeek- V3
93.0
74.0
Table Ablation studies of retrosynthesis planning based direct queries. We report the Solve rate under N 100.
the optimization loop of generative models, such that synthesizability is optimized for, rather than enforced in the generation process (Guo Schwaller, 2024a;b).
C. Extended Experiment Results
C.1. Performance of direct user queries for multi-step retrosynthesis tasks
In Table 5, we show the performance of directly querying LLMs with target molecules. Each LLM was queried 100 times to ensure fair comparison. On the USPTO easy dataset, both LLMs solved fewer than 10 routes, and on the Pistachio Hard dataset, they failed in nearly all cases. These results suggest the models do not merely "remember" synthetic routes from their training data.
C.2. Performance of advanced reasoning LLM (DeepSeek-Ri)
LLM
USPTO Easy
USPTO-190
GPT-4o
91.0
64.7
DeepSeck- V3
93.0
62.1
DeepSeek-RL
95.0
57.9
Table Ablation studies Sa--ayyy-l using DeepSeek-R1, We report the Solve rate under 100.
We conduct ablation studies on advanced reasoning LLM DeepSeek-RR R1 (Guo et al., 2025) and show the results Table Interestingly, although DeepSeek-R1 is designed as reasoning model, it performs worse than DeepSeek V3. Moreover, because DeepSeek-RR R1 includes its thinking process in the output, its overall cost is roughly three times higher than that of DeepSeek-V3.
C.3. Case Study
15

---

LLM-Augmeeted Chemical Synthesis and Design Decision Programs
Salmeterol
31-550-CAS-17125655
Benzyl deprotection
OH
OH
SN2
alman
Benzyl protection error Toluene (left) should contain a halogen, such as bromine. There are also selectivity problems as both alcohols could be protected
Figure 4, Proposed synthesis route to Salmeterol, Available building blocks are highlighted in yellow.
16

---

L..M-Augmented Chemical Synthesis and Design Decision Programs
31-398-CAS-618448B (near match), missing a protection group at highlighted position
5H6
N
NHz
31-172-CAS-6234755 (near match)
missing several
protectinnccappng groups
The synthesis route will suffer from selectivity problems but this is limitation of the templates
11-084-CAS-132366977 (near match)
missing several
protection/capping groups
Bia
Figure 5. Proposed synthesis route to SH6. Available building blocks are highlighted in yellow.
17

---

LLM--Agmmeted Chemical Synthesis and Design Decision Programs
DDR1 00537
Acyl chloride coupling
HN
NH2
Nucleophilic aromatic substitution may be possible even though hydroxyl is poor leaving group, the pyrrolopyridine ring can stabilize the negative charge
H&N
NH
There are CAS matches but the reagents are dissimilar such that we do not want to cite any reference This is possible reaction but selectivity poor. Self coupling is possible
H&N
OH
31-371-CAS-15413531
Figure 6. Proposed synthesis route to a DDRI inhibitor. Available building blocks are highlighted in yellow.
18

---

L.MM-Augmentdd Chemical Synthesis and Design Decision Programs
Lenalidomide
31-522CCAS-1296838
NH
NH2
31-031-CAS-21505591
N
Problematic step
Figure 7. Proposed synthesis route to Lenalidomode. Available building blocks are highlighted in yellow.
We apply LLM--Sn-PPanncee to propose synthetic routes four bio-active molccules: Salmeterol (Figure 4), 5H6 (Figure 5), DDR1 inhibitor (Figure 6), and Lenalidomide (Figure 7). For all molecules, synthetic route combining the fixed templates and building blocks stock is successfully proposed. To assess the feasibility of the proposed reaction sequences, we look for literature precedent using SciFinder (Gabrielson, 2018; CAS) and annotate the CAS number for matched reaction steps. For reaction steps without literature reference. we propose a plausible reaction transformation. Lastly, and most importantly, we highlight reactivity and selectivity problems across all routes for transparency. Since LLM-Syn-Plann Planner uses fixed template set, proposed synthetic routes inherit the limitations of the templates. Consequently, for all synthetic routes, there is least instance of reactivity problem which would likely involve modifying the route if it were be performed in the lab. With improved templates, the chemical reliability of LLM--Sn-Plannee will improve.
D. Prompts
show prompts of INITIALIZATION and MUTATION for LLM-SSn-Planner, And LLM operators prompt for LLM-Syn-Designer.
LLM-Syn Syn--Snn--- Syn-Planner INITIALIZATION prompts
As professionsl. chenist specialized in synthesis analysis, you are tasked wich generating retrosyrthesis reute target molecule provided
format.
A retzosyntheess route is series of retrosyn:heeia steps that stazte from the Larget molecule and ends with some commercially purchesable compcunds. The reactions from the USPTO dataset. Please alse
19

---

L..M-Augmented Chemical Synthesis and Design Decision Programs
Top from LLM-syn-designer, JNK3 = 0.96
NH.
OH
Figure Top molecule of jnk3 found by L.M.-Syn-Designer
corsider reactions stercochamistrry.
The route should be a 1ist of steps wzapped in <ROUTE></BOUTE with EXPLANATTORNDDDPLAAATTTOO after it. Each step in the list should be dictionary. You need to melecules we nead to synthesiza or purchase. In each step, you need to select a molecule from the 'Molecule set'
molecule which consists of the
product molecule the backward reaction in this step, you need to remove the product molecule from the molecule set and add reactants you in tala step. In the next melacule set, than the starting molecule set should be the "Updated molecule set' from the previous step. the last step, all the melecules in the "Ugdated molecule set! should be parchasable. Here example:
step
backward reaction find
reactente.
taking
this updated set as
'Cpdated
see!
<ROUTE>
'Kolecule set':
"[Tazget Mclecule]",
'kationsl's : step analysis, 'Praduct"; [Product molecule]", 'Reaction'; "[Reaction templäte]",
'Reactants's [Reactenti,, Reactant2]
'Uppdated molecsle set "[Reactant1, Reactant2]*
'Nolecule set': "[Resctantl, Seactant2]",
Rat tonaloall Step analysta,, 'Produet"; "[Product molecule]", 'Reection'; "(Reaction template)", 'Reactanta': "[subResctanti, subFeactant2]",
1pdattdd molecule set": "[Resctantl, subResctantt, aubReactant2)"
</ROUTE>
<EXPLANATION:: Explanation fer the whole route. </EXPLANATIONAA)
Requizementa:
The "Molecule set' contains molecules we need to syntherize at this stage. In the first step, it should be the target molecule. in the following steps, it should be the rupdated molecule set! from the previous step.
each step should your analyais for aynthesis planning be in the string format wrapped with molecule plan synthesize
The 'Rational""
step. ahould
'Molecule set'. The molecule should be molecule from the 'Molecule set" in List. The molecule amilea should be wrapped with
step. should
'seaction' is reaction that synchesize the product molecule. should reactior tamplate ahould be in SMTLRS format. For exampla, [ProdustsReactenttl Rescctttt))
list. The molecule amiles should
'Reactants"
reaction. It should
wrapped with
The 'updated molscule should molecules need purchase synchesize taking seaction. To the 'Updated molecule set', you need remove the product molecule from the 'oleecue
20

---

LLMM-Augmented Chemical Synthesis and Design Decision Programs
set' and ther add the reaciants In this step into it. In the Last atep, all the molecules in the "Updated molecule setr should be purchasable. 7. in the <EMPLABAT15E>, you should analyze the whele route and the molecules in the "Uddated malecule sat in the lest step are all purchanable. My target molecule (Target Molecule)
To asstst you, example retrosyntheata rouses that are either close to the target melecule represent will provided.
<ROUTE>
Retrieved route here
</ROUTE>
Please propose retrosyntheale route for my target molecule. The provided reference routes may be helpful. You also design syathetis route based
knowledge.
LLM-Syn-Planner MUTATION prompts
As professional chenist specializing in synthesis analysie, you are tasked with modifying rotythhheeeei target molecules provided SMILES format.
& retrosynthesàs route is a series of retrosynchesia steps that starts from the given target molecule set and ends with some commercielly purchasable compounds. In the zoute, you need to keep a melecule set, which the molecules need. in the first step, the molecule the user. In each step, you naed to provids a backward reaction and update the molecule act. specifically, it.
should be the
molecule given
the product molecule of reactior from the molecule
then
reactants
By doing so, you reactiors are from the USPTO dataset. Flease also take reactions in atereochemistryy into consideration. For example, B-configuration or E-confiquration.
melezule
molecules commercially purchasable.
The route should be list of steps wrepped in <ROUTE></SOUTE>> with <XMPAALNCCIOO<</XEEEEE LINAAAATTIOOO after it. Each step in tha 1ist should be a dictionary. You need to keep molecule set, which consists of the step,
molecule from the 'Molecule set! as the product molecule in this step and use backward reaction to find the reactente. After teking backward reaction this step, you reactanta you find into the molecule set, and ther name this updated set the 'Updated nolecule sat' in this step. In the next step, the starting molecule set should be the 'Updated molecule set' from the previous step. the last stap, all the molecules in the 'Updated molecule set' should be purchasable. Here la an example:
molecules
aynthesize
purchase,
remove the product
the
the
<ROUTE>
'ooocuul set': "[Target Maleculo]", RRat tonal Step analysta, 'Product's * "[Product molecule]", "Reaction'; "[Reaction template]", 'Beactants'; [[eacteent,, Reactant2]",
'Updated moleaule set"; "(Reactartl, Reastant2)*
'Molecule set'; "[Reactanti, Reactant2]", 'Rational's Step analysis, 'Product': "[Froduct molecule]",
'Reaction'; "[Reaction templata]",
'Reactanta': "[subReactantl, subReactant2j""
'Updated molecule set': "[Resctantl, subReactantl, aubReacLant2)*
1
</ROUTE>
SEXPLANATION:: Explanation for the whole route. </EXPLANATIONPM
Requi rementa:
The Moleccue set' contains molecules need to synthesize at this stage. In the first step, should be the target molecule set. In the following steps, it should be the 'Updased molecule set! Erem previcus step,
2. The 'Rational' part in each step should be your amalysis for synchesis plaaning in this step. It should wrapped 3. "Product" is the molecule plan to synthesize in this step. It should be from the 'Molecule aet The molecule should be à molecule from the 'Molecule set" in List. The molecule amilea should be wrapped with
in string
4. "'eactioo' is reaction that can synchesize the product moiecule. it should on a list. The SMILES format. Fer example, [FroductoReactanl..i Reeccattt) 5. 'Reactantg' are the reactants of the reaction. it should be en a list. The molscule smiles should be wrapped with
seaction template should
6. The 'Updated molacule setr ahould be molecules reed purchase or synthestze after taking this seaction. To get the 'Ugdated molecule set', you need to renove the product molecule from the 'Molecule set! and then aid the reactents in this step into it. In the last step, all the molecules in the "Updated
21

---

LLM-Augmented Chemical Synthesis and Design Decision Programs
molecule setr should be purchasable,
7. In the <EXPLAMATIO>>, you should analyze the whole route and easure the molecules in the "Updated molecule set! in the last step all purchasable.
target molecule (Target Molecule seil Here the feedback for the roste: (Peedback)
To assist you, example zetrosynchesis rcuzes that are close to the target molecules in the sterting molecule will be provided.
<BOUTE>
Retrieved route here </ROUTE>
Please propese retrosyntiesis zeute for the starting melecule set. The provided reference routes may be helpful, You
based your knowledge. All the molacules should be in SMILES format. For example, 0.2 ahould be CICI in SMTL8S format. Br2 should be Brär in SMTMES format, should
design synthetic
SXILES be [0]Br SMILES format. should Bydroger atems are implicitly understord unless explicit clerity is zeeded.
SMILES format.
LLM-Syn-Desight Designer prompts
have
and score
accres. score measures moleculará biological activity
against JNK3.
Malecule SMILES, Molecule SMILES, Molecule 2 score
molecule that has higher JNK3 Flease prepose synthesizable malecule that has higher CNK3 score. You con eithar make crassevara and mutations based on the given just propose molecule based your knowledge.
syathesize
Your output should
format:
TEXPLANATION analysie</ExPLMMATIOO
<MOLECULE>The
your proposed molecu><</MOLEEUDE
requirements:
1. In the <EXPLANATION>, you should analyze how to edit the given molecules to batter property score and then propose your edited molecule or your proposed new molecule, and how to syntheaize your proposed/edited molecule.
2. in the <ccLECUL>>, you should provide the SXILES the molecule you prepose.
22


Reviews:
[{"review_id": "hocXr3glxp", "reviewer": "Reviewer_diPA", "summary": "This paper explores the use of Large Language Models (LLMs) for the challenging task of multi-step retrosynthesis planning and extends this to synthesizable molecular design. The authors introduce a novel approach, LLM-Syn-Planner, which uses an evolutionary search algorithm where the LLM generates and optimizes entire retrosynthetic pathways. The method employs a sequential decision-making representation for synthesis routes, a three-level evaluation process (molecule, reaction, route), and a partial reward mechanism based on the SC score for incomplete routes. Experiments are conducted on USPTO and Pistachio datasets, comparing the LLM-augmented approach with traditional single-step models combined with search algorithms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and thoughtful questions. We are ecstatic that the reviewer finds our method “crucial for tackling the combinatorial complexity of retrosynthesis”. We have updated the new results here: https://drive.google.com/file/d/1Car72zPjMG9__41PfJAGbMkbNObYHjBS/view?usp=sharing\n\n>Error analysis\n\nWe first did an error analysis and observed that the main errors in LLM-Syn-Planner mainly come from the partial reward mechanism—a strong SC Score does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop. To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.\n\n>Underperformance\n\nAfter resolving the infinite loop issue in our error analysis, we present the updated results in Table 1. LLM-Syn-Planner now achieves superior performance compared to the baseline specialized models. Additionally, we include a new open-source LLM, DeepSeek-V3. Our findings show that both LLMs perform well within our framework, demonstrating the effectiveness of LLM-Syn-Former.\n\n> Additional ablation studies\n\nWe conducted several ablation studies to evaluate different design choices: route format, the use of molecule RAG, the reward signal, EA parameters, and prompt robustness. \n\nIn a separate ablation study for direct user queries (Table 3), we queried each LLM 100 times for a fair comparison. On the USPTO easy dataset, both LLMs solved fewer than 10 routes, and on the Pistachio Hard dataset, they failed in nearly all cases. These results suggest the models do not merely “remember” synthetic routes from their training data.\n\nAdditionally, Figure 1 provides a cost analysis, while Table 4 presents an ablation study using DeepSeek-R1, and Table 5 studies the statistical significance.\n\n\n>problematic claim & our technical contribution\n\nWe agree with the reviewer the search algorithm itself is not new. The key contribution of this work is to apply the idea of an EA to generate the entire decision trajectory to the problem of retrosynthesis (which is new). In addition, the workflow we propose such as how LLMs can serve as the proposal and how a proper proxy and retrieval can augment the process is new. We will rephrase the text to reflect this. \n\nOur primary goal in this work is to investigate how effectively LLMs can be leveraged for retrosynthesis tasks. The main contribution is our demonstration that an LLM-based tree-sampler EA method can deliver strong performance in multi-step retrosynthesis planning—surpassing even specialized models. We believe this finding not only validates the utility of LLMs in retrosynthesis but also offers a valuable contribution to the broader research community.\n\n\n>acknowledge EA for molecule design, Partial reward in RL and RAG in NLP\n\nThank you for pointing out the literature. We cited Jensen 2019 and here we adopt a similar EA framework, but we design it for a sequential decision-making problem that is highly different (search for a single molecule vs a synthesis route). We use LLMs as the proposal with additional consideration of partial reward and retrieval-based augmentation. We will acknowledge them in the next version.\n\n>Compared with MolLEO\n\nOur work and MolLEO share only the concept of combining EA with LLMs, but address fundamentally different domains. For task domains, MolLEO focuses on small molecule optimization (naturally represented in SMILES strings). But LLM-Syn-Planner is designed for a sequential decision-making problem. For task challenges, molecule optimization uses LLM to directly edit the molecule and can easily get ground truth feedback from an oracle function. However, retrosynthesis planning poses a more complex, sequential decision-making challenge within a constrained search space. \n\n>Scalability Concerns\n\nWe have shown that the open-source model DeepSeek-V3 can outperform specialized models, suggesting that as open-source LLMs continue to improve, so will LLM-Syn-Planner, thereby reducing computational costs. Currently, the primary efficiency bottleneck is the OpenAI API rate limit, which caps the number of tokens processed each minute. Another bottleneck stems from the raw reaction database; classifying or indexing these reactions would improve the efficiency of RAG.\n\n>question about backward reaction\n\nIn retrosynthesis planning, especially in top-down planning algorithms, the reaction is considered in reverse. We denote this by writing the reaction as Product >> Reactant1.Reactant2, is often referred to as a backward reaction.\n\nWe sincerely thank the reviewer for their valuable feedback. We hope our rebuttal has effectively addressed their questions and concerns. If the reviewer is satisfied with our responses, we kindly ask them to consider a reassessment of our paper. We remain more than happy to address any additional questions or concerns that may arise.", "claims_and_evidence": "Claim: \"We propose a novel way of searching sequence-level decisions with a smooth reward function and partial feedback by sampling from the space of decision sequences (full multi-step synthetic pathways) rather than individual states (a single reaction step).\"\n\n1. \"Novel way of searching sequence-level decisions...\"\n\nEvidence: The paper does describe a search process operating at the level of entire synthesis routes (sequences of decisions). This contrasts with many traditional retrosynthesis methods that use search algorithms (like MCTS or A*) combined with single-step prediction models. The LLM-Syn-Planner generates entire pathways, which are then evaluated and mutated.\n\nProblematic Aspects: The novelty here is questionable, and this is where the claim is weakest. Evolutionary algorithms that operate on entire sequences are not new.  Using an LLM to generate those sequences within an evolutionary framework is a new application, but the underlying search mechanism itself is not novel.  The paper needs to be very careful about how it frames this as \"novel.\" It's a novel application of an evolutionary algorithm, but not a novel search algorithm itself. \n\n2.  \"...with a smooth reward function and partial feedback...\"\n\n  Evidence: The paper does describe a reward function that incorporates the `sc_score` (synthetic complexity score) of intermediate molecules. This is presented as a form of \"partial reward\" because it provides feedback on incomplete routes, unlike a binary reward that only rewards complete, valid syntheses. The `sc_score` itself acts as a smoother measure of progress than a simple binary success/failure.\n\nProblematic Aspects: The use of intermediate rewards in reinforcement learning and search is very common; it's not inherently novel. The paper needs to acknowledge this and position its contribution appropriately. The partial feedback by comparing it against one with only the final reward. It is also very common.", "methods_and_evaluation_criteria": "The paper's proposed methods and evaluation criteria are generally appropriate for the task of retrosynthesis planning and synthesizable molecular design. The use of an evolutionary algorithm with an LLM as a mutation operator is a reasonable approach, and the design choices, such as the sequential route representation, partial reward based on sc_score, and retrieval-augmented generation, are well-justified. The evaluation on standard benchmark datasets (USPTO, Pistachio) using common metrics (solve rate) allows for comparison with prior work. However, concerns remain regarding the potential computational cost of the method and the need for reporting statistical significance and more detailed analysis for molecular design. Furthermore, the chosen experimental methodology, while sound, resulted in performance that did not surpass existing, non-LLM approaches. This indicates that, while the methods make sense, they do not demonstrate an advantage in the current implementation.", "theoretical_claims": "Nope", "experimental_designs_or_analyses": "I have carefully examined the experimental designs and analyses presented in the paper, focusing on the comparison to baselines (Table 2) and the ablation studies (Tables 4, 5, and 6). Several issues undermine the soundness and validity of the experimental results:\n\nUnderperformance: \nThe most significant concern is the underperformance of LLM-Syn-Planner compared to several non-LLM baselines, as shown in Table 2. \n\nLimited Ablation Studies: \nWhile the ablation studies are a positive step, they are limited in scope. The authors should consider ablating other key components of the method, such as the partial reward mechanism, and perform a more thorough sensitivity analysis of the evolutionary algorithm's hyperparameters. The ablation results also suffer from the same lack of statistical significance.", "relation_to_broader_scientific_literature": "The key contributions of the paper should be carefully positioned within the existing scientific literature. While the application of LLMs to retrosynthesis is a timely and relevant topic, several aspects of the proposed method are not fundamentally novel. Specifically:\n\nThe use of an evolutionary algorithm for search is well-established in chemistry, particularly for molecular design (e.g., Jensen, Jan H. \"A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration of chemical space.\" Chemical science 10.12 (2019): 3567-3572.). The authors should acknowledge prior work in this area and clearly differentiate their contribution.\n\nThe concept of operating on entire synthesis routes (sequences of decisions) is not unique to this work. While many retrosynthesis methods focus on single-step prediction, others, such as planning algorithms and prior uses of evolutionary algorithms, also consider longer-range dependencies.\n\nThe use of an LLM as a genetic operator is most closely related to the MolLEO paper (Wang et al., 2024), which applies a similar idea to molecular optimization. The authors should explicitly discuss this connection and highlight the differences in their approach.\n\nPartial rewards and retrieval-augmented generation (RAG) are general techniques, not specific to this work. The authors should acknowledge their broader use in reinforcement learning and NLP.\n\nThe paper's primary contribution lies in the specific combination of these techniques within an LLM-driven framework for retrosynthesis. The authors should emphasize the application and integration of these ideas, rather than claiming broad novelty for the individual components. A more thorough discussion of related work, clearly positioning the paper within the existing literature, is crucial.", "essential_references_not_discussed": "Nope", "other_strengths_and_weaknesses": "Strengths:\n1. Timely and Relevant Topic. The use of LLMs in scientific domains, particularly for complex decision-making, is a highly relevant and active area of research. Applying LLMs to retrosynthesis, a core challenge in organic chemistry, is a timely and appropriate topic for ICML.\n\n2. Exploration of Route-Level Generation: The paper's attempt to move beyond single-step prediction and generate entire synthetic routes is a notable strength. This approach aims to leverage LLMs' potential for long-range planning, which is crucial for tackling the combinatorial complexity of retrosynthesis.\n\n3. Innovative Combination of Techniques:  The integration of an evolutionary algorithm with LLM-based mutation and RAG is a well-motivated approach. The use of partial rewards, sequential representation, and RAG addresses some known limitations of LLMs in structured output generation and long-horizon planning tasks.\n\n4.  Evaluation on Multiple Tasks: Evaluating the method on both retrosynthesis planning and synthesizable molecular design demonstrates a broader applicability and strengthens the empirical assessment.\n\nWeaknesses:\n1. Limited Novelty: While the combination of techniques is specific to this work, the individual components (evolutionary algorithms, RAG, LLM prompting for chemistry) are well-established. The paper's core novelty lies in the specific application and integration, which might be considered incremental by some reviewers.\n\n2. Underperformance Compared to Non-LLM Baselines: The most significant weakness is the empirical performance.  Table 2 clearly shows that LLM-Syn-Planner, while performing well in absolute terms, underperforms compared to several established non-LLM baselines (Retro*, and MCTS combined with specialized single-step models) on the standard retrosynthesis planning tasks. This significantly weakens the claim of the method's effectiveness in practice.  The paper would be significantly stronger if it could demonstrate a clear advantage over existing methods.\n\n3. Lack of Insight into LLM Reasoning: The paper doesn't deeply analyze why the LLM generates the routes it does.  Is it truly leveraging chemical knowledge, or is it relying on pattern matching from the training data? Understanding the LLM's internal reasoning (or lack thereof) would provide valuable insights. More error analysis or qualitative examples of successful/failed routes would be beneficial.\n\n4. Scalability Concerns (Potentially): While not explicitly addressed, the computational cost of using an LLM within an evolutionary loop might be significant.  The paper mentions a 30-minute time limit per molecule, but further discussion of scalability and efficiency would be helpful.", "other_comments_or_suggestions": "Nope", "questions_for_authors": "Line 91, \"At each step, we use a backward reaction to decompose\", can authors clarify how this backward reaction is implemented?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KWVncHXbUS", "reviewer": "Reviewer_hjfV", "summary": "This paper explores the potential of LLMs in addressing retrosynthesis planning and synthesizable  molecular design tasks. The authors first formulate retrosynthesis planning as a sequential decision-making problem with sequential route format that is suitable for LLMs. The authors introduce an evolutionary search algorithm called LLM-Syn-Planner, which allows LLMs to generate and optimize entire retrosynthetic pathways directly. They evaluate the algorithm on multiple USPTO and Pistachio datasets.  The results show that LLM-Syn-Planner performs comparably to or better than these traditional single-step  methods, but not yet competitive as single-step predictors in MCTS/Retro*. For synthesizable molecular  design, LLM-Syn-Designer, an extension of LLM-Syn-Planner, effectively balances optimization efficiency and synthesizability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback! We are glad that the reviewer considers our method to be “novel” and our paper to be “well-written”. We wholeheartedly agree with the reviewer that this method can “be easily adapted to molecular design problems, unifying LLM-augmented reaction decision programs”.\n\nWe have uploaded the results of the new experiments and the required algorithm box in https://drive.google.com/file/d/1IsuWkKzzLrNM0aYdbKx-3n2YnqkaECgg/view?usp=sharing\n\n>New LLM experiments: \n\nWe first did an error analysis for LLM-Syn-Planner and observed that the main errors in LLM-Syn-Planner mainly come from the partial reward mechanism—a strong SC Score does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop. To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.\n\nBased on this, we conducted experiments on an open-source model DeepSeek-V3, and the results are shown in Table 1. Both LLMs achieve comparable or even surpass the baseline specialized models, which demonstrates the effectiveness of the LLM-Syn-Planner framework. To the best of our knowledge, we are the first to show that current LLMs can achieve decent performance on multi-step retrosynthesis planning tasks, paving the way for an autonomous LLM agent framework for molecule synthesis and optimization.\n\nMoreover, we also tested DeepSeek R1. The results are shown in Table 4. Interestingly, although DeepSeek R1 is designed as a reasoning model, it performs worse than DeepSeek V3. \n\n>Evaluation metrics\n\nWe conducted a cost analysis shown in Figure 1. The figure shows the average cost of each discovered route across the four datasets. As the route length increases, the cost of finding that route also rises, but at an acceptable rate. As open-source LLMs improve, it is reasonable to believe that LLM-Syn-Planner will also improve, so computational cost will be mitigated in the near future. For the route length, due to the time constraint for rebuttal, we will add it in our future version. \n\n> Definition of the functions INITIALIZATION and MUTATION\n\nWe show the definition in Algorithm 1 and Algorithm 2.\n\nINITIALIZATION queries the LLM to produce an initial route for the given target molecule. It uses a RAG strategy by first identifying molecules similar to target_molecule in retrieval_db, retrieving their known routes, and providing these references to the LLM to guide route generation. \n\nMUTATION prompts the LLM to revise any invalid parts of current_route beginning at invalid_step_index. The LLM may directly fix the incorrect step, substitute it with a more suitable reaction, or even propose an alternative multi-step path if needed.\n\n>typo in the algorithm box\n\nYes in line 228, R_0 should be P_0. In line 231, t should start from 0. Thanks for pointing out. We will fix this typo in the next version.\n\n>solve rate with more model calls (N)?\n\nWhen the number of model calls for LLM-Syn-Planner increases from 100 to 300, the solve rate shows a significant improvement. However, increasing the number of model calls from 300 to 500 yields only minimal additional gains. One factor is that our partial reward does not always correlate with final synthetic success (e.g., a molecule with an SC score of 1 might still be extremely difficult to synthesize in practice).\n\n>For reaction-level evaluation, how is the chemical feasibility of the proposed reaction assessed?\n\nTo assess the chemical feasibility at the reaction level, we verify whether the product molecule in this step could validly undergo the backward reaction. In practice, we use the rdchiralRun function from the rdchiral package to check if the product matches the reaction template such that the backward reaction is recognized. If rdchiralRun confirms that the product can serve as a valid reactant under the backward reaction, we regard the forward reaction as chemically feasible.\n\n> How does the LLM identify substructures, and in what form are these substructures represented and used to extract templates? (line 197)\n\nWe begin by prompting the LLM to identify relevant substructures and functional groups in the product molecule. In response, the LLM provides these substructures in SMILES format. Next, we verify each suggested substructure SMILES to ensure it is indeed present within the product structure. Once validated, we use these SMILES to compute Tanimoto similarities between the identified substructures and the product molecules in our reference reaction database to extract templates.\n\nWe hope that our responses were sufficient in clarifying all the great questions asked by the reviewer. We sincerely thank the reviewer for their time and consideration. We would be happy to engage in further discussions if the reviewer has any additional questions or concerns.", "claims_and_evidence": "Yes, the claims are supported by clear and convincing experimental evidence.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria are well-suited for the problem studied in the paper.", "theoretical_claims": "No, there are no proofs or theoretical claims in the paper.", "experimental_designs_or_analyses": "Yes, the experimental designs appear sound, particularly in evaluating retrosynthesis planning and  synthesizable molecular design using datasets like USPTO and Pistachio.", "relation_to_broader_scientific_literature": "The key contributions of this paper are closely related to the broader field of AI for Science, particularly in chemistry and drug discovery. This aligns with the growing trend of using AI, including LLMs, to tackle complex scientific problems, as explored in works like [Segler'18] and [Chervonyi'25].\n\n[Segler'18] Segler, Marwin HS, Mike Preuss, and Mark P. Waller. \"Planning chemical syntheses with deep \nneural networks and symbolic AI.\" Nature 555.7698 (2018): 604-610.\n\n[Chervonyi'25] Chervonyi, Yuri, et al. \"Gold-medalist Performance in Solving Olympiad Geometry with \nAlphaGeometry2.\" arXiv preprint arXiv:2502.03544 (2025).", "essential_references_not_discussed": "No, related works are properly discussed.", "other_strengths_and_weaknesses": "Strengths:\n* The paper is clearly written and well-formatted.\n* Unlike previous methods that use LLMs as single-step reaction predictors, this work introduces a novel  evolutionary approach that allows LLMs to generate, evaluate, and modify pathways at the route level.\n* The proposed algorithm can be easily adapted to molecular design problems, unifying LLM-augmented \nreaction decision programs.\n* The code is available, enhancing reproducibility.\n\nWeaknesses:\n* The experiments are limited to GPT-4o as the only LLM tested. The authors could expand their evaluation to include other models, such as open-source models like Qwen-2.5, Llama-3, or recent strong reasoning models like o1 or r1.\n* The evaluation metrics are limited to success rate. Including additional metrics such as route lengths and costs would provide a more comprehensive assessment of the proposed methods.", "other_comments_or_suggestions": "* A clear and formal definition of the functions INITIALIZATION and MUTATION , including their arguments and return values, would improve clarity.\n* line 228: should R_0 be P_0 \n* Line 231: t should start from 0", "questions_for_authors": "* Does the solve rate continue to improve with more model calls (N)?\n* For reaction-level evaluation, how is the chemical feasibility of the proposed reaction assessed?\n* For the LLM as a single-step prediction model, how does the LLM identify substructures, and in what form are these substructures represented and used to extract templates? (line 197)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nLHYDB4hNo", "reviewer": "Reviewer_kcQx", "summary": "This work attempts to investigate whether LLMs can solve complex chemical sequences and proposes a framework that uses LLMs without tuning for conducting retrosynthesis and molecular design tasks. An effective method for formatting synthesis sequences is introduced. Additionally, a novel approach for searching sequence-level decisions via a reward function and partial feedback is presented, enabling multi-step retrosynthesis. Through this unified framework, the authors demonstrate the efficacy of the proposed method for chemical synthesis tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback! We are thrilled the reviewer finds our LLM-based retrosynthesis framework “could be further extended to an autonomous laboratory framework or a molecular synthesis and generation pipeline using LLMs”.  \n\nBelow, we address the reviewer’s questions about experiments as well as the technical contribution of the paper. We have provided additional experiment results in the following link: https://drive.google.com/file/d/1ANfpzJLJgJh83saOwBzK4AMgenb41Jnc/view?usp=sharing\n\n>Performance of LLM-Syn-Planner\n\nFirst, we conducted error analysis for LLM-Syn-Planner and observed that the main shortcomings stem not from the LLM itself but from the partial reward mechanism—namely, that a strong SCScore does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop.\n\nTo address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.\n\n>Other general LLMs\n\nWe have included new experimental results for the open-source models DeepSeek V3 and DeepSeek R1. The performance of DeepSeek V3 is shown in Table 1, while the results for DeepSeek R1 appear in Table 4. Interestingly, although DeepSeek R1 is designed as a reasoning model, it performs worse than DeepSeek V3. Moreover, because DeepSeek R1 includes its thinking process in the output, its overall cost is roughly three times higher than that of DeepSeek V3.\n\n>Technical contribution \n\nOur primary goal in this work is to investigate how effectively LLMs can be leveraged for retrosynthesis tasks. The key contribution of this work is to apply the idea of an evolutionary algorithm to generate the entire decision trajectory to the problem of retrosynthesis (which is new). In addition, the workflow we propose such as how LLMs can serve as the proposal and how a proper proxy and retrieval can augment the process is new. We will rephrase the text to reflect this. The strongest contribution, in our humble opinions, of this work is the demonstration of the strong performance of the retrosynthesis planning problem with our LLM-based tree-sampler EA method. To the best of our knowledge, we are the first to show that LLMs can achieve comparable or even better performance compared with specialized models. This also provides evidence for solving other science problems with a similar structure. \n\n\n>A cost analysis is also needed\n\nIn Figure 1, we present a cost analysis showing the average cost of each discovered route across the four datasets. As the route length increases, the cost of finding that route also rises, but at an acceptable rate. Given the rapid advancements in LLMs, we anticipate that these costs will decrease over time. For example, Llama 3 outperforms Llama 2 significantly despite being released only one year later.\n\n>Ablation of the prompt\n\nIn Table 2, we present an ablation study examining how the <Explanation> component in the overall prompt and the ‘Rational’ tag at each step affect performance. Our findings show that both elements enhance the language model’s reasoning capabilities, underscoring the importance of including thinking/explanation steps in the prompt.\n\nWe are happy to address any additional comments or concerns the reviewer may have. Otherwise, we would appreciate it if the reviewer continues to view this work positively. Thank you once again for your time and thoughtful feedback.", "claims_and_evidence": "Rather than claiming that LLMs can outperform or surpass previous specialized approaches, the authors investigate the capability of LLMs in chemical domains by assessing their problem-solving ability in complex tasks. Without relying on tuning methods such as fine-tuning or instruction tuning, they solely depend on parameterized and retrieved knowledge, as well as the models' own reasoning abilities. They design a well-structured framework that combines prompts and the GA algorithm, targeting reaction pathway optimization.", "methods_and_evaluation_criteria": "The proposed framework utilizes a reaction database, LLM, and the GA algorithm with partial feedback and rewards. As a unified scheme, it presents a reasonable combination and demonstrates effective performance. The evaluation criteria are well-established for assessing the proposed framework.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "Comparisons with other specialized models and experiments on two tasks are well conducted. However, experiments using other general LLMs are necessary to genuinely investigate the capability of LLMs in solving complex problems in the chemistry domain.", "relation_to_broader_scientific_literature": "This paper is related to machine learning-based organic retrosynthesis and molecular optimization works. It could be further extended to an autonomous laboratory framework or a molecular synthesis and generation pipeline using LLMs.", "essential_references_not_discussed": "This work effectively covers the key references.", "other_strengths_and_weaknesses": "It is a clever combination of existing modules, enhanced with curated feedback and a reward function. In essence, the proposed scheme is an extension of MOLLEO into the retrosynthesis task, with added reward and feedback. Solving multi-step retrosynthesis and molecular design using LLMs without tuning seems reasonable, but the technical novelty appears somewhat limited.", "other_comments_or_suggestions": "This framework uses only GPT-4o, so experiments with other general LLMs are necessary. A cost analysis is also needed, as the 30-minute call limit per molecule could be expensive.", "questions_for_authors": "Looking at the prompt template, it contains many explanations and instructions. Is the proposed scheme robust to variations in the prompt format?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6HC9EV7JN0", "reviewer": "Reviewer_cmv4", "summary": "This paper attempts to address the problem of multi-step retrosynthesis planning using large language models (LLMs). The authors first propose a format for representing a synthesis pathway, using a sequential reaction step format to characterize the synthesis pathway. This approach allows large models to avoid the traditional difficulties associated with understanding deeply nested tree structures. Additionally, the authors introduce a generation + refinement pipeline, which involves first generating a complete synthesis pathway in one go and then using three different levels of rewards + prompts to guide the LLM to further evolve the generated pathway, ultimately obtaining a correct synthesis pathway. The authors conducted experimental validations on four subsets of synthesis pathways derived from the USPTO and Pistachio databases. They also applied the proposed method to molecular property optimization, aiming to ensure the synthesizability of molecules while optimizing their properties. Experiments were carried out for three different properties.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are extremely appreciative of the reviewer’s feedback and suggestions to improve our work. We are glad that the reviewer considers our work to “potentially introduce a new paradigm for future multi-step retrosynthesis planning”.\n\nWe provide additional experiment results in the following link: https://drive.google.com/file/d/1RIcdPo-uV5Fibbllk2COJl9niUhrVAFk/view?usp=sharing\n\n>fair comparison\n\nWe follow the Rootaligned paper to set the number of test-time augmentation to 20. In updated Table 1, we extend the search time for each molecule to 60 minutes. Only Rootaligned cannot finish all calls. Due to the time constraint for rebuttal, we will adjust the test-time augmentation in our future version. Note we do not intend to limit the solving time for comparison benefit, but only to rule out methods with extremely long solving times.\n\n>whether the synthesis path proposed by the model has been seen during pretraining. Experiments to show the routes for the provided molecules.\n\nWe conducted an ablation study for direct user queries and presented the results in Table 3. The results suggest that the models do not simply “remember” synthetic routes from their training data. We provide the routes for four suggested molecules discovered by our method in the PDF.\n\n>for synthesizable molecular design\n\nIn contrast to approaches like [1] that start with an unsynthesizable molecule and then “project” it to a synthesizable form—often compromising similarity and potentially degrading properties (For instance, Table 2 in [1] observes that the average similarity can be modest)—our method jointly optimizes molecular properties and ensures synthesizability from the outset. By directly sampling candidate molecules with high properties and verifying their synthetic accessibility through route sampling, we avoid an additional “projection” step and minimize property loss. We will discuss [1] in our related work as an alternative way to find optimized molecules with synthesis pathways.\n\n> The size of the molecule set will increase\n\nWhile it is true that the set can increase with each additional step, our framework mitigates this issue in practice. Specifically, during mutation, we provide the LLM only with the partial route starting from the first invalid step, rather than resupplying the entire route. Furthermore, purchasable molecules in the set can be filtered out and do not need to be reintroduced to the LLM. Our experiments indicate that this approach remains manageable in real-world scenarios. \n\n>Replace the chemical reaction proposed by the LLM with a retrieved one. Label leakage.\n\nThis procedure does not introduce label leakage because we rely only on reactions known to exist (i.e., those in the database). In fact, the single-step specialized models are also trained on reactions extracted from USPTO-FULL. In the original Retro* paper, they even filter the test route dataset by only keeping the routes whose reactions are all covered by the top 50 predictions by the single-step model (Section 5.1.1).\n\nThe main role of retrieving is to use as a template and approximate information as a reference.\n\n> How to verify the feasibility of the synthesis pathway\n\nYes. Since the reaction templates used are from reliable databases, we can apply the reaction template using the chemical tools rdchiral to verify.\n\n> Practical advantages of the proposed method\n\nThrough detailed analysis of the failure cases, we observed that the main issue is not from the LLM itself but from the partial reward mechanism—namely, that optimal SCScore does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop.\n\nTo address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines. This indicates that LLMs offer a powerful framework for complex, multi-step decision-making in scientific domains, and we are the first to show that LLMs can achieve comparable or even better performance compared with specialized models. This also provides evidence for solving other science problems with a similar structure. \n\nRegarding the reaction database, it is worth noting that specialized models also require such databases: for instance, localRetro formulates one-step retrosynthesis as a classification problem, necessitating a pre-defined set of reaction data. \n\nAt last, we would be happy to engage in further discussions with the reviewer regarding these topics or any additional concerns the reviewer has – please let us know! We thank the reviewer again for their time spent with our paper. If the reviewer finds that our new experiments, analyses, and discussions are useful for improving the paper, we would be grateful if the reviewer would consider a fresher evaluation of our paper.\n\n[1] Luo S, et al. Projecting Molecules into Synthesizable Chemical Spaces, ICML2024\n\n[2] Chen B, et al. Retro*..., ICML2020", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "This paper does not involve any theoretical claims", "experimental_designs_or_analyses": "I have checked all the experimental designs and analyses. Here are the problems:\n\n- for retrosynthesis planning\n\n  - When using RootAligned for single-step retrosynthesis prediction, did you employ test-time augmentation? Although this can improve the solve rate, it significantly reduces the model's response speed. The original paper of RootAligned[3] set the number of test-time augmentation to 20, meaning that the same sample would undergo 20 inferences. If you reduce this parameter to 10 or 5, you might be able to increase the number of model responses per unit time at the cost of a slight decrease in the accuracy of single-step retrosynthesis prediction.\n\n  - Setting a maximum solving time is not fair, as GPT-4o is deployed on a very powerful cluster while the baselines are not. The authors should deploy these baseline models on a server as powerful as the OpenAI cluster and then set a unified maximum solving time; otherwise, it is unfair.\n\n  - The datasets used in the experiments are relatively old, and most of them are accompanied by detailed original papers and patent articles. These articles may have already become part of the pretraining corpus for GPT-4o. It is difficult for the authors to prove whether the synthesis path proposed by the model has been seen during pretraining. The authors should present some recently marketed or complex molecules as case studies to demonstrate that the proposed method truly works. The following are some convincing choices:\n\n    - Pacritinib, SMILES: `C1=CC2=CC(=C1)COCC=CCOCC1=C(OCCN3CCCC3)C=CC(=C1)NC1=NC=CC2=N1`\n    - Daprodustat, SMILES `O=C(O)CNC(=O)C1C(=O)N(C2CCCCC2)C(=O)N(C2CCCCC2)C1=O`\n    - Nirmatrelvir, SMILES `C(N[C@@H](C[C@H]1C(=O)NCC1)C#N)(=O)[C@@H]2[C@@]3([C@@](C3(C)C)(CN2C([C@@H](NC(C(F)(F)F)=O)[C@](C)(C)C)=O)[H])[H]`\n    - 5-HT6 receptor, SMILES `O=S(=O)(NC=1C=C(C=2OC3(CCC2C1)CCC3)N4CCNCC4)C=5C=CC=CC5F`\n\n    These molecules have been used as case studies in references [1, 2, 3], and all of them have been predicted with reasonable synthesis routes by single-step retrosynthesis models combined with manual expansion by experts. The reviewer believes that these cases are molecules that AI can accurately predict synthesis routes for but are still challenging.\n\n- for synthesizable molecular design\n\n  - The proposed method only compared some molecular optimization methods, but these methods do not guarantee the synthesizability of the optimized molecules. In fact, there have already been works on improving the synthesizability of molecules without compromising their properties (see **Essential References Not Discussed**). Experiments and analyses about synthesizable molecular design need to combine molecular property optimization with molecular synthesizability optimization as a baseline, instead of unilaterally claiming that molecular property optimization methods cannot guarantee synthesizability and thus the proposed method is better.\n\n**Reference**\n\n[1] Yao L, Guo W, Wang Z, et al. Node-aligned graph-to-graph: Elevating template-free deep learning approaches in single-step retrosynthesis[J]. JACS Au, 2024, 4(3): 992-1003.\n\n[2] Zeng K , Yang B , Zhao X ,et al.Ualign: pushing the limit of template-free retrosynthesis prediction with unsupervised SMILES alignment[J].Journal of Cheminformatics, 2024, 16(1):1-16.DOI:10.1186/s13321-024-00877-2.\n\n[3] Zhong Z, Song J, Feng Z, et al. Root-aligned SMILES: a tight representation for chemical reaction prediction[J]. Chemical Science, 2022, 13(31): 9023-9034.", "relation_to_broader_scientific_literature": "The idea proposed in this paper of using prompts to generate a synthesis pathway in one go and then optimize it is groundbreaking in multi-step retrosynthesis, and is distinct from other previous works on multi-step retrosynthesis planning.", "essential_references_not_discussed": "In this submission, the design of synthesizable molecules is discussed, and the authors' model is compared with other molecule optimization methods, claiming that their model can optimize properties while ensuring synthesizability, which other baselines cannot achieve. However, an excellent work [1] appeared at ICML 2024 that converts unsynthesizable molecules into synthesizable ones while as much as possible ensuring the molecular properties. By using this ICML-published article to optimize the synthesizability of all baseline-optimized molecules, both the properties and synthesizability of the molecules would be ensured to some extent. The submission did not design any experiments to demonstrate that the proposed method could achieve better results than calling the molecule optimization model and this ICML 2024 work in series.\n\n**Reference**\n\n[1] Luo S, Gao W, Wu Z, et al. Projecting Molecules into Synthesizable Chemical Spaces[C]//International Conference on Machine Learning. PMLR, 2024: 33289-33304.\n\n\n#", "other_strengths_and_weaknesses": "**Strengths**\n\nThis article pioneers an innovative approach, namely the idea of generating a complete synthesis pathway in one go and then refining it. This could potentially introduce a new paradigm for future multi-step retrosynthesis planning.\n\n**Weakness**\n\nIn the sequential structure formatting of the synthesis pathway designed in the article, the size of the molecule set in each step will increase linearly with the number of steps. Consequently, the total number of tokens in the entire synthesis pathway will have an almost quadratic relationship with the number of steps. This is clearly not concise.", "other_comments_or_suggestions": "I have no other comments.", "questions_for_authors": "1. Why is it necessary to replace the chemical reaction proposed by the LLM with a retrieved one during the **Evaluation** process? Does this operation alter the updated molecule set of the replaced step and the subsequent synthesis steps? **It is particularly important to note that  the reaction database is constructed from USPTO-FULL, which contains all the single-step reactions of the routes in USPTO Easy and USPTO-190. If such a replacement is actually carried out, the method would involve the problem of label leakage.**\n2. How does the paper verify the **feasibility** of the synthesis pathway when calculating the solve rate? Does it check whether the reactants of each step can produce the desired product through the reaction?\n3. What are the **practical advantages** of the proposed method? Retrosynthesis planning is a very application-driven topic. The proposed method does not outperform specialized models in terms of performance, and it relies on large-scale computing power (needing to call powerful LLM APIs) and a pre-defined reaction database, while specialized models can run under more modest computing conditions (for example, models like localRetro can perform a single-step retrosynthesis prediction within a second on a 1080ti graphics card) without the need for a pre-defined reaction database.\n\nIf the authors can address all the issues proposed above and the issues in **Experimental Designs Or Analyses**, I will be willing to raise my score.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a framework for LLM-driven multi-step retrosynthesis and synthesizable molecular design. The main contributions include:
- A novel integration of evolutionary algorithms with LLMs for sequence-level decision-making in retrosynthesis, enabling end-to-end pathway generation and refinement.
- A reward mechanism incorporating partial feedback (e.g., SCScore) for intermediate route assessment, and the use of retrieval-augmented generation to improve chemical validity.
- Empirical evaluation across multiple retrosynthesis and molecular design benchmarks (USPTO, Pistachio), with evidence showing that LLM-Syn-Planner matches or outperforms specialized baselines in several settings, even without fine-tuning.

The authors provided extensive responses, including new experiments with open-source LLMs (DeepSeek), additional ablation studies, error analysis, and clarifications of reaction verification and deployment cost. Reviewer diPA upgraded their score after acknowledging improved performance. Reviewer cmv4 remains concerned about practical cost, but noted improvements and raised his score.

Despite some lingering concerns, the reviewers unanimously support its acceptance.

The authors are suggested to (1) clarify **novelty claims to focus on the integration and application, not algorithmic development**, (2) address cost-effectiveness and fairness concerns more transparently, and (3) deepen discussion on real-world generalizability, including LLM limitations and verification of synthetic validity.
Decision: Accept (poster)