SemiRetro: Semi-template framework boosts deep retrosynthesis prediction
Authors: Zhangyang Gao
Cheng Tan
Lirong Wu
Haitao Lin
Stan Z. Li
Venue: ICLR 2022 Submitted (2022)
Keywords: Retrosynthesis prediction
molecular graph learning

Abstract: Retrosynthesis brings scientific and societal benefits by inferring possible reaction routes toward novel molecules. Recently, template-based (TB) and template-free (TF) molecule graph learning methods have shown promising results to solve this problem. TB methods are more accurate using pre-encoded reaction templates, and TF methods are more scalable by decomposing retrosynthesis into subproblems, i.e., center identification and synthon completion. To combine both advantages of TB and TF, we suggest breaking a full-template into several semi-templates and embedding them into the two-step TF framework. Since many semi-templates are reduplicative, the template redundancy can be reduced while the essential chemical knowledge is still preserved to facilitate synthon completion. We call our method SemiRetro and introduce a directed relational graph attention (DRGAT) layer to extract expressive features for better center identification. Experimental results show that SemiRetro significantly outperforms both existing TB and TF methods. In scalability, SemiRetro covers 96.9\% data using 150 semi-templates, while previous template-based GLN requires 11,647 templates to cover 93.3\% data. In top-1 accuracy, SemiRetro exceeds template-free G2G 3.4\% (class known) and 6.4\% (class unknown). Besides, SemiReto has better interpretability and training efficiency than existing methods.

Under review as a conference paper at ICLR 2022
SEMIRETRO: SEMI-TEMPLATE FRAMEWORK BOOSTS DEEP RETROSYNTHESIS PREDICTION
Anonymous authors Paper under double-blind review
ABSTRACT
Retrosynthesis brings scientific and societal benefits by inferring possible tion routes toward novel molecules. Recently, template-based (TB) and template- free (TF) molecule graph learning methods have shown promising results to solve this problem. TB methods are more accurate using pre-encoded reaction tem- plates, and TF methods are more scalable by decomposing retrosynthesis into subproblems. i.e., center identification and synthon completion. To combine both advantages of TB and TF, we suggest breaking a full-template into several semi- templates and embedding them into the two-step TF framework. Since many semi- templates are reduplicative, the template redundancy can be reduced while the essential chemical knowledge is still preserved to facilitate synthon completion. We call our method SemiRetro and introduce a directed relational graph atten- tion (DRGAT) layer extract expressive features for better center identification. Experimental results show that SemiRetro significantly outperforms both existing TB and TF methods. In scalability, SemiRetro covers 96.9% data using 150 semi- templates, while previous template-based GI.N requires 1 11,647 templates to cover 93.3% data. In top-1 accuracy, SemiRetro exceeds template-free G2G 3.4% (class known) and 6.4% (class unknown). Besides, SemiReto has better interpretability and training efficiency than existing methods.
INTRODUCTION
plays a crucial role in synthesis plan- ning and drug discovery, which aims infer possible reactants for synthesizing a target molecule. This problem is quite challenging due to the vast search space, multiple theoretically correct syn- thetic paths, and incomplete understanding of the reaction mechanism, thus requiring considerable expertise and experience. Fortunately, with the rapid accumulation of chemical data, machine learn- ing is promising to solve this problem (Coley et al. 2018 Segler et al.) 2018). this paper, we focus on the single-step version predicting the reactants from given product.
Retrosynthesis prediction (Corey & Wipke 1969 Corey 1991
Common deep-learning-based retrosynthesis works can be divided into template-based (TB) (Co- ley et al.) 2017b| Segler & Waller 2017: Dai et al.) 2019) and template-free (TF) (Liu et al., 2017 Karpov et al.) 2019) methods. Generally, TB methods achieve high accuracy by leveraging reaction templates, which encode the molecular changes during the reaction. However, the usage of templates brings some shortcomings, such as high computation cost and incomplete rule coverage, limiting the scalability. To improve the scalability, class of chemically inspired TF methods (Shi et al., 2020 Yan et al. 2020) (see Fig. [1 have achieved dramatical success, which decompose retrosynthesis into subproblems: center identification and ii) synthon completion. Center identification increases the model scalability by breaking down the target molecule into virtual synthons without utilizing templates. Synthon completion simplifies reactant generation by taking synthons as potential start- ing molecules, i.e., predicting residual molecules and attaching them to synthons to get reactants. Although various TF methods have been proposed, the top-k retrosynthesis accuracy remains poor, especially when the reaction class is unknown. Can we find more accurate way predict potential reactants while keeping the scalability?
To address the aforementioned problem, we suggest combining the advantages of TB and TF ap- proaches, and propose a novel framework namely SemiRetro. Specifically, we break a full-template
In single-step retrosynthesis, the synthesis route length is i.c., only one reaction needs he inferred.

---

Under review as a conference paper at ICLR 2022
into several simpler semi-templates and embed them into the two-step TF framework. As many semi-templates are reduplicative, the template redundancy can be reduced while the essential chem- ical knowledge is still preserved to facilitate synthon completion. Moreover, introduce directed relational graph attention (DRGAT) layer to extract more expressive molecular features to improve center identification accuracy. Finally, we combine the center identification and synthon completion modules in a unified framework to accomplish retrosynthesis predictions.
We evaluate the effectiveness of SemiRetro on the benchmark data set USPTO-50K, and compare with recent state-of-the-art TB and TF methods. We show that SemiRetro significantly outperforms these methods. In scalability, SemiRetro covers 96.9% of data using 150 semi-templates, while previous template-bess based GLN requires 11,647 templates to cover 93.3% of data. In top-1 accuracy, SemiRetro exceeds template-free G2G 3.4% (class known) and 6.4% (class unknown). Owing to the semi-template, SemiReto is more interpretable than template-free G2G and RetroXpert in synthon completion. Moreover, SemiRetro trains at least 6 times faster than G2G, RetroXpert, and GLN. All these results show that the proposed SemiRetro boosts the scalability and accuracy of deep retrosynthesis prediction.
2 RELATED WORK
Template-based models TB methods infer reactants from the product through shared chemical transformation patterns, namely reaction templates. These templates are either hand-crafted by hu- man experts (Hartenfeller et al. 2011 Szymkuć et al. 2016) or automatically extracted by algo- rithms (Coley et al.) 201 7a Law et al. 2009). For product molecule, due to the vast search space, multiple qualified templates, and non- non-unique matching sites for each template, it is challenging to select and apply the proper template to generate chemically feasible reactants. To handle those chal- lenges, (Coley et al. (2)) suggests sharing the same templates among similar products. (Segler & al. 2019) employ neural models for template selection with molecule finger- print as input. The ofate-of-th-aae GIN (Dai et al.] (219)) learns the joint distribution of templates and products by decomposing templates into pre-reaction and post-reaction parts and introducing logic variables to apply structure constraints. TB methods are interpretable and accurate because they embed rich chemical knowledge into the algorithm. However, the vast space of templates and incomplete coverage severely limit their scalability.
Waller 2017:Bayoo
Template-free models Instead of explicitly using templates, TF approaches learn chemical trans- formations by the model. (Liu et al.] 2017; Karpov et al., 2019) solve the retrosynthesis problem with seq2seq models, e.g. Transformer (Vaswani et al. (017), LSTM (Hochreiter & Schmidhuber] 1997). based on the SMILES representation of molecules. Despite the convenience of modeling, SMILES cannot fully utilize the inherent chemical structures and may generate invalid SMILES strings. Therefore, (Zheng et al.) (019) propose self-corrected transformer to fix the syntax er- rors of candidate reactants. Recently, G2G (Shi al., 2020). RetroXpert (Yan et al.) (020) and GraphRetro (Somnath et al. 2021) achieve tfte-offthhe-ae performance by decomposing the ret- rosynthesis into two sub-proue problems: i) center identification and ii) synthon completion, as shown in Fig. Center identification increases the model scalability by breaking down the target molecule into virtual synthons without utilizing templates, among which G2G reports the highest accuracy. Synthon completion simplifies the complexity of reactant generation by taking synthons as potential starting molecules. For example, RetroXpert and G2G treat it as a SMILES or graph sequence trans- lation problem from synthon to reactant. GraphRetro completes synthons by predicting pre-defined leaving groups, but it does not provide end-to-end open-source algorithms for attaching leaving groups and model construction. Generally, these TF methods are more scalable but perform worse than TB GLN in top-1 accuracy.
Challenges Although the two-step TF framework significantly improves the algorithm's 's scalabil- ity, the overall accuracy is relatively low. A possible solution to this issue is to enhance submodules, i.e. center identification and synthon completion. 1) To the best of our knowledge, current GNN models only work well for center identification when the reaction class is known; Otherwise, the model performance degrades rapidly. How develop more suitable model that works well with unknown classes is the first challenge. 2) In addition, synthon completion is the major bottleneck affecting the overall accuracy. Specifically, predicting and attaching residuals for each synthon
2

---

Under review as a conference paper at ICLR 2022
difficult because the residual structures could be complex, attaching residuals into synthons may violate chemical rules, and various residuals may agree with the same synthon (e.g.. F, CI. Br, and I have similar chemical properties). For researchers, scalability, interpretability, and training effi- ciency are also important. How to develop more accurate, interpretable, and efficient synthon completion model while maintaining the scalability is the second challenge.
Step1: Center identification
DRGAY
13
L4
Higteboouncy
Synthon
SOTA GWN6
Synthon
Step2: Synthon completion
GLN22019 9
Template-based
full template
produce macove
plssina mortenl
Detter scalabilty
! Higher acouracy
BemiRettro/2221)
Semi template
semi-template 1
reeiceett
Synthon
semi-template
Symthon
reaclant
Efficient to train
Higher acouracy Better interpatabilty
Cand tinal Intry wntinn groveationn
Con Conpiiional
enqueerie genacaics
Template-free
Tou (NeC SJC 22w2H4
input
"prooduco reactant
you copeaa segouctpe
input
101 21
(020)
RetroXpart/22222
Figure Overview of SemiRetro. decomposite retrosynthesis into two steps: center identification and synthon completion. In step 1, we use DRGAT to extract molecule features for predicting reaction centers. By breaking product bonds in these centers, synthons can be obtained. In step 2, we use another DRGAT model to predict the semi-template for each synthon. The final reactants can be deduced from reaction centers, synthons, and semi-templates by using the residual attachment algorithm.
DEFINITION AND OVERVIEW
Molecule representation There are two types of dominant representations, i.e., SMILES string [Weininger] (988) and molecular graph. SMILES commonly used in early works (Liu et al. 2017 Schwaller et al., 2018 Zheng al. 2019 Schwaller et al. 2019: Tetko et al. (020) due its simpleness. Many NLP models can directly applied to solve related problems in an end-to-ent fashion. However, these models cannot guarantee the chemical correctness of the output molecules because they ignore structure information to some extent. Similar to recent breakthroughs (Dai et al., 2019 Shi et al. 2020 Yan et al. 2020 Somnath al.) (222), we take the molecule as labeled graph (AA, X, E), where A, X and E are adjacency matrix, atom features and bond features, seeing Table. M Under the graph framework. we can effectively apply chemical constraints to ensure the validity of output molecules. Besides, maaph-baeed methods are more controllable and interpretable than SMILES- ones.
Problem definition Retrosynthesis aims to infer the set of reactants [ that can generate the product P. Formally, that is to leam mapping function Fo:
Fo P {R,
N; 1=1
3

---

Under review as a conference paper at ICLR 2022
Considering the unknown by by-products, the law of conservation of atoms longer holds here, which makes the problem quite challenging because the algorithm needs to generate new atoms and bonds to get potential reactants automatically.
Overview As shown in Fig. we adopt the two-step TF framework due to its scalability and effec- tiveness. Our method is distinguished from previous works in two folds: 1) We propose relational graph attention (DRGAT) layer to improve the center identification performance; 2) We use semi- templates to facilitate synthon completion, which significantly reduces the problem complexity.
Table 1: Commonly used symbols
Symbol
Description
E) A
Molecular graph with adjacency matrix A, atom features and bond features A E {0, 1iin with the number atoms O., 1 indicates that there is hond and and vice versa.
between
X
E Redd, is the dimension of atom features. æ; is feature vector atom i.
E
E E Rm,,, b bond features dimension and m is the number of bonds.
CL R2 S.. P
feature vector of bond (i,j).
a molecular graph of the i-th reactant, the j-th synthon and the product.
4 E {0, 1), indicating whether atom is the reaction center or not.
9J
C. E {0, 1), indicating whether bond (f.j) is the reaction center or not.
4 METHODOLOOGY
4.1 CENTER IDENTIFICATION
Center identification plays vital role in the two-step retrosynthesis because enrors caused by this step directly lead to the final failures. Previous works have limitations, e.g., RetroXpert Yan et al., 2020) provides incomplete prediction without considering atom centers, and G2G (Shi et al.) 2020) performs poorly when the reaction class is unknown. How to obtain comprehensive and accurate center identification results is still worth exploring.
Reaction centers We consider both atom centers and bond centers the product molecule. As shown in Fig. [ from the product its corresponding reactants, either some atoms add residuals by dehydrogenation without breaking the product structure (case 1), or some bonds are broken to allow new residues to attach (case 2). Both these atoms and bonds are called reaction centers.
Case1: 1: atom center
Case2: bond center Ca0
Synthon
syrithon 2
product
residual
synthon
residuel
rredduall 2
Figure 2: Reaction centers. Products, reactants, and residuals are circled in blue, green, and red, respectively. We label atoms in reaction centers with solid circles. In case 1, the centered atom adds residual by removing hydrogen atoms without changing the product structure. In case 2, the centered bond in the product broken to generate synthons, to which new residuals are attached.
Directed relational GAT Commonly used graph neural networks (Defterrard et al. 2016: Kipf
& Welling, 2016: Velicković et al., 2017) mainly focus on and I edges, ignoring edge direction and multiple types, thus failing to capture expressive molecular features. As to molecules, different bonds represent different interatomic interactions, resulting in multi-relational graph. Meanwhile, atoms at the end of the same bond may gain or lose electrons differently, leading to directionality. Considering these factors, we propose directed relational graph attention (DRGAT) layer based on the general information propagation framework Zheng et al. (2021). as shown in Fig. 3 During message passing, DRGAT extracts source and destination node s features via independent MLPs to consider the bond direction and use the multi-head edge controlled attention mechanism consider the multi-relational properties. We add shortcut connections from the input to the output in each layer and concatenate hidden representations of all layers to form the final node representation.
4

---

Under review as a conference paper at ICLR 2022
message passing
arc; MLP (A:)
MLP
proppe (oBA))(MMLP((N))
Ays by
MLR
hi, b'
MLF
(M
(n
prop
at MLPa(eu)
SPD odges
key python code
att
and + utt
loEdge
edges(.1]
MLPP
node Sesturee__ss
AteEdge
att.s
ally
www.ster_eeaaannene www.smmeaennnnaameemmmm all dsl.immmmm
llAgg
Eess, attes
Edge feature
MLP.
Edga feature
cit
dal,
att prople
(4644)
Edge
N2 (orc)lor
Figure 3: DRGAT: Directed Relational GAT. DRGAT contains two submodules: directed message passing (DMP) and edge-controlled attention (ECA). DMP uses different MLP to learn features of the source (src) and target (dst) atoms during message passing. ECA utilizes both atom features and bond features to learn the attention weights.
Labeling and learning reaction centers We use the same labeling algorithm as G2G to identify ground truth reaction centers, where the core idea is comparing each pair of atoms in the product P with that in reactant R, We denote the atom center as G E {0, 1} and bond center as C. E {0, 1} in the product P. During the learning process, atoms features (h,) j- are learned from the product P by applying stacked DRGAT, and the input bond features are {ei, = 1}. Then, we get the representations of atom i and bond (i. as
h: h; RReddoul ( and higj eijl [h:] h Readout ( |Pl
atom representation
bond representation
where we use average Readout operation, and || is vector concatenation operation. Further, we predict the atom center probability p; and bond center probability Pr,j via MLPs:
P: M.MPPPPoo.. (h., and Pr.j
ML..
(3)
Finally, center identification can be reduced to a binary classification, whose loss function is:
C1
Σ P
logg pi + ci) log +
Cij Pi.j + ci.)) log log)
(4)
alom center
hond
In summary, we propose a directed relational graph attention (DRGAT) layer to learn expressive atom and bond features for accurate center identification prediction. We consider both atom center and bond center to provide comprehensive results. In section. 5.2,. we show that our method can achieve state-of-the-art accuracy, especially with reaction class unknown.
4.2 SYNTHON COMPLETION
Synthon completion is the main bottleneck of two-step TF retrosynthesis, which is responsible for predicting and attaching residuals for each synthon. This task is challenging because the residual structures could be complex to predict, attaching residuals into synthons may violate chemical rules, and various residuals may agree with the same synthon. Because of these complexities, previous usually inaccurate, unexplainable, and cumbersome. Introduc-
synthon completion approaches ing the necessary chemical knowledge to improve interpretabilityy and accuracy can be promising solution. However, how to provide attractive scalability and training efficiency is new challenge.
5

---

Under review as a conference paper at ICLR 2022
Semi-templates The semi-template used in this paper is the local reaction pattern of each synthon, as shown Fig. Different from GraphRetro (Somnath et al. 2021), our semi- semiltem contains not only residuals but also synthon's 's local structure. Similar the work of forward reaction predic- tion (Segler & Waller 2016), semi-template splits binary reaction into two half reactions. Notably, represent possible synthon atoms that match the semi-template, signifi- cantly reducing redundancy. We extract semi-template from each synthon-reactant pair by removing reactant atoms that have exact matches in the synthon. There are two interesting observations: 1) Top- 150 semi-templates cover 96.9% samples; 2) Reactants can be deterministically generated from semi-tempe cemiltemplates and synthons (introduced later). Based on these observations, synthon completion can be further simplified as classification problem. In other words, we need to predict the semi- template type for each synthon, and the total number classes 150+1. The first 150 classes are top-150 semi-templates, and the st class indicates uncovered classes.
dummy
full template
supporting atoms
Caf reactant
No0
semi-template
Synthon
N2 reactent
Syntton
semi-template
Figure 4: Semi-template full-template. Semi-template the local reaction template of each synthon. full-template can be decomposed into several simpler semi-templates.
Learning semi-templates For each synthon Sj. denote its semi-template label as L,, S L S
) 151. We use stacked DRGAT's to extract atom features (h, |5 from Sj- In each synthon, atoms that exactly match the semi-template are called supporting atoms, seeing circled atoms in Fig. Given the supporting atom set (io, ...... the semi-template representation is:
h, Readout (h; |S
Readout( (hy/) e {io, i1. ..))
(5)
Based on this representation, we can get the predicted semi-template i, as:
f arg max Pj.ci
Pj Softma((ML.P(ff.))
(6)
1se5151
Denote [ (+) as the indicator function, the cross-entropy loss used for training is:
== Σ Σ log((,)))
J ISB 15c5151
Applying semi-templates Once reaction centers, synthons, and corresponding semi-templates are known, we can deduce reactants with almost 100% accuracy. This is not theoretical claim; We provide a practical residual attachment algorithm the appendix and open source code on GitHub.
In summary, we suggest using the semi-templates to improve synthon completion performance. Firstly, reducing this complex task to a classification problem helps promote training efficiency and accuracy. Secondly, the high coverage of semi-tere aeiitemplates significantly enhanced the scalability of TB methods. Thirdly, the deterministic residual attachment algorithm improves interpretability and accuracy. In section. 5.3 will show the effectiveness of the proposed method.

---

Under review as a conference paper at ICLR 2022
5 EXPERIMENTS
As mentioned earlier, the main contributions of this paper are proposing a DRGAT layer for central identification and suggesting to use semi-template for synthon completion. The effectiveness of the proposed method is evaluated by systematic experiments, which focus on answering these questions:
Q1: For center identification (CI), how much performance gain can be obtained from DRGAT?
Q2: For synthon completion (SC), can semi- cemi-templates reduce template redundancy and improve the synthon completion performance? And why?
Q3: For retrosynthesis, how do we integrate CI and SC models into unified retrosynthesis frame- work? Can SemiRetro outperform existing template-based and template-free methods?
5.1
BASIC SETTING
We evaluate SemiRetro on the widely used benchmark dataset USPTO-50K (Schneider et al,, 2016 to show its effectiveness. USPTO-50K contains 50k mappedd reactions with 10 reaction types. Following (Dai et al., 2019 Shi et al. 2020 Yan et al.] 20200,, the training/valiie vaniiing/aidattion/esstt splits is 8:1:1. As mentioned in previous works, the USPTO-50k dataset contains shortcut in that the product atom with stom-mappinn "1" is part of the reaction center in 75% of the cases. In our
Data
graph-based methods, this shortcut will not introduced by forbidding atom position encoding.
Baselines Template-based GLN (Dai al.) 2019), template-free G2G (Shi et al., 2020) and RetroXpert (Yan et al. 2020) are primary baselines, which not only achieve state-of-the-ee per- formance, but also provide open-source PyTorrc code that allows verify their effectiveness. To show broad superiority. we also comapre SemiRetro with other baselines, incuding RetroSim (Coley) cet al. 20117)) NeuralSym (Segler & Waller) (017). SCROP (Zheng et al., 2019). LV-Transformer [Chen et al. 2019), GraphRetro (Somnath et al. 202T, MEGAN (Sacha et al. [202T). and MHNreact (Seidl et al. 202T). As the retrosynthesis task is quite complex, subtle implementation differences or mistakes may cause critical performance fluctuations. We prefer comparing SemiReto with open- source methods whose results are more reliable. Energy-based approaches (Sun et al.] 2020) are ignored because they are more like plug-and-play training strategies and result filters, focusing on enhancing existing models. For simplicity, we leave the use of energy function in the future and concentrate comparing original retrosynthesis models.
Metrics This paper uses consistent metrics derived from previous literature for both submodule and overall performance. 1). Center identification: We report the accuracy of breaking input product into synthons. 2). Synthon completion: We present the accuracy of generating reactants from ground truth input synthons. When a product has multiple synthons, the final prediction correct if and only if all reactants are correct. 3). Retrosytthesi:: The metric is similar to that of synthon comple- tion, except that the input synthons are also predicted by center identification. In other words, the retrosynthesis is correct if and only if both center identification and synthon completion are correct. Since there may be multiple valid routes for synthesizing a product, we report top-k accuracy.
Implementation details Thanks to the elegant implementation of G2G (Shi et al., 2020), we can develop our SemiRetro in a unified PyTorch framework (Paszke et al. 2019), namely TorchDrug. We use the same data preprocessing algorithm and code framework as G2G in center identification, ensuring the only difference is the GNN used for feature extraction. Besides, we use the open- source cheminformatics software RDkit [Landrum] 2016) to preprocess molecules and SMILES strings. The graph feature extractor consists of 6 stacked DRGAT, with the embedding size 256 for each layer. We train the proposed model for 30 and 100 epochs in center identification and synthon completion with batch size 64 on single NVIDIA V100 GPU. The training costs of different methods can be found in the appendix. We run all experiments three times and report the means of their performance in default. To avoid the label leakage Yan et al. 22000; Sommath al. 2021 ignore atom mapping numbers during the training and evaluation phases.
5.2 CENTER IDENTIFICATION (Q1)
Objective and setting This section studies how much center identification performance gain can be obtained from the proposed DRGAT, especially when the reaction class is unknown. The data preprocessing, training, and evaluating process is similar to G2G. The primary difference between our SemiRetro and G2G is the graph feature extractor, where we DRGAT while G2G uses RGCN
7

---

Under review as a conference paper at ICLR 2022
(Schlichtkrull et al. 2018). We trained our model up 30 epochs, which occupied about 2940 MB of GPU memory, where the batch size is 64, and the learning rate is le-5.
Results and analysis 1) Highest accuracy: As shown in 2 the proposed SemiRetro outperforms baselines on all the cases with different k. For example, SemiRetro achieves the highest top- 1, top2,
top-3, top-5 accuracy on both reaction class known and unknown settings. 2) Better potential: Since the possible synthesis routes toward product may be multiple, the top-k accuracy (k: > 1) is important, and the performance gain of SemiRetro rises as k: increases, indicating the better potential. In particular, SemiRetro achieves nearly perfect top-5 accuracy on the setting of reaction class known (acc = 99.4%) and unknown (acc 98.4%). 3) Better adaptability: In a more general and complex case, where the reaction class is unknown, SemiRetro can significantly exceed SOTA methods. For example, SemiRetro outperforms competitors by at least 10% the top-3 and top- 5 accuracy. These results show that the center identification performance has been dramatically improved using DRGAT, which is good first step towards accurate retrosynthesis prediction.
Top-k center identification accuracy
Reaction class known
Reaction class unknown
3
5
RetroXpert Yan 2020
86.0
64.9
GraphRetro (Somnath 2021
84.6
92.2
93.7
94.5
70.8
85.1
89.5
92.7
G2G (Shi et at. Z020
90.2
94.5
94.9
95.0
75.8
83.9
85.3
85.6
Semikelro (our)
90.9
97.2
98.5
99.4
80.5
92.8
96.0
98.4
Improvement
+0.7
+2.7
+3.6
+4.4
+4.7
+3.9
+10.7
+12.8
Table 2: Top-k center identification accuracy. The best and sub sub-optimum results are highlighted in bold and underline.
5.3 SYNTHON COMPLETION (Q2)
Objective and setting This section reveals the effectiveness of using semi-template in three folds: 1) reducing the template redundancy, 2) improving the accuracy, and 3) promoting interpretability and training efficiency. Firstly, count the full-templates of GLN and semi-templates introduced in this paper. We visualize the distribution and coverage of top-k templates for analyzing the re- dundancy. Secondly, present the accuracy of synthon completion with ground truth synthon inputs. The final reactants are obtained by predicting the semi-templates and applying the residual attachment algorithm. Thirdly, we compare the interpretability and training efficiency of different methods in short. We trained our model up to 100 epochs, which occupied about 2320 MB of GPU memory, where the batch size 64, and the learning rate 1e-4.
full-distriution
semi-disiribution
full-coverage
semi-coverage
0.8949
0.0025
1.0
0.25
0.8
0.8949
0.0020
0.8
0.20
probabilty
cofer rat
prgbability
0.00015
0.15
oover
0.10
9.0010
1030664
0.2673
0.2
0.05
0.00
0.0
0.0000
0.0
20
30
40
50
50
100
150
200
250
300
350
400
450
500
(a) top 1-50
(b) top
50-150
Figure 5: SemiRetro reduces the template redundancy.
lop-k synthon completion accuracy
Reaction class known
Reaction class unknown
5
10
10
G2G (Shi et al. 2020
66.8
87.2
91.5
93,9
61.1
81.5
86.7
90.0
Semiketro
70.2
87.7
91.5
93.2
66.3
85.6
90.0
92.7
Improvement
+3.4
40.5
40.0
+5.2
+4.1
43.3
+2.7
Table 3: Top-k synthon completion accuracy.
8

---

Under review as a conference paper at ICLR 2022
Results and analysis (1) Reduce redundancy: In Fig. 5 show the distribution and coverage of top-k full-templates and semi-templates, where the former distribution sharper than the latter, indicating higher top-k coverage. For example, the top-150 remi-templates cover the case of 96.87% while the full-templates only cover 39.64%. Using semi-templates can reduce 1 11,647 full- templates into 150 semi-templates and increase the cover rate from 93.3% to 96.87% (2) Higher accuracy As shown in Table. 3 SemiRetro outperforms G2G in most cases. This improvement comes from two parts: Firstly, semi-templates reduce the difficulty of predicting residual structures; Secondly, semi-templates eliminate the problem of residual attachment by using deterministic algorithm. When the reaction class is unknown, the improvement more dramatic. (3) More interpretable and efficient By using semi-templates, the residual attachment process is controllable and interpretable. In addition, our model can be trained least 8 times faster than previous synthon completion models such as GLN, G2G, and RetroXpert, seeing the appendix for details.
5.4 RETROSYNTHESIL (Q3)
Objective and setting We explain how to combine center identification and synthon completion to provide end-to-end retrosynthesis predictions. We use a probability tree search the top-k results, seeing Fig. 6 where the probability product of two-step predictions is used to rank these results.
Input
Step center identification
Top2 prediction
Step2: synthon completion
Tap3 prediction
P
Rank
p=0.3
0.15
2
p=0.5
Synthon completion model
p=0.2
S
0.10
p=0.1
0.05
Product molscull/
identfiiatiin model
p=0.4
10
R
0.16
p=0.4
Synthon completion
p=0.3
S
0.12
p=0.2
0.08
5
Figure The retrosynthesis example: combining top-2 CI and top-3 SC obtain top-6 retrosynthesis results. Note that S, indicates the i-th synthon, and R(2) is the j-th predicted reactant of S,.
accuracy
Reuction class known
Reaction class unknown
1
3
5
10
1
3
5
10
RetroSim (Coley et al
7b
52.9
73.8
81.2
88.1
37.3
54.7
63.3
74.1
NeuralSyni (Segier & Water 2017
55.3
76.0
81.4
85.1
44.4
65.3
72.4
78.9
GLN DDii al 2019
64.2
79.1
85.2
90.0
52.5
69.0
75.6
83.7
SCKOP (Zhnng al 2019
59.0
74.8
78.1
81.1
43.7
60.0
05.2
68.7
LV-Trumstommr
2019
40.5
65.1
72.8
79.4
G2G (Shi et al. 20z0
61.0
81.3
86.0
88.7
48.9
67.6
72.5
75.5
TF
Retroxpai ae 2020
62.1
75.8
78.5
80.9
50.4
61.1
62.3
63.4
GraphRetro
2021
63.9
81.5
85.2
88.1
53.7
68.3
72.2
75.5
MEGGAN (Sacha el ai.
2021
60.7
82.0
87.5
91.6
48.1
70.7
78.4
86.1
MHNreacT (Sedi (SeNd
et 2021
50.5
73.9
$1.0
87.9
SemiRetro
87.6
90.2
55.3
76.5
81.4
Our Improvmenn to GLN
+0.2
+4.6
+2.4
+0.2
+2.8
+7.5
+5.8
+1.7
Improvement G2G
+3.4
+2.4
+1.6
+1.5
+4.9
+8.9
+8.6
+6.0
Table 4: Overall performance. The best and sub-optimum results are highlighted in bold and underline. Only open source works participate in rigorous comparisons, such GLN, G2G. and RetroXpert.
Results and analysis (1) Higher accuracy: SemiRetro achieves the highest accuracy in most settings, seeing Table. 4 As to previous works with open source code, template-free G2G and RetroXpert are more scalable than template-based GLN while sacrificing the top-1 accuracy. We use semi-template to reduce the template redundancy and improve the accuracy simultaneously. (2) Better adaptability We observe the performance gain of SemiRetro increases when the reaction class is unknown, suggesting method can work better in general and complex cases.
6 CONCLUSION
We propose SemiRetro for retrosynthesis prediction, which achieves better accuracy and attractive scalability than previous methods. Specifically, the DRGAT achieves the highest center identifica- tion accuracy. The semi-template improves both the accuracy and scalability of synthon completion. Moreover, SemiRetro has favorable interpretability and training efficiency. We hope this work will promote the development of deep retrosynthesis prediction.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Javier L Baylon, Nicholas A Cilfone, Jeffrey R Gulcher, and Thomas W Chittenden. Enhancing ret- rosynthetic reaction prediction with deep learning using multiscale reaction classification. Journal of chemical information and modeling, 59(2):673-688 2019.
Benson Chen, Tianxiao Shen, Tommi S Jaakkola, and Regina Barzilay. Learning to make general- izable and diverse predictions for retrosynthesis. arXiv preprint arXiv:1910.0963. 2019.
Connor W Coley, Regina Barzilay, Tommi S Jaakkola, William H Green, and Klavs Jensen. Pre-
diction of organic reaction outcomes using machine learning, ACS central science, 3(5):434-443 2017a.
Connor W Coley, Luke Rogers, William H Green, and Klavs F Jensen. Computer-assisted retrosyn- thesis based on molecular similarity. ACS central science, 3(12):1337-1255 2017b.
Connor W Coley, William H Green, and Klavs F Jensen. Machine learning in computer-aided synthesis planning. Accounts of chemical research, 51(5):1281-1289, 2018.
Elias James Corey. The logic of chemical synthesis: multistep synthesis of complex carbogenic molecules (nobel lecture). Angewandte Chemie International Edition in English, 30(5):455 465, 1991.
Elias James Corey and W Todd Wipke. Computer-assisted design of complex organic syntheses. Science, 166(3902):1788 178-19,, 1969.
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. Retrosynthesis prediction with conditional graph logic network. Advances Neural Information Processing Systems, 32:8872- 8882, 2019.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks graphs with fast localized spectral filtering. Advances in neural information processing systems, 29:3844-3852, 2016.
Markus Hartenfeller, Martin Eberle, Peter Meier, Cristina Nieto-Oberhuber, Karl-Heinz Altmann, Gisbert Schneider. Edgar Jacoby, and Steffen Renner. A collection of robust organic synthesis reactions for in silico molecule design. Journal of chemical information and modeling, 51(()): 3093-3098, 2011.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Pavel Karpov, Guillaume Godin, and Igor Tetko. A transformer model for retrosynthesis. In International Conference on Artificial Neural Networks, pp. 817-830. Springer, 2019.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net-
works. arXiv preprint
2016.
Greg Landrum. Rdkit: phoen--ourceecceemifffnnnon software. 2016. htLp:tts::///is
com/ rdkit/ http://ww../ckttteelaaaaeee/ssss
2016
09
James Law, Zsolt Zsoldos, Aniko Simon, Darryl Reid, Yang Liu, Sing Yoong Khew, A Peter John- son, Sarah Major, Robert A Wade, and Howard Y Ando. Route designer: retrosynthetic anal- ysis tool utilizing automated retrosynthetic rule generation. Journal of chemical information and modcling, 4963):593-602, 2009.
Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, and Vijay Pande. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS central science, 3(10):1033-1113, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32: 8026-8037, 2019.
10

---

Under review as a conference paper at ICLR 2022
Mikolaj Sacha, Mikolaj Blaz, Piotr Byrski, Pawel Pabrooskki-ummnnki Mikolaj Chrominski, Rafal Loska, Pawel Wiodarczyy--Pruzzyssti and Stanistaw Jastrzebski. Molecule edit graph atten- tion network: modeling chemical reactions sequences of graph edits. Journal of Chemical Information and Modeling, 61(7):3273-3288 2021.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. European semantic web conference, pp. 593-607. Springer, 2018.
Nadine Schneider, Nikolaus Stiefl, and Gregory A Landrum. What's what: The (nearly) definitive guide to reaction role assignment. Journal of chemical information and modeling, 56(12):2336 2346, 2016.
Philippe Schwaller, Theophile Gaudin, David Lanyi, Costas Bekas, and Teodoro Laino. "found in translation": predicting outcomes of complex organic chemistry reactions using neural sequence- to-sequence models. Chemical science, (728))6091-6099) 2018.
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Costas Bekas, et al. Molecular transformer for chemical reaction prediction and uncertainty estimation. 2019.
Marwin HS Segler and Mark Waller. Modelling chemical reasoning to predict reactions. arXiv preprint arXiv: 10XKv:1608.000 160..668111111 07117, 2016.
Marwin HS Segler and Mark P Waller. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry-A European Journal, 23625):5966-5771, 2017.
Marwin HS Segler, Mike Preuss, and Mark Waller. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):004-610, 2018.
Philipp Seidl, Philipp Renz, Natalia Dyubankova, Paulo Neves, Jonas Verhoeven, Marwin Segler, Jörg K Wegner, Sepp Hochreiter, and Günter Klambauer. Modern hopfield networks for few-and shoo-shot reaction template prediction. arXiv preprint arXiv:2104.0322 2021.
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph graphs framework for retrosynthesis prediction. In International Conference on Machine Learning, pp. 8818-8822 PMLR, 2020.
Vignesh Ram Somnath, Charlotte Bunne, Connor W. Coley, Andreas Krause, and Regina Barzi- lay. Learning graph models for retrosynthesis prediction. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021. URL
SnONpX2_ude _Q_
Ruoxi Sun, Hanjun Dai, Li Li, Steven Kearnes, and Bo Dai. Energy-based view of retrosynthesis. arXiv preprint arXiv:2007.13437 2020.
Sara Szymkuć, Ewa Gajewska, Tomasz Klucznik, Karol Molga, Piotr Dittwald, Michal Startek, Michat Bajezyk, and Bartosz A Grzybowski. Cosputer-assisted synthetic planning: the end of the beginning. Angewandte Chemie International Edition, 55(20):5904-59) 2016.
Igor V Tetko, Pavel Karpov, Ruud Van Deursen, and Guillaume Godin. State-of-the-ard augmented nlp transformer models for direct and single-step retrosynthesis. Nature communications, 11(1): 1-11, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint 1rXiv 2017.
David Weininger. Smiles, chemical language and information system. 1. introduction method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36 1988.
11

---

Under review as a conference paper at ICLR 2022
Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, JINYU YANG, Yang Yu, and Junzhou Huang. Retroxpert: Decompose retrosynthesis prediction like a chemist. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 11248-11258, Curran Asso- ciates, Inc., 2020. /ttpps:////////eeeeete neweeerriiiinnn..nneee.eeeee newr iss. cc//paper/2020/ff1ee pdf
81
Qinkai Zheng. Houyi Li, Peng Zhang, Zhixiong Yang, Guowei Zhang, Xintan Zeng, and Yongchao Liu. Gipa: General information propagation algorithm for graph learning. arXiv preprint arXiv:2105000035, 2021.
Shuangjia Zheng, Jiahua Rao, Zhongyue Zhang, Jun Xu, and Yuedong Yang. Predicting ret- rosynthetic reactions using self-corrected transformer neural networks. Journal of Chemical Information and Modeling, 60(1):47-55, 2019.
12

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
Center identification We show top-2 center identification predictions in Fig. 7 where synthons obtained from breaking edge centers for downstream synthon completion. We present the prob- ability of each prediction where the total probability of top-2 predictions exceeds 98%, indicating strong inductive confidence. Since the top-2 predictions are accurate enough, seeing Table. 2 we use them for synthon completion.
Case ground Iruth is stom center
Tap1 prediction (p Thue prodicdio
Tap2 prediction
Faloo prodicton
Case2: ground truth is edge center
Top1 prediction (p 800
Top2 prediction (p 184%)
Figure 7: Visualize results of center identification. Casel: the ground truth is atom center, and the top- prediction is correct with the probability 99.3%. Case2: The ground truth is edge center, and the top-2 prediction is correct with the probability 18.4%.
Synthon completion In Fig. we present the process of predicting multiple reactants of the same product. This process provides an end-to-end view of synthon completion, containing semi- template prediction, top-k results search, and semi-template application. By default, we choose the top-5 synthon completion results for each center identification output as part of the final top- 10 retrosynthesis results.
Na
Synthon
unggeetty
Reactant
HD
C36
C20
C2S C24
Synthon
Reactant
Figure The overall pipeline synthon completion. The input synthons are the outputs of the center iden- tification module, coming from the same product molecule. We get the top-5 semi- template predictions and their probabilitics of each synthon using SemiRetro (synthon completion network), then generate the joint dis- tribution of semi-templates. We choose the top-5 predictions from this joint distribution and apply the residual attachment algorithm (introduced later) to get the final reactants.
13

---

Under review as a conference paper at ICLR 2022
Table 5: Residual attachment algorithm. For easy and quick understanding, demonstrate the idea by visual samples. The detailed implementation can be found open-source code.
Input: Synthon, reaction center, and semi-template
Output: Reactant oblained by applying the semi-template on the synthon
Step1: map atoms within the template
100
101
301
Step2: map bonds within the template
$100
101
1100
101
1:10
10
Step3: match template with the synthon
Constraint: reaction center must be within the matching area
Step4: Attach the right template into the synthon
100
remove the left template
add the right template
final result
The platform for experiment is ubuntu 18.04, with Intel@ Xcon@ Gold 6240R Processor and 256GB memory. We use single NVIDIA V100 to train models, where the CUDA version is 10.2.
Platform
Retrosynthesis
Center identification
Synthon completion
GLN
RetroXpert
G2G
SemiRetro
RetroXpert
G2G
SemiReero
time/epoch
785s
440s
58s
48s
330s
322s
40s
GPU memory/sample
274.7MB
55.7MB
46.1MB
45,9MB
147.7MB
65.7MB
36.3MB
total epochs
80
100
different methods. We the open-source code of these methods on the same platform, reporting the training time per epoch and occupied GPU memory per sample. We also show the total training epochs mentioned in their paper (preferred) or code. If the author reports training steps, we calculate epochssotal stepssotal /ttoosassssttttstsseee sttepaatrrvll.
Table The training
14

---

Under review as a conference paper at ICLR 2022
Important details We follow the setting of G2Gs, which provides open-source code hawpaa/oooooeeerepa.a.aaannggee
G2Gs use different atom features in their open- source code for center identification and synthon completion. In this paper, features of G2Gs. We have also tried to combine all these atom features and use the same set of features in center identification and synthon completion models. The combined atom features do not make a significant difference.
the same atom
Name
Description
Atom type # Hs Degree Valence Aromaticity Ring
Type of atom (ex. C, N, O), by atomic number one-ho hot embedding for the total number of Hs (explicit and implicit) on the atom one-hot embedding for the degree of the atom in the molecule including Hs one-hot embedding for the total valence (explicit + implicit) of the atom Whether this atom part of aromatic system. whether the atom is in ring
Table 7: Atom features for center identification.
Name
Description
Bond type Bond direction
one-hot embedding for the type of the bond one-hot embedding for the direction of the bond one-hot embedding for the stereo configuration of the bond whether the bond is considered to be conjugated the length of the bond
Stereo
Conjugation
Bond length
Table Bond features for center identification.
Name
Description
Atom type
Type of atom (ex. N, O), by atomic number
# Hs
one-hot embedding for the total number of Hs (explicit and implicit) on the atom one-hot embedding for the degree of the atom in the molecule including Hs whether the atom is in a ring
Degree
Ring
Ring 3
whether the atom is in a ring of size 3 whether the atom is in a ring of size whether the atom is in a ring of size 5 whether the atom is in a ring of size whether the atom is in ring of size larger than 6
Ring 4
Ring 5
Ring 6
Ring 6+
Table Atom features for synthon completion. Note that synthon completion do not use bond features.
15


Reviews:
[{"review_id": "mzXxGuQYRk", "reviewer": "Reviewer_XCBM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths\nS1. The approach obtains superior performance (wrt accuracy) compared to many previous retrosynthesis methods. \n\nS2. The \"semi-templates\" used can cover a larger space of reactions compared with using a similar number of full templates (Figure 5). This leads to improved training efficiency.\n\nS3. The method is faster to train and takes up less GPU space (which I assume also means it has less parameters...?) than many previous methods. (Table 6).\n\nS4. I found the paper clear to read. In particular the figures were helpful for explaining the approach.\n\n# Weaknesses\nW1. A large part of the performance of this approach seems to stem from DRGAT's superior reaction center prediction (compared to e.g. the R-GCN used by Shi et al., 2020) -- in particular see Table 2. However, it is not clear exactly where this performance comes from: as well as introducing an edge based attention mechanism the authors also use more layers than the R-GCN. It would be helpful to have an ablation study. How does it compare generally to well-tuned, off-the-shelf GNNs...?\n\nW2. I have concerns about the novelty of the paper. SemiRetro seems quite similar to Somnath et al. (2020)'s GraphRetro (that method also has a two step process that first predicts the reaction center to create synthons and then adds \"leaving groups\" to create reactants). I would like to have seen a more detailed description of how the work here differs? For instance, is predicting semi-templates distinctly different to predicting leaving groups?\n\nW3. SemiRetro's training efficiency and scalability are highlighted as one of the advantages of the method (see e.g. abstract) and the speed improvements listed in Table 6 seem impressive. It would therefore have been nice to see experiments on larger retrosynthesis datasets, e.g. the larger USPTO one used by Dai el al. (2019), to show the practical advantages of this.\n\nW4. In the retrosynthesis experiments there are missing comparisons to state-of-the-art (SOTA) methods, e.g.:  \n> Sun, Ruoxi, et al. \"Energy-based View of Retrosynthesis.\" arXiv preprint arXiv:2007.13437 (2020).  \n\n> Sacha, Mikołaj, et al. \"Molecule edit graph attention network: modeling chemical reactions as sequences of graph edits.\" Journal of Chemical Information and Modeling 61.7 (2021): 3273-3284.    Preprint: https://arxiv.org/abs/2006.15426\n\nIt would be good to have these included.\n\n# Questions to authors\nQ1. Can you compare to other synthon completion methods in Table 3. For instance, Somnath et al. (2020)?\n\nQ2. How do you make use of the reaction class when it is known? Do you just use it to mask out inapplicable templates or do you also provide the information to the reaction center network?\n\nQ3. In the right hand subplot of Figure 5, the dotted green light appears to end above 1. Is this correct?\n\nQ4. I'm a little confused by equation 5, in particular the second readout term. Are the node embeddings, $\\mathbf{h_k}$, taken from message passing run on (a) the template graph or (b) from the synthon graph?\n\nQ5. On p.6 you state: \"Once reaction centers, synthons, and corresponding semi-templates are known, we can deduce reactants with almost 100% accuracy.\" Why _\"almost\"_ 100%? When does it fail?\n\nQ6. I would be interested to have more details about why you think SemiRetro has better \"interpretability\" than previous approaches? (Previous approaches have also used templates, and the reaction center prediction here still seems to be done in a black box manner?)\n\n# Typos/Minor Comments\nA few minor comments I had (all easily fixable) are:  \ni. p.1 \"(seeing Fig. 1)\" to \"(see Fig. 1)\" .  \nii. Figure 5 -- the subcaptions are both labeled a.  \niii. p.6 \"Reactants can be determinately generated from semi-templates and synthons\": \"determinately\" should be \"deterministically\"?  \niv. Sum term in yellow box on right-hand part of Figure 3, I believe you mean $i \\in \\mathcal{N}_j$?  \nv. p.7 \"As mentioned in previous works, the USPTO-50k dataset contains a shortcut in that the product atom with atom-mapping ”1” is part of the reaction center in 75% of the cases.\". It's great that you draw attention to the issues with information leakage and atom mapping. However, perhaps provide a reference to the previous works for non-expert readers.  \nvi. p.7 \"Implement details\" to \"Implementation details\".  \n", "summary_of_the_paper": "* This paper is interested in the problem of one-step retrosynthesis, i.e., predicting the reactants that formed a given product.\n* For this task, the authors propose a new model (SemiRetro) that combines ideas from previous template-based (TB) and template-free (TF) methods. \n* SemiRetro consists of two parts:  \n\ti. First, DRGAT (a new GNN architecture based on edge attention) predicts the reaction center, and breaks down the product molecule into synthons.  \n\tii. Second, \"semi-templates\" (chosen from a fixed set by a classification model) is used to complete the synthons and assemble the reactants.\n* The authors show:\n\t1. DRGAT works much better than previous methods for identifying reaction centers (specifically the R-GCN used in Shi et al. (2020)'s G2G) -- see Table 2.\n\t2. That the use of semi-templates allows SemiRetro to cover a wider range of reactions than previous template-based methods while also using less unique templates -- see Figure 5.\n\t3. SemiRetro outperforms many previous TB and TF methods in terms of top-k accuracy, particularly in the reaction class unknown case -- see Table 4.\n", "summary_of_the_review": "SemiRetro seems an interesting method for retrosynthesis that combines ideas from both template-based and template-free approaches. It also seems to get impressive empirical performance (with the caveat that some SOTA methods are missing from the comparison -- see W4). However, I have currently gone with a lower overall score as I worry the method is similar to previous work (see W2) and I feel the paper could do a better job at explaining where the performance gains come from (see W1).\n\nThe other scores:\n- correctness: see Q6 & W3. \n- technical novelty: see W2.\n- empirical novelty: the paper mainly makes use of existing benchmarks/evaluation techniques.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hNADFaCBwi", "reviewer": "Reviewer_83gD", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(Sorry I re-organize the review to meet the guideline. No changes in the contents/opinions from the initial submission.)\n\n### points\n\n(+): a new variant of retrosynthesis problem using a local-structure aware semi-template\n\n(++): good numerical performances confirmed in the sub-problems and the main retrosynthesis\n\n(-): insufficient explanations of DRGAT\n\n(-): possible missing reference and comparison\n\n(-); visibility of figures are generally low\n\n### comments\n\nI like the idea of this paper, and enjoy reading the manuscript. \n\nIn my understanding, the main stream of the proposed model follows the typical template-free retrosynthesis. \nIn the current TF approaches, the center identification sub-problem affects a lot in the final retrosynthesis problem. \nThe proposed DRGAT seems performing great in this sub-problem (Sec. 5.2). \nHowever, the explanation of the DRGAT model only appears in the Figure 3. I expect the revised manuscript add a few sentences/equations to convey the main idea of the model. \n\nThe semi-template model for synthon completion is quite interesting. \nIt seems for me a good alternative of the GraphRetro's residual prediction approach. \nQuestion: Why GraphRetro and RetroXpert are not studied in the Sec. 5.3? In terms of the synthon completion, GraphRetro's residual modeling may perform as good as the proposed semi-template.  \n\nI found that one of the important previous work is missing from the discussions and the experiments; energy-based models.\nPlease discuss the relationship with the proposed model, and if it is appropriate, compare in experiments. \n\n@misc{\nsun2021energybased,\ntitle={Energy-based View of Retrosynthesis},\nauthor={Ruoxi Sun and Hanjun Dai and Li Li and Steven Kearnes and Bo Dai},\nyear={2021},\nurl={https://openreview.net/forum?id=0Hj3tFCSjUd}\n}\n\n\nReadability: In general, figures are low-visibility. Especially, atoms, bonds, molecules, equations are not displayed well on screens. \nI hope these figures are improved in the camera ready as much as possible. For example, thick/strong lines and fonts, larger symbols, and so on. ", "summary_of_the_paper": "This paper studies a retrosynthesis problem of chemical reactions. \nThis paper proposes a new variant of retrosynthesis models, with mainly two new components: DRGAT for powerful embedding computation and the semi-templates for accurate synthon completions. \nEspecially, the propose semi-template takes the local (neighboring) bonding structure into consideration. This significantly reduces the redundancy of the templates, allowing connecting the template-based retrosynthesis (less scalable but powerful) and the template-free retrosynthesis (scalable but less powerful). \nThe experimental validations show the efficacy of the new components for each subtask, and the final retrosynthesis performances exceeds the existing retrosynthesis models. ", "summary_of_the_review": "It is great to see the stead improvement of the retrosynthesis. \nThe semi-template idea sounds reasonable and good to me. \nThe explanations on the DRGAT is clearly not enough, and in general, figures are hard to see on the screen. \nMy positive score is with the expectation that the insufficient explanations and low-visibility figures are properly addressed in revisions or the camera-ready. \n\n=============================================================\n\nAfter author feedback;\nThank you again authors for the efforts to revise the manuscript and discussions. \nOne of the reviewers raise the concern about the leakage problem of the USPTO dataset, and I agree the problem is important. \nSome strong competitors are also introduced by other reviewers. \nAs a result, we observe a few but important updates that are not reflected in the revised manuscript. \nI still believe the proposed semi-template approach is new and valuable to the community, but this time I cannot recommend this manuscript for acceptance. I expect the authors to come up with a manuscript including all new updates in near-future conferences. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aq6d6hdCGxq", "reviewer": "Reviewer_G14H", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*strengths*\n- good results\n- reasonable model\n\n*weaknesses*\nablation of representation is missing: It could be argued that the most gains are from the new GNN architecture. It could be instructive to try out the same model with the same architecture that G2G, GLN, or NeuralSym uses. Or maybe with DRGAT,  G2G, GLN, or NeuralSym would also perform better?\n\nbaselines is missing: I would suggest to add\n- Sacha et al - MEGAN model https://arxiv.org/abs/2006.15426\n- Sun et al https://arxiv.org/abs/2007.13437 \n- Seidl et al https://arxiv.org/abs/2104.03279 to the results table, which outperforms SemiRetro in top10 accuracy.\n\n\n\nnotes:\nIn the introduction, the authors write:\n`Fortunately, with the rapid accumulation of chemical data, machine learning is promising to solve this problem (Szymkuc et al., 2016; Coley et al., 2018; Segler et al., 2018).` - However, Szymkuc et al 2016 argue *against* the use of data-driven approaches, so I would suggest to not to cite them in this context.\n", "summary_of_the_paper": "A new algorithm for retrosynthesis prediction based on a new GNN architecture and semi templates is presented, which yields state of the art results on some benchmark tasks.", "summary_of_the_review": "interesting model, experiments could be stronger", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "UNgFnZcwFX", "reviewer": "Reviewer_XXdN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nThe paper is well organized and written. Easy to follow.\n\nThe authors propose to break full-template into several simpler semi-templates, and conduct synthon completion based on the semi-templates. The template redundancy can be reduced since many semi-templates are reduplicative. As a result, they can obtain 150 semi-templates that cover 96.9% of reactions. \n\nTo better recognize the reaction center, a directional relational graph attention layer (DRGAT) is introduced to extract more expressive molecular features.\n\nThe proposed method achieves the SOTA 64.4% and 55.3% top-1 accuracy for reaction class known and unknown settings, respectively. Besides, the authors also show the detailed performance of two steps of the retrosynthesis method.\n\n\nWeaknesses:\n\nThe proposed semi-template method is interesting, however, it has already been published in a journal article [1]. The authors of [1] name the semi-template as the local reaction template. If I understand them right, the semi-template in this paper and local reaction template are the same stuff. Please correct me if I am wrong. If that is the case, it will make the paper less significant. \n\nThe presented DRGAT layer is based on the previous framework, what is the major contribution or novelty of DRGAT that comes from the authors? Could the authors clarify that?\n\nWhat is the detailed composition of atom and bond features mentioned in Table 1? I can not find them in the submission file.\n\nHave the authors removed the original mapping numbers from test data? If not, could the authors run inference without the original mapping numbers, and confirm the reported results can be reproduced? As the authors mentioned, this is an information shortcut with the USPTO dataset. Sometimes, it is really difficult to recognize the shortcut. So it is best to completely remove the potential of shortcuts by replacing the original mapping numbers for test data.\n\nThe proposed method seems quite generalizable, could the authors also report results on the USPTO-full dataset like GLN to demonstrate the scalability of the proposed method?\n\n[1] Deep Retrosynthetic Reaction Prediction using Local Reactivity and Global Attention. Shuan Chen and Yousung Jung. JACS Au 2021 1 (10), 1612-1620.\n\n", "summary_of_the_paper": "This paper presents a novel template-based retrosynthesis prediction method. The authors reduce template redundancy and obtain 150 abstract semi-templates, which can cover 96.9% of data samples. Besides, the authors propose directed relational graph attention to find the reaction center. Experiments show the proposed method can achieve superior retrosynthesis performance.\n", "summary_of_the_review": "The proposed method has achieved impressive experimental results on the small dataset USPTO-50K. The proposed semi-template is novel and effective, however, it has been published in previous work. The presented DRGAT is quite incremental. The synthon completion method is the same as the GraphRetro. The overall framework is very similar to previous work G2Gs and GraphRetro. In summary, except for the performance boost, no new ideas or methods are proposed in this paper, and the contribution and novelty of this paper are limited.\n\n*****************************************\nAfter reading the author's response. I am quite confident there is an information leak in both G2Gs and this work. The authors did not canonicalize the test SMILES during model evaluation which is not a correct way to evaluate their method. They can not reproduce the reported results with canonicalized test SMILES. Please the authors canonicalize the test SMILES and update experimental results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new semiretro algorithm by combining the two major approaches of retrysynthesis, the template based method and the template free method - breaking a full-template into several semi-templates and embedding them into the two-step template-free framework.  They also obtained state of the art performance in this task based on the recent GNN architecture. Although all reviewers were satisfied with the idea and excellent performance of this paper, the possibility of information leakage in their experiments was raised through the discussion period, and the authors seem to agree to this to some extent.

In conclusion, it is difficult to say that accurate and rigorous experimental verification of the proposed method has been made yet. I encourage the authors to resubmit the paper after correcting errors in their experiments.
Decision: Reject