Data Transfer Approaches to Improve Seq-to-Seq Retrosynthesis
Authors: Katsuhiko Ishiguro
Kazuya Ujihara
Ryohto Sawada
Hirotaka Akita
Masaaki Kotera
Venue:  (2021)
Keywords: retrosynthesis
data transfer
transfer learninig
pre-training
fine-tuning
self-training

Abstract: Retrosynthesis is a problem to infer reactant compounds to synthesize a given
product compound through chemical reactions. Recent studies on retrosynthesis
focus on proposing more sophisticated prediction models, but the dataset to feed
the models also plays an essential role in achieving the best generalizing models.
Generally, a dataset that is best suited for a specific task tends to be small. In
such a case, it is the standard solution to transfer knowledge from a large or
clean dataset in the same domain. In this paper, we conduct a systematic and
intensive examination of data transfer approaches on end-to-end generative models,
in application to retrosynthesis. Experimental results show that typical data transfer
methods can improve test prediction scores of an off-the-shelf Transformer baseline
model. Especially, the pre-training plus fine-tuning approach boosts the accuracy
scores of the baseline, achieving the new state-of-the-art. In addition, we conduct a
manual inspection for the erroneous prediction results. The inspection shows that
the pre-training plus fine-tuning models can generate chemically appropriate or
sensible proposals in almost all cases.

Under review as a conference paper at ICLR 2021
DATA TRANSFER APPROACHES TO IMPROVE SEQ-TO- SEQ RETROSYNTHESIS
Anonymous authors Paper under double-blind review
ABSTRACT
Retrosynthesis problem infer reactant compounds to synthesize a given product compound through chemical reactions. Recent studies on retrosynthesis focus on proposing more sophisticated prediction models, but the dataset to feed the models also plays essential role in achieving the best generalizing models. Generally, dataset that best suited for specific task tends to be small. such a case, it is the standard solution to transfer knowledge from large clean dataset the same domain. In this paper, we conduct a systematic and intensive examination of data transfer approaches end-to-end generative models, in application to retrosynthesis. Experimental results show that typical data transfer methods improve test prediction scores of off-the-shelf Transformer baseline model. Especially, the pre-training plus fine-tuning approach boosts the accuracy scores of the baseline, achieving the new state-of-the-art, In addition, we conduct a manual inspection for the erroneous prediction results. The inspection shows that the pre-training plus fine-ture finettuning models can generate chemically appropriate or sensible proposals in almost all cases.
INTRODUCTION
Retrosynthesis, first identified by Corey Wipke (969)), is a fundamental chemical problem to infer a set of reactant compounds that can be synthesized into desired product compound through series of chemical reactions. The search space of sets of compounds is innately huge. Further, product compound can be synthesized through different series of reactions from different reactant compound sets. Such difficulties require the huge efforts of human chemical experts and the large knowledge base to build retrosynthesis engine for years. Thus, expectations of machine-learning (ML) based retrosynthesis engines is growing in recent years. The need for the retrosynthesis becomes intensive in these days along with the development of in silico (computational) chemical compound generations (Jin et al. 2018 Kusner et al. 2017), which are also applied to new drug discovery for COVID-19 (Cantürk et al. 2020; Chenthamarakshn
al.. 2020). These generation models can generate unseen compounds in computers but do not answer how to synthesize them in practical. Retrosynthesis engines can help chemists and pharmacists fill this gap.
Practical retrosynthesis planning requires strong model to learn inherent biases in the target dataset while keeping generalization performance to generate unseen (test) product compounds. The current trend is to focus on developing such a strong ML model architecture such as seq-to-seq models (Liu
al. 2017 Karpov et al., 2019) and graph-to-graphs models (Shi et al. 2020 Yan et al. 2020 Somnath et al. 2020), which achieve the State-of thethf--tArtt (SotA) retrosynthesis accuracy.
the only issue to consider. In the current deep neural network (DNN) era, the quantity (many samples) and the quality (less noisy, corrupted samples) of the available dataset often governs the final performance of the MI. model. The problem is that there are only few large and high-quality supervised training datasets that are available publicly. Instead, only a small and/or a noisy dataset is usually available for the target application task. To cope with this problem, data transfer approaches are widely employed as ordinary research pipelines in computer vision (CV), natural language processing (NLP), and machine translation (MT) domains (Kornblith et al., 2019 Xie al. 2020 He et al. 2020 Khan et al. 2019: Sennrich et al.) 2016). A data transfer approach tries to transfer knowledge to help a difficult training on the small and/or noisy target dataset. That knowledge is imported from augmented dataset, which is usually a large or
However, model architecture

---

Under review as a conference paper at ICLR 2021
Y: Reactant(s)
X: Product(s)
Form. p(M(); Retrosynthesis model
Graph representations
HgN
M(]|): Synthesis
(Somnath al., 2020)
(Yan et al., 2020)
(Shi al., 2020
xX Retrosynthesis
SMILES representations
(I.iu et al., 2017)
ccccICv-O.CCccCIccCC....
Ccoc(calecCCO)ccce1000
(Kurpov et al., 2018)
Figure I: Overview of the retrosynthesis problem.
clean dataset in the same domain but does not share the same lask the same assumptions with the target dataset. Such augmented datasets are beneficial if the quantity or the quality of the augmented dataset superior compared to the target dataset. However, this data transfer approach still well investigated in the previous retrosynthesis studies as we explained above.
In this paper, we conduct systematic investigation of the effect of data transfer for the improvement of the retrosynthesis models. We examine three standard methods of data transfer: joint training, self-training, and pre-training plus fine-tuning, The result shows that every data transfer method can improve the test prediction accuracy of an off-the-shelf Transformer retrosynthesis model. Especially, Transformer with pre-training plus fine-tuning achieves comparable performance with, and in some cases better performance than, the SotA models. In addition, conducted an intensive manual inspection for the erroneous prediction results. This inspection clarifies the limitations of our approaches. But at the same time, it reveals that the pre-training plus fine-tuning model generate chemically appropriate or sensible proposals more than 99% cases of top-1 predictions.
2
RELATED WORKS
2.1 MACHINE LEARNING-BASER RETROSYNTHESIS
Computers are used to aid retrosynthesis design for decades, dating back to Corey & Wipke (1969). However, it takes many years see the rise of generalizable and scalable MI-baase approaches, instead of rule- bile-based approaches (e.g. Chen & Baldi (2009)), which cannot extrapolate beyond the rules given, and first principle-based approaches (e.g. Wang et al. (2014), which suffer from prohibitively heavy computations.
The most recent ML retrosynthesis employs Deep Neural Network (DNN) for its main component (e.g. (Wei et al.) 2016)). Among them, Liu et al. (2017) first introduced the LSTM seq-to-seg model. Their model handled the compounds with the SMILES (Weininger) (988)-formal string representation, and the retrosynthesis problem was solved as the MT problem. Later, Karpov et al. (2019) replaced the LSTM with the Transformer (Vaswani et al. 2017). which is the current baseline seq-to-seq DNN, and achieves a good performance.
Recently, Shi et al. (2020) introduced graph-to-graphs approach. Graph-to-graphs approach can treat the compounds molecular graphs with help of graph neural networks. This approach well matches human experts' intuition and the very recent model (Somnath et al. 2020) greatly improved the accuracy, outperforming the former Atate-of-the-Art (SotA) model based on the deep logic network (Dai et al. 2019).
As seen, most efforts of ML-based retrosynthesis researches are dedicated stronger DNN archi- tectures. In this paper, we propose another approach to improve the SotA of retrosynthesis, i.e., by transferring the knowledge of additional datasets that are not directly prepared for the users' target tasks.
2.2 DATA TRANSFER IN GENERAL ML DOMAINS
Generally, the performances of the statistical ML model dependent the design of the model (variable dependencies) and the datasets. In the dataset side, the both of the quality (fewer noises in features, less mislabels conflicting samples) and the quantity (many sample sizes, not-few and
2

---

Under review as a conference paper at ICLR 2021
not-too- -any numbers of class labels) of the training dataset are crucial. However, it is difficult, or practically impossible, to prepare a large dataset of high-quality samples for users' target tasks. To cope with this problem, data transfer techniques are widely employed as ordinary research pipelines in CV, NLP, and MT domains.
Pre-training is train model with some datasets prior fine-tuning, which is further training with the target dataset. Sometimes the pre-trained model is deployed as is as a general-purpose baseline. One of the first major successes of data transfer for DNN is the supervised pre-training
in CV. In that circumstance, pre-trained binaries of successful networks (Krizhessky et al. 2012
Simonyan & Zisserman 2015| He al., 2016) included in deep learning frameworks (e.g. Caffe) played crucial roles in ad advancing other CV studies. Today it is a common pipeline to employ the pre-trained model as a part of the larger and stronger networks (Girshick et al., 2014 Long et al. 2015, Xiong et al. 2018: Korbllihh et al. 2019). One distinct characteristic of the pre-training in the NLP domain is the large sizes in the dataset to feed and the train models. Very large networks such as BERT (Devlin et al.) 2018)) and GPT-2 (Radford et al. 2019) are trained with massive amounts of the corpus (dataset), which were unrealistically huge for most engineers and researchers to deal with, were distributed in binaries parameters. These pre-traines models were incorporated in each developer's s system and contributed to push limits of several NLP studies and applications (e.g. (Lee) et al. 2019 Lan al. 2020 Yang et al. 2019)).
Self-training, another data transfer method studied in this paper. has longer history than the pre- training, back to the seminal work by [Scudder (1955). In the self-training approach, we first train a base model with the labeled target dataset, which is often small but high-quality. Then, a larger dataset that contains unlabeled or attached with low-quality noisy labels is relabeled by the base model. The relabeled dataset is used to augment the small target dataset to obtain a stronger model. The same technique is often referred as pseudo labeling, one of the elemental techniques to boost performances in program competitions. Recently self-traing trainiriiig draws attention in several domains, such as CV (Xie et al. 2020), NL.P (He et al. 2020), and speech recognition (Khan et al., 2019). An advantage of self-training in against the pre-training is that the self-training does not require the large labeled dataset thanks to the relabeling, even for the supervised task. This characteristic is employed in the MT application: it is difficult to collect qualified translation datasets (parallel corpus) for minor languages. Thus the self-training (referred to as back translation) is intensively used to augment the pseudo parallel corpus (Sennrich et al. 2016).
Zoph et al. (2020) examined the data transfer pipeline and showed that the self- self-training is superior to the widely adopted pre-training in object recognition (segmentation) tasks. They studied the data transfer at discriminative models. In this paper, we focus to examine the effectiveness of the data transfer methods in peuuencc-to-sequenee generative models.
2.2.1 1 DATA TRANSFER IN BIO- AND CHEMO-INFORMATICS
A few researchers in bioinformatics and chemoinformatics show interest in the data transfer approach to learn a universal numerical representation (embedding) of properties of the compounds that is
informative enough to predict multiple quantities. HHond al. (2019) employed Transformer encoder-decoder to pre-train the intermediate representation of compounds and applied the pre-trained
encoder for downstream tasks combining with a separate predictor network. Li & Fourches) (2020) followed the masked prediction procedure of BERT (Devlin et al. 2018) for pre-training. Wang al. 2019) employed an LSTM but reports that the pre-trained model performs well in some small datasets. Compared to these previous studies, our study differs in the following two essential points. First, as we have explained earlier, we deal with data transfer for sequence-tor requence-to-sequene sequenceeee generative models for retrosynthesis while these studies focus on discriminative models similar to (Zoph et al.) 2020 Second, these previous studies did not perform systematic comparisons of different transfer methods for the target tasks. In contrast, perform systematic and intensive comparisons among transfer methods to find the best data transfer methods for retrosynthesis.
There are two exceptions that study the transfer learning in generative models in chemoinformatics. Pesciullesi et al.) (2020) improved accuracy of synthesis prediction by applying the Molecular Transformer model to carbohydrate reactions using transfer learning. Though apparently similar, the synthesis and the retrosynthesis belong to different classes of problems: a set of reactants yields a specific product compound while a product can be decomposed into multiple reactant sets.
3

---

Under review as a conference paper at ICLR 2021
Chen al. (219)) proposed to pre-train retrosynthesis model in slightly different way. They synthesized a pre- pre-training dataset by manipulating the target dataset randomly or using pre-defined rules and did not rely on the other large datasets. They reported such pre-training improve retrosynthesis accuracy, though the gains were not large. Their problem setting was different from ours. We focused on the case where additional huge dataset is available other than the target dataset to mitigate the quantity-and-qualin problem. We will show that our approach can achieve much greater improvements in retrosynthesis accuracy.
still able to
DATA TRANSFER METHODS FOR RETROSYNTHESIS
In this section, we first give a general formulation of ML-based retrosynthesis problem. Then, we explain several data transfer methods using that formulation.
3.1 RETROSYNTHESIS PROBLEM FORMULATION
Retrosynthesis is problem to map product compound item to appropriate set y of reactant compounds [FigureT). A sample for a retrosynthesis model is a pair (z.y) where x denotes a product, y denotes set of reactants. A retrosynthesis model . is (stochastic) mapping between and y: conventionally formulated as a likelihood distribution PM (y]æ; 6) with the parameters 8MM- Formulation of the likelihood determines the model characteristics. Most previous studies focused on developing new sophisticated formulations e.g. seq-to-seq models (Liu et al. 2017 Karpov et al.)
2019) and graph-to-graphe models (Shi et al. 2020 Somnath et al. 2020 Yan et 2020).
In this paper, we simply set .M naive seq-to-seq Transformer (Vaswani et al. 2017) since we focus on the data transfer rather than the model architecture. Now the model fixed, so we omit M hereafter. Under a seq-to-seq formulation of retrosynthesis, both of the product æ and the set of reactants y are represented as strings (a sequence of characters). We adopt the standard string representation of chemical compounds, referred to as SMILES (Weininger, 1988). Basically any compound can be uniquely represented as a SMILES format string. y consists of possibly multiple compounds. In such a case, we simply concatenate the multiple SMILES strings into longer string with delimiters.
Assume dataset D is set of reaction samples where the product and the reactants are represented as SMILES strings. The ith sample of the dataset is indexed by i: a pair (I,, yi) thus D = {(x,, y:)]- A training dataset Dunin is used to find good parameters 8. that optimize objective function, which is typically the maximum (log- )likelihoo criterion:
lt arg, max
log (3)(2))
(Z. Dran
We usually implement the above maximum likelihood criterion the minimization of the cross- entropy loss L(Drrain. We minimize the cross-entropy loss by an iterative parameter update algorithm such stochastic gradient descent:
gD Init, ge+1 8² - VV
The iterations are scheduled by validation scores computed on validation dataset D and the learned model is evaluated by test scores computed on test dataset Dies- These three datasets should have no intersection samples.
3.1.1 BASELINE: SINGLE MODEL TRAINING (WITHOUT DATA TRANSFER)
The standard setting of most of the previous retrosynthesis models followed simple scenario. First, we prepare a dataset to which we want to fit the model. We refer to this dataset as a target dataset DT The whole DT is split into three sub-datasets: a training set DJ validation set DI and test set Tea The single model training is:
0 Singgle = argg max
Σ
(E.D
where the training procedure is scheduled by D and evaluated on DT In this setting, we have no dataset other than the target dataset, thus do not (can not) conduct any data transfer.
4

---

Under review as a conference paper at ICLR 2021
3.2 DATA TRANSFER FOR RETROSYNTHESIS
Next, we consider another scenario where are given additional dataset other than the target dataset. We call this additional dataset as augment dataset DA Typically augment datasets have larger sample sizes than those of the target datasets. It is also usual that empirical distributions of p(y(z) can different between D and DA, Still, we want to transfer some knowledge from D to the model parameter to improve the test scores computed the target test set lest
3.2.1 JOINT TRAINING
One of the most simple ways of data transfer is joint training. We split DA into a training set D validation set DA and test set D
Tran
In the joint training, we optimize the parameters that satisfy the training objective on both Train and DAnin In our setting, the model and the objective are shared among datasets, so we simply train on the concatenated training sets:
joint argo max
log (3)l2a)) where Djoint Drain U D Train
(4)
The training process is scheduled by D and evaluated DJ
The above formulation (Eq of the joint training is restrictive: only available when the domain of the data pair (Ir y) are shared among the target dataset and the augment dataset. For example, in our cases, the canonicalization rule the SMILES compound string representation must be the same in (z) of Drain and D Train
3.3 SELF-TRAINING
Self-training is well-known technique for data transfer, and it is also known as pseudo labeling. Unlike the joint training, we can conduct the self-training even if the domain differs between Di and DA Irain
We first conduct the single model training (using D solely) and obtain 0 single Then, we decode all product of the augment dataset with the learned parameter Single The generated pseudo labels is used to replace the reactants of the new augment training set. Then we perform the joint training by concatenating D Train and the pseudo-labeled DA Decoding with the single model trained by the target dataset will ease the difficulty of transferring knowledge due to the inconsistent p(ylæ) between D and D We can formulate the self self-training follows:
Ú: argy max log p (y) 24; (Gingee) for E Diiain
(5)
D PrendoTrain
{x,, 0:)} for all x, E D
(6)
0g arga max (or EDrr
Σ
log (yn(2r)) where Daelr Di U D A
PscudoTrain
The training process of Eq is scheduled by D and evaluated on DIr
3.4 PRE-TRAINING PLUS FINE-TUNING
In the joint training and the self-training the final best parameter is computed via the training on the concatenated training sets Djoint and Deetf. The sample sizes of these concatenated training sets can be large, resulting in slower training convergence. Pre-training plus fine-tuning method handles this problem by training a model in advance with the augment dataset A The pre-trained model is simply loaded as the initial model, and is tuned to the smaller target dataset. The later fine fine-tuning finishes faster general, beneficial for the cases when we have potentially multiple target datasets for deployments.
5

---

Under review as a conference paper at ICLR 2021
The pre-training plus fine-tuning approach thus naturally requires two training phases. In the pre- training phase (the first phase), we preform single model training solely on the augment dataset:
p pretrain argo max Σ
logp (yil
where the training procedure is scheduled by Da and evaluated D
In the fine-tuning phase, we train the model with D not from the scratch but from Mo
poanan
0
Pinetune
pretrain
g+++1
linetume
f- 0 fineuune L L(D Train
(9)
irsture
ffinetune
(10)
where the training procedure is scheduled by V and evaluated on D Test"
4 EXPERIMENT
In this section, we first assess the effects of the several data transfer methods, compared with the Transformer that is trained naively. After that, we show the comparisons with the most recent SotA retrosynthesis models.
4.1 PROCEDURE
4.1.1
DATASET
Most ML-based retrosynthesis studies adopt the USPTO database, which is collection of chemical reaction formulas automatically OCR-scdnned from the US Patent documents.
The target dataset is the USPTO-50K dataset, which contains curated 50K samples provided by Lowe (2012). This dataset serves as the standard benchmark in the retrosynthesis studies. The characteristic of this dataset is that all samples are classified in one of ten major reaction classes in advance (Schnei) der et al., 2016). This means the distribution of products and reactants are skewed. In this study, the reaction class information is only used to summarize the prediction results and is not used for filtering the dataset or for prediction.
For data transfer, we prepare two augment datasets. Since these two datasets are not filtered based on the reaction classes, their distributions of compounds are clearly different from that of the USPTO- 50K dataset. The first and the main augment dataset is referred to USPTO-Full dataset, which contains 877K samples curated by Lowel (017)). The second augment dataset is referred to as USPTO-MIT dataset, which contains 479K samples curated by Din al. (2017). Experimental results on USPTO-MIT included the appendix for sample sizes assessment.
The three datasets have number of shared reaction samples noisy instances. For fairer evaluations, we need data cleansing beforehand. The details of the data cleansing are presented in the appendix.
4.1.2 MODEL ARCUITECTURE, TRAINING, AND EVALUATIONS
In order to give fair assessments of the effects of data transfer methods, we fix the architecture the seq-to-seq model, namely the Transformer Vaswani et al. (2017) regardless of the transfer methods. We may obtain better results if we optimize the architecture for each transfer method.
All models are optimized via Adam (Kingma & Ba 2015). Following the seminal paper of Karpov cet al. (2019). learning rates are scheduled in a cyclic manner with the warm-up iterations excepting the fine-tune training. In the mineitinng. we find that a standard non-cyclic scheduler (Klein et al. 2017) perform good, thus we adopt in our experiments.
We adopt the n-best accuracy We use l = 50 beam search to list and sort the predictions, and compute n = 1,3, 5, 10, 20, 50-best accuracy scores, following the normal procedure in the literature.
of the predicted results on the test the evaluation metric.
Previous works (Dai et al. 2019; Somnath et al., 2020; Yan et al. 2020 use the validation sets to exponentially decay the learning rate. Instead,
the validation set to choose and output

---

Under review as a conference paper at ICLR 2021
Table 1: n-best accuracy of retrosynthesis tasks on USP,O-50K, with different data-transfer training methods. Augment dataset is the cleansed USPTO-Full. Larger values are better. Averages and standard deviations of 5 runs are presented. Bold faces indicate the best scores at the specific n-accuracy.
n-best accuracy (%)
Training Method Single model
n1
Tt 1=3
7=
n n 10
n= 20
11= 50
35.3 + 1.4
52.8 + 1.4
58.9 + 1.3
64.5 + 1.2
68.8 + 1.2
72.1 + 1.3
(No Transfer)
Joint Training
39.1 + 1.3
63.4 + 0.9
71.9 0.5
80.1 + 0.2
85.4 0.3
89.4 + 0.2
Self-Training
41.5 1.0
60.4 + 0.7
66.1 + 0.7
71.8 + 0.6
75.3 + 0.5
78.0 + 0.3
Pre-training
57.4 0.4
77.6 + 0.4
83.1 +: 0.2
87.4 0.4
89.6 0.3
90.9 1 0.2
+ Fine-Tune
a snapshot that records the best validation perplexity. All accuracy scores of our experiments are computed these best val-score snapshots.
Full descriptions about the experimental procedure are presented in the appendix.
4.2
RESULTS
4.2.1
COMPARING DATA TRANSFER METHODS
Table D shows the n-best accuracy scores over several data transfer methods. We use the USPTO-50K dataset as the target dataset, and the cleansed USPTO-Full dataset the augment dataset this table.
Generally, all data transfer methods are successful in improving the n-best accuracy for all choices of n. Among them, the pre-training plus fine-tuning method achieves the remarkable gains (more than 20 points for most n)). This result is indeed beneficial for the researchers who conduct not computational experiments but actual synthesis experiments because it reduces the number of required experiment trials. In this study, we did not perform optimization of the model itself because our main goal was to confirm the effect of data transfer, but if necessary, we can perform optimization with the existing model immediately.
is interesting to compare this result with the previous study by Zoph et al.) 2020). which examines the self-training and the pre-training plus fine-tuning in the image recognition tasks. Their conclusion that pre-training is less effective than the self-training to improve generalization performance. The authors explain this is because there is no universal representation (embedding) of generic images. In this study, which is not for image recognition but for retrosynthesis, we observe that the pre- pre-training plus fine-tuning is evidently better than the self-training. Perhaps the chemical compound strings may have a universal representation because of restrictions and patterns naturally imposed on the chemical compounds in nature.
4.2.2 COMPARISONS WITH THE STATE- THE------TTTTTS
Next, we compare our best data transfer model with the SotA model of the retrosynthesis. As the figure shows, our best pre-training plus fine-tuning model performs as good the latest SotA model in the literature, which is built on a off-the-shed simple Transformer. Moreover, our model exceeds the known best accuracy at n 10 and 20.
None of the compared studies provided the standard deviations of the obtained accuracy scores. Thus it is difficult to conclude definitively that one model better than other models from this result. However, our model performed the best in = 10, 20-best accuracy, which is preferable for those who conduct synthesis experiments, for many reasons. First, there is usually more than one correct answer for retrosynthesis analysis, even if it consists of only one-step reaction. Second, the reaction with the highest score is not always the best in practice. Thus, is expected to output various solutions within a limited number (for example, 10) of answers rather than producing only one answer. In other words, a predictor that makes 10 good suggestions is pragmatically more highly valued than predictor that makes only suggestion that considered the best.
7

---

Under review as a conference paper at ICLR 2021
Table 2: n-best accuracy of retrosynthesis tasks USPTO-50K, comparing with the most recent SotA models. Bold faces indicate the best scores at the specific n-aceuracy. All numbers of the compared models are borrowed from their original papers. N/A indicates the accuracy scores are not reported in the published papers.
n-best accuracy (%)
Name GLN
Model Arch.
11=3
n=
n
10
n--20
11
20
50
Logic Network
52.5
69.0
75.6
83.7
88.5
92.4
Dai al.
2019)
G2Gs S
Graph-to-Grrph
48.9
67.6
72.5
75.5
N/A
N/A
Shi et al.
2020)
RetroX
pert
Graph-to-Graph
65.6
78.7
80.8
83.3
84.6
86.0
Yan al.
2020)
GraphRetro
Graph-to-Graph
63.8
80.5
84.1
85.9
N/A
87.2
(Somnath et al.
22200)
Our Pre-training
Seq-to-Sey
57.4
77.6
83.1
87.4
89.6
90.9
Fine-Tune
4.3
MORE ANALYSES
Due to page limitation, we describe other additional analyses in appendixes. Here only summarize the main messages of these analyses. Please find the appendix for the details.
Additional experiments with smaller USPTO-MIT dataset shows that the sample size augment dataset clearly affects the generalization performance of the transferred models.
2. We find that the 1-best accuracy and n-best accuracy show quite different evolution curves against training iterations, especially for the single training model. Such behaviors mentioned in the literature so far, possibly opening new question for model training. This behavior
observed in the pre-training plus fine-tuning, which another advantage over other transfer methods.
3. We confirm that the pre-training plus fine-tuning improves the class-wise prediction accuracy for all 10 major reaction classes of USPTO-50K. The augment dataset especially help prediction of difficult bond/ring formation reactions.
4. The manual inspection for the erroneous prediction results reveals that there are a few reactions that are still difficult to perform reasonable retrosynthesis prediction.
5. At the same time, the manual inspection results show that over 99% of top-1 predictions are chemically reasonable and appropriate hypotheses, even the hypotheses do not include the exact "gold" reactant hypothesis.
5 CONCLUSION
In this paper, we conducted systematic investigation on the effect of data transfer for the improve- of the computational retrosynthesis system. The result proved that the most typical data transfer methods can improve the test prediction accuracy of a retrosynthesis model. Especially, Transformer with pre-training plus fine-tuning updated the SotA of the 10- and 20-best retrosynthesis accuracy. We also confirmed that the pre- pre-training plus fine-tuning model can generate chemically appropriate or sensible proposals more than 99% cases top-1 predictions through our manual inspections of the predictions.
We only validated the simplest transfer techniques in this work. future work, we are interested in applying more sophisticated transfer learning methods. For example, freezing part of the NN during fine-tuning may be effective according to the results in CV domain. It is also interesting for the retrosynthesis community to apply the data transfer methods for graph-to-graphs models such as (Somnath et al. 2020). We observed that the USPTO-Full dataset is transferable to the USPTO-50K dataset, which has biased distribution of reaction types. To examine the transferability of the USPTO-FF dataset. we need to test the transfers to other retrosynthesis datasets, which are small, biased and collected for specific purposes.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Semih Cantürk, Aman Singh, Patrick St-Amant, and Jason Behrmann. Machine-learning driven drug repurposing for COVID-19, arXiv, 2020.
Benson Chen, Tianxiao Shen, Tommi Jaakkola, and Regina Barzilay. Learning to make generaliz- able and diverse predictions for retrosynthesis. arXiv, 2019.
Jonathan H. Chen and Pierre Baldi. No Electron Left Behind: A Rule-Based Expert System To Predict Chemical Reactions and Reaction Mechanisms. Journal of Chemical Information and Modeling, 49(9):2034- 2009.
Vijil Chenthamarakshan Payel Das, Samuel C. Hoffman, Hendrik Strobelt, Inkit Padhi, Kar Wai Lim, Benjamin Hoover, Matteo Manica, Jannis Born, Teodoro Laino, and Aleksandra Mojsilovic. CogMol: Target-specific and selective drug design for COVID-19 using deep generative models. arXiv, 2020.
EJ Corey and W Todd Wipke. Computer-assisted design of complex organic syntheses. Science, 166 199, 1969.
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. Retrosynthesis prediction with conditional graph logic network. In Proceedings of the Advances in neural information processing systems 32 (NeurIPS), pp. 8872-8882, 2019.
Jacob Devlin, Ming-wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv, 2018.
Girshick, Ross, Donahue, Jeff, Darrell, Trevor, Malik, and Jitendra. Rich feature hierachies for accurate object detection. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2014.
Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. Revisiting self-training for neural sequence generation. In Proceedings of the 8th International Conference Learning Reprexenta- tions (ICLR), 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2016.
Shion Honda, Shoi Shi, and Hiroki R. Ueda. SMILES transformer: Pre-trained molecular fingerprint for low data drug discovery. arXiv, 2019.
Wengong Jin, Connor W Coley, Regina Barzilay, and Tommi Jaakkola. Predicting Organic Reaction Outcomes with Weisfeiler-Lehie Network. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS), pp. 1-10, 2017.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction Tree Variational Autoencoder for Molecular Graph Generation. In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.
Pavel Karpov, Guillaume Godin, and Igor V. Tetko. A transformer model for retrosynthesis. In Proceedings of the International Conference on Artificial Neural Networks (ICANN): Workshop and special sessions Artificial Neural Networks and Machine Learning, pp. 817-830, 2019.
Jocab Khan, Ann Lee, and Awni Hannun. Self-training for end-to-end speech recognition. In Pro- ceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICAASP), 2019.
Diederik Kingma and Jimmy Lei Ba. Adam: a Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICIR), 2015.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. OpenNMT: Open-source source toolkit for neural machine translation. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long papers) (ACL), 2017.

---

Under review as a conference paper at ICLR 2021
Simon Kornblith, Jonathon Shlens, and Quoc V. Le. Do better Imagenet models transfer better? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the Advances in Neural Information Processing Systems 25 (NIPS), pp. 1-9, 2012.
Matt J. Kusner, Brooks Paige, and Jose Miguel Hernandez-Lobato. Grammar Variational Autoencoder. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A Lite BERT for Self-supervised Lerarning of Language Representations, In Proceedings of the 8th International Conference on Learning Representations (ICLR), pp. 1-17, 2020.
Greg Landrum and others. RDKit: Open-source cheminformatics, 2006.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformaties, 36(4): (1:4):1234-111 1240, 2019.
Xinhao Li and Denis Fourches. Inductive transfer learning for molecular activity prediction: Next-Gen QSAR Models with MolPMoFiT. Journal of Cheminformatiex, 12(1):27, 2020.
Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, and Vijay Pande. Retrosynthetic Reaction Prediction Using Neural Lequence-to-Sequene Models. ACS Central Science, 3(10): 1001110031111 1113, 2017.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
Daniel Lowe. Extraction of Chemical Structures and Reactions from the Literature. PhD thesis, University of Cambridge, 2012.
Daniel Lowe. Chemical reactions from US patents (1976-Sep2016), 2017. URL https://
Figshare.com
Giorgio Pesciullesi, Philippe Schwaller, Teodoro Laino, and Jean-Louis Reymond. Transfer learning enables the molecular transformer predict regio- and stereoselective reactions on carbohydrates. Nature Communications, 11(1):8874, 2020.
Alee Radford, Jeffrey Wu, Rewon Child, David Luan, Dairo Amodei, and llya Sutskever. Language Models are Unsupervised Multitask Learneres, 2019. ULL/httpps///peeaaaccoo/////
betterr-lnnguage-moeeee
GPT2.
Nadine Schneider, Nikolaus Stiefl, and Gregory A. Landrum. What' What: The (Nearly) Definitive Guide to Reaction Role Assignment. Journal of Chemical Information and Modeling, 56(12): 2336-2346 2016.
H. Scudder. Probability of error of some adaptive pattern-recognition machines. IEEE Trans. Information Theory, 11(3):363-371, 1965.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation mod- els with monolingual data. In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: Long papers) (ACL), pp. 86-96, 2016.
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph to graphs framework for retrosynthesis prediction. Proceedings of the 37th International Conference on Machine Learning (ICML), pp. 2003.12725, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
10

---

Under review as a conference paper at ICLR 2021
Vignesh Ram Somnath, Charlotte Bunne, Connor W. Coley, Andreas Krause, and Regina Barzilay. Learning graph models for template-fiee free retrosynthesis. arXiv, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention All You Need. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS), 2017.
Lee-Ping Wang, Alexey Titov, Robert McGibbon, Fang Liu, Vijay S. Pande, and Todd J. Martínez. Discovering chemistry with an initio nanoreactor. Nature Chemistry, 6(12):1044 1044, 2014.
Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang. SMILES AMILES-BERT: Large scale unsupervised pre-training for molecular property prediction. In Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics, pp- 429-436 2019.
Jennifer N. Wei, David Duvenaud, and Alán Aspuru-Guzic Neural Networks for the Prediction of Organic Chemistry Reactions, ACS Central Science, 2 2(10):725-732, 2016.
David Weininger. SMILES, chemical language and information system. 1. introduction to method- ology and encoding rules. Journal of Chemical Information and Modeling, 28(1):31-36, 1988.
Xie, Qizhe, Hovy, Eduard, Luong, Minh-Thang student improves imagenet classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Thang Luong, and Quoe V. Le. Self-training with noisy
Fu Xiong, Yang Xiao, Zhiguo Cao, Kaicheng Gong, Zhiwen Fang, and Joey Tianyi Zhou. Towards Good Practices on Building Effective CNN Baseline Model for Person Re-identification. Optics Letters, pp. 1-7, 2018.
Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, and Jun- zhou Huang. RetroXpert: Decompose retrosynthesis prediction like a chemist. chemrxiv, pp. 11869692.v3, 2020.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.
XLNet: Generalized autoregressive pretraining for language understanding. In Proceedings of the Advances in neural information processing systems 32 (NeurIPS), pp. 5753-5763, 2019.
Barret Zoph, Golnaz Ghiasi, Tsung- Yi Lin, Yin Cui, Hanxiao Liu, Ekin D. Cubuk, and Quoe Le. Rethinking pre-training and self-training. arXiv, 2020.

---

Under review as a conference paper at ICLR 2021
APPENDIX A EXPERIMENT PROCEDURE: DETAILS
A.1 DATASET
Most ML-based retrosynthesis studies adopt the USPTO database, which is collection of chemical reaction formulas automatically oca-ncddded from the US Patent documents. We use the filtered subsets of the USPTO database in this study.
The target dataset is the USPTO-500 dataset, which contains curated 50K samples provided by Lowe (2012). This dataset serves as the standard benchmark in the retrosynthesis studies. The dataset is split into 40K train, 5K validation, and 5K test sets following (Liu et al.) 2017). The distinct characteristic of this dataset that all 50K samples are classified in one ten major reaction classes in advance (Schneider et al. 22016). This means the distribution of products and reactants are highly skewed. In this study, the reaction class information is only used to summarize the prediction results and is not used for filtering the dataset or for prediction.
To perform data transfer, we prepare two augment datasets. Since these two datasets are not filtered based on the reaction classes, their distributions of compounds are clearly different from that of the USPTO-50K dataset. The first and the main augment dataset is referred to as USPTO-FUE dataset, which is curated by Lowe (017). The USPTO-Full dataset is split into 879K, 49K, and 49K samples for training, validation, and test sets, respectively. The second augment dataset is referred to as USPTO-MIT dataset, which is curated by Jin al. (2017). The USPTO-MIT dataset is split into 409K, 30K, and 40K samples for training, validation, and test sets, respectively[] Experimental results on this dataset are presented in the appendix for assessment of the sample sizes of augment datasets.
A.2 DATA CLEANSING
The training sets of the augment datasets (USPTO-Full and USPTO-MIT) contain many reaction SMILES samples that are incorporated in one of the subsets of the target dataset USPTO-50K. Such contaminated samples inhibit fair predictions, because of the duplicated training samples (duplicated samples given more weight than other samples) or the data leaks (if a test sample is included in the training set, the model answer correctly by simply memorizing the sample without learning the features). Thus, we remove all reaction samples from the training sets of the two augment datasets whose product SMILES are included in any subsets of the target dataset. As result, the USPTO-MIT training set is reduced to 384K samples. Similarly, the USPTO-Full training set shrinks to 844K samples.
We adopt a canonical SMILES format throughout the experiments. However, there are multiple canonicalization rules in this field. In fact, we confirmed the canonicalization rules are not unified among the three datasets. Non-unified canonicalizations may hinder effective training, and may also induce unexpected data leaks because the same compound with different SMILES string cannot be detected. Therefore we conducted unified canonicalization for all datasets using the RDKit tool (Landrum & others 2006) with a specific version.
A.3 MODEL. ARCHITECTURE, TRAINING, AND EVALUATIONS
In order to give fair assessments of the effects of data transfer methods, we fix the architecture of the seq-to-seq model, namely the Transformer Vaswani et al. (2017) regardless of the transfer methods. This means the reported results are based on the sub-optimal Transformer architecture; may obtain better results if we optimize the architecture for each transfer method. The number of self-attention layers is set to 3, and the dimensions of the latent vectors are set to 500 in all layers. We adopt the positional encoding, which find it essential achieve good scores through preliminary experiments. We limit the maximum number of tokens (length of a sequence) up to 200 avoid memory shortage of GPUs. We use the off-the-shelf implementation of the Transformer by OpenNMT-py?
Downloaded from opennemmtorg
12

---

Under review as a conference paper at ICLR 2021
Table 3: n-best accuracy of retrosynthesis tasks USPTO-50K, with different data-transfer training methods. Augment dataset is the smaller USPTO-MIT. Other details are the same with the Table n-best accuracy (%)
Training Method
n
n=
n=5 5
n 10
n1 20
n== 50
Single model
35.3
- 1.4
52.8 + 1.4
58.9 + 1.3
64.5 + 1.2
68.8 + 1.2
72.1 + 1.3
(No Transfer)
Joint Training
38,4 0.9
60.7 + 0.5
67.8 + 0.4
75,2 + 0.3
80.4 0.4
84.9 1 0.3
Self-Training
41.2 0.3
60.2 = 0.4
66.2 - 0.2
71.9 0.3
75.5 0.5
78.2 0.5
Pre-training
52.2 0.4
73.1 + 0.4
78.8 0.4
83.7 0.3
86.3 0.3
88.2 = 0.3
+ Fine-Tune
Following the seminal paper of Karpoy ct al. (2019), learning rates are scheduled in cyclic manner with the warm-up iterations excepting the fine-tune training. In the fine-tuning, we find that a standard non-cyclic scheduler (Klein et al. 2017) performs good, thus we adopt it our experiments.
All models are optimized via Adam (Kingma & Ba 2015).
We adopt the n-best accuracy score of the predicted results on the test set as the evaluation metric. Remember that sample pair of the imput product SMILES string and the expected reactant SMILES string. However, the provided reactant SMILES is not necessarily the only correct an- swer. Thus we predict the k-best possible predictions for each test input. If the expected reactant SMILES is included in the best-n prediction, we assume the retrosynthesis model is successful in predicting the answer. We use = 50 beam search to list and sort the predictions, and compute 1, 5, 10, 20, 50-best accuracy scores, following the normal procedure in the literature.
We note that there are possible choices how to employ validation split samples. Previous works (Dai al. 2019 Somnath al. 2020 Yan et al.) 2020) use the validation sets to exponentially decay the learning rate. However, we adopt the cyclic learning rate scheduler following the seminal Transformer- based model (Karpov al. 2019) and found it indeed effective in our experiments. Therefore, use the validation set to choose and output a snapshot that records the best validation perplexity. All accuracy scores of our experiments are computed on these best validation score snapshots. We do not employ any ensemble model for fair comparisons with the previous studies.
APPENDIX B SMALLER AUGMENT DATASET RESULTS IN INFERIOR PERFORMANCE
Table 3 shows the n-best accuracy scores over several data transfer methods, but this time we use the smaller USPTO-MIT dataset as the augment dataset in this table. Our expectation is the gains of data transfer methods decreases with the smaller augment dataset.
The joint training and the pre-training plus fine-tuning record worse scores compared to the USPTO- Full cases (Table [ Surprisingly, the gain of the self-training is not affected by the sample size of the augment dataset.
APPENDIX C TRAINING EVOLUTION AND TEST GENERALIZATION PERFORMANCE IN DIFFERENT TRANSFER METHODS
We thoroughly trained several models with larger numbers of mini-batch iterations (or epochs), which has not been conducted in any previous studies.
how the three metrics (the average values of five runs) on the target dataset (train and test sets) change as the training iteration increases. Spikes indicate the periods of the cyclic learning rate schedulers. Figure 2 consists of 4x3 panels, where the four rows correspond transfer training methods: single model (no transfer), joint training, self self-training, and pre- pre-training plus fine-tuning and the three columns correspond the three different metrics. The left column shows
four different
We only consider the fine-tuning because we have access to the target dataset during pre-training in usual cases.
13

---

Under review as a conference paper at ICLR 2021
the perplexity cerplexttyoonn the training set. We observe all transfer training methods successfully decrease the training perplexity, namely, better fit for the training set.
The middle and the right columns show the 1-best and the 20-best accuracy scores the test set, respectively. The remarkable difference is observed among different models, especially between the single (1st row) and the self-training (3rd row) models. More importantly, this analysis clearly showed different behavior between 1-best and the ( 1) -bet accuracy curves in the single model.
One may naturally expect and hypothesize that 1-best and the n(> 1) 1)-best accuracy curves behave similarly: but no previous retrosynthesis studies have examined, or even discussed, this hypothesis. We showed in this study that the hypothesis does not hold for single model training on USPTO-50K, which is the default training strategy in previous studies.
This observation, that the 1-best and the n-best accuracy curves may be totally different, is important especially when trying to verify the top-n result experimentally (not in a computational experiment, but in an actual experiment that is both time-conse consuming and expensive). One possible reason for this difference between the 1-best and the n-best accuracy is the form of the objective function. The maximum likelihood of objective L updates the parameter so as to maximize the 1-best accuracy on the training set, not to maximize the n-best accuracy.
is also remarkable that the two curves of the pre-training plus fine-tuning (4th row) are not significantly different, but rather very similar. The trained model quickly hits the peaks of the I-best and the 20-best accuracy around 10K iterations. After hitting the peaks, two curves decrease rapidly. The model gradually forgets the beneficial knowledge transferred from the augment dataset by keeping the training iterations, and the model will converge to the single train model after many iterations. Thus it is important whether experiments, the validation score monitoring are al always successful in identifying this peak, yielding the good scores for all top-n accuracy in Tables. 1 The fact that the two curves of the pre-training plus fine-tuning approach are very similar is another ad advantage for the data transfer in retrosynthesis. The joint training (2nd row) also shows the similar curves in test scores, but the pre-training plus fine-tuning approach is better in two aspects: final accuracy scores (Table and necessary numbers of iterations (roughly between 70K and 250K iterations) required achieve the best-val-se val-score snapshot.
detect the peaks to early-stop the fine-tuning, In our
C.I CONCERNING THE SINGLE MODEL SCORE
It may seem strange that the 1.3,5- best accuracy scores of the single model are lower than those of the known results in Karpov et al. 2
In our experiment, the best-val-score snapshots of the single model are always chosen from the earlier iterations (less than 10,000 iterations), resulting low I-best scores of the single model compared to the known results (Karpov et al.) (019). According to Figure the 1-best accuracy exceeds the known score if we choose the snapshot of e.g. 140,000 iterations. However, the snapshot achieves worse 20-best accuracy compared to the best-val-score snapshot.
APPENDIX D MORE IN-DEPTH EXAMINATION OF PREDICTED REACTANTS
Next, we examine the predicted reactant SMILES to see how the data transfer contributes to the improvement of the computational retrosynthesis prediction from the perspective of actual chemical experiment. We evaluate 428 test samples (æ,,, U,)) E Dir whose 50-best predictions ûرs do not include the "gold" reactant SMILES string 3/, We note again that the gold reactant set yy is not the sole correct retrosynthesis prediction, but the current studies adopt this "hard" criterion. During the inspection, we realize that USPTO-50K test dataset still contains number of errors (mislabels) within the test set despite the curating efforts of the original authors (Lowe) 2012). Therefore we exclude these mislabeled samples from this in-depth analysis.
D.1 SUCCESSFUL TRANSFER CASES
Table shows the accuracy of the single training model and our best-transferred model (pre-training plus fine-tuning) for each reaction class. From the table, we observe the heterocycle formation is
*The most similar model 71, achieved 39.8, 59.1, and 63.9, respectively
14

---

Under review as a conference paper at ICLR 2021
Training Perplexity
1-Best Test Accuracy
20-Best Test Accuracy
42
71
41
70
Single training (No Transfer)
40
69
1.2
39
68-
38
67-
1.0
40K
80K
120K150K
40K
80K iterations
120K 150K
40K
BOK
120K 150K
Iterations
iterations
1.7.
40
86
1.6
38
84-
1.5
36
82
Joint Training
1.4
80-
1.3
78-
1.2
76-
1.1
28
74.
100K
200K
300K
100K
200K
300K
0
100K
200K
300K
lterations
Iterations
Iterations
76.0-
1.4
43
75.5
TyHthy Hyutttun
Self-training
1.3.
42
75.0
74.5
1.2
74.0
1.1
40
73.5-
39
73.0
100K
200K
300K
100K
200K
300K
0
100K
200K
300K
iterations
iterations
iterations
58 0
90.0
1.08
57.5
89.5
Pre-training
1.06
56 5
89.0
plus Fine-tuning 1.04
55.5 5
88.5.
1.02
0
88.0.
87.5.
0 20K 40K 60K 80K 100K
20K 40K 60K 80K 100K
20K 40K 60K BOK 100K fine-tuning iterations
fine-tuning Iterations
fine-tuning iterations
Figure 2: Relationships between Training iterations and Train/Test scores. For all panels, the horizontal axis denotes the training iterations. First row: Single model, Second row: joint training. Third row: Self-training, Fourth row: Pre-training plus Fine-tuning. The left column shows the training perplexities. The middle column shows the 1-best test accuracy scores. The right column shows the 20-best test accuracy scores.
difficult to predict with single training of a seq-to-seq model (Liu et al., (017). The heterocycle formation reaction not only forms some new bonds but also aromatizes some atoms through reaction, which changes many capital letters in SMILES string to lowercase. Therefore, it is expected that will be difficult learn atom-to-atom mapping with small training samples. Figure 3 shows examples that fail in the single model but succeed in the transfer model. Significant improvements are observed the prediction of complicated reactions such as heterocyclic reactions (Ist row), suggesting that pre-training plus fine-tuning with large augment dataset enables to learn such large changes. Deals-Alder reaction (2nd row) amother example of complicated reactions, where two C-C bonds are formed well some atom and bond types change. In general, the single model is not good at generating the ring formation reactions and only returns odd answers, while the transferred model is able return correct answers such reactions.
15

---

Under review as a conference paper at ICLR 2021
Table 4: Difference in 50-best accuracy for each reaction class.
50-best accuracy
Number of samples
Reaction class
Single model
Transferred model
Heteroatom alkylation/arylation
RX_1
1497
0.78
0.96
Acylation and related processes
RX 2
1176
0.85
0.98
C-C bond formation
RX 3
553
0,54
0.79
Heterocycle formation
RX_4
90
0.40
0.86
Protection
RX_5
64
0.64
0,83
Deprotection
RX 6
812
0.69
0.88
Reduction
RX 7
457
0.77
0.96
Oxidation
RX 8
0.77
0.93
Functional group interconversion
RX
176
0.57
0.90
Functional group addition
RX_10
22
0.55
0.86
Predicted by transferred model and ground truth
Product
Predicted reaciants by single model
HgN
HgN
OH
OH OH
Figure 3: Examples of being able make predictions with pre-training and fine-tuning. The top-1 predictions for the single model are reasonable but the model does not predict reactants for constructing ring. The 1st row (a) An example of heterocycle formation (RX_4). Synthesis N- substituted pyrrole from 1,4-diketone and alkylamine. Single model does not correctly propose reactants built pyrrole ring. The 2nd row (b) An example of C-C bond formation (RX_3). Synthetic organic chemists easily come with the Diels-Alder reaction when they see bicyclo[2.2.1 htwp:ttt...pppp-22___2 Jhept-2-ene ring system but single model does not predict any Diels-Ales Alder reaction within the top-50.
D.2 DIFFICULT CASES
Figure shows difficult examples that could not be predicted correctly even by our best model. If there are multiple similar substituents in a compound, the transferred model sometimes chooses them wrong. another case, our model fails to generate valid SMILES string. This indicates that the augment dataset still does not contain sufficient amount of reactions for polycyelic aromatic hydrocarbons. We need more data augmentation to prevent that from happening, but augmentation may be difficult to do such chemical groups are rare.
D.3
MOST PREDICTIONS ARE CHEMICALLY APPROPRIATE, THOUGH NoT EXACTLY MATCH THE GOLD ANSWER
Having confirmed the predicted results based on our expertise synthetic organic chemistry, less than 0.5% of the top 1 reactions were found to be wrong, and less than 0.2% of the cases did not output any organically correct reactions all. This a difficulty the evaluation of retrosynthesis: there are multiple reasonable (appropriate) hypotheses for reactant predictions, and the n- bebest accuracy does not perfectly match problem. At the same time, surprising that our model achieves such high accuracy without using domain knowledge or graph representation of compounds.
16

---

Under review as a conference paper at ICLR 2021
Product
Predicted transferred model
Ground truth
CHa(CHah
CHa(CHel1
CHa(CHah 14
W
CHalCHalia
CHy(CHa)
CHy(CHa)
(CHaleoOH
Br(CHe)isOH
Br(CH2layOH
(CHglgCOgH
Cl-O)(CH))CCOy
b
(Invalid SMILES)
O=C1OICC0Cocccoccccooooo ccofece 1clofeeckk2
O=CCO)CCCai-OOklz2ooooooooo cccfoselele223645
Figure 4: Examples of failed predictions. The 1st row (a). The model fails when there multiple similar substituents like long chain hydrocarbons. The 2nd row(b) Our model can predict Clem- mensen reduction reactions, but outputs incorrect SMILES substrings corresponding to the coronene substructure.
17


Reviews:
[{"review_id": "hisof0IqJln", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work examines different datasets and strategies for pretraining a Transformer model to perform one-step retrosynthesis as a SMILES-to-SMILES translation task. Three approaches are evaluated: joint training, joint training on self-labeled data, and pretraining/fine-tuning.\n\nThe related work section and problem formulation setting are rather long, but provide a reasonable overview of approaches used to date. It may behoove the authors to move some of the results and discussion from the appendix to the main text and shorten the introduction in a revision/resubmission.\n\nThe target dataset selected in this work is the USPTO-50K set, which has become a standard benchmark; that is a reasonable choice. The two larger datasets used for augmentation are “USPTO-Full” and “USPTO-MIT”. All three of these datasets were taken from the patent literature (if that wasn’t obvious by the name). In particular, they should all be strict subsets of USPTO-Full if the same preprocessing steps are used.\n\nMy major concern is that the pre-training + fine-tuning approach might not be a reasonable approach to use with these datasets. The pretraining setting is merely training on a larger dataset for the same task of identical noise/quality. The breadth of transformations in USPTO-Full is larger, but it should contain a superset of reactions and reaction types compared to USPTO-50K. This is a relatively minor domain shift, so it’s not surprising that the pretraining approach works well. I appreciate that the authors have removed exact matches where the product SMILES are identical (hopefully after removing atom mapping and canonicalizing – please confirm), but there will be an abundance of very similar training examples even without exact matches.\n\nI would certainly not consider this approach to achieve the new state-of-the-art, as is claimed in the abstract because it is merely using additional data for training, which other studies *intentionally* do not do to allow for a fair comparison. \n\nThe results in Figure 2 showing the tradeoffs between top-1 and top-20 test accuracy are interesting. This does suggest using the validation set to monitor for overfitting is worthwhile.\n\nThere is a claim that over 99% of top-1 predictions are reasonable and appropriate hypotheses. However, there is no way to evaluate this because—as far as I can tell—this data is not contained in the submission. \n\nThis work does not make any significant contributions from an algorithmic or theoretical perspective and requires additional clarifications and discussion to justify the significance of its empirical results. I do find aspects of the analysis novel, but those aspects might be more interesting to an application- or domain-focused venue.\n\nFactual corrections:\n-\tThe statement “any compound can be uniquely represented as a SMILES format string” is potentially confusing. Many distinct SMILES strings can describe the same compound; SMILES strings are not unique. \n-\tThe USPTO-50K dataset was not curated by Lowe (2012). It is from https://doi.org/10.1021/ci5006614 \n-\tThe dataset released by Lowe contains over one million reaction examples; is the 877K number after filtering?\n\n\nMinor corrections:\n-\tFigure 1 labels retrosynthesis as $p(X|Y)$ rather than $p(Y|X)$; the notation of $X$ and $Y$ is slightly inconsistent with the notation in 3.1 using $x$ and $y$.\n-\tParameters in the likelihood function are referred to as both $\\theta$ and $\\theta_\\mathcal{M}$\n-\tOptional parameters are referred to as both $\\theta^*$ and $\\theta_{\\mathcal{M}^*}$\n-\tThe notation for argmax expressions is a little unconventional\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "bTMoOJMy8cp", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary of the paper\nThis paper proposes to improve retrosynthesis models with pre-training and self-training techniques. For pre-training, the model is trained on the USPTO reaction dataset and fine-tuned on USPTO-50K dataset. For self-training, the model is trained on artificial reaction instances generated by the model (i.e., back-translation). The pre-training based approach greatly improves the seq2seq retrosynthesis model and achieves comparable performance against graph based methods.\n\n### Strength\n1) The paper conducts extensive experiments for various training paradigms: joint training, self-training and pre-training. For pre-training, authors try different options of the pre-training dataset (USPTO, USPTO-MIT). \n2) The self-training approach is an interesting application of back-translation.\n\n### Weakness\n1) Technical novelty is very weak. The pre-training approach is not novel. It is simply training on a larger training set, leveraging additional data. The self-training approach is not novel either -- it is a simple application of back-translation, which is well-studied in machine translation in NLP.\n2) The performance is weak. From Table 2, we can see that current state-of-the-art are graph-to-graph based approaches. The best result of this paper is worse than the state-of-the-art (for 1,3,5-best accuracy), even though it is trained on additional data.\n3) In principle, the pre-training method is not limited to seq2seq retrosynthesis models, but authors only apply pre-training for the seq2seq transformer architecture, which performs much worse than graph-to-graph methods.\n\n### Overall evaluation and suggestions\nI vote for rejection. Unfortunately, the weakness of the paper greatly outweighs the strength. I am afraid the technical novelty is too weak for ICLR. The results are unsatisfactory due to lower 1-best accuracy compared to prior methods. To improve the paper, authors should try to apply pre-training to graph-to-graph models, which may lead to new state-of-the-art results. Since your pre-training is just training on a larger dataset, it shouldn't be too hard to do.\n\n### Post rebuttal\nI would like to thank all the reviewers for valuable feedback. My review score stays the same.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BqLOEkw_01w", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary & Strengths\n- Recent works for retrosynthesis focus on developing neural architectures in the USPTO-50 benchmark. In contrast, this paper aims at improving existing models using additional reaction datasets with transfer learning techniques.\n- This paper tries to apply three techniques (joint-training, self-training, and pre-training+fine-tuning) into a seq-to-seq model, Transformer. As reported in the paper, pre-training+fine-tuning achieves significant improvements over the baseline. The obtained results are comparable with SOTA methods that utilize different knowledge (e.g., atom-mapping or templates) instead of the augmented datasets.\n\n---\n\nConcern \\#1: Limited contribution\n- This paper just applies the simplest transfer methods to the existing seq-to-seq model. Although transfer learning for retrosynthesis is an interesting topic, I think the methodology contribution is very limited.\n- Additionally, there is no insightful (experimental) analysis.\n\nConcern \\#2: Experiments are not enough\n- The provided experiments are insufficient to demonstrate the effectiveness of transfer learning. For example, it is hard to guarantee that the used transfer learning techniques are useful for other retrosynthesis frameworks (e.g., GLN or G2Gs). I want to note that the transfer learning literature in other domains often experiments with various scenarios (e.g., different architectures, tasks, datasets) to demonstrate the effectiveness.\n- Why the gap between pre-training+fine-tuning and joint-training is too large? I think the gap is somewhat surprising, but there is no explanation and analysis of this.\n- Is the top-k accuracy only one merit of transfer learning? In vision tasks, pre-training can improve robustness [Hendrycks'19]. If other metrics (e.g., diversity [Schwaller'19]) are evaluated, the paper would be stronger.\n\n[Hendrycks'19] Using Pre-Training Can Improve Model Robustness and Uncertainty, ICML 2019 \\\n[Schwaller'19] Evaluation Metrics for Single-Step Retrosynthetic Models, Second Workshop on Machine Learning and the Physical Sciences (NeurIPS 2019)\n\n---\n\nConclusion: The topic is interesting, but contribution seems to be very limited. More extensive experiments/analyses and/or a novel transfer technique specialized in retrosynthesis should be presented.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2tjlrJeLtvM", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper uses finetuning for SEQ2SEQ translation models to improve performance for retrosynthesis prediction. In particular, transformer models are finetuned on a large dataset to improve performance on a smaller dataset. \nSuch an investigation of fine tuning approaches for retrosynthesis is interesting for the experts working in the area and therefore it is much appreciated that the authors did this study.  \n \nWhile promising, however, the work is still premature. Unfortunately, the performed experiments do not fully support the claims made, in particular, the investigation is neither \"systematic and intensive\". Furthermore, a SOTA baseline is not cited (MEGAN, see below), which means the approach reported here actually does not achieve SOTA in any task, in contrast to what the authors claim. \n \nThe underlying idea of the paper is somewhat incremental and lacks novel insight on the ML side. With additional experiments, the paper might be better suited for a good chemoinformatics journal where it would find an expert audience, however, for a leading ML conference, this is too little. Transfer learning for reaction prediction transformer models has already been explored by Pesciullesi et al. (2020) and Chen et al. (2019). \n \nThis reviewer is not convinced that the comparison provided in the paper are meaningful. \nAll of the models in the study, not just Transformer, could be improved by finetuning. In a \"systematic and intensive\" study, the authors should apply the fine tuning to the other models as well, e.g. GLN, RetroXpert, G2G, GraphRetro, and others, and not just to Transformers, where all of the hard work of implementation has been done already by someone else. Given that Transformer without finetuning is outperformed by all of these models, it is likely that finetuning these other models on the larger dataset will also outperform finetuned Transformer models. These key experiments are missing. \n \nregarding the experimental setup used and data:\nThe USPTO50k dataset the authors use for validation is a random subset of the larger USPTO full dataset. It is therefore no surprise that a model (pre)trained on the full dataset performs better than a model trained only on a subset of that dataset. \n \nWhy do the authors not compare the performance on the full dataset then directly? In a realistic scenario, the user would train on the largest dataset available, and not on a small subset. \n \nMissing references: \n \n- Segler et al perform fine tuning of sequence based models on SMILES https://arxiv.org/abs/1701.01329 (2017)  \n- Winter et al perform pretraining using sequence to sequence models on SMILES https://doi.org/10.1039/C8SC04175J (2018) \n- Sacha et al report the MEGAN model https://arxiv.org/abs/2006.15426 which performs better than the fine-tuned transformer presented here in in top10 and top20 accuracy (87.6, 91.6) vs (87.4, 89.6). ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While the authors thought that the paper had some strong experimental comparisons, there were serious concerns with novelty and paper claims. For a stronger ML paper the authors would need to either: (a) design a new training methodology beyond pre-training that is better suited for leveraging multiple datasets for Retrosynthesis, (b) design a new model for Retrosynthesis that is better able to leverage mutliple datasets, (c) design new evaluation metrics to describe how well current methods perform in Retrosynthesis and/or metrics that describe how well methods can use data from different sources. That said, if the authors were interested to submit to non-ML venues then I agree with R2 that chemistry venues may be better suited to the paper in its current form. 
Decision: Reject