Energy-based View of Retrosynthesis
Authors: Ruoxi Sun
Hanjun Dai
Li Li
Steven Kearnes
Bo Dai
Venue:  (2021)
Keywords: Applications
Retrosynthesis
Energy-based Model

Abstract: Retrosynthesis—the process of identifying a set of reactants to synthesize a target molecule—is of vital importance to material design and drug discovery. Existing machine learning approaches based on language models and graph neural networks have achieved encouraging results. However, the inner connections of these models are rarely discussed, and rigorous evaluations of these models are largely in need. In this paper, we propose a framework that unifies sequence- and graph-based methods as energy-based models (EBMs) with different energy functions. This unified point of view establishes connections between different models and identifies the differences between them, thereby promoting the understanding of model design. We also provide a comprehensive assessment of performance to the community. Moreover, we present a novel “dual” variant within the framework that performs consistent training over Bayesian forward- and backward-prediction by constraining the agreement between the two directions. This model improves the state of the art for template-free approaches where the reaction type is unknown and known.


Under review as a conference paper at ICLR 2021
ENERGY-BASED VIEW OF RETROSYNTHESIS
Anonymous authors
Paper under double-blind review
ABSTRACT
Retrosynthesis is the process of identifying set of reactants to synthesize a target molecule. is of vital importance to material design and drug discovery. Ex- isting machine learning approaches based on language models and graph neural networks have achieved encouraging results. However, the inner connections these models are rarely discussed, and rigorous evaluations of these models are largely in need. In this paper, we propose a framework that unifies sequence- and graph-based methods energy-based models (EBMs) with different energy functions. This unified view establishes connections and reveals the differences between models, thereby enhancing our understanding of model design. We also provide comprehensive assessment of performance to the community. More- over, we present a novel dual variant within the framework that performs consis- tent training to induce the agreement between forward- and backward- ward--reddittion. This model improves the state-of-the-ar of template-free methods with or without reaction types.
INTRODUCTION
Retrosynthesis is critical problem in organic chem- istry and drug discovery (Corey) 1988 1991 Segler
let al.] 2018b [Szymkuć
al.] 2016 Kalthof
Input: Product
Output:
ett al. 2020). As the reverse process of chemical syn- thesis (Coley et al. 2017a 2019), retrosynthesis aims
to find the set of reactants that can synthesize the pro- vided target via chemical reactions (Fig [T]. Since the search space of theoretically feasible reactant candi- dates is enormous, models should be designed carefully to have the expression power to learn complex chemical rules and maintain computational efficiency.
Figure 1: Retrosynthesis and SMILES.
Recent machine learning applications retrosynthesis, including sequence- and graph-based models, have made significant progress (Segler Waller, 2017a Segler et al.] 2018b Johans- dimensinsi token sequences
al. 2020). Sequence-based models treat molecules (SMILES Weininger 1988), bottom of Fig I and formulate retrosynthesis as a sequence-to- sequence problem, where recent advances in neural machine translation (Vaswani et al., 2017] Schwaller et al. 2019) can be applied. In this principle, the LSTM-based encoder-decoder frame- works and, more recently, cansformer-based approaches have achieved promising results (Liu al.) 2017 Schwaller et al. 2019 Zheng et al., 2019). On the other hand, graph-based models have natural representation of human-interpretabble molecular graphs, where chemical rules are easily applied. Graph-based based approaches that perform graph matching with chemical rules ("templates"; definition in Sec 3.2) or reaction centers have reached encouraging results (Dai et al., 2019 Shi et al. 2020). In this paper, we focus one-step retrosynthesis, which is also the foundation of multi-step retrosynthesis (Segler et al. 2018b).
Our goal here is to provide unified view of both sequence- and graph-based retrosynthesis models using an energy-based model (EBM) framework. It is beneficial because: First, the model design with EBM is very flexible. Within this framework, both types of models can be formulated as different EBM variants by instantiating the energy score functions into specific forms. Second, EBM provides principled ways for training models, including maximum likelihood estimator, pseudo- likelihood, etc. Third, a unified view is critical to provide insights into different EBM variants, it is easy extract commonalities and differences between EBM variants, understand strengths

---

Under review as a conference paper at ICLR 2021
and limitations in model design, compare the complexity of learning inference, and inspire novel EBM variants. To summarize our contributions:
We propose unified energy- based model (EBM) framework that integrates sequence- and graph-basee based models for retrosynthesis. To our best knowledge, this is the first effort to unify and exploit inner connectivity between different models.
We perform ngooouss evaluations by running tens of experiments on different model de- signs. We believe revealing the performance the community contributes the develop- ment of retrosynthesis models.
Inspired by such a unified framework, we propose a novel dual EBM variant that per- forms consistent training over forward and backward prediction directions. dual model impreves the -tate-of-the-art aceerarc aecuracy by 9.9% for full automate cetemmatt-emplate-eeee and 2.7%
for Cor-template-based
The goal of this paper is to investigate the performance of different models under the setup without any hand-crafted chemistry features, e-g. reaction center, during training. Incorporating these hand- crafted chemistry features usually can boost accuracy significantly regardless of the model design. So adding features is not the focus of our paper. See discussion in Appendix A.2
A.3 A.4
ENERGY-BASER MODEL FOR RETROSYNTHESIS
Retrosynthesis is to predict set of reactant molecules from a product molecule. We denote the product as y, and the set of reactants predicted for one-step retrosynthesis as X. The key for ret- rosynthesis is to model the conditional probability p(X (((ly) (Dai et al. 2019 Shi et al. 2020 Liu itt al. 2017; Schwaller et al. 2019). EBM provides a common theoretical Iramework that can unity many retrosynthesis models, including but not limited to existing models.
An EBM (LeCun et al. 2006; Hinton 2012) de- fines the distribution using energy function. Without loss generality, we define the joint distribution of product and reactants follows:
Algorithm EBM framework
[Train Phase]: Learning
Input: Reactants X and products y.
1. Parameterize X and in Sequence or
Po (X,,)) exp(- (X, v))
(1)
Graph format.
((0)
2. Design Eg (e.g. dual, perturbed, bidi-
where the partition function ( Ey Ex exp(-Eo(X, is normaliza- tion constant ensure valid probability distribution. Since the design of Eg is free of choice, EBMs can be used unify many retrosynthesis models by instantiating the en- ergy function E(0) with different designs and approximation of the partition function. Note there is a trade-off between model expression capacity and learning tractability. EBM is also easy to obtain arbitrary conditioning with different partition functions. For example, the forward prediction probability for reaction outcome prediction po(y) can be written as exp(- Ex(aa)) with the same form of energy function. Overall, the proposed framework works as follows: (1) design and train an energy function Eg (Sec B and Sec H. and (2) use Eg for inference in retrosynthesis (Sec 5). See Fig 2 and Algorithm [ Based how parameterize reactant and product molecule and y, the model designs can be divided into two categories: requence-based and graph-based models.
rectional, graph- graph-based, etc} // See
3. Select training loss to learn Eg and ob- tain 0" // Sec Return 0"
(ees Phase]: Inference // Sec Input: 8", g ,Proposal P. // Sec 4. Obtain list of X candidates by P. L test + P(y)tst)
5. X arg min Egg (X, y test)
Return: X
Exp(EaxXX (X(,
MODEL DESIGN
3.1
SEQUENCE-BASER MODELS
Here we describe several sequence-based parametriztion to instantiate our EBM framework, which SMILES string representations of molecules. We first define the sequence-based notations.
2

---

Under review as a conference paper at ICLR 2021
Given reactant molecule z, denote its SMILES representation as s(z). Superseript s(x)() de- notes the character i-th position of the SMILES string. For simplicity, use x(() when possible. Reactants of a chemical reaction are usually a collection of molecules: X = {01, 22, 2 j 2 Txx}}; ). where ž; is the j-th reactant molecule. The SMILES representation of molecule set X, denoted as For
s(X), is a concatenation of s(x) for every x in X with "." in between: "s(z)).s(z2)...5(11 simplicity, we use X as the short form of s( ((X))) to denote the i-th position of this concatenated SMILES.
3.1.1 FULL ENERGY-BASED MODEL
We start by proposing most flexible EBM that imposes the minimum restrictions design of Eg. All the variants proposed in See 3.1 are special instantiations this model (e.g. by specifying different Eg). The EBM is defined as follows:
exp (X,,
p(X Ex'E((M exp Eo(X' exp((-E((X,)) v))
Here the energy function Fgg : (M) M + I takes molecule set and molecule input, and outputs a scalar value. .M defines the set of all possible molecules. (-) represents the power set. 9 (M) denotes domain of reactant X. Due to the intractability of the partition function, training involves additional information e.g., template or approximation of the partition (See Sec
3.1.2 ORDERED MODEL
One design of energy function is factoring the input sequence in autoregressive manner Schwaller et al. 2019; Segler et al. 2018a).
(Sutskever et al.
2014,
|x(X]
|s(X)
log exp (bo( .y) e((((()))
(X(w)) exp
px (x exp
Exes exp (1(:i1)))) y) eecc)
(3)
y) is parameterized by a transformer ho(p, g) : ShPl x Slv + RISI where [S] is vocabulary size. e(c) is one-hot vector with dimension set to 1. This choice of hp g) enables efficient computing of the partition function, as outputs a vector with length equal to |S to represent logits (unnormalized log probability) for each value in vocabulary. maximum likelihood estimator (MLE) is feasible for training, as this factorization allows tractable partition function.
where po (X (x (1==-11,,
3.1.3 DUAL MODEL
A different design is to leverage duality of retrosynthesis and reaction prediction. They are a pair of mutual reversible pro- cesses that factorize the joint distribution in different orders, where reaction predic- tion "forward direction" - p(yl y X)) and retrosynthesis is the "backward direction" -p(X w). With additional prior modeling, the joint probability p(X, y) factorizes either ((1lv))()) or p(y] X)p(X). We propose training framework that lever- ages on the duality of the forward and backward directions and performs consis- tent training between the two to bridge the divergence. The advantage of incorpo- rating forward direction p(y) X)) has been showed (Guo et al.] 2020). where the au- thors use sequential Monte Carlo tree al- gorithm to search for reactants that agree with forward prediction score.
Algorithm 2 Dual Model
[Train Phase]: Learning:
Input: Reactants and product
Let 0 = {%, a.
Define Eg as Eq
44
Eg log P. (X)
log (y) X) + log Pr, (X (())
1. Train backward:
n' arg min,, Lduet arg max, Ellog (X)ll)
2. Train prior and forward: Plug in 17"
m (X, v) = --X,(Xx + 1- (Xlv)
9
7², a" = arg min,,a Ieduat
= -Epmit log (X) + log
[Test Phase]: Inference: Input: 8* {~* """"", ",, 17") gest Proposal P. L +- P(y X = arg minxer. Ea. (X, pest) Return X*
3

---

Under review as a conference paper at ICLR 2021
The advantage of the duality of reversible processes has been demonstrated in other applications as well. He et al. (2016) trained a reinforcement learning (policy gradient) model to achieve duality in natural language processing and improved performances. [Wei et al. (2019) treated code summary and code generation as a pair of dual tasks, and improved efficacy by imposing symmetry between attention weights of LSTT encoder-decoder in forward and backward directions. Despite of their encouraging results, these models are not ideal for stable and efficient training for retrosynthesis, for example, policy gradient methods suffer from high variance. Therefore we propose a novel training method that is simple yet efficient for retrosynthesis task. We impose duality constraints by training forward direction on a mixture of samples drawn from the backward and original dataset. To our best knowledge, we are the first to apply duality to retrosynthesis and to impose duality constraints by samples drawn from one direction. The EBM is defined as:
(4) where prior p(X), forward likelihood p()) and backward posterior P(X are modeled as autoregressive models (See 3.1.), parameterized by transformers with parameters y, a, and 17. Note energy function can be designed free of choice.
p( (Xly) x exp (logp,(X) + log Pau ( X) + log (x =
Our consistent training is achieved by minimizing the "dual loss", where the duality con- straints in the equation below are imposed to penalize KL divergence of the two directions, i.e., (L(bakkward] forward). For simplicity. we fix the backward probability in the dual loss, and there- fore entropy (backward) is dropped.
,",0" ,ny" = -arg min laual
(5)
3,0.0
Ldual - Blogg, Pal
+ BIC, Dog (X) logpo (vlx)] (og) + Blog P (x
forward direccion
duality constraints
backward direction
Bpp px) pol E[oog (y (xlv)
where É indicates expectation over empirical data distribution 8). The duality constraints ,EE, log + log Pa P X)] is the expectation of the forward direction log py (X) + logp. X) with respect to empirical backward data distribution F. Fx where F.y Exly are ap- proximated by samples drawn from (X] (y), as is given so p(y) = 1 and |3 is scaled parameter. In our implementation we use size k-beam search to draw samples. Combining "forward" and "duality constraints" " terms (Eq 7... we can see that the first term of the dual loss is to train the forward di- rection on the mixture distribution of the original data and samples drawn from backward directions Put every piece together (Algorithm 2 and Fig 3 in Appendix). Here is our training procedure. Since we parameterize the three probabilities separately, the optimization of dual loss breaks into two steps:
miix y) = B X, y) + 14 ((y)
Step 1: Train backward. 17 does not depend on forward direction under empirical data distribu- tion. 77" = arg min, Ldual = arg max,, Ellog P,, (X|))) 77 can be learned by MLE.
Step 2:
Train prior and forward. We plug '1 into p v). 7", a- arg minl, Lauat
arg max,,a Bygii [log p.y (X) log Pa (y X)]. 7, can be learned by MLE.
Please also see ablation study of each components of dual loss in Appendix See A.6
3.1.4 PERTURBED MODEL
In contrast to the ordered model that factorizes the sequence in one direction, we use a perturbed sequential model to achieve stochastic bidirectional factorization adapted from XL.eet (Yang et al.) 2019). In particular, this model permutes the factorization order (while maintaining position encod- ing of the original order) that is used in the forward autoregressive model.
p(X 2) ((X(21), x(*2), ,..,, (=)(((0)) ly) II i=1 where the permutation order 2 is a permutation of the original order sequence 2, = [1,2, XX]
sxx) Po (x (1(1122)1)))) y) (8)
and 2; denotes the i-th element of permutation z. Here is treated as hidden variable.
4

---

Under review as a conference paper at ICLR 2021
(09:--1XX al)
Product Representation
2(0)
Reactants
88ILBS
Orderad
3h5 x
66.19 *
acanend Jnckccen
Pertarbed
0 +0
50.05
4 +
Graph
CLM
+
174.92
G2G
X+
Figure 2: EBM framework for retrosynthesis. Given the product as input, the EBM framework (1) represents it as SMILES or graph, (2) designs and trains the energy function Eg. (3) ranks re- actant candidates with the trained energy score Eig., and (4) identifies the top K reactant candidates. The best candidate has the lowest energy score (denoted by a star). The list of reactant candidates is obtained via templates or directly from the trained model.
During training, permutation order is randomly sampled and uses the following training objective: p(X y)) = cxp 2
[X| log Po (X(#)) ( 1), .y)
(9)
and the corresponding parameterization:
cxp (h(X 24, v) TeeTT)eee+++ e(x*)))
po (x (2 ,) = log Eees exp (h(X 24, y) T TT(e))
(10)
where 2; encodes which position index the permutation order to predict next, implemented by a second position attention (in addition to the primary context attention). Note that Eq (9) is actually
a lower bound the latent variable model, due to Jensen's 's inequality. However, we focus on this model design for simplicity of permuting order in training. The lower-bound approximation is tractable for training.
3.1.5 BIDIRECTIONAL MODEL
An alternative way to achieve bidirectional context conditioning is the denoising auto-encoding model. We adapt bidirectional model from BERT (Devlin et al. 2018) to our application. The conditional probability p(X(z) is factorized into product of conditional distributions of one single random variable conditioning on the others,
|s(X)]
p(Xyy) exp( Σ
log pe (X (X .))
(11)
As presented in Wang & Cho 2019), although the model is similar to MRF Kindermann 1980), the marginal of each dimension in Eq (IT) does not have a simple form as BERT training objective. may result in a mismatch between the model and the learning objective. This model can be trained by needo-likelihood (Sec 4.2)
3.2 GRAPH- BRAPH-BASED MODEL
Compared with the sequence-basee based model, the graph-based methods present chemical molecules, with vertices as atoms and edges as chemical bonds. This natural parameterization allows straight- forward application of chemistry knowledge by sub-graph matching with templates or reaction cen- ters. We instantiated three representative gragh-based approaches, namely NeuralSym (Segler & Waller 2017b), GLN (Dai et al.) 2019) and G2G (Shi et al. 2020). from the framework. Firstly, we introduce an important concept template, which can assist modeling, learning, and inference.
Templates are reaction rules extracted from existing reactions. They are formed by reaction centers (a set of atoms changed, e.g. form or break bonds). A template T consists of a product-subgraph pattern (fyy) and reactants-subgran pattern(s) (Ex), denoted as T := ty - tx, where X is a molec- ular set. We overload the notation to define a template operator T(.) : M + ((M) which takes
product as input, and returns set of candidate reactant sets. T(-) works as follows: enumerate all the templates with product-subgrapp ty matching with the given product y and define S(y) = (T : fy y, VT E T). where T are available templates; then reconstruct the reactant candidates by instantiating reactant-subee subbaapss the matched templates = {X : ix E X, YT e S(y)). The output of T(-) is R. T(-) can be implemented by chemistry toolbox RDKit (Landrum) 2016).
5

---

Under review as a conference paper at ICLR 2021
3.2.1 TEMPLATE PREDICTION: NEURALSYM
NeuralSym is a template-basee based method, which treats the template prediction as multi-class classifi- cation. The corresponding probability model under the EBM framework be written as:
p(Xly) & Erer (x) E T(y)]
(12)
where f(-) is a neural network that embeds molecule graph y, and er is the embedding of template T. Learning such model requires only optimizing the cross entropy, despite that the number of potential templates could be very large.
3.2.2
GRAPII-MATCHING WITH TEMPLATE: GLN
Dai et al. (2019) proposed a method of graph matching the reactants and products with their corre- sponding components in the template model the reactants and template jointly, with the model:
(13) where wi and w2 are graph matching score functions, and the o(-) operators defines the hard tem- plate matching results. This model assigns zero probability the reactions that do not match with the template. p(X (t) can be obtained by marginalizing over all templates.
p(X, Tly) x exp(wi (T, y) + w2(X, T, y)) bo (T))yyr(()
3.2.3 GRAPH MATCHING WITH REACTION CENTERS, G2G
In contrast with GLN, Shi et al. (2020) proposed a method to predict reaction center directly. This method closely imitates chemistry experts when performing retrosynthesis: first identify reaction centers (i.e. where the bond breaks, denoted as c), then reconstruct X.
p(Xlv) exp (log P(e v)p(cw))
(14)
All three methods mentioned above require the additional atom- atom- -aapping as supervision during train- ing, while NeuralSym and GLN require template information during inference. So NeuralSym and GLN are template-ber basedd methods. Since atom mapping plus reaction centers have almost same information as templates, we denote G2G method as www.eete--abeeeeeeeeeeeeeeeee approach.
4 LEARNING
Training EBMs is to learn parameters 0. In particular, we introduce three ways to learn exact (if applicable) or approximate maximum likelihood estimation (MLF) for full energy-based model (Sec [31.1.. as this model includes other sequence-based EBM variants (ordered, perturbed, bidi- rectional, etc) by instantiating Eg accordingly. Training EBMs with MLE is non-trivial because the partition function Z(0) in Eq (I) is generally intractable. Computing 2() involves approximation or additional information.
4.1 APPROXIMATE MLE: INTEGRATION USING TEMPLATE.
We use additional chemistry information: Templates. Direct MLE is not feasible because the par- tition function of Eq (2) involves enumerating full molecular set M, which is intractable. Here we use templates to get finite support of the partition function. Specifically, we use template operator to extract set of reactant candidates associated with denoted as T(y) As the size of T(y) is about tens to hundreds (not computationally prohibitive), we can perform exact inference of Eq ( to obtain the MLE. We denote this training scheme template learning.
4.2 APPROXIMATE MLE: PSEUDO-LIKELIHOOD.
Alternatively, we can provide an approximation of Eg (2) via pseudo-likelihood (Besag) 1975) to enable training. Pseudo-likelihood factorizes the joint distribution into the produet of conditional probabilities of each variable given the rest. Theoretically, the pseudo-likelihodd estimator yields an exact solution if the data is generated by a model p(Xly) and number of data points n + (i.e., is consistent) (Besag 1975). For the full model, training is performed as:
|=(X]|
|=(X)
exp (go (go (X,
p(Xyy) exp(
(x x x)) = exp( Σ log Enes exp (gx v; X X X'00 c))
(15)
where the superscript - indicates sequence except the i-th token and go (p. : Stpl x Slal + R
transformer architecture that maps two sequences to a scalar. As bidirectional model Sec 3.1.5 and training approaches See 4.2 (approximate joint probability) factorizes in the same way, pseudo- likelihood is a convenient way train this model.

---

Under review as a conference paper at ICLR 2021
4.3 EXACT MLE: TRACTABLE FACTORIZATION.
This training procedure only works for special case of the full model, which has tractable fac- torization of the joint probability, e.g., autoregressive models in ordered model (Sec 3.1.2) and perturbed models (Sec 3.1.4).
5 INFERENCE
With the trained Eig., inference identifies the best X that minimizes the energy function for given g/Sa, i.c. X arg minxex Egg "**). Directly solving the above minimization is again intractable, but the energy function can generally be used for ranking. Let R denote the rank of candidate X; for the given yes (lower is better).
(R(X1) R(X2) - Eg- (X-1, Eg- (X),
(16)
Practically, as illustrated in Fig 2 one can use either template-based or template-free method to come up with initial proposals for ranking, follows.
Template-baser based Ranking (TB). Templates can be used to extract list of proposed reactant can- didates by using templates. We use template operator T(-) (defined in Sec 3.2) to propose list of candidate reactant sets from the input product y. Template-free Ranking (TF). In this pa- per, template-free ranking makes proposals using the learned structure prediction model. We a simple autoregressive form for p(Xly), which can draw the top most likely samples from this distribution using beam search, which is computational efficient.
6 EXPERIMENTS
Experiment setup: Dataset and evaluation used follow existing work (Coley et al., 2017b; Dai al. 2019: Liu et al., 2017 Shi al., 2020). We mainly evaluate method benchmark dataset named USPTO-50K, which includes 50k reactions falling into ten reaction types from the US patent literature. we split the datasets into train/vliiattonntess with percentage 80% 11%6//10//52 10%. Our evaluation metric is the top-k exact match accuracy, referring to the percentage of examples where the ground truth reactant Following the common practice, we use RDKit (Landrum (016)) to canonicalize the SMILES string from different representations in different methods. We augment USPTO with random SMILES as follows: (1) Replace each molecule in reactant set or product using random SMILES; (2) Random permute the order of reactant molecules.
was found within the top i predictions made by the model.
We first present the evaluation of our best EBM variant (dual model) against existing methods (Ap- pendix A.2) for both template-based and template- free approaches in Sec 6.1 Table M our main results. Dual model statistics are extracted from Table 2 and 3 Then we provide comprehensive seque
study on different variants sequence- based EBMs in Sec 6.2 Table serves as an ablation study to understand performance of different models. Then provide template-fin timelate-free results in Table which are our second main results. We provide time and complexity analysis in Appendix
6.1 COMPARISON AGAINST THE STATE- TTAE--OF-THE-AR
Table presents the main results. All the baseline results are copied from existing works as we share the same experiment protocol. The dual model is trained with randomized SMILES to inject order invariance information of molecule graph traversal. Note that other methods like graph-based variants do not require such randomization as the graph representation is already order invariant. We see that, regarding top 1 accuracy when reaction type unknown known, our proposed dual model outperforms the current state-of-the-arr methods by 9.9% and 6.7% for template-fiee setting, and 2. 2.7% and 3.5% for remplate-based setting. Note that Dual-TF has quite close top-1 accuracy as Dual-TB, which demonstrates the discriminative ability of the designed energy function. The Dual- TB has higher top 10 accuracy due the higher coverage from templates compared the proposal obtained by p(X y) model. This suggest that with better proposal during inference, we can further boost the current performance.
6.2 REQUENCE-BASED VARIANT EVALUATION
In this section, we mainly compare different energy based sequence models described Sec 3.1 with both template-based and template-free evaluation criteria.

---

Under review as a conference paper at ICLR 2021
Table I: Top K exact match accuracy of existing methods
Reaction type unknown
Reaction type known
Calegory
Model
topl
top3
top5
top10
top1
top3
top5
top10
retrosim (Coley et NyuralSym Septer & Waller 2017b
37.3
63.3
52.9
73.8
88.1
44.4
65.3
72.4
78.9
55.3
76.0
81.4
85.1
TB
52.5
75.6
79.1
Dual-TB (Ours)
55.2
74.6
80.5
86.9
67.7
84.8
88.0
92.0
G2Gs Shi al. 2020
48.9
67.6
72.5
75.5
61.0
81.3
86.0
88.7
Semi TB
RetroXpert
A120
65.6
78.7
80.8
83.3
70.4
83,4
85.3
86.8
GraphRls1R0 (Somiall
2020
64.2
80.5
84.1
85.9
67.8
82.7
85.3
87.0
LSTM (Liu et 2017
37.4
52.4
61.7
TF
Transformar (2nin cl 2019
43.7
60.0
65.2
68.7
59.0
74,8
78.
81.1
Dual-TF (Ours)
53.6
74.6
65.7
85.9
-Dual-TB/TF: Dual model with cemplate-based or -free ranking.
Description of existing methods see Appendix A.2
Table 2: Template-Bas Basee Proposal: Top K accuracy of sequence variants
Reaction type unknown
Reaction type known
Dataset
Models
Top
Top 3
Top 5
Top 10
Top
Top 3
Top 5
Top
Full model
39.5
63.5
73.0
83.8
55.0
79.9
86.3
92.0
Ordered
47.0
67.4
75.4
83.1
60.9
80,9
85.8
90.2
USPTO 50k
Perturbed
42.9
58.7
63.9
69.6
56.6
73.6
77.2
81.6
Bidirectional
16.9
34.4
45.6
61.1
31.4
57.0
69.8
81.3
Dual
48.4
69.1
77.0
84.4
61.7
81.5
86.9
91.1
Ordered
54.2
72.0
77.7
84.2
66.4
82,9
87.4
91.0
Augmented USPTO 50k
Perturbed
47.3
64.6
70.4
75.8
64.2
79.8
83.3
86.4
Ridirectional
23.5
43.7
54.3
69.5
41.9
66.3
75.6
84.6
Dual
55.2
74.6
80.5
86.9
67.7
84.8
88.9
92.0
6.2.1 CIMPLATTE-BSSEE RANKING
Table provides the results of template-based ranking (Sec 5 TB) for each sequence model vari- ant described in See 3.1 Random SMILES has been shown usefuu for RNN and LSTM models Arús-Pous et al. (209)) We want to convey the message to the community that augmentation using random SMILES is critical for ensuring best performance for transformer by preventing overfit- ting. Without reiterating good performance for the dual variant, we focus on discussion of variants with undesired performance. The perturbed sequential model (Sec 3.1.4) and bidirectional model (Sec 3.1.5) are inferior to dual or ordered models, where the main reason possibly comes from the fact that the learning objective approximates the actual model in Eq ( and Eq IT] poorly, and thus leads to discrepancy between training and inference (See more discussion in Appendix). The full model (Sec B.1.I) despite being most flexible and achieving best top 10 performance when type is given, would suffer from high computation cost due to the explicit integration even with the tem- plates. addition to the understanding of individual models throughout the comprehensive study, we find it is important to balance the trade-off between model capacity and learning tractability. A powerful model without effective training would be even inferior to some well trained simple mod- els. Our dual model makes a good balance between capacity and learning tractability. Please
discussion about semi-template based methods in Appendix A.2 A.3 A.4
6.2.2 TEMPLATE-FREE RANKING
Table presents the results for template-free evaluation (Sec 5 TF). Template-free evaluation ap- proach proposed in this paper requires proposal model with good coverage and ranking model with good accuracy. We explored various combinations of proposal-ranking pairs. The proposal model evaluated is the ordered model trained on USPT050K and augmented USPTO50K. respec- tively. The ranking model is the dual model trained on augmented data, as it performs the best in Table Our best performer is ordered-pre pproposa (USPTO duK)-dual-ranking (aug USPTO 50K) model. We reached top 1 accuracy as 53.6% and top 10 accuracy as 77.0% when type is unknown. Note this accuracy are already close to template- bbased state-of-the-aat dual model improves accuracy upon proposal is given Fig 4 in Appendix, where it shows how the energy based re-ranking refines the initial proposal. One interesting observation is that, the proposal ordered model trained on augmented data has higher top accuracy but much lower top 10 accu- racy. This indicates that such proposal has quite low coverage in the prediction space. We observed that the model learned augmented dataset learns various representations of the same molecule (due to usage of random SMILES). A certain percentage of proposed candidates are the same after canonicalization, which is good for top prediction but undesired for proposal.
case study showing how
8

---

Under review as a conference paper at ICLR 2021
Table 3: Template-free: Translation Proposal and Dual Ranking
Type
Proposal
Re-rank
Proposal model
Top
Top
Top 10
Top.50
50
Top 100
Rank model
Top
Top 3
Top
Top 10
Ordered UPSPTO
44.4
64.9
09.9
17.2
78.0
Dual trained
68.6
70.7
74.6
77.0
No
Ordered Aug USPTO
55.6
Ang USPTO
Zhong 2019
43.7
68.7
Ohdcred on USPTO
56.0
76.1
79.7
85.2
86.4
Dual trained om
65.7
81.9
84.7
85.9
Yes
Ordered on Ang USPTO
64.7
68.5
67.3
69.7
75.7
Aug USPTO
66.2
76.1
76.8
70.7
SOTA (SCROP Zheng el 2019
88.0
74.8
78.1
61.
CONCLUSION
In this paper we proposed an unified EBM framework that integrates multiple sequence- and graph- based variants for retrosynthesis. Assisted by comprchensive assessment, we provide critical understanding of different designs. Based on this, we proposed a novel variant- dual model, which outperforms state-of- -ftt------e-tttttt in both template-basee based and template-fres setting.
REFERENCES
Josep Arús-Pous, Simon Viet Johansson, Oleksii Prykhodko, Esben Jannik Bjerrum, Christian Tyr- chan, Jean-Louis Reymond, Hongming Chen, and Ola Engkvist. Randomized smiles strings improve the quality of molecular generative models. Journal of cheminformaties, 11(1):1-13, 2019.
Julian Besag. Statistical analysis of non-lattice data. Journal of the Royal Statistical Society: Series D (The Statistician), 24(3):179-195 1975.
Connor W Coley, Regina Barzilay, Tommi S Jaakkola, William H Green, and Klavs F Jensen. Pre- diction of organic reaction outcomes using machine learning. ACS central science, 3(5):434-443, 2017a.
Connor W Coley, Luke Rogers, William H Green, and Klavs F Jenssen. Computer-assisted retrosyn- thesis based molecular similarity. ACS central science, 3(12):1277-1255, 2017b.
Connor W Coley, Dale A Thomas, Justin AM Lummiss, Jonathan N Jaworski, Christopher P Breen, Victor Schultz, Travis Hart, Joshua S Fishman, Luke Rogers, Hanyu Gao, et al. A robotic platform for flow synthesis of organic compounds informed by ai planning. Science, (65()) 1566, 2019.
EJ Corey. Robert robinson lecture. retrosynthetic thinking-essennn and examples. Chemical Society Reviews, 11:13,, 1988.
Elias James Corey. The logic of chemical synthesis: multistep synthesis of complex carbogenic molecules (nobel lecture). Angewandte Chemie International Edition in English, 30(5):455-445 465, 1991.
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. Retrosynthesis prediction with conditional graph logic network. In Advances in Neural Information Processing Systems, pp. 8870-8880, 2019.
Andrew Dalke. Deepsmiles: An adaptation smiles for use in. 2018.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint P.Xiv:1810.04500 2018.
Zhongliang Guo, Stephen Wu, Mitsuru Ohno, and Ryo Yoshida. A bayesian algorithm for retrosyn- thesis. arXiv preprint arXiv:2003.03199 2020.
Di He, Yingee Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie- Yan Liu, and Wei- Ying Ma. Dual learning for machine translation. In Advances in neural information processing systems, pp. 820- 828, 2016.

---

Under review as a conference paper at ICLR 2021
Geoffrey E Hinton. A practical guide to training restricted boltzmann machines. In Neural networks: Tricks of the trade, pp. 599-619. Springer, 2012.
Simon Johansson, Amol Thakkar, Thierry Kogej, Esben Bjerrum, Samuel Genheden, Tomas Bastys, Christos Kannas, Alexander Schliep, Hongming Chen, and Ola Engkvist. Ai-assisted synthesis prediction. Drug Discovery Today: Technologies, 2020.
Ross Kindermann. Markov random fields and their applications. American mathematical society, 1980.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69800 2014.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.022: .02810, 2017.
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alán Aspuru-Guzit Self- ies: robust representation of semantically constrained graphs with an example application in chemistry. arXiv preprint
2019.
Landrum. Rdkit: Open-source cheminformatics software, 2016.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. A tutorial energy- based learning. Predicting structured data, 1()), 2006.
Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, and Vijay Pande. Retrosynthetic reaction prediction using neural grquence-to-sqqeene models. ACS central science, 3(10):1103-1133 2017.
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: model for encertainty-calibaann chemical reaction prediction. ACS central science, 5(9):1572-1583, 2019.
Marwin HS Segler and Mark Waller. Modelling chemical reasoning to predict and invent reactions. Chemistry-A European Journal, 23(25)66118-11 18-6128, 2017a.
Marwin HS Segler and Mark P Waller. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry-A European Journal, 23(25):596659771 2017b.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2018a.
Marwin HS Segler, Mike Preuss, and Mark P Waller. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604--60, 2018b.
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph graphs framework for retrosynthesis prediction. arXiv preprint 1222:20033.12225, 2020.
Vignesh Ram Somnath, Charlotte Bunne, Connor W Coley, Andreas Krause, and Regina Barzilay. Learning graph models for template-free retrosynthesis. arXiv preprint Sikiv:20066070333 2020.
Felix
Frederik Sandfort, Marwin HS Segler, and Frank Glorius. Machine learn- ing the ropes: principles, applications and directions in synthetic chemistry. Chemical Society Reviews, 49617661154-6168 2020.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112, 2014.
Sara Szymkuć, Ewa P Gajewska, Tomasz Klucznik, Karol Molga, Piotr Dittwald, Michal Startek, Michal Bajezyk, and Bartosz Grzybowski. Computer-assisted synthetic planning: The end of the beginning. Angewandte Chemie International Edition, 55(20):50045933) 2016.
10

---

Under review as a conference paper at ICLR 2021
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp.
2017.
Alex Wang and Kyunghyun Cho. Bert has mouth, and must speak: Bert as a markov random field language model. arXiv preprint 1rXiv:1902.0444 1902.049444 2019.
Bolin Wei, Ge Li, Xin Xia, Zhiyi Fu, and Zhi Jin. Code generation as dual task of code summa-
rization. In Advances in Neural Information Processing Systems, pp.
2019.
David Weininger. Smiles, chemical language and information system. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang. Yang Yu, and Junzhou Huang. Retroxpert: Decompose retrosynthesis prediction like chemist. 2020.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. XInet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pp. 3764-5764, 2019.
Shuangjia Zheng, Jiahua Rao, Zhongyue Zhang, Jun Xu, and Yuedong Yang. Predicting ret- rosynthetic reactions using self-corrected transformer neural networks. Journal of Chemical Information and Modeling, 2019.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 TERMINOLOGY: REACTION CENTER AND SYNTHONS
Reaction center of a chemical reaction are the bonds that are broken or formed during a chemical reaction. For relrosynthesis, reaction centers are bonds exist in product, but do not exist in reactants. One chemical reaction may have multplee reaction centers. Synthons are the sub-parts extracted from the products by breaking the bonds in the reaction center. Synthons are usually not valid molecules with to indicate the broken ends in the reaction centers.
A.2 EXISTING METHODS
We evaluate of our approach against several existing methods, including both template-basee and template-free approaches. Specifically, for Template-free ones: Transformer (Zheng et al. 2019) is a transformer based approach that trains a second transformer to identify the wrong transla- tions and remove them. LSTM (Liu al., (017)) is sequence to sequence approach that use LSTM as encoder and decoder. For Template-
ones: retros im. (Coley et al. (017b) selects template for target molecules using fingerprint based similarity measure between targets and tem- plates; neura ym (Segler & Waller 2017b) performs selection of templates as multiple-class problem using MLP; GLN builds a template induced graphical model and makes prediction with approximated MAP.
For Semi-Template based ones: The three methods: G2Gs (Shi et al. 2020).. GRetroXpert YYan) let al. (020), and GraphRETRO (Somnath et al.) 2020), share the same idea: infer reaction cen- ter to generate synthons, and then complete the missing pieces (aka "leaving groups") in synthons to generate reactants. The three methods share the same first step phrasing identification of re- action center bond as classification problem. They have different second steps: from synthons to reactants. G2G uses graph generation via variational inference. Retroxpert use transformer. GraphRETRO predefine s list of possible leaving groups and phrase the problem as a classification problem. Please note G2G and Ret rexpert are generative model that can be applied to unseen molecules, whereas GraphRETRO requires the pre-defined list, which simplies the problem signif- icantly and may not apply to molecules with unseen leaving groups
A.3 TEMPLATE-FREE METHODS AND SEMI-TEMPLATE BASED METHOOSS
In this paper, we refer template-free methods as the approaches that do not require any chem- istry hand-crafted features, including templates, sub-parts of templates, reaction centers, and atom mapping, More precisely, our experiment setup is full automatic template-free methods. We are interested in this set up because
These handcrafted features are often not accessible for real world application. Get- ting these features ether requires intensive human efforts or non-trivial computational chal- lenges. For example, the ground truth "reaction center" used in remi-template based meth- ods, have to be manually labeled by human experts when atom coommapping is not available. So it is prohibitive for large datasets. Alternatively, reaction centers can be computationally extracted using atom-mapping. However, obtaining good quality atom mapping for large datasets itself is an equally challenging problem.
Think beyond existing rule (e.g. handcrafted features). The future retrosynthesis mod- els should be able to think beyond existing chemical rules, and achieve full automation without handcrafted features. We refer existing rules as templates, reaction center, atom mapping, reaction type, etc. Thinking beyond exiting rules denotes generating unknown chemical rules. Similar to the computer vision field (e.g. imagenet classification), people used handcrafted features at the beginning and all move to full automation without any handerafted features because full automation achieves much better results. Unfortunately, we not there yet. Including hand-crac crafted features often lead to dramatic improvement (10%+) regardless of model design. For example, well- well-known examples are reaction type and templates. Please also so next section.
12

---

Under review as a conference paper at ICLR 2021
In this paper, refer methods that use additional chemistry features during training semi- template-based methods. For example, G2Gs (Shi et al. 2020), GRetroXpert (Yan et al. and GraphRETRO (Somnath et al. 2020). See description of the methods in Appendix A.2 These
2020)
methods do not apply template-man canplatt-mmaching explicitly, so they are not template-basee methods. How-
ever, they use "reaction centers" as additional information to supervise their algorithm during train- ing. The reaction centers preserve almost equivalent information as templates once reaction cen- ters are given, the product can be broken into two parts (denoted as synthons). The templates can be recovered by removing nonessential atoms from synthons and product. In addition to reaction cen- ters, RetroXpert also add additional features "the product side of template" to its atom features. GraphRETRO has pre-defined list of leaving groups. Therefore, we denote G2G, RetroXpert and GraphRETRO as semi-template based methods.
A.4 DUAL MODEL PERFORMANCE WITH REACTION CENTER GIVEN
Semi- Semi-template based methods Ret Retrexpert Yan al. 2020), and GraphRETRO (Somnath et al. 2020) achieved impressive results. Their performance even outperforms template-based approach.
The models presented in this paper have different experiment setup from Ret roxpert and GraphRETRO. Our setup does not use additional chemical information: reaction center, during training, which makes a head-to-head comparison not meaningful. To further investigate the effect of this additional chemistry features, we asked what will happen if our models also have the infor- mation of "reaction center"" Can we have the same or better performance when reaction center is given? As a proof of concept, we incorporated the reaction center to the dual model as follows: we first generate synthons from product using the given reaction centers, and the then concatenate the generated synthons to the input of our dual model.
Table 4: Template-free with reaction center: Translation Proposal and Dual Ranking
Type
Proposal
Re-rank
Proposal model
Top
Top
Top 10
Top 50
Top 100
Rank model
Top
Top
Top 5
Top 10
Drdered UPSPTO
65.8
Dual trained
70.1
86.3
89.0
90.3
Aug USPTO
Ordered on Aug USPTO
68.7
70.0
70.5
72.1
74.7
69.9
74.4
74.6
74.7
RetroXpent (Yan at 5020
65.6
78.7
80.8
86.8
Ordered on USPTO
67.7
85.7
88.0
91.6
91.9
Dual trained on Aug USPTO
73.2
88.4
90.8
91.6
Ordered on Aug USPTO
77.4
77.7
77.7
RetroXpert Yan et al.
2020
70.4
83.4
85.3
86.8
Compared with Table 3 (when no synthons information is available), Table 4 shows adding reaction centers can boost accuracy significantly: When reaction type is unknown, the accuracy of final retrosynthesis improves from 54.5% to 70.1%. When reaction type is known, the accuracy improves from 66.2% 73.2%. An interesting observation is that when reaction centers presented, the effect of reaction type and augmentation on positively increasing the accuracy is reduced. One explanation is reaction center is so important that dominates all the other useful features or tricks.
Although in the above experiments, we are given the reaction center, that we do not infer them, Ta- ble Dle4still still serves well as proof of concept that incorporating useful handcrafted chemistry features can significantly boost accuracy regardless of model design. To be more precise, Table is an upper bond of Dual model when we use inferred reaction center as input if we are given reaction center as additional information during training. Dual with reaction center given is NOT remplate-free methods. It is semi-template based methods as RetroXpert, GraphRETRO. G2G.
In addition to reaction centers, RetroXpert also add additional features "the product side of template" to its atom features. GraphRETRO has a pre-defined list of leaving groups Incorporating those information (i.e. product side if template) in our setup may boost accuracy significantly again.
We want to avoid pursuing the path of chasing the state of the art by incorporating many useful hand- crafted features, as it is contradictory to full automatic template-free methods. We want to deliver the message to the community that the performance of models should be compared under the same setup. The performance gain may be result of using extra information, instead of modeling.
13

---

Under review as a conference paper at ICLR 2021
A.5
DUAL MODEL FIGURE
Forvard
Dual Conetratnt
Rackoaard acscenlngnO0 +legolyf) -lggaott
Learning
Figure 3: Dual model. (a) Learning consists of training three transformers: prior p(X) (green), likelihood p(y| (X) (blue), and backward p(X y) (orange). Dual model penalizes the divergence between forward p( X X))(p(()) and backward direction p(y) X) with Dual constraint (highlighted). (b) Inference Given reactant candidates list, we rank them using Eq 4
A.6 ABLATION STUDY OF THE DUAL LOSS
We perform ablation study investigate whether training with the dual loss, in particular the dual constraint, is the reason for the performance improvement. Recall the dual loss is given in Eq O and the dual constraint is the middle term in Eq 66_ See Table [ The "dual" row indicates training
with dual loss and the results are taken from Table ! E log P. (X) + log (y(X)) + log Pr, (Xly))
is dual loss without the dual constraint. Ellog Po ()) + log Pry [[y)) is without prior log py (X). log Pty (X ([y) [y]] is only including backward direction. Table shows each component of the dual loss contribute to the performance positively.
Table 5: Ablation Study of dual loss when reaction type is known
Aug USPTO
Top 1
Top
Top 5
Top 10
Dual
67.7
84.8
88.9
92.0
E Elog P-, (X) + Po
Pa (y) + (X)
67.0
84.7
88.9
91.95
E llog Pa ()) + 1, (X]y]
66.1
82.8
87.6
91.3
log Pr, (Xly)
60.9
80.9
85.8
90.2
A.7 TIME AND SPACE COMPLEXITY ANALYSIS
In this section, we provide time and space complexity regarding model design choices. As the main bottleneck is the computation of transformer model, we measure the complexity in the unit of trans- former model calls. For all the models, the inference only requires the evaluation (un-normalized) score function, thus the complexity is O(1); For training, the methods that factored have easy form of likelihood computation, where a diagonal mask is applied to input sequence so that au- toregressive is done in parallel (not [s(x:) times), so it requires (11) model calls. This include seterrdd/erurhhdddddderciiooaaoe
models. For the full model trained with pseudo-likelihood. it requires O[XX] (SS) calls due to the evaluation per each dimension and character in vocabu- lary. Things would be bit better when trained with template-based method, in which it requires ((I(()) calls, which is proportional the number of candidates after applying template operator.
As the memory bottleneck is also the transformer model, it has the same order of growth as time complexity with respect to sequence length and vocabulary size. summary we can see the Full model has much higher cost for training, which might lead to inferior performance. Our dual model with a consistency training objective has the same order of complexity than other autoregressive ones, while yields higher capacity and thus better performance.
A.8 TRANSFORMER ARCHITECTURE
The implementation of variants in framework is based on OpenNMT-py (Klein et al. 2017). Fol- lowing /Vaswani al. 2017), transformer is implemented as encoder and decoder, each has
14

---

Under review as a conference paper at ICLR 2021
self-attention layers with 8 heads and weed-forward layer of size 2048. We model size and word embedding size as 256. Batch size contains 4096 tokens, which approximately contains 20-
200 sequences depending on the length of sequence. We trained for 500K steps, where each update uses accumulative gradients of four batches. The optimization uses Adam (Kingma & Ba 2014)
optimizer with 81 0.9 and $2 0.998 with learning rate descrihed in (Vaswani al. 2017)
using 8000 warm up steps. The training takes about 48 hours on single NVIDIA Tesla V100. The setup is true for training transformeer-ased models, including ordered sequential model (See 3.1.2). perturbed sequential model (.1.(3.1.4) bidirectional model (Sec 3.1.3). dual model (Sec 3.1.3.11.3). As for full model (Sec 3.I.I). each sample contains 20-500 candidates. We implemented each batch only contains one sample. Its tens or hundreds of candidates are computed in parallel within the batch. The model parameters are updated when accumulating 100 batches to perform one step of update.
follows:
A.9 EXAMPLE OF CASE STUDY
Here we provide another case study showing with dual model ranking (Sec 3.1.3) the accuracy improves upon translation proposal. Please see Fig and Fig
NH
MH
Translation Proposal
ground
Dual Model Re Re-rank
NH
2
NHg
ranking
Figure 4: Dual ranking improves upon translation proposal. Left and right column are the top three candidates from translation proposal and dual re-ranking of the proposal. Ground truth (GT) is given at the top and is labeled orange in the middle. By dual re-ranking, the GT ranks the first place, whereas the 3rd place in the proposal. Note that the first place in the proposal is only atom different from GT (Br vs I), indicating the dual model is able to identify small changes in structure.
A.10 DISCUSSION
V11 Full model (Sec 3.1. Full model (Sec 3.1.1 with template learning reaches accuracy of 39.5% and 53.7% on USPTO50K data-sets. Full model is partially limited by expensive computation due to the number of candidates per product.
V.2 Perturbed sequential model (Sec Perturbed sequential model has about ~ 4% accuracy loss top accuracy compared with ordered model (.122. We argue the reason are as follows: firstly, designed Eg the middle term of Eq. Eq (9) to facilitate perturbing the order during training. following (Yang et al. 2019). However, due to Jensen's 's inequality, this design not equal P(Xly), which causes discrepancy in ranking (inference).
V.3 Bidirectional model (Sec 3.1.5) Bidirectional model, however, does not perform well in our makes the prediction of one position given all the rest y) almost perfect (99.9% accuracy in token-level). However, due to
experiments. The bidirectional-wwarene
of the sequence the gap between pseudo-liked likelihood and maximum likelihood, i.c., log P PX(y)) the performance for predicting the whole sequence will be inferior, observed in the experiments.
15

---

Under review as a conference paper at ICLR 2021
Translation Proposal
ground truth
Dual Model Re-rank
NH
Mpt
Mge
ranking
Figure 5: Dual ranking improves upon translation proposal. Another example. Descriptions see 4
A.11
ALTERNATIVE OF SMILES: DEEPSMILES AND SELFIES
In this section, explore the effect of prepossessing procedure of sequence-based model, e.g. inline representation of molecular graph, effecting performance of sequence-base model. In particular, deepSMILES (Dakke 2018) and SELFIES (Krenn et al. 2019) alternatives to SMILES. Without loss of fairness, we evaluated these representations using Ordered sequential model (Sec The results indicate SMILES work the best. We speculate the reason are deepSMILES and SELFIES on average longer than SMILES, leading to higher probability of making mistakes on token level and therefore low sequence-level accuracy.
Table 6: deepSMILLES and SELFIES
SMILES
Models
Top 47.0
Top 67.4
Top 75.4
Top 10
Ordered
83.1
deepSMILES
Ordered
46.08
65.87
73.54
81.51
Selfies
Ordered
43,00
62.51
70.16
79.07
A.12 TRANSFORMER IMPLEMENTATION OF PERMUTATION INVARIANT OF REACTANT SET
Transformer has a position encoding to mark the different locations on an input sequence. We modified the position encoding such that each molecule starts with 0 encoding. instead of the catenated position in the reactants sequence. The results are Table 7 We can see that this position encoding is beneficial for non-augment data, but not augment data, as the latter has already consid- ered the permutation invariance order of reactants by data augmentation. In this paper, we use data augmentation to maintain order-invariant for reactants.
16

---

Under review as a conference paper at ICLR 2021
Table 7: Transformer model with permutation invariant position encoding
Reaction type is unknown
USPTO 50k
Models
Top
Top
Top
Top 5
Top 10
Ordered
46.97
60.71
67.39
75.35
83.14
Ordered Permutation invariant
47.29
61.29
68.08
75.37
83.36
Augmented data
Ordered
54.24
66.33
72.02
77.67
84.22
Ordered Permutation invariant
53.45
66.61
72.58
78.33
85.42
17


Reviews:
[{"review_id": "jHp9SfIkHw", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary of the paper\nThis paper proposes an energy based model (EBM) for retrosynthesis. The best model (dual model) leverages the duality of retrosynthesis and reaction prediction. The EBM contains three factors: prior on reactants $p(X)$, forward reaction probability $p(y | X) and backward posterior $P(X|y)$. The duality loss penalizes the KL divergence of the two directions. The forward predictor is trained on a mixture of original data and samples drawn from the backward predictor. The dual model shows improvement over template-based and template-free baselines.\n\n### Strength\n1. The dual model is novel and interesting. EBM provides a unified view of forward and backward reaction prediction. It will be interesting to see how this improves the forward prediction performance.\n2. EBM is a flexible framework, which can be applied to both template-based and template-free approaches.\n\n### Weakness (and questions)\n1. Clarity: The paper describes EBM view of many methods, ranging from sequence and graph based methods. As a result, each subsection is too sketchy and lots of details are missing. For example:\n 1. For Dual-TB (ours), what is the exact model architecture? I understand that the candidates are generated from reaction templates, but what's the parametrization of $p_\\alpha, p_\\gamma, p_\\eta$? Is it GLN?\n 2. For Dual-TF (ours), what is the model architecture? Is it transformer?\n 3. What is the augmented USPTO 50k? Is it randomized SMILES? For Dual-TB (ours), is it trained on augmented USPTO 50k? Why SMILES randomization matters for templated based methods?\n 4. $X$ is a set of compounds, how do you generate a set of compounds in an order-invariant manner?\n 5. For duality constraint, you sample from backward predictor using beam search. Why not the other way around? Why not sample from forward predictor and train your backward predictor on the additional samples?\n\n2. At inference time, it is hard to sample from EBM. Therefore, authors propose to rank the candidates generated from other models (reaction templates or transformers). This really limits the performance (and applicability) of the approach. To my knowledge, you can do MCMC based on Langevin dynamics to sample from EBM. Is this not possible for retrosynthesis?\n\n3. The best model is this paper actually performs much worse than the state-of-the-art models. For instance, RetroXpert (Yan et al., NeurIPS 2020) achieves 65.6% top-1 accuracy (reaction type unknown) and 70.4% top-1 accuracy (reaction type known). This is much better than the dual model (55.2%, 67.7%). Somnath et al, 2020 also achieves 64.2% top-1 accuracy (reaction type unknown), which is much higher than dual model (55.2%).\n\n### Overall evaluation\nI vote for weak reject of the paper, primarily due to the weaker result and lack of clarity. I believe the dual formulation can be applied to these above state-of-the-art models (if code is available). I suspect the weak result is primarily due to the base model (e.g., transformer). I am happy to adjust my score if there are stronger results and the clarity can be improved. One suggestion for clarity is to put perturbed / bidirectional models into appendix since they are not helpful anyway... \n\n### Post Rebuttal\nI would like to thank authors for their response. I think the paper needs to be improved further to get accepted. I do believe that the proposed dual learning method is promising, but empirical evaluation is still lacking. So my review score stays the same.\n\n[1] Yan et al., RetroXpert: Decompose Retrosynthesis Prediction like A Chemist, NeurIPS 2020.\n\n[2] Somnath et al., Learning Graph Models for Template-Free Retrosynthesis, 2020", "extra_scores": {}, "extra_text": {}}, {"review_id": "_99sSejhs7", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "SUMMARY\n\nThis paper uses the statistical physics-inspired energy-based model formalism to study the by now \"canonical\" problem of retrosynthesis using deep learning. The authors use an interesting variant that combines forward and backward prediction. The authors use template-based and template-free models.\n\n\nPROS\n\n- This reviewer believes that energy-based models have an elegance and connection to statistical mechanics that should be explored more in the area of machine learning. This work goes in this interesting direction.\n- Based on the above, and as far as the reviewer is appraised, this is a unique, non-derivative direction in the field and therefore deserving of consideration for acceptance.\n- The dual model seems to be very useful given the increase in template-based and non-template-based model performance. This could be applied to other transformer-based tasks in chemistry and graph-based ML\n- The authors compared their models to a variety of SOTA models and approaches, they also were thorough and explored both DeepSMILES and SELFIES.\n\n\nCONS\n- Some of the mathematical formalism could be moved to supplementary to allow for better discussion.\n\n\nMINOR FORMATTING\n- The authors may want to give the manuscript a pass for grammar. There are missing articles in a few sentences.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "FDC34zfDlvL", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary #\nThis paper introduces a re-interpretation of seq2seq and graph2graph retrosynthesis models with energy based models (EBMs). \nEBM is a general log-linear framework to model joint distributions of a feature variable X and a target variable y, first introduced to connect discriminative (DNN) models with generative (DNN) models. This paper shows the reformulations of typical seq2seq retrosynthesis models and graph2graph models. Also, the paper proposes a dual training model that optimizes both the forward path model and the backward path model. \nExperimental results show that the dual training models perform better than conventional retrosynthesis models in template-based and template-free retrosynthesis frameworks. \n\n# Comments #\n\nI have two major concerns. \n\nFirst, I cannot clearly understand a new insight or knowledge that is brought by the EBM-based re-formulation of retrosynthesis models. \nAny probabilistic models can be described by the full joint distribution of all variables. In my understanding, the EBM is a way of modeling the joints with the log-linear model plus the potential function. \nIt seems for me that the current manuscript is successful in re-wiring the existing retrosynthesis frameworks into EBMs, but that is all. \nPlease clarify what readers can learn about \"connections and ... differences between models, ...understanding of model design [abstract]\" from the EMB modelings. \n\nSecond, the State-of-the-Art of the retrosynthesis in the manuscript is somewhat outdated. \nTo the best of my knowledge, the current best-performing retrosynthesis models are [1] and [2], and the scores of these models are higher than the reported results of the proposed dual models. \n\nI think the current manuscript needs modifications to appropriately cite these papers and replace the standpoint of the submitted work. \n\nPersonally, I am interested in how the EMB can interpret these latest models. \n\n\n[1] Somnath+, \"Learning graph models for template-free retrosynthesis\", arXiv:2006.07038, June 2020. \n[2] Yan+, \"RetroXpert: Decompose retrosynthesis prediction like a chemist\", chemrxiv:11869692.v3, June 2020. \n\n# Evaluation points #\n\n(+) First to apply the EBM for retrosynthesis prediction models\n\n(+) Proposed a forward/backward simultaneous (dual) training\n\n(--) It is unclear what we can learn by rewriting the retrosynthesis models with EBM. \n\n(-) Reported results do not update the State-of-the-Art prediction accuracy of retrosynthesis models in the literature\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "74mJtnpyS1e", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors use the framework of energy based models to describe several known approaches for ML-based retrosynthesis in a unified way. This allows to combine retrosynthetic (backward) and reaction prediction (forward) in a principled way.  \nBased on this analysis, a dual model is proposed which used a duality constraint as a regulariser, leading to improved performance over several baselines models, but not over SOTA (see below!).\n\nThe theoretical analysis alone is very interesting, and well described. However, I have a few concerns with missing ablation experiments and the positioning, which could be addressed to make the paper stronger in my opinion.\n\nOverall, I think this paper should be accepted at ICLR after the following points have been addressed (or the authors commit to provide the additional data in the camera ready version if the time of the rebuttal phase is too short to run additional experiments). However, in the current form, the paper is not ready. My evaluation is for the current form of the paper, and I am happy to change it significantly during the rebuttal.\n\n\nAblation Experiments:\n\nIs the duality constraint actually needed? It would be important to perform the ablation experiment where $\\beta$ is set to 0.\nIn other words, what happens if you just use $\\log p(X|y) + \\log  p(y|X)$ to rank the candidates? If the authors perform these additional experiments and report the numbers, I will increase my score regardless of the outcome of the experiments.\n\n\nPrior work:\n\nThe state of the art claim is not correct. Yan et al achieve higher performance on the same dataset https://chemrxiv.org/articles/preprint/Interpretable_Retrosynthesis_Prediction_in_Two_Steps/11869692/2 which has already been published last February! \nTherefore, please remove the SOTA claim from the paper, and acknowledge the Yan et al work. \nFor this reviewer well-motivated modelling, honest analysis and proper experimentation is more important than chasing SOTA, and not achieving SOTA will not affect the evaluation negatively.\n\nAlso, the connection of forward reaction and retrosynthesis prediction via Bayes Theorem has been studied here https://arxiv.org/abs/2003.03190 which should be acknowledged.\n\nOther work preceding has also used a combination of backward and forward prediction. For example, Segler et al Nature 2018 and Coley et al Science 2019 use a model for p(X|y) to propose disconnections combined with a model for p(X,y) to remove low probability solution.\n\nThe usefulness of SMILES augmentation has been previously shown, please cite the previous work by Arus Pous et al https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0393-0 \n\nThe Coley et al 2017a paper that the authors cite in the introduction is a wonderful paper, however, it is not concerned with retrosynthesis. I would suggest to cite the review article by Strieth-Kalthoff et al https://doi.org/10.1039/C9CS00786E instead, which provides a good overview over ML approaches for (retro)synthesis. Furthermore, I would suggest to cite Segler & Waller 2017 in the introduction, which was the first paper to suggest deep neural networks for both retrosynthesis & reaction prediction (see also https://doi.org/10.1016/j.ddtec.2020.06.002 )\nColey 2017a should be cited later in the context of forward prediction.\n\nComments:\n\nTable 3 is a bit unclear. With Type, do you mean the reaction type is given?\n\nsmall things:\n- in eq 13, the is a low dot, should this have been a centred dot for multiplication?\n\n- Please don't use Google house fonts for your figures,  to maintain anonymity.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Before the discussion phase nearly all reviewers had doubts about the comparison of the current work with state-of-the-art works (notably Yan et al., 2020, RetroXpert, and GraphRETRO). The authors then compared with these works and emphasized that these works rely on hand-crafted features. They argue that the fairest comparison is the one where each method uses the same sort of features during train/test time. This is because in certain real world settings we may not have accurate estimates of such features (e.g., atom mappings, templates, reaction centers). However, in the revised version of the paper the authors did not adhere to this concept of fair comparison in Table 4 of Appendix A.4. Here their method uses reaction centers as input while baselines do not. While the authors claimed that the comparison here was designed to show how reaction centers provided as input improved performance, this doesn't seem like a good way to show it: to isolate the improvement due to reaction center inputs you should fix everything else, i.e., the rest of the method. 

Apart from the above contradiction, I buy the arguments of reviewers that distinguishing between methods that use hand-crafted features and those that do not is not a meaningful distinction. One can apply atom-mapping or reaction center discovery algorithms as data preprocessing before applying other methods. Ablation studies where such preprocessing is added or removed are interesting, but it is completely fair for any method to use such preprocessing before applying their method, it is up to the modeller. 

I would have argued for acceptance had the authors either (a) just included results from SOTA methods (one, RetroXpert was published 1 month after the ICLR submission deadline), and/or (b) reran their approach with such preprocessing. However the authors ended up hurting the submission by emphasizing a difference between using handcrafted features and not, then contradicting their experimental setup in Table 4.

This is a good paper, but I agree it is not ready to be accepted at ICLR. I recommend the authors do the following: (a) use any preprocessing they want for their method and compare with the state-of-the-art, (b) if they want they can run their method without any preprocessing as an interesting ablation study, (c) remove Table 4 (as (b) already does this type of an ablation study), (d) describe recent work through the lense of EBM, (e) resubmit to a strong ML conference. The new submission will be much stronger.
Decision: Reject