Towards understanding retrosynthesis by energy-based models
Authors: Ruoxi Sun
Hanjun Dai
Li Li
Steven Kearnes
Bo Dai
Venue: NeurIPS 2021 Poster (2021)
Keywords: chemical application
energy-based model

Abstract: Retrosynthesis is the process of identifying a set of reactants to synthesize a target molecule. It is of vital importance to material design and drug discovery. Existing machine learning approaches based on language models and graph neural networks have achieved encouraging results. However, the inner connections of these models are rarely discussed, and rigorous evaluations of these models are largely in need. In this paper, we propose a framework that unifies sequence- and graph-based methods as energy-based models (EBMs) with different energy functions. This unified view establishes connections and reveals the differences between models, thereby enhancing our understanding of model design. We also provide a comprehensive assessment of performance to the community. Moreover, we present a novel dual variant within the framework that performs consistent training to induce the agreement between forward- and backward-prediction. This model improves the state-of-the-art of template-free methods with or without reaction types.  

Towards understanding retrosynthesis by energy-based models
Ruoxi Sun', Hanjun Dai2, Li Li Steven Kearnes', and Bo Dai²
Google Cloud AI Google Brain 'Google Research (ruoxis, hadai, leeley, kearnes, bodai) @google.com
Abstract
Retrosynthesis is the process of identifying set of reactants to synthesize a target molecule. It is critical material design and drug discovery. Existing machine learning approaches based on language models and graph neural networks have achieved encouraging results. However, the inner connections of these models are rarely discussed, and rigorous evaluations of these models are largely in need. In this paper, we propose a framework that unifies sequence- and graph-based methods energy-base models (EBMs) with different energy functions. This unified view establishes connections and reveals the differences between models, thereby enhances our understanding of model design. We also provide comprehensive assessment of performance to the community. Additionally, we present a novel dual variant within the framework that performs consistent training to induce the agreement between forward- and backward- couudddeedcctio.. This model improves the state-of-the-an of template-free methods with or without reaction types.
Retrosynthesis is critical problem in organic chemistry and drug discovery [H5). As the reverse process of chemical synthesis 76, retrosynthesis aims to find the set of reactants that can synthesize the provided target via chemical reactions (Fig I.. Since the search space of theoretically feasible reactant candidates is enormous, models should be designed carefully to have the expression power to learn complex chemical rules and maintain computational efficiency.
Recent machine learning applications on retrosynthe- sis, including sequence- and graph-based models, have made significant progress [3] 9.. Sequence-based models treat molecules as one-dimensional token se- quences (SMILES [IO], bottom of Fig [ and formu- late retrosynthesis as sequence-to-sequenee where recent advances in neural machine translation can be applied. In this principle, the LSTM- based encoder-decoder frameworks and, more recently, transformer-basee approaches have achieved promising
Input: Product
Output: Reactants
problem,
Deicent Lortee---eeetee S0etemme
Ceternt (nd(///nne
DiOstecncei
[1 [2]
Figure 1: Retrosynthesis and SMILES.
results 13 12 [4]. On the other hand, graph-based models have a natural representation of human- interpretable molecular graphs, where chemical rules are easily applied. Graph-based approaches that perform graph matching with templates (e.g. chemical rules) or reaction centers have reached encouraging results. Among those, G2Gs [5], RetroXpert [16] and GraphRETRO (17] outperform template-bass based methods by inferring reaction centers in a supervised way. In this paper, we focus on one-step retrosynthesis, which is also the foundation of multi-step retrosynthesis 3
Our goal here is to provide unified view of both sequence- and graph-based retrosynthesis models using energy-based model (EBM) framework. It is beneficial because: First, the model design with EBM very flexible. Within this framework, both types of models can be formulated as different
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

EBM variants by instantiating the energy function into specific forms. Second, EBM provides principled ways for training models, including maximum likelihood estimator, -iuuo-likelihood etc. Third, a unified view is critical to provide insights into different EBM variants, as it is easy to extract commonalities and differences between EBM variants, understand strengths and limitations in model design, compare the complexity of learning or inference, and inspire novel EBM variants. To summarize our contributions:
We propose a unified energy- based model (EBM) framework that integrates sequence- and graph-based models for retrosynthesis. To our best knowledge, this the first effort to unify and exploit inner connectivity between different models.
We perform rigorous evaluations by running tens of experiments on different model de- signs. Revealing the performance to the community contributes to the development of retrosynthesis models.
Inspired by such unified framework, we propose novel generalized dual EBM variant that performs consistent training over forward and backward prediction directions. This model improves the state-of-the-arr by 4.3%.
Energy-based model for Retrosynthesis
Retrosynthesis is to predict a set of reactant molecules from product molecule. We de- note the product y, and the set of reactants predicted for one-step retrosynthesis as X. The key for retrosynthesis is to model the conditional probability p( (XIy)) EBM provides a common theoretical framework that can unify many ret- rosynthesis models, including but not limited to existing models.
Algorithm 1 EBM framework
[Train Phase]: Learning
Input: Reactants X and products y.-
1. Parameterize X and y in Sequence or Graph format.
2. Design Eg {e.g. dual, perturbed, bidirec- tional, graph-based, based, etc) // Sec
An EBM defines the distribution using an energy function [88 [9] Without loss of generality, we define the joint distribution of product and reactants as follows:
3. Select training loss to learn Eg and obtain 6* Sec
Return 0*
[Test Phase]: Inference // Sec
Po (X, = exp( 1op(-Fo(XX Ex(
Input: 0", yytest, Proposal P. // Sec
(())
(1)
4. Obtain a list of candidates by P. Lest + P(ytext)
where the partition function 7(0)
5. x arg min me Eg (X,y Return: X
E, Ex exp is a normaliza- tion constant ensure a valid probability distribution. Since the design of Es is free of choice, EBMs can be used unify many retrosynthesis models by instantiating the energy function E(0) with various designs. Note there is trade-off between model expression capacity and learning tractability. EBM is also easy to obtain conditioning with different partition functions. The forward
prediction probability for reaction outcome prediction pe (yyl X) can be written as Σ exp( Fo (")) with the same form of energy function.
exp(=Ex(X,))
The proposed framework works as follows: Step 1, design and train an energy function Eg (Sec 2
and Sec 3, and Step 2 use Eg for inference in retrosynthesis (Sec 4). See Fig and Algorithm I
2
Model Design
Based on how to parameterize reactant and product molecule X and y, the model designs can be divided into two categories: sequence-based and graph-based models.
2.1
Sequence-based Models
Here we describe several sequence-based parametriztion to instantiate our EBM framework, which use SMILES string as representations of molecules. We first define the sequence-basse based notations.

---

Qul-AxXX,g)) 219)
Product
Representation
Reactants
Po(X;a)
Ordered
-0,00 +
46.19
Octceet -náef 3cence3 (eenec))neSosos cac2ndjao1
Perturbed
0 +
Bidizectional
000 1:0
130.45
Graph
Dual
GLN
+
176.92
G2G
x
196.30
Figure 2: EBM framework for retrosynthesis. Given the product as input, the EBM framework (I) represents the product molecule SMILES sequence graph, (2) designs and trains the energy function Eg. (3) ranks reactant candidates with the trained energy score Eg-, and (4) identifies the top K reactant candidates. The best candidate has the lowest energy score (denoted by star). The list of reactant candidates is obtained via templates (template based proposal) or directly generated by the trained model (template free proposal).
Given reactant molecule x, denote its SMILES representation s(x). Superscript (1x) denotes the character at i-th position the SMILES string. For simplicity, we use a(() when possible. Reactants of a chemical reaction are usually a collection of molecules: X = {z1, T2, Ij. x x where æj is the j-th reactant molecule. The SMILES representation of a molecule set X, denoted as s(X), ), is concatenation of s(x) for every x in X with "" in between:
We X the short form of s(X) s(X)() denote the i-th position of the concatenated SMILES.
2.1.1 Full energy-based model
We start by proposing a most flexible EBM that imposes the minimum restrictions on the design of Eg. All the variants proposed in Sec 2.c2.11p are special instantiations of this model (e.g. by specifying different Eg). The EBM is defined as follows:
exp (x
pf p(XW))
(2)
EXIE(M)) exp (- EX()
x exp( (x(X.))) v))
(3)
Here the energy function Fg (M) M + IR takes a molecule set and a molecule as input, and outputs scalar value. M defines the set of all possible molecules. (-)) represents the power set. P (M) denotes domain of reactant sets X. Due to the intractability of the partition function, training involves additional information e.g., template approximation of the partition (See Sec 3)
2.1.2 Ordered model
One design of energy function is factoring the input sequence in an autoregressive manner
12
20
|=(X)
(X)) = exp
log po (+) (x(
(4)
Is(X)
exp (ho 2) e(x())))
exp Σ log
(5)
Enesexy exp (X y) e((c))
where Po( (x (xx-))) is parameterized by a transformer (p. 9) S/v S/vl + RIS where
S is vocabulary. e(c) is one-hot vector with dimension set to 1. This choice of he q) enables efficient computing of the partition function, as it outputs vector with length equal to |S| to represent logits (unnormalized log probability) for each value in vocabulary. Here, maximum likelihood estimator (MLE) is feasible for training, as this factorization allows tractable partition function.
2.1.3 Dual model
A different design is to leverage duality of retrosynthesis and reaction prediction. They are pair of mutual reversible processes that factorize the joint distribution different orders, where reaction prediction is "forward direction". (((y)X) and retrosynthesis is the "backward direction".

---

p(X]y). With additional prior modeling, the joint probability p(X, y) factorizes to either p(X (x)))())) or ply ()()). We propose training framework that leverages on the duality of the forward and backward directions and performs consistent training between the two to bridge the divergence.
The advantage of the duality of re- versible processes has been demon- strated other applications as well.
Algorithm 2 Dual Model
He et al. [2D] trained reinforce- ment learning (policy gradient) model to achieve duality in natural lan- guage processing and improved per- formances. Wei et al. [22] treated
[Train Phase]: Learning:
Input: Reactants X and product y.
Let 0 = {7, a7, 7)
Define Eg as Eq
code summary and code generation as pair of dual tasks, and improved effi- cacy by imposing symmetry between attention weights of LSTM encoder- decoder in forward and backward di- rections. Despite of their encourag- ing results, these models are not ideal for stable and efficient training for ret- rosynthesis, as policy gradient meth- ods suffer from high variance and LSTM has sub-optimal performance. Therefore propose novel training method that simple yet efficient for retrosynthesis task. We impose dual- ity constraints by training forward direction on mixture of samples drawn from the backward and original dataset. To best knowledge, we are the first to apply duality retrosynthesis and to impose duality constraints by samples drawn from one direction. The EBM is defined:
Eg = log (X)
log Pa
Pa (y(X) + p,,
1. Train backward:
1)" = arg min, Ldvat arg max, Ellog (Xly)
2. Train prior and forward: Plug in rj
pm
1430(X,
1430(y)p, (Xly)
2, a arg miny, Ldual
= -Ewger [x
Pyy (X) + log log
[Test Phase]: Inference:
Input: 0" ("",""""""""""""" a"" ,n"" ,. ges Proposal
L - P(y X = arg minxeL Eg- (X,y yeatt
Return X
p(X (1y) x exp [o + log (vlX) + (Xgp((((1))
(6)
=
y))
(7)
where prior p(X). ). forward likelihood X(y) and backward posterior P(X are modeled as autoregressive models (See parameterized by transformers with parameters 7, a, and 17. Note energy function can be designed free of choice.
The consistent training is achieved by minimizing the "dual loss", where the duality con- straints in the equation below are imposed to penalize KL divergence of the two directions, i.e., [I.(backward] forward). For simplicity, we fix the backward probability in the dual loss, and therefore entropy H (backward) is dropped.
x", o ,n' = arg min ldual
(8)
3,0.0)
Edual
D log p. (X) + log (o)
(9)
forward direction
+ BR. Ex (x)gp,()) + log (yl x)] + Blog Pa (Xly
(10)
duality constraints
backward direction
-E pii (X.u)
[log p., (X) + log (y] Blog P., (Xlv))
(11)
where E indicates expectation over empirical data distribution p X, y). The duality constraints B., Ex [log (X) + log Po is the expectation of the forward direction log logpy(X) Po Pa Xy) with respect to empirical backward data distribution É, Ex where ExEx are approximated by samples drawn from (yl)), as y is given so p(y) 1. f scale parameter. In our implementation we use size k-beam search to draw samples efficiently. Combining "forward" and "duality constraints" terms (Eq[IT, [T]. we can see that the first term of the dual loss to train the forward direction on the mixture distribution of the original data and samples drawn from backward directions miix(Xx 3) _ y) + 1438(y)), (X (Xly). Put every piece together (Algorithm

---

and Fig in Appendix). Here is our training procedure. Since we parameterize the three probabilities separately, the optimization of dual loss breaks into two steps:
Step 1: Train backward. 1 does not depend on forward direction under empirical data distribution.
'7" arg miny Ldual arg max,, Ellog P.y (X (x())) [b)] 17 can be learned by MLE.
Step 2: Train prior and forward. We plug 7/" into P"" (X.y). v). 7", xr = arg minly Ldual
arg max,,a Ep Bppry
+ log Pa (y X)] 7, a can be learned by MLE.
We provide ablation study of each component of dual loss in Appendix [.. The results show that each component in dual loss contribute to the final performance positively.
2.1.4 Perturbed model
In contrast the ordered model that factorizes the sequence in one direction, sequential model to achieve stochastic bidirectional factorization adapted from XLNet [23]. In particular, this model permutes the factorization order (while maintaining position encoding of the original order) that is used in the forward autoregressive model.
perturbed
[x(X)
p(X 2) = ((X(=)), xx(x)) x (b)
II 1=1
(12)
where the permutation order = is permutation of original order sequence 2, = [1, and 2, denotes the i-th element of permutation z. Here is treated as hidden variable.
|X]
2.1.5 Bidirectional model
alternative way to achieve bidirectional context conditioning the denoising auto-encoding model.
We adapt bidirectional model from BERT (24) to our application. The conditional probability p(X ((X] (v) factorized into product of conditional distributions of one random variable conditioning on others,
|a(X]I
p(X = exp(
Σ
log po (x (x
(13)
although the model is similar to MRF (26. the marginal of each dimension Eq (13) does not have a simple form as in BERT training objective. It may result in a mismatch between the model and the learning objective. This model can be trained by pscudo-likelihood (See (.2c
As presented in Wang and Cho 25
2.2 Graph-based Model
Compared with the sequence-based model, the graph-based methods present chemical molecules, with vertices as atoms and edges as chemical bonds. This natural parameterization allows straightforward application of chemistry knowledge by sub-graph matching with templates or reaction centers. We instantiated three representative gragh- based approaches, namely NeuralSym (27). GLN (28 and G2G [15), from the framework. Firstly, we introduce important concept template, which can assist modeling, learning, and inference.
Templates are reaction rules extracted from existing reactions. They are formed by reaction centers of atoms changed, e.g. form or break bonds). A template T' consists of a product-subgraph pattern (f,,) and reactants-subgraph pattern(s) (tx), denoted as T := by + txx where X is molecular set. We overload the notation define template operator T(-) M (M) which takes product as input, and returns set of candidate reactant sets. T(.) works as follows: enumerate all the templates with product-subgrape ty matching with the given product and define S(y) = (T: ty E y. VT E T). where T are available templates; then reconstruct the reactant candidates by instantiating reactant-subgraphs of the matched templates R (X tx X, YT e S(y)). The
output of T(.) is R. T(.) can be implemented by chemistry toolbox RDKit 29
2.2.1 Template prediction: NeuralSym
NeuralSym template-base method, which treats the template prediction as multi-class clas- sification. The corresponding probability model under the EBM framework can be written as:
5

---

p(Xyy) & <TET ( E T(y)
(14)
where f()) is neural network that embeds molecule graph y, and er is the embedding of template T. Learning such model requires only optimizing the cross entropy, despite that the number of potential templates could be very large.
2.2.2
Graph-matching with template: GLN
Dai et al. (28 proposed method of graph matching the reactants and products with their corre- sponding components in the template to model the reactants and template jointly, with the model:
p X, Tly) x exp(wn (T, + w22 (X,,X, T, v) (y) (T))
(15)
where wi and w2 are graph matching score functions, and the ((-) operators defines the hard template matching results. This model assigns
probability the reactions that do not match with the template. p(X(y) can be obtained by marginalizing over all templates.
2.2.3 Graph matching with reaction centers, G2G and GraphRETRO
In contrast with GLN, a few recent works G2Gs [15] and GraphRETRO [17] proposed to predict reaction center directly. These methods closely imitate chemistry experts when performing retrosyn- thesis: first identify reaction centers (i.e. where the bond breaks, denoted as c), then reconstruct X.
p(X x exp (log (Eeey p(X|c, v)p(clu))
(16)
All the methods mentioned above require the additional atom- nom-mapping as supervision during training. while NeuralSym and GLN require template information during inference. So NeuralSym and GLN are template- based methods. Since atom mapping plus reaction centers have almost same information templates, we denote G2G and GraphRETRO method as semi-template-based approach.
Learning
Training EBMs is to learn parameters 0. In particular, we introduce three ways to learn exact (if applicable) or approximate maximum likelihood estimation (MLE) for full energy-based model (..c as this model includes other eqquence-based EBM variants (ordered, perturbed, bidirec- tional, etc) by instantiating Eg accordingly. Training EBMs with MLE is non-trivial because the partition function (0) in Eq (I) is generally intractable. Computing Z(0) involves approximation or additional information.
3.1 Approximate MLE: integration using template.
We use additional chemistry information: Templates. Direct MLE is not feasible because the partition function of Eq 33) involves enumerating full molecular set M, which is intractable. Here we use templates to get finite support of the partition function. Specifically, we use template operator to extract a set of reactant candidates associated with denoted as T(y). ) As the size of T(y) is about tens hundreds (not computationally prohibitive), we perform exact inference of Eq 33) obtain the MIE, We denote this training scheme as template learning.
3.2 Approximate MLE: pseudo-likelihodd.
Alternatively, we can provide an approximation of Eq via pseudo-likelibeee likelihood [30] enable training. likelooo factorizes the joint distribution into the product of conditional probabilities of each variable given the rest. Theoretically, the pseudo-likelihood estimator yields an exact solution if the data is generated by model p( X [y)) and number of data points n > co (i.e., it is consistent) For the full model, training performed as:
30
[x(X)
exp (ge X, v))
p(X (y) RJ exp
log po (X (0 v)) exp(
Σ log
ECes exp (go(X', y; X 4 = X x c))
(17)
where the superscript indicates sequence except the i-th token and go 4) SiPl x Slvl H IR transformer architecture that maps two sequences scalar. As bidirectional model Sec 2.1.5

---

and training approaches See 3.2
(aproximaae joint probability) factorizes in the same way, pseudo-
likelihood a convenient way train this model.
3.3 Exact MLE: tractable factorization.
This training procedure works for special case of the full model, which has a tractable factorization of the joint probability, e.g., autoregressive models in ordered (Sec and perturbed (Sec 2.1.4).
3.4
Generalized sequence model.
Generalized sequence model first infer latent variable S" = arg max p(Sy) and then infer X arg max pXX ly. 55*) as the vanilla sequence-bad based model with S provided as additional input (e.g. concatenate S" and y).
Inference
With the trained Eg-, inference identifies the best X that minimizes the energy function for given your, iee X - arg x Eg- (X, y Directly solving the above minimization is again intractable, but the energy function can generally be used for ranking. Let I denote the rank of candidate X; for the given y (lower is better).
(R(X1) R(X2) = Eg- (X1,8 Eg- (X2, (0))
(18)
Practically, as illustrated in Fig 2 one can use either template-base or template-free method to come up with initial proposals for ranking, as follows.
femplate-based Proposing (TB). Templates can be used to extract list of proposed reactant
candidates by using templates. We use template operator T() (defined in See 2.2 to propose list of candidate reactant sets from the input product y. Template-free Proposing (TF). In this paper, remplate-free ranking makes proposals using the learned prediction model. We use simple autoregressive form for pi ((|yy (yy) (Ordered model), which can draw the top K most likely samples from this distribution using beam search, which is computational efficient.
Experiments
5.1 Experiment setup
We evaluate our method on a benchmark dataset named USPTO-50k, which includes 50k reactions falling into reaction types from the US patent literature. The datasets are split into train/valide lainn/alidation/test with percentage of 80% 10% 100% 11%% Our evaluation metric the top-k exact match accuracy, referring to the percentage of examples where the ground truth reactant set was found within the top A predictions made by the model. Following the common practice, we use RDKit 29 canonicalize the SMILES string. For sequence-basee models, we incorporate the augmentation trick to ensure best performance. The procedures are follows: (1) Replace each molecule in reactant
Dataset and evaluation used follow existing work [31 28 13 15
product using random SMILES; (2) Random permute the order of reactant molecules. The augmentated SMILES are different linearizations of the same molecules. can prevent sequence- based models (transformer) from over-fitting. However, the augmentation does not improve performance for graph-based models, as graph- models take graph format as input which is invariant for different augmentations.
5.2 Existing methods
We evaluate of our approach against several existing methods, including both template-based, semi- template based and template-free approaches. Template-free methods: Transformes [14 is a transformer based approach that trains a second transformer to identify the wrong translations and remove them. LSTM [13] is a sequence to sequence approach that use LSTM as encoder and decoder. Template-basee methods: retrosim [31] selects template for target molecules using fingerprint based similarity measure between targets and templates; neuralsym [27] performs selection of templates as multiple-class problem using MLP; GLN builds template induced graphical model and makes prediction with approximated MAP.
Semi- TemilTempllte based methods: G2Gs [1], GRetroXpert [66, and GraphRETRO [7 share the same idea: infer reaction center to generate synthons, and then complete the missing pieces (aka "leaving groups")) in synthons to generate reactants. These methods use "reaction centers" as additional information to supervise their algorithm. The reaction centers preserve key information in templates. So denote them as "Semi-Toee Semi-Template".
7

---

5.3 Template-free evaluation
Table I shows our best EBM variant (the dual model) evaluated in a template free setup. We first perform evaluations on all the EBM variants introduced in Sec to select the best EBM variant. The results show that the dual model outperforms other EBM variants by a clear margin (Table 3 in Appendix). The evaluation is on template-basee proposing to ensure the proposal list of candidate molecules the same for all the variants.
Then we pursue further on template-free setup. An ideal model requires a proposal model with good coverage and ranking model with good accuracy. We explored various combinations of proposal-ranking pairs. The proposal model evaluated is the ordered model trained on USPTOS0K and augmented USPT050K, respectively. The ranking model is the dual model trained on augmented data, as it performs the best in Table 4 Our best performer is ordered- ordered-proposal (USPTO 50K)- dual-ranking (aug USPTO 50K) model. case study showing how dual model improves accuracy upon proposal is given in Fig where it shows how the energy based re-ranking refines the initial proposal. One interesting observation that, the proposal ordered model trained on augmented data has higher top 1 accuracy but much lower top 10 accuracy, than the one trained without augmentation. This indicates that the proposal using augmented data has low coverage the prediction space. We observed that the model learned on augmented dataset learns various representations of the same molecule (due to usage of random SMILES). A certain percentage of proposed candidates are the same after canonicalization, which is good for top 1 prediction during ranking but undesired for proposal.
Table 1: Template-free: Dual model: Translation Proposal and Dual Ranking
Proposal
Re-min
Type
Proposal model
Top 1
Top
Top 10
Top.50
Top 100
Rank model
Top
Top 3
Top
Top
Ordered on UPSPTO
11.1
61.9
69.9
77.2
78.0
Dual trained
53.6
70.7
71.0
77.0
Ordered on Aug USPTO
53.2
54.7
55.6
60.5
Aug USPTO
60.0
60.4
60.5
SOTA (RetroXpest 16
50.4
61.1
62.3
63.4
Ordered USPTO
76.1
79.7
Dual trained
81.9
Yes
Ordered on Aug USPTO
64.7
66.5
67.3
69.7
75.7
Aug USPTO
66.2
75.1
75.6
75.7
SOTA (RetrXXeett
16
62.1
75.8
78.5
80.9
5.4
Ablation Study of the dual loss
Since the dual variant serves as the backbone variant in the previous section, we perform additional ablation study investigate the performance of the dual variant with respect different designs of the dual loss. Table 2 shows that each component of the dual loss contribute positively to the final performance. The dual constraint leads additional improvement on the top of other components, which is more challenging to achieve in a higher accuracy region.
The evaluation of Table 2 is under the same setup as Table 4 uses template- based proposal for fair and easy comparison. The notations of Table 2 are as follows: The "dual" row are entries taken from Table showing results trained with dual loss. To recap, the dual loss is defined in Eq (10) and the dual constraint is its middle term. É Eloop,
p, (X) + log Pa (a (x) + log (Xly)) is the dual
(X [v)] is the dual loss without the prior log (X). log (X()) is only including backward direction.
loss without the dual constraint. Ellog (y)X) ) + log p., (X
Table Ablation Study of dual loss design when reaction type is known
Aug USPTO
Top 1
Top 3
Top 5
Top 10
Dual
67.7
84.8
88.9
92.0
E Log P., (X) + log Pa (y X) + log p,, (X [y)]
67.0
84.7
88.9
91.95
Elog Pa, (y) X) x)=+ + +log P,, (X (Xly)
66.1
82.8
87.6
91.3
T log Pry (1y)]
60.9
80.9
85.8
90.2
8

---

br
Translation Proposal
ground cruth
Dual Model Re-rank
NH
ranking
Figure 3: Dual ranking improves upon translation proposal. Left and right column are the top three candidates from translation proposal and dual re-ranking of the proposal. Ground truth (GT) given at the top and labeled orange in the middle. By dual re-ranking, the GT ranks the first place, whereas the 3rd place in the proposal. Note that the first place in the proposal is only one atom different from GT (Br vs 1), indicating the dual model is able identify small changes in structure. Another example is given in Fig 5ig5inn in Appendix.
Table 3: Top K exact match accuracy of existing methods
Reaction type unknown
Reaction type known
Category
Model
topl
top3
top5
top10
top1
top3
top5
top10
retrosim 31
37.3
54.7
63.3
74.1
52.9
73.8
81.2
88.1
NeuralSym
27
44.4
65.3
72.4
78.9
55.3
76.0
81.4
85.1
TB
GLN
52.5
69.0
75.6
83.7
64.2
79.1
85.2
90.0
G2Gs
15
48.9
67.6
72.5
75.5
61.0
81.3
86.0
88.7
Semi-TB
GraphRETRO
53.7
68.3
72.2
75.5
63.9
81.5
85.2
88.1
RetroXpert [16
50.4
61.1
62.3
63.4
62.1
75.8
78.5
80.9
LSTM [13
37.4
52.4
57.0
61.7
TF
Transformer
43.7
60.0
65.2
68.7
59.0
74.8
78.
81.1
Dual (Ours)
53.6
70.7
74.6
77.0
65.7
81.9
84.7
85.9
5.5
Comparison against the state-of-the-art
Table 3 presents presents the main results. All the baseline results are extracted from existing works as we share the same experiment protocol. The dual model is trained with randomized SMILES to inject order invariance information of molecule graph traversal. Note that other methods like graph-based variants do not require such randomization We can see that, regarding top accuracy, our proposed dual model outperforms the current state- of-the-art methods. Semi-template methods are those require ground truth reaction centers during training supervision, whereas generalized dual model does not require this additional information, yet still output perform the best semi-tent semi-template models by 1.6% when reaction type is known. This demonstrates the advantages of the dual model. RetroXpert results are the updated results taken from https://gi //gptt/ thub com/uta- uuuututaaaae wutataallleeeeea swwle/Retroolt /Rterooperrtt
the graph representation is already order invariant.
Conclusion
In this paper we proposed an unified EBM framework that integrates multiple sequence- and graph- based variants for retrosynthesis. Assisted by comprehensive assessment, we provide critical understanding of different designs. Based this, proposed novel variant generalized dual model, which outperforms state-of-the-ar in template free manner.


Reviews:
[{"review_id": "BqgxR-fC0XX", "reviewer": "Reviewer_r7dk", "summary": "This paper presents a unified framework with energy-based models for retrosynthesis prediction. Different methods (sequence and graph methods) can be untied with different energy functions within the framework. A dual variant model is presented and it performs consistent forward and backward training. Experimental results demonstrate this consistent training improves the retrosynthesis prediction significantly.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors present a general EBM framework to unify existing sequence-based and graph-based methods. Through investigating different designs of the EMB framework, a dual variant model is proved to achieve the best performance on USPTO-50K dataset. The proposed dual model also trains a forward reaction prediction model, combined with the retrosynthesis prediction, the top-k predictions can be ranked according to the overall prediction scores. While previous methods rank predictions without incorporating forward-prediction scores.\n\nOne major concern for me is that the authors present their methods and variants with abstract formulations, while implementation details are largely ignored. This makes their method difficult to understand. For example, line #101, p(Xi|X1:i-1, y) is parameterized by the Transformer model h(p, q), how does h(p, q) output a logits vector for each value in vocabulary? Does the formula p(Xi|X1:i-1, y) indicate the input is X1:i-1 and y, and output is Xi? Line 142, could the authors add more details to explain how KL divergence is related to the duality constraints?\n\nAnother concern for me is the preprocessing of the USPTO-50k dataset. It is found that there is a potential information leak with the dataset reported by the authors of RetroXpert. Have the authors removed the atom mapping and then canonicalized the product SMILES during the inference? Have the authors also run rxnmapper on testing data? It is not necessary to have mapping information for testing data, the authors should remove any mapping information for testing data and then run the inference. Are the authors aware of any potential information leak with the proposed method?\n\nThe authors claim that \"we propose a unified energy-based model (EBM) framework that integrates sequence- and graph-based models for retrosynthesis.\" and I do not think this is a significant contribution. EBM is a general framework and it is not surprising that EBM can be used to explain different models, especially the graph-based method GLN itself is energy-based. \n\nThe Transformer has been extensively explored for retrosynthesis, the authors did not clarify the difference between their different variants and existing Transformer-based methods. The proposed generalized sequence models at line #216 are similar to the RetroXpert, except that the synthons are directly predicted here. The ordered model at line #98 is similar to the one-stage Transformer methods such as SCROP.\n\nLine #223, previous methods infer reaction center from provided atom-mapping information, which is extracted from third-party software like Indigo. The authors use rxnmapper to find the atom-mapping information. I do not think there is an essential difference.\n\nWhat is the difference between formula (4) and (13)？\n\nWhen incorporating templates, what is the difference between the proposed method and GLN except for the sequence and graph-based?\n\nCould the authors also report results on USPTO-full as GLN to demonstrate the scalability? \n\nLine #250, the explanation of the superscript should be moved to line #173 when it first appears.\n\nLine#121, the RHS of the equation in Algorithm 2 should have a minus symbol.\n\n \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Not", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "AI_mdzUmXLs", "reviewer": "Reviewer_2KjZ", "summary": "The paper summarizes many different models for for retrosynthesis under the EBM framework.\nIt introduces dual training.\nBy performing reaction center annotation, state of the art results are claimed.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors unify many recently proposed models for chemical retrosynthesis under the framework of energy based models (EBM)\nFurthermore they propose dual training for re-ranking the predictions.\nThe authors report state of the art performance.\n\nSeveral works (e.g. Segler 2018, Coley 2019) have combined retro-prediction with forward scoring for re-ranking, so the novelty of this aspect is somewhat limited.\n\nThe presentation of the paper is somewhat dense, which mostly due to the length limit.\n\nOverall i think the paper could eventually fit into NEURIPS and is interesting, but there are several open questions which need to be addressed before the publication can be recommended. (see update after rebuttal)\n\n\nPlease note that retroXpert https://github.com/uta-smile/RetroXpert and graphretro https://arxiv.org/abs/2006.07038 results have been updated (they are lower than originally reported). so these results should be updated.\n\nQuestions:\n\nThe description of the reaction center assignment using Schwaller's method is very brief, and in the current form unclear how it is implemented. Reaction enters can only be assigned to full reactions (you need to have the reactants AND product for that). It is unclear how this can actually work during inference. Given the later corrected results of the 2 papers results I am a bit skeptical how this is actually done, and I would like to see a very clear and long description on what is actually done here.\n\nI also had a look at the code, but could not find the data, or how this additional step is performed in the code.\n\nI am looking forward to discuss this in the rebuttal phase.\n\n______\nEdit after discussion:\n\nThis reviewer has raised their score to reflect the improvements the authors will make to the manuscript after discussion with the reviewers.\nThis paper should be published in NeurIPS\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No concerns wrt Societal Impact: basic research.", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns, basic research.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "2RANBdgwI60", "reviewer": "Reviewer_EqsE", "summary": "This submission tackles the problem of one-step retrosynthesis, a classic problem in chemistry, through the lens of energy-based models. The framing of the problem in this manner is elegant. The quantitative results may need to be clarified or resolved to ensure that the comparison to prior SOTA is fair (i.e., not using an additional model or leaking data of the ground truth reactants).", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThe framework presented here is an elegant way to think about the complementarity between retrosynthesis and forward synthesis. It is described well. \n\n1.\tThis is a revised submission from ICLR 2021 where the authors were unfortunately punished for lacking comparisons to SOTA RetroXpert and GraphRetro, both of which ended up having data leakages that inflated performance by a large margin. Please refer to https://github.com/uta-smile/RetroXpert for an explanation of the leak. Were the sequence-based models provided with atom-mapped SMILES or unmapped SMILES? Were the product and reactant SMILES strings recanonicalized after stripping atom mapping using RDKit? The current SOTA top-1 accuracy when reaction type is unknown is closer to ~53% using the revised RetroXpert and GraphRetro results. The high results reported in Table 2 may be explained by the fact that the “Generalized” model implicitly uses information about the true reactants (see comment below). \n\n2.\tThe notation in Equation 6 is confusing to me; is $p(X|y) = \\exp(- E_\\theta(X,y))$ what the authors intended to convey?\n\n3.\tThe categorization of methods into template-based and semi-template based has an unclear role to me. Is there a need to draw these strong distinctions? Atom mapping can be approximated through maximum common substructure heuristics, so it is “free” in a sense. None of the datasets used for benchmarking these tools relied on manual labeling (cf. line 226). This limitation of methods requiring atom mapping is overstated.\n\n4.\tThe setting displayed in Table 1 where the inferred reaction center is provided is an unusual choice as it does not reflect a real use case for these models in chemistry. Providing the inferred reaction center (which is inferred given knowledge of the true reactants) to the “Generalized” model makes the comparison to previously reported results invalid, but would explain the 69.9% reported in Table 2 in the absence of the data leak.\n\nIf the quantitative results are clarified or resolved to ensure that the comparison to prior SOTA is fair (i.e., not using an additional model or leaking data of the ground truth reactants) I believe this paper is deserving of a 7 (accept).\n\n-----\nI have raised my score to 7, as the authors have addressed the (potential) information leak and atypical evaluation with the generalized model", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are not discussed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "-YOMGk1RFTA", "reviewer": "Reviewer_vxrH", "summary": "This paper attempts to describe sequence-based and graph-based one-step retrosynthesis models, which are the current mainstreams of retrosynthesis, in a unified manner with the Energy-Based Model (EBM).\nThis paper reformulates several existing models with the EBM, mainly focusing on the difference in the energy function definitions. \nIn addition, a new EBM variant is proposed in this paper, and its performance is updated the State-of-the-Art.　\n\nAside from the mainstreams of the machine learning community, AIzynthfinder recently draws many attention as a free and powerful multi-step retrosynthesis engine for computational chemistry. Some comments on this engine may be beneficial for placing the submitted work in a broader context. \n\nThakkar A, Kogej T, Reymond J-L, et al (2019) Datasets and their influence on the development of computer assisted synthesis planning tools in the pharmaceutical domain. Chem Sci. https://doi.org/10.1039/C9SC04944D\nGenheden S, Thakkar A, Chadimova V, et al (2020) AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning. J. Cheminf. https://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00472-1\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Originality: \nAs far as I know, this paper is the first attempt to describe various Retrosynthesis models in a unified manner in EBM, and it is novel in this respect.\nThe bi-directional training of sequence models (Dual in this paper) has been shown to be useful in several previous studies, as cited in the text, and is not novel in itself.\nHowever, the use of sampling to stably apply the method to retrosynthesis is a good idea. \n\n## Quality: \nI find no evident errors in terms of technical content. \n\n## Clarity: \nIt is good that the paper carefully introduces the EBM.\nOn the other hand, since many models have to be covered in one paper, the explanation of each model is limited to a minimum.   I think that the current manuscript requires the reader to have some expertises in retrosynthesis.\nIn Sec. 2, various existing models are actually formulated by EBM. \nIn the current manuscript, readers have to go back and forth between pages while remembering the formulas to confirm the differences between the model equations.\nIf it is possible, it would be easier for readers if the qualitative characteristics of each formula are summarized in one table in the appendix.\n\n## Significance:\nNumerical experiments show that a new State-of-the-Art for the one-step retrosynthesis has been achieved, which strongly supports the usefulness of the generalized Dual model proposed in the paper.\nIn addition, the fact that an unsupervised atom mapping transformer can achieve great performance is good news for readers who have difficulty in handling graph data (in fact, it is not an easy task for many ones).\nThe Appendix contains many additional experiments and information to evaluate the model from various points of view, which is useful to help the reader understand it.\n\n\n(+) EBM interpretation of several retrosynthesis models are new\n(+) Specific formulation for dual (bi-directional) training of generalized sequence models.\n(+) new SotA\n(+) appendix informative\n(-) cumbersome notations and presentations (partially inevitable because this paper covers many models: understandable)\n\n## after feedbacks\nFeedbacks from authors and additional experiment report solves many concerns of reviewers. \nI believe this paper is worth publication in NeurIPS.  \nI keep my positive score. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No specific opinions", "review_text": "", "extra_scores": {"ethical_concerns": "No specific concerns", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper tries to include all the existing retrosynthesis algorithms into the framework of Energy-Based Model (EBM). Based the this, it derives a dual model to measure the energy of a pair (Reactants, Product), which depends on both the backward, forward energy function and their consistency. Experiments demonstrate this will improve the top-1 accuracy by 1.3~1.5%, compared with only using the backward model. 
Decision: Accept (Poster)