E NERGY - BASED V IEW OF R ETROSYNTHESIS
Anonymous authors
Paper under double-blind review
A BSTRACT
Retrosynthesis is the process of identifying a set of reactants to synthesize a target molecule.
It is of vital importance to material design and drug discovery.
Existing machine learning approaches based on language models and graph neural networks have achieved encouraging results.
However, the inner connections of these models are rarely discussed, and rigorous evaluations of these models are largely in need.
In this paper, we propose a framework that uniﬁes sequenceand graph-based methods as energy-based models (EBMs) with different energy functions.
This uniﬁed view establishes connections and reveals the differences between models, thereby enhancing our understanding of model design.
We also provide a comprehensive assessment of performance to the community.
Moreover, we present a novel dual variant within the framework that performs consistent training to induce the agreement between forward- and backward-prediction.
This model improves the state-of-the-art of template-free methods with or without reaction types.
I NTRODUCTION
[Figure]
Retrosynthesis is a critical problem in organic chemistry and drug discovery (Corey, 1988; 1991; Segler et al., 2018b; Szymku ´ c et al., 2016; Strieth-Kalthoff et al., 2020).
As the reverse process of chemical synthesis (Coley et al., 2017a; 2019), retrosynthesis aims to ﬁnd the set of reactants that can synthesize the provided target via chemical reactions (Fig 1).
Since the search space of theoretically feasible reactant candidates is enormous, models should be designed carefully to have the expression power to learn complex chemical rules and maintain computational efﬁciency.
Recent machine learning applications on retrosynthesis, including sequence- and graph-based models, have made signiﬁcant progress (Segler & Waller, 2017a; Segler et al., 2018b; Johansson et al., 2020).
Sequence-based models treat molecules as one-dimensional token sequences (SMILES (Weininger, 1988), bottom of Fig 1) and formulate retrosynthesis as a sequence-tosequence problem, where recent advances in neural machine translation (Vaswani et al., 2017; Schwaller et al., 2019) can be applied.
In this principle, the LSTM-based encoder-decoder frameworks and, more recently, transformer-based approaches have achieved promising results (Liu et al., 2017; Schwaller et al., 2019; Zheng et al., 2019).
On the other hand, graph-based models have a natural representation of human-interpretable molecular graphs, where chemical rules are easily applied.
Graph-based approaches that perform graph matching with chemical rules (“templates”; deﬁnition in Sec 3.2) or reaction centers have reached encouraging results (Dai et al., 2019; Shi et al., 2020).
In this paper, we focus on one-step retrosynthesis, which is also the foundation of multi-step retrosynthesis (Segler et al., 2018b).
Our goal here is to provide a uniﬁed view of both sequence- and graph-based retrosynthesis models using an energy-based model (EBM) framework.
It is beneﬁcial because: First, the model design with EBM is very ﬂexible.
Within this framework, both types of models can be formulated as different EBM variants by instantiating the energy score functions into speciﬁc forms.
Second, EBM provides principled ways for training models, including maximum likelihood estimator, pseudolikelihood, etc.
Third, a uniﬁed view is critical to provide insights into different EBM variants, as it is easy to extract commonalities and differences between EBM variants, understand strengths and limitations in model design, compare the complexity of learning or inference, and inspire novel
EBM variants.
To summarize our contributions:
• We propose a uniﬁed energy-based model (EBM) framework that integrates sequence- and graph-based models for retrosynthesis.
To our best knowledge, this is the ﬁrst effort to unify and exploit inner connectivity between different models.
• We perform rigorous evaluations by running tens of experiments on different model designs.
We believe revealing the performance to the community contributes to the development of retrosynthesis models.
• Inspired by such a uniﬁed framework, we propose a novel dual EBM variant that performs consistent training over forward and backward prediction directions.
dual model improves the state-of-the-art accuracy by 9 .
9% for full automate template-free and 2 .
7% for template-based.
The goal of this paper is to investigate the performance of different models under the setup without any hand-crafted chemistry features, e.g. reaction center, during training.
Incorporating these handcrafted chemistry features usually can boost accuracy signiﬁcantly regardless of the model design.
So adding features is not the focus of our paper.
See discussion in Appendix A.2, A.3, A.4.
E NERGY - BASED MODEL FOR R ETROSYNTHESIS
Retrosynthesis is to predict a set of reactant molecules from a product molecule.
We denote the product as y , and the set of reactants predicted for one-step retrosynthesis as X .
The key for retrosynthesis is to model the conditional probability p ( X | y )
(Dai et al., 2019; Shi et al., 2020; Liu et al., 2017; Schwaller et al., 2019).
EBM provides a common theoretical framework that can unify many retrosynthesis models, including but not limited to existing models.
[Train Phase]: Learning Input : Reactants X and products y . 1. Parameterize X and y in Sequence or Graph format.
Design E θ { e.g. dual, perturbed, bidirectional, graph-based, etc } /
An EBM (LeCun et al., 2006; Hinton, 2012) deﬁnes the distribution using an energy function.
Without loss of generality, we deﬁne the joint distribution of product and reactants as follows:
[Equation]
where the partition function Z ( θ )
=
[Equation]
y (cid:80) X exp( − E θ ( X, y )) is a normalization constant to ensure a valid probability distribution.
Since the design of E θ is free of choice, EBMs can be used to unify many retrosynthesis models by instantiating the energy function E ( θ ) with different designs and approximation of the partition function.
Note there is a trade-off between model expression capacity and learning tractability.
EBM is also easy to obtain arbitrary conditioning with different partition functions.
For example, the forward prediction probability for reaction outcome prediction p θ ( y | X ) can be written as exp( − E θ ( X,y ))
(cid:80) y (cid:48) exp( − E θ ( X,y (cid:48) )) with the same form of energy function.
Overall, the proposed framework works as follows: (1) design and train an energy function
E θ (Sec 3 and Sec 4), and (2) use E θ for inference in retrosynthesis (Sec 5).
See Fig 2 and Algorithm 1.
Based on how to parameterize reactant and product molecule X and y , the model designs can be divided into two categories: sequence-based and graph-based models.
M ODEL D ESIGN 3. S EQUENCE - BASED M ODELS
Here we describe several sequence-based parametriztion to instantiate our EBM framework, which use SMILES string as representations of molecules.
We ﬁrst deﬁne the sequence-based notations.
Given a reactant molecule x , we denote its SMILES representation as s ( x ) .
Superscript s ( x ) ( i ) denotes the character at i -th position of the SMILES string.
For simplicity, we use x ( i ) when possible.
Reactants of a chemical reaction are usually a collection of molecules: X = { x 1 , x 2 , .., x j , .., x | X | } , where x j is the j -th reactant molecule.
The SMILES representation of a molecule set X , denoted as s ( X ) , is a concatenation of s ( x ) for every x in X with “.”
in between: “ s ( x 1 ) .s
( x 2 ) ...
s ( x | X | ) ”.
For simplicity, we use X ( i ) as the short form of s ( X ) ( i ) to denote the i -th position of this concatenated
SMILES.
3.1. F ULL ENERGY - BASED MODEL
We start by proposing a most ﬂexible EBM that imposes the minimum restrictions on design of
E θ .
All the variants proposed in Sec 3.1 are special instantiations of this model (e.g. by specifying different E θ ).
The EBM is deﬁned as follows:
[Equation]
Here the energy function E θ : P ( M ) × M (cid:55)→ R takes a molecule set and a molecule as input, and outputs a scalar value.
M deﬁnes the set of all possible molecules.
P ( · ) represents the power set.
P ( M ) denotes domain of reactant sets X .
Due to the intractability of the partition function, training involves additional information e.g., template or approximation of the partition (See Sec 4).
3.1. O RDERED MODEL
One design of energy function is factoring the input sequence in an autoregressive manner (Sutskever et al., 2014; Schwaller et al., 2019; Segler et al., 2018a).
[Equation]
where p θ ( X ( i ) | X (1: i − 1) , y ) is parameterized by a transformer h θ ( p, q ) :
S | p | × S | q | (cid:55)→ R | S | where | S | is vocabulary size.
e ( c ) is a one-hot vector with dimension c set to 1.
This choice of h θ ( p, q ) enables efﬁcient computing of the partition function, as it outputs a vector with length equal to | S | to represent logits (unnormalized log probability) for each value in vocabulary.
maximum likelihood estimator (MLE) is feasible for training, as this factorization allows tractable partition function.
3.1. D UAL MODEL
Algorithm [Train Phase]: Learning :
Input: Reactants X and product
y.
Let θ = { γ, α, η }
Deﬁne E θ as Eq (4) [Equation]
Train backward :
[Equation]
forward
p
[Equation]
X, y )
+ [Equation]
+
[Equation]
| y ) γ
[Equation]
[Equation] arg min γ, [Equation] L dual = (cid:98) E p mix ( X,
[Equation]
[Equation]
p [Equation] ( X ) + log p
[Equation]
( y | X )]
[Test Phase]: Inference :
Input : θ ∗ = { γ ∗ , α ∗ , η ∗ } , y test , Proposal P.
L ← P ( y test ) X ∗ = arg min X ∈ L E θ ∗ ( X, y test )
Return X ∗
A different design is to leverage on duality of retrosynthesis and reaction prediction.
They are a pair of mutual reversible processes that factorize the joint distribution in different orders, where reaction prediction is “forward direction” – p ( y | X ) ) and retrosynthesis is the “backward direction” – p ( X | y ) .
With additional prior modeling, the joint probability p ( X, y ) factorizes to either p ( X | y ) p ( y ) or p ( y
| X ) p ( X ) .
We propose a training framework that leverages on the duality of the forward and backward directions and performs consistent training between the two to bridge the divergence.
The advantage of incorporating forward direction p ( y | X ) has been showed (Guo et al., 2020), where the authors use a sequential Monte Carlo tree algorithm to search for reactants that agree with forward prediction score.
The advantage of the duality of reversible processes has been demonstrated in other applications as well.
He et al.
(2016) trained a reinforcement learning (policy gradient) model to achieve duality in natural language processing and improved performances.
Wei et al.
(2019) treated code summary and code generation as a pair of dual tasks, and improved efﬁcacy by imposing symmetry between attention weights of LSTM encoder-decoder in forward and backward directions.
Despite of their encouraging results, these models are not ideal for stable and efﬁcient training for retrosynthesis, for example, policy gradient methods suffer from high variance.
Therefore we propose a novel training method that is simple yet efﬁcient for retrosynthesis task.
We impose duality constraints by training forward direction on a mixture of samples drawn from the backward and original dataset.
To our best knowledge, we are the ﬁrst to apply duality to retrosynthesis and to impose duality constraints by samples drawn from one direction.
The EBM is deﬁned as:
[Equation]
where prior p ( X ) , forward likelihood p ( y | X ) , and backward posterior P ( X | y ) are modeled as autoregressive models (Sec 3.1.2), parameterized by transformers with parameters γ , α , and η .
Note energy function can be designed free of choice.
Our consistent training is achieved by minimizing the “dual loss”, where the duality constraints in the equation below are imposed to penalize KL divergence of the two directions, i.e.,
KL(backward | forward) .
For simplicity, we ﬁx the backward probability in the dual loss, and therefore entropy H (backward) is dropped.
[Equation] [Equation]
where (cid:98) E indicates expectation over empirical data distribution ˆ p ( X, y ) .
The duality constraints β (cid:98)
E y (cid:98)
E X |
y [log p
γ
( X ) + log p
α
( y | X )] is the expectation of the forward direction log p γ ( X ) + log p
α
( y | X ) with respect to empirical backward data distribution
ˆ E y
ˆ E X
| y , where ˆ E y
ˆ E X
| y are approximated by samples drawn from p η
( X | y ) , as y is given so p ( y )
= 1 and β is scaled parameter.
In our implementation we use size k -beam search to draw samples.
Combining “forward” and “duality constraints” terms (Eq 7), we can see that the ﬁrst term of the dual loss is to train the forward direction on the mixture distribution of the original data and samples drawn from backward directions p mix ( X, y )
= 1 1+ β ˆ p ( X, y )
+ β 1+ β ˆ p ( y )
p η
( X | y ) .
Put every piece together (Algorithm 2 and Fig 3 in
Appendix).
Here is our training procedure.
Since we parameterize the three probabilities separately, the optimization of dual loss breaks into two steps:
• Step 1: Train backward .
η does not depend on forward direction under empirical data distribution.
η ∗ = arg min
η L dual = arg max η (cid:98) E [log p
η
( X | y )] .
η can be learned by MLE.
• Step 2: Train prior and forward .
We plug η ∗ into p mix η ∗ ( X, y ) .
γ ∗ , α ∗ = arg min γ,α L dual = arg max γ,α (cid:98)
E p mix η ∗ ( X,y ) [log p γ ( X ) + log p
α
( y | X )] .
γ, α can be learned by MLE. Please also
see ablation study of
each
components of dual loss in Appendix Sec A.6.
3.1. P ERTURBED MODEL
In contrast to the ordered model that factorizes the sequence in one direction, we use a perturbed sequential model to achieve stochastic bidirectional factorization adapted from XLNet (Yang et al., 2019).
In particular, this model permutes the factorization order (while maintaining position encoding of the original order) that is used in the forward autoregressive model.
[Equation]
where the permutation order z is a permutation of the original order sequence z o = [1 , 2 , . .
, | X | ] and z i denotes the i -th element of permutation z .
Here z is treated as hidden variable. [Figure]
During training, permutation order z is randomly sampled and uses the following training objective:
[Equation] [Equation]
and the corresponding parameterization:
[Equation] [Equation]
where z i encodes which position index in the permutation order to predict next, implemented by a second position attention (in addition to the primary context attention).
Note that Eq (9) is actually a lower bound of the latent variable model, due to Jensen’s inequality.
However, we focus on this model design for simplicity of permuting order in training.
The lower-bound approximation is tractable for training.
3.1. B IDIRECTIONAL MODEL
An alternative way to achieve bidirectional context conditioning is the denoising auto-encoding model.
We adapt bidirectional model from BERT (Devlin et al., 2018) to our application.
The conditional probability p ( X | y ) is factorized into product of conditional distributions of one single random variable conditioning on the others,
[Equation]
As presented in Wang & Cho (2019), although the model is similar to MRF (Kindermann, 1980), the marginal of each dimension in Eq (11) does not have a simple form as in BERT training objective.
It may result in a mismatch between the model and the learning objective.
This model can be trained by pseudo-likelihood (Sec 4.2)
3. G RAPH - BASED M ODEL
Compared with the sequence-based model, the graph-based methods present chemical molecules, with vertices as atoms and edges as chemical bonds.
This natural parameterization allows straightforward application of chemistry knowledge by sub-graph matching with templates or reaction centers.
We instantiated three representative gragh-based approaches, namely NeuralSym (Segler &
Waller, 2017b), GLN (Dai et al., 2019) and G2G (Shi et al., 2020), from the framework.
Firstly, we introduce an important concept template , which can assist modeling, learning, and inference.
Templates are reaction rules extracted from existing reactions.
They are formed by reaction centers (a set of atoms changed, e.g. to form or break bonds).
A template T consists of a product-subgraph pattern ( t y ) and reactants-subgraph pattern(s) ( t X ), denoted as T :=
t y
→ t X , where X is a molecular set.
We overload the notation to deﬁne a template operator T ( · ) :
M (cid:55)→ P ( M ) which takes a product as input, and returns a set of candidate reactant sets.
T ( · ) works as follows:
enumerate all the templates with product-subgraph t y matching with the given product y and deﬁne S ( y ) =
{ T : t y ∈ y, ∀ T ∈ T } , where T are available templates; then reconstruct the reactant candidates by instantiating reactant-subgraphs of the matched templates R =
{ X : t X ∈ X, ∀ T ∈ S ( y ) } .
The output of T ( · ) is R .
T ( · ) can be implemented by chemistry toolbox RDKit (Landrum, 2016).
3.2. T EMPLATE PREDICTION : N EURAL S YM
NeuralSym is a template-based method, which treats the template prediction as multi-class classiﬁcation.
The corresponding probability model under the EBM framework can be written as:
[Equation]
[Equation] where f ( · ) is a neural network that embeds molecule graph y , and e T is the embedding of template
T .
Learning such model requires only optimizing the cross entropy, despite that the number of potential templates could be very large.
3.2. G RAPH - MATCHING WITH TEMPLATE : GLN
Dai et al.
(2019) proposed a method of graph matching the reactants and products with their corresponding components in the template to model the reactants and template jointly, with the model:
[Equation]
where w 1 and w 2 are graph matching score functions, and the φ ( · ) operators deﬁnes the hard template matching results.
This model assigns zero probability to the reactions that do not match with the template.
p ( X | y ) can be obtained by marginalizing over all templates.
3.2. G RAPH MATCHING WITH REACTION CENTERS , G2G
In contrast with GLN, Shi et al.
(2020) proposed a method to predict reaction center directly.
This method closely imitates chemistry experts when performing retrosynthesis: ﬁrst identify reaction centers (i.e. where the bond breaks, denoted as c ), then reconstruct X .
[Equation]
All three methods mentioned above require the additional atom-mapping as supervision during training, while NeuralSym and GLN require template information during inference.
So NeuralSym and
GLN are template-based methods.
Since atom mapping plus reaction centers have almost same information as templates, we denote G2G method as semi-template-based approach.
L EARNING
Training EBMs is to learn parameters θ .
In particular, we introduce three ways to learn exact (if applicable) or approximate maximum likelihood estimation (MLE) for full energy-based model (Sec 3.1.1), as this model includes other sequence-based EBM variants (ordered, perturbed, bidirectional, etc) by instantiating E θ accordingly.
Training EBMs with MLE is non-trivial because the partition function Z ( θ ) in Eq (1) is generally intractable.
Computing Z ( θ ) involves approximation or additional information.
4. A PPROXIMATE MLE: INTEGRATION USING TEMPLATE .
We use additional chemistry information: Templates.
Direct MLE is not feasible because the partition function of Eq (2) involves enumerating full molecular set M , which is intractable.
Here we use templates to get a ﬁnite support of the partition function.
Speciﬁcally, we use template operator to extract a set of reactant candidates associated with y , denoted as T ( y )
As the size of T ( y ) is about tens to hundreds (not computationally prohibitive), we can perform exact inference of Eq (2) to obtain the MLE.
We denote this training scheme as template learning.
4. A PPROXIMATE MLE: PSEUDO - LIKELIHOOD .
Alternatively, we can provide an approximation of Eq (2) via pseudo-likelihood (Besag, 1975) to enable training.
Pseudo-likelihood factorizes the joint distribution into the product of conditional probabilities of each variable given the rest.
Theoretically, the pseudo-likelihood estimator yields an exact solution if the data is generated by a model p ( X | y ) and number of data points n → ∞ (i.e., it is consistent) (Besag, 1975).
For the full model, training is performed as: p ( X | y )
≈ exp( | s ( X ) | (cid:88) i =
[Equation]
where the superscript ¬ indicates sequence except the i -th token and g θ ( p, q ) : S | p | × S | q | (cid:55)→ R is a transformer architecture that maps two sequences to a scalar.
As bidirectional model Sec 3.1.
and training approaches Sec 4.2 (approximate joint probability) factorizes in the same way, pseudolikelihood is a convenient way to train this model.
4. EXACT MLE: TRACTABLE FACTORIZATION .
This training procedure only works for a special case of the full model, which has a tractable factorization of the joint probability, e.g., autoregressive models in ordered model (Sec 3.1.2) and perturbed models (Sec 3.1.4).
I NFERENCE
With the trained E θ ∗ , inference identiﬁes the best X that minimizes the energy function for given y test , i.e. X test =
arg min
X ∈X E θ ∗ ( X, y test ) .
Directly solving the above minimization is again intractable, but the energy function can generally be used for ranking.
Let R denote the rank of candidate
X i for the given y test (lower is better).
[Equation]
Practically, as illustrated in Fig 2, one can use either template-based or template-free method to come up with initial proposals for ranking, as follows.
Template-based Ranking (TB).
Templates can be used to extract a list of proposed reactant candidates by using templates.
We use template operator T ( · ) (deﬁned in Sec 3.2) to propose a list of candidate reactant sets from the input product y .
Template-free Ranking (TF).
In this paper, template-free ranking makes proposals using the learned structure prediction model.
We use a simple autoregressive form for p ( X | y ) , which can draw the top K most likely samples from this distribution using beam search, which is computational efﬁcient.
E XPERIMENTS
Experiment setup : Dataset and evaluation used follow existing work (Coley et al., 2017b; Dai et al., 2019; Liu et al., 2017; Shi et al., 2020).
We mainly evaluate our method on a benchmark dataset named USPTO-50k, which includes 50k reactions falling into ten reaction types from the US patent literature.
we split the datasets into train/validation/test with percentage of 80% / 10% / 10% .
Our evaluation metric is the top- k exact match accuracy, referring to the percentage of examples where the ground truth reactant set was found within the top k predictions made by the model.
Following the common practice, we use RDKit (Landrum, 2016) to canonicalize the SMILES string from different representations in different methods.
We augment USPTO with random SMILES as follows: (1) Replace each molecule in reactant set or product using random SMILES; (2) Random permute the order of reactant molecules.
We ﬁrst present the evaluation of our best EBM variant (dual model) against existing methods (Appendix A.2) for both template-based and template-free approaches in Sec 6.1.
Table 1 are our main results.
Dual model statistics are extracted from Table 2 and 3.
Then we provide comprehensive study on different variants of sequence-based EBMs in Sec 6.2.
Table 2 serves as an ablation study to understand performance of different models.
Then we provide template-free results in Table 3, which are our second main results.
We provide time and complexity analysis in Appendix A.7.
C OMPARISON AGAINST THE STATE - OF - THE - ART
Table 1 presents the main results.
All the baseline results are copied from existing works as we share the same experiment protocol.
The dual model is trained with randomized SMILES to inject order invariance information of molecule graph traversal.
Note that other methods like graph-based variants do not require such randomization as the graph representation is already order invariant.
We can see that, regarding top 1 accuracy when reaction type is unknown or known, our proposed dual model outperforms the current state-of-the-art methods by 9 . 9% and 6 .
7% for template-free setting, and 2 .
7% and 3 .
5% for template-based setting.
Note that Dual-TF has quite close top-1 accuracy as
Dual-TB, which demonstrates the discriminative ability of the designed energy function.
The DualTB has higher top 10 accuracy due to the higher coverage from templates compared to the proposal obtained by p ( X | y ) model.
This suggest that with better proposal during inference, we can further boost the current performance.
6. S EQUENCE - BASED VARIANT EVALUATION
In this section, we mainly compare different energy based sequence models described in Sec 3.1, with both template-based and template-free evaluation criteria. [Table] [Table]
6.2. T EMPLATE - BASED RANKING
Table 2 provides the results of template-based ranking (Sec 5 TB) for each sequence model variant described in Sec 3.1. Random SMILES has been shown useful for RNN and LSTM models
Ar ´ us-Pous et al. We want to convey the message to the community that augmentation using random SMILES is critical for ensuring best performance for transformer by preventing overﬁtting.
Without reiterating good performance for the dual variant, we focus on discussion of variants with undesired performance.
The perturbed sequential model (Sec 3.1.4) and bidirectional model (Sec 3.1.5) are inferior to dual or ordered models, where the main reason possibly comes from the fact that the learning objective approximates the actual model in Eq (9) and Eq (11) poorly, and thus leads to discrepancy between training and inference (See more discussion in Appendix).
The full model (Sec 3.1.1) despite being most ﬂexible and achieving best top 10 performance when type is given, would suffer from high computation cost due to the explicit integration even with the templates.
In addition to the understanding of individual models throughout the comprehensive study, we ﬁnd it is important to balance the trade-off between model capacity and learning tractability.
A powerful model without effective training would be even inferior to some well trained simple models.
Our dual model makes a good balance between capacity and learning tractability.
Please see discussion about semi-template based methods in Appendix A.2, A.3, A.4.
6.2. T EMPLATE - FREE RANKING
Table 3 presents the results for template-free evaluation (Sec 5 TF).
Template-free evaluation approach proposed in this paper requires a proposal model with good coverage and a ranking model with good accuracy.
We explored various combinations of proposal-ranking pairs.
The proposal model evaluated is the ordered model trained on USPTO50K and augmented USPTO50K, respectively.
The ranking model is the dual model trained on augmented data, as it performs the best in
Table 2.
Our best performer is ordered-proposal (USPTO 50K)-dual-ranking (aug USPTO 50K) model.
We reached top 1 accuracy as 53 . 6% and top 10 accuracy as 77 . 0% when type is unknown.
Note this accuracy are already close to template-based state-of-the-art.
A case study showing how dual model improves accuracy upon proposal is given Fig 4 in Appendix, where it shows how the energy based re-ranking reﬁnes the initial proposal.
One interesting observation is that, the proposal ordered model trained on augmented data has higher top 1 accuracy but much lower top 10 accuracy.
This indicates that such proposal has quite low coverage in the prediction space.
We observed that the model learned on augmented dataset learns various representations of the same molecule (due to usage of random SMILES)
A candidates proposal. [Table]
C ONCLUSION
In this paper we proposed an uniﬁed EBM framework that integrates multiple sequence- and graphbased variants for retrosynthesis.
Assisted by a comprehensive assessment, we provide a critical understanding of different designs. Based on this [Table]
we proposed a novel variant—dual model [Table] which outperforms state-of-the-art in both template-based and template-free setting [Table]
A A PPENDIX A. T ERMINOLOGY : R EACTION CENTER AND S YNTHONS
Reaction center of a chemical reaction are the bonds that are broken or formed during a chemical reaction.
For retrosynthesis, reaction centers are bonds exist in product, but do not exist in reactants.
One chemical reaction may have multiple reaction centers.
Synthons are the sub-parts extracted from the products by breaking the bonds in the reaction center.
Synthons are usually not valid molecules with ∗ to indicate the broken ends in the reaction centers.
We evaluate of our approach against several existing methods, including both template-based and template-free approaches.
Speciﬁcally, for Template-free ones : Transformer (Zheng et al., 2019) is a transformer based approach that trains a second transformer to identify the wrong translations and remove them.
LSTM (Liu et al., 2017) is a sequence to sequence approach that use LSTM as encoder and decoder.
For Template-based ones: retrosim .
(Coley et al., 2017b) selects template for target molecules using ﬁngerprint based similarity measure between targets and templates; neuralsym (Segler & Waller, 2017b) performs selection of templates as a multiple-class problem using MLP; GLN builds a template induced graphical model and makes prediction with approximated MAP. For Semi-Template based ones : The three methods: G2Gs (Shi et al., 2020), GRetroXpert (Yan et al., 2020), and GraphRETRO (Somnath et al., 2020), share the same idea:
infer reaction center to generate synthons, and then complete the missing pieces (aka “leaving groups”) in synthons to generate reactants. The three methods share the same ﬁrst step – phrasing identiﬁcation of reaction center bond as a classiﬁcation problem. They have different second steps: from synthons to reactants. G2G uses graph generation via variational inference.
RetroXpert use transformer. GraphRETRO predeﬁne s list of possible leaving groups and phrase the problem as a classiﬁcation problem. Please note G2G and RetroXpert are generative model that can be applied to unseen molecules, whereas GraphRETRO requires the pre-deﬁned list, which simplies the problem significantly and may not apply to molecules with unseen leaving groups. A. T EMPLATE - FREE METHODS AND S EMI - TEMPLATE BASED METHODS
In this paper, we refer template-free methods as the approaches that do not require any chemistry hand-crafted features, including templates, sub-parts of templates, reaction centers, and atom mapping.
More precisely, our experiment setup is full automatic template-free methods .
We are interested in this set up because
• These handcrafted features are often not accessible for real world application.
Getting these features ether requires intensive human efforts or non-trivial computational challenges.
For example, the ground truth “reaction center” used in semi-template based methods, have to be manually labeled by human experts when atom-mapping is not available.
So it is prohibitive for large datasets.
Alternatively, reaction centers can be computationally extracted using atom-mapping.
However, obtaining good quality atom mapping for large datasets itself is an equally challenging problem.
• Think beyond existing rule (e
g. handcrafted features).
The future retrosynthesis models should be able to think beyond existing chemical rules, and achieve full automation without handcrafted features.
We refer existing rules as templates, reaction center, atom mapping, reaction type, etc.
Thinking beyond exiting rules denotes generating unknown chemical rules.
Similar to the computer vision ﬁeld (e
g. imagenet classiﬁcation), people used handcrafted features at the beginning and all move to full automation without any handcrafted features because full automation achieves much better results.
Unfortunately, we are not there yet.
Including hand-crafted features often lead to a dramatic improvement ( 10%+ ) regardless of model design.
For example, well-known examples are reaction type and templates.
Please also so next section.
In this paper, we refer methods that use additional chemistry features during training as semitemplate-based methods.
For example, G2Gs (Shi et al., 2020), GRetroXpert
(Yan et al., 2020), and GraphRETRO (Somnath et al., 2020).
See description of the methods in Appendix A.2.
These methods do not apply template-matching explicitly, so they are not template-based methods. However, they use “reaction centers” as additional information to supervise their algorithm during training. The reaction centers preserve almost equivalent information as templates – once reaction centers are given, the product can be broken into two parts (denoted as synthons). The templates can be recovered by removing nonessential atoms from synthons and product.
In addition to reaction centers, RetroXpert also add additional features “the product side of template” to its atom features. GraphRETRO has a pre-deﬁned list of leaving groups. Therefore, we denote G2G, RetroXpert and GraphRETRO as semi-template based methods.
Semi-template based methods RetroXpert (Yan et al., 2020), and GraphRETRO (Somnath et al., 2020) achieved impressive results. Their performance even outperforms template-based approach.
The models presented in this paper have a different experiment setup from RetroXpert and GraphRETRO . Our setup does not use additional chemical information: reaction center, during training, which makes a head-to-head comparison not meaningful. To further investigate the effect of this additional chemistry features, we asked what will happen if our models also have the information of ”reaction center”?
Can we have the same or better performance when reaction center is given? As a proof of concept, we incorporated the reaction center to the dual model as follows: we ﬁrst generate synthons from product using the given reaction centers, and the then concatenate the generated synthons to the input of our dual model. [Table] Compared with Table 3 (when no synthons information is available), Table 4 shows adding reaction centers can boost accuracy signiﬁcantly: When reaction type is unknown, the accuracy of ﬁnal retrosynthesis improves from 54 .
5% to 70 . When reaction type is known, the accuracy improves from 66 . 2% to 73 . An interesting observation is that when reaction centers are presented, the effect of reaction type and augmentation on positively increasing the accuracy is reduced.
One explanation is reaction center is so important that it dominates all the other useful features or tricks. Although in the above experiments, we are given the reaction center, that is we do not infer them, Table 4 still serves well as a proof of concept that incorporating useful handcrafted chemistry features can signiﬁcantly boost accuracy regardless of model design. In addition to reaction centers, RetroXpert also add additional features “the product side of template” to its atom features. GraphRETRO has a pre-deﬁned list of leaving groups.
Incorporating those information (i.e. product side if template) in our setup may boost accuracy signiﬁcantly again. We want to avoid pursuing the path of chasing the state of the art by incorporating many useful handcrafted features, as it is contradictory to full automatic template-free methods. We want to deliver the message to the community that the performance of models should be compared under the same setup. The performance gain may be a result of using extra information, instead of modeling.
encoder decoder y p(X|y) X encoder decoder
X p(y|X) y encoder decoder . p(X)
X encoder decoder
X p(y|X) y encoder decoder . p(X)
X
Forward (X, y) ~ p(X, y) (a) Learning (b) Inference (X, y) ~ p(X, y) ^ ^ ^ ~ p(y) (X, y) p(X|y) model
Backward Dual Constraint data encoder decoder . p(X)
X score=log +log +log encoder decoder
X p(y|X)
y encoder decoder y p(X|y)
X
A. A BLATION S TUDY OF THE DUAL LOSS
We perform ablation study to investigate whether training with the dual loss, in particular the dual constraint, is the reason for the performance improvement. Recall the dual loss is given in Eq (6) and the dual constraint is the middle term in Eq (6). See Table 5.
The ”dual” row indicates training with dual loss and the results are taken from Table 1.
(cid:98) E [log p
γ
( X ) + log p
α
( y | X ) + log p
η ( X | y )] is dual loss without the dual constraint.
(cid:98) E [log p
α
( y | X ) + log p
η ( X | y )] is without prior log p γ ( X ) .
log p η
( X | y )] is only including backward direction.
Table 5 shows each component of the dual loss contribute to the performance positively
[Table]
[Table]
A. T IME AND SPACE COMPLEXITY ANALYSIS
In this section [Table] we provide time and space complexity regarding model design choices.
As the main bottleneck is the computation of transformer model, we measure the complexity in the unit of transformer model calls.
For all the models, the inference only requires the evaluation of (un-normalized) score function, thus the complexity is O (1) ; For training, the methods that factored have an easy form of likelihood computation, where a diagonal mask is applied to input sequence so that autoregressive is done in parallel (not | s ( x ) | times), so it requires O (1) model calls.
This include ordered/perturbed/bidirectional/dual models.
For the full model trained with pseudo-likelihood, it requires O ( | X | · | S | ) calls due to the evaluation per each dimension and character in vocabulary.
Things would be a bit better when trained with template-based method, in which it requires
O ( | T ( y ) | ) calls, which is proportional to the number of candidates after applying template operator.
As the memory bottleneck is also the transformer model, it has the same order of growth as time complexity with respect to sequence length and vocabulary size.
In summary we can see the Full model has much higher cost for training, which might lead to inferior performance.
Our dual model with a consistency training objective has the same order of complexity than other autoregressive ones, while yields higher capacity and thus better performance.
A. T RANSFORMER A RCHITECTURE
The implementation of variants in framework is based on OpenNMT-py (Klein et al., 2017).
Following (Vaswani et al., 2017), transformer is implemented as encoder and decoder, each has a self-attention layers with 8 heads and a feed-forward layer of size 2048.
We use model size and word embedding size as 256.
Batch size contains 4096 tokens, which approximately contains 20200 sequences depending on the length of sequence.
We trained for 500K steps, where each update uses accumulative gradients of four batches.
The optimization uses Adam (Kingma & Ba, 2014) optimizer with β 1 = 0 .
9 and β 2 = 0 .
998 with learning rate described in (Vaswani et al., 2017) using 8000 warm up steps.
The training takes about 48 hours on a single NVIDIA Tesla V100.
The setup is true for training transformer-based models, including ordered sequential model (Sec 3.1.2), perturbed sequential model (Sec 3.1.4), bidirectional model (Sec 3.1.5), dual model (Sec 3.1.3).
As for full model (Sec 3.1.1), each sample contains 20-500 candidates.
We implemented as follows: each batch only contains one sample.
Its tens or hundreds of candidates are computed in parallel within the batch.
The model parameters are updated when accumulating 100 batches to perform one step of update.
A. E XAMPLE OF CASE STUDY
Here we provide another case study showing with dual model ranking (Sec 3.1.3), the accuracy improves upon translation proposal. Please see Fig 4 and Fig 5.
NH O
Cl
ONH
Br
NH O
Cl
ONH
Br
N
O OH O
Cl O NH
Br
NH O
Cl
O NH
Br
NH O
Cl
O NH
Br
NH O
Cl
ONH
Br
N NH 2 OO
Cl O NH
Br
N NH 2 OO
Cl O NH
Br
N NH 2 OO
Cl O NH
Br
N NH 2 OO
Cl O NH Br
N NH 2 OO Cl O NH
Br
N NH 2 OO
Cl O NH
Br
N NH 2 OO
Cl O NH
Br
O
Br
NH
O
I
NH
O
I
NH
O
Br
NH NH
O
Br
NH
N OO
Cl O NH
Br
NHground truth ranking . .
Translation Proposal
Dual Model Re-rank
A. D ISCUSSION
V.1 Full model (Sec 3.1.1)
Full model (Sec 3.1.1) with template learning reaches accuracy of 39 .
5% and 53 .
7% on USPTO50k data-sets.
Full model is partially limited by expensive computation due to the number of candidates per product. V.2 Perturbed sequential model (Sec 3.1.4) Perturbed sequential model has about ∼ 4% accuracy loss in top 1 accuracy compared with ordered model (Sec 3.1.2). We argue the reason are as follows: ﬁrstly, we designed E θ as the middle term of Eq. Eq (9) to facilitate perturbing the order during training, following (Yang et al., 2019).
However, due to Jensen’s inequality, this design is not equal to P ( X | y ) , which causes discrepancy in ranking (inference). V.3 Bidirectional model (Sec 3.1.5) Bidirectional model, however, does not perform well in our experiments. The bidirectional-awareness makes the prediction of one position given all the rest of the sequence p ( X ( i ) | X ¬ i , y ) almost perfect ( 99 . 9% accuracy in token-level).
However, due to the gap between pseudo-likelihood and maximum likelihood, i.e., log P ( X | y ) , the performance for predicting the whole sequence will be inferior, as we observed in the experiments.
O
NH O O
OH
NH O O
O
O NH O
Mg+ Mg+
O
O NH
O HO
O
NH O O
OH
NH
O O O O
O
OH
NH O O
OH
NH O O
OH
NH O O
OH
NH O O
O
NH O O
OH
NH O O ground truth ranking . . . . . .
Translation Proposal
Dual Model Re-rank
OH
NH O O
OH
NH O O
A LTERNATIVE OF SMILES
SMILES AND SELFIES
In this section, we explore the effect of prepossessing procedure of sequence-based model, e.g. inline representation of molecular graph, in effecting performance of sequence-based model.
In particular, deepSMILES (Dalke, 2018) and SELFIES (Krenn et al., 2019) are alternatives to SMILES.
Without loss of fairness, we evaluated these representations using Ordered sequential model (Sec 3.1.2)The results indicate SMILES work the best.
We speculate the reason are deepSMILES and SELFIES are on average longer than SMILES, leading to higher probability of making mistakes on token level and therefore low sequence-level accuracy.
[Table] A.
T RANSFORMER IMPLEMENTATION OF P ERMUTATION I NVARIANT OF REACTANT SET
Transformer has a position encoding to mark the different locations on an input sequence.
We modiﬁed the position encoding such that each molecule starts with 0 encoding, instead of the concatenated position in the reactants sequence.
The results are Table 7.
We can see that this position encoding is beneﬁcial for non-augment data, but not augment data, as the latter has already considered the permutation invariance order of reactants by data augmentation.
In this paper, we use data augmentation to maintain order-invariant for reactants.
[Table]