SemiRetro: Semi-template framework boosts deep retrosynthesis prediction

1 INTRODUCTION . Retrosynthesis prediction ( Corey & Wipke , 1969 ; Corey , 1991 ) plays a crucial role in synthesis planning and drug discovery , which aims to infer possible reactants for synthesizing a target molecule . This problem is quite challenging due to the vast search space , multiple theoretically correct synthetic paths , and incomplete understanding of the reaction mechanism , thus requiring considerable expertise and experience . Fortunately , with the rapid accumulation of chemical data , machine learning is promising to solve this problem ( Coley et al. , 2018 ; Segler et al. , 2018 ) . In this paper , we focus on the single-step version 1 : predicting the reactants from a given product . Common deep-learning-based retrosynthesis works can be divided into template-based ( TB ) ( Coley et al. , 2017b ; Segler & Waller , 2017 ; Dai et al. , 2019 ) and template-free ( TF ) ( Liu et al. , 2017 ; Karpov et al. , 2019 ) methods . Generally , TB methods achieve high accuracy by leveraging reaction templates , which encode the molecular changes during the reaction . However , the usage of templates brings some shortcomings , such as high computation cost and incomplete rule coverage , limiting the scalability . To improve the scalability , a class of chemically inspired TF methods ( Shi et al. , 2020 ; Yan et al. , 2020 ) ( see Fig . 1 ) have achieved dramatical success , which decompose retrosynthesis into subproblems : i ) center identification and ii ) synthon completion . Center identification increases the model scalability by breaking down the target molecule into virtual synthons without utilizing templates . Synthon completion simplifies reactant generation by taking synthons as potential starting molecules , i.e. , predicting residual molecules and attaching them to synthons to get reactants . Although various TF methods have been proposed , the top-k retrosynthesis accuracy remains poor , especially when the reaction class is unknown . Can we find a more accurate way to predict potential reactants while keeping the scalability ? To address the aforementioned problem , we suggest combining the advantages of TB and TF approaches , and propose a novel framework namely SemiRetro . Specifically , we break a full-template 1In single-step retrosynthesis , the synthesis route length is 1 , i.e. , only one reaction needs to be inferred . into several simpler semi-templates and embed them into the two-step TF framework . As many semi-templates are reduplicative , the template redundancy can be reduced while the essential chemical knowledge is still preserved to facilitate synthon completion . Moreover , we introduce a directed relational graph attention ( DRGAT ) layer to extract more expressive molecular features to improve center identification accuracy . Finally , we combine the center identification and synthon completion modules in a unified framework to accomplish retrosynthesis predictions . We evaluate the effectiveness of SemiRetro on the benchmark data set USPTO-50k , and compare it with recent state-of-the-art TB and TF methods . We show that SemiRetro significantly outperforms these methods . In scalability , SemiRetro covers 96.9 % of data using 150 semi-templates , while previous template-based GLN requires 11,647 templates to cover 93.3 % of data . In top-1 accuracy , SemiRetro exceeds template-free G2G 3.4 % ( class known ) and 6.4 % ( class unknown ) . Owing to the semi-template , SemiReto is more interpretable than template-free G2G and RetroXpert in synthon completion . Moreover , SemiRetro trains at least 6 times faster than G2G , RetroXpert , and GLN . All these results show that the proposed SemiRetro boosts the scalability and accuracy of deep retrosynthesis prediction . 2 RELATED WORK . Template-based models TB methods infer reactants from the product through shared chemical transformation patterns , namely reaction templates . These templates are either hand-crafted by human experts ( Hartenfeller et al. , 2011 ; Szymkuć et al. , 2016 ) or automatically extracted by algorithms ( Coley et al. , 2017a ; Law et al. , 2009 ) . For a product molecule , due to the vast search space , multiple qualified templates , and non-unique matching sites for each template , it is challenging to select and apply the proper template to generate chemically feasible reactants . To handle those challenges , ( Coley et al. , 2017b ) suggests sharing the same templates among similar products . ( Segler & Waller , 2017 ; Baylon et al. , 2019 ) employ neural models for template selection with molecule fingerprint as input . The state-of-the-art GLN ( Dai et al. , 2019 ) learns the joint distribution of templates and products by decomposing templates into pre-reaction and post-reaction parts and introducing logic variables to apply structure constraints . TB methods are interpretable and accurate because they embed rich chemical knowledge into the algorithm . However , the vast space of templates and incomplete coverage severely limit their scalability . Template-free models Instead of explicitly using templates , TF approaches learn chemical transformations by the model . ( Liu et al. , 2017 ; Karpov et al. , 2019 ) solve the retrosynthesis problem with seq2seq models , e.g . Transformer ( Vaswani et al. , 2017 ) , LSTM ( Hochreiter & Schmidhuber , 1997 ) , based on the SMILES representation of molecules . Despite the convenience of modeling , SMILES can not fully utilize the inherent chemical structures and may generate invalid SMILES strings . Therefore , ( Zheng et al. , 2019 ) propose a self-corrected transformer to fix the syntax errors of candidate reactants . Recently , G2G ( Shi et al. , 2020 ) , RetroXpert ( Yan et al. , 2020 ) and GraphRetro ( Somnath et al. , 2021 ) achieve state-of-the-art performance by decomposing the retrosynthesis into two sub-problems : i ) center identification and ii ) synthon completion , as shown in Fig . 1 . Center identification increases the model scalability by breaking down the target molecule into virtual synthons without utilizing templates , among which G2G reports the highest accuracy . Synthon completion simplifies the complexity of reactant generation by taking synthons as potential starting molecules . For example , RetroXpert and G2G treat it as a SMILES or graph sequence translation problem from synthon to reactant . GraphRetro completes synthons by predicting pre-defined leaving groups , but it does not provide end-to-end open-source algorithms for attaching leaving groups and model construction . Generally , these TF methods are more scalable but perform worse than TB GLN in top-1 accuracy . Challenges Although the two-step TF framework significantly improves the algorithm ’ s scalability , the overall accuracy is relatively low . A possible solution to this issue is to enhance submodules , i.e. , center identification and synthon completion . 1 ) To the best of our knowledge , current GNN models only work well for center identification when the reaction class is known ; Otherwise , the model performance degrades rapidly . How to develop a more suitable model that works well with unknown classes is the first challenge . 2 ) In addition , synthon completion is the major bottleneck affecting the overall accuracy . Specifically , predicting and attaching residuals for each synthon are difficult because the residual structures could be complex , attaching residuals into synthons may violate chemical rules , and various residuals may agree with the same synthon ( e.g. , F , CI , Br , and I have similar chemical properties ) . For researchers , scalability , interpretability , and training efficiency are also important . How to develop a more accurate , interpretable , and efficient synthon completion model while maintaining the scalability is the second challenge . 3 DEFINITION AND OVERVIEW . Molecule representation There are two types of dominant representations , i.e. , SMILES string ( Weininger , 1988 ) and molecular graph . SMILES is commonly used in early works ( Liu et al. , 2017 ; Schwaller et al. , 2018 ; Zheng et al. , 2019 ; Schwaller et al. , 2019 ; Tetko et al. , 2020 ) due to its simpleness . Many NLP models can be directly applied to solve related problems in an end-to-end fashion . However , these models can not guarantee the chemical correctness of the output molecules because they ignore structure information to some extent . Similar to recent breakthroughs ( Dai et al. , 2019 ; Shi et al. , 2020 ; Yan et al. , 2020 ; Somnath et al. , 2021 ) , we take the molecule as a labeled graph G ( A , X , E ) , where A , X and E are adjacency matrix , atom features and bond features , seeing Table . 1 . Under the graph framework , we can effectively apply chemical constraints to ensure the validity of output molecules . Besides , graph-based methods are more controllable and interpretable than SMILES-based ones . Problem definition Retrosynthesis aims to infer the set of reactants { Ri } Ni=1 that can generate the product P . Formally , that is to learn a mapping function Fθ : Fθ : P 7→ { Ri } N1i=1 . ( 1 ) Considering the unknown by-products , the law of conservation of atoms no longer holds here , which makes the problem quite challenging because the algorithm needs to generate new atoms and bonds to get potential reactants automatically . Overview As shown in Fig . 1 , we adopt the two-step TF framework due to its scalability and effectiveness . Our method is distinguished from previous works in two folds : 1 ) We propose a relational graph attention ( DRGAT ) layer to improve the center identification performance ; 2 ) We use semitemplates to facilitate synthon completion , which significantly reduces the problem complexity . 4 METHODOLOGY . 4.1 CENTER IDENTIFICATION . Center identification plays a vital role in the two-step retrosynthesis because errors caused by this step directly lead to the final failures . Previous works have limitations , e.g. , RetroXpert ( Yan et al. , 2020 ) provides incomplete prediction without considering atom centers , and G2G ( Shi et al. , 2020 ) performs poorly when the reaction class is unknown . How to obtain comprehensive and accurate center identification results is still worth exploring . Reaction centers We consider both atom centers and bond centers in the product molecule . As shown in Fig . 2 , from the product to its corresponding reactants , either some atoms add residuals by dehydrogenation without breaking the product structure ( case 1 ) , or some bonds are broken to allow new residues to attach ( case 2 ) . Both these atoms and bonds are called reaction centers . Directed relational GAT Commonly used graph neural networks ( Defferrard et al. , 2016 ; Kipf & Welling , 2016 ; Veličković et al. , 2017 ) mainly focus on 0 and 1 edges , ignoring edge direction and multiple types , thus failing to capture expressive molecular features . As to molecules , different bonds represent different interatomic interactions , resulting in a multi-relational graph . Meanwhile , atoms at the end of the same bond may gain or lose electrons differently , leading to directionality . Considering these factors , we propose a directed relational graph attention ( DRGAT ) layer based on the general information propagation framework Zheng et al . ( 2021 ) , as shown in Fig . 3 . During message passing , DRGAT extracts source and destination node ’ s features via independent MLPs to consider the bond direction and use the multi-head edge controlled attention mechanism to consider the multi-relational properties . We add shortcut connections from the input to the output in each layer and concatenate hidden representations of all layers to form the final node representation . Labeling and learning reaction centers We use the same labeling algorithm as G2G to identify ground truth reaction centers , where the core idea is comparing each pair of atoms in the product P with that in a reactantRi . We denote the atom center as ci ∈ { 0 , 1 } and bond center as ci , j ∈ { 0 , 1 } in the product P . During the learning process , atoms features { hi } |P|i=1 are learned from the product P by applying stacked DRGAT , and the input bond features are { ei , j |ai , j = 1 } . Then , we get the representations of atom i and bond ( i , j ) as ĥi = hi||Readout ( { hs } |P|s=1 ) ︸ ︷︷ ︸ atom representation and ĥi , j = eij ||hi||hj ||Readout ( { hs } |P|s=1 ) ︸ ︷︷ ︸ bond representation , ( 2 ) where we use average Readout operation , and || is vector concatenation operation . Further , we predict the atom center probability pi and bond center probability pi , j via MLPs : pi = MLP6 ( ĥi ) and pi , j = MLP7 ( ĥi , j ) . ( 3 ) Finally , center identification can be reduced to a binary classification , whose loss function is : L1 = ∑ P ( ∑ i ci log pi + ( 1− ci ) log pi︸ ︷︷ ︸ atom center + ∑ i , j ci , j log pi , j + ( 1− ci , j ) log pi , j︸ ︷︷ ︸ bond center ) . ( 4 ) In summary , we propose a directed relational graph attention ( DRGAT ) layer to learn expressive atom and bond features for accurate center identification prediction . We consider both atom center and bond center to provide comprehensive results . In section . 5.2 , we show that our method can achieve state-of-the-art accuracy , especially with reaction class unknown .

Summary: This paper studies a retrosynthesis problem of chemical reactions.  This paper proposes a new variant of retrosynthesis models, with mainly two new components: DRGAT for powerful embedding computation and the semi-templates for accurate synthon completions.  Especially, the propose semi-template takes the local (neighboring) bonding structure into consideration. This significantly reduces the redundancy of the templates, allowing connecting the template-based retrosynthesis (less scalable but powerful) and the template-free retrosynthesis (scalable but less powerful).  The experimental validations show the efficacy of the new components for each subtask, and the final retrosynthesis performances exceeds the existing retrosynthesis models.