# Learning Hierarchical Representations for Explainable Chemical Reaction Prediction
## 3 Method
### Model Structure
#### 3.1.2 Coarse-Fine Level Sequence Encoding  
The architecture overview of the coarse-fine level sequence encoding is shown in Figure 2, and we elaborate it as follows: Since reactions often occur between different functional groups in which the atoms are always adjacent when the reactions are represented by SMILES, we use a 1D CNN to extract the local context information from the transformer-based local features \([\mathbf{x}_{1}^{t},\dots,\mathbf{x}_{N}^{t}]\) to fully exploit the functional group-level representations. To map the global feature \(\mathbf{x}_{global}^{t}\) into the same space with the local features, we employ a nonlinear fully connected feed forward network (FFN) to obtain the \"local context\" feature of the global node. Accordingly, we denote the extracted local context features by \(\mathbf{x}^{t}=\left[\mathbf{x}_{global}^{t},\mathbf{x}_{1}^{t},\mathbf{x}_{2}^{t},\dots,\mathbf{x }_{N}^{t}\right]\). Figure 1: Schematic overview of our method for chemical reaction representation. The entire model structure of the coarse-fine level chemical-reaction representation is shown in **(a)**. The Left part of (b) illustrates the data augmentation for coarse-level contrastive learning using two examples. The right part of (b) shows two examples of reaction classification and yield prediction, respectively. For more explainable modeling and representative encoding, we divide the following model into two branches to separately conduct Coarse-level Sequence Encoding (CSE) and Fine level Sequence Encoding (FSE). The coarse-level represents the molecules with specific bonds that are altered in the reaction, such as the \"C-C bond formation\" and the \"Oxidations\". The fine level represents the molecules in the reaction as specific functional groups or atoms, such as the \"Alcohol to aldehyde oxidation\" and the \"Fischer-Speier esterification\". Figure 3 illustrates examples of the coarse-level and fine-level differences between different reactions. Figure 3a is an oxidation reaction and Figure 3b,c are acylation reactions, where the bond of \"-OH\" alters into \"=O\" in Figure 3a and the bonds of \"=C=O\" are both altered in Figure 3b,c. Figure 3b is different from Figure 3c because Figure 3b contains \"-NH\({}_{2}\)\" and \"-N=C=O\" in reactants but Figure 3c contains \"-COO-\" in the product. For CSE, we apply a sparse attention mechanism to encode the sequence into the coarse-level encoding feature \(\mathbf{c}\), given by  
\[\mathbf{c}=\left[\mathbf{x}_{1}^{l},\ldots,\mathbf{x}_{N}^{l}\right]\cdot\mathbf{\alpha}_{ coarse}, \tag{1}\]  
where \(\mathbf{\alpha}_{coarse}\) represents the attention coefficient vector at the coarse level. The \(\mathbf{\alpha}_{coarse}\) is obtained by optimizing  
\[\operatorname*{arg\,min}_{\mathbf{a}\in\mathbf{B}^{N}}\parallel\mathbf{\alpha}-\widetilde {\mathbf{\alpha}}_{coarse}\parallel, \tag{2}\]  
Figure 3: Examples of coarse-level and fine-level differences. The pink color remarks the main changes during the reactions. (**a**) is different from (**b**,**c**) at the coarse level. (**b**,**c**) are different at the fine level. Figure 2: Architecture of the coarse-fine level sequence encoding module. where \(\widetilde{\mathbf{\alpha}}_{coarse}=[\widetilde{\mathbf{\alpha}}_{1},\dots,\widetilde{\mathbf{ \alpha}}_{N}]^{\top}\), \(\widetilde{\mathbf{\alpha}}_{i}\) is calculated via a two-layer non-linear fully connected network with \(\left[\mathbf{x}_{global}^{t},\mathbf{x}_{i}^{t}\right]\) as input, and \(\mathbf{B}^{N}=\left\{\mathbf{b}\in(\mathbf{R}^{+})^{N}\big{|}\mathbf{1}^{\top}\mathbf{b}=1\right\}\) is the \(N\)-dimensional simplex in which each vector has positive elements with total summation of 1. The optimization depicted in Equation (2) is solved by projecting \(\widetilde{\mathbf{\alpha}}_{coarse}\) onto \(\mathbf{B}^{N}\).