# Learning Hierarchical Representations for Explainable Chemical Reaction Prediction
## 4 Experiments  
### Datasets  
The proposed method is evaluated on three publicly available datasets, i.e., the Schneider 50k [18] and the USPTO 1k TPL [9] datasets for reaction classification, as well as the USPTO yield [19] dataset for yield prediction. Table 1 briefly summarizes these datasets.  
The Schneider 50k dataset contains 50k reactions collected from the granted United States patents. The reactions are annotated with 50 template labels and 9 superclass labels using a substructure-based expert system. We use the superclass labels as the coarse- level labels in our method. We follow the data split in [18] to use 10k reactions for training and the remaining 40k reactions for testing. The performance is evaluated by the Accuracy, CEN [34], and MCC [35; 36] scores, respectively.  
The latter two datasets are both derived from the USPTO database [19], and both consisted of about 500k chemical reactions with different annotations. The USPTO data are highly noisy and biased.  
The USPTO 1k TPL dataset is divided into a training and validation set and a test set, where the former contains 90% data and the latter contains 10% data [9]. The performance on the reaction classification task is evaluated by the Accuracy, CEN, and MCC scores, respectively. We also conduct more challenge tests on this dataset by only selecting 32 training samples per class, named USP-few, and use the macro F1 score for evaluations following [12].  
The USPTO yield dataset contains a gram set and a sub-gram set. Following [10], we used two split methods, i.e., the random split and the time split, to validate the prediction performance of the proposed method. The time split denoted that data published in and before 2012 are used for training and validation, and the remaining were used for test. Since the data are quite noisy, the output yield of the data is smoothed as  
\[\overline{y}_{i}=\frac{1}{5}(\sum_{y_{i}^{NN}\in NN(y_{i})}y_{i}^{NN}+2y_{i}), \tag{11}\]  
where \(y_{i}\) and \(\overline{y}_{i}\) represent the yield values of the \(i\)-th data before and after the smoothing operation, respectively, and \(NN(y_{i})\) denotes the set of the 3-nearest neighbors of \(y_{i}\). The performance on the yield prediction task is evaluated by the R\({}^{2}\) score.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Datasets** & **Schneider 50k [18]** & **USPTO 1k TPL [9]** & **USPTO Yield [19]** \\ \hline Task & reaction classification & reaction classification & yield prediction \\ \hline Reaction Number & 50k & 445k & 500k \\ \hline Class Number & 9 super classes & \multirow{2}{*}{1000 template classes} & \multirow{2}{*}{-} \\  & 50 template classes & & \\ \hline \multirow{6}{*}{Split Strategy} & \multirow{2}{*}{10k for training 40k for testing} & 90\% for training 10\% for testing & Random split \\ \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \hline \hline \multirow{3}{*}{Metric} & Accuracy, CEN [34], and MCC [35; 36] & Accuracy, CEN [35; 36], and F1-score & R\({}^{2}\) \\ \cline{1-1} \cline{3-4}  & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets used in this paper.