# Bridging the Vocabulary Gap: Using Side Information for Deep Knowledge Tracing
## 6 Results
### Results on the New Dataset
\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**A09-Test**} & \multicolumn{2}{c}{**A12-Test**} & \multicolumn{2}{c}{**EN-Test**} & \multicolumn{2}{c}{**Junyi-Test**} & \multicolumn{2}{c}{**Eedi-Test**} \\  & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** \\ \hline DKT-A09 * & 79.87 & 72.72 & 60.92 & 67.46 & 52.63 & 50.41 & 54.00 & 75.53 & 50.62 & 53.13 \\ DKT-A12 & 65.78 & 61.70 & 72.87 & 73.12 & 48.41 & 49.51 & 56.10 & 78.56 & 52.87 & 61.38 \\ DKT-EN & 51.03 & 51.42 & 49.56 & 50.54 & 68.67 & 64.67 & 52.58 & 65.66 & 50.11 & 52.67 \\ DKT-Junyi & 51.02 & 52.41 & 52.20 & 59.88 & 46.66 & 47.20 & 69.99 & 77.83 & 51.22 & 53.49 \\ DKT-Eedi & 70.74 & 65.07 & 63.79 & 67.61 & 44.91 & 46.68 & 56.84 & 88.53 & 74.45 & 70.04 \\ \hline SAINT-A09 & 75.62 & 69.18 & 50.60 & 58.07 & 56.08 & 52.94 & 48.96 & 58.21 & 49.99 & 54.58 \\ SAINT-A12 & 48.15 & 52.10 & 65.06 & 70.02 & 53.14 & 55.41 & 50.73 & 77.20 & 49.62 & 59.59 \\ SAINT-EN & 51.03 & 51.42 & 49.56 & 50.54 & 77.14 & 70.13 & 50.33 & 40.12 & 49.13 & 46.82 \\ SAINT-Junyi & 49.82 & 56.22 & 50.35 & 69.74 & 48.11 & 55.89 & 70.20 & 88.61 & 49.44 & 63.15 \\ SAINT-Eedi & 53.68 & 54.61 & 50.12 & 59.51 & 44.06 & 51.71 & 48.10 & 60.28 & 63.66 & 62.62 \\ \hline \hline \end{tabular}
* The DKVMN model was trained on the A09-train dataset and evaluated on the A09-test, A12-test, EN-test, Junyi-test, and Eedi-test datasets; the same strategy was applied to other models.  
\end{table} 
Table 5: Results of DLKT models on new datasets. Table 6 presents the results of the SIKT model trained on one dataset and predicted on another dataset. Firstly, after the model had trained on one dataset, it cloud directly predict other datasets, and achieve comparable performance, such as the SIKT-EN model obtained by training on EN (AUC: 72.95, ACC: 65.64) being able to predict on A09 (AUC: 76.65, ACC: 65.98), A12 (AUC: 71.24, ACC: 56.47) Junyi (AUC: 67.71, ACC: 71.02), and Eedi (AUC: 61.29, ACC: 58.84) directly, and the situation was similar for other datasets. This indicates that although the model has not seen the students, problems, or knowledge in the other datasets, it can still achieve good performance on the new dataset. Secondly, the AUC and ACC were similar when SIKT-A12 was tested on A09 (AUC: 79.76, ACC: 72.87) compared to SIKT-A09 tested on A09 (AUC: 80.73, ACC: 72.31). It is shown that the model can acquire more shared knowledge when learning similar tasks because these two datasets come from different years in the same learning system. Thirdly, it is also interesting to note that SIKT-A12 and SIKT-EN even showed higher test performances on A09-test than on A12-test and EN-test. This indicates that the model can perform well on a smaller number of datasets after learning a database that contains a larger number of SKQ. Therefore, we further merged all the datasets to train SIKT. Finally, it can be seen that during the joint training, the model achieved the highest performance on five datasets, as it was able to learn the feature distribution of all datasets at once and perform the best on all five datasets simultaneously. This also means that using SIKT can train data from multiple datasets simultaneously, without constantly bloating the model due to the increase in SKQ, solving the problem of the DLKT model being unable to expand quickly. Overall, we found that SIKT can be trained on one dataset and directly predict on new datasets, while maintaining considerable performance. It can also learn all tasks at the same time and achieve acceptable performance. What this means is that if the model is allowed to learn from as many different datasets with a wide distribution of data, it obtain a more general model that can support more KT tasks.