# Bridging the Vocabulary Gap: Using Side Information for Deep Knowledge Tracing
## 6 Results
### Ablation Experiments Based on Different Features  
In order to further verify the actual role of side information in the KT, we removed the side information of students, questions, and knowledge in turn, and reported the SIKT model effect of missing different side information. The AUC results on five datasets are shown in Table 7. Firstly, when removing features related to student knowledge mastery, the performance of the model on five datasets decreased by 1.18 (A09), 3.14 (A12), 0.08 (EN), 0.97 (Junyi), and 1.85 (Eedi). When removing features related to student's abilities, the performance of the model on five datasets decreased by 3.21 (A09), 2.17 (A12), 0.99 (EN), 0.09 (Junyi), and 2.89 (Eedi). When features related to student characteristics are removed, the performance of the model decreases by the largest amounts. It decreased by 6.67 (A09), 6.94 (A12), 1.00 (EN), 2.47 (Junyi), and 11.14 (Eedi) on the five datasets. This indicates that in KT, the representation of students' abilities is very important because it can largely determine whether the student can answer a certain question. Second, after removing the  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**A09-Test**} & \multicolumn{2}{c}{**A12-Test**} & \multicolumn{2}{c}{**EN-Test**} & \multicolumn{2}{c}{**Junyi-Test**} & \multicolumn{2}{c}{**Eedi-Test**} \\  & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** \\ \hline SIKT-A09 1 & 80.73 & 72.31 & 73.95 & 65.20 & 69.73 & 65.48 & 66.67 & 78.90 & 69.03 & 59.48 \\ SIKT-A12 & 79.76 & 72.87 & 75.90 & 73.20 & 70.95 & 67.88 & 65.29 & 70.63 & 63.20 & 58.22 \\ SIKT-EN & 76.65 & 65.98 & 71.24 & 56.47 & 72.95 & 65.64 & 67.71 & 71.02 & 61.29 & 58.84 \\ SIKT-Junyi & 69.98 & 62.10 & 65.32 & 61.76 & 61.11 & 60.00 & 72.71 & 88.49 & 68.60 & 64.37 \\ SIKT-Eedi & 75.26 & 60.43 & 67.69 & 60.27 & 59.41 & 56.49 & 60.66 & 70.22 & 78.32 & 59.87 \\ \hline Join train 2 & 80.77 & 72.36 & 76.04 & 73.15 & 73.11 & 65.70 & 72.80 & 88.50 & 79.01 & 64.07 \\ \hline \hline \end{tabular}  
* The SIKT-A09 model was trained on the A09-train dataset and evaluated on the A09-test, A12-test, EN-test, Junyi-test, and Eedi-test datasets. The same strategy was applied to other models. 2 The model was trained by mixing the training datasets from all datasets together, and then its performance on different test sets was evaluated.  
\end{table}
Table 6: Results of SIKT model on new datasets.  
side information of the question, the AUC decreased by 3.95 (A09), 6.07 (A12), 3.53 (EN), 4.17 (Junyi), and 3.86 (Eedi). This indicates that the side information of the question is also a very important feature because the difficulty of the question can also determine whether the student can answer the question correctly. Finally, it is also interesting to note that when removing knowledge-related side information, the AUC only decreased by 0.91 (A09), 1.54 (A12), 0.05 (EN), 0.37 (Junyi), and 0.09 (Eedi). It indicates that in KT, knowledge-related side information has the smallest impact on the effect. This is a counterintuitive finding; the explanation we can think of is whether KT primarily focuses on the relationship between students and questions, where knowledge plays a certain connecting role between the two, that is, questions classified with different knowledge are presented to students. In summary, it can be found that the side information related to students and questions has the greatest impact on model performance, while the side information related to knowledge has the smallest impact on model performance.