# Bridging the Vocabulary Gap: Using Side Information for Deep Knowledge Tracing
## 6 Results
### Explainability Analysis Using SHAP  
To further analyze the importance of each feature, this study used Shapley values to analyze the importance and contribution of each feature in the prediction. Shapley Additive Explanations (SHAP) is a method used to explain model predictions by calculating the contribution of each feature to the model's output [46]. Figure 4 shows the importance of all features, ranked from high to low. The darker red indicates higher feature values, and the lighter blue indicates lower feature values. The horizontal axis is the SHAP value of the corresponding feature, which is the impact size of that feature on the model output. If the SHAP value is negative, it means that the feature will decrease the probability of the student answering correctly. If it is positive, it will increase the probability. We can see that _q_acc, sq_acc, sk_acc,_ and _q_mt_ were the four most important features, which is also consistent with previous experimental results indicating that features about questions and students have the greatest impact on model performance. When _q_acc_ has a higher value, the model tends to predict that the student can answer the question correctly, because most of the students can answer this question correctly. Similarly, when _sq_acc_ or _sk_acc_ are higher, the model also tends to predict that the student can answer the question correctly because the student has a stronger ability or has mastered the knowledge well enough. On the contrary, when the average time to answer a question (_q_mt_) is longer, it indicates that the question is more difficult, so the student is also less likely to answer it correctly. When the number of questions the student has answered for a knowledge (_sk_n_) increases, it means that the student has not yet mastered that knowledge fluently, so they need more practice. Therefore, the model tends to predict that the student cannot answer the question correctly. Specifically, we used two case studies to explain how each feature affects the output of the model. As shown in Figures 5 and 6, blue represents that the feature has a negative impact on the prediction, and red represents that the feature has a positive impact on the prediction. In Case 1, the model output was 0.84, indicating a high likelihood of a correct answer. Three features significantly impacted this outcome: _sq_acc_ was 70.38, _sk_acc_ was 66.67, and _q_acc_ was 40.51. The first two had a positive impact on the results, while the \begin{table}
\begin{tabular}{l l l l l l l} \hline
**Input** & **Remove** & **A09** & **A12** & **EN** & **Junyi** & **Eedi** \\ \hline
5–14 & 1–4 [FOOTNOTE:]Footnote : Removed student master. [ENDFOOTNOTE] & 79.55 (\(-\)1.18) & 72.76 (\(-\)3.14) & 72.87 (\(-\)0.08) & 71.74 (\(-\)0.97) & 76.47 (\(-\)1.85) \\
1–4, 9–14 & 5–8 [FOOTNOTE:]Footnote : Removed student master. [ENDFOOTNOTE] & 77.52 (\(-\)3.21) & 73.73 (\(-\)2.17) & 71.96 (\(-\)0.99) & 72.62 (\(-\)0.09) & 75.43 (\(-\)2.89) \\
9–14 & 1–8 [FOOTNOTE:]Footnote : Removed student master. [ENDFOOTNOTE] & 73.97 (\(-\)6.76) & 68.96 (\(-\)6.94) & 71.95 (\(-\)1.00) & 70.24 (\(-\)2.47) & 67.18 (\(-\)11.14) \\
1–8, 12–14  & 79.82 (\(-\)0.91) & 74.36 (\(-\)1.54) & 72.90 (\(-\)0.05) & 72.34(\(-\)0.37) & 78.23 (\(-\)0.09) \\
1–11 & 12–14 [FOOTNOTE:]Footnote : Removed student master. [ENDFOOTNOTE] & 76.78 (\(-\)3.95) & 69.83 (\(-\)6.07) & 69.42 (\(-\)3.53) & 68.54 (\(-\)4.17) & 74.46 (\(-\)3.86) \\
1–14 & - & 80.73 & 75.90 & 72.95 & 72.71 & 78.32 \\ \hline \end{tabular}
\end{table} 
Table 7: AUC results of SIKT model when removing different side information. third had a negative impact. This means that although the accuracy of this question was only 40.51, this student had stronger ability and had basically mastered this knowledge. Therefore, the model predicted that the student can answer this question correctly. In Case 2, the situation was somewhat different, and the model output was 0.43.