# Knowledge Graph and Personalized Answer Sequences for Programming Knowledge Tracing
## 4 Experiments
### Experiment Results  
#### 4.5.1 Comparative Experiment Analysis  
After multiple experiments, the final performance of all models on the two datasets is shown in Table 3, with the best results highlighted in bold. Baseline KT models provided by the pykt [36] library are used. Table 3 compares various KT models using two key metrics: AUC and ACC. The AUC measures the ability of a model to distinguish between correct and incorrect answers, with higher values indicating better performance. The ACC reflects the overall percentage of correct predictions made by the model. These metrics are critical for evaluating the effectiveness of KT models in predicting students' future performance. Overall, the GPPKT model proposed in this study is shown to perform well on both datasets after the knowledge graph and the student answer sequence are fully considered. On the Luogu dataset, an average improvement of 7.75% in AUC is exhibited by GPPKT compared to other methods, with a relative improvement of approximately 5.49% observed compared to the second-best method, ATKT. The ACC is recorded at 0.8472, showing an improvement of 1.51% over the average performance of other methods. On the Codeforces dataset, an average AUC improvement of 10.30% is demonstrated by GPPKT compared to other methods, with a relative improvement of about 3.01% seen compared to the second-best method, ATKT. The ACC is recorded at 0.8799, indicating an improvement of 2.53% over the average performance of other methods. Additionally, it is observed that similar performance is exhibited by DKT and DKT+, which are among the first to use deep knowledge tracing. This is due to the fact that programming exercises primarily consist of subjective questions, and students' understanding of KCs is relatively stable, leading to fewer fluctuations in the predicted state. DKT+ was developed specifically to address the problem of fluctuations in the prediction state. Although some improvement in preventing overfitting is provided by DKT+, it does not account for the inherent relationships between KCs. Furthermore, similar performance is exhibited by Deep-IRT and DKVMN. On the Luogu dataset, AUC values of 0.8020 and 0.8058 are recorded, respectively, and on the Codeforces dataset, AUC values of 0.6702 and 0.6828 are recorded, respectively. DKVMN is designed to dynamically update the mastery levels of relevant KCs and performs well on objective exercises. However, in programming KT, where subjective exercises prevail and students often engage in continuous answering behavior, the impact of KC forgetting is not significant. Deep-IRT is an improvement upon the DKVMN model; although students' learning abilities are also considered, it indirectly reveals the limitations of traditional KT methods when addressing personalized answers for subjective exercises. Both SAKT and SAINT are based on transformer models, and similar AUC values are observed for \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{2}{c}{**Luogu**} & \multicolumn{2}{c}{**Codeforces**} \\ \cline{2-5}  & **AUC** & **ACC** & **AUC** & **ACC** \\ \hline DKT & 0.8114 & 0.8397 & 0.6804 & 0.8574 \\ DKT+ & 0.8104 & 0.8388 & 0.7037 & 0.8589 \\ Deep-IRT & 0.8020 & 0.8271 & 0.6702 & 0.8526 \\ DKVMN & 0.8058 & 0.8300 & 0.6828 & 0.8573 \\ IEKT & 0.8230 & 0.8309 & 0.7308 & 0.8567 \\ SAKT & 0.8301 & 0.8307 & 0.6708 & 0.8550 \\ SAINT & 0.8263 & 0.8254 & 0.6763 & 0.8577 \\ AKT & 0.8313 & 0.8394 & 0.7491 & 0.8595 \\ ATKT & 0.8380 & 0.8418 & 0.7543 & 0.8614 \\
**GPPKT (our)** & **0.8840** & **0.8472** & **0.7770** & **0.8799** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparative experiments. both. However, relationships between exercises and students' learning abilities are not considered by either model. A certain gap between IEKT and GPPKT is also observed. Although the IEKT model integrates individual cognition and learning processes, the relationship between KCs is not considered.