# A Multi-Layer Attention Knowledge Tracking Method with Self-Supervised Noise Tolerance
## 6 Experiment
### Performance Evaluation  
The MASKT model introduces a multi-layer attention network to find the link between topic-skill relationships, represents topic and skill information models, and overall performs better than using a single skill information or topic information model. Figure 11 shows the results of the overall performance evaluation of the MASKT model. Experiments using both AUC and RMSE for more comprehensive comparisons reveal significant differences in performance on most datasets. Examples of the results are presented here: Figure 11a shows on the Algebra05 dataset; the SAINT model exhibits a superior prediction performance, presumably because Transformer captures the complex relationship between exercises and answers more effectively. The SKVMN model outperforms SAINT and other datasets on the ASSIST2012 dataset, possibly because the SKVMN model has enhanced time-series capabilities and loop modeling to better capture the dependencies between knowledge states. In addition the AKT model's introduction of problem-skill relationships in the Transformer architecture allows the model to capture learning features well, and is in the second or third position overall. Finally, the MASKT model employs a specific data cleaning model that introduces an effective self-supervised learning method and hierarchical attention mechanism to guide the model to enhance knowledge feature extraction and knowledge state representation. The comparative metrics based on AUC and RMSE MASKT consistently outperforms all the models, and is able to maintain a stable performance beyond 0.2 percentage points on the AUC of all the models. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Algebra05** & **ASSIST09** & **ASSIST12** & **ASSIST15** & **EdNet** \\ \hline MASKT-F & 0.8047 & 0.7751 & 0.7341 & 0.7451 & 0.7654 \\ MASKT-D & 0.7992 & 0.7796 & 0.7473 & 0.7337 & 0.7631 \\ MASKT-R & 0.7949 & 0.7841 & 0.7482 & 0.7401 & 0.7573 \\ MASKT & 0.8103 & 0.7944 & 0.7620 & 0.7553 & 0.7714 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of Best AUC values in ablation experiments. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Algebra05** & **ASSIST09** & **ASSIST12** & **ASSIST15** & **EdNet** \\ \hline MASKT-D & 0.7992 & 0.7796 & 0.7473 & 0.7337 & 0.7631 \\ MD-Dynamic & 0.8095 & 0.7953 & 0.7579 & 0.7426 & 0.7694 \\ MASKT & 0.8103 & 0.7944 & 0.7620 & 0.7553 & 0.7714 \\ MASKT-DY & 0.8196 & 0.8109 & 0.7673 & 0.7518 & 0.7786 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of best AUC values in static/dynamic mask experiments. The ablation experiment performance comparison shown in Figure 12 is designed to explore the optimal \(\lambda\) value by adjusting the regularization proportion \(\lambda\) within the loss function. For newly introduced datasets, conclusions are drawn by tuning the \(\lambda\) parameter within the MASKT model and observing its impact on prediction performance. As shown in Figure 12, across various datasets, once \(\lambda\) exceeds a specific threshold, the overall model performance experiences a notable decline. The core objective of the MASKT model is to balance prediction performance and sequence feature correlation, thereby enhancing both interpretability and robustness. To verify this balancing mechanism, comparative experiments are conducted among single-noise-processing methods (filling, delete, random), the full MASKT model with multi-noise processing, and a dynamic training strategy (denoted as Dynamic) serving as a baseline. Figure 12a demonstrates that the MASKT model achieves relatively superior performance, highlighting the effectiveness of noise restoration on this dataset. However, the random noise method proves less effective. At higher \(\lambda\) values, the amplified validation loss weight exposes the shortcomings of the random method, which lacks a consistent processing strategy and thus suffers from conflict-driven degradation.