# A Multi-Layer Attention Knowledge Tracking Method with Self-Supervised Noise Tolerance
## 1 Introduction  
Knowledge tracing is an important branch of education, including predicting learners' answer behavior, classifying knowledge points, and analyzing cognitive states. With the recent developments in deep learning technology, its application in knowledge tracing has also begun to spread. In traditional methods, the Bayesian knowledge tracking model (BKT) is a commonly used method for tracking student skill status [1]. Before the popularization of deep learning, it was used as a basic model for improvement and innovation and has strong interpretability. The rapid development of deep learning methods has prompted researchers to apply Recursive Neural Networks (RNNs) in the field of intelligent education [2], with performance superior to traditional models such as BKT. Masked language models perform well in handling missing information and noisy data, and use upstream multi-task training to provide a reliable foundation for downstream applications. The Multi-layer Attention Self-supervised Knowledge Tracing Method uses masked language models to help modelsin understanding language structure and information features, thereby improving the predictive performance of the model.  
This paper proposes a MASKT model that combines self-supervised learning with multi-layer attention networks for knowledge tracing. Specifically, since the data in the current knowledge tracing field contain different features, random forest is used to filter out highly correlated features in the initial learning interaction sequence and classify them into positive and negative correlations to enhance the compatibility between the model and the dataset. The processed sequence is then input into the self-supervised masking task encoder for secondary processing. By performing three different noise restoration tasks, namely fragmentation, discrepancy, and reordering, the sequence structure information is destroyed to reduce the risk of model overfitting. The output sequence is then input as a learning sequence into two special attention encoders: a node encoder and a dimension encoder, which are used to capture potential connections in historical learning sequences in a hierarchical manner to ensure that dependency information at different granularity levels is modeled. This enables the more comprehensive capture of key features and improves model prediction performance. During the optimization process, the MASKT model incorporates noise processing and multi-level semantic information into learning interactions and knowledge state representations. In this way, MASKT balances the significance of knowledge state and prediction performance. The main contributions of this paper are as follows:  
This paper designs a multi-level encoder based on attention to learn the dynamic input of masked language models, so as to more efficiently alleviate the long-dependency problem at different levels through hierarchical decomposition of sequences. Additionally, a random forest filtering strategy is designed to rank dataset attributes based on positive or negative correlations, ensuring the order of feature importance learning. This promotes MASKT's ability to learn more representative knowledge states and interactions. Extensive experiments on five public datasets demonstrate that MASKT outperforms other baseline models. Furthermore, MASKT adjusts parameters through ablation experiments to analyze the impacts of different noise sequences on model performance.