# A Multi-Layer Attention Knowledge Tracking Method with Self-Supervised Noise Tolerance
## 3 MASKT Model
### Self-Supervised Sequence Tasks  
Inspired by the BART model noise-reduction autoencoder method, MASKT uses multiple noise sequences to assist model training [16]. Noise tasks include input sequence filling, deletion, reordering, etc. They are divided into the following three aspects, as shown in Figure 3: F noise, D noise, and R noise. The original sequence of the student is \(E=(0,e_{0},e_{1}\dots e_{n-1})\), \(n=8\), and the maximum length is 16. In real-world student response scenarios, learner interaction sequences typically consist of a question sequence, an answer sequence, and a set of knowledge points. Considering that post-learned behavioral factors influence answer outcomes in real-world response processes, this hinders accurate analysis of learning behavior and knowledge mastery. Therefore, this paper proposes a simulation noise to emulate error-prone response behaviors in real-world scenarios. Three main types of noise are defined and discussed:  
F noise (Fragmentation Noise: This type of noise refers to a situation in which the question sequence, \(q_{t}\), and the knowledge point set, \(kc_{t}\), are complete in the interaction sequence tuple, but part of the answer sequence, \(a_{t}\), is missing. This noise is used to simulate situations where the recording of student answers is incomplete due to reasons such as students leaving the answer midway, network connection problems, or incomplete data recording, as shown in Figure 4. The answer sequences are then uniformly adjusted in length using two methods. The first method sets a sequence length threshold, padding short sequences with zeros and splitting long sequences. The second method uses mask-marked insertion. Filling positions are marked specifically, and invalid bits in the sequences are filtered out and recorded as \(e_{i}^{\delta f}=(e_{0}^{\delta f},e_{1}^{\delta f}\dots e_{n}^{\delta f})\). D noise (discrepancy noise): This type of noise simulates situations in interactive sequence tuples where the problem sequences \(q_{i}\) and \(kc_{i}\) are the same at different time steps \(i\) and \(j\), but the corresponding answer sequences, \(a_{i}\), are inconsistent, as shown in Figure 5. This simulates situations where students' understanding and responses to the same question change over time or where they make mistakes in their answers. This noise makes it difficult to analyze the consistency of students' knowledge mastery. For this noise, it is possible to use the [mask] to randomly select one sequence from the two unequal sequences for marking. This prevents the model from over-relying on certain features and reduces the risk of overfitting. This is denoted as \(e_{t}^{\delta f}=(e_{0}^{\delta f},e_{1}^{\delta d}\dots e_{n}^{\delta f})\). Figure 3: Original sequence structure of student interaction. R noise (reordering noise): In learner interaction sequences, some learner response sequences are misaligned with the question sequences, causing confusion in the overall interaction sequence. This noise simulates synchronization issues during data collection or sorting errors during data processing. R noise severely affects the time-series analysis of student responses, thereby affecting the accuracy of modeling student learning paths and knowledge mastery. As shown in Figure 6, according to the minimum length of sequence \(q_{t}\), assuming that the minimum length is \(n=4\), we mark the beginning and end of each question sequence \(q_{t}\), and re-check whether all grouped question sequences \(q_{t}\) and answer sequences \(a_{t}\) correspond to each other. We set the hyperparameter \(\omega\) such that when the matching of \(q_{t}\) and \(a_{t}\) is less than the \(\omega\) threshold. Judgment sequence endings are misplaced in such a way that \(a_{t}\) is less than \(q_{t}\), recorded as if it were a defective mismatch. Then, the starting marker position is deleted, and a subsequent \(a_{t}\) left shift is added.