# A Multi-Layer Attention Knowledge Tracking Method with Self-Supervised Noise Tolerance
## 3 MASKT Model
### Feature Embedding Screening  
As an ensemble learning method, random forest consists of multiple decision trees and can effectively deal with nonlinear relationships between features and high-dimensional and redundant features. Compared with supervised methods such as linear discriminant analysis, random forest is more stable and robust in dealing with nonlinear feature relationships, and is particularly suitable for complex and diverse feature structures such as educational behavior data [14]. Compared with the \"black box\" feature extraction method of deep neural networks, random forest provides a clear ranking of feature importance, which can more intuitively understand which behavioral features (such as the number of prompts, the correctness of answering questions, etc.) play a key role in predicting learning performance, which is conducive to teaching intervention and strategy formulation [15]. Therefore, with the purpose of using the least feature vectors and carrying the most student information, the importance of the screened behavioral features is ranked as the pre-feature screening module of the self-supervised learning model, providing more refined and discriminative input features for subsequent modeling, and being able to more specifically retain features that have a significant impact on the prediction results. Since the attributes contained in the learners' answers also have different characteristics, such as differences in the relevant attributes when the answer is correct or incorrect, we divide the questions into two levels. Learner-question pair: When the answer is correct, a bipartite graph is formed by the learners' correct answers in the dataset. Therefore,  
Figure 1: MASKT overall architecture. we believe that there may be a correlation between the questions that the same learner answered correctly and incorrectly. The positive feedback correlation score is denoted as \(UF^{+}\), representing the learner-question pair where the answer is incorrect, while the negative feedback correlation score is denoted as \(UF^{-}\). The principle involves bootstrapping random sampling of the training set during training to form \(n\) datasets, generating \(N\) decision trees from these datasets, and using out-of-bag (OOB) data to calculate errors and evaluate the accuracy of the random forest. In a random forest, assuming there are N trees, the correlation of features is evaluated using Formula (2). \[Imp=\frac{1}{N}|errOOB_{1}-errOOB_{2}| \tag{2}\]  
\(Imp\) indicates the relevance score of the feature, \(i\) indicates the number of features, \(N\) indicates the number of decision trees in the random forest, \(errOOB_{1}\) indicates the out-of-bag error without interference, and \(errOOB_{2}\) indicates the out-of-bag error with noise interference. The evaluation criterion for feature importance is whether the accuracy of the OOB data significantly decreases after noise is added. If the accuracy decreases, this indicates that the feature has a significant impact on the prediction results of the model and is marked as an important feature. The above method is transferred and applied to the evaluation of behavioral features. The behavioral features obtained are sorted by importance. Before transferring them to the MASKT model, the learner's answer behavior features extracted using random forests are weighted using PCA (Principal Component Analysis). During the answering process, learners' interactions with different questions are also different, and there is a necessary relationship between interaction relevance and answer accuracy. Based on the learners' answer records, the relevance between answer interactions and answer results is quantified as follows:  
\[Correlation(x_{j}) \begin{cases}\frac{\sum_{i=1}^{|N_{j}|}|e_{ij}=0|}{|N_{j}|},| N_{j}|\\ 0.5,else\end{cases}  \tag{3}\]  
\(N_{j}\) is the set of learners who have answered the current question. \(e_{ij}\) is the response result of learner \(i\) to exercise \(j\), where 1 or 0 indicates correct/incorrect.