## 4 Experiment and Analysis
### Comparison and Analysis of Experimental Results
#### 4.5.2 Ablation Experiment and Visualization  
In order to verify and analyze the effectiveness of the components in the model of this paper, as well as the optimal parameter settings, ablation experiments were conducted in this paper.  
**Information Difficulty Extraction Module Ablation**  
In this section, we replace the multi-task learning part of the information difficulty extraction module in this model using two parallel Transformers (without sharing parameters and inputs) to obtain the difficulty embedding vector of the predicted questions and the difficulty embedding vector of the skills, respectively, while the rest of the model remains the same as the original model. The results of the ablation experiments are shown in Table 7.  
From the results in the table, it can be concluded that the model with the information difficulty extraction module had a better performance, and the improvement was especially obvious for the Assistments09 data set. This may be due to the fact that the question-based interaction records in the Assistments09 data set were more sparse (refer to Table 1), and the multi-task learning allowed the model to learn more and richer features by sharing inputs and underlying parameters, thus improving the prediction performance of the model.  
**Student Ability Extraction Module Ablation**  
In this section, we show how the improved attention mechanism in the student competence module affected the performance of the model and how different behavioral characteristics in the performance bias function affected the status of students.  
**MTLKT-NA**: Represents calculating weights between vectors without using the attention mechanism in the model, instead simply adding up the historical interaction embedded vectors to obtain the current knowledge state of the students.  
**MTLKT-A**: Indicates that attention mechanisms were used but performance bias functions were not.  
**MTLKT-attempt**: Represents the use of a student's \"number of attempts\" as a parameter in the bias function to improve the attention mechanism.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Data Sets**} & \multicolumn{2}{c}{**Assistments09**} & \multicolumn{2}{c}{**Assistments12**} & \multicolumn{2}{c}{**Assistments17**} \\ \cline{2-7}  & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** \\ \hline Compared Model & 0.7550 & 0.7324 & 0.7669 & 0.7438 & 0.7800 & 0.7575 \\ \hline MTLKT & 0.7703 & 0.7659 & 0.7780 & 0.7542 & 0.7860 & 0.7653 \\ \hline Elevation & 1.53\% & 3.35\% & 1.11\% & 1.04\% & 0.6\% & 0.78\% \\ \hline \hline \end{tabular}
\end{table}
Table 7: Information difficulty extraction module ablation results.  
Figure 5: Comparison with pre-trained model.  
From the experimental results in Table 8, we could conclude that the use of the attention mechanism improved the accuracy of the model, and the introduction of the performance bias function with appropriate improvements to the attention mechanism could better simulate the impact of student behavioral performance on knowledge state during the question-answering process, thus improving the accuracy of the prediction. MTLKT used \"response time \"as a parameter in the performance bias function, the performances for all three data sets were higher than that of MTLKT-attempt, and the \"response time\" was more representative of student performance than their \"number of attempts\". It had a greater impact on student knowledge state.