# Collaborative Learning Groupings Incorporating Deep Knowledge Tracing Optimization Strategies
## 4 Evaluation of Student Knowledge State Diagnosis
### Performance Comparison  
To validate the efficacy of the proposed knowledge tracing optimization strategy centered around the DKVMN-EKC model, we conducted student score prediction experiments on three datasets separately. In each experiment, 80% of the students were randomly allocated from the dataset for training purposes, while the remaining 20% were for testing the trained models. To mitigate the impact of Q matrix sparsity, we conducted a sensitivity analysis by randomly masking 10-30% of the knowledge point associations. The results showed that DKVMN-EKC maintained stable performance (AUC > 0.72) under moderate sparsity, indicating robustness to incomplete Q matrix annotations. The following three models were selected as the baseline for comparison in this study:  
* DINA model [27]: This is a common model for modeling students' knowledge states in the field of educational psychology. It uses a Q matrix to map the relationship between exercises and knowledge points and considers the two influencing factors of errors and guesses to obtain a binary estimate of students' knowledge states. * IRT model [28]: This model is based on the relationship between a student's ability and the percentage of correct responses. It integrates the student's ability with the situation in the question paper (difficulty, differentiation, etc.) and assesses the student's unidimensional ability values to diagnose potential traits. * DKVMN model [23]: This model represents and tracks the conceptual state of each knowledge point using a dynamic key-value pair memory network to characterize a student's knowledge state as a high-dimensional continuous feature. It is the basis for the DKVMN-EKC model. The student score prediction problem is a classification problem. Therefore, this experiment uses two evaluation metrics commonly used in classification: area under the curve (AUC) and accuracy (ACC), to measure the model's performance. The experimental findings are presented in Table 2.  
Analysis of the experimental results led to the following conclusions:  
1. The DKVMN-EKC model surpassed the DINA model for all three datasets, with the most significant improvement in Item II, where the AUC improved by 0.16 and the ACC improved by 0.05. The DINA model represents students' mastery of knowledge points as discrete two-dimensional variables. However, unlike the DKVMN-EKC model, it disregards the dynamic shifts in students' knowledge and mastery states during the response process. Instead, it only considers two influencing factors: student error and guessing, which are modeled in an excessively simplified manner. 2. The DKVMN-EKC model outperformed the IRT model on all three datasets, with the most significant improvement in Item II, where the AUC improved by 0.22 and ACC improved by 0.05. The IRT model uses a continuous variable to assess students' latent characteristics with difficulty in diagnosing students' mastery of their individual knowledge points, in contrast to the DKVMN-EKC model. 3. Compared to the DKVMN model, the DKVMN-EKC model performed slightly worse on both the Math2 and Item II datasets but was still within acceptable limits. This is \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Math1**} & \multicolumn{2}{c}{**Math2**} & \multicolumn{2}{c}{**Item II**} \\ \cline{2-7}  & **AUC** & **ACC** & **AUC** & **ACC** & **AUC** & **ACC** \\ \hline DINA & 0.66 & 0.63 & 0.68 & 0.64 & 0.59 & 0.69 \\ IRT & 0.69 & 0.64 & 0.71 & 0.65 & 0.53 & 0.70 \\ DKVMN & 0.74 & 0.68 & 0.77 & 0.69 & 0.77 & 0.75 \\ DKVMN-EKC & 0.74 & 0.68 & 0.76 & 0.69 & 0.75 & 0.74 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of AUC and ACC indicators by model. because, compared to the DKVMN model, the DKVMN-EKC model introduces a Q matrix, which inevitably leads to a degree of matrix sparsity; thus, the performance of the model is slightly degraded.