## 6 Results and Discussion
### Visualization  
Figure 6 shows the distribution of the questions belonging to a knowledge concept in the embedding space. For the sake of visualization, we used the t-SNE technique to map the original space to a 3D space. Figure 6a represents the Rasch model, while Figure 6b represents our QQE model. The deep knowledge tracing algorithms based on the Rasch model [13; 14; 16] assume that all the questions belonging to a knowledge concept can be expressed as a linear combination of two vectors, which are entirely determined by the knowledge concept. Therefore, in the original embedding space, the data points representing these questions are distributed along a line segment. When mapped to the 3D space, they are squeezed into a particularly narrow manifold. On the other hand, after learning with the QQE model, the embedding points of the questions were distributed in a spherical space that resembled a Gaussian distribution. This distribution can more naturally reflect the relationship between the questions and the knowledge concept. This difference shows that our model alleviates the overparameterization problem without introducing overly strong prior assumptions, resulting in better representations and better predictions. Figure 7 illustrates the hotmaps of using and not using the multiband attention obtained from the query and key multiplication. Both Figure 7a and Figure 7b utilize eight individual heads. In Figure 7a, the eight heads adopt the same temporal decay, while in Figure 7b, each of the eight heads employs different temporal decay. It can be observed that different bands can capture attention within different time interval ranges, which enables a better adaptive fusion of features from different bands. Whether or not the multiband strategy is used, the attention model requires the multihead attention mechanism. Each head employing a different band will not increase computational overhead, thus leaving the computational complexity unchanged. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Metric** & **256** & **512** & **1024** & **2048** \\ \hline \multirow{2}{*}{Assist09} & RMSE & 40.91 & 40.37 & 40.25 & 40.32 \\  & AUC & 81.33 & 81.64 & 81.82 & 81.76 \\ \hline \multirow{2}{*}{Assist17} & RMSE & 37.92 & 37.68 & 37.36 & 37.44 \\  & AUC & 84.91 & 85.32 & 85.75 & 85.61 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The performance comparison of the proposed model with 256, 512, 1024, and 2048 codebook sizes on the Assist09 and Assist17 datasets. Figure 6: The distribution of a knowledge concept’s questions in the embedding space: (**a**) The questions of a knowledge concept in the Rasch model. (**b**) The questions of a knowledge concept in the QQE model. Figure 8 shows the distribution of the knowledge concepts and questions on the Assist09 dataset in the embedding space, which are mapped to a two-dimensional space using t-SNE. Each dot represents a question, and each plus sign represents a concept, with different colors distinguishing the questions belonging to different concepts. Figure 8a-d shows the distribution of 10 knowledge concepts, 20 knowledge concepts, 50 knowledge concepts, and 100 knowledge concepts that were randomly selected, as well as their associated questions, respectively. It indicates that our model has successfully extracted meaningful, discriminative embedding features from the raw one-hot encoding vectors. These features capture essential differences between different knowledge concepts such that the questions containing the same knowledge concept are clustered together in the low-dimensional embedding space, while distinct knowledge concepts maintain appreciable distances from one another. Even in the reduced-dimensional view, the model is able to effectively identify and distinguish among knowledge concepts. The good separation of the questions and knowledge concepts also indicates that the model has learned embeddings that preserve the inherent knowledge structure of the data.