# Calibrated Q-Matrix-Enhanced Deep Knowledge Tracing with Relational Attention Mechanism
## 4 Methods
### Results and Discussion
#### 5.2.1 Students Performance Predictions (RQ1)  
Table 5 presents the performance of all baseline models, the RAKT model, and the QRAKT model. According to Table 5, our QRAKT model outperformed all the baseline models with a noticeable gap. DKT+ performed better than DKT due to the fact that the inputs were reconstructed and making the student performance consistent in each timestamp on the two datasets. SAKT further improved the model performance to some extent on ASSIST2012, indicating the significance of the relevance between past interactions and the next exercise. DKVMN had high performance compared with the DKT+ model, the DKT model, and the SRAKT model on the Eedi dataset because this model can learn representations based on nonlinear d high performance compared with the DKT+ model, the DKT model, and the SRAKT model on the Eedi dataset because this model can learn representations based on nonlinear transformations. Compared with SAKT, EKT considers the knowledge concepts' information and exercise relation and obtained better results in terms of the AUC, ACC, and Stability rate (Sta). However, the EKT model ignored the student behaviors when tracking the knowledge state of students. The EKPT model considered the student behaviors including the learning behavior and the forgetting behavior and applied the traditional Q-matrix to model the relationship between exercises and knowledge concepts to improve the performance on student performance prediction. However, the EKPT model still has some drawbacks. The first is that the Q-matrix does not consider the hierarchical knowledge levels in knowledge concepts. The second is that two situations were ignored: the student may guess an answer to a question or provide the wrong answer for the questions because of his/her carelessness. In order to solve these problems, the RAKT model incorporates the relation modeling of exercises, the forgetting theory, and student behaviors with two types of attention mechanisms to improve the performance in terms of the AUC, ACC, and Stability rate (Sta).  
DKVMN had high performance compared with the DKT+ model, the DKT model, and the SRAKT model on the Eedi dataset because this model can learn representations based on nonlinear transformations. Furthermore, the relative position and relation modeling of exercises are used in RAKT to trace the knowledge state of students and obtain a better performance than all baseline models on the two datasets. This demonstrated that the relation modeling of exercises and forgetting theory and student behaviors also need to be considered in knowledge tracing. The extension of RAKT, QRAKT, had the best performance compared to the other models on the two educational datasets. The QRAKT model used the knowledge vector matrix, the Q-matrix, to generate the embedding of the raw exercise and considered different hierarchical levels of knowledge on the knowledge graph to validate the Q-matrix compared to RAKT. Therefore, the results of the QRAKT model were more interpretable and accurate than simply using the word2 vector method in the RAKT model.  
As shown in Figure 4, the different performance of these models on the two datasets was due to the ASSIST2012 dataset having more student response data than the Eedi dataset. Therefore, those models that are suitable for processing an extensive dataset will lead to better performance on the AUC and ACC on the ASSIST2012 dataset, such as the SAKT model. Those models that are appropriate for handling a small dataset will perform better on the Eedi dataset, such as the DKVMN model.