## 2 Related Works
### Attentive Knowledge Tracing  
The transformer model was initially introduced for neural machine translation [25]. Unlike traditional deep learning models that rely on recurrence, the transformer model completely relies on the self-attention mechanism to capture global dependencies within a sequence. This approach has demonstrated exceptional abilities in feature extraction and dependency capture while maintaining high computational efficiency. Several noteworthy pretraining models based on the transformer architecture, such as BERT [29] and GPT [30], have achieved state-of-the-art performance on various natural language processing tasks.  
[PERSON] et al. [12] introduced a self-attentive model called SAKT (self-attentive knowledge tracing), which incorporates the transformer architecture to effectively capture the long-term dependencies in student learning interactions. This application of the transformer model directly contributes to achieving impressive performance in knowledge tracing. [PERSON] et al. [31] introduced an adaptive sparse self-attention network, which not only generates the missing features but also provides detailed predictions of student performance. [PERSON] et al. [32] identified a vibration issue in DKT and proposed an attention-based KT model to address it. They also incorporated the use of finite state automaton for a comprehensive understanding of knowledge state transitions. [PERSON] et al. [33] proposed an approach called separated self-attentive neural knowledge tracing (SAINT) to enhance the self-attentive computation for facilitating knowledge tracing adaptation. The SAINT model utilizes an encoder-decoder structure, where the embeddings of exercises and answers are independently encoded and decoded using self-attention layers. [PERSON] et al. [34] introduced the SAINT+ model, which integrates two temporal features into SAINT. These features include the response time for each exercise and the duration between consecutive learning interactions. [PERSON] et al. [13] proposed a context-aware attentive knowledge tracing (AKT) model that combines the self-attention mechanism with psychometric models. The unique aspect of AKT is its implementation of a novel monotonic attention mechanism, which operates under the assumptions that the learning process is influenced by forgetting and that student knowledge tends to diminish over time. [PERSON] et al. [14] introduced a contrastive learning framework for knowledge tracing (CL4T) that focuses on revealing semantically similar or dissimilar examples of learning history to better understand their relationships.  
Compared with other knowledge tracing methods, the attentive knowledge tracing or transformer framework has shown promising results in terms of predicting the future answers of learners.  
Overall, knowledge tracing plays a crucial role in adaptive learning systems, offering insights into student progress and enabling personalized interventions. The transformer-based knowledge tracing framework, with its self-adaptive nature and enhanced modeling capabilities, has the potential to revolutionize the field of educational technology and improve student learning experiences.