# Calibrated Q-Matrix-Enhanced Deep Knowledge Tracing with Relational Attention Mechanism
## 4 Methods
### Relation Attention Knowledge Tracing Model
#### 4.1.2 Attention Layer  
This layer consists of two sublayers, including the positional attention layer and the relational attention layer, and combines the relation modeling coefficient, R, to generate the final attention weights:  
* **Positional attention layer**. The positional attention layer can be applied to consider the relative position of each word in an exercise in order to obtain a higher interpretation ability than the absolute position in the traditional attention mechanism. Specifically, positional attention has h attention heads, and each attention head has two input sequences \(X_{i}^{s}\) and \(X_{j}^{s}=(x_{1},x_{2}..x_{n})\), where \(x_{i}\in\mathbb{R}^{d}\). The edges between \(X_{i}^{s}\) and \(X_{j}^{s}\) are represented by vectors \(a_{i,j}^{v},a_{i,j}^{K}\in\mathbb{R}^{d}\). In this positional attention layer, the maximum relative position of the input sequences is clipped to a value k. This process enables the model to generalize to sequence lengths that are not seen during the training process. \(a_{ij}^{v}\) and \(a_{ij}^{K}\) are calculated based on the maximum distance clipping using the following formula, and the relative position representations are \(P^{K}=(P_{-k}^{k}...P_{k}^{K})\) and \(P^{V}=(P_{-k}^{V}...P_{k}^{V})\). \[a_{ij}^{k} =P_{clip}^{k}(j-i,k)\] \[clip(pos,k) =max(-k,min(k,x))\] Then, \(e_{i,j}\) is calculated based on a compatibility function that compares two input elements, \(x_{i}\) and \(x_{j}\). \(W^{Q}\) and \(W^{K}\) are the parameter matrices. \(d_{z}\) is the dimension of the new sequence of Z. \[e_{i,j}=\frac{x_{i}W^{Q}(x_{j}W^{k})^{T}+x_{i}W^{Q}(a_{i,j}^{K})^{T}}{\sqrt{d_ {z}}}\] (7) Next, a weight coefficient \(a_{i,j}\) is calculated based on \(e_{i,j}\) by applying the softmax function. \[a_{i,j}=\frac{exp(e_{i,j})}{\sum_{i=1}^{n}exp(e_{i,k})}\] (8) Lastly, the final output of the positional attention layer, \(Z_{i}\), is computed as follows: \[Z_{i}=\sum_{j=1}^{n}a_{ij}(x_{j}W^{V})\] (9)
* **Relational attention layer**: In this layer, the output of the positional attention layer using Formula (9) is used as the input, and the relation coefficient R is incorporated into the relation attention mechanism, which pays more attention to some relevant  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & \multicolumn{4}{c}{**exercise i**} \\ \hline  & & F & T & total \\
**exercise j** & F & n\({}_{00}\) & n\({}_{01}\) & n\({}_{0*}\) \\  & T & n\({}_{10}\) & n\({}_{11}\) & n\({}_{1*}\) \\ \hline  & total & n\({}_{*0}\) & n\({}_{*1}\) & n \\ \hline \hline \end{tabular}
\end{table}
Table 2: The contingency table for exercise \(i\) and exercise \(j\). This table is used to calculate the Phi coefficient between two exercises. In this contingency table, “F” means the student answers the exercise incorrectly and “T” means the student answers the exercise correctly.  
interactions of students. \(\alpha\) is the attention weight of this relation attention mechanism using the following formula.  
\[\alpha_{i}=\frac{exp(e_{i})}{\sum_{k=0}^{n-1}exp(e_{k})}\quad e_{i}=\frac{E_{e_{n }}W^{Q}(Z_{j}W^{K})^{T}}{\sqrt{d}} \tag{10}\]  
where \(E_{e_{n}}\) represents the word embedding of exercise e\({}_{n}\), \(Z_{j}\) represents the input element of this layer using Formula (9), and \(W^{Q}\in\mathbb{R}^{d\times d}\) and \(W^{K}\in\mathbb{R}^{d\times d}\) are thequery matrix and key matrix for this relational attention layer. Next, the traditional attention weight \(\alpha_{i}\) and relation coefficient weight R is incorporated as the relational attention weight \(\gamma_{i}\):  
\[\gamma_{i}=\delta\alpha_{i}+(1-\delta)R_{i} \tag{11}\]  
Finally, the output of this layer \(O\in\mathbb{R}^{d}\) is obtained by the following formula:  
\[O=\sum_{i=1}^{n-1}\gamma_{i}Z_{i}W^{\mathrm{p}} \tag{12}\]  
where \(W^{V}\) is the value matrix of the attention mechanism.