# SFBKT: A Synthetically Forgetting Behavior Method for Knowledge Tracing
## 3 Proposed Method
### Model Architecture
#### 3.2.2 Using CTLSTM to Obtain Student Knowledge Status  
In continuous-time LSTM, the knowledge state vector is dependent on the memory unit vector \(c_{t}\) as the lstm network. The key difference is that each memory unit \(c\) decays exponentially at a rate \(\delta\) towards a steady-state value \(\overline{c}\) in subsequent time intervals after the event. Formula (5) depicts how the hidden states \(h_{t}\) are continually obtained from the memory cells \(c_{t}\) as the cells decay, where \(O_{i}\) is the output gate following the \(\mathrm{i}-th\) event. \(h_{t}\) is analogous to \(h_{i}\) in an LSTM language model [35], summarizing the past knowledge state changes caused by learning sequences \(\{k_{0},k_{1},\cdots,k_{i-1}\}\). Afterwards, the model predicts the probabilities of answering correctly for all skills, based on the hidden states \(h(t_{i+1})\). The resulting output \(\hat{Y}(t_{i+1})\) is a vector of equal length to the number of questions. Each entry in \(\hat{Y}(t_{i+1})\) represents the predicted probability that the student answered a specific question correctly. Therefore, the prediction \(\hat{y}_{i+1}\) for question \(q_{i+1}\) can be obtained from the entry corresponding to \(q_{i+1}\) in \(\hat{Y}(t_{i+1})\). The above can be summarized as follows:  
\[h(t)=O_{i}\odot(2\sigma(2c(t))-1)\ for\ te(t_{i},t_{i+1}] \tag{5}\]  
\[\hat{Y}(t_{i+1})=\sigma\big{(}W^{out}h(t_{i+1})+b^{out}\big{)} \tag{6}\]  
In our model, \(h_{t}\) also reflects memory decay caused by the time interval \(t_{i}-t_{i-1}\) between the current event and the last event. This interval \((t_{i-1},t_{i}]\) ends when the next event \(k_{i}\) occurs stochastically at some time \(t_{i}\). At this point, the continuous-time LSTM reads \(\left(k_{i},t_{i}\right)\) and updates the current (decayed) hidden cells \(c_{(t)}\) to new initial values \(c_{i+1}\), based on the current (decayed) hidden state \(h(t_{i})\). When information is input at time \(t_{i}\), the update formula of the model is similar to traditional LSTM. The overall process is illustrated in Formulas (7)-(13). \[i_{i+1}\leftarrow\sigma(W_{i}x_{i}+U_{i}h(t_{i})+d_{i}) \tag{7}\]  
\[f_{i+1}\leftarrow\sigma\Big{(}W_{f}x_{i}+U_{f}h(t_{i})+d_{f}\Big{)} \tag{8}\]  
\[z_{i+1}\gets 2\sigma(W_{z}x_{i}+U_{z}h(t_{i})+d_{z})-1 \tag{9}\]  
\[o_{i+1}\leftarrow\sigma(W_{\omega}x_{i}+U_{o}h(t_{i})+d_{o}) \tag{10}\]  
\[c_{i+1}\gets f_{i+1}\odot c(t_{i})+i_{i+1}\odot z_{i+1} \tag{11}\]  
\[\overset{-}{c}_{i+1}\leftarrow\overset{-}{f}_{i+1}\odot\overset{-}{c}_{i}+ \overset{-}{i}_{i+1}\odot z_{i+1} \tag{12}\]  
\[\delta_{i+1}\gets f(W_{d}x_{i}+U_{d}h(t_{i})+d_{d}) \tag{13}\]  
The vector \(x_{i}\) is derived from Formulas (2)-(4), which represents the new exercise item information combined with the forgotten factor in the input module. The symbol \(\sigma\) is the logistic sigmoid function, and \(i\), \(f\), \(o\), \(c\) are respectively the input gate, forget gate, output gate and cell activation vectors. The upright-font subscripts \(i\), \(f\), \(z\), \(c\) and \(o\) are not variables but constant labels that distinguish different \(W\), \(U\) and \(d\) tensors. The \(\overset{-}{f}\) and \(\overset{-}{i}\) in Formula (12) are defined analogously to \(f\) and \(i\) but with different weights. The key difference between this method and the traditional LSTM is that the update depends not only on the knowledge state at \(t_{i-1}\) time, but also on the time interval between this time and the last practice, which affects the update of the knowledge state. As time progresses, the memory cell vector changes according to Formula (14). Specifically, the memory cell vector at the moment when a new practice occurs continues to decay deterministically from \(c_{i+1}\) to the target \(\overset{-}{c}_{i+1}\) at different rates.