# Knowledge Tracing Model and Student Profile Based on Clustering-Neural-Network
## 2 Frame of C-B-KT
### Model Training  
For all combined data, we use a Bert\(-\)KT model (B\(-\)KT) with the following structure (Figure 3) to conduct training separately.  
Question Embedding. Text information \(e_{t}\) automatically learns the semantic representation code \(x_{t}\) of each question, that is, question embedding. Each word \(w_{t_{j}}\) in the question \(e_{t_{j}}\) is converted into a \(d_{0}\)-dimensional word vector \(x_{t_{j}}\), using the weighted average to get the question vector. The equation is  
\[x_{t}=\frac{1}{m}\sum_{j=1}^{m}x_{t_{j}} \tag{1}\]  
where \(x_{t_{j}}\in R^{d_{0}}\) represents the word, \(w_{t_{j}}\) is the word vector obtained after Bert processing, \(m\) represents the number of words in question \(e_{t}\), and \(x_{t}\) represents the question vector.  
Student embedding is the modeling of the different training processes of different students and traces students' hidden knowledge state in different training processes, that is,  
Figure 3: There are two main parts to the framework process: the question embedding (gray) and the student embedding (blue). \(S_{t}\) is an improvement of the KT model, which represents the state of students’ knowledge and increases the interpretability.  
Figure 2: The dimensions of the two groups of data are too large, and Principal Component Analysis (PCA) is used to compress them into two dimensions, namely PC1 and PC2. (**a**) shows the clustering results of the students. (**b**) shows the clustering results of the question. Three colors represent three clusters, and different clusters are combined for training.  
student embedding. The KT framework relies on two basic assumptions: (1) the student's knowledge state is related to the question information and answer results; (2) students usually learn and forget their long-term continuous questions. Based on the above assumptions, this paper uses the LSTM model to embed students. This is the equation for LSTM:  
\[\begin{split} f_{t}&=\sigma(W_{f}\cdot[h_{t-1},x_{t}] +b_{f})\\ i_{t}&=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})\\ \tilde{C_{t}}&=tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C} )\\ o_{t}&=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \end{split} \tag{2}\]  
where \(W_{*}\in R^{d_{h}}\), which is the weight matrix between the input layer and the hidden layer; \(b_{*}\in R^{d_{h}}\), which is the deviation vector between the input layer and the hidden layer; and \(d_{h}\) is the input dimension obtained in the question embedding.  
In the process of training, correct or incorrect answers to questions will affect students' status in different ways, and it is necessary to distinguish their different effects on specific students. Specifically, when training on the t-th question in the sequence, the input of the model should be embedded with question \(x_{t}\) and the corresponding score \(r_{t}\). First, the score \(r_{t}\) is extended to the question embedding \(x_{t}\). With the same dimension \(R^{d_{0}+d_{g}}\), the feature vector \(\mathbf{0}=(0,0,\ldots,0)\) is obtained, and then the score vector and the question vector are combined to get the input vector \(x_{t}^{+}\), as follows:  
\[x_{t}^{+}=\begin{cases}[x_{t}\oplus\mathbf{0}]&r_{t}>0\\ \mathbf{0}\oplus x_{t}&r_{t}<0\end{cases} \tag{3}\]  
The neurons of the model are divided into upper and lower parts (half of the value of the input vector is 0), where \(W_{*}\) can be divided into \(W_{*}^{+},W_{*}^{-}\in R^{2d_{h}}\). This can be understood as using two models to capture the impact of the question \(e_{t}\) and reflect it in real-time during the training process.