## 3 The Proposed Method
### Information Difficulty Extraction Module
#### 3.1.4 Shared Layer  
The shared layer consisted of a multi-head attention layer, a normalization layer, and a forward feedback layer. The input was obtained by summing each embedding vector with the position encoding \(\mathrm{P_{i}}\):  
\[\mathrm{O}=\widetilde{\mathsf{Q}}+\widetilde{\mathsf{S}}+\widetilde{\mathsf{ T}}+\widetilde{\mathsf{H}}+\mathrm{P_{i}} \tag{6}\]  
where \(\mathrm{O}\in R^{d}\).  
**Multi-head self-attentive layer**. In the self-attentive mechanism, \(\mathsf{Q}\) is usually used to denote the query, \(\mathsf{K}\) denotes the key, and \(\mathsf{V}\) denotes the value. The attention weights were calculated by first dotting \(\mathsf{Q}\) and \(\mathsf{K}\). After that, to prevent the result from being too large to affect the subsequent gradient propagation, the dotted product result was divided by \(\sqrt{d}\) and, finally, input to the SoftMax function for normalization. The attention weights are calculated as follows:  
\[\mathrm{Attention(Q,K,V)}=\mathrm{softmax}\left(\frac{\mathrm{QK}^{\mathsf{T} }}{\sqrt{\mathsf{d}}}\right)\mathsf{V} \tag{7}\]  
When we predicted the information related to question \(\mathsf{t}\), we should only consider the interaction information of the first \(\mathsf{t}-1\) questions, so we used the upper triangle mask to mask the information of the subsequent positions.  
\[\mathrm{head_{i}}=\mathrm{Softmax}\left(\mathrm{Mask}\left(\frac{\mathrm{Q_{i }K_{i}^{\mathsf{T}}}}{\sqrt{\mathsf{d}}}\right)\right)\mathrm{V_{i}} \tag{8}\]  
\[\begin{array}{rcl}Q_{i}&=&\left[q_{1}^{i},\cdots,q_{t}^{i}\right]=\mathbf{O} W_{i}^{Q}\\ K_{i}&=&\left[k_{1}^{i},\cdots,k_{t}^{i}\right]=\mathbf{O}W_{i}^{K}\\ V_{i}&=&\left[v_{1}^{i},\cdots,v_{t}^{i}\right]=\mathbf{O}W_{i}^{V}\end{array} \tag{9}\]The multi-head attention layer projected Q, K, and V into different potential spaces through different matrices \(W^{Q}\), \(W^{K}\), and \(W^{V}\) to capture rich feature information to form different heads and stitched these information heads together.  
\[\text{Multihead}(\text{Q,K,V})=\text{ Concat}(\text{head}_{1},\text{head}_{2}\ldots\text{head}_{i}W^{O}\text{ }) \tag{10}\]  
**Feed forward layer.** Since the output of the input vector after the multi-headed self-attentive layer was still a linear combination, we used a feed forward layer to convert the linear combination to a nonlinear combination. The feed forward layer consisted of two linear transformations activated by a ReLU function to add nonlinearity to the model.  
\[\text{FFN}(\text{x})=\text{ ReLU}(\text{x}W_{1}+b_{1})W_{2}+b_{2} \tag{11}\]  
where x = Multihead(Q, K, V), and \(W_{1}\), \(W_{2}\), \(b_{1}\), \(b_{2}\) are the weight matrices and bias vectors, which are continuously updated in training.  
**Residual connect.** He introduced the concept of residual connect in [26], where the primary role of residual linking was to propagate low-level features to higher levels, which made it easier for the model to utilize low-level information. Residual connection is a common deep-learning technique that builds deeper networks in a more stable manner.  
**Normalization layer.** The input information was normalized after passing through the multi-head attentive layer and the feed forward layer. The research of [PERSON] in [27] showed that a neural network could be more stable and fast by normalizing the input, and gradient disappearance and explosion could be prevented simultaneously.