# Mining on Students' Execution Logs and Repairing Compilation Errors Based on Deep Learning
## 3 Mining on Students' Execution Logs
The calculation of dynamic routing can be described by the following formula:

\[\mathrm{c}_{ij}=\mathrm{softmax}(b_{ij}) \tag{3}\]

\[\hat{\mathbf{n}}_{j|i}=\mathbf{W}_{ij}\mathbf{\mu}_{i} \tag{4}\]

\[\mathbf{s}_{j}=\sum_{i}c_{ij}\hat{\mathbf{n}}_{j|i} \tag{5}\]

Figure 2:0

Figure 2:1

where \(\mathrm{c}_{ij}\) represents the coupling coefficients, indicating the routing probabilities from the lower-order capsules to the higher-order capsules, and \(\hat{\mathbf{n}}_{j|i}\) represents the output vector of the i-th capsule in the previous layer, which is obtained by multiplying the input vector \(\mathbf{u}_{i}\) with the corresponding transformation matrix \(\mathbf{W}_{ij}\). \(\mathbf{s}_{j}\) represents the weighted sum of all the output vectors from the previous layer's capsules to the current layer, and \(\mathbf{v}_{j}\) represents the output vector of the capsule. The function \(f\), also known as the squash function, compresses the element values of the corresponding vectors into the range of 0 to 1. After the aforementioned calculations, the output vector \(\mathbf{h}_{v}\) is obtained. The dimension of \(\mathbf{h}_{v}\) is N \(\times\) capsules' dimension, where N represents the number of capsules. We ensure that the dimension of the capsule network matches the dimension of the input hidden state to facilitate its merging with other semantic information. Due to the limited correlation between different batch samples, we have added a regularization layer with LayerNorm [25] after the capsule network layer. This layer only normalizes the dimensions within each sentence, aiming to reduce overfitting and accelerate the convergence of the model. We employed an attention mechanism to enhance the propagation of higher-order feature information within the context, resulting in the generation of the final hidden states \(\mathbf{g}_{x_{i}}{}^{final}\) and \(\mathbf{g}_{m}{}^{final}\). The purpose of the attention calculation is to allocate importance to different capsule output vectors based on their relevance within the input, thereby enhancing the fusion of higher-order feature information and generating the final hidden state. The following steps illustrate the calculation process of attention using the hidden state \(\mathbf{g}_{x_{i}}\)\({}^{final}\) as an example. Similarly, the attention calculation process for \(\mathbf{g}_{m}\)\({}^{final}\) is derived following the same procedure. Figure 2:2

Figure 2:3

Figure 2:4

Figure 2:5

After updates from the graph attention and capsule network layers, the information is propagated in the respective local contexts using two types of BiLSTM models. One type of model, referred to as BiLSTM\({}^{(2)}_{code}\), is used for the code statement sequences, while the other, BiLSTM\({}^{(2)}_{msg}\), is applied to the error description sequences. As a result, the final hidden state \(\mathbf{r}_{i}\) is obtained for each line \(i\). Figure 2:6

The final hidden states are fused using the BiLSTM\({}^{(3)}_{code}\) to obtain a merged embedding sequence \(\mathbf{o}_{1:L}\) that encompasses embeddings from different lines. Figure 2:7

#### 5.2.4 Decoding Layer

Given the hidden states \(\mathbf{o}_{1:L}\) obtained from the previous layer, we modeled the probabilities of each line \(k\in\{1,2,\dots,L\}\) being an erroneous line using a feedforward network. Additionally, we employed a pointer-generator network [26] as a decoder to model the probability distribution of the repair sequence \(y_{k}\) for the erroneous lines, aiming to address the issue of numerous out-of-vocabulary words caused by different variable names. The formulas are as follows:

Figure 2:8

Figure 2:9

During the final model prediction, the model selects the erroneous line index \(k\) with the highest probability as the error localization result. To generate potential repair patches, beam search is used to output sequences \(y_{k}\) with higher joint probability density. These sequences serve as candidate patches for the repair prediction.