# Mining on Students' Execution Logs and Repairing Compilation Errors Based on Deep Learning
## 3 Mining on Students' Execution Logs
Additionally, given the compiler error description sequence \(M_{err}=(M_{1},M_{2},\ldots)\), we encoded it using another BiLSTM network, denoted as BiLSTM\({}^{(1)}_{msg}\), which outputs the corresponding hidden state \(\mathbf{h}_{M_{i}}\). Based on the error line number \(i_{err}\) indicated by the compiler, we calculated the position offset \(\Delta i=i_{err}-i\) for each code line relative to this error line. To incorporate this position offset information into the source code sequence, we used position embedding. Each position offset \(\Delta i\) was concatenated with the corresponding hidden state \(\mathbf{h}\) of the code sequence, resulting in the final representation \(\mathbf{h}_{x_{ij}}\) for each line of code. Indeed, after encoding the code sequence with the position embedding and incorporating the position offset information, the resulting code context vectors contained the relevant information associated with the error position. Additionally, the compilation error description information had been encoded as well, enabling the subsequent integration of corresponding semantic information. This encoding process facilitated capturing the contextual relationships between the code lines and the error position, as well as the semantic representations of the error descriptions, ultimately aiding in the subsequent steps of the model. #### 5.2.2 Graph Attention Layer

To enable the model to learn the correspondence between compiler error descriptions and the source code, as well as to track key symbols within the source code sequence and maintain relevant syntactic dependencies, a graph attention network [24] was used. This network allows the associated information contained within the constructed program-feedback graph to be propagated within the context, enabling the model to simulate the process of symbol tracking for program repair. Based on the adjacency relationships among symbols in the program-feedback graph, the weight calculation for the multi-head attention of each adjacent symbol to a given symbol is conducted, and the state information is updated accordingly. Each layer is computed using the following formula:

\[\mathbf{\varsigma}^{n}=\text{Attention}_{G}(\mathbf{\mathsf{h}}^{n-1}) \tag{1}\]

Figure 4: Model architecture of ACNPR. \[\mathbf{h}^{n}=\mathrm{MLP}\Big{(}\Big{[}\mathbf{h}^{n-1};\mathbf{c}^{n}\Big{]} \Big{)} \tag{2}\]

where \(\mathbf{h}^{n-1}\), \(\mathbf{h}^{n}\) denote the input/output representation of each token at the n-th layer, including the code state information \(\mathbf{h}_{x_{ij}}\) and the error description information \(\mathbf{h}_{M_{l}}\). \(\mathrm{Attention}_{\mathrm{G}}(\mathbf{h}_{i})\) is used to calculate the attention weights of symbol \(t\) on the programmable graph \(G\) with its neighboring nodes and then takes their weighted average. A gating mechanism is incorporated in the graph attention layer to replace the activation function, preserving the scale of each dimension and enhancing non-linear capabilities. MLP refers to a feedforward network used to integrate the attention weights into the context vector, resulting in the final output state information represented as \(\mathbf{g}_{x_{i}}\) and \(\mathbf{g}_{m}\). #### 5.2.3 Capsule Network Layer

After updating through the graph attention layer, the hidden state includes the semantic features of the input code sequence and the compiler feedback information from the updated program-feedback graph. The dimensions of these vectors are batch size \(\times\) sequence length \(\times\) hidden state dimension. The capsule network is used to detect the internal states of the input code sequence features, encapsulating them in vector form and resulting in output vectors representing the corresponding probabilities and characteristics of higher-level features. The internal weights of the capsules are learned using dynamic routing, where the weights determine how vectors from the lower layer will enter the higher-level vectors.