## 4 Proposed Approach  
In the proposed model, an LSTM-AttM network is used as a seq2 seq language model for error assessment and detection as well as source code classification. We trained the proposed model using correct source code. The model then generated the error probability through the softmax layer for each error candidate word based on the context vector \(c_{t}\) of all previous hidden states and the current state output \(h_{t}\). The estimated error probability is also used to classify the source code as either correct  
Figure 1: Internal structure of a simple long short-term memory (LSTM) unit. (error-free) or incorrect. The proposed LSTM-AttM model can identify many kinds of error (logic, syntax, semantic, etc.) in source code to increase source code reliability. The workflow of the proposed model is shown in Figure 2.  
_Proposed LSTM-AttM Model Architecture_  
The attention mechanism has been adapted for performing various tasks [32; 33; 34; 35]. It is most commonly used in seq2 seq modeling. A neural network that utilizes the attention mechanism is called an attentive neural network. The conventional seq2 seq model cannot properly process a long sequence of input because only the last hidden state of the input is used as a context vector for output [36]. The attention mechanism maps the most relevant words from the input sequence and then assigns a higher weight to these words to enhance the output accuracy. We incorporated the attention mechanism with LSTM, as shown in Figure 3, to better predict short and long sequences of source code. The proposed LSTM-AttM model creates a potential application domain in programming education arena. Figure 3: Architecture of proposed LSTM-AttM network model. Figure 2: Workflow of proposed model: (**a**) word embedding and encoding process, (**b**) training of LSTM combined with the attention mechanism (LSTM-AttM) network using IDs, and (**c**) error probability prediction followed by the softmax layer and source code classification. Attention is a vector or dense output layer with a softmax function. It is used to enhance the performance of machine translators and seq2 seq models. Recently, the attention mechanism has achieved great success in machine translation tasks. A machine translator sometimes compresses long sequenced sentences into a fixed-length vector. Therefore, information may be lost. The attention mechanism mitigates this problem. Although LSTM has outstanding performance in terms of capturing long-range dependencies, a hidden state carries all the information into a fixed-length vector [36]. The attention mechanism has been applied to neural language models such as LSTM to overcome this problem [37]. The attention mechanism allows a neural language model to retrieve and make use of pertinent information in all previous hidden states, improving network retention. The mathematical details of the attention mechanism are described in previous work [38]. For attention, we use external memory \(M\) for previous hidden states, which is denoted as \(M_{t}=\left[h_{t-M}\dots\dots\dots\dots\ \ h_{t-1}\right]e\,\mathbb{R}^{kM}\). At time step \(t\), the context vector \(c_{t}\) and attention weight \(\alpha_{t}\). Now, the model uses the attention layer between \(h_{t}\) and the hidden states in \(M_{t}\). We defined our attention-based LSTM model by the following equations. \[A_{t}=M_{t}.h_{t} \tag{7}\]  
\[\alpha_{t}=softmax(A_{t}) \tag{8}\]  
\[c_{t}=M_{t}\alpha_{t}^{T} \tag{9}\]  
For predicting the next word at time step \(t\), the calculation is based on current hidden states \(h_{t}\) and context vector \(c_{t}\). The vocabulary spaces are obtained using the softmax function to produce the final probability \(y_{t}\epsilon\,\mathbb{R}^{p}\). \(G_{t}\) is an output vector.