# Grammar Correction for Multiple Errors in Chinese Based on Prompt Templates
## 3 Methodology
### Error-Detection Layer
#### 3.1.1 MacBERT Layer  
In the process of detecting errors, we use MacBERT to convert the input text into vector representations. MacBERT is a pre-trained model optimized for Chinese language tasks. Through training on a large-scale Chinese corpus, it learns rich representations of the Chinese language. MacBERT is a variant of BERT, and its structure is similar to BERT, consisting of compositional embedding layers and multiple stacked transformer encoder layers. The embedding layer transforms the input text into a mathematical representation and includes token, positional, and segment embeddings. Token embedding converts each character of the input text into a fixed-dimensional word-embedding vector. Positional embedding captures the positional information within the input sequence, encoding the position of each word in the sequence. It is added after word embedding. Segment embedding is used to differentiate relationships between different sentences. Each word is marked as belonging to the first sentence (Segment A) or the second sentence (Segment B). The outputs of these three embedding layers are element-wise summed and passed as input to the transformer encoder layers. However, unlike BERT, MacBERT improves the masking strategy used during pre-training. It discards the strategy of masking tokens with the whole word [MASK] and, instead, uses a similar word replacement for masking. When a masked word is not similar, a random word is selected for replacement. In this strategy, 15% of the characters in the input sentence are masked, 80% are replaced with similar words, 10% are replaced with random words, and the remaining 10% are kept unchanged. As shown in Table 2, in the masking strategy of BERT, masking the original sentence would result in \") [MASK] [MASK]\" (The cat [MASK] an [MASK]). In the masking strategy of MacBERT, masking the original sentence would result in \")  
The first part is a sigmoid layer, which decides which values to update, and the second part is a tanh layer, which generates a new candidate value vector \(\widetilde{c}\) that could be added to the state. The old cell state \(c_{t-1}\) is then updated to obtain the new cell state \(c_{t}\). Finally, the output gate determines which values to output. Specifically, it first applies a sigmoid layer to determine the output portion and then processes the state with a tanh function. The two are multiplied to obtain the final hidden state \(h_{t}\). The calculation formulas for each part of an LSTM unit are as follows, with the formula for the forget gate shown in Equation (1):  
\[f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{1}\]  
In the equations, \(W_{f}\) represents the weight matrix for the forget gate, \(b_{f}\) is the bias term for the forget gate, \(x_{t}\) is the input at the current time step, and \(h_{t-1}\) is the previous hidden state. The input gate consists of the activation value \(i_{t}\) and the candidate value vector \(\widetilde{c}_{t}\), which are calculated as shown in Equations (2) and (3):  
\[i_{t}=\sigma(W_{t}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{2}\]  
\[\widetilde{c}_{t}=\tanh(W_{c}\cdot[h_{t-1},x_{t}]+b_{c}) \tag{3}\]  
In this case, \(W_{t}\) represents the weight matrix for the input gate, \(b_{i}\) is the bias term for the input gate, \(W_{c}\) is the weight matrix for the candidate value vector, and \(b_{c}\) is the bias term for the candidate value vector. We can now obtain the updated cell state \(c_{t}\), which is calculated as shown in Equation (4):  
\[c_{t}=f_{t}*c_{t-1}+i_{t}*\widetilde{c}_{t} \tag{4}\]  
In the equations, \(c_{t-1}\) represents the previous cell state, \(f_{t}\) is the activation value of the forget gate, \(i_{t}\) is the activation value of the input gate, and \(\widetilde{c}_{t}\) is the candidate value vector. Finally, we obtain the hidden state \(h_{t}\) through the output gate, which is calculated as shown in Equations (5) and (6):  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{5}\]  
\[h_{t}=o_{t}*\tanh(c_{t}) \tag{6}\]  
Figure 2: LSTM Unit Architecture Diagram.