TemplateGEC: Improving Grammatical Error Correction with Detection Template
Year: 2023

Abstract:
Grammatical error correction (GEC) can be divided into sequence-to-edit (Seq2Edit) and sequence-to-sequence (Seq2Seq) frameworks, both of which have their pros and cons. To utilize the strengths and make up for the shortcomings of these frameworks, this paper proposes a novel method, TemplateGEC, which capitalizes on the capabilities of both Seq2Edit and Seq2Seq frameworks in error detection and correction respectively. TemplateGEC utilizes the detection labels from a Seq2Edit model, to construct the template as the input. A Seq2Seq model is employed to enforce consistency between the predictions of different templates by utilizing consistency learning. Experimental results on the Chinese NLPCC18, English BEA19 and CoNLL14 benchmarks show the effectiveness and robustness of TemplateGEC. Further analysis reveals the potential of our method in performing human-in-the-loop GEC. Source code and scripts are available at https: //github.com/li-aolong/TemplateGEC.

heading: 1 Introduction
text: Grammatical error correction (GEC) is a fundamental task in natural language processing that focuses on identifying and correcting grammatical errors in written text (Ng et al., 2013, 2014). The utilization of GEC is wide-ranging, including but not limited to, improving the quality of machine translation (Popović, 2018), increasing the readability of text (Liao et al., 2020), and assisting non-native speakers in language proficiency (Knill et al., 2019). There has been a significant amount of research in the field of GEC (Yuan and Briscoe, 2016; Bryant et al., 2017a; Ren et al., 2018; Zhou et al., 2018; Awasthi et al., 2019; Lai et al., 2022; Gong et al., 2022; Zhang et al., 2022b), which can be broadly
∗ Work done when Yinghao Li was a remote intern at Harbin Institute of Technology, Shenzhen.

heading: He prefer study in home.

heading: He prefers to study at home.
text: classified into two categories: Sequence-to-Edit (Seq2Edit) and Sequence-to-Sequence (Seq2Seq).
Seq2Edit GEC typically involves converting a source sentence into a sequence of editing operations (Stahlberg and Kumar, 2020; Omelianchuk et al., 2020). Despite certain limitations, such as the manual selection of edits and the use of a dictionary (Awasthi et al., 2019; Malmi et al., 2019), Seq2Edit GEC have specific advantages for grammatical error detection due to its high understanding ability (Omelianchuk et al., 2020). Seq2Seq GEC, on the other hand, which approaches GEC as a monolingual translation problem (Ge et al., 2018; Sun et al., 2021), has the advantage of better generation ability of the corrected sentence. However, Seq2Seq GEC still encounters the challenge of over-correction (Park et al., 2020).
In this paper, we propose a novel approach, named TemplateGEC, to merge both frameworks and leverage their respective strengths for grammatical error detection and correction. The proposed approach, as illustrated in Figure 1, utilizes a source template to introduce the detection label from Seq2Edit GEC to Seq2Seq GEC. This enables the Seq2Seq GEC model to make more accurate predictions with the assistance of the detection label. However, the predicted labels from Seq2Edit models may not always be accurate and may contain errors. To enhance the robustness of the model
6878
to these inaccurately predicted labels, we propose incorporating gold labels through consistency learning. Experimental results on widely-used English and Chinese GEC benchmarks show the effectiveness and robustness of TemplateGEC. Additional analysis reveals its high potential for human-in-theloop GEC through the proposed detection template. Our main contributions are as follows:
• We explore the integration of Seq2Edit and Seq2Seq GEC, by leveraging their respective strengths of understanding and generation.
• We propose a detection template to introduce detection information from Seq2Edit GEC to Seq2Seq GEC, which allows the model to make more accurate predictions.
• We introduce a gold label-assisted consistency learning method to enhance the robustness of the model to inaccurately predicted labels.
• Our proposed method shows high upper bounds of utilizing gold labels, which has the potential to inspire new research in the area of human-in-the-loop GEC.

heading: 2 Related Work

heading: 2.1 Sequence-to-Edit GEC
text: Seq2Edit GEC generally predicts the operation for each token in a sentence, such as insertion, deletion, etc. LaserTagger (Malmi et al., 2019) transforms a source text into a sequence of token-level edit operations, which consist of keeping, deleting, adding and swapping. PIE (Awasthi et al., 2019) reduces the local sequence editing problem to a sequence labeling setup and utilizes BERT to non-autoregressively label input tokens with edits. Stahlberg and Kumar (2020) propose a sequence editing model named Seq2Edits, in which the prediction target is a sequence of edit operations applied to the source. GECToR (Omelianchuk et al., 2020) introduces custom g-transformations in addition to the conventional edit operations, such as capitalization change, merging of two tokens, changing word suffixes and so on. A limitation of Seq2Edit is that it heavily relies on the manual construction of editing operations. This dependence on manual curation renders the model less transferable and results in a lower degree of fluency in the output (Li et al., 2022b). Conversely, its strength is demonstrated in its ability to effectively perform
error detection (Yuan et al., 2021), which is facilitated by the accurate prediction of each input category, as opposed to a focus on text fluency.

heading: 2.2 Sequence-to-Sequence GEC
text: Seq2Seq GEC encodes the erroneous sentence through the encoder and uses the decoder to generate each error-free token, which has been well explored (Liu et al., 2021; Wang et al., 2021; Li et al., 2022a; Fang et al., 2023a). The seq2Seq model is more suitable to generate fluent sentences while the decoding speed of it is slower. Zhao et al. (2019) employ a copy-augmented framework and copy unchanged tokens from the sentence pair to the target sentence. Kaneko et al. (2020) explore how to effectively incorporate pre-trained knowledge into the encoder-decoder framework. Qorib et al. (2022) propose a simple logistic regression method to combine GEC models much more effectively. It is noted that constructing pseudo datasets is most useful on GEC task, as noise can be easily injected into error-free sentences automatically, and receive large pseudo sentence pairs which can be used to pre-train GEC models (Zhao et al., 2019; Zhou et al., 2020; Lichtarge et al., 2019; Kiyono et al., 2020; Yasunaga et al., 2021; Sun et al., 2022; Fang et al., 2023b).
Previous works have preliminary attempted to incorporate detection label knowledge into GEC models in order to improve correction results. Chen et al. (2020) use error spans and source sentences as input and output correct spans. Yuan et al. (2021) take detection labels as auxiliary input and using for re-ranking. In our work, we propose a simple and effective way to exploit detection information, providing a nice alternative for this line of research.

heading: 3 TemplateGEC
text: This section introduces the proposed method as illustrated in Figure 2. TemplateGEC integrates detection information generated by a Seq2Edit model and fuses the information into a Seq2Seq model for model enhancement.

heading: 3.1 Definition of Error Detection Label
text: To incorporate the detection information, we first acquire the error label for a given input sentence. This label is then utilized to identify the specific words or phrases in the sentence that contain grammatical errors. Given the source input sentence x = x1, x2, ..., xN , the error detection label of the
sentence can be formulated as:
l = l1, l2, ..., lN , ln ∈ {C, I}, (1)
where C denotes correct and I denotes incorrect. As shown in the left part of Figure 2, the source sentence is transformed into two detection labels: a predicted label lp and a gold label lg.
Predicted Label The predicted label represents the predicted error positions in a sentence obtained from a detection model. Due to its improved understanding capabilities, the Seq2Edit architecture is used to train a language model with a fullyconnected layer as the output layer, which classifies the source tokens as correct or incorrect. We utilize the detection component of the Seq2Edit model to get the predicted label lp, which might contain errors. As shown in Figure 2, there are two detection errors marked as red in the predicted label lp.
Gold Label The gold label, which will be utilized by our model, indicates the true location of errors in a sentence. Based on the parallel source and target pairs, we use ERRANT (Bryant et al., 2017b) to extract the edits, from which we can obtain the gold label lg of the source sentence.

heading: 3.2 Detection Template Construction
text: We introduce the detection template which incorporates detection information by transforming the input sentence in a specific manner. This template is constructed by concatenating a detection prefix with a modified version of the input sentence, utilizing a specialized token <sep> as a delimiter. The detection template t can be formulated as:
t = d <sep> x′, (2)
where d and x′ represent the detection prefix and modified source, respectively. The detection tem-
plate t is utilized as input for the Seq2Seq model, instead of the original source, as shown in Figure 2.
Detection Prefix The detection prefix is made up by concatenating the error parts and corresponding ordered special tokens. Error parts represent the continuous tokens that are labeled as I. A source sentence may contain multiple error parts, each comprising a varying number of words. As shown in Figure 2, there are two error parts annotated by the predicted label that are “answer” and “the”, while there is only one error part “answer at” annotated by the gold label, due to the continuous label of I. We extract all the error parts E = E1, E2, ..., EI of the source sentence according to the detection labels, then we use d to represent the detection prefix, which can be given by:
d = S1E1S2E2...SIEI , (3)
where Si is the i-th ordered special token. As shown in Figure 2, the detection prefix dp is made up of two error parts and their corresponding special tokens <S1> and <S2> and so dg is.
Detection Template All the error parts E with the number of I can divide the source sentence x into I + 1 correct parts, which can be given by:
x = X0E1X1...XI−1EIXI , (4)
where Xi denotes the i-th correct part of x. Taking the predicted error parts Ep illustrated in Figure 2 for example, two error parts (“answer” and “the”) divide the source sentence three parts (“It is difficult”, “at” and “question”). Then the predicted modified source sentence x′p is obtained by replacing the error parts, present in the source sentence, with corresponding ordered special tokens (“<S1>” and “<S2>”). The modified source sentence is:
x′ = X0S1X1...XI−1SIXI . (5)
Once x′p is obtained, the predicted template tp is constructed by concatenating the detection prefix dp and modified source sentence x′p. So does tg.

heading: 3.3 Gold Label-Assisted Consistency learning
text: Motivation The proposed template incorporates detection information in the hope that the model can more accurately correct errors at the corresponding positions. However, when the detection information is incorrect, the model may make wrong modifications to correct words, resulting in a decrease of model performance. To overcome this problem and make the model more robust to the predicted error detection information, we propose using gold label-assisted consistency learning to help the model increase consistency in the output of the predicted detection template and gold detection template, thus improving the model performance.
Training Objective We adopt a Seq2Seq model based on the Transformer (Vaswani et al., 2017) architecture as the backbone of our method. As outlined in Section 3.1, we are able to obtain both predicted and gold detection labels for a given source sentence. The templates constructed by these types of labels, represented by tp and tg respectively, are then fed into the Seq2Seq model as shown in Figure 2. The two losses can be defined as:
Lp = − logP (y|tp;θ); Lg = − logP (y|tg;θ),
(6)
where θ is the set of parameters to optimize, y is the target sequence.
Consistency Learning Following Liang et al. (2021); Wang et al. (2022a), we introduce the consistency loss to our model, which maximizes the similarity of the output distributions with predicted and gold detection information. KL divergence is a measure of the difference between two probability distributions, which is a non-symmetric measure. We set KL divergence as our consistency loss to maximize the consistency between the distributions of the predictions for tp and tg, thus the consistency
loss Lc is defined as:
Lc = 1
2 [KL(P (y|tp;θ)||P (y|tg;θ)) +KL(P (y|tg;θ)||P (y|tp;θ))]. (7)
This final training loss is:
L = 1 2 (Lp + Lg) + βLc, (8)
where β is a hyper-parameter representing the coefficient of consistency loss.
Discussion The two cross-entropy loss items encourage the model to generate the corresponding targets for the templates tp and tg, which make the model learn how to construct the distributions of predicted and gold detection templates. Based on the two distributions, the consistency loss reduces the distance between the two distributions (Wang et al., 2022b; Li et al., 2022c; Liu et al., 2023). By enforcing consistency between predicted and gold labels, the model can learn more robust and reliable representations of the dataset, which can lead to improved performance for the GEC task. In the inference stage, only predicted detection labels are used to generate the template tp which is fed into the model, since the gold detection label is not available that is suitable for practical application.

heading: 4 Experiments

heading: 4.1 Error Label Detection
text: Setup For the English, to obtain the predicted labels, we train a Seq2Edit model based on ELECTRA (Clark et al., 2020) using the same configurations following Yuan et al. (2021). Additionally, for the comparison of detection performance, we treat GECToR (Omelianchuk et al., 2020) as the detection model. We use the available best-trained RoBERTa model of GECToR1 to infer the BEA19Dev set and CoNLL14-Test set for English, and NLPCC18-Test for Chinese. We use ERRANT to extract the predicted labels according to the hypotheses of GECToR. For Chinese, as there are
1https://github.com/grammarly/gector
no public detection results for NLPCC18-Test, we train the same Chinese Seq2Edit model as the English experiment to obtain the predicted labels. Besides, ERRANT is used to extract the gold detection labels from all the training data.
Comparasion Results As shown in Table 1, the results of ELECTRA we reproduced are lower than the other two models for English datasets. Based on the superior performance of the GECToR model on the BEA-Dev set and CoNLL14-Test 1 set, as well as its proximity to the best results of another dataset, we select GECToR as the detection model for obtaining the detection labels.
Error Label Preparation Based on the results, we use the open-sourced GECToR model to detect all the English data and our reproduced ELECTRA model to produce predicted labels for Chinese data. Gold labels are extracted by ERRANT.

heading: 4.2 Grammatical Error Correction
text: Dataset For the English, we use cLang-8 (Rothe et al., 2021) as training data, which is a clean version of the original Lang-8 corpus (Mizumoto et al., 2011; Tajiri et al., 2012). Following Yuan et al. (2021), we use BEA-Dev (Bryant et al., 2019) and BEA-Test as the development and test datasets, both of which consist of W&I (Yannakoudakis et al., 2018) and LOCNESS (Granger, 2014). Additionally, we include the CoNLL14Test set (Ng et al., 2014) in the test dataset. For the
Chinese, following Zhang et al. (2022b), we use NLPCC18-Train (Zhao et al., 2018) as the training set, MuCGEC-Dev (Zhang et al., 2022a) as the development set and NLPCC18-Test as the test set. Table 3 shows the statistics of the used datasets.
Model The models we use are based on Transformer (Vaswani et al., 2017) architecture. For English, Transformer-large, BART-large (Lewis et al., 2020) and T5-large (Raffel et al., 2020) are first used as our baseline models, which are finetuned with the original format of training data. For Chinese, Transformer-large and Chinese BARTlarge (Shao et al., 2021) models are used as the baseline in the same way. Due to the absence of a Chinese version of the T5 model, the experiments conducted in Chinese do not incorporate the use of the T5 model. Then we train the models mentioned above with only the predicted template for comparison rather than the original sources. At last, the proposed TemplateGEC is trained with the predicted and gold template described in 3.2, and consistency loss is applied in the training stage. The hyper-parameter β is set to 1 and other main hyper-parameters used in English and Chinese experiments are shown in Table 2. All experiments are run on a GeForce RTX 3090 GPU.
Evaluation Metrics For English experiments, following Yuan et al. (2021), we use ERRANT and M2 (Dahlmeier and Ng, 2012) to evaluate GEC models on BEA-Test set and CoNLL14-Test set, respectively. For Chinese experiments, following Zhang et al. (2022b), we use M2 as the metric on the NLPCC18-Test set. Precision, recall, and F0.5 values are reported for all the experiments.
Comparison with Previous Works Table 4 shows the main results for English and Chinese GEC tasks, which are compared with previous sin-
gle models. GECToR (Omelianchuk et al., 2020) treats GEC as a sequence labeling task and assigns the proposed operation labels to each token in the source sentence. Multi-encoder (Yuan et al., 2021) additionally employs an encoder to handle the detection input and uses a re-ranking strategy based on the detection outputs to improve the GEC performance. T5-large (Rothe et al., 2021) directly takes the original source sentence as input and generates the prediction outputs with T5-large. TypeDriven (Lai et al., 2022) proposes a TypeDriven Multi-Turn Corrections approach for GEC, which trains the model to exploit interdependence between different types of errors. SynGEC (Zhang et al., 2022b) adapts the dependency syntax into GEC models to improve performance.
Main Results As shown in Table 4, utilizing pretrained models results in a marked improvement in performance across all datasets, in comparison to models that are not pre-trained. Compared to the baseline, when the detection template is introduced, the performance of the majority of the models improves, particularly in models that are not pre-trained, while the improvement in pre-trained models is less significant. The results with weak or declining performance may be attributed to the possibility that the model is not effectively addressing errors present in the template. The proposed method TemplateGEC, utilizing both the template and consistency learning, achieves the best F0.5 values on all the datasets when compared to other methods. This indicates that the incorporation of
consistency learning allows the model to make more accurate corrections with the help of error labels provided in the template. The improvement of the proposed methods is primarily driven by an increase in precision, with some recall values experiencing a decline. This phenomenon is encouraged in GEC tasks since ignoring an error is not as bad as proposing a wrong correction (Ng et al., 2014).

heading: 5 Analysis

heading: 5.1 Potential of Human-in-the-loop GEC
text: Upper Bound Results In order to determine the performance upper bound of TemplateGEC, we initially evaluate its performance using gold detection labels during the testing stage. Subsequently, we conduct additional experiments where the gold labels are utilized both in the training and testing stages. As shown in Tabel 5, in contrast to the benchmark models and the label-based TemplateGEC utilizing predicted labels, utilizing gold labels in the TemplateGEC results in a marked improvement in performance, especially when pretrained knowledge is not fully introduced (i.e., the results of Transformer). This serves as evidence that the proposed template plays a significant role in impacting the performance of the GEC system. The upper bound results of both BART and T5 models exhibit a significant improvement and are relatively comparable. This suggests that by training and testing TemplateGEC with the correct error distribution, it is possible to achieve superior performance compared to the predicted error distribution.
Potential Direction In addition to the proposed TemplateGEC, we contend that our method has the potential for application in the development of a human-in-the-loop GEC system. We envision a scenario where a user inputs a sentence in need of correction or refinement, and our GEC model supports the identification of specific spans within the sentence that the user has identified as being in error or uncertain. Given this scenario, the TemplateGEC system can convert the identified error spans to the template format outlined in Section 3.2, resulting in the GEC model placing increased emphasis on these specific areas during the correction. The incorporation of user interaction in the TemplateGEC system allows for the utilization of user-annotated spans as the gold standard for error detection labels, resulting in improved error correction capabilities and increased efficiency as the need for a separate detection model is eliminated.

heading: 5.2 Ablation Study
text: In order to evaluate the effectiveness of the various components in TemplateGEC, we conduct multiple experimental evaluations using a variety of model configurations, testing them on both the BEA-Dev and BEA-Test datasets. In each experimental setting, we conduct evaluations, varying only one component while keeping the remaining constant.
Effect of Detection Model Given that the proposed template method incorporates the use of detection label knowledge, the performance of the TemplateGEC may be impacted by the performance of various detection models. In order to investigate the relationship between TemplateGEC and the detection model, we replace the GECToR model with the ELECTRA-based detection model described in Section 4.1. The results show that both detection models can produce reliable detection labels, indicating that the proposed method can accommodate various detection models.
Effect of Detection Class As stated in Yuan et al. (2021), the performance of GEC models incorporating detection labels is influenced by the choice of detection class. The results indicate that the F0.5 score of the 4-class detection is slightly lower than that of the 2-class detection on average. As the performance of the 4-class detection model is suboptimal, further research is required to explore methods to enhance the TemplateGEC system with more fine-grained class detection labels.
Effect of Detection Template To investigate the significance of designing appropriate detection templates, we create a simple template that concatenates the detection prefix with the original source
sentence without any reformatting of the source. The simple template ts is:
ts = d <sep> x. (9)
The results show that the use of the simple template results in a decrease in performance, highlighting the effectiveness and appropriateness of our method in incorporating detection labels.
Effect of Consistency Loss Type Various loss functions can be used to measure how different two probability distributions are from each other, to find the divergence by employing different loss functions, we change the KL divergence loss to the Mean Squared Error (MSE) loss between two output representations. The results show that when MSE loss is adopted as the consistency function, a certain degree of performance degradation will be observed, which indicates that KL divergence loss is more appropriate for enhancing the model performance in our method.
Effect of Coefficient of Consistency Loss To learn the influence of the coefficient of consistency loss, we test several different values of β. Default value of β = 1, and we test for β = 2 and β = 3. The results show that our default setting β = 1
can get the best F0.5 score. The result reveals that the consistency between the predicted and gold predictions is not always the higher the better.

heading: 5.3 Model Robustness
text: Error Detection Robustness As shown in Table 7, we explore the performance of different models under various sentence error numbers and the results. The baseline model achieves the best results when the error number is zero or more than three, while the template-only method shows a weak performance. It may be due to the unbalanced data distribution and the performance of the detection. Based on the results, TemplateGEC is still competitive in the two situations and outperforms the baseline in other situations, which is attributed to consistency learning. By introducing gold labels, the model is guided in the right direction even though the predicted labels may be wrong. It indicates that TemplateGEC is robust for different error numbers and performs better when there are few errors.
Error Type Robustness To explore if TemplateGEC can correct every error type well, the results of three error types, which are categorized as M (Missing), R (Replacement), and U (Unnecessary), are computed and shown in Table 9. Results show that compared with the baseline model, TemplateGEC gets the better F0.5 score for all error types, especially the replacement and unnecessary types, which demonstrates the robustness of TemplateGEC on the error type level.
Case Study Table 8 illustrates how TemplateGEC works better than the baseline model
in terms of model robustness. For the first example, the baseline model fails to correct the error. In contrast, despite the correction being incorrect, the template-only model attempt to correct the error indicated by the predicted label, which also confirms the effectiveness of the template. Based on the results of the template-only model, TemplateGEC successfully corrects the unnecessary type of error, corresponding to the ability of TemplateGEC reflected in Table 9. For the second example, the template-only model still modifies the corresponding positions indicated by the template, but one of them is wrongly corrected, which is misguided by the predicted label. However, we surprisingly observe that TemplateGEC ignores this misdirection and corrects the whole source sentence successfully. This result strongly suggests that our model can make correct corrections even when the prior information is wrong, which fully demonstrates the reliability and robustness of our method.

heading: 6 Conclusion
text: This paper presents a new method for GEC, called TemplateGEC, which integrates the Seq2Edit and Seq2Seq frameworks, leveraging their strengths in error detection and correction. TemplateGEC converts the original erroneous sentence into a novel template format that incorporates predicted and gold error detection labels, which are generated by a Seq2Edit model. Besides, TemplateGEC incorporates gold label-assisted consistency learning to enhance performance by maximizing the consistency between the predictions of the predicted and gold templates through the use of a Seq2Seq model. Experimental results on widely-used English and Chinese benchmarks show that TemplateGEC exhibits competitive performance in comparison to previous GEC methods. Additional analysis suggests that the proposed method is a promising approach for human-in-the-loop GEC and confirms that TemplateGEC is effective and robust. We will investigate the feasibility of adapting TemplateGEC to other languages and assess its effectiveness through additional experimentation in our future work.
Limitations
The primary limitation of the proposed model is computational efficiency. Specifically, during the training phase, the input size of the model is more than double that of traditional models, which is due to the inclusion of both predicted and gold
templates. Besides, the source sentences are transformed into longer sequences, resulting in an increased memory footprint and longer training time. Additionally, both during the training and testing phase, an additional step of preparing detection labels for the data further contributes to the increased processing time. In future research, we aim to investigate methods for achieving comparable or superior performance while reducing the input size and addressing these limitations, building upon the foundation of our current work. Additionally, TemplateGEC does not support the joint training of the Seq2Edit model. We will further explore how to jointly train the Seq2Edit model in future work, particularly focusing on the continuous modeling of detection labels based on an end-to-end model.
Ethics Statement
Our work aims to develop and evaluate algorithms that automatically detect and correct grammatical errors in written English and Chinese text. We use publicly available datasets for training and evaluation purposes. These datasets consist of anonymized and de-identified text samples, ensuring the privacy and confidentiality of the original authors. We are committed to conducting our research in an ethical and responsible manner.

heading: Acknowledgments
text: Heyan Huang is the corresponding author. This work was supported in part by the National Natural Science Foundation of China (Grant Nos. U21B2009, 62206076), the Science and Technology Development Fund, Macau SAR (Grant Nos. FDCT/060/2022/AFJ, FDCT/0070/2022/AMJ), Shenzhen College Stability Support Plan (Grant Nos. GXWD20220811173340003, GXWD20220817123150002), Shenzhen Science and Technology Program (Grant No. RCBS20221008093121053), CCF Fund for Young Scholars and the Multi-year Research Grant from the University of Macau (Grant No. MYRG2020-00054-FST).

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7
7 A2. Did you discuss any potential risks of your work? The data we used are publicly available and do not contain this issue.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1, 6
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The datas we used are public available and do not contain this issue.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. The data we used are publicly available and do not contain this issue.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. The data we used are publicly available and do not contain this issue.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. The data we used are publicly available and do not contain this issue.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? The data we used are publicly available and do not contain this issue.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: We will investigate the feasibility of adapting TemplateGEC to other languages and assess its effectiveness through additional experimentation in our future work. Limitations
The primary limitation of the proposed model is computational efficiency. Specifically, during the training phase, the input size of the model is more than double that of traditional models, which is due to the inclusion of both predicted and gold
templates. Besides, the source sentences are transformed into longer sequences, resulting in an increased memory footprint and longer training time. Additionally, both during the training and testing phase, an additional step of preparing detection labels for the data further contributes to the increased processing time. In future research, we aim to investigate methods for achieving comparable or superior performance while reducing the input size and addressing these limitations, building upon the foundation of our current work. Additionally, TemplateGEC does not support the joint training of the Seq2Edit model. We will further explore how to jointly train the Seq2Edit model in future work, particularly focusing on the continuous modeling of detection labels based on an end-to-end model