FCGEC : Fine-Grained Corpus for C hinese Grammatical Error Correction (2022)

Abstract: Grammatical Error Correction (GEC) has been broadly applied in automatic correction and proofreading system recently. However, it is still immature in Chinese GEC due to limited high-quality data from native speakers in terms of category and scale. In this paper, we present FCGEC, a fine-grained corpus to detect, identify and correct the grammatical errors. FCGEC is a human-annotated corpus with multiple references, consisting of 41,340 sentences collected mainly from multi-choice questions in public school Chinese examinations. Furthermore, we propose a Switch-Tagger-Generator (STG) baseline model to correct the grammatical errors in low-resource settings. Compared to other GEC benchmark models, experimental results illustrate that STG outperforms them on our FCGEC. However, there exists a significant gap between benchmark models and humans that encourages future models to bridge it.

## introduction grammatical error correction (gec) is a complex task, aiming at detecting, identifying and correcting various grammatical errors in a given sentence. gec has recently attracted more attention due to its ability to correct and proofread the text, which can serve a variety of industries such as education, media and publishing @xcite . however, chinese gec (cgec) is still confronted with the following three obstacles: (1) lack of data. the major obstacle in cgec is that the high-quality manually annotated data is limited compared to other languages @xcite @xcite @xcite . there are only five publicly accessible datasets in cgec: nlpcc18 @xcite , cged @xcite , ctc-qua, yaclc @xcite and mucgec @xcite . (2) data sources are nonnative speakers. the sentences in nlpcc18, cged, yaclc and mucgec are all collected from chinese as a foreign language (cfl) learner sources. however, massive errors from native speakers rarely arise in these sources. therefore, the native speaker errors are more challenging with the inclusion of pragmatic data. though ctc-qua covers grammatical errors in native speakers, it has insufficient scale with 972 sentences. (3) limited multiple references. for an erroneous sentence, there tends to be different correction methods. the sentences revised by the model may be correct, but different from the ground truth. this may cause unexpected performance degradation @xcite . besides, more references can offer various correction schemas enabling the model to accommodate more scenarios. among cgec, only mucgec and yaclc provide rich references. to tackle aforementioned obstacles, we present fcgec, a large-scale fine-grained gec corpus with multiple references. the sentences in fcgec are mainly collected from multi-choice questions in public school chinese examinations. therefore, our fcgec is more challenging since it involves more pragmatic data in the examinations of native speakers. as for multiple references, we assign 2 to 4 annotators on each sentence, thus more references can be attained in this way. moreover, we generate more references in the annotation process through techniques with synonym substitution. in order to correct the grammatical errors, recent works are mostly based on two categories of benchmark models. sequence-to-sequence (seq2seq) approaches regard gec as a generation task that straightforward converts an erroneous sentence to the correct one @xcite @xcite . however, training such a generation model requires more computational resources due to the autoregressive decoder. moreover, the generated style of seq2seq models is more arbitrary, which is not well applicable for gec task. more recently, sequence-to-edit (seq2edit) approaches gain interests which take gec as a token-level labeling task @xcite via different edits, such as insert, delete, etc. nevertheless, previous work falls short on altering the word order and correcting errors simultaneously with iterating. to fill these gaps, we propose switch-tagger-generator (stg) model as an effective baseline to correct grammatical errors in low-resource settings inspired by @xcite . our stg can be decomposed into three modules: switch module determines the permutation of characters while tagger module identifies the operation tags of each character in the sequence. notably, benefiting from carefully designed compound tags, we eliminate the necessity for iteration. as for generator module, we adopt non-autoregressive approach to fill in the characters that do not appear in the source. in summary, our contributions are as follows: ## benchmark models we divide the gec task into a classification task and a correction task. the classification task involves the error detection and error type identification. we adopt the pre-trained language models (plms) based approaches for these tasks. as for the correction task, we propose a stg model to correct errors. meanwhile, two categories of mainstream gec models (seq2seq and seq2edit) are applied as benchmark models for our fcgec. ## related work there already exists a lot of work on grammatical error correction for datasets and approaches. in terms of the dataset, most researches focus on english gec. nucle @xcite , an early annotated corpus of gec research, collects the erroneous sentences from students' essays in nus. jfleg @xcite is constructed from toefl exam with native sounding judgement. w&i @xcite collects the texts from non-native english students around the world in an online web platform and then manually an- ## conclusion in this paper, we construct a large-scale corpus for chinese grammatical error detection, identification and correction. compared to previous cgec corpus, our fcgec is more complicated and challenging with pragmatic data. furthermore, we provide multiple references so that the models can be evaluated for better performance. furthermore, we propose a stg model to correct grammatical errors. extensive experiments demonstrate that our stg outperforms the baselines and achieves the state-ofthe-art performance. however, experiments show that there exists a notable gap between cutting-edge models and humans. therefore, it encourages the future gec models to bridge the gap.