C hinese Grammatical Error Diagnosis with Graph Convolution Network and Multi-task Learning (2020)

Abstract: This paper describes our participating system on the Chinese Grammatical Error Diagnosis (CGED) 2020 shared task. For the detection subtask, we propose two BERT-based approaches 1) with syntactic dependency trees enhancing the model performance and 2) under the multi-task learning framework to combine the sequence labeling and the sequence-to-sequence (seq2seq) models. For the correction subtask, we utilize the masked language model, the seq2seq model and the spelling check model to generate corrections based on the detection results. Finally, our system achieves the highest recall rate on the top-3 correction and the second best F1 score on identification level and position level.

## introduction chinese has become an influential language all over the world. more and more people choose chinese as a second/foreign language (csl/cfl). their writings usually contain grammatical errors including spelling and collocation errors. for instance, a japanese learner may write "我苹果喜欢" (i apple like) while its correct expression should be "我 喜欢苹果" (i like the apple). the inconsistency of chinese and japanese grammatical structures will lead to different expression order. grammatical structure in chinese is different from other languages and affects expression. the previous works used to do feature engineering including pretrained features and parsing features to improve performance. in this paper, we fertilize the representations from bert with the syntactic dependency tree and propose a multi-task learning of error detection and correction. we employ three strategies based on bert for correction based on detection results. experiment shows that our system is effective on both detection and correction level. our contributions are summarized as follows: this paper is organized as follows. section 2 describes the cged task. section 3 describes our system for grammatical error detection and correction. section 4 reports the experimental results conducted by the proposed methods. section 5 concludes this work. ## chinese grammatical error diagnosis the cged shared task has been held since 2014. several sets of training data have been released written by cfl learners which contain a lot of grammatical errors. for detection, the cged defines four types of errors: (1) r (redundant word wrong input: correct input: smoking can cause many diseases smoking can many diseases errors);(2) m (missing words); (3) w (word ordering errors);(4) s (word selection errors) as shown in figure 1. the performance is measured at detection level, identification level and position level. for correction, systems are required to recommend at most 3 corrections for missing and selection errors. ## conclusion this article describes our system in the cged shared task. we proposed two approaches including bert-gcn-crf model and multi-task learning to improve the baseline model to detect grammatical errors. we also designed three approaches including masked language model, seq2seq and spelling check to correct these errors. we got first place in the recall rate of the top-3 correction and got the second highest f1 scores at the identification level and position level.