CYUT Team C hinese Grammatical Error Diagnosis System Report in NLPTEA -2020 CGED Shared Task (2020)

Abstract: This paper reports our Chinese Grammatical Error Diagnosis system in the NLPTEA-2020 CGED shared task. In 2020, we sent two runs with two approaches. The first one is a combination of conditional random fields (CRF) and a BERT model deep-learning approach. The second one is a BERT model deep-learning approach. The official results shows that our run1 achieved the highest precision rate 0.9875 with the lowest false positive rate 0.0163 on detection, while run2 gives a more balanced performance.

## introduction learning chinese is very popular for foreigners, but it is difficult for them to write correct sentence. grammatical error detection is a big challenge for the chinese learners as a second language. learning chinese sentences will rely too much on the teacher to correct the wrong sentences. it is not easy for learners to get timely feedback. therefore, how to use existing technology to detect and correct the grammatical errors that learners make has become a hot topic. since 2014 @xcite ) @xcite ) @xcite @xcite , the nlp-tea workshop provides a series chinese grammar error detection (cged) shared tasks to promote the research on grammar error diagnosis. the organizers ask professional teachers to label the errors in learners' sentences. there are four types of label in the sentences: redundant (r), selection (s), disorder (w), and missing (m). the goal of the task is to build a system that can predict whether a sentence is wrong and correct it. in previous years, we participated in the nlptea cged (wu et al., 1 https://lang-8.com/ 2018) and shows that such a system can be precision oriented or recall oriented for different users. since the emerging of deep learning, we find that sequence-to-sequence models have good effect on grammar correction, and the bert model @xcite is the best sequence-to-sequence pre-training language model using a large number of data sets. the pretrained model is trained with mask language model (mlm) to enhance the strength of the model. in run1 of 2020, we use bert as the first level of our identification. we fine-tuning the bert model with the lang-8 1 corpus and all the data from nlptea since 2016 to 2020, so that the model can be used to predict correct and incorrect sentences, and reproduce the wrong sentences. the error types are determined by crf. in run2 is not used to determine the wrong and correct sentences. in the following sections, we will introduce related work and our approaches, then discuss the formal test results, and give conclusion and future works. ## related work grammar error detection and correction is now a popular research topic in natural language processing @xcite . previous works show that crf model can be used to integrate various features to build a good system. better results can be achieved by using the precollected collocation word database. recently, researchers use deep learning models to solve this issue. the most common models are sequence-to-sequence @xcite and convolutional neural network @xcite models. the idea of sequence-to-sequence is to translate the wrong into correct sentences just like translation between two languages. a corrected sentence is generated from the wrong sentence, and it is believed that multiple revisions will give better results. the convolutional neural network originally is used to process images, now shifted to process text. with the two-dimension processing power, it is easier for the model to read the context of the text. since 2019, the bert gives many state-ofthe-art results on several nlp applications. this shows the great influence of bert on natural language processing. spelling check is a similar task to the grammar correction @xcite . the authors use the bert internal model to find typos. although its effect is not the best, it achieves the purpose the author wants. ## method this year, we mainly focused on minimizing the false alarm on error detection. since the system is to help foreign learners, we hope that less errors judged by the model will not cause learners to feel frustration. bert is a pre-trained language model. since the original pre-training model was not trained for chinese grammar correction, we have to train it with our corpus. for different task, better results can be achieved by fine-tuning the pre-trained language models with additional training corpus. moreover, bert has achieved excellent results in various projects, such as single classification tasks, sentence-labelling tasks, and question answering tasks. in addition to the bert model, we also use conditional random fields (crf) to double check tine wrong sentences detected by bert, and select the type and location of the errors.