S yn GEC : Syntax-Enhanced Grammatical Error Correction with a Tailored GEC -Oriented Parser (2022)

Abstract: This work proposes a syntax-enhanced grammatical error correction (GEC) approach named SynGEC that effectively incorporates dependency syntactic information into the encoder part of GEC models. The key challenge for this idea is that off-the-shelf parsers are unreliable when processing ungrammatical sentences. To confront this challenge, we propose to build a tailored GEC-oriented parser (GOPar) using parallel GEC training data as a pivot. First, we design an extended syntax representation scheme that allows us to represent both grammatical errors and syntax in a unified tree structure. Then, we obtain parse trees of the source incorrect sentences by projecting trees of the target correct sentences. Finally, we train GOPar with such projected trees. For GEC, we employ the graph convolution network to encode source-side syntactic information produced by GOPar, and fuse them with the outputs of the Transformer encoder. Experiments on mainstream English and Chinese GEC datasets show that our proposed SynGEC approach consistently and substantially outperforms strong baselines and achieves competitive performance. Our code and data are all publicly available at https://github.com/HillZhang1999/SynGEC.

## introduction given an ungrammatical sentence, the grammatical error correction (gec) task aims to produce a grammatical target sentence with the intended meaning @xcite . recent mainstream approaches treat gec as a monolingual machine translation (mt) task @xcite . standard encoder-decoder based mt models, e.g., transformer @xcite , have ˚corresponding author. 1 although this work focuses on the dependency syntax structure, syngec can also be extended to the constituency syntax structure straightforwardly. emerged as a dominant paradigm and achieved state-of-the-art (sota) results on various gec benchmarks @xcite @xcite . despite their impressive achievements, most work treats the input sentence as a sequence of tokens, without explicitly exploiting syntactic or semantic information. compared with mt, gec has two peculiarities that directly motivate this work. first, the training data for gec models is much less abundant, which may be alleviated by incorporating linguistic structure knowledge like syntax. as shown in in this paper, we propose syngec, an approach that can effectively inject the syntactic structure of the input sentence into the encoder part of gec models. the critical challenge here is that off-theshelf parsers are unreliable when handling ungrammatical sentences. on the one hand, off-the-shelf parsers are trained on clean treebanks that only consist of grammatical sentences. when parsing ungrammatical sentences, their performance may sharply degrade due to the input mismatch. on the other hand, mainstream syntax representation schemes, adopted by existing treebanks, do not cover the non-canonical structures arising from grammatical errors. in consequence, under such schemes, it is sometimes difficult to find a plausible syntactic tree to properly parse an ungrammatical sentence (e.g., the sentence in figure 1(d ## our gec-oriented parser this section describes our tailored gopar, a dependency parser that is more competent in parsing ungrammatical sentences than off-the-shelf parsers. ## the depgcn-based gec model this section describes our depgcn-based gec model, whose architecture is shown in figure 3 . we adopt gcn @xcite to encode the dependency syntax trees of the source sentence. then, we feed the encoded syntactic information into a transformer-based gec model. ## conclusions this paper presents a syngec approach that incorporates adapted dependency syntax into gec models. the key idea is adjusting vanilla parsers to accommodate ungrammatical sentences. we first extend the standard syntax representation scheme to use a unified tree structure to encode both grammatical errors and syntactic structure. then we obtain high-quality parse trees of ungrammatical sentences by projecting target-side trees into sourceside ones in parallel gec training data, which are ultimately used for training a tailored parser named gopar. we employ gcn to encode syntax produced by gopar. experiments on mainstream datasets in two languages show that syngec is effective and achieves sota results.