Bidirectional Transformer Reranker for Grammatical Error Correction (2023)

Abstract: Pre-trained seq2seq models have achieved state-of-the-art results in the grammatical error correction task. However, these models still suffer from a prediction bias due to their unidirectional decoding. Thus, we propose a bidirectional Transformer reranker (BTR), that re-estimates the probability of each candidate sentence generated by the pre-trained seq2seq model. The BTR preserves the seq2seq-style Transformer architecture but utilizes a BERT-style self-attention mechanism in the decoder to compute the probability of each target token by using masked language modeling to capture bidirectional representations from the target context. For guiding the reranking, the BTR adopts negative sampling in the objective function to minimize the unlikelihood. During inference, the BTR gives final results after comparing the reranked top-1 results with the original ones by an acceptance threshold. Experimental results show that, in reranking candidates from a pre-trained seq2seq model, T5-base, the BTR on top of T5-base could yield 65.47 and 71.27 F0.5 scores on the CoNLL-14 and BEA test sets, respectively, and yield 59.52 GLEU score on the JFLEG corpus, with improvements of 0.36, 0.76 and 0.48 points compared with the original T5-base. Furthermore, when reranking candidates from T5-large, the BTR on top of T5-base improved the original T5-large by 0.26 points on the BEA test set.

## introduction grammatical error correction (gec) is a sequenceto-sequence task which requires a model to aim to correct an ungrammatical sentence. an example is presented in previous studies have shown that gec can be approached as machine translation by using a seq2seq model @xcite with a transformer @xcite architecture @xcite @xcite @xcite . as a neural model consists of an encoder and a decoder, the seq2seq architecture typically requires a large amount of training data. because gec suffers from limited training data, applying a seq2seq model for gec results in a low-resource setting, that can be handled by introducing synthetic data for training @xcite @xcite . however, as pointed out by @xcite , the use of synthetic data in gec may result in a distributional shift and require language-specific tuning, which can be time-consuming and resource-intensive. considering the limitations of the synthetic data, the current trend is to utilize the learned and general representations from a pre-trained model, such as bert @xcite , xlnet @xcite , bart @xcite , and t5 @xcite , which have been trained on large corpora and shown to be effective for various downstream tasks. according to @xcite , incorporating a pre-trained masked language model (mlm) into a seq2seq model could facilitate correction. in addition, as reported by @xcite , the pre-trained t5 model achieved stateof-the-art results on gec benchmarks for four languages after successive fine-tuning with the cleaned lang-8 corpus (clang-8) @xcite . although the seq2seq model with pre-trained representations has shown to be effective for gec, its performance was still constrained by its unidirectional decoding. as suggested by @xcite , for an ungrammatical sentence, a fully pre-source speed camera can be placed in many locations along a highway . gold 1 speed cameras can be placed in many locations along a highway . candidate 1 (roberta, t5gec) a speed camera can be placed in many locations along a highway . candidate 2 (btr, r2l) speed cameras can be placed in many locations along a highway . candidate 3 a speed camera can be placed in many locations along a highway . we regard the state-of-the-art model for gec @xcite , a pre-trained transformer model, t5 (either t5-base or t5-large), as our base model and utilize its generated candidates for reranking. because the btr can inherit learned representations from a pre-trained transformer model, we construct the btr on top of t5-base. our experimental results showed that, by reranking candidates from a fully pre-trained and fine-tuned t5-base model, the btr on top of t5-base can achieve an f 0.5 score of 65.47 on the conll-14 benchmark. the btr on top of t5-base also outperformed t5-base on the bea test set by 0.76 points, achieving an f 0.5 score of 71.27. adopting negative sampling for the btr also generated a peaked probability distribution for ranking, and so grammatical suggestions could be selected by using λ. furthermore, the btr on top of t5-base was robust even when reranking candidates from t5-large and improved the performance by 0.26 points on the bea test set. ## related work for directly predicting the target corrections from corresponding input tokens, @xcite and @xcite regarded the encoder of the transformer model as a nonautoregressive gec sequence tagger. the experimental results of @xcite showed that, compared with the randomly initialized lstm (hochreiter and schmidhuber, 1997), the pre-trained models, such as roberta @xcite , gpt-2 @xcite , and al-bert @xcite , can achieve higher f 0.5 scores as a tagger. @xcite considered gec as a seq2seq task and introduced the shallow aggressive decoding (sad) for the decoder of the transformer. with the sad, the performance of a pre-trained seq2seq model, bart, surpassed the sequence taggers of @xcite . the t5 xxl model is a pre-trained seq2seq model with 11b parameters @xcite . after finetuning with the clang-8 corpus, t5 xxl and mt5 xxl @xcite , a multilingual version of t5, achieved state-of-the-art results on gec bench-marks in four languages: english, czech, german, and russian @xcite . this demonstrated that performing a single fine-tuning step for a fully pre-trained seq2seq model is a simple and effective method for gec without incorporating a copy mechanism @xcite , the sad or the output from a pre-trained mlm @xcite . despite the improvements brought about by the pre-trained representations, the conventional seq2seq structure suffers from a prediction bias due to its unidirectional decoding. according to @xcite , by using beam search, a fully pretrained seq2seq gec model @xcite can generate several high-quality grammatical hypotheses, which include one that is more grammatical than the selected one. to address the shortcoming of the unidirectional decoding, previous studies @xcite @xcite introduced reversed representations to rerank the hypotheses. @xcite and @xcite utilized a seq2seq gec model that decodes in the opposite direction (right-to-left) to rerank candidates, which was effective to select a more grammatical sentence than the original one. this finding motivated us to use a bidirectional decoding method for our model. instead of using a seq2seq model, @xcite fine-tuned bert as a reranker to evaluate the grammatical quality of a sentence. by using masked language modeling, bert learned deep bidirectional representations to distinguish between grammatical and ungrammatical sentences. however, bert did not account for the positions of corrections, as it discarded the source sentence and considered only the target sentence. this made it difficult for bert, as a reranker, to recognize the most suitable corrected sentence for an ungrammatical sentence. @xcite proposed the use of pseudo-log-likelihood scores (pll) for reranking. they demonstrated that roberta, with the pll for reranking, outperformed the conventional language model gpt-2 when reranking candidates in speech recognition and machine translation tasks. @xcite also claimed that the pre-trained model, mpnet @xcite , was more effective than gpt-2 when using pll for reranking in discourse segmentation and parsing. @xcite proposed a bidirectional transformer-based alignment (btba) model, which aims to assess the alignment between the source and target tokens in machine transla-tion. to achieve this, btba masked and predicted the current token with attention to both left and right sides of the target context to produce alignments for the current token. specifically, to assess alignments from the attention scores in all crossattention layers, the decoder in btba discarded the last feed-forward layer of the transformer model and directly predicted masked tokens from the output of the last cross-attention layer. even though the target context on both sides was taken into consideration, one limitation of btba was that the computed alignments ignored the representation of the current token. to produce more accurate alignments, @xcite introduced full context based optimization (fcbo) for fine-tuning, in which btba no longer masks the target sentence to use the full target context. in our research, to determine the most appropriate correction for a given erroneous sentence, we model the btr as a seq2seq reranker, which encodes the erroneous sentence using an encoder and decodes a corrected sentence using a decoder. in contrast to the conventional seq2seq model, we use masked language modeling to mask and predict each target token in the decoder and estimate the sentence probability for each candidate using pll. unlike btba, the btr preserves the last feed-forward layer in the decoder to predict masked tokens more accurately. because the original data of the masked tokens should be invisible in the prediction, the fcbo fine-tuning step is not used in the btr. compared with btba, the btr keeps the structure of the transformer model and can easily inherit parameters from pre-trained models. ## preliminary because the decoder of the btr uses masked language modeling to rerank candidates based on the pll, in this section, we explain how a transformer-based gec model generates the candidates, the masked language modeling used in bert, and how to compute the pll. ## bidirectional transformer reranker (btr) the btr uses masked language modeling in the decoder to estimate the probability of a corrected sentence. given an ungrammatical sentence x, a base gec model first generates the top-a corrected sentences y a , as described in section 3.1. assume y @xmath0. the btr selects and accepts the most optimal corrected sentence y bt r from y a on the basis of the estimated sentence probability, as described in the following. figure 2 shows the overview of the btr for the whole procedure. ## conclusion we proposed a bidirectional transformer reranker (btr) to rerank several top candidates generated by a pre-trained seq2seq model for gec. for a fully pre-trained model, t5-base, the btr could achieve 65.47 and 71.27 f 0.5 scores on the conll-14 and bea test sets. our experimental results showed that the btr on top of t5-base with limited pretraining steps could improve both precision and recall for candidates from t5-base. since using negative sampling for the btr generates a peaked probability distribution for ranking, introducing a threshold λ benefits the acceptance of the suggestion from the btr. furthermore, the btr on top of t5-base could rerank candidates generated from t5-large and yielded better performance. this finding suggests the effectiveness of the btr even in experiments with limited gpu resources. while the btr in our experiments lacked sufficient pretraining, it should further improve the performance with full pre-training for reranking in future.