Adjusting the Precision-Recall Trade-Off with Align-and-Predict Decoding for Grammatical Error Correction (2022)

Abstract: Modern writing assistance applications are always equipped with a Grammatical Error Correction (GEC) model to correct errors in user-entered sentences. Different scenarios have varying requirements for correction behavior, e.g., performing more precise corrections (high precision) or providing more candidates for users (high recall). However, previous works adjust such trade-off only for sequence labeling approaches. In this paper, we propose a simple yet effective counterpart – Align-and-Predict Decoding (APD) for the most popular sequence-to-sequence models to offer more flexibility for the precision-recall trade-off. During inference, APD aligns the already generated sequence with input and adjusts scores of the following tokens. Experiments in both English and Chinese GEC benchmarks show that our approach not only adapts a single model to precision-oriented and recall-oriented inference, but also maximizes its potential to achieve state-of-the-art results. Our code is available at https://github.com/AutoTemp/Align-and-Predict .

## introduction modern writing assistance applications (e.g., microsoft office word foot_0 , google docs foot_1 and grammarly foot_2 ) always contain grammatical error correction (gec) modules @xcite @xcite to correct errors in user-entered sentences. such applications usually require gec models to perform different correction tendencies and behaviors according to practical scenarios and user preferences @xcite . for instance, as shown in ## align-and-predict decoding beam search @xcite @xcite is a widely used algorithm for decoding sequences on all generation tasks, such as translation @xcite , dialogue @xcite , etc. multiple modifications to beam search that force the outputs to include pre-defined lexical constraints (i.e., words and phrases) have been proposed @xcite . fortunately, the input and output sentences of gec overlap significantly and the input tokens are natural constraints for correction generation. this assumption is an objective characteristic of gec and has been made in many previous works @xcite @xcite . thus, we propose a novel decoding approach -align-and-predict decoding (apd), which leverage the characteristic of gec to adjust behavior and tendencies of inference. the overview of apd is shown in figure 1 . given an input sentence @xmath0, . . . , x n ), we maintain k hypotheses at the time step t during inference as beam search does: since the output of gec is highly constrained by the input sequence, we assume that h @xmath1. then, we match the suffix of each hypothesis h @xmath2. if there exists a unique substring x k-j , ..., x k (@xmath3, ..., y i t , the next token of the hypothesis h @xmath4, which we store in the set n i t . 4 formally, as beam search does, we expand current hypotheses and construct possible candidates for the next time step t + 1 with all tokens in the vocabulary. the candidate ĥi t,v of the i-th hypothesis is obtained as follows: ## conclusion we propose a novel language-independent decoding approach to offer more flexibility to adjust the precision-recall trade-off of inference for seq2seq gec models, making it adaptive to various realworld application scenarios. it can not only adapt a single model to precision-oriented and recalloriented inference, but also be used as a simple trick for better overall performance. on both english and chinese gec benchmarks, our approach further improves the state-of-the-art seq2seq gec model by precision-recall trade-off. in the future, we plan to apply it to other sentence rewriting tasks, such as paraphrasing and style transfer.