Venue: EMNLP 2017

# Neural Sequence-Labelling Models for Grammatical Error Correction

 Helen Yannakoudakis, Marek Rei, Oistein E. Andersen and Zheng Yuan

The ALTA Institute

Computer Laboratory

University of Cambridge

{hy260,mr472,oa223,zy249}@cl.cam.ac.uk

###### Abstract

We propose an approach to \(N\)-best list re-ranking using neural sequence-labelling models. We train a compositional model for error _detection_ that calculates the probability of each token in a sentence being correct or incorrect, utilising the full sentence as context. Using the error detection model, we then re-rank the \(N\) best hypotheses generated by statistical machine translation systems. Our approach achieves state-of-the-art results on error correction for three different datasets, and it has the additional advantage of only using a small set of easily computed features that require no linguistic input.

## 1 Introduction

Grammatical Error Correction (GEC) in non-native text attempts to automatically detect and correct errors that are typical of those found in learner writing. High precision and good coverage of learner errors is important in the development of GEC systems. Phrase-based Statistical Machine Translation (SMT) approaches to GEC have attracted considerable attention in recent years as they have been shown to achieve state-of-the-art results Felice et al. (2014); Junczys-Dowmunt and Grundkiewicz (2016). Given an ungrammatical input sentence, the task is formulated as "translating" it to its grammatical counterpart. Using a parallel dataset of input sentences and their corrected counterparts, SMT systems are typically trained to correct all error types in text without requiring any further linguistic input. To further adapt SMT approaches to the task of GEC and tackle the paucity of error-annotated learner data, previous work has investigated a number of extensions, ranging from the addition of further features into the decoding process Felice et al. (2014) via re-ranking the SMT decoder's output Yuan et al. (2016) to neural-network adaptation components to SMT Chollampatt et al. (2016).

In this paper, we propose an approach to \(N\)-best list re-ranking using neural sequence-labelling models. \(N\)-best list re-ranking allows for fast experimentation since the decoding process remains unchanged and only needs to be performed once. Crucially, it can be applied to any GEC system that can produce multiple alternative hypotheses. More specifically, we train a neural compositional model for error _detection_ that calculates the probability of each token in a sentence being correct or incorrect, utilising the full sentence as context. Using the error detection model, we then re-rank the \(N\) best hypotheses generated by the SMT system. Detection models can be more fine-tuned to finer nuances of grammaticality and acceptability, and therefore better able to distinguish between correct and incorrect versions of a sentence.

Our approach achieves state-of-the-art results on GEC for three different datasets, and it has the additional advantage of using only a small set of easily computed features that require no linguistic information, in contrast to previous work that has utilised a large set of features in a supervised setting Hoang et al. (2016); Yuan et al. (2016).

## 2 Previous work

The first approaches to GEC primarily treat the task as a classification problem over vectors of contextual lexical and syntactic features extracted from a fixed window around the target token. A large body of work has investigated error-type-specific models, and in particular models targeting preposition and article errors, which are among the most frequent ones in non-native English learner writing Chodorow et al. (2007); De Felice and Pul

man, 2008; Han et al., 2010; Tetreault et al., 2010; Han et al., 2006; Tetreault and Chodorow, 2008; Gamon et al., 2008; Gamon, 2010; Rozovskaya and Roth, 2010; Rozovskaya et al., 2012; Dale and Kilgarriff, 2011; Leacock et al., 2014). Core components of one of the top systems in the CoNLL 2013 and 2014 shared tasks on GEC (Ng et al., 2013, 2014) include Averaged Perceptron classifiers, native-language error correction priors in Naive Bayes models, and joint inference frameworks capturing interactions between errors (e.g., noun number and verb agreement errors) (Rozovskaya et al., 2012, 2014, 2011; Rozovskaya and Roth, 2011). The power of the classification paradigm comes from its ability to generalise well to unseen examples, without necessarily requiring error-annotated learner data (Rozovskaya and Roth, 2016).

One of the first approaches to GEC as an SMT task is the one by Brockett et al. (2006), who generate artificial data based on hand-crafted rules to train a model that can correct countability errors. Dahlmeier and Ng (2011) focus on correcting collocation errors based on paraphrases extracted from parallel corpora, while Dahlmeier and Ng (2012a) are the first to investigate a discriminatively trained beam-search decoder for full-sentence correction, focusing on five different error types: spelling, articles, prepositions, punctuation insertion, and noun number. Yoshimoto et al. (2013) utilise SMT to tackle determiner and preposition errors, while Yuan and Felice (2013) use POS-factored, phrase-based SMT systems, trained on both learner and artificially generated data to tackle determiner, preposition, noun number, verb form, and subject-verb agreement errors. The SMT approach has better capacity to correct complex errors, and it only requires parallel corrected sentences as input.

Two state-of-the-art systems in the 2014 CoNLL shared task on correction of all errors regardless of type use SMT systems: Felice et al. (2014) use a hybrid approach that includes a rule-based and an SMT system augmented by a large web-based language model and combined with correction-type estimation to filter out error types with zero precision. Junczys-Dowmunt and Grundkiewicz (2016) investigate parameter tuning based on the MaxMatch (\(M^{2}\)) scorer, the shared-task evaluation metric (Dahlmeier and Ng, 2012b; Ng et al., 2014), and experiment with different optimisers and interactions of dense and sparse features.

Susanto et al. (2014) and Rozovskaya and Roth (2016) explore combinations of SMT systems and classifiers, the latter showing substantial improvements over the CoNLL state of the art. Chollampatt et al. (2016a) integrate a neural network joint model that has been adapted using native-language-specific learner text as a feature in SMT, while Chollampatt et al. (2016b) integrate a neural network global lexicon model and a neural network joint model to exploit continuous space representations of words rather than discrete ones, and learn non-linear mappings. Yuan and Briscoe (2016) present a Neural Machine Translation (NMT) model and propose an approach that tackles the rare-word problem in NMT.

Yuan et al. (2016) and Mizumoto and Matsumoto (2016) employ supervised discriminative methods to re-rank the SMT decoder's \(N\)-best list output based on language model and syntactic features respectively. Hoang et al. (2016) also exploit syntactic features in a supervised framework, but further extend their approach to generate new hypotheses. Our approach is similar in spirit, but differs in the following aspects: inspired by the work of Rei and Yannakoudakis (2016) who tackle error _detection_ rather than _correction_ within a neural network framework, we develop a neural sequence-labelling model for error _detection_ to calculate the probability of each token in a sentence as being correct or incorrect; using the error detection model, we propose a small set of features that require no linguistic processing to re-rank the \(N\) best hypotheses. We evaluate our approach on three different GEC datasets and achieve state-of-the-art results, outperforming all previous approaches to GEC.

## 3 Datasets

We use the First Certificate in English (FCE) dataset (Yannakoudakis et al., 2011), and the NUS Corpus of Learner English (NUCLE) (Dahlmeier et al., 2013) that was used in the CoNLL GEC shared tasks. Both datasets are annotated with the language errors committed and suggested corrections from expert annotators. The former consists of upper-intermediate learner texts written by speakers from a number of different native language backgrounds, while the latter consists of essays written by advanced undergraduate university 

students from an Asian language background. We use the public FCE train/test split, and the NUCLE train/test set used in CoNLL 2014 (the test set has been annotated by two different annotators).

We also use the publicly available Lang-8 corpus Mizumoto et al. (2012); Tajiri et al. (2012) and the JHU FLuency-Extended GUG corpus J-FLEG Napoles et al. (2017). Lang-8 contains learner English from lang-8.com, a language-learning social networking service, which has been corrected by native speakers. JFLEG is a newly released corpus for GEC evaluation that contains fluency edits to make the text more native-like in addition to correcting grammatical errors, and contains learner data from a range of proficiency levels.

We use Lang-8 and the FCE and CoNLL training sets to train our neural sequence-labelling model, and test correction performance on JFLEG, and the FCE and CoNLL test sets. For JFLEG, we use the 754 sentences on which Napoles et al. (2017) have already benchmarked four leading GEC systems. As our development set, we use a subset of the FCE training data.

## 4 Neural sequence labelling

We treat error detection as a sequence labelling task and assign a label to each token in the input sentence, indicating whether it is correct or incorrect in context. These binary gold labels can be automatically extracted from the manual error annotation available in our data (see Section 3). Similarly to Rei and Yannakoudakis (2016), we construct a bidirectional recurrent neural network for detecting writing errors. The system receives a series of tokens \([w_{1}...w_{T}]\) as input, and predicts a probability distribution over the possible labels for each token.

Every token \(w_{t}\) is first mapped to a token representation \(\widetilde{x}_{t}\), which is also optimised during training. These embeddings are composed together into context-specific representations using a bidirectional LSTM Hochreiter and Schmidhuber (1997):

\[\overrightarrow{h_{t}}=\text{LSTM}(\widetilde{x}_{t},\overrightarrow{h_{ t-1}}) \tag{1}\]

\[\overleftarrow{h_{t}}=\text{LSTM}(\widetilde{x}_{t},\overleftarrow{h_{t+1}}) \tag{2}\]

\[h_{t}=[\overrightarrow{h_{t}};\overleftarrow{h_{t}}] \tag{3}\]

where \(\widetilde{x}_{t}\) is the token representation at position \(t\), \(\overrightarrow{h_{t}}\) is the hidden state of the forward-moving LSTM, \(\overleftarrow{h_{t}}\) is the hidden state of the backward-moving LSTM, and \(h_{t}\) is the concatenation of both hidden states. A feedforward hidden layer with tanh activation is then used to map the representations from both directions into a more suitable combined space, and allow the model to learn higher-level features:

\[d_{t}=\text{tanh}\,W_{d}h_{t} \tag{4}\]

where \(W_{d}\) is a weight matrix. Finally, a softmax output layer predicts the label distribution for each token, given the input sequence:

\[P(y_{t}|w_{1}....w_{T})=\text{softmax}\,W_{o}d_{t} \tag{5}\]

where \(W_{o}\) is an output weight matrix.

We also make use of the character-level architecture proposed by Rei et al. (2016), allowing the model to learn morphological patterns and capture out-of-vocabulary words. Each individual character is mapped to a character embedding and a bidirectional LSTM is used to combine them together into a character-based token representation. This vector \(m\), constructed only from individual characters, is then combined with the regular token embedding \(x_{t}\) using an adaptive gating mechanism:

\[z=\sigma\big{(}W_{z_{1}}\cdot\text{tanh}(W_{z_{2}}x_{t}+W_{z_{3}}m)\big{)} \tag{6}\]

\[\widetilde{x}_{t}=z\cdot x_{t}+(1-z)\cdot m \tag{7}\]

where \(W_{z_{1}}\), \(W_{z_{2}}\) and \(W_{z_{3}}\) are weight matrices, \(z\) is a dynamically calculated gating vector, and \(\widetilde{x}_{t}\) is the resulting token representation at position \(t\).

We optimise the model by minimising cross-entropy between the predicted label distributions and the annotated labels. In addition to training the error detection objective, we make use of a multi-task loss function and train specific parts of the architecture as language models. This provides the model with a more informative loss function, while also encouraging it to learn more general compositional features and acting as a regulariser Rei (2017). First, two extra hidden layers are constructed:

\[\overrightarrow{m_{t}}=\text{tanh}\overrightarrow{W}_{m}\overrightarrow{h_{t}}\] (8) 

\[\widehat{m}_{t}=\text{tanh}\,\widehat{W}_{m}\overleftarrow{h}_{t} \tag{9}\]

where \(\overrightarrow{W}_{m}\) and \(\overleftarrow{W}_{m}\) are direction-specific weight matrices, used for connecting a forward or backward LSTM hidden state to a separate layer. The surrounding tokens are then predicted based on each hidden state using a softmax output layer:

\[P(w_{t+1}|w_{1}...w_{t})=\text{softmax}\,\overrightarrow{W}_{q} \overrightarrow{m_{t}} \tag{10}\] \[P(w_{t-1}|w_{t}...w_{T})=\text{softmax}\,\overleftarrow{W}_{q} \overleftarrow{m_{t}} \tag{11}\]

During training, the following cost function is minimised, which combines the error detection loss function with the two language modeling objectives:

\[E= -\sum_{t=1}^{T}\text{log}\,P(y_{t}|w_{t}...w_{T})\] \[-\gamma\sum_{t=1}^{T-1}\text{log}\,P(w_{t+1}|w_{1}...w_{t}) \tag{12}\] \[-\gamma\sum_{t=2}^{T}\text{log}\,P(w_{t-1}|w_{t}...w_{T})\]

where \(\gamma\) is a weight that controls the importance of language modeling in relation to the error detection objective. Figure 1 shows the error detection network architecture.

### Experimental settings

All digits in the text are replaced with the character '0'. Tokens that occur less than \(2\) times in the training data share an out-of-vocabulary (OOV) token embedding, whereas the character-level component still operates over the original tokens. The model hyperparameters are tuned based on F\({}_{0.5}\) on the FCE development set (Section 3) and \(\gamma\) is set to \(0.1\).1 The model is optimised using Adam Kingma and Ba (2015), and training is stopped when F\({}_{0.5}\) does not improve on the development set over 5 epochs. Token representations have size 300 and are initialised with pre-trained word2vec embeddings trained on Google News Mikolov et al. (2013). The character representations have size 50 and are initialised randomly. The LSTM hidden layers have size 200 for each direction.

Footnote 1: Lower \(\gamma\) values tend to give better error detection results as this essentially prioritises the error detection objective.

### Error detection performance

Rei and Yannakoudakis (2016)'s error detection framework uses token-level embeddings, bidirectional LSTMs for context representation, and a multi-layer architecture for learning more complex features. They train their model on the public FCE training set, and benchmark their results on the FCE and CoNLL test sets (Baseline LSTMFCE). We also train and test our detection model on the same data and evaluate the effectiveness of our approach (LSTMFCE). In Table 1, we can see that our architecture achieves a higher performance on both FCE and CoNLL, and particularly for FCE (7% higher F\({}_{0.5}\)) and CoNLL test annotation 2 (around 2% higher F\({}_{0.5}\)). When we use a larger training set that also includes the CoNLL training data and the public Lang-8 corpus (see Section 3), performance improves even further (LSTM), particularly for CoNLL test annotation 1 (at least 8% higher F\({}_{0.5}\) compared to LSTMFCE). We use this model in the experiments reported in the following sections.

## 5 Statistical machine translation

SMT attempts to identify the \(1\)-best correction hypothesis \(c^{*}\) of an input sentence \(s\) that maximises the following:

\[c^{*}=\operatorname*{arg\,max}_{c}p_{\text{LM}}(c)\,p(s|c) \tag{13}\]

Figure 1: Error detection network architecture that is repeated for all the words in a sentence (illustration for the word “cat”).

 

A Language Model (LM) is used to estimate the correction hypothesis probability from a corpus of correct English, and a translation model to estimate the conditional from a parallel corpus of corrected learner sentences. State-of-the-art SMT systems are phrase-based [14] in that they use phrases as "translation" units and therefore allow many-to-many "translation" mappings. The translation model is decomposed into a phrase-translation probability model and a phrase re-ordering probability model, and the -best correction hypothesis is of the following log-linear form [25]:

\[c^{*}=\operatorname*{arg\,max}_{c}\,\exp\,\sum_{i=1}^{K}\lambda_{i}\,h_{i}(c,s) \tag{14}\]

where represents a feature function (e.g., phrase-translation probability) and the feature weight.

In this work, we employ two SMT systems: Yuan et al. yuan2016learning2 and Junczys-Dowmun and Grundkiewicz zhanghang2016learning. We apply our re-ranking approach to each SMT system's -best list using features derived from the neural sequence-labelling model for error detection described in the previous section, improve each of the SMT systems, and achieve state-of-the-art results on all three GEC datasets: FCE, CoNLL and JFLEG.

Footnote 2: Yuan et al. yuan2016learning propose a supervised -best list re-ranking approach; however, we only use their baseline SMT system.

### -best list re-ranking

For each SMT system, we generate the list of all the best candidate hypotheses. We then use the following set of features (tuned on the FCE development set, see Section3) to assign a score to each candidate, and determine a new ranking for each SMT model:

Sentence probabilityOur error detection model outputs a probabilty indicating whether a token is likely to be correct or incorrect in context. We therefore use as a feature the overall sentence probability, calculated based on the probability of each of its tokens being correct: \(\sum\limits_{w}\text{log}\,P(w)\)

Levenshtein distanceWe first use Levenshtein distance (LD) to identify which tokens in the original/source sentence have been corrected by the candidate hypothesis. We then identify the tokens that our detection model predicts as incorrect (i.e., the probability of being incorrect is greater than ). These give us two different sets of annotations for the source sentence: tokens in the source sentence that the candidate hypothesis identifies as incorrect; and tokens in the source sentence that the error detection model identifies as incorrect. We then convert these annotations to binary sequences - i.e., 1 if the token is identified as incorrect, and 0 otherwise - and use as a feature the LD between those binary representations. More specifically, we would like to select the candidate sentence that has the smallest LD from the binary sequence created by the detection model: \(\frac{1}{\text{LD}}\)

True and false positivesGiven the binary sequences described above, we also use as a feature the ratio of true positives (TP) to false positives (FP) by treating the error detection model as the "gold standard". Specifically, we count how many times the candidate hypothesis agrees or not with the detection model on the tokens identified as incorrect: \(\frac{\text{TP}}{\text{FP}}\)

We use a linear combination of the above three scores together with the overall score (i.e., original rank) given by the SMT system (we do not include any other SMT features) to re-rank each SMT system's -best list in an unsupervised way. The new -best correction hypothesis is then the one that maximises:

\[c^{*}=\operatorname*{arg\,max}_{c}\sum_{i=1}^{K}\lambda_{i}\,h_{i}(c) \tag{15}\]

\begin{table}
\begin{tabular}{l|r r r|r r r|r r r} \hline  & \multicolumn{4}{c|}{FCE test set} & \multicolumn{4}{c|}{CoNLL test set annotation 1} & \multicolumn{4}{c}{CoNLL test annotation 2} \\ \hline System & P & R & F\({}_{0.5}\) & P & R & F\({}_{0.5}\) & P & R & F\({}_{0.5}\) \\ \hline Baseline LSTM\({}_{\text{FCE}}\) & 46.10 & 28.50 & 41.10 & 15.40 & 22.80 & 16.40 & 23.60 & 25.10 & 23.90 \\ LSTM\({}_{\text{FCE}}\) & 58.88 & 28.92 & 48.48 & 17.68 & 19.07 & 17.86 & 27.62 & 21.18 & 25.88 \\ \hline LSTM & 79.10 & 21.19 & **51.14** & 54.51 & 8.69 & **26.53** & 69.60 & 7.91 & **27.18** \\ \hline \end{tabular}
\end{table}
Table 1: Token-level error detection performance of our detection models (LSTM\({}_{\text{FCE}}\) and LSTM) on FCE and the two CoNLL 2014 test set annotations. Baseline LSTM\({}_{\text{FCE}}\) and LSTM\({}_{\text{FCE}}\) are trained only on the public FCE training set.

 

where \(h\) represents the score assigned to candidate hypothesis \(c\) according to feature \(i\); \(\lambda\) is a parameter that controls the effect feature \(i\) has on the final ranking; and \(K=4\) as we have four different features (three features presented in this section, plus the original score output by the SMT system). \(\lambda\)s are tuned on the FCE development set and are set to \(1\), except for the sentence probability feature which has \(\lambda=1.5\).3

Footnote 3: We experimented with a small set of values (from \(0\) to \(2\) with increments of \(.1\)), though not exhaustively.

## 6 Evaluation

We evaluate the effectiveness of our re-ranking approach on three different datasets: FCE, CoNLL 2014 and JFLEG. We report F\({}_{0.5}\) using the shared task's \(M^{2}\) scorer Dahlmeier and Ng (2012), and GLEU scores Napoles et al. (2015). The latter is based on a variant of BLEU Papineni et al. (2002) that is designed to reward correct edits and penalise ungrammatical ones. As mentioned in Section 5, we re-rank the \(10\)-best lists of two SMT systems: Yuan et al. (2016) (CAMB16SMT) and Junczys-Dowmunt and Grundkiewicz (2016) (AMU16SMT). The results are presented in Table 2.

We replicate the AMU16SMT system to obtain the \(10\)-best output, and report results using this version (AMU16SMT (replicated)). Compared to the original results on CoNLL reported in their paper (AMU16SMT (reported)), we obtain slightly lower performance.4 We can see that AMU16SMT is the current state of the art on CoNLL, with an F\({}_{0.5}\) of 49.49. On the other hand, CAMB16SMT generalises better on FCE and JFLEG: 52.90 and 52.44 F\({}_{0.5}\) respectively. The lower performance of AMU16SMT can be attributed to the fact that it is tuned for the CoNLL shared task.

Footnote 4: The differences are likely to be caused by different versions of the NLTK tokeniser and/or Moses.

The current state of the art on FCE is a neural machine translation system, CAMB16SMT Yuan and Briscoe (2016), which is also the best model on JFLEG in terms of GLEU. The rest of the baselines we report are: Rozovskaya and Roth (2016), who explore combinations of SMT systems and classifiers (VT16SMT + classifiers); Chollampatt et al. (2016), who integrate a neural network joint model that has been adapted using native-language-specific learner text as a feature in SMT (NUS16SMT+NNIM); and Hoang et al. (2016), who perform supervised \(N\)-best list re-ranking using a large set of features, and further extend their approach to generate new hypotheses (NUS16SMT + re-ranker).5

Footnote 5: We note that Napoles et al. (2017) use an updated version of GLEU to evaluate AMU16SMT (reported), NN16SMT+NNIM and CAMB16SMT on JFLEG. We therefore also use this updated version throughout all GLEU evaluations on JFLEG.

 

When using our LSTM detection model to re-rank the \(10\)-best list (+ LSTM), we can see that performance improves across all three datasets for both SMT systems. F\({}_{0.5}\) performance of CAMB16SMT on FCE improves from \(52.90\) to \(54.15\), on CoNLL from \(37.33\) to \(39.53\), and on JFLEG from \(52.44\) to \(53.50\) (the latter demonstrating that the detection model also helps with fluency edits). This improved result is also better than the state of the art CAMB168MT on FCE.6 When looking at AMU16SMT, we can see that re-ranking (+ LSTM) further improves the best result on CoNLL from \(49.34\) (replicated) to \(49.66\) F\({}_{0.5}\), and there is a similar level of improvement for both FCE and JFLEG.

Footnote 6: We note that CAMB16SMT outperforms the re-ranking approach by Yuan et al. (2016).

As a further experiment, we re-train our error detection model on the same training data as CAMB16SMT (+ LSTMcamb). More specifically, we use the Cambridge Learner Corpus (CLC) Nicholls (2003), a collection of learner texts of various proficiency levels, written in response to exam prompts and manually annotated with the errors committed (around 2M sentence pairs). In Table 2, we can see that the detection model further improves performance across all datasets and SMT systems. Compared to just doing SMT with CAMB16SMT, re-ranking improves F\({}_{0.5}\) from \(52.90\) to \(55.60\) on FCE (performance increases further even though CAMB16SMT's training set includes a large set of FCE data), from \(37.33\) to \(42.44\) on CoNLL, and from \(52.44\) to \(54.66\) on JFLEG. The largest improvement is on CoNLL (5%), which is likely because CoNLL is not included in the training set. AMU16SMIT (replicated) is specifically tuned for CoNLL; nevertheless, the detection model also improves F\({}_{0.5}\) on CoNLL from \(49.34\) to \(51.08\). Re-ranking using a small set of detection-based features produces state-of-the-art results on all three datasets (we note that CAMB16SMT generalises better across all).

We next run ablation tests to investigate the extent to which each feature contributes to performance. Results obtained on the FCE test set after excluding each of the features of the 'CAMB16SMT + LSTMcamb' re-ranking system are presented in Table 3. Overall, all features have a positive effect on performance, though the sentence probability feature does have the biggest impact: its removal is responsible for a \(1.47\) and \(1.11\) decrease of F\({}_{0.5}\) and GLEU respectively. A similar pattern is observed on the other datasets too.

### Oracle

To calculate an upper bound per SMT system per dataset, we calculate character-level LD between each candidate hypothesis in the \(10\)-best list and the gold corrected sentence. We then calculate an oracle score by selecting the candidate hypothesis that has the smallest LD. Essentially the oracle is telling us the maximum performance that can be obtained with the given \(10\)-best list on each dataset. For datasets for which we have more than one annotation available, we select the oracle that gives the highest F\({}_{0.5}\).

In Table 2, we can see that, overall, CAMB16SMT has a higher oracle performance compared to AMU16SMT. More specifically, the maximum attainable F\({}_{0.5}\) on FCE is \(71.60\), on CoNLL \(58.13\), and on JFLEG \(61.92\). This shows empirically that the \(10\)-best list has great potential and should be exploited further. AMU16SMT has a lower oracle performance overall, though again this can be attributed to the fact that it is specifically tuned for CoNLL.

### \(N\)-best list size

Next, we examine performance as the \(N\)-best list varies in size, ranging from \(1\) to \(10\) (Table 4). We observe a positive effect: the larger the size, the better the model for all datasets. F\({}_{0.5}\) does not seem to have reached a plateau with \(n<10\), which suggests that increasing the size of the list further can potentially lead to better results. We do, however, observe that large improvements are obtained when increasing the size from \(1\) to \(3\), suggesting that, most of the time, better alternatives are identified within the top 3 candidate hypotheses. This, however, is not the case for the oracle (F\({}^{\text{oracle}}_{0.5}\)), which consistently increases as \(n\) gets larger.

\begin{table}
\begin{tabular}{l|r r} \hline \multicolumn{3}{c}{CAMB16SMT + LSTMcamb} \\ \hline Ablated feature & F\({}_{0.5}\) & GLEU \\ \hline None & 55.60 & 71.76 \\ Sentence probability & 54.13 & 70.65 \\ Levenshtein distance & 55.42 & 71.78 \\ True/false positives & 55.14 & 71.75 \\ \hline \end{tabular}
\end{table}
Table 3: Ablation tests on the FCE test set when removing one feature of the re-ranking system at a time.

 



### Error type performance

In Table 5, we can see example source sentences, together with their corrected counterparts (reference), 1-best candidates by CAMB16\({}_{\text{SMT}}\) and 1-best candidates by CAMB16\({}_{\text{SMT}}\) + LSTM\({}_{\text{camb}}\). Re-ranking seems to fix errors such as subject-verb agreement ("the Computer help" to "the computer helps") and verb form ("I recommend you to visit" to "I recommend visiting"). In this section, we perform an analysis of performance per type to get a better understanding of where the strength of the re-ranking detection model comes from.

Until recently, GEC performance per error type was only analysed in terms of recall, as system output is not annotated. Recently, however, Bryant et al. (2017) proposed an approach to automatically annotating GEC output with error type information, which utilises a linguistically-enhanced alignment to automatically extract the edits between pairs of source sentences and their corrected counterparts, and a dataset-independent rule-based classifier to classify the edits into error types. Human evaluation showed that the predicted error types were rated as "Good" or "Acceptable" 95% of the time. We use their publicly available code to analyse per-error-type performance before and after re-ranking.

Table 6 presents the performance for a subset of error types that are affected the most before and after re-ranking CAMB16\({}_{\text{SMT}}\) on the FCE test set. The error types are interpreted as follows: Missing error; **R**eplace error; **U**nnecessary error. The largest improvement is observed in replacement errors referring to possessive nouns (R:NOUN:POSS) and verb agreement (R:VERB:SVA); and in unnecessary errors referring to adverbs (U:ADV), determiners (U:DET), pronouns (U:PRON), and verb tense (U:VERB:TENSE).

The LSTM architecture allows the network to learn advanced composition rules and remember dependencies over longer distances (e.g., R:VERB:SVA improves from \(58.38\) to \(69.40\)). The network's language modelling objectives allow it to learn better and more general compositional features (e.g., U:ADV improves from \(13.51\) to \(22.73\)), while the character-level architecture facilitates modelling of morphological patterns [e.g., replacement errors referring to verb form (R:VERB:FORM) improve from \(53.62\) to \(58.06\)]. Between M, R, and U errors, the largest improvement is observed in U, for which there is at least 5% improvement in F\({}_{0.5}\).7

Footnote 7: U improves from \(38.44\) to \(43.77\); M from \(43.43\) to \(45.40\); R from \(53.25\) to \(55.33\).

Overall, re-ranking improves F\({}_{0.5}\) across error types; however, there i

\begin{table}
\begin{tabular}{l|c c c c|c c} \hline  & \multicolumn{4}{c|}{CAMB16\({}_{\text{SMT}}\)} \\ \hline  & \multicolumn{3}{c|}{FCE test set} & CoNLL test set & JFLEG \\ \hline N-best list & F\({}_{0.5}\) & P\({}^{\text{scale}}_{0.5}\) & P\({}^{\text{scale}}_{0.5}\) & P\({}^{\text{scale}}_{0.5}\) & P\({}^{\text{scale}}_{0.5}\) & P\({}^{\text{scale}}_{0.5}\) \\ \hline
1 & 52.90 & 52.90 & 37.33 & 37.33 & 52.44 & 52.44 \\
2 & 54.28 & 62.10 & 39.87 & 47.03 & 53.04 & 55.64 \\
3 & 54.96 & 65.93 & 41.00 & 51.12 & 53.24 & 57.69 \\
4 & 55.18 & 68.05 & 41.83 & 53.13 & 53.47 & 58.93 \\
5 & 55.33 & 69.47 & 42.12 & 54.80 & 54.01 & 59.66 \\
6 & 55.43 & 70.24 & 42.49 & 55.53 & 54.18 & 60.34 \\
7 & 55.48 & 70.74 & 42.60 & 56.54 & 54.45 & 61.14 \\
8 & 55.47 & 71.00 & 42.63 & 57.04 & 54.47 & 61.28 \\
9 & 55.51 & 71.27 & **42.65** & 57.23 & 54.64 & 61.55 \\
10 & **55.60** & 71.60 & 42.44 & 58.13 & **54.66** & 61.92 \\ \hline \end{tabular}
\end{table}
Table 4: Re-ranking performance using LSTM\({}_{\text{camb}}\) as the \(N\)-best list varies in size from \(1\) to \(10\) for CAMB16\({}_{\text{SMT}}\) and its oracle.

\begin{table}
\begin{tabular}{|p{142.3pt}|} \hline \(Source\) \\ \hline Iwork with children on the Computer help my Job bat affected to \\ \hline _CAMB16\({}_{\text{SMT}}\)_ \\ \hline IV work with children and the Computer help my Job bat affected to \\ \hline _CAMB16\({}_{\text{SMT}}\)_ + LSTM\({}_{\text{camb}}\)_ \\ \hline Iwork with children and the **computer helps** my Job bat affected to \\ \hline _Reference_ \\ \hline IV work with children and the computer helps me in my job but affects it too. \\ \hline _Source_ \\ \hline It takes 25 minutes that is convenient to us \\ \hline _CAMB16\({}_{\text{SMT}}\)_ \\ \hline It takes 25 minutes that is convenient for us \\ \hline _CAMB16\({}_{\text{SMT}}\)_ + LSTM\({}_{\text{camb}}\)_ \\ \hline It takes 25 minutes , **which** is convenient for us \\ \hline _Reference_ \\ \hline It takes 25 minutes , which is convenient for us \\ \hline \end{tabular} 
\begin{tabular}{|p{142.3pt}|} \hline \(Camb16\({}_{\text{SMT}}\) \\ \hline \(Camb16\({}_{\text{SMT}}\) \\ \hline \(Camb16\({}_{\text{SMT}}\) + LSTM\({}_{\text{camb}}\) \\ \hline \(Recommcal\)) \\ \hline Ireference) \\ \hline Especially youngsters misuse this kind of invention \\ \hline _CAMB16\({}_{\text{SMT}}\)_ \\ \hline Especially youngsters misuse this kind of invention \\ \hline _CAMB16\({}_{\text{SMT}}\) + LSTM\({}_{\text{camb}}\)_ \\ \hline _In particular_ , youngers misuse this kind of invention \\ \hline _Reference_ \\ \hline Especially youngsters misuse this kind of invention \\ \hline \end{tabular}
\end{table}
Table 5: Source sentences along with gold corrections (reference), 1-best candidates by CAMB16\({}_{\text{SMT}}\) and by CAMB16\({}_{\text{SMT}}\) + LSTM\({}_{\text{camb}}\)

negatively affected (Table 6, bottom part); for example, performance on missing errors referring to verb form (M:VERB:FORM) drops from \(50.00\) to \(38.46\), and on replace contraction errors (R:CONTR) from \(50.00\) to \(27.78\). Importantly, such an analysis allows us to examine the strengths and weaknesses of the models, which is key for the deployment of GEC systems.

## 7 Conclusion

To the best of our knowledge, no prior work has investigated the impact of detection models on correction performance. We proposed an approach to \(N\)-best list re-ranking using a neural sequence-labelling model that calculates the probability of each token in a sentence being correct or incorrect in context. Detection models can be more fine-tuned to finer nuances of grammaticality, and therefore better able to distinguish between correct and incorrect versions of a sentence. Using a linear combination of a small set of features derived from the detection model output, we re-ranked the \(N\)-best list of SMT systems and achieved state-of-the-art results on GEC on three different datasets. Our approach can be applied to any GEC system that produces multiple alternative hypotheses. Our results demonstrate the benefits of integrating detection approaches with correction systems, and how one can complement the other.

## Acknowledgments

Special thanks to Christopher Bryant, Mariano Felice, and Ted Briscoe, as well as the anonymous reviewers for their valuable contributions at various stages.

## References

* Brockett et al. (2006) Chris Brockett, William B Dolan, and Michael Gamon. 2006. Correcting ESL errors using phrasal SMT techniques. In _Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics_, pages 249-256. Association for Computational Linguistics.
* Bryant et al. (2017) Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
* Chodorow et al. (2007) Martin Chodorow, Joel R Tetreault, and Na-Rae Han. 2007. Detection of grammatical errors involving prepositions. In _Proceedings of the fourth ACLISGEM workshop on prepositions_, pages 25-30. Association for Computational Linguistics.
* Chollampatt et al. (2016a) Shamil Chollampatt, Duc Tam Hoang, and Hwee Tou Ng. 2016a. Adapting grammatical error correction based on the native language of writers with neural network joint models. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_, pages 1901-1911.
* Chollampatt et al. (2016b) Shamil Chollampatt, Kaveh Taghipour, and Hwee Tou Ng. 2016b. Neural network translation models for grammatical error correction. _arXiv preprint arXiv:1606.00189_.
* Dahlmeier and Ng (2011) Daniel Dahlmeier and Hwee Tou Ng. 2011. Correcting semantic collocation errors with L1-induced paraphrases. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_, pages 107-117. Association for Computational Linguistics.
* Dahlmeier and Ng (2012a) Daniel Dahlmeier and Hwee Tou Ng. 2012a. A beamsearch decoder for grammatical error correction. In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_, pages 568-578. Association for Computational Linguistics.

\begin{table}
\begin{tabular}{l|c|c} \hline  & CAMBL16SMT & CAMBL16SMT + LSTM\({}_{\text{camb}}\) \\ \hline Type & F\({}_{0.5}\) & F\({}_{0.5}\) \\ \hline M:ADV & 25.00 & 31.25 \\ M:VERB & 25.42 & 29.85 \\ R:NOUN:NUM & 56.60 & 62.50 \\ R:NOUN:POSS & 35.71 & **55.56** \\ R:OTHER & 34.99 & 38.75 \\ R:PRON & 26.88 & 33.33 \\ R:VERB:FORM & 53.62 & 58.06 \\ R:VERB:SVA & 58.38 & **69.40** \\ R:VERB:TENSE & 31.94 & 36.29 \\ U:ADV & 13.51 & **22.73** \\ U:DET & 46.27 & **55.30** \\ U:NOUN & 10.10 & 15.72 \\ U:PREP & 47.62 & 53.40 \\ U:PRON & 30.77 & **39.33** \\ U:PUNCT & 51.22 & 58.38 \\ U:VERB:TENSE & 28.41 & **41.67** \\ \hline M:PEP & 43.69 & 39.43 \\ M:VERB:FORM & 50.00 & **38.46** \\ R:ADJ & 45.45 & 37.67 \\ R:CONTR & 50.00 & **27.78** \\ R:WO & 53.63 & 48.74 \\ \hline \end{tabular}
\end{table}
Table 6: Error-type performance before and after re-ranking on the FCE test set (largest impact highlighted in bold; bottom part of the table displays negative effects on performance).

 

Daniel Dahlmeier and Hwee Tou Ng. 2012b. Better evaluation for grammatical error correction. In _Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 568-572. Association for Computational Linguistics.
* Dahlmeier et al. (2013) Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner english: The NUS corpus of learner English. In _Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications_, pages 22-31.
* Dale and Kilgarriff (2011) Robert Dale and Adam Kilgarriff. 2011. Helping our own: The HOO 2011 pilot shared task. In _Proceedings of the 13th European Workshop on Natural Language Generation_, pages 242-249. Association for Computational Linguistics.
* De Felice and Pulman (2008) Rachele De Felice and Stephen G Pulman. 2008. A classifier-based approach to preposition and determiner error correction in L2 English. In _Proceedings of the 22nd International Conference on Computational Linguistics_, pages 169-176. Association for Computational Linguistics.
* Felice et al. (2014) Mariano Felice, Zheng Yuan, Oistein E. Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. 2014. Grammatical error correction using hybrid systems and type filtering. In _Proceedings of the 18th Conference on Computational Natural Language Learning: Shared Task_, pages 15-24. Association for Computational Linguistics.
* Gamon (2010) Michael Gamon. 2010. Using mostly native data to correct errors in learners' writing: a meta-classifier approach. In _Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics_, pages 163-171. Association for Computational Linguistics.
* Gamon et al. (2008) Michael Gamon, Jianfeng Gao, Chris Brockett, Alexandre Klementiev, William B Dolan, Dmitriy Belenko, and Lucy Vanderwende. 2008. Using contextual speller techniques and language modeling for ESL error correction. In _IJCNLP_, volume 8, pages 449-456.
* Han et al. (2006) Na-Rae Han, Martin Chodorow, and Claudia Leacock. 2006. Detecting errors in English article usage by non-native speakers. _Natural Language Engineering_, 12(1):115-129.
* Han et al. (2010) Na-Rae Han, Joel Tetreault, Soo-Hwa Lee, and JinYoung Ha. 2010. Using error-annotated ESL data to develop an ESL error correction system. In _Proceedings of LREC. Emi Izumi, Kiyotaka Uchimoto and Hitoshi Isahara_.
* Hoang et al. (2016) Duc Tam Hoang, Shamil Chollampatt, and Hwee Tou Ng. 2016. Exploiting n-best hypotheses to improve an smt approach to grammatical error correction. _arXiv preprint arXiv:1606.00210_.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural Computation_, 9.
* Junczys-Dowmunut and Grundkiewicz (2016) Marcin Junczys-Dowmunut and Roman Grundkiewicz. 2016. Phrase-based machine translation is state-of-the-art for automatic grammatical error correction. _arXiv preprint arXiv:1605.06353_.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: a method for stochastic optimization. In _International Conference on Learning Representations_.
* Koehn et al. (2003) Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In _Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology_, pages 48-54. Association for Computational Linguistics.
* Leacock et al. (2014) Claudia Leacock, Martin Chodorow, Michael Gamon, and Joel Tetreault. 2014. Automated grammatical error detection for language learners, second edition. _Synthesis lectures on human language technologies_, 7(1):1-170.
* Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In _Proceedings of the International Conference on Learning Representations (ICLR 2013)_.
* Mizumoto et al. (2012) Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, Masaaki Nagata, and Yu Matsumoto. 2012. The effect of learner corpus size in grammatical error correction of ESL writings. In _24th International Conference on Computational Linguistics_, pages 863-872.
* Mizumoto and Matsumoto (2016) Tomoya Mizumoto and Yuji Matsumoto. 2016. Discriminative reranking for grammatical error correction with statistical machine translation. In _Proceedings of the North American Chapter of the Association for Computational Linguistics_, pages 1133-1138.
* Napoles et al. (2015) Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground truth for grammatical error correction metrics. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing_, volume 2, pages 588-593.
* Napoles et al. (2017) Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. JFLEG: A fluency corpus and benchmark for grammatical error correction. _arXiv preprint arXiv:1702.04066_.
* Ng et al. (2014) Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadwinnotto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In _Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task_, pages 1-14, Baltimore, Maryland. Association for Computational Linguistics 

Hwee Tou Ng, Siew Mei Wu, Yuanbin Wu, Christian Hadwinoto, and Joel Tetreault. 2013. The CoNLL-2013 shared task on grammatical error correction. In _Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task_, pages 1-12. Association for Computational Linguistics.
* error coding and analysis for lexicography and ELT. In _Proceedings of the Corpus Linguistics 2003 Conference_.
* Och and Ney (2002) Franz Josef Och and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_, pages 295-302. Association for Computational Linguistics.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting on association for computational linguistics_, pages 311-318. Association for Computational Linguistics.
* Rei (2017) Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
* Rei et al. (2016) Marek Rei, Gamal K. O. Crichton, and Sampo Pyysalo. 2016. Attending to characters in neural sequence labeling models. In _Proceedings of the 26th International Conference on Computational Linguistics (COLING-2016)_.
* Rei and Yannakoudakis (2016) Marek Rei and Helen Yannakoudakis. 2016. Compositional sequence labeling models for error detection in learner writing. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics_.
* Rozovskaya et al. (2014) Alla Rozovskaya, Kai-Wei Chang, Mark Simmons, Dan Roth, and Nizar Habash. 2014. The Illinois-Columbia system in the CoNLL-2014 shared task. In _Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task_, pages 34-42. Association for Computational Linguistics.
* Rozovskaya and Roth (2010) Alla Rozovskaya and Dan Roth. 2010. Generating confusion sets for context-sensitive error correction. In _Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing_, pages 961-970. Association for Computational Linguistics.
* Rozovskaya and Roth (2011) Alla Rozovskaya and Dan Roth. 2011. Algorithm selection and model adaptation for ESL correction tasks. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 924-933. Association for Computational Linguistics.
* Rozovskaya and Roth (2016) Alla Rozovskaya and Dan Roth. 2016. Grammatical error correction: Machine translation and classifiers. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics_, pages 2205-2215.
* Rozovskaya et al. (2011) Alla Rozovskaya, Mark SAMmons, Joshua Gioja, and Dan Roth. 2011. University of Illinois system in HOO text correction shared task. In _Proceedings of the 13th European Workshop on Natural Language Generation_, pages 263-266. Association for Computational Linguistics.
* Rozovskaya et al. (2012) Alla Rozovskaya, Mark SAMmons, and Dan Roth. 2012. The UI system in the HOO 2012 shared task on error correction. In _Proceedings of the Seventh Workshop on Building Educational Applications Using NLP_, pages 272-280. Association for Computational Linguistics.
* Susanto et al. (2014) Raymond Hendy Susanto, Peter Phandi, and Hwee Tou Ng. 2014. System combination for grammatical error correction. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_, pages 951-962.
* Taijiri et al. (2012) Yoshikazu Taijiri, Mamoru Komachi, and Yuji Matsumoto. 2012. Tense and aspect error correction for ESL learners using global context. In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2_, pages 198-202. Association for Computational Linguistics.
* Tetreault et al. (2010) Joel Tetreault, Jennifer Foster, and Martin Chodorow. 2010. Using parse features for preposition selection and error detection. In _Proceedings of the ACL 2010 conference short papers_, pages 353-358. Association for Computational Linguistics.
* Tetreault and Chodorow (2008) Joel R Tetreault and Martin Chodorow. 2008. The ups and downs of preposition error detection in ESL writing. In _Proceedings of the 22nd International Conference on Computational Linguistics_, pages 865-872. Association for Computational Linguistics.
* Yannakoudakis et al. (2011) Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading esol texts. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 180-189. Association for Computational Linguistics.
* Yoshimoto et al. (2013) Ippei Yoshimoto, Tomoya Kose, Kensuke Mitsuzawa, Keisuke Sakaguchi, Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, and Yuji Matsumoto. 2013. NAIST at 2013 CoNLL grammatical error correction shared task. In _CoNLL Shared Task_, pages 26-33.
* Yuan and Briscoe (2016) Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In _Proceedings of the North American Chapter of the Association for Computational Linguistics_, pages 380-386 

* Yuan et al. (2016) Zheng Yuan, Ted Briscoe, and Mariano Felice. 2016. Candidate re-ranking for SMT-based grammatical error correction. In _Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications_, pages 256-266.
* Yuan and Felice (2013) Zheng Yuan and Mariano Felice. 2013. Constrained grammatical error correction using statistical machine translation. In _Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task_, pages 52-61.