A Crash Course in Automatic Grammatical Error Correction (2020)

Abstract: Grammatical Error Correction (GEC) is the task of automatically detecting and correcting all types of errors in written text. Although most research has focused on correcting errors in the context of English as a Second Language (ESL), GEC can also be applied to other languages and native text. The main application of a GEC system is thus to assist humans with their writing. Academic and commercial interest in GEC has grown significantly since the Helping Our Own (HOO) and Conference on Natural Language Learning (CoNLL) shared tasks in 2011-14, and a record-breaking 24 teams took part in the recent Building Educational Applications (BEA) shared task. Given this interest, and the recent shift towards neural approaches, we believe the time is right to offer a tutorial on GEC for researchers who may be new to the field or who are interested in the current state of the art and future challenges. With this in mind, the main goal of this tutorial is not only to bring attendees up to speed with GEC in general, but also examine the development of neural-based GEC systems.

## content the tutorial will begin by introducing the task and describing some of its challenges in terms of the complexity of natural languages and technical limitations. for example, closed class errors, such as prepositions, are often easier to correct than open class errors, such as content words, because the confusion sets are smaller and they tend to require less semantic knowledge. it will then continue by providing an overview of the evolution of approaches to the problem, including: after the break, the tutorial will continue by focusing on the development of state-of-the-art neural gec systems @xcite . in particular, although smt systems outperformed nmt systems for much of the past five years, a number of techniques, such as transfer learning from monolingual data, strong regularization, model ensembling, and using a large-scale language model, have closed this gap. we will also highlight the main challenges in neural gec -data sparsity and sub-optimal decoding -and describe several proposed techniques to overcome them @xcite @xcite @xcite @xcite @xcite . finally, we will close the tutorial by discussing the findings of the bea-2019 shared task @xcite , and outlining future work and research directions. in particular, we will highlight the advent of low resource gec, which strives to make it possible to build gec systems for any language without human-annotated training data ( cf. bryant and briscoe (2018), stahlberg et al. (2019), alikaniotis et al. (2019) ). ## tutorial outline a brief outline of the tutorial: ## prerequisites and reading list a basic knowledge of machine learning (e.g. familiarity with precision, recall, f-score metrics, and ensemble methods) and neural approaches to natural language processing (e.g. the concept of sequenceto-sequence encoder-decoder neural models) will be helpful to understand the content of the tutorial. suggested reading list: christopher bryant 4 is a research associate in the alta institute, 5 based in the department of computer science and technology at the university of cambridge. his research interests include automatic grammatical error detection and correction, automatic corpus annotation, and computer aided language learning (call). he completed his phd on "automatic annotation for grammatical error correction" in 2019 and is the lead developer of the associated error annotation toolkit (errant). most recently, he also led the organisation of the bea shared task on gec in 2019, and has previously given a tutorial on gec to an international audience at the first summer school on machine learning for digital english language teaching in 2017. 6 mariano felice