To Err Is Human, but Llamas Can Learn It Too (2024)

Abstract: This study explores enhancing grammatical error correction (GEC) through automatic error generation (AEG) using language models (LMs). Specifically, we fine-tune Llama 2 LMs for error generation and find that this approach yields synthetic errors akin to human errors. Next, we train GEC Llama models using these artificial errors and outperform previous state-of-the-art error correction models, with gains ranging between 0.8 and 6 F0.5 points across all tested languages (German, Ukrainian, and Estonian). Moreover, we demonstrate that generating errors by fine-tuning smaller sequence-to-sequence models and prompting large commercial LMs (GPT3.5 and GPT4) also results in synthetic errors beneficially affecting error generation models. We openly release trained models for error generation and correction as well as all the synthesized error datasets for the covered languages.

## introduction the grammatical error correction (gec) task aims to correct spelling and grammatical errors in text, making it valuable for a wide range of people. the best-performing gec approaches currently use deep learning models @xcite @xcite , and several others), which are known to be datahungry. simultaneously, the availability of openly accessible error correction data is severely limited, even for languages typically considered highresource in other tasks, such as german and arabic @xcite . this lack of data complicates the development of effective gec systems for these and other even less-resourced languages. the scarcity of gec data is commonly addressed through the creation of synthetic data, where errors are automatically added into correct sentences -also called artificial error generation @xcite . in low-resource settings, the overwhelmingly most employed approach for aeg is applying random probabilistic perturbation (deletion, insertion, replacement) of words and/or characters @xcite @xcite . alternatives include usage of intricate hand-crafted rules and confusion sets @xcite @xcite and automatically learning to generate errors @xcite @xcite -also referred to as back-translation (bt) * . however, to the best of our knowledge, none of the related work on aeg uses pre-trained foundation models or applies this methodology in a low-resource setting. this gap is precisely the focus of the present work: we are using pre-trained language models for synthetic error generation and demonstrate the simplicity and effectiveness of the approach in lowresource scenarios. we approach the task by finetuning open large language models (llms) based on llama 2 @xcite for error generation and correction, resulting in quality aeg data and state-of-the-art gec models even when very limited human error data is available. our analysis shows that the resulting errors can be categorized similarly to human errors. we also compare finetuning approach to prompting commercial llms (gpt-3.5 and gpt-4: openai, 2023) to perform aeg, as well as include other open models commonly employed for gec and tune them for aeg: mt5 @xcite and nllb @xcite . our final goal and evaluation setting is improving gec for languages with limited gec data. in particular, we focus on german, ukrainian, and estonian gec. when pre-trained on our llmgenerated synthetic errors, the resulting gec mod-els achieve the best current results on the included benchmarks in all three evaluated cases, including previous state-of-the-art and 4-shot gpt-4. we publicly release aeg and gec models from our work and the generated data. the datasets include one million sentences for german, ukrainian, and estonian, each processed with three different models, as well as an additional set of 100k sentences with gpt models. in summary, our contributions are as follows: the paper is structured as follows. we outline related work in section 2, methodology experimental settings in section 3, and results in section 4. additional questions on the same topic are discussed in section 6 and the paper is concluded in section 5. ## related work the use of synthetic data is a common concept in gec. the first effective neural method proposed by @xcite approaches gec as low-resource machine translation (mt) translating from erroneous text to correct text, making it a relatively resource-heavy method encouraging synthetic data generation. over the years, there have been different approaches to deliberately introducing errors into monolingual text, like rule-based and probabilistic methods, methods based on confusion sets and error patterns, models trained for error generation and using round-trip translation @xcite . one widely adopted approach to generating synthetic data involves the probabilistic addition of † github.com/tartunlp/gec-llm errors to monolingual corpora. this technique encompasses inserting, deleting, substituting, or moving characters or words without considering the context, as described by @xcite , and @xcite . additionally, @xcite introduced a "reverse speller" approach that suggests word replacements from confusion sets based on the speller's corrections. this method has been applied to several languages such as german, czech, russian, ukrainian, icelandic and estonian @xcite @xcite @xcite . as we show later, errors generated with the context-free probabilistic method differ from human errors and thus cover a much smaller number of error types, shown by significantly lower gec recall. learned methods of error generation typically require more resources. before the widespread adoption of transformers and mt, various studies explored alternative approaches for training models for error generation. for instance, @xcite and rei et al. (2017) utilized statistical machine translation to generate errors, while xie et al. (2018) and yuan et al. (2019) experimented with cnns for this purpose. additionally, kasewa et al. (2018) investigated using rnn-based sequence-tosequence models with attention mechanisms. moving towards more modern mt architectures, htut and tetreault (2019) tested various model frameworks, including transformers, and kiyono et al. (2019) specifically employed transformer models. both of the latter studies trained models from scratch, utilizing datasets ranging from approximately 500,000 to over a million error correction examples to train the aeg system. in contrast, our work generates up to 1 million sentences with synthetic error while using between 9k and 33k human error sentences to fine-tune the base models. during the last few years, there has been no one error-generation method that has proved its superiority. it depends on language and available resources. for english @xcite train seq2edit models (stahlberg and kumar, 2020) from scratch for learning to create diverse sets of errors. as mentioned in the beginning, synthetic probabilistic errors have found wide use for different languages. for instance, ingólfsdóttir et al. (2023) combine probabilistic character/word permutations with a rule-based approach for icelandic and kara et al. (2023) curate special rules for generating turkish data. to the best of our knowledge, no works focus specifically on error generation by llms; however, several studies have evaluated the performance of commercial llms in this task. @xcite found that while gpt-3.5 performs significantly worse than other systems in terms of precision, it excels in recall. similar results were reported by @xcite , who observed that gpt models tend to overcorrect rather than undercorrect errors. this finding is also supported by @xcite , who noted that gpt models are particularly strong at making fluency edits. while there are few studies that use or evaluate open-source llms, zhang et al. (2023) explore the use of the llama model @xcite for writing assistance. next, we present the key methodological details of our work. ## methodology and experiments the primary target of our work is to apply generative language models to aeg via fine-tuning. additionally, we experiment with prompting llms to perform the same task and include two seq2seq models that are fine-tuned to do the same. the efficiency of proposed aeg solutions is evaluated by using them to improve gec. thus, we also fine-tune generative llms to perform the gec task and compare the results to promptingbased gec results and related work. the general pipeline of our approach is straight-forward: ## results in this section, we evaluate the performance of llama-based models for gec and aeg tasks. we then compare the aeg effectiveness between nllb and mt5 models against llama-based models to see if smaller, more efficient models can generate quality data. separately, we assess aeg through prompting with gpt-3.5-turbo versus llama models with trained error generation. finally, we examine the quality of generated errors against human data and probabilistic reversespeller errors and compare the error type distributions for estonian. ## conclusion in conclusion, our research demonstrates the significant potential of llama-based llms in addressing the challenges of gec for low-resource settings. we have successfully developed state-of-the-art systems for estonian, ukrainian, and german by leveraging these models as both correctors and synthetic data generators. we also explore other methods for aeg and show that prompting stronger commercial llms is another way of generating high-quality data, and fine-tuning smaller models also has potential when the resources are more limited.