Introducing O mni GEC : A Silver Multilingual Dataset for Grammatical Error Correction (2025)

Abstract: In this paper, we introduce OmniGEC, a collection of multilingual silver-standard datasets for the task of Grammatical Error Correction (GEC), covering eleven languages: Czech, English, Estonian, German, Greek, Icelandic, Italian, Latvian, Slovene, Swedish, and Ukrainian. These datasets facilitate the development of multilingual GEC solutions and help bridge the data gap in adapting English GEC solutions to multilingual GEC. The texts in the datasets originate from three sources: Wikipedia edits for the eleven target languages, subreddits from Reddit in the eleven target languages, and the Ukrainian-only UberText 2.0 social media corpus. While Wikipedia edits were derived from human-made corrections, the Reddit and UberText 2.0 data were automatically corrected with the GPT-4o-mini model. The quality of the corrections in the datasets was evaluated both automatically and manually. Finally, we fine-tune two open-source large language models — Aya-Expanse (8B) and Gemma-3 (12B) — on the multilingual OmniGEC corpora and achieve state-of-the-art (SOTA) results for paragraph-level multilingual GEC. The dataset collection and the best-performing models are available on Hugging Face.

## introduction the english gec spearheaded advancements in gec, and some of the developed methods and approaches can be directly applied to other languages. for instance, the authors of the recent survey paper @xcite mention that for ensembling and ranking the results, a high diversity between possible corrections results in higher scores. this approach can be applied and validated for a variety of languages. at the same time, many solutions are english-centric and unadjustable to other languages, creating language bias @xcite . for example, the gector model @xcite , used for ranking the proposed grammatical corrections, is specifically trained to work with english, and its adaptation to other languages would be extremely high-effort. with the introduction of transformer-based models @xcite and modern large language models (llms), the landscape in modern gec shifted drastically @xcite : (1) synthetic data generation has started to be used more often to rely less on high-quality parallel data @xcite , and (2) open-source llms opened new possibilities to approach the gec task with various prompting and fine-tuning techniques @xcite . these models and methods have been successfully applied to the english language but have not been validated in the multilingual setting. ## data in this section, we describe the creation of the omnigec datasets that cover eleven languages: czech, english, estonian, german, greek, icelandic, italian, latvian, slovene, swedish, and ukrainian. the language selection was based on the multigec-2025 shared task for further data and model comparability. for consistency in data-related measurements, we employ gpt-4o & gpt-4o-mini's tokenizer (openai, 2024a), and for model-related technicalities, we use gemma-3 and aya-expanse's tokenizers respectively. ## quality evaluation to assess the quality of corrections in the om-nigec datasets, we used automated metrics and human feedback. we evaluated only the ukrainianlanguage subcorpora due to time and human resource constraints and acknowledge the need for a further multilingual assessment. nevertheless, we believe that the evaluation results still provide insights into the quality of corrections in the dataset. for both evaluation tracks, we sampled 1,500 random examples from each of the three subcorpora, which totalled in 4,500 samples for evaluation. ## experiments in this section, we experiment with the omnigec dataset in the setting of the multigec-2025 shared task. ## results in this section, we explore the results of our experiments, which include model performance in two multigec-2025 tracks and the performance changes with the addition of omnigec training data. ## conclusions in this research, we presented the omnigec collection of multilingual silver-standard gec corpora. we found that including more silver-grade training data improves accuracy in multilingual gec. we demonstrated the performance increase by training aya-expanse (8b) and gemma-3-12b-it models on multigec and omnigec datasets, which yielded the best results for paragraph-based editing models outperforming previous leaders trained solely on multigec data. aya-expanse (8b), being a smaller model with fewer excess languages, adapted more easily to the multilingual gec but has its limitations, like fewer relevant pre-training languages. these limitations can be addressed through fine-tuning on large-scale datasets in the target languages. gemma-3-12b-it, a larger model, despite having more parameters, yielded worse results than aya-expanse when trained solely on a small golden gec dataset but after adding a large silver dataset for fine-tuning, outperformed aya-expanse and established a new paragraph-based editing sota score. we publish omnigec and processing pipelines to open-source and expect omnigec to be continuously updated with new data, growing both in new samples and languages. the reddit-multigec and wikiedits-multigec subcorpora can be continuously updated with corrections. together with our exploratory work, these resources aim to facilitate new developments in multilingual gec with new models, approaches, and techniques. in future work, we plan to further research multilingual gec by assessing more models, sentencebased editing, which yielded better results in the multigec-2025 shared task, and preference optimization methods, like dpo @xcite , made possible in this task with prepared human-in-the-loop scores in omnigec. on top of that, the ablation studies will be an important area for future research: (a) more thorough research on data quantity versus quality with ubertext-gec, which includes nearly 10 times more language data than reddit-multigec for the ukrainian case study, and (b) per-language lora adapters to unveil the cross-language relationships, if any. finally, we expect the ubertext-gec case study to trailblaze research toward the sota ukrainian gec model in both paragraph-based and sentencebased editing. we expect all these methods to easily adapt to other languages, improving multilingual gec scores.