m E d IT : Multilingual Text Editing via Instruction Tuning (2024)

Abstract: We introduce mEdIT, a multi-lingual extension to CoEdIT – the recent state-of-the-art text editing models for writing assistance. mEdIT models are trained by fine-tuning multi-lingual large, pre-trained language models (LLMs) via instruction tuning. They are designed to take instructions from the user specifying the attributes of the desired text in the form of natural language instructions, such as “Grammatik korrigieren” (German) or “이 텍스 트를 단순화” (Korean). We build mEdIT by curating data from multiple publicly available human-annotated text editing datasets for three text editing tasks (Grammatical Error Correction (GEC), Text Simplification, and Paraphrasing) across diverse languages belonging to six different language families. We detail the design and training of mEdIT models and demonstrate their strong performance on many multi-lingual text editing benchmarks against other multilingual LLMs. We also find that mEdIT generalizes effectively to new languages over multilingual baselines. We publicly release our data, code, and trained models.

## introduction large language models (llms) have made remarkable progress toward generating fluent and coherent text in a wide variety of tasks and domains to support writing assistance @xcite @xcite ; inter alia). in particular, llms have been adapted to perform many complex text editing tasks like gec @xcite @xcite ), text simplification @xcite , paraphrasing @xcite , and formality and tone rewriting @xcite , among others. however, most ## quantitative results: mte quality we split the models into three main groups. the first group (a) consists of the "no edits" baseline, the second group (b) is the untrained baseline, where models are evaluated in a zero-shot setting without any task-specific fine-tuning, while the third group (c) is our set of multi-lingual models trained on task-specific datasets. for all our experiments, we aggregate the models' performance by text editing tasks. specifically, we aggregate the metrics for each task using the harmonic means of its constituents. specifically, we use (1-self-bleu) 4 and semantic accuracy for paraphrasing, sari and bleu for simplification, and f 0.5 and gleu for gec. we scale all metrics to lie between figure 3 : overall performance comparison of all baselines against trained models. we calculate the aggregated performance across all tasks using the harmonic mean of task-specific scores. baselines are "no edits (copy)", "english-only" ("-en,"), and our trained models are marked as "clm" and "seq2seq," respectively. the aggregated performance is calculated as described in § 5. 0 and 100. we show full results on all models in appendix c for the best-performing setup. ## human evaluations we conduct human evaluations of our model outputs by proficient linguists (and native language speakers of the respective languages) on 50 test inputs (per task-language) to ensure they meet the instructional task-specific constraints across the various languages since text editing is often subjective, and automatic metrics are often limited in their effectiveness and accuracy. we evaluate our best-performing medit model foot_6 based on its strong performance ( ## conclusions we present medit -an open-sourced dataset and set of multilingual instruction-tuned llms capable of following natural language instructions in seven languages to perform various textual editing tasks. it is the first publicly available set of models that heavily outperforms numerous multilingual llms on multiple tasks in different languages. positive feedback from human evaluations shows that medit can assist writers with various aspects of the text revision process at scale by following natural language instructions in multiple languages. experiments on various multilingual nlp tasks demonstrate that medit models outperform both their corresponding non-fine-tuned and fine-tuned models on other multilingual instruction datasets for text editing, in addition to achieving strong performance on languages unseen during the taskspecific fine-tuning. by making our data/models publicly available, we hope to help make advances in multilingual intelligent writing assistants.