M 2 L ingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models (2025)

Abstract: Collecting instruction fine-tuning (IFT) data is a resource and time intensive task especially in multilingual setting where finding proficient native speakers is challenging. Moreover, traditional data collection is prone to privacy risks, toxicity and lacks scalability. While, fully synthetic datasets are a promising alternative, research on their use in multilingual domain is limited as existing approaches still rely on machine translation to improve multilingual performance. To bridge this gap we introduce M2Lingual, the first fully synthetic, multi-turn multilingual dataset having 175K conversations across 70 languages with a balanced mix of high, low and mid-resourced languages. M2Lingual is constructed using a cost-efficient and scalable method that uses our novel two-step Evol prompt taxonomy to transform a small set of human written instructions to complex and challenging conversations. Results across three model families, six baseline datasets and evaluation spanning 31 languages demonstrates the effectiveness of M2Lingual over other datasets.

## introduction the recent success of large language models (llms) @xcite @xcite can be largely attributed to the availability of large, diverse, and high quality instruction fine-tuning (ift) datasets @xcite @xcite . however, the majority of ift datasets are in english with very limited coverage for other languages @xcite . existing multilingual ift datasets can be divided into those that require human involvement and those that rely on machine translation ( fully synthetic datasets offer a promising solution to address the above concerns. not only do synthetic datasets address the high cost of data collection, toxicity and privacy concerns, english synthetic datasets like wizardlm, vicuna, ultrachat, etc have been proven to significantly enhance the performance of llms in english @xcite @xcite . however, there is a lack of research on synthetic datasets in the multilingual domain that encompass a wide range of languages, nlp tasks, and multi-turn conversations. to address this gap, we present the following contributions: 1. we introduce m2lingual, the first fully synthetic, multi-turn, and diverse multilingual dataset, containing 175k complex and challenging conversations across 70+ languages and 19 nlp tasks built with the evol taxonomy. 2. we construct a novel, two-step evol taxonomy (figure 2 ), covering 19 nlp tasks, each with 9 distinct methods to transform seed instructions to make them more complex and challenging. additionally, to synthesize multi-turn conversations, we develop 21 evol prompts to increase engagement. this controlled setup ensures a balances representation of different languages, dataset size multi turn? langs resource level task specific? general instructions? translated dataset? fully synthetic? low high aya dataset @xcite 200k ir pairs ✗ 70 37 (1) 32 ✗ ✓ ✗ ✗ multialpaca @xcite 132k ir pairs ✗ 11 0 11 ✗ ✓ ✗ ✓ m-alpaca @xcite 52k ir pairs ✗ 12 0 12 ✗ ✗ ✓ ✓ bactrian-x @xcite 3.4m ir pairs ✗ 52 15( 1 especially low resource languages (figure 4 ) which is challenging to achieve in real-world scenarios @xcite . the evol taxonomy enables a fully-synthetic, scalable, and cost-efficient method for constructing enriched multi-turn multilingual conversational ift dataset which is extendable to any task and language. 3. we provide detailed analyses highlighting the impact of seed instructions, each step of the data enrichment and synthesis process. additional analysis on low resource languages, content moderation, conversation length, and language distribution, demonstrate the superiority of m2lingual over other datasets. ## related work multilingual instruction finetuning. due to the widespread availability of high-resource language pretraining corpora multilingual instruction finetuning has proven to be a cost effective solution for improving performance @xcite @xcite . several approaches have been adopted to expand access to multilingual ift corpora. notable among these are datasets derived from nlp tasks (e.g., flant5, supernatural instructions) @xcite @xcite human-generated datasets such as aya @xcite and openassistant @xcite involve humans creating conversation topics, writing questions, and crafting responses. while these datasets are typically high quality, their creation is extremely resource and time intensive. moreover, finding native speakers for diverse languages is challenging, with potential annotator errors and uneven data distribution. @xcite , making it difficult to scale these datasets. human-ai generated datasets such as lm-sys @xcite , wildchat @xcite and @xcite are less resource-intensive than purely humangenerated ones, as they involve humans interacting with llms to generate conversations. however, they still present challenges, as humans must write instructions and create diverse questions in native languages, a process that remains timeconsuming. additionally, this approach can raise privacy concerns @xcite , introduce toxic data @xcite , and result in lowcomplexity conversations @xcite . finally, machine-translated datasets such as bactrainx @xcite offer a more resource efficient method. however, such datasets often suffer from translation artifacts known as translationese @xcite that fail to capture linguistic nuances @xcite . on the contrary to these, our presented m2lingual dataset utilizes ift seeds from native speakers across various languages (section 3.1) and applies task specific mutation in each language (section 3.2), thus maintaining linguistic nuances in respective individual language. m2lingual's generation pipeline is also completely synthetic ( synthetic datasets. fully synthetic datasets have emerged as a promising alternative towards addressing constraints with existing data generation methods. popular english synthetic datasets, such as alpaca @xcite , wizardlm @xcite , and vicuna @xcite , generate new instructions from a small initial set using methods like self-instruct @xcite or evol-instruct @xcite , and have shown strong performance. however, there is limited research on leveraging synthetic datasets to enhance multilingual capabilities, with the exception of multialpaca @xcite , which uses self-instruct. this approach has been shown to be susceptible to repetitive and noisy outputs @xcite , and suffers from low performance (tables 12 & 13 ). ## methodology m2lingual has three main synthesis steps. ## results importance of evol -the seed + evol rows in ## additional analysis low-resource languages. ## limitations and ethical considerations m2lingual covers over 70 languages in total, with dialects added in with evol as well -a significant number of languages, more than all relevant datasets ( some major limitations of m2lingual include the limited conversation length, possible presence of toxic data, and dependence on gpt-4 translated mt-bench for low-resource language evaluation. while potentially longer conversations could be built with evol, it would take significantly more resources to extend each conversation beyond the current limit. for toxicity, our seed dataset aya does not contain specific flags for toxic, harmful, or offensive speech @xcite , and aya authors report that they believe there is a low risk for these in aya data. however, to mitigate risk, we conduct moderation analysis of the generated evol ir pairs for m2lingual, and find that less than 0.2% of the generated data was flagged, which we filter out before making the data public. lastly, we conduct limited manual evaluation of the gpt-4 generated low-resource multilignual mt-bench data generated by gpt4, and find that it performs satisfactorily well. however, improving evaluation on low-resource data remains an area of future work.