Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets (2024)

Abstract: Advancements in Large Language Models (LLMs) have significantly enhanced instruction-following capabilities. However, most Instruction Fine-Tuning (IFT) datasets are predominantly in English, limiting model performance in other languages. Traditional methods for creating multilingual IFT datasets—such as translating existing English IFT datasets or converting existing NLP datasets into IFT datasets by templating—struggle to capture linguistic nuances and ensure prompt (instruction) diversity. To address this issue, we propose a novel method for collecting multilingual IFT datasets that preserves linguistic naturalness and ensures prompt diversity. This approach leverages English-focused LLMs, monolingual corpora, and a scoring function to create high-quality, diversified IFT datasets in multiple languages. Experiments demonstrate that LLMs finetuned using these IFT datasets show notable improvements in both generative and discriminative tasks, indicating enhanced language comprehension by LLMs in non-English contexts. Specifically, on the multilingual summarization task, LLMs using our IFT dataset achieved 17.57% and 15.23% improvements over LLMs fine-tuned with translation-based and template-based datasets, respectively.

## introduction recent advancements in natural language processing (nlp) have showcased remarkable progress, particularly in its instruction-following capabilities. notably, large language models (llms) like @xcite @xcite have demonstrated significant prowess in this area @xcite @xcite . after the pretraining stage, llms are fine-tuned on † correspondence to sathishreddy.indurthi@zoom.us figure 1 : the incorrectly translated telugu instructionresponse pair is from the aya collection @xcite , which was translated from an english instructionresponse pair in the dolly v2 dataset @xcite . the correct telugu instruction-response pair was provided by a native telugu speaker. instruction fine-tuning (ift) datasets followed by an optional alignment tuning (at) based on the availability of the training datasets. ift datasets consist of instruction prompt-response pairs and have proven instrumental in enhancing the efficacy and overall instruction-following abilities of llms @xcite @xcite @xcite @xcite . however, a notable disparity persists between the abundance of instruction prompts available in english compared to other languages. while over 7k foot_0 languages are spoken worldwide, a staggering 73% of prevalent ift datasets primarily cater to english alone @xcite . while llms often demonstrate proficiency in understanding and generating text across multiple languages, the language imbalance in training datasets has led to suboptimal performance in non-english contexts @xcite @xcite . to enhance llms' ability to follow non-english instructions, figure 2 : lack of diversity in templated datasets: the template created by human annotators has been repeated several thousand times in the templated adversarial qa dataset from the aya collection @xcite various studies have explored fine-tuning llms on multilingual instruction fine-tuning (ift) datasets @xcite @xcite @xcite @xcite . however, creating such multilingual ift datasets is challenging. previous efforts have primarily focused on two approaches: translating existing english ift datasets or templating existing natural language processing (nlp) datasets in non-english languages through native speakers to form iftstyle datasets. each approach has its drawbacks, highlighting the need for more effective methods. translating english ift datasets poses significant challenges, primarily because it fails to capture the nuances and intricacies unique to each language @xcite . additionally, the translation process often introduces errors, leading to suboptimal performance when fine-tuning llms on these translated datasets, as the models absorb these errors during training @xcite @xcite . for example, in figure 1 , the first translated response (red) is incorrect, even though it differs from the correct response (blue) by just one word. the red and blue words are used in different contexts in telugu and do not have direct translations in english. despite being generated by a state-of-the-art translation model, the first translation (red) fails to capture the correct meaning. thus, relying entirely on translated data poses significant challenges in accurately reflecting the nuances of non-english languages. comparatively, the templating approach avoids the introduction of translation errors. however, achieving high diversity through templated approaches is challenging and often tedious due to the manual effort required @xcite . for instance, as shown in figure 2 , one of the templated datasets contains the same instruction repeated several thousand times, resulting in a lack of diversity in the ift dataset. to address the issues of translation and templated approaches, we introduce an efficient method to collect high-quality multilingual ift datasets. the proposed method preserves the nuances of languages, avoids errors, and creates a diverse set of ift examples for multiple languages. this is achieved by leveraging an english-focused llm and the availability of monolingual corpora in each non-english language. we also employ a scoring function to control the quality of generated ift examples. by relying on english-focused llms, we can tap into their extensive capabilities and transfer these abilities across diverse linguistic contexts. utilizing monolingual corpora allows us to capture the unique linguistic and cultural nuances of each language, enhancing performance and accuracy in multilingual applications. additionally, the robust scoring function ensures that the knowledge and capabilities derived from englishcentric llms are appropriately adapted and optimized for non-english languages. extensive experiments on both generative and discriminative tasks demonstrate the effectiveness of the multilingual ift datasets resulting from our proposed method. compared to models fine-tuned on ift datasets created using translation and templated approaches, the model fine-tuned on ift datasets from our method achieves an average improvement of 11.1% in generative tasks and 6.9% in discriminative tasks. furthermore, these improvements are obtained with an ift dataset less than half the size of those created using templated and translation methods, highlighting the superior quality and diversity of the ift dataset generated by our approach. ## method a fundamental component in the development of multilingual large language models (mllms) lies in the acquisition of training datasets, crucially needed throughout distinct phases: pretraining (pt), instruction fine-tuning (ift), and alignment tuning (at). while obtaining the necessary monolingual datasets for pretraining is relatively straightforward, acquiring datasets for instruction fine-tuning and alignment tuning presents significant challenges due to the costs and human effort involved. to address these challenges while maintaining linguistic characteristics and diversity, we propose a framework for creating ift datasets for multiple languages. the framework consists of five stages, illustrated in figure 3 and described below: we use llm as a judge, employing the prompt p s to assess the quality of (i en , r en ) pair. this results in a score, denoted as s: pairs with a score greater than or equal to a predefined threshold (λ) are used for fine-tuning, while those below this threshold are removed from finetuning phase. subsequently, we form a training pair (i x , r x ). here, i x serves as a pseudo instruction, while r x represents natural text in the same non-english language. during the llm fine-tuning stage, despite potential unnaturalness and errors in i x arising from the instruction generation and translation process, the model is trained to generate r x , which is typically a natural and error-free output sourced from the monolingual corpus or existing humancurated nlp datasets. leveraging such pairs enhances the model's ability to handle instruction errors and improves its overall language comprehension. the sample task prompts (p i ) and scoring prompt (ps) used in equation 1 and equation 2 are provided in ## results in we evaluate the performance of models finetuned on datasets collected using our approach against models fine-tuned on datasets obtained through translation and template-based methods. specifically, we compare the aya-tm and llama-3-8b-tm models, which are trained on templatebased datasets as described in @xcite . additionally, we assess the aya-tr and llama-3-8b-tr models, which are trained on translationbased datasets detailed in @xcite . both types of datasets include the aya-human annotated dataset foot_3 . furthermore, we compare these with the bactrian-x model @xcite , fine-tuned on a dataset comprising translated english instructions and their corresponding multilingual responses generated using chatgpt. our rougelsum tel hin jpn spa templated approaches aya-tm 18.0 33.8 7.9 24.2 llama-3-8b-tm 19.6 36.4 17.8 26.8 ## conclusion in conclusion, our research addresses the notable disparity in instruction fine-tuning (ift) datasets, predominantly centered on english, by proposing a novel method for collecting multilingual ift datasets. by leveraging english-focused llms and monolingual corpora, our approach maintains the naturalness of specific languages and ensures diversity in the datasets. the quality control through a scoring function further enhances the effectiveness of the generated datasets. our extensive experiments on generative tasks demonstrate that the models trained with our multilingual ift datasets significantly outperform those trained on traditional translated and templated datasets. moreover, our models show substantial improvements in discriminative tasks, indicating a better comprehension of language. these results underscore the importance of diverse and high-quality multilingual datasets in enhancing the performance of large language models across various languages. our method provides a viable solution to the challenges faced in creating effective multilingual ift datasets, paving the way for more inclusive and capable language models. future research can build upon this approach to further refine and expand the capabilities of llms in a broader range of linguistic contexts.