Multilingual Instruction Tuning With Just a Pinch of Multilinguality (2024)

Abstract: As instruction-tuned large language models (LLMs) gain global adoption, their ability to follow instructions in multiple languages becomes increasingly crucial. In this work, we investigate how multilinguality during instruction tuning of a multilingual LLM affects instruction-following across languages from the pre-training corpus. We first show that many languages transfer some instruction-following capabilities to other languages from even monolingual tuning. Furthermore, we find that only 40 multilingual examples integrated in an English tuning set substantially improve multilingual instruction-following, both in seen and unseen languages during tuning. In general, we observe that models tuned on multilingual mixtures exhibit comparable or superior performance in multiple languages compared to monolingually tuned models, despite training on 10x fewer examples in those languages. Finally, we find that diversifying the instruction tuning set with even just 2-4 languages significantly improves cross-lingual generalization. Our results suggest that building massively multilingual instruction-tuned models can be done with only a very small set of multilingual instruction-responses.

## introduction instruction tuning is a fundamental aspect of building modern general-purpose large language models (llms), involving fine-tuning a pre-trained model on pairs of instructions and corresponding responses @xcite @xcite . for these models to be globally applicable, they must operate on a wide range of languages, yet, most instruction tuning datasets are typically limited to english. while curating naturally occurring instructions and responses for every language is challenging, cross-lingual transfer has emerged as a promising approach, in which a model is fine-tuned using one language, and acquiring similar abilities in another @xcite @xcite @xcite . the ability to follow instructions for languages seen only at pre-training can significantly expand the applicability of llms, allowing them to be used by more people worldwide. in this work, we show that instruction-tuning of multilingual llms transfers across languages better than previously known, and that even minimal language diversity in the tuning set can further unlock instruction-following generalization to languages that are unseen during instruction tuning. we investigate the effect of multilingual data on instruction-following across languages using an llm pre-trained on hundreds of languages @xcite , and high-quality, open-ended instructions and responses @xcite translated into 11 languages, across different families and writing systems. initially, we examine the transferability of monolingual instruction tuning across different languages. naturally, tuning using each language individually enhances performance within that language. notably, we find that this also translates into instruction-following capabilities across other languages, and that tuning with english, italian, or spanish yields the best average multilingual performance. inspired by this result, we turn to ask how much multilingual data is required to improve multilingual instruction-following, while preserving english performance. we find that replacing even just 40 english training examples with multilingual examples, significantly improves instructionfollowing in those languages. surprisingly, this small amount of language-diverse examples also improves performance for languages that are only seen during pre-training and are not represented in the instruction tuning set at all. the next question we tackle is whether increasing the number of languages in the tuning set can enhance generalization to new languages from the pre-training corpus. we find that tuning using a few languages enables better performance for languages unseen during tuning, compared to monolingual tuning with the same number of examples. finally, we test two potential factors that might influence the degree of cross-lingual transfer: language similarity and the amount of languagespecific pre-training data, but find no significant correlations. overall, our results provide recipes for multilingual instruction tuning that improves cross-lingual generalization, while preserving performance on english, under a fixed budget. in particular, we find that capable multilingual instruction-following models can be tuned even with a minimal amount of multilingual data. ## measuring multilingual instruction-following our objective is to discover how multilinguality during instruction tuning affects general-purpose instruction-following across languages. we break this down to multiple questions, including how well can monolingual instruction tuning transfer to other languages, how many multilingual examples can enhance multilingual instruction-following while preserving english performance, and whether increasing the number of languages can result in improved cross-lingual generalization. in this section we elaborate on the data, evaluation protocol, models we use, and the human annotation process to ensure the models quality. ## potential factors of transferability following the results from the previous sections, a natural question arises: what factors can predict the language code slavic script mutually family intelligible russian ru east cyrillic -serbian sr south cyrillic croatian croatian hr south latin serbian slovenian sl south latin -polish pl west latin -slovak sk west latin czech czech cs west latin slovak ## related work cross-lingual transfer the success of the pretraining-fine-tuning paradigm @xcite ignited a new line of work on cross-lingual transfer. @xcite and @xcite showed that the multilingual variant of bert can be fine-tuned on a specific task in one language and preform this task on another language, and @xcite reported similar findings with a recurrent neural network. conneau et al. (2020a) introduced xlm-r, a multilingual pre-trained encoder with strong cross-lingual abilities. phang et al. (2020) showed that intermediate training on an english task improves xlm-r's transfer across languages further, and pfeiffer et al. (2020) suggested an adapter-based framework to improve cross-lingual and task generalization. hu et al. (2020) proposed a benchmark for cross-lingual generalization consists of 40 languages across 9 nlp tasks. @xcite found that the depth of the network matters for cross-lingual transfer, and @xcite showed that parameter sharing is more important than shared vocabulary. @xcite delved into the influence of specific examples from the training data on the performance in other languages, and @xcite investigated how pre-training bert-based models using different language pairs affects cross-lingual downstream performance. going beyond encoderonly models, @xcite proposed mt5, a multilingual variant of t5 @xcite , and showed the significance of model scaling for crosslingual transfer in generation tasks. @xcite explored trasferability in english-centric models @xcite using four tasks. in contrast to most cross-lingual transfer literature that is focused on task-specific fine-tuning, we explore trends of cross-lingual generalization for general-purpose instruction-following llms. multilingual instruction tuning initially, works on instruction tuning @xcite @xcite focused on cross-task generalization in english. subsequently, a large body of work was dedicated to multilingual instruction tuning. @xcite found that tuning models with english datasets enables zero-shot cross-lingual abilities to new languages. the authors also found that this holds for languages that the model has never intentionally seen during pre-training, and that multilingual training improves generalization to new tasks. @xcite investigated the effects of full parameter training vs low-rank adaptation @xcite and monolingual vs multilingual instruction tuning using the stanford alpaca @xcite data, machine translated into 5 languages. @xcite trained multilingual instruction-following models for 26 languages with reinforcement learning from human feedback @xcite suggested instruction tuning llms by prepending the instruction and response translated into a pivot language (e.g english) to the response in the target language. concurrently with our work, @xcite found that only a few languages in the tuning set result in better cross-lingual transfer to new languages for english-centric llms. in this work, we consider transfer from monolingual instruction tuning from 12 languages, rather than exclusively on english. furthermore, we examine multilingual instruction-following using an llm pre-trained on hundreds of languages, which might be a key to unlocking more transfer to languages not represented during tuning. importantly, we unveil the potential of just a small amount of language diversity in the instruction tuning set for this cross-lingual generalization. ## conclusion we demonstrate that cross-lingual transfer offers a promising avenue for building multilingual instruction-following llms. our findings across different languages suggest that even monolingual instruction tuning using only one language can result in improved instruction-following capabilities in other languages. moreover, incorporating even a small set of a few dozen multilingual examples can significantly enhance instruction-following performance for both the languages the model is tuned on, and ones that were only seen during pre-training. additionally, training on such multilingual datasets achieves comparable or even superior performance compared to monolingual tuning for some languages. we observe a similar trend when exploring the effect of total number of languages in the tuning set, as even splitting the train set to only two languages improves generalization to new languages, compared to monolingual tuning. these findings pave the way for efficient and scalable development of multilingual llms capable of understanding and following instructions across languages with minimal multilingual supervision.