LLM s Beyond E nglish: Scaling the Multilingual Capability of LLM s with Cross-Lingual Feedback (2024)

Abstract: To democratize large language models (LLMs) to most natural languages, it is imperative to make these models capable of understanding and generating texts in many languages, in particular low-resource ones. While recent multilingual LLMs demonstrate remarkable performance in such capabilities, these LLMs still support a limited number of human languages due to the lack of training data for low resource languages. Moreover, these LLMs are not yet aligned with human preference for downstream tasks, which is crucial for the success of LLMs in English. In this paper, we introduce xLLaMA-100 and xBLOOM-100 (collectively xLLMs-100), which scale the multilingual capabilities of LLaMA and BLOOM to 100 languages. To do so, we construct two datasets: a multilingual instruction dataset including 100 languages, which represents the largest language coverage to date, and a cross-lingual human feedback dataset encompassing 30 languages. We perform multilingual instruction tuning on the constructed instruction data and further align the LLMs with human feedback using the DPO algorithm on our cross-lingual human feedback dataset. We evaluate the multilingual understanding and generating capabilities of xLLMs-100 on five multilingual benchmarks. Experimental results show that xLLMs-100 consistently outperforms its peers across the benchmarks by considerable margins, defining a new state-of-the-art multilingual LLM that supports 100 languages.

## introduction despite the impressive improvements have been made recently, the majority of large language models (llms), such as llama-2 @xcite , llama-3 (ai@meta, 2024) and bloom @xcite , are predominantly trained on english texts and support only a limited number of non-english languages. for instance, while llama-2 and bloom support 25 and 46 languages, respectively, their performance varies significantly across different languages. however, there are currently more than 7,000 languages spoken in the world foot_0 and only a few of them are used for training llms. scaling llms' multilingual capabilities is challenging due to the scarcity of multilingual instruction data available for fine-tuning, particularly for low-resource languages. the primary advantage of llms lies in their ability to learn task execution, in particular mapping an input text to an output text, through a textual instruction. a task instruction, input, and output text are assumed to be in the same language. however, these elements can be in different languages for many downstream tasks, which is the most intuitive manifestation of the multilingual capabilities of llms. we consider two types of multilingual capability in llms. first, when the instructions for llms are expressed in different languages, llms should understand these instructions and generate a correct output. we refer to this feature of llms as the understanding capability. second, llms should be able to generate the correct response in the target language and perform consistently well on (almost) all languages when a fixed language (e.g., english) is used as the instruction language. we name this capability of llms the generating capability. we categorize the previous work on scaling the multilingual capability of llms in two groups. the first group includes approaches that continue training the llms using as much of the training corpora as possible. the training corpora are either multilingual parallel data for machine trans-lation tasks @xcite , or multilingual instruction data for instruction tuning @xcite @xcite @xcite . the second group includes approaches that align non-english instructions with english instructions through cross-lingual prompting in the inference stage @xcite . while achieving impressive performance, both groups suffer from major problems. first, they only support a small number of languages, and most of the world's languages are still being left behind. second, they use large corpora for supervised fine tuning (sft), neglecting the exploration of alignment to human preferences. to address the aforementioned issues, we aim to scale the two multilingual capabilities of llms at the same time. to improve the understanding capability of llms, we construct a multilingual instruction dataset with 100 languages by translating instructions from alpaca @xcite via chatgpt foot_1 and google translate api foot_2 . we finetune llms on our constructed multilingual dataset using parameter efficient fine tuning (peft; @xcite . the languages of instruction and output are always the same language in the existing human feedback dataset @xcite , which limits the generating capability of llms. to enhance the generating capability, we construct a cross-lingual human feedback data (i.e., instruction and output are different languages) covering 30 languages foot_3 . we then further align the llms with human feedback using the dpo algorithm @xcite . finally, we obtain an llm that supports 100 languages which has the capability to understand the instructions of 100 languages and supports output in 100 languages. we conduct a comprehensive evaluation to verify the effectiveness of xllms-100 on five diverse multilingual benchmarks, covering understanding (paws-x; @xcite , reasoning (xcopa; @xcite , generation @xcite @xcite and expert-written (self-instruct; @xcite tasks in the zero-shot setting. each benchmark includes multilingual evaluation data covering both high-resource and low-resource languages. the experimental results clearly demonstrate that xllms-100 significantly enhances both the understanding and generating capabilities of llms simultaneously across all benchmarks. furthermore, our extensive analysis experiments reveal that xllms-100 not only mitigate the off-target problem (i.e., llms generate the text into an incorrect language @xcite ) but also enhance language democratization (i.e., democratization degree of tasks between languages @xcite ) compared with strong llms. in summary, we make the following contributions: (i) we construct two datasets, one of which contains a multilingual instructions in 100 languages, and the other one contains cross-lingual human preferences in 30 languages. (ii) we evaluate the multilingual capabilities of llms in two dimensions: understanding and generating capability. unlike previous studies that assess these capabilities in isolation, we urge the community to consider both capabilities when evaluating the multilingual performance of llms. (iii) we scale the multilingual capabilities of llms to perform well across 100 languages. ## method to scale the multilingual capabilities of llms, we construct two datasets: a multilingual instruction dataset (section 3.1) and a cross-lingual feedback dataset (section 3.2). then, we evaluate the quality of the translated instructions and generated responses on our constructed datasets (section 3.3). finally, we introduce the training process of our multilingual instruction fine-tuning (section 3.4). data statistics and analyses can be found in the appendix a. ## results our goal is to simultaneously evaluate the understanding capability and the generating capability of the llms. to evaluate the understanding capability of llms, we use google translate api to translate the instructions for each task into different languages. to evaluate the generating capability of an llm, we use english as the instruction language during the inference phase. we choose english because llms demonstrate effective comprehension of english instructions, thus avoiding any potential impact on the generating capability of llms due to misunderstanding of instructions. the off-the-shelf llms demonstrate limited multilingual capabilities, in particular in generation tasks, where performance is exceedingly poor. after fine-tuning the llms using the multilingual instruction dataset, there is a marginal (but not significant) improvement in the multilingual capability of the llms across most benchmarks, especially when describing instructions in non-english languages. for example, the understanding capability of the bx-based models (bx llama and bx bloom ) drops significantly on the self-instruct*, xl-sum, and flores benchmarks. this phenomenon is due to the fact that the bx-based models are finetuned in 52 languages and do not completely support all the languages in the benchmarks. ## analysis in section 5 we show the effectiveness of our approach compared with previous work. in this section, we study the performance of our multilingual model in detail. ## conclusions to enhance the multilingual capability of llms in two dimensions (understanding and generating), we present xllms-100, two llms that support 100 languages. we train xllms-100 on a novel multilingual instruction dataset containing 100 languages. to improve the generating capability, we construct a cross-lingual human feedback dataset to further align the llms with human feedback and to enable the llms to generate output in multiple languages. experiments on five benchmarks demonstrate the effectiveness of our datasets and also the multilingual capability of our models both on high-resource languages and low-resource languages.