LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback (2024)

Abstract: To democratize large language models (LLMs) to most natural languages, it is imperative to make these models capable of understanding and generating texts in many languages, in particular low-resource ones. While recent multilingual LLMs demonstrate remarkable performance in such capabilities, these LLMs still support a limited number of human languages due to the lack of training data for low-resource languages. Moreover, these LLMs are not yet aligned with human preference for downstream tasks, which is crucial for the success of LLMs in English. In this paper, we introduce xLLaMA-100 and xBLOOM-100 (collectively xLLMs-100), which scale the multilingual capabilities of LLaMA and BLOOM to 100 languages. To do so, we construct two datasets: a multilingual instruction dataset including 100 languages, which represents the largest language coverage to date, and a crosslingual human feedback dataset encompassing 30 languages. We perform multilingual instruction tuning on the constructed instruction data and further align the LLMs with human feedback using the DPO algorithm on our crosslingual human feedback dataset. We evaluate the multilingual understanding and generating capabilities of xLLMs-100 on five multilingual benchmarks. Experimental results show that xLLMs-100 consistently outperforms its peers across the benchmarks by considerable margins, defining a new state-of-the-art multilingual LLM that supports 100 languages1.

heading: 1 Introduction
text: Despite the impressive improvements have been made recently, the majority of large language models (LLMs), such as LLaMA-2 (Touvron et al., 2023), LLaMA-3 (AI@Meta, 2024) and
∗ Work done during internship at Bosch AI. 1The code, datasets, and models are publicly available at
https://github.com/boschresearch/ACL24-MLLM.
BLOOM (BigScience et al., 2022), are predominantly trained on English texts and support only a limited number of non-English languages. For instance, while LLaMA-2 and BLOOM support 25 and 46 languages, respectively, their performance varies significantly across different languages. However, there are currently more than 7,000 languages spoken in the world2 and only a few of them are used for training LLMs. Scaling LLMs’ multilingual capabilities is challenging due to the scarcity of multilingual instruction data available for fine-tuning, particularly for low-resource languages.
The primary advantage of LLMs lies in their ability to learn task execution, in particular mapping an input text to an output text, through a textual instruction. A task instruction, input, and output text are assumed to be in the same language. However, these elements can be in different languages for many downstream tasks, which is the most intuitive manifestation of the multilingual capabilities of LLMs. We consider two types of multilingual capability in LLMs. First, when the instructions for LLMs are expressed in different languages, LLMs should understand these instructions and generate a correct output. We refer to this feature of LLMs as the understanding capability. Second, LLMs should be able to generate the correct response in the target language and perform consistently well on (almost) all languages when a fixed language (e.g., English) is used as the instruction language. We name this capability of LLMs the generating capability.
We categorize the previous work on scaling the multilingual capability of LLMs in two groups. The first group includes approaches that continue training the LLMs using as much of the training corpora as possible. The training corpora are either multilingual parallel data for machine trans-
2https://www.ethnologue.com
8186
lation tasks (Yang et al., 2023; Zhu et al., 2023), or multilingual instruction data for instruction tuning (Üstün et al., 2024; Groeneveld et al., 2024; Luo et al., 2023; Li et al., 2023; Lai et al., 2023a). The second group includes approaches that align non-English instructions with English instructions through cross-lingual prompting in the inference stage (Huang et al., 2023; Etxaniz et al., 2023). While achieving impressive performance, both groups suffer from major problems. First, they only support a small number of languages, and most of the world’s languages are still being left behind. Second, they use large corpora for supervised fine tuning (SFT), neglecting the exploration of alignment to human preferences.
To address the aforementioned issues, we aim to scale the two multilingual capabilities of LLMs at the same time. To improve the understanding capability of LLMs, we construct a multilingual instruction dataset with 100 languages by translating instructions from Alpaca (Taori et al., 2023) via ChatGPT3 and Google Translate API4. We finetune LLMs on our constructed multilingual dataset using parameter efficient fine tuning (PEFT; Hu et al., 2021). The languages of instruction and output are always the same language in the existing human feedback dataset (Taori et al., 2023), which limits the generating capability of LLMs. To enhance the generating capability, we construct a cross-lingual human feedback data (i.e., instruction and output are different languages) covering 30 languages5. We then further align the LLMs with human feedback using the DPO algorithm (Rafailov et al., 2023). Finally, we obtain an LLM that supports 100 languages which has the capability to understand the instructions of 100 languages and supports output in 100 languages.
We conduct a comprehensive evaluation to verify the effectiveness of xLLMs-100 on five diverse multilingual benchmarks, covering understanding (PAWS-X; Yang et al., 2019), reasoning (XCOPA; Ponti et al., 2020), generation (XL-Sum; Hasan et al., 2021 and FLORES101; Goyal et al., 2022) and expert-written (SelfInstruct; Wang et al., 2023) tasks in the zero-shot setting. Each benchmark includes multilingual evaluation data covering both high-resource and low-resource languages. The experimental results
3https://chat.openai.com 4https://translate.google.com 5We include 30 languages for that ChatGPT provides high
quality feedback.
clearly demonstrate that xLLMs-100 significantly enhances both the understanding and generating capabilities of LLMs simultaneously across all benchmarks. Furthermore, our extensive analysis experiments reveal that xLLMs-100 not only mitigate the off-target problem (i.e., LLMs generate the text into an incorrect language (Zhang et al., 2020)) but also enhance language democratization (i.e., democratization degree of tasks between languages (Huang et al., 2023)) compared with strong LLMs.
In summary, we make the following contributions: (i) We construct two datasets, one of which contains a multilingual instructions in 100 languages, and the other one contains cross-lingual human preferences in 30 languages. (ii) We evaluate the multilingual capabilities of LLMs in two dimensions: understanding and generating capability. Unlike previous studies that assess these capabilities in isolation, we urge the community to consider both capabilities when evaluating the multilingual performance of LLMs. (iii) We scale the multilingual capabilities of LLMs to perform well across 100 languages.

heading: 2 Related Work
text: Multilingual Capabilities of LLMs. Recent studies have applied LLMs to various NLP tasks in a multilingual setting (Üstün et al., 2024; Groeneveld et al., 2024; Lai et al., 2023a; Weissweiler et al., 2023). In general, two kinds of corpora have been used to improve the multilingual capabilities of LLMs: multilingual parallel corpora (BigTranslate; Yang et al., 2023) and multilingual instruction datasets (Bactrian-X; Li et al., 2023 and xP3; Muennighoff et al., 2023). Yang et al. (2023) continued training LLaMA (Touvron et al., 2023) using a multilingual parallel corpus that covers 102 languages and achieved good results on a multilingual machine translation task. However, the results from our initial experiments indicate that the performance of this model on non-machine translation tasks is not as good as its performance on the evaluated machine translation task (see Section 6.2 for more details). Li et al. (2023) finetunes LLaMA-2 using a multilingual instruction dataset including 52 languages, and achieved good performance on several multilingual NLP tasks. In contrast, we construct a multilingual instruction dataset including 100 languages. To the best of our knowledge, our dataset is the multilingual instruction dataset with the largest language coverage to date.
Aligning LLMs with Human Feedback. A prominent method for LLM training is reinforcement learning from human feedback (RLHF; Ouyang et al., 2022), which learns from human feedback instead of relying on a pre-defined reward function. Despite its popularity, there are significant flaws. Collecting human preferences is time consuming. The instability of the RLHF method during training also poses a significant challenge in learning the optimal reward function from human preference data (Schulman et al., 2017). Recently, the DPO algorithm (Rafailov et al., 2023) has demonstrated that this challenge can be addressed by fine-tuning LLMs to align with human preferences using a supervised learning regime, without the requirement of explicit reward modeling or reinforcement learning. DPO has only been applied to monolingual human feedback data (i.e., instructions, inputs, and outputs are in the same language). While such fine-tuned LLMs are beneficial for monolingual tasks, they have not been shown to generalize to tasks that require text understanding and generation in various languages. In this paper, we construct a large-scale cross-lingual human preference dataset covering 30 languages and show promising results in multiple NLP tasks.

heading: 3 Method
text: To scale the multilingual capabilities of LLMs, we construct two datasets: a multilingual instruction dataset (Section 3.1) and a cross-lingual feedback dataset (Section 3.2). Then, we evaluate the quality of the translated instructions and generated responses on our constructed datasets (Section 3.3). Finally, we introduce the training process of our multilingual instruction fine-tuning (Section 3.4). Data statistics and analyses can be found in the Appendix A.

heading: 3.1 Multilingual Instruction Dataset
text: Alpaca (Taori et al., 2023) contains 52K instruction and demonstration pairs generated by OpenAI’s text-davinci-003 engine using the self-instruct technique (Wang et al., 2023). Some of the existing multilingual instruction datasets are primarily translated from the Alpaca datasets. For instance, Lai et al. (2023a) and Li et al. (2023) expanded the Alpaca dataset to include 26 and 52 languages, respectively. To further scale the multilingual capabilities of LLMs, we expand the Alpaca dataset to 100 languages. Our construction process con-
tains two steps: instruction translation and hybrid response generation.
Instruction Translation. We use Google Translate API to translate English instructions and inputs in the Alpaca dataset into 100 languages covered in the FLORES-101 dataset (Goyal et al., 2022). We chose the Google Translate API because it outperforms state-of-the-art translators such as NLLB, DeepL6, and GPT-4 (Achiam et al., 2023) in translation performance across multiple languages (Yang et al., 2023; Robinson et al., 2023). For languages that are not supported by Google Translate, we employ the NLLB model (Costa-jussà et al., 2022) for translation, as it is currently considered the state-ofthe-art for multilingual translation, particularly for low-resource languages. Similar to Bactrian-X (Li et al., 2023), we do not translate instructions that contain program-related text.
Hybrid Response Generation. Intuitively, there are two alternatives to obtain responses in various languages: the translation-based method and the generation-based method. The translation-based approach involves directly translating Alpaca’s English responses into one of the 100 target languages using either the Google Translate API or the stateof-the-art multilingual machine translation model. Generation-based methods, like Bactrian-X (Li et al., 2023), take instructions that have been translated into the desired target language and feed them into LLMs (e.g., ChatGPT), resulting in a response expressed in the target language. Both of these methods can produce responses in the target language, however, they also have notable limitations. Without the context, the translation-based approach usually translates responses in a different style from the native speaker and has the potential problem of translationese (Riley et al., 2020). More importantly, the understanding and generating capabilities of LLMs varies significantly across languages, and it is extremely difficult for the generation-based method to generate a high quality response in lowresource languages. To solve the above problems, we generate responses in a hybrid mode. We motivate our approach with the translation performance (i.e., translation task from English to other languages) of ChatGPT in the FLORES benchmark demonstrated in Lu et al. (2023). We assume that languages exhibiting poor translation performance (typically with BLEU scores below 10) by Chat-
6www.deepl.com
GPT also possess limited generating capabilities. Consequently, for languages with poor translation quality, we directly translate the English responses in Alpaca into the specific target language using the Google Translate API or NLLB model, while languages with good translation quality have their responses generated by ChatGPT. ChatGPT’s responses are preferable because they seem to have a more consistent style. Table 11 in the Appendix A details the translator (Google Translate API or the NLLB model) used for each language’s translation instructions, as well as the method (ChatGPT, Google Translate API, or the NLLB model) employed to generate responses.

heading: 3.2 Cross-Lingual Feedback Dataset
text: Aligning LLMs with human preferences is crucial in enhancing the truthfulness of their generated responses. Most datasets with human feedback are monolingual, e.g., English (Peng et al., 2023) and Chinese (Sun et al., 2023). Recently, Lai et al. (2023a) extend the human feedback dataset to 26 languages using two rounds of dialogue (translation and ranking) via ChatGPT. However, one of the biggest problems with human feedback data nowadays is that their instructions, inputs and outputs are in the same language, which limits the generative capability of LLMs. To enhance the generating capability, we construct a cross-lingual human feedback dataset covering 30 languages. This dataset provides both the instruction and output in different languages. Such a design is advantageous as it simulates a wide range of generating scenarios, thereby enhancing the generating capa-
bility of LLMs. For instance, if we have human preference data available in 30 languages, we can simulate up to 30× 29 = 870 generation scenarios. The construction process has two steps: instruction design and response generation. We show an example generated cross-lingual human feedback in Figure 1.
Instruction Design. Given a source language ℓs, we first translate the instruction written in English into the instruction written in ℓs using Google Translate API. We denote the instruction written in ℓs as Iℓs . Then, we randomly select one of the 30 languages, excluding ℓs, as the target language ℓt and design an instruction written in ℓs to return an output written in ℓt. We denote the instruction written in ℓt as Iℓt . Finally, we merge Iℓs and Iℓt to construct a new instruction as Iℓtℓs .
Response Generation. Inspired by Lai et al. (2023a), we rank the responses from ChatGPT according to their quality for the instruction and input text. Given the new instruction Iℓtℓs , we use ChatGPT to generate the responses in target language ℓt and rank the responses. The ranking process scores different responses based on three factors: correctness, coherence and naturalness. We take the response with the highest score as the accepted response, denoted as Ra and the response with the lowest score as the rejected response, denoted as Rr. Note that, both the Ra and Rr are written in ℓt.

heading: 3.3 Instruction and Response Quality
text: To evaluate the quality of the instructions in our constructed dataset, we randomly choose 50 in-
structions per language and translate them into English using the Google Translate API. Then, we evaluate the BLEU (Post, 2018) and COMET (Rei et al., 2020) of the back-translated instructions against the original English instructions in Alpaca. Table 1 shows the number of languages within each interval segment for BLEU and COMET scores. We find that most languages have BLEU scores between 20 and 60 and COMET scores between 60 and 90, indicating that the quality of the constructed instructions is high.
To evaluate the quality of the generated responses, we randomly select 5 high-resource (Arabic, Chinese, French, German and Hindi) and 5 low-resource languages (Armenian, Gujarati, Kannada, Malayalam and Marathi), and evaluate 100 responses in each language. We use two metrics for this evaluation. First, similar to our evaluation on the instruction quality, we back-translate the responses into English and calculate the BLEU score. Second, we assess content preservation (CP), which is a crucial metric in text style transfer (Jin et al., 2022). It measures the degree of meaning preservation between two texts by calculating the cosine similarity between the vectors of the original and generated texts. We use this metrics by mapping the language-specific responses and the original English responses in Alpaca into the same vector space using a multilingual sentence embedding model (LaBSE; Feng et al., 2022), then we compute their cosine distance. As shown in Table 2, the BLEU scores for responses in high-resource languages ranges from 73.16 to 80.27 and effectively preserved meaning (i.e., CP ranges from 0.81 to 0.91). In addition, the CP score is more than 0.8 for high-resource languages and about 0.6 for lowresource ones, which indicate enough quality for the purpose of this research.

heading: 3.4 Multilingual Instruction Tuning
text: To enhance both the understanding and generating capabilities of LLMs, we employ a two-step training process: supervised fine-tuning and alignment with human preferences.
Supervised Fine-tuning (SFT). Starting with an LLM, e.g., LLaMA (Touvron et al., 2023) or BLOOM (BigScience et al., 2022), we perform supervised fine-tuning on the LLM using our constructed multilingual instruction dataset. Since finetuning on full parameters across all layers of an LLM is computationally expensive, we apply a parameter-efficient fine-tuning technique, specifically LoRA (Hu et al., 2021). LoRA incorporates trainable rank decomposition matrices into the LLM layers and only updates the newly introduced parameters while freezing all parameters of the original LLMs. LoRA has been shown to achieve comparable performance to full parameter fine-tuning methods on LLMs (Chen et al., 2023).
Aligning LLMs with human feedback. Common practice for aligning LLMs with human feedback is to use reinforcement learning (RLHF; Ouyang et al., 2022) to employ the Proximal Policy Optimization (PPO; (Schulman et al., 2017)) algorithm to maximizes the reward of the model. However, RLHF suffers from instability of reinforcement technical and requires significant computational resources (Casper et al., 2023). To save computational resources, we further fine-tune the trained SFT model from the last step with the DPO algorithm (Rafailov et al., 2023) with our constructed cross-linguistic human feedback dataset. Note that we also use LoRA in the whole DPO training process.

heading: 4 Experiments

heading: 4.1 Datasets and Tasks
text: We evaluate xLLMs-100 on five typical benchmarks including generation, reasoning, understanding and expert-written tasks that measure the multilingual capabilities of LLMs, including both highresource and low-resource languages. Understanding Task. We evaluate our LLM for the paraphrase identification task on PAWSX (Yang et al., 2019) benchmark on 7 languages. The benchmark provides two sentences and asks the model to determine whether they paraphrase each other or not.
Generation Task. We evaluate the multilingual capability of our LLM on the FLORES-101 (Goyal et al., 2022) and XL-Sum (Hasan et al., 2021) benchmarks. FLORES-101 is a machine translation benchmark, containing parallel sentences between 101 languages. FLORES-101 lets us evaluate our LLM for tasks in which input and output texts are in different languages. XL-Sum is a summarization benchmark covering 44 languages, where an LLM should summarize a long text into a short text. XL-Sum lets us evaluate LLMs for tasks where input and output texts are in the same language.
Reasoning Task. We evaluate the commonsense reasoning task using the XCOPA (Ponti et al., 2020) benchmark, which includes texts written in 11 languages. A data sample in XCOPA consists of one premise and two choices, and requires the model to select which choice is the effect or cause of the given premise.
Expert-written Task. We use the Google Translate API to translate the Self-Instruct (Wang et al., 2023) benchmark from English to five high-resource languages (Arabic, Czech, German, Chinese, Hindi) and five low-resource languages (Armenian, Kyrgyz, Yoruba, Tamil, Mongolian). We call this dataset Self-Instruct*. This benchmark contains the instruction, input and output in each instance, and requires that the LLM predicts the correct answer in the correct target language.

heading: 4.2 Baselines
text: We compare xLLMs-100 with the following baselines:
Off-the-shelf LLMs. We evaluate LLaMA-2 (Touvron et al., 2023) and BLOOM (BigScience et al., 2022) as vanilla LLM baselines without additional finetuning.
Publicly available multilingual instructiontuned models: Bactrian-X is the instruction-tuned model proposed by Li et al. (2023). These models were instruction-tuned on 52 languages. They released models based on LLaMA and BLOOM. We refer to them as BXLLaMA and BXBLOOM, respectively.
Supervised Fine-Tuning (SFT): We performed instruction tuning (Taori et al., 2023) by utilizing our constructed multilingual instruction dataset. We denote these models as SFTLLaMA and SFTBLOOM.

heading: 4.3 Implementation
text: We use the 7B model of LLaMA (chat version; Llama-2-7b-chat-hf) and BLOOM (basic version; bloom-7b1) in all experiments as the base model. We train our model using PyTorch with the HuggingFace transformers7 and PEFT8 implementation. Hyperparameters used for training our model can be found in Appendix B. We evaluate our model in the zero-shot setting. Recent studies have shown that slight modifications in input prompts can lead to varied results (Loya et al., 2023). To ensure the reproducibility, we present the prompts used for each task in the experiment in Appendix C. To mitigate over-fitting, we set the number of epochs to 1 for both our multilingual fine-tuning and DPO training processes.

heading: 4.4 Experimental Settings
text: Similar to Huang et al. (2023), to save inference time, we randomly select 200 test samples for each language in FLORES-101 and 250 test samples for each language in XL-Sum. For FLORES-101, we have two settings: translation tasks from English to other languages and translation tasks from the other languages to English, denoted as FLORES(f) and FLORES(t), respectively. In addition to XL-Sum and PAWS-X benchmark, we categorize the languages in each dataset into low-resource and highresource languages based on the language classification in Costa-jussà et al. (2022). We classify the languages in XL-Sum dataset into low, mid, and high categories according to Hasan et al. (2021). The PAWS-X dataset does not include low-resource languages.

heading: 4.5 Evaluation Metric
text: For FLORES-101, we report case-sensitive detokenized BLEU with SacreBLEU9(Post, 2018). For the XCOPA and PAWS-X benchmarks, we utilize the accuracy score for evaluation. For the XLSum and Self-Instruct* benchmark, we report the multilingual ROUGE-1 score implemented by Lin (2004).

heading: 5 Results
text: Our goal is to simultaneously evaluate the understanding capability and the generating capability
7https://github.com/huggingface/transformers 8https://github.com/huggingface/peft 9https://github.com/mjpost/sacrebleu
of the LLMs. To evaluate the understanding capability of LLMs, we use Google Translate API to translate the instructions for each task into different languages. To evaluate the generating capability of an LLM, we use English as the instruction language during the inference phase. We choose English because LLMs demonstrate effective comprehension of English instructions, thus avoiding any potential impact on the generating capability of LLMs due to misunderstanding of instructions. Table 3 presents the average score for all languages in each benchmark. For more comprehensive results of each benchmark per language, please refer to Appendix D.
The off-the-shelf LLMs demonstrate limited multilingual capabilities, in particular in generation tasks, where performance is exceedingly poor. After fine-tuning the LLMs using the multilingual instruction dataset, there is a marginal (but not significant) improvement in the multilingual capability of the LLMs across most benchmarks, especially when describing instructions in non-English languages. For example, the understanding capability of the BX-based models (BXLLaMA and BXBLOOM)
drops significantly on the Self-Instruct*, XL-Sum, and FLORES benchmarks. This phenomenon is due to the fact that the BX-based models are finetuned in 52 languages and do not completely support all the languages in the benchmarks.
Low-Resource vs High-Resource. We observe that the multilingual capabilities of the examined LLMs are significantly superior in high-resource languages to those in low-resource languages. It is a common problem for multilingual models that the scarcity of training corpora in low-resource languages makes it challenging to train a robust decoder, leading to the generation of incorrect outputs (Lai et al., 2023b).
Understanding Capability vs Generating Capability. Our findings indicate that instructions written in English outperform instructions written in non-English languages, aligning with our initial expectations. This phenomenon can be attributed to the accumulation of biases in LLMs’ understanding of instructions across different languages. Such biases can introduce errors during the generation of results, resulting in incorrect outputs or outputs in the wrong language (i.e., off-target problem; Zhang
et al., 2020). xLLMs-100. Compared with the baseline mod-
els, our models demonstrate consistent improvements for both multilingual capabilities across all examined tasks. In comparison to the SFT-based models, xLLMs-100 increases the step of alignment with human preferences, resulting in additional improvements. This observation highlights the significance of aligning with human preferences after supervised fine-tuning. Furthermore, our model significantly outperforms other models on generative task datasets such as XL-Sum and FLORES, showing the effectiveness of our dataset and human feedback finetuning.

heading: 6 Analysis
text: In Section 5 we show the effectiveness of our approach compared with previous work. In this section, we study the performance of our multilingual model in detail.

heading: 6.1 Different Human Feedback Datasets
text: To investigate the importance of cross-lingual properties in aligning LLMs with human preferences, we conduct an ablation study on the same five benchmarks as shown in Table 3. In particular, we employ the DPO algorithm (Rafailov et al., 2023) to finetune our model, xLLMs-100, on two distinct datasets. The first dataset is our constructed cross-lingual human feedback dataset, where instructions and outputs are in different languages. The second dataset is a traditional monolingual human feedback dataset (Lai et al., 2023a), where both instructions and outputs are in the same language. Table 4 show the results categorized by lowand high-resource languages. We observe that (1) Aligning xLLMs-100 using our cross-lingual hu-
man feedback dataset yields superior results compared with using monolingual human feedback. This improvement is evident for datasets with generation tasks such as XL-SUM and FLORES, showing that our novel cross-lingual human feedback dataset effectively simulates the multilingual generation task, reducing the possibilities of generating incorrect outputs. (2) Finetuning xLLMs100 on our cross-lingual human feedback dataset is more effective for low-resource languages than high-resource ones. This is due to the fact that highresource languages already exhibit strong understanding and generation capabilities in the vanilla LLMs (as shown in Table 3), which mitigates the impact of further finetuning xLLMs-100 on the cross-lingual preference data. (3) It is worth noting that despite aligning xLLMs-100 with crosslingual human preferences, its performance in lowresource languages is still not as good as that in high-resource languages. Although our constructed cross-lingual feedback dataset enhances multilingual performance, the inclusion of additional languages (our dataset currently includes 30 languages) might be necessary to support all lowresource languages in the evaluation benchmarks.

heading: 6.2 Different Datasets for Multilingual Tuning
text: In Section 1, we introduced two types of datasets to enhance the multilingual capabilities of LLMs: multilingual parallel corpus and multilingual instruction dataset. We study how these two types of data impact the multilingual capabilities of LLMs. To do so, we conduct comprehensive comparison experiments on these two types of dataset. For the multilingual parallel corpus, we use the NLLB dataset (Costa-jussà et al., 2022) collected by allenai10. We use the same set of 100 languages as xLLMs-100 for the sake of a fair comparison.
The experimental results presented in Table 5 clearly show that utilizing the multilingual parallel corpus leads to a significant improvement in the machine translation task. However, there is a notable decrease in performance observed in non machine translation tasks. This phenomenon is referred to as catastrophic forgetting (McCloskey and Cohen, 1989), where the model after fine-tuning achieves better performance on the new task at the expense of the model’s performance on other tasks. On the other hand, the model fine-tuned with multilingual instruction data demonstrates improvement across all tasks, indicating that attention should be given to the models’ performance on a broad range of tasks during fine-tuning, rather than focusing solely on performance gains in a single task. It is obvious that the multilingual instruction dataset is a better choice compare to the multilingual parallel dataset when finetuning LLMs.

heading: 6.3 Off-Target Analysis
text: Off-target (Zhang et al., 2020) refers to the generation of output in an incorrect language, which is a common issue in multilingual models. Following the approach of Lai et al. (2023c), we calculate the Off-Target Ratio (OTR), which represents the proportion of output sentences generated by a multilingual model that are in the wrong language.
10https://huggingface.co/datasets/allenai/nllb
This metric helps to assess the accuracy of language generation in multilingual models. Table 6 shows the results on the XL-Sum and FLORES benchmark. We observe that the off-target problem is more prominent in low-resource languages. Although our model has made some progress in addressing this issue, there is still significant room for improvement. Overcoming the off-target problem in low-resource languages continues to be a challenge that necessitates further research and development efforts.

heading: 6.4 Language Democratization
text: Language democratization, as proposed by Huang et al. (2023), is a metric used to evaluate the level of task democratization across different languages of a multilingual model. This metric is obtained by calculating the average percentage of different languages relative to the best performing language among all languages. It provides insights into the fairness and equality of performance across different languages in a multilingual model. According to Table 7, we observe that xLLMs-100 demonstrates a higher degree of linguistic democratization its peers across four out of six examined benchmarks. This implies that our model exhibits a smaller performance gap between different languages, indicating a more equal and fair distribution of performance across languages. This is a positive outcome, suggesting that our model is successful in reducing disparities and achieving more balanced performance across various languages.

heading: 7 Conclusions
text: To enhance the multilingual capability of LLMs in two dimensions (understanding and generating), we present xLLMs-100, two LLMs that support 100 languages. We train xLLMs-100 on a novel multilingual instruction dataset containing 100 languages. To improve the generating capability, we construct a cross-lingual human feedback dataset to further align the LLMs with human feedback and to enable the LLMs to generate output in multiple languages. Experiments on five benchmarks demonstrate the effectiveness of our datasets and also the multilingual capability of our models both on high-resource languages and low-resource languages.

heading: 8 Limitations
text: This work has the following limitations: (i) To make our computations environment friendly, our experiments have so far been limited to 7B size on LLaMA and BLOOM. However, our dataset has the potential to be deployed for larger LLMs (e.g., 13B and 70B models). We hope that the community will contribute to realizing this potential using our dataset. (ii) Our constructed human feedback dataset currently covers 30 languages. Given that ChatGPT also faces difficulties in obtaining highquality feedback across a majority of languages, determining how to extend the cross-lingual human feedback dataset would be promising future work. (iii) As shown in Appendix E and F, a noticeable discrepancy persists between xLLMs-100, small language models (SLMs) and the current state-ofthe-art LLMs, such as ChatGPT. This disparity is primarily due to models like ChatGPT leveraging larger quantities of data and model sizes. Therefore, bridging this gap will be a critical objective in future research. (iv) We do not conduct an analysis on toxicity (Deshpande et al., 2023), domain (Lai et al., 2022a,b), bias and fairness (Gallegos et al., 2023) aspects of xLLMs-100, which should be discussed more in future work.

heading: Acknowledgement
text: This publication was partially supported by LMUexcellent, funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Länder; and by the German Research Foundation (DFG; grant FR 2829/4-1).

heading: A Datasets
text: We construct two datasets for multilingual tuning: multilingual instruction dataset in 100 languages and cross-lingual human feedback dataset in 30 languages. We tokenize the inputs, instructions, and outputs of these two datasets separately, and present the statistics of the results in Table 12. We observe that both LLaMA have a large average number of tokens, indicating that the original LLMs do not adequately support all languages in our constructed datasets. The sentences are segmented into smaller units, which increases the difficulty of capturing semantic information. Addressing the tokenizer issue in LLMs is an important step towards expanding LLMs to new languages, and we would like to address this in future work. In addition, we show the languages covered in our constructed cross-lingual dataset in Table 8. Finally, the translator used for each language when translating the instructions and responses are shown in Table 11.

heading: B Model Configuration
text: The training process of xLLMs-100 contains two steps. In supervised fine-tuning, we use LORA to fine-tune LLMs on our constructed multilingual instruction dataset. In aligning LLMs with human feedback, we use DPO with LORA to fine-tune the LLMs. We train xLLMs-100 on one machine with 8 A100 80GB GPUs. The hyper-parameters for
xLLMs-100 are presented in Table 9.

heading: C Prompt Setting
text: To evaluate the generating capability of LLMs, we use English as the prompt language as shown in Table 13. To evaluate the understanding capability of LLMs, we use Google Translate API to translate the English prompt to the other languages.

heading: D Complete Results
text: We present the results of all languages for each benchmark: PAWS-X (Table 16), XCOPA (Table 18), Self-Instruct* (Table 17), XLSum (Table 19, Table 20) and FLORES (Table 21, 22, 23, 24, 25, 26, 27 and Table 28).

heading: E Multilingual Capability of LLMs
text: Table 10 illustrates the significant variation in performance achieved by LLMs across different examined languages, with the highest scores observed when tasks are described and presented in English. Furthermore, LLMs exhibit varying degrees of understanding capabilities across different languages, with some cases where they fail to understand the
language. For instance, when using Hindi as the instruction language, ChatGPT lacks an understanding of the instructions, leading to subpar performance.

heading: F Comparison with Fine-tuned SLM
text: While it is not feasible to evaluate a single small language model (SLM) across all the benchmarks we employed, we omit these results from Table 3. However, we do provide some experimental findings for reference. As shown in Table 14, our model does not perform as well as certain SLM fine-tuned on specific benchmarks (e.g., XL-Sum), we successfully narrow the performance gap between the original large language models (e.g., LLaMA) and closed-source large models (e.g., ChatGPT). This achievement is particularly meaningful as we refrain from using any task-specific data for finetuning. Interestingly, when SLM are not fine-tuned for a specific task, Table 15 demonstrates that our model significantly outperform the SLM in the SelfInstruct* benchmark.
Generating Capability
ar cs de zh hi hy ky yo ta mn Avg_L Avg_H
LLaMA 7.09 7.95 8.70 15.23 5.10 5.39 4.85 6.95 5.44 4.28 5.38 8.81 BXLLaMA 9.11 7.57 9.51 14.27 8.53 5.52 6.94 10.43 5.79 6.37 7.01 9.80 SFTLLaMA 10.06 8.54 9.77 19.40 12.98 7.77 5.27 9.73 7.62 5.11 7.10 12.15 xLLMs-100 12.44 12.78 15.49 19.90 12.94 3.79 9.28 11.48 10.82 10.42 9.16 14.71
BLOOM 6.52 4.90 6.12 11.13 6.41 2.37 3.44 5.99 5.20 3.34 4.07 7.01 BXBLOOM 5.74 7.08 9.40 12.26 6.55 3.32 6.58 8.15 5.34 6.02 5.88 8.21 SFTBLOOM 6.30 11.63 13.35 18.24 8.04 7.24 7.18 9.10 7.89 7.47 7.78 11.51 xLLMs-100 11.12 10.69 13.36 19.22 13.71 10.51 9.24 12.72 9.48 8.88 10.17 13.62
Understanding Capability
ar cs de zh hi hy ky yo ta mn Avg_L Avg_H LLaMA 6.71 12.64 13.52 20.19 9.79 5.59 4.50 14.82 5.08 5.45 7.09 12.57 BXLLaMA 8.89 7.88 11.99 21.09 9.54 5.31 4.36 11.64 4.88 5.37 6.31 11.88 SFTLLaMA 10.67 9.33 13.25 17.87 12.47 19.10 7.47 12.29 8.76 7.90 7.32 12.72 xLLMs-100 13.69 12.90 15.31 21.31 13.56 23.19 8.94 13.35 9.31 9.93 12.94 15.35
BLOOM 9.41 6.97 12.29 2.80 11.87 7.85 5.05 11.33 8.03 5.56 7.56 8.67 BXBLOOM 7.22 6.50 7.66 4.43 14.72 0.00 1.06 7.70 9.86 5.43 4.81 8.11 SFTBLOOM 8.89 7.88 11.99 21.10 9.54 5.31 4.36 11.63 4.88 5.37 6.31 11.88 xLLMs-100 12.06 8.35 13.07 22.45 10.83 6.48 5.16 13.31 8.33 6.43 7.94 13.35
Table 17: ROUGE-1 scores on the Self-Instruct* benchmark (Wang et al., 2023) and average scores on low-resource (Avg_L) and high-resource (Avg_H) languages.
Generating Capability
et ht id it qu sw ta th tr vi zh Avg_L Avg_H
LLaMA 50.00 49.80 51.00 51.80 48.40 50.40 49.80 52.40 50.60 50.80 55.20 49.33 51.52 BXLLaMA 50.60 47.60 49.40 49.40 48.60 50.20 47.80 49.20 49.00 50.60 50.40 48.00 49.85 SFTLLaMA 50.40 50.00 49.60 50.00 47.00 50.00 49.80 50.00 50.20 50.20 50.00 48.93 50.05 xLLMs-100 50.00 51.00 54.80 60.60 48.20 52.60 49.93 53.20 54.00 51.20 61.00 49.71 54.68
BLOOM 48.00 49.16 47.84 48.27 50.22 50.63 50.16 50.83 50.72 49.37 50.11 49.85 49.47 BXBLOOM 48.14 43.57 50.60 49.32 49.80 50.20 49.80 50.60 50.40 48.88 51.67 47.72 49.98 SFTBLOOM 49.00 50.00 48.80 49.20 46.80 47.80 50.60 49.20 47.40 51.00 51.80 49.13 49.28 xLLMs-100 51.22 51.59 52.35 51.42 52.83 51.28 52.66 51.50 51.73 53.26 55.34 52.36 52.26
Understanding Capability
et ht id it qu sw ta th tr vi zh Avg_L Avg_H
LLaMA 47.83 48.24 51.80 50.23 50.60 48.52 43.47 42.57 45.14 46.37 45.26 47.44 47.22 BXLLaMA 48.59 49.17 52.26 50.84 51.06 49.27 48.36 46.07 47.49 48.93 48.58 49.53 49.00 SFTLLaMA 49.26 50.28 52.85 51.44 51.59 50.17 48.70 48.26 48.37 49.34 49.15 50.19 49.86 xLLMs-100 52.80 49.60 53.20 53.17 52.48 51.00 52.51 50.75 50.37 51.75 52.66 51.53 51.96
BLOOM 46.15 43.27 50.14 46.25 48.28 47.34 41.25 48.74 48.37 53.16 52.96 44.27 49.14 BXBLOOM 47.26 45.14 52.68 47.85 49.94 48.50 43.77 49.16 48.93 54.25 54.16 46.28 50.35 SFTBLOOM 50.14 49.27 54.35 50.22 52.63 50.48 46.37 51.06 50.45 56.47 55.29 49.42 52.31 xLLMs-100 52.70 51.20 58.96 52.97 54.55 53.29 51.75 53.26 54.25 60.37 58.93 52.50 55.59
Table 18: Accuracy scores on the XCOPA benchmark (Ponti et al., 2020).

heading: Future Work
text: Addressing the tokenizer issue in LLMs is an important step towards expanding LLMs to new languages, and we would like to address this in future work. In addition, we show the languages covered in our constructed cross-lingual dataset in Table 8. Finally, the translator used for each language when translating the instructions and responses are shown in Table 11