X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
Authors: Haoran Xu
Kenton Murray
Philipp Koehn
Hieu Hoang
Akiko Eriguchi
Huda Khayrallah
Venue: ICLR 2025 Spotlight (2025)
Keywords: Large Language Model
Machine Translation
Multilingual
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We present X-ALMA, a multilingual machine translation model that prioritizes quality over quantity by delivering top-tier performance across 50 diverse languages, regardless of their resource levels

Abstract: Large language models (LLMs) have achieved remarkable success across various NLP tasks with a focus on English due to English-centric pre-training and limited multilingual data. In this work, we focus on the problem of translation, and 
while some multilingual LLMs claim to support for hundreds of languages, models often fail to provide high-quality responses for mid- and low-resource languages, leading to imbalanced performance heavily skewed in favor of high-resource languages. We introduce **X-ALMA**, a model designed to ensure top-tier performance across 50 diverse languages, regardless of their resource levels. X-ALMA surpasses state-of-the-art open-source multilingual LLMs, such as Aya-101 and Aya-23, in every single translation direction on the FLORES-200 and WMT'23 test datasets according to COMET-22. This is achieved by plug-and-play language-specific module architecture to prevent language conflicts during training and a carefully designed training regimen with novel optimization methods to maximize the translation performance. After the final stage of training regimen, our proposed **A**daptive **R**ejection **P**reference **O**ptimization (**ARPO**) surpasses existing preference optimization methods in translation tasks.

Published as a conference paper at ICLR 2025
X-ALMA: PLUG & PLAY MODULES AND ADAPTIVE REJECTION FOR QUALITY TRANSLATION AT SCALE
Haoran Xu" Kenton Murray Philipp Koehn" Hieu Hoang Akiko I. Eriguchi Huda Khayrallah Microsoff, "Johns Hopkins University, Amazon haoranxuêmi crosoffft..rroo..ooooooooooooffe com, (kenton, phi] 0 ft.. fhihoan, akikoe waakkkoe@emiioooooo Jemicrooof.. com, hudakh amazon. com
ABSTRACT
Large language models (LLMs) have achieved remarkable success across vari- NLP tasks with focus on English due English-centric pre-training and limited multilingual data. In this work, we focus on the problem of translation, and while some multilingual LLMs claim to support for hundreds of languages, models often fail to provide high-quality responses for mid- and low-resource languages, leading imbalanced performance heavily skewed in favor of high- resource languages. We introduce X-ALMA, model designed to ensure top- tier performance across 50 diverse languages, regardless of their resource lev- els. X-ALMA surpasses state-of-the-ar open-source multilingual LLMs, such as Aya-101 (Üstün et al. 2024) and Aya-23 (Aryabumi et al. 2024). in every single translation direction on the FLORES-204 200 and 2M test datasets according to COMET-22. This is achieved by plug-and-play language-specific module ar- chitecture to prevent language conflicts during training and carefully designed training regimen with novel optimization methods to maximize the translation performance. After the final stage of training regimen, our proposed Adaptive- Rejection Preference Optimization (ARPO) surpasses existing preference opti- mization methods in translation tasks]
INTRODUCTION
Large language models (LLMs) such as the GPT series (Brown et al. 2020: OpenAl 2023), Mistral Jaan et al. 2023), LLaMA series (Touvron et al.] [023a bJ Dubey et al. (024). Gemma series Team et al. 2024a b), inter alia have demonstrated impressive performance across various NLP tasks. However, the efficacy of LLMS has primarily been evaluated on English tasks, with their multilingual capabilities receiving less attention due to the models being predominantly pre- traitrinnd on English and the scarcity of multilingual data. Recently, there has been a shift towards multilingual studies in LLMs. For instance, LLaMA 3 and 3.1 (Dubey et al. 2024 expand the vocabulary from 32K to 128K and pre-train on multilingual texts; Üstün et al. multilingual generative model supporting 101 languages; and Big Translatee (Yang al. 2023 and
2024 have introduced Aya-101, a
LLaMAX (Lu et al. 2024) scale LLM-based multilingual translation models to over 100 languages.
Despite the increased language support in LLMs, their performance across most languages falls short of practical application expectations, especially for mid- and low-resource languages (weak- ness 1). Furthermore, the performance of high-resource languages tends be inferior compared to LLMS trained with fewer languages, a phenomenon known as the 'curse of multilinguality (Coneaau et al. 2020) (weakness 2). The weaknesses are prevalent in most current state-of-the art (SoTA) massively multilingual models: overall quality decreases as the number of supported languages increases. Although methods such as building models by focusing on smaller number of high-resource languages like German and Chinese can achieve satisfactory performance for these languages and mitigate these weaknesses (Aryabumi et al. 2024 Xu et al. 2024a b Alves et al.
I Work done at Microsoft.
'Code is released at hettps://githhe wtw..tttoo.o....eeaeeeeeeeeeeeee ixte/AxXxAAMMMM Models and Dataset are released
htt:s::/huggingg www.aa/aaaeeeccmo Micc//XAAMM

---

Published as conference paper at ICLR 2025
2024). they neglect the needs of mid- and low-resource languages. In this work, address these weaknesses and build a multilingual model that achieves consistently high performance across 50 diverse languages, regardless of resource level, with focus multilingual machine translation.
89.5
X-ALMA 50 languages
88.5
ALMA- é Llaquugge
Avg. COMET-22
Aya-23-338 23 languages
87.5
Aye- 23 88 languages
86.5
Apa-100: 10B lorlaggssees
85.5
ILaNA33 languages
84.5
200 languages
83.5
4
16
64
256
Number of Languages Supported
Depiction of the general inverse trend between the number of supported languages and average translation performance. While many state-of-the-ar multilingual models claim to support hundreds of languages, the translation quality is not as high as in models trained on fewer languages, particularly for mid- and low-resource languages. This is reflected in the trend of decreasing average scores more languages are supported. contrast, we propose X-ALMA, which extends ALMA-R (Xu et al., 2024a b) by supporting 44 additional diverse languages with even higher average perfor- mance, offering top performance across all supported languages, regardless resource level.
Figure 1:
To visualize these weaknesses, let closely examine current models in the context of multilingual translation. We evaluate each model on the overlapping set of languages that are supported by the model and the 50 languages focus on in this paper[] As shown in Figure there is clear trend: as the number of supported languages increases, the average translation performance decreases. This intuitively understandable,
difficult for mid- and low-resource languages reach the same level of performance as high-resource languages, thus lowering the overall average. For instance, ALMA-R (Xu et al., 2024b) achieves the highest average translation performance across the lan- guages it supports, while NLLB-200 2-200 (Team et al., 2022) exhibits the lowest average performance on 50 languages, largely due poorer results low-resource languages. Although this comparison not entirely fair due to the varying number of languages tested, it provides general indication of above-mentioned weaknesses multilingual models
Despite the ability of current multilingual models to support hundreds of languages, the hollow purple star "s' in the figure represents our ideal model, where the inclusion more languages does not diminish the average performance. In this work, introduce
multilingual translation model, X-ALMA, represented by the solid golden star * in Figure which extends ALMA(-R) (Xu al,, 2024a b) from 6 languages 50 languages. ALMA-R is of pop-pffoorming translation models built on LLMs, comparable to WMT winners and GPT-4-turbo. Despite the addition of 44 more achieves slightly higher average performance compared to ALMA-R.
languages, X-ALMA
We summarize our main contributions as follows, including our model architecture design and train- ing methodology.
Plug-and-Play Architecture: For capacity reasons, we design X-ALMA with several different modules with each module serving a group of similar languages. These modules can either be plugged into the base model individually for the inference of target randugges-reducnng the neces-
This to depict trend, we acknowledge scores are not directly comparable across languages. Here evaluated these models using FL.ORF--200 (Team et al. 2022) test data and reporting the average COMET-22 (Rei al. 2022) across all languages, to from Ennlish.
2

---

Published as a conference paper at ICLR 2025
sity of loading all expert parameters and saving memory- or all modules can be loaded together in a mixture-on mixture-of-expprts of-experts (MoE) way (Shazeer et al. 2017 Lepikhin et al. 2021).
Effective Training Recipe: The training regimen for X-ALMA consists of three pre-training stages and two post-training stages, each crucial for achieving optimal performance. Furthermore, in the final stage, introduce Adaptive- -eeiieeeonneeiinnn Preference Optimization (ARPO), designed to maxi- mize performance and address the 'over-rejection' issue (detailed explanation in Section 4 in trans- lation preference learning, which current optimization methods struggle to resolve.
State-of-the-Art Performance and Data Relcase: X-ALMA outperforms existing open-source multilingual translation models across 50 diverse languages for every single direction only training on publicly available data, as measured by COMET-22. To enable future work, we also release the preference learning data for 50 languages and the model checkpoints.
BACKGROUND
2.1 PROBLEM DEFINITION
We consider a decoder-only LLM, denoted as To. parameterized by 8, for multilingual machine translation tasks. Let D represent the multilingual dataset. consisting of pairs of source sentence r and the corresponding perfect translation y, represented as D = {æ, y). Given prompt that instructs the model to perform the translation, our goal is to maximize the log-likelihood of the multilingual parallel dataset D: maxg ((z,y)> ne Te [y]]
2.2 RELATED WORK
Multilingual Translation Massively Multilingual Translation models JJohnson et al. 2017). in- cluding open-source models such as PRISM Thompson & Post (2020a b). M2M-100 (Fan et al. 2020), and NLLB (Team et al. 2022) combine translation between many language pairs in a single
encoder-decoder model. T5 (Raffel al. 2020 and mTT5 (Xue et al. 2021 considered translation
one of multitask learning.
LLM-Based Translation Initially, decoder-only LLMs struggled to match the performance of con- ventional encoder-decoder models for MT. For example, GPT-3.5 slightly under-performen the con- current WMT winners (Hendy et al., 2023). and large open-source models like OPT-175B (Zhang cet al. 2022) performed worse than the 1.3B parameter NLLB model (Team et al. 2022), even on high-resource languages, as demonstrated by Zhu et al. (2024b). This lead to an increased interest in smaller LLMs, such as 7B or 13B models, because even smaller models like NLLB-113B showed strong translation capabilities. However, first generation -ased MT models such as TIM (Zeng et al. 2023), SWIE (Chen et al., 2023), and BayLing (Zhang et al. 2023) still lag behind encoder- decoder models performance. The under performance of LLMs on translation lead to hybrid approaches combining LLMs with dedicated NMT models (Petrick al. 2023 Hoang al. 2024). Recently, GPT-4 (OpenAIl (203) has been reported to achieve top performance in the WMT com- petition (Koemi et al.. 2023), and smaller L.-M-based models, like ALMA(-R) (Xu et al. 2024a b) and Tower (Alves et al. 2024), have demonstrated comparable performance to GPT-4 by employ- ing their specialized training methods. However, the high performance of LLM-based translation models is limited to small subset of languages.
Massively Multilingual LLM The limited scope of languages in LLM-based MT models stems primarily from English-focused pre-training and the use of restricted vocabularies. However, this limitation has driven interest in expanding these models to support a broader range of languages. The simplest approach to extending current LLM-based MT models involves expanding the vocabulary and training on large amounts of parallel data across additional languages (Yang et al. (023), but this approach has been shown to degrade model performance. Aya-101 (Ustún et al. 2024) revisits the encoder-decoder architecture, building a multilingual model based on the largest MTS (Xue et al.) 2020), designed not only for translation but also for general multilingual QA. Similarly, LLaMAX (Lu et al. 2024) extends LLaMA-2 and LLaMA-3 to over 100 languages. However, multilingual models often suffer from reduced performance mid- and low-resoures languages, which can also negatively impact high-resource language performance. To mitigate this, the decoder-only model Aya-23 (Aryabumi al. 2024) focuses exclusively on 23 high-resource languages to maximize
3

---

Published as a conference paper at ICLR 2025
T.MODULE
LAUCURGE- SPECIFIO MODULE
GROUP 2. MODULE
BASE MODEL
GROUP MODULE
Figure 2: High-level architecture design of the plug-and-play multilingual model. Each language group is assigned a specific module that works alongside the base model. These language -eguuag--specffic modules handle inputs exclusively from their respective language groups, enabling the model to effectively adapt to different linguistic characteristics while leveraging the shared base model for comprehensive multilingual learning.
their performance and avoid the 'curse of multilinguality'. While limiting the number of supported languages can indeed alleviate some challenges, reverses the goal of building truly multilingual models and neglects the needs of mid- and low-resource languages. In this paper, we expand ALMA- R from 6 languages to 50, ensuring robust performance across all languages.
3 METHODS
3.1 MODEL ARCHITECTURE
Our model architecture consists of: (1) a dense base model, and (2) multiple language-specific (LS) modules. The core concept of LS modules is to prevent conflicts between languages during training, such as gradient conflicts (Wang et al., 2021. This design has similarities with the mixture-of- experts (MoE) approach (Shazeer et al.) 2017: Lepikhin et al. (02T), but diverges by not using neural-based gate to assign tokens to experts (LS modules). Instead, similar to Xu al.] (2023), the assignment is hard-gated, i.e., input data is assigned exclusively to the module designated for its language. Consequently, only the base model and the corresponding LS module are activated, depending the input language. Languages sharing common LS module. An overview of the model architecture illustrated in Figure
categorized into distinct groups, with each group
In detail, the base model architecture is built upon the LLaMA-2 architecture (Touvron et al. 2023b Each LS module comprises low-rank adaptations (LoRAs) (Hu et al., (2021) integrated into all linear laLP)aaerrs
layers within the attention and multi-layer perceptron (MLP) layers. The total number of parameters for each LS module is approximately 15% of the base model.
Why This Design? While model architectures such MoE activate only expert per example, all experts must still reside in GPU memory during training and inference, necessitating high-end GPUs. Moreover, MoE has been reported for its parameter inefficiency in multilingual settings, e.g., hard-gated language assignment can achieve similar performance to MoE while using 4 times fewer parameters (Xu al., 2023). Compared to MoE, our design offers three distinct model- loading strategies for both training and inference: (1) selectively loading single, on- LS module, which alleviates GPU memory constraints; (2) merging LS modules with the base model to generate a new L.S L.M model that retains the same parameter count the base model, facilitating subsequent use; and (3) loading both the base model and all L.S modules as larger, combined model, similar to the approach employed by MoE.
3.2 LANGUAGE GROUPING
In this paper, we consider a total of 50 languages, encompassing 14 scripts and 18 language fam- ilies, to capture the linguistic diversity. The languages categorized into 8 groups based on two
4

---

Published as a conference paper at ICLR 2025
Table 1:
Language grouping based on linguistic features and balanced number of languages.
Group ID
Linguistic Feature
Languages
Germanic languages
a f, da, de, is, no, sv, (en) ca, es, gl, it, pt, ro, (en)
2
Romance Languages
3
Eastern and Southern Slavic Languages
bg, mk, ru, sr, uk, (en)
4
Southeast Asian Languages
fr, id, mg, ms, th, vi, (en) cs, el, hu, 1t, 1v, pv, (en) et, fi, ja, ka, ko, zh, (en)
5
Central and Eastern European Languages
6
Eurasian Language Mix
7 Turkic and Semitic Languages
Indo-Aryan Languages
gu, hi, mr, ne, ur, (en)
ar, az, fa, he, kk, ky, tz, uz, (en)
criteria: (1) each group should consist of languages that are as similar as possible, and (2) the number of languages in each group should be balanced. We opted not to use automated tools like Leng2Vec (Littell et al. 2017) for grouping, we found that manual grouping based on human linguistic knowledge yields more accurate classification in line with our criteria. The specific lan- guages within each group presented in Table with their ISO-539-1 code. Note that English (en) is included in all groups to ensure that each group can perform English-centric translation. More detailed information on these languages is provided in Appendix A
3.3 TRAINING RECIPE
We provide a comprehensive description of the training recipe for the X-ALMA model, including three pre-training stages and two post- training stages. An overview of this training recipe is depicted in the workflow diagram in Appendix B The specifics of each stage are elaborated upon as follows.
Pre-Training Stage 1: Monolingual Fine-Tuning Base Model The first stage of pre-training is dedicated exclusively to the base model. During this phase, we fine-tune the base model using 20B monolingual tokens from all 50 languages, with sampling ratio proportional to the size of the available monolingual data for each language, as suggested by Xu et al. (2024a). This stage aims to facilitate the model's acquisition fundamental knowledge across all languages.
Fine-Tuning Tuning Laneuage-Speccfic Modules In all subsequent stages, the base model remains frozen, and the focus shifts to fine-tuning LS modules. During the second stage of pre- pre-training, each LS module is fine-tuned with 10B monolingual tokens exclusively from the languages within its respective group. This stage is designed to enable each LS module to emphasize learning general knowledge across the specific languages.
Fre-Training Stage 2: Monolingual Fine-
Pre-Training Stage 3: Mseudo-Monolingual Fine-tuning In this stage, we continue to fine-tune data from each module's language group. This pseudo-
the LS modules using monolingual data is constructed from parallel sentences. While previous studies have indicated that simple instruction tuning with large volume of parallel sentences for instruction tuning can degrade model performance (Xu et al. 2024a Zhu et al. 2024a). recent research demonstrates that utilizing parallel data in the pre-training stage can enhance multilingual alignment (Alves et al. 2024| Kondo et al.) 2024 Lu et al, 2024). Similar to these approaches, we combine each available translation pair to create new sentence in either <source sentence> sentence><<target sentence> or <targer sentence><<sone soorce sentence>> manner, with the order of the source and target sentence in each pair determined randomly. We then concatenate all the combined translations to construct the pseudo- monolingual data. Each LS module is fine-tuned on 1.25B tokens.
Post-Traine Training Stage 1: Supervised Fine-tuning Building on the insights from prior research that small but high-quality multilingual datasets are sufficient yield impressive performance (Maillard ctt al.) 20233 Xu et al., (2024)), we supervised fine-tune (SFT) the model using a small, high-quality parallel dataset at this stage with the translation prompt suggested by Xu et al. (204a). This fine- tuning is performed using a simple causal language modeling (CLM) loss.
Post-Training Stage 2: Preference Optimization We also introduce Adaptive Rejection Prefer- ence Optimization (ARPO) to further enhance translation quality across all languages. ARPO is designed to address the 'over-rejection' issue in MT preference learning, challenge that other pref-
5

---

Published as a conference paper at ICLR 2025
erence optimization methods struggle to manage effectively. We will elaborate motivations, methodology, and preference data construction in the following section.
ADAPTIVE-REJECTION PREFERENCE OPTIMIZATION
4.1 LIMITATIONS IN CURRENT PREFERENCE LEARNING
When constructing preference data for MT, it is essential that the dis-preferred translation is also of high quality to ensure meaningful model improvement (Xu al. 2024b). This results in a scenario where the preferred and dis-preferree translations are often very similar. differing by only a few words, which quite different from the preference data used in open-ended question-answering (QA) tasks (A detailed example is shown in Appendix (C. While many preference optimization methods have proven effective in various NLP tasks (Rafailov et al.) 2024 Azar et al.) [024] Hong al. 2024. Meng et al. 2024), we find that they are not well-suited for the MT task because they lend to reject the entire dis- dis-preferred translation which is similar to the preferred one. This approach can inadvertently lead to the rejection of most tokens in the preferred translation well, resulting in phenomenon we term over overrrejection where the writing style of the translation outputs is forced away from the preferred data distribution (further analysis and examples in Section 6).
Mathematically speaking, the preference optimization problem can be generally formulated given a
dataset D = { ((((), Viu Vi i=="" where each data point consists of prompt (source sentence) x, preferred response (translation) Ye. and a dis- dis-preferred response y!, for total of N data points: (ro(U - ro(() (z)
(t) (i)
(1)
where f(-) : R + IR is a general non- non-linear function. In many instances, such as in the DPO method RRfailovv et al. 2024), is the negative lellikklihood of the Bradley-Terry objective, i.e., f()) = log (-). Here, ro x) represents the reward of y. calculated according to log-probability of the policy model parameterized by A. In the case of DPO, the reward function is defined as ra (ylzz) 8 log( (To ()) - 3 log( (Trr())))) where 3 is hyperparameter and Tre is the reference model.
As indicated by Equation when Yw and yr are too similar, the difference between ro ro (x) and re (w]lz) tends to be small and even near O. Consequently, the near-zero difference between rewards causes the preference loss to a constant value, such as f() = - log (0) in the case of DPO. This makes challenging for the optimization process to distinguish between the two options, hindering meaningful improvements.
The challenge becomes even more pronounced in a finite-data regime. While an infinite number of response pairs with small but precise preference differences could mitigate the optimization difficul- ties, translation preference data is often sparse and may contain noise (e.g., the Al-labeled preference data used by Xu et al.) (2024b) contains only 2K samples per direction). Consequently, the model is prone to overfitting to these minor differences, which poses a significant empirical challenge and can lead to suboptimal learning outcomes, particularly when dealing with large response (translation) space, as is the case with LLMs.
4.2 ADAPTIVE REJECTION
To mitigate the over-rejection, introduce an adaptive penalty, denoted Ta. which controls the strength of the dis- dis-preferred term in the loss:
CARP
E(z.ow
(rro(uulzz) To (y 91) ro ro((ul=))
(2)
The value of Te is determined by the similarity between yo and yı, ranging from 0 to To (u vi) min(e":=sov....)) 1,
(3)
where yj is a hyperparameter, and z (y m1) is a function that quantifies the distance between the the preferred and dis dis-preferred responses by measuring absolute difference of their average log- likelihoods:
log( no [x)
log (Te (y1 (z)
zo(yw: vI) = abs(
(4)

---

Published as a conference paper at ICLR 2025
When Vu and Y very similar, the absolute difference between their averaged log- sigllikeiihooss is small, resulting in Ty close to O, thereby reducing the impact of the dis preferred term on the loss and mitigate rejection on this translation. Conversely, when the difference between yo and yr is large, To close to 1, turning the loss back to a standard preference optimization loss.
In the multilingual MT task, start with contrastive preference optimization (CPO) (Xu et al.) 2024b), which has proven to be one of the most effective optimization methods for translation.
Ccpo (r, )-D log ( log no (y x)) - 3 log To (o xx) log To (1
(5)
BC
preference loss
CPO consists of two components: preference loss and behavior cloning (BC) loss (Hejna et al.
The BC loss helps prevent the model from drifting too far from the original task. Then, incorporate adaptive rejection into the preference term in CPO, resulting in new loss function:
2023
CARPO
log log ( (x) -0- To (Yer 3: log Te (w(()z) +log To (Vullz)) (6)
5
EXPERIMENTS
5.1 DATA
Monolingual and Parallel Data Following the introduction of 50 languages in Section 3.2 we focus on 98 Inglish-centrin translation directions, both into and from English. We test on Flores-200 test data (Team et al. 2022) and 2M (Koemi et al. 2023 For pre-training stages and 2, we
use monolingual data from OSCAR (Ortiz Su' arez et al. 2019). In pre-training stage 3, we construct
menuoo-monlinguaal data using NLLB (Schwenk et al. 202 Heffernan et al. 2022, Team et al. 2022 and OPUS TTedemmann 2012 Zhang et al. 2020)) parallel training data. Web crawled data (included in NLLB) has been shown to contain substantial mis-aligned and mis-translated segments KKhayrallah & Koehn 2018 Kreutzer et al.
2022) and low-quality machine translated segments Thompson et al. 2024). Therefore, in the SFT step building on the insights from Xu et al. (2024a) that small amount of high-quality data can significantly enhance translation performance-
use the Flores-200 dev set and NTREX (Barrault et al.) 2019; Federmann et al. 2022) test data as our training data to ensure the quality. Given that both Flores-200 and NTREX are mulli-way-parallll datasets (all languages share the same English source sentences), we also incorporate the WMT' 15- 22 test data in training. The final data size in the SFT stage for each direction ranges from 3K 7K, with an average of 4K per direction.
Preference Data Construction Given the scareity of preference datasets for multilingual MT, we describe our approach to constructing preference data for 50 languages. Starting with the parallel data used in SFT, for each source sentence x, we generate a translation yxalma using X-ALMA that has been fine-tuned through SFT. Then, the reference translation Yret is designated the preferred translation, and Vxaimg as the dis-preferred one, forming our initial preference dataset, denoted as Dj = {zz Yeef: Vsalmm) Unlike Xu et al. (2024b), avoid the use of reference-free methods like XCOMET (Guerreiro et al. 2023) for ranking translations in preference data construction to avoid potential bias, the same metrics are used for evaluation. As result, DD1 might contain some noise due to the assumption that reference translations are always preferred. To reduce this noise, for high-resouree languages, we also employ GPT-4o to produce revised translations Vept conditioned on (z, Vsalma), ). drawing on studies that show post-editing by LLMs can improve translation quality (Ki & Carpuat| 2024) Feng et al., 2024 Raunak et al. 2023). We show the prompts in Appendix [D] Thus, our second preference dataset is defined as D2 = {-. Mgpt, Dsalma }. We then concatenate the two datasets to form the final preference dataset, denoted as D = DDD.
5.2 EVALUATION
We report COMET-22 (Rei al. 2022) main metric suggested by Freitag al. (0033 2024]. In Appendix E we also include XCOMET-XL reithoutreferrnness (Guerreiro et al. 2023) as
recommended by Xu al. (2024b), and BLEU (Papineni al. 2002 Post 2018) for completeness.
httts::/huggingffaac htt.://gnnnnag.eeooootataaa/aaaa

---

Published as a conference paper at ICLR 2025
The overall results of Flores test data across each language group in en-xxx Scores are reported using COMET-22. X-ALMA outperforms both massively multilingual models, such as Aya-101, and models focus specifically on high-resouree languages, like Aya-23, 'All' repre- sents the average performance across all languages in the group, while 'High' refers to the average performance for higu-resource languages the group. Bold numbers represent the highest scores.
Table 2:
Models
All
Group
High
All
Group
High
All
Group
High
All
Group
High
All
Group
High
All
Group
High
All
Group
High
All
Group 8
High
LLLMA-3. 1.B--...---
83.7
75.1
76.3
71.1
NLL.8-338
88.2
88.8
88.3
88.1
89.4
89.1
87.1
88.2
89.2
89.8
87.5
87.5
80.1
80.9
88.1
87.5
LLaMAX3-Alpase
86.4
86.9
86.8
86.6
85,7
82.0
81.7
86.2
86.6
87.1
86.0
87.3
76.5
76.6
82.6
83.6
Aya-101
87.5
76.2
86.8
Aya-23-8B
75.1
84.7
86.6
86.6
74.4
75.7
74.6
88.7
70.6
77.3
67.1
79.8
68.9
79.3
76.0
87.9
Ay--23-3518
79.6
86.5
87.1
87.0
77.6
78.5
76,7
88,6
82.1
86.0
73,9
84,4
61.9
79.1
68.8
87.8
10.3
16.7
30.2
36.5
9012"
S81
59.
9905
D6.7
90.
9074
82.6
11.
38.
15.
X-ALMA
89.6
89.9
89.4
89.0
90.9
90.5
88.6
89.5
91.0
91.1
90.6
90.8
83.2
81.9
89.4
89.2
Table The overall COMET-22 scores of Flores test data across each language group xx xx- Similarly, X-ALMA outperforms all baselines.
Models
Group
Group
All
High
All
High
All
High
All
High
All
High
All
High
All
High
All
High
LLAMA-J. Insttact
68.3
77.6
10.9
76.9
51.2
53.8
65.6
76.4
54.8
60.2
55.8
66.9
47.5
53.7
53.7
D7.5
NLL8.3.38
79.1
81.8
84.5
85.0
84.3
83.8
81.1
85.4
74.9
76.0
76.1
77.3
88.3
88.9
79.5
88.3
88.5
88.1
87.9
87.0
86.8
86.2
87.9
87.0
87.2
81.7
87.7
83.6
58.9
84.7
87.7
Aya-101
87.2
88.2
87.6
87.6
85.4
85.5
86.2
87.6
86.3
86.5
86.5
86.7
84.8
87.5
85.9
87.1
Aya-23-8B
84.6
88.2
87.9
87.7
83.3
83.3
79,8
88.5
82.9
85.2
79.9
86.1
71.8
89.1
76.7
88.0
Ayn-23-35B
87.4
88.9
88.8
88.6
86.3
86.2
82.3
88
86.4
87.2
85.9
88.0
79.4
82.7
88.6
X-ALMMA (only SPT)
89.1
89.2
88.8
88.6
87.9
87.3
87.7
88.8
87.9
38.1
88.2
88.3
89.3
89.8
87.5
38.4
88.1
88.0
88.2
88.7
88.0
5.3 TRAINING SETUP
We use ALMA- 13B-Pretrain (Xu et al., 2024a) our backbone model, which is pre-trained on 6 languages and based on LLaMA-2 (Touvron et al. 22223b). Following Xu al. (2024a), we pre-train the backbone model with batch size of 256, warm-up ratio of 0.01, and sequences containing up to 512 tokens. In the post-training stage, the model is fine-tuned for many-to-many multilingual translation manner using epoch with batch size of 128, and other settings remain unchanged. For preference learning, we set 1) as 1.5 and 39 as 0.1 for all experiments.
5.4 BASELINES
We use the strongest open-source massively multilingual translation models as our baselines, in- cluding NLLB-200 (Team et al. 2022). Aya-101 (Ustün et al. 2024), and LLaMAX3-Alpaca (Lu al. 2024). Additionally, we compare our model's translation performance with Aya-23-8B and Aya-35B (Aryabumi et al. 2024) to demonstrate that increasing the number of supported languages does not compromise the performance of high-resource languages, effectively mitigating the curse of multilinguality. We also include LLaMA-3.1-88 1.1-BBB.BBBB..... -.n-stttttttt baseline to assess the performance of one of the latest strong LLMs in multilingual translation.
5.5 RESULTS
We present the average performance for each language group in both en-*xx and xx-tn direc- tions on the Flores-200 test data in Tables 2 and 3 The results for WMT'23 in both directions are provided in Table Detailed results for each translation direction can also be found in Appendix
General inerrcctiontuuned LLaMA-3. signifi- cantly lags behind models specifically designed for translation, so we primarily focus on other mod- els. X-ALMA outperforms other massively multilingual models such as NLLB-3.3B. LLaMAX3- Alpaca-8B, and Aya-101 on average across all language groups, both into and from English for both Flores-200 and WMT'23 test sets. Furthermore, X-ALMA surpasses Aya-23-8B and Aya-23- 35B-both of which are tailored for high-resource languages on average across all high nigh-resource languages in each group. In fact, as detailed in Appendix E X-ALMA surpasses all baselines in all translation directions according to COMET-22 and outperforms in 97 out of 98 directions based on XCOMET-XI,, achieving top translation performance for all languages considered.
Compared with SOTA Multilingual Open Models:
8

---

Published as a conference paper at ICLR 2025
Table Results on 2M dataset reported using COMET-22. The symbol represents lations from English into the target language, while &- indicates translations into English.
Models
he
Avg.
ALMA-R-13B
84.0
85.5
85.0
80.6
85.5
83.3
TowerInstmet-7BB
83.1
84.6
80.5
83.1
NLLBB-133
79.7
66.6
79.6
67.8
81.6
65.8
83.8
76.7
82.8
79.0
83.6
79.9
81.8
72.6
LlamaX3-8B
73.3
79.4
81.5
79.3
81.8
80.1
81.6
81.3
80.6
84.9
82.5
83.0
80.2
81.3
Aya-101
75.1
81.6
78.6
73.7
84.6
77.3
81.4
82.7
84.5
82.9
81.0
80.2
Aya-23-8B
80.4
82.1
85.3
78.8
86.5
80.2
81.6
84.3
85.0
84.3
84.9
84.2
82.1
Aya-23-3518
80.7
82.3
84,6
79.7
86,4
81.6
84.7
82.2
84.0
85.7
84.1
85.9
84.1
82.9
X-ALMA (onfy Tonfy SFT)
84.1
83.3
86.
80.3
86.8
806
85.9
62.4
85.
86.4
86.1
84.4
83.7
833
84.4
85.7
87.5
82.4
85.5
Table Average performance comparison of various preference optimization methods for en- and xx-ten on Group 6.
Avg. en-txx
Avg. xX-ten
Models
COMET-22
XCOMET-XL
BLEU
COMET-22
XCOMET-XL
XALMA (only SFT)
90.1
32.1
88.2
77.4
+PO
0.7
53.6
51.1
7.1
79.2
64.6
+ BC
23.5
90.2
80.0
27.8
87.5
77.4
SimPO
0.0
I6.7
133
0.0
16.4
8.9
+ BC
23.3
89.7
78.7
26.6
87.1
76.5
+ KTO
22.1
89.8
79.2
26.4
87.1
76.5
+ BC
26.4
90.3
80.4
29.2
87.5
77.1
FORPO
23.0
85.8
75.9
22.7
81.3
70.8
+PO
22.2
90.2
79.9
26.5
87.8
77.0
+APPÓ (Final X-ALMA)
27.8
90.6
81.3
32.2
88.7
78.4
Effectiveness of ARPO: ARPO delivers consistent improvements compared to SFT-only models in
Flores-200 and 23 Similarly, shown in the full results in Appendix [H] ARPO enhances performance in every translation direction as measured by COMET-22 and delivers improvements in 95 out of 98 directions according XCOMET-XL. We also compare the effectiveness of ARPO against other preference optimization methods in Section6.1
6 ANALYSIS
All analyses are conducted on languages in Group 6, as it is the most challenging group to learn due its mix of typologically diverse Asian and European languages.
6.1 PREFERENCE OPTIMIZATION COMPARISON
Here, we compare ARPO with other popular optimization methods, including DPO (Rafailov al., 2024), KTO (Ethayarajh et al. 2024), ORPO (Hong et al. 22024). SimPO (Meng et al. 2024). and the original CPO [Xu et al. 2024b). As indicated by CPO findings, directly applying preference learning to the MT task can harm the model, but adding a behavior cloning (BC) regularizer can stabilize training and improve the performance (Xu al. 2024b). Following them, we also incorporate BC regularizer into optimization methods that do not originally include to provide a fair comparison. Table presents the comparison of preference optimization methods across all three metrics. As shown, ARPO clearly outperforms all baselines.
Over-Rejection Over-rejection manifests itself under the significant shift writing style away from the preferred data distribution. We observe clear and big BLEU scores drop across all other preference optimization methods, indicating decline in lexical matching. However, for certain methods, such as DPO + BC, KTO + BC, and CPO, both COMET scores do not decrease dras- tically (and in some cases even improve slightly for en-xx), suggesting that the models still pro- duce accurate translations that maintain the same semantic meaning, but with different writing style. Some translation examples generated by CPO are provided in Appendix F Unlike ARPO, methods such CPO tend produce wider range of writing styles convey the same meaning

---

Published as a conference paper at ICLR 2025
as the reference, many of which accurate and non-detrimental. However, excessive shifts in writing style still can introduce translation errors that negatively impact overall quality. These small number of errors, concealed within lot of stylistic deviations, where over-rejection occurs. As hypothesized in Section 4 the significant style shift is caused by the model rejecting dis-preferred translations that similar preferred ones, leading styles from the preferred data. However, ARPO addresses this issue by constraining the stylistic variation within a more controlled range, thereby mitigating errors caused by over-rejection.
excessive rejection of certain writing
For other methods such as naive DPO and SimPO, which even though work well in other NLP tasks, the over-rejectoon severely impairs the model's ability to generate meaningful translations. The introduction of ARPO significantly mitigate the over-rejection issue (stable BLEU scores) and maximize the translation qualities (the highest scores in two COMET metries).
6.2 ABLATION STUDY
Training Recipe We investigate the impact of each step in the training recipe on model perfor- mance. The average results for Group 6, both into and from English, are presented in the left part of Figure 3 The results show clear trend of consistent performance improvement with each step in the training process. Note that 'None' is the initial checkpoint in our recipe, ALMA- ALMA-11B-Prettee -3eetrraan
Parallel Data for SFT For SFT, we use high-quality parallel data from three sources: NTREX, WMT, and the Flores-200 dev set. Here, we investigate how combining parallel datasets affects per- formance during the SFT stage. As shown on the right of Figure using only NTREX data already achieves impressive average translation performance for Group 6. Adding high-quality WMT data further boosts average performance, particularly for translations into English data. We hypothesize that this improvement stems from the increased diversity of English data, which mitigates overfitting to the NTREX English domain-4 known issue with multi-way- wat--waay-ppaleeee data, as observed by Ahaa roni et al. 2019). Conversely, incorporating more Flores-201 200 dev data (also multi-way-paree -prallel) into training does not result in significant gains, also suggesting that the strong translation performance is not driven by in-domiin Flores-200 data.
90
Beugrial Thim
50.4
20L4
Pued BuAs
X7.6
STREEN
15
STRRX WhIT
WOEE
Aog. COMETT-22
80
COMICT
78.2
SS1
63-
64.5
Direction
Directim
Figure 3: Left: ablation study each stage of the training recipe, demonstrating that adding each stage leads to consistent performance improvements. Right: ablation study on the impact of parallel data composition during the SFT stage. Adding WMT data to NTREX significantly enhances model performance, while adding Flores-200 data provides no noticeable improvement.
7 CONCLUSION
We tackled the challenge of achieving high translation quality while scaling to large number of languages, a limitation seen in many state-of-the-art multilingual models. We have introduced X- ALMA, an LLMM-based multilingual translation system that prioritizes translation quality across all supported 50 languages, regardless of resource level. X-ALMA surpasses SoTA open models such as Aya-101 and Aya-23 in all translation directions on the -OOORES-200 and 2M test datasets, as measured by COMET-22. X-ALMA is built on a plug-and and-play architecture with language- specific modules, complemented by carefully designed training recipe. In particular, the final stage of the recipe, ARPO, achieves further performance gains and outperforms existing preference optimization methods in translation tasks, while successfully mitigating the over-rejection issue.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We thank anonymous reviewers for their insightful feedback. We express our profound appreciation HyoJung Han, Jack Zhang, Tianjian Li, Thamme Gowda, Tom Kocmi, Young Jin Kim, Hany Has- san Awadalla, Marcin Junczys-Dowman Grundkiewicz, Arul Menezes, and Vishal Chowdhary for their engaging and valuable discussions that greatly enriched our work.
Vikas Raunak, Matt Post, Anoop Kunchukuttan, Roman
REFERENCES
Roee Aharoni, Melvin Johnson, and Orhan Firat. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3874-3884 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. httLhttss://cclaaa
doi: 10.18653/v1/N19-113
wrg. wrg.llloy.oogg111990 1388
Duarte M Alves, José Pombal, Nuno Guerreiro, Pedro H Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, et al. Tower: An open multilingual large language model for translation-related tasks. arXiv preprint arXiv:2402. 17733, 2024.
Viraat Aryabumi, John Dang, Dwarak Talupur,, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, et al. Aya 23: Open weight leases to further multilingual progress. arXiv preprint 2rXi:2240..10032 2024.
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Danicle Calandricllo. A general theoretical paradigm to understand learn- ing from human preferences. In International Conference on Artificial Intelligence and Statistics, pp. 4447-4455. PMLR, 2024.
Loic Barrault, Ondiej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Gra- ham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz. Math- ias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. Findings of the 2019 conference on machine translation (WMT19).
Ondfej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, and Karin Verspoor (eds.), Proceedings of the Fourth Conference Ma- chine Translation (Volume 2: Shared Task Papers, Day 1), pp. 1-61, Florence, Italy, Au- gust 2019. Association for Computational Linguistics. doi: 10. 10.18853//1//19-1 ht.tps: a/cl antho 1 ogy. wwwoorgggg.. /W1 9-03
URL
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Yijie Chen, Yijin Liu, Fandong Meng. Yufeng Chen, Jinan Xu, and Jie Zhou. Improving translation faithfulness of large language models via augmenting instructions. arXiv preprint a308v:2308. 1 12674, 2023.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440-8448 8451, Online, July 2020. Association for Computational Linguistics. doi: 10. 1e..18653111222...c. 2at.:000..l-mai..17... 747. JRLpetpe. //acllantholooy..oo wrt.:yyrrrg/20000 acl-mallll. 747
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 2024.
11

---

Published as a conference paper at ICLR 2025
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Man- deep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. Be- yond english-centric multilingual machine translation. CoRR, abs/2010.11125, 2020. URL
htttps://arxiv. wttp:/iwwwrrg.r//oooo//ooo wttp://papaaaooooo//////// 11125
Christian Federmann, Tom Koemi, and Ying Xin. NTREX-128 news test references for MT evaluation of 128 languages. In Proceedings of the First Workshop on Scaling Up Multilin- gual Evaluation, pp. 21-24, Online, nov 2022. Association for Computational Linguisties. URL, httts::/acclathool www.ooooo..oooooooo..o..o.oo...o wwt.or//////... 2022. sumeval-1.4
Zhaopeng Feng, Ruizhe Chen, Yan Zhang, Zijie Meng, and Zuozhu Liu. Ladder: A model- agnostic framework boosting Ilm-based machine translation to the next level. arXiv preprint
DrXiv:2406.15741 2024.
Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Koemi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, and George Foster. Results of WMT23 metrics shared task: Metries might be guilty but references are not innocent. In Philipp Koehn, Barry Haddow, Tom Koemi, and Christof Monz (eds.), Proceedings of the Eighth Conference on Machine Translation, pp. 578-628, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023, wmtt://22222m311111m 1.5- URL
www.antoooooyoooooooooooooo 2023. wmt 11.517
Markus Freitag, Nitika Mathur, Daniel Deutsch, Chi-Kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Frederic Blain, Tom Koemi, Jiayi Wang, David Ifeoluwa Adelani, Marianna Buchicchio, Chrysoula Zerva, and Alon Lavie. Are LLMs breaking MT metrics? results of the WMT24 metrics shared task. In Barry Haddow, Tom Koemi, Philipp Koehn, and Christof Monz (eds.), Proceedings of the Ninth Conference Machine Translation, pp. 47-81, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024. wwm- I.2. URL httpt::://// /aap:://ccllee anthol
/202//2024442 wmt 1.2/
Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André FT Martins. xcomet: Transparent machine translation evaluation through fine-grained error detection. arXiv preprint urXiv:2310. 10482, 2023.
Kevin Heffernan, Onur Çelebi, and Holger Schwenk. Bitext mining using distilled sentence rep- resentations for low-resource languages. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 2101-2112 121 Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 18.18653/ http:///w22222222 1282::///022...innnnneeemmmm findings-emmllp.544
URL httph:tt httpsss//aeeaaatooooogoooooo org./2222..../222.....
Findings Saicinggs-emilp.111 wm..p.mmmppp.p.
Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W Bradley Knox, and Dorsa Sadigh. Contrastive prefence learning: Learning from human feedback without rl. arXiv preprint arXiv:2310.13639 2023.
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Mat- sushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. How good are gpt models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210 2023.
Hieu Hoang, Huda Khayrallah, and Marcin Junczys-Dowmunt On-the-fly fusion of large language models and machine translation. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguisties: NAACL 2024, pp. 520-532, Mex- ico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653//1/2024 Ainings-naacll.n URL ac anttolootoyoooo.oooooooooo wttp:///./....//iioneeeeeeeeee// www.fiiiininnsieee
35/
Jiwoo Hong. Noah Lee, and James Thome. Orpo: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07661, 2(4):5, 2024.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. International Conference on Learning Representations, 2021.
12

---

Published as a conference paper at ICLR 2025
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Melvin Johnson, Mike Schuster, Quoe V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenbrrg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google's multilingual neural machine translation system: Enabling zero-shot transla- tion. Transactions of the Association for Computational Linguistics, 5:339-351, 2017. doi: n0.1162/taclaa.c a 00065. htttsttts:////t
Huda Khayrallah and Philipp Koehn. On the impact of various types of noise neural machine translation. In Alexandra Birch, Andrew Finch, Thang Luong, Graham Neubig, and Yusuke Oda (eds.), Proceedings of the 2nd Workshop Neural Machine Translation and Generation, pp. 74-83, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653////118-2709 URL htt.tp/s//cclaattoooooooooo
2709/
Dayeon Ki and Marine Carpuat. Guiding large language models to post-edit machine translation with error annotations. arXiv preprint arXiv:2404.0785/, 2024.
Tom Koemi, Eleftherios Avramidis, Rachel Bawden, Ondicj Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Had- dow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, Toshiaki Nakazawa, Martin Popel, Maja Popović, and Mariya Shmatova. Findings of the 2023 conference on machine translation (WMT23):: LLMs are here but not quite there yet. In Philipp Koehn, Barry Haddow, Tom Koemi, and Christof Monz (eds.), Proceedings of the Eighth Conference on Machine Translation, pp. 1-42, Singapore, December 2023. Association for Com- putational Linguistics. URL https: /ttp:p::/acccaaa /wtp/aaaaaaaanaaaaaaaaaaaaaoalea org/ 2023. wnt 1.1
Minato Kondo, Takehito Utsuro, and Masaaki Nagata. Enhancing translation accuracy of large lan- guage models through continual pre-training on parallel data. arXiv preprint arXiv:2407.03144 1445 2024.
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orsikkh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytonë Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Pa- padimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangirā, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat. Bonaven- ture F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefegō Ahia, Sweta Agrawal, and Mofe- toluwa Adeyemi. Quality at a glance: An audit of web-crawled multilingual datasets. Transac- tions of the Association for Computational Linguistics, 10:50-72, 01 2022. ISSN 2307-387X. 00447
doi: 16.1162/ a 004474 URL
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. [GS] Jhard: Scaling giant models with condi- tional computation and automatic sharding. In International Conference Learning Represen- n
tations, 2021. httttttttt/ppeereeeeeee wwt.eeewwwweeeeneeeeeeeeee wtw.mmmddwdwee//e/mmmmm
Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Mirella Lapata, Phil Blunsom, and Alexander Koller (eds.), Proceedings of the 15th the Association for Computational Linguistics: Volume 2, Short Papers, pp. 8-14, Valencia, Spain, April 2017. Association for Computational Linguistics. URL httph:tp.: antho .og...rr... E. 7020
Conference of the European Chapter the
Yinquan Lu, Wenhao Zhu, Lei Li, Yu Qiao, and Fei Yuan. Llamax: Scaling linguistic horizons of I1m by enhancing translation capabilities beyond 100 languages. arXiv preprint urXiv:2407.05975, 2024.
13

---

Published as a conference paper at ICLR 2025
Jean Maillard, Cynthia Gao, Elahe Kalbassi, Kaushik Ram Sadagopan, Vedanuj Goswami, Philipp Koehn, Angela Fan, and Francisco Guzmán. Small data, big impact: Leveraging minimal data for effective machine translation. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguisties (Volume Long Papers), pp. 2740-2756, 2023.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734, 2024.
OpenAl. Gpt-4 technical report, 2023.
Pedro Javier Ortiz Su' arez, Benoit Sagot, and Laurent Romary. Asynchronous pipelines for process- ing huge corpora on medium to low resource infrastructures. Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. Cardiff, 22nd July 2019, pp. 16, Mannheim, 2019. reso ving. de /ur nbn : de bsz mh 90215.
Institut f'ur Deutsche Sprache. doi: 10.14618/ids 8/ds-- 9021.
URL
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Frithjof Petrick, Christian Herold, Pavel Petrushkov, Shahram Khadivi, and Hermann Ney. Document-level language models for machine translation. In Philipp Koehn, Barry Haddow, Tom Kocmi, and Christof Monz (eds.), Proceedings of the Eighth Conference on Machine Trans- lation, pp. 375-391, Singapore, December 2023. Association for Computational Linguistics. doi: 20118653/11/2222 wwt- 1.39. URL httph:tt///// pettpsss//aclanthollogooo www.tthoooo..ogg//22222
orgg:.org//// 2023 wmn 1.397
Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Con- ference on Machine Translation: Research Papers, pp. 186-191, Brussels, Belgium, Octo- ber 2018. Association for Computational Linguisties. doi: 10. 1.1163//1//18--3119 URL
https: htts:///aaaaathologgoo org. wtt.oo////111111111 6319
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(1), January 2020. ISSN 1532-4435.
Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Awadalla, and Arul Menezes. Leveraging GPT-4 for automatic translation post-editing. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 12009-12024, Singapore, December 2023, Association for Computational Linguistics. doi: 10. 10.18653/v1/2023, findings-emnlp.8064 URL
findings emnlp.
804
Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 578-585, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. org/2022.
Association for Computational Linguistics. URL seetps://aclantten
wmt-1.55
Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, Armand Joulin, and Angela Fan. CCMatrix: Mining billions high-quality parallel sentences on the web. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6490-6500, Online. August 2021. Association for Computational Linguistics. doi: 00.18653/v1/2021 wtt./2201..clloooo lon.long.507. URL https:
7/acl anchannhoollooooooooo ogy www.hhollooo...g////// aacl 1ong
14

---

Published as a conference paper at ICLR 2025
Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hin- ton, and Jeff Dean. Outrageously large neural networks: The sparsely-gate gated mixture-of- experts layer. In International Conference on Learning Representations, 2017. URL https::: //openreview.
d=B1 ckMDglg
Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F Karlsson, Abinaya Mahendiran, Wei- Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, al. Aya dataset: An open-access collection for multilingual instruction tuning. arXiv preprint wrXiv:2402.06611 2024.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint 00:::2403.00225 2024a.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju. Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint
18, 2024b.
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shan- non Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No language left be- hind: Scaling human-centered machine translation. 2022. URL pettps://erxiv.orrg// 2207 04672
Brian Thompson and Matt Post. Automatic machine translation evaluation in many languages via zero-shot paraphrasing. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 90-121, Online, November 2020a. Association for Compu- tational Linguistics. doi: 2..18653vv122222
mainlp-main..... URL httts:::/aalatthhl.lo.
org/ 2020.2020.. emn ma-mlin...n 87
Brian Thompson and Matt Post. Paraphrase generation as zero-shot multilingual translation: Dis- entangling semantic similarity from lexical and syntactic diversity. In Proceedings of the Fifth Conference on Machine Translation, pp. 561-570, Online, November 2020b. Association for Computational Linguisties. URL https::: / a aaatholotooogo org/2020.0000.. wmt 1. 67/
Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico. A shocking amount of the web is machine translated: Insights from multi-way parallelism. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Com- putational Linguistics: ACL 2024, pp. 1763-1775, Bangkok, Thailand, August 2024. Associ- htt22222..innnns---ii madiings-all.103.
ation for Computational Linguistics. doi: 10. 1021116633/1222444. wwwiiinnnnnsaiiccc 1033
URL https:
7/aclanthology.. orq//024 htwpchoooooy.rr//2224444
Jörg Tiedemann. Parallel data, tools and interfaces in OPUS. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Ugur Dogan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.). Proceedings of the Eighth International Conference on
Language Resources and Evaluation (LREC' pp. 2214-2218, Istanbul, Turkey, May 2012. European Language Resources A Association (ELRA). URL http:://ttwtt www.-reccooo cone-ccoof..o org/ proeeedingg//ree2222////////
Paper. pdf
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprini arXiv:2302.13971. 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint w0X2v22307.09288, 2023b.
15

---

Published as a conference paper at ICLR 2025
Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei- Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. Aya model: An in- struction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827, 2024.
Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In International Confer- ence on Learning Representations, 2021. URL htttps://openreesees net.eeewweeeeeeeeeeeeeeeeeeeeeee formm/f
FIvE WK 1H
Haoran Xu, Weiting Tan, Shuyue Li, Yunmo Chen, Benjamin Van Durme, Philipp Koehn, and Kenton Murray. Condensing multilingual knowledge with lightweight language-specific mod- ules. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 1575-1587, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10. 1/2023..022 http:////..m....mmmiiiii emnlp-main.97. URL 2tt.:y..orr22222.mllmmmm
97
Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift machine translation: Boosting translation performance of large language models. In The Twelfth Intera- tional Conference on Learning Representations, 2024a. URL hittps:/cpperreieww .neteveieewee///////
Forum?id=fart? F
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries of Ilm performance in machine translation. In Forty-first International Conference on Machine Learning, 2024b.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.1 1934, 2020.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer. In Kristina Toutanova, Anna Rumshisky, Luke Zeutleooee, Dilek Hakkani-Tur, Iz Beltagy. Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 483-498 Online, June 2021. Association for Computational org/
Linguistics. doi: 10.18653/w1/002 1/202 mmacl-main.41 URL 2021. nwa.laacallcco...... m..maiin4111..
Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. Bigtrans: Augmenting large lan- guage models with multilingual translation capability over 100 languages. arXiv preprint 1:Xi1:2305.18098 2023.
Jiali Zeng, Fandong Meng, Yongjing Yin, and Jie Zhou. Tim: Teaching large language models to translate with comparison. arXiv preprint arXiv:2307 230,00200, 2023.
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. Improving massively multilin- gual neural machine translation and zero-shot translation. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguisties, pp. 1628-1639, 1639, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl main. 148. URL httpss:/aclanthhloog httphhoo.oooooorr0220...... www.:m2....lmmm......44.....
Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma. Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, et al. Bayling: Bridging cross-lingual alignment and instruction following through interactive translation for large language models. arXiv preprint arXiv:2306. 10968. 2023.
Susan Zhang, Stephen Roller. Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li. Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint 2rX.v:2205.01065 2022.
16

---

Published as a conference paper at ICLR 2025
Dawei Zhu, Sony Trenous, Xiaoyu Shen, Dietrich Klakow, Bill Byme, and Eva Hasler. A preference-driven paradigm for enhanced translation with large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.). Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies (Volume I: Long Papers), pp. 3385-3403 Mexico City, Mexico, June 2024a. Asso- ciation for Computational Linguistics. doi: 10.1 16633////2224mmaccoooogggg
URL https:
//aclanthology.oo wwtpy//4.gg/00222020 naacl Long. 186
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. Multilingual machine translation with large language models: Empirical results and analysis. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.). Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2765-2781. I, Mexico City, Mexico, June 2024b. Association for Computational Linguistics. doi: 10. 8653/ www../ii.....ss---n.. naacl. URL waw.ggssnnaal111...
http://aalaanthoooy..oooo
Vilém Zouhar, Shuoyang Ding, Anna Currey, Tatyana Badeka, Jenyuan Wang, and Brian Thomp- son. Fine-tuned machine translation metrics struggle in unseen domains. In Lun-Wa Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics Volume 2: Short Papers), pp. 488-500, Bangkok, Thailand, Au- gust 2024. Association for Computational Linguisties. doi: 10. 1211186/3vvll w0t.////l/2222.../ a000224.c-ssoo...... URL https: / antho wwwphhhloo...oggg/o/o. www.hhoooog.oor//222....oo ac ssh.rt.ssshtttt... .57
17

---

Published as a conference paper at ICLR 2025
APPENDIX CONTENTS
Appendix Sections
Contents
Appendix A
Language Information
Appendix B
Illustration of Training Recipe
Appendix C
Reward Differences of MT and QA
Appendix D
Prompts
Appendix E
Full Results of All Directions
Appendix
Examples Over-Rejection
A LANGUAGE INFORMATION
We provide detailed information the eight language groups, including their scripts, language families, and resource levels, in 6 Each group includes English to ensure that each language- specific module supports English-centric translation and prevent catastrophic forgetting of En- glish. While we primarily grouped languages based on linguistic similarity, the grouping is not perfect. This is due to the need to balance the number of languages in each group and the inherent nature of language resources. For example, Group is mix of Asian and European languages, and although most languages in Group 4 are Southeast Asian languages, include French as an additional bonus language to facilitate cross-lingual transfer, especially since most languages in this group are low- and mid-resource.
Table 6: Detailed information of all langauges.
Language
ISO-639-1
Script Latin
Family
Subgroup Germanic
Resource High
English
en
Indo-European
Group 1: Germanic languages
Afrikaans af Latin
Indo-European
Germanic
Mid
Danish da
Latin
Indo-European
Germanic
Mid
Dutch nl
Latin
muro-eu
Germanic
High
German de
Latin
muro-Eur
Germanic
High
Icelandic
is
Latin
Indo-European
Germanic
Low
Norwegian no Latin
Indo-European
Germanic
Low
Swedish sv Latin
Indo-European
Germanic
High
Group 2: Romance Languages
Catalan
ca
Latin
Indo-European
Italic
High
Galician gl
Latin
Indo-European
Italic
Mid
Italian it
Latin
-uropen
Italic
High
Portuguese
pt
Latin
Indo-European
Italic
High
Romanian ro
Latin
Indo-European
Italic
Mid
Spanish
es
Latin
Indo-European
Italic
High
Group 3: Eastern and Southern Slavie Languages
Bulgarian
bg Cyrillic
mndo-European
Balto-Slavic
Mid
Macedonian
mk Cyrillic
-udo-Eu
Balto-Slavic
Low
Russian ru
Cyrillic
Indo-European
Balto-Slavic
High
Serbian sr
Cyrillic
Indo-European
Balto-Slavic
High
Ukrainian uk Cyrillic
Indo Europen
Balto-Slave Slavic
Mid
Group 4: Southeast Asian Languages
French
id Latin
fr
Indo-European
Italic
High
Indonesian
Latin
Austronesian
Malayo-Polynesian
Mid
Malagasy
mg Latin
Austronesian
Low
Malay
ms Latin
Austronesian
Malayo-Polyneste
Mid
Thai th
Thai
Tai-Kadai
Kam-Tai
Mid
Vietnamese vi Latin
Austronesian
Vietic
High
Continued on next page
18

---

zh Published as a conference paper at ICLR 2025
Language ISO-639-1
Script
Family
Subgroup
Resource
Group 5:
Central and Eastern European Languages
Czech
CS
Latin
Indo-European
Balto-Slavic
High
Greek
el
Greek
mndo-European
Traeco-Phrygiin
Mid
Hungarian
hu
Latin
Uralic
Finnic
High
Latvian lv
Latin
Indo-European
Balto-Slavic
Mid
Lithuanian
It
Latin
Indo-European
Balto-Slavic
Mid
Polish pl
Latin
Indo-European
Balto-Slavic
High
Group 6: Eurasian Language Mix
Chinese
Han
Sino-Tibetal
Sinitic
High
Estonian et
Latin
Uralic
Finnic
Mid
Finnish fi
Latin
Uralic
Finnic
High
Georgian ka
Georgian
Kartvelian
Georgian-Zan
Mid
Japanese ja
ko Japanese Hangul
Japonic
Japanesic
High
Korean
Koreanic
Korean
High
Group 7: Indo-Aryan Languages
Gujarati gu
Gujarati
Indo-Eun Europen
Indo-Aryan
Low
Hindi hi
Devanagari
Indo-Aryan
High
Marathi mr Devanagari
Indo-European
Indo-Aryan
Low
Nepali ne Devanagari
Indo-European
Indo-Aryan
Low
Urdu
Arabic
Indo-Aryan
Mid
Group 8: Turkic and Semitic Languages
Arabic
ar
Arabic
Afro-Asiatic
Semitic
High
Azerbaijani
az
Arabic/Latin
Turkic
Common Turkic
Low
Hebrew
he
Hebrew
Afro-Asiatio
Semitic
Mid
Kazakh
kk
Cyrillic
Turkic
Common Turkic
Mid
Kyrgyz
ky
Cyrillic
Turkic
Common Turkic
Low
Persian fa
Arabic
Indo-Eure aado-European
Iranian
High
Turkish tr
Latin
Turkic
Common Turkic
High
Uzbek
Latin
Turkic
Common Turkic
Low
B ILLUSTRATION OF TRAINING RECIPE
Here, we illustrate an overview of our 5-step training recipe in Figure
2
Pat- Shape
Per Tree Pige Calmuddon
Gateu Cannoloooee Moscaged
Mosloged of Lorgugee
Speniee Hor Gialy
Coller
BAŠE MODEL
BAŠE MODEL
BAŚE MODEL
This diagram of the multi-stage process of fine-tuning multilingual model. In Pre- Training Stage the base model is fine-tuned using 20B tokens of monolingual data from 50 languages. The process continues with Pre- Pre-Training Stage 2, where language-specifs modules are fine-tuned with 10B monolingual tokens. Pre-Truining Stage 3 introduces pseudo-monolingur, fine-tuning, using randomly concatenated parallel sentences to improve multilingual alignment. The model then undergoes Tost-Training Stage I, where SFT is performed on high-quality parallel data, followed by Post- -astnTrannng Stage 2, which applies Adaptive Contrastive Preference Optimization to address over-rejection issues in translation preference learning.
Figure 4:
19

---

Published as a conference paper at ICLR 2025
C REWARD DIFFERENCES OF MT AND QA
Here, we present a comparison of the reward difference between machine translation (MT) tasks and open-ended question answering (QA) tasks in preference learning. Figure 5 illustrates the mulative distribution of reward differences for the MT preference dataset, as described in Section [5] alongside the multilingual preference data from the Aya open-ended QA dataset (Singh et al.) 2024) for languages in Group 6. The reward differences are sorted in ascending order, and their cumulative probabilities are displayed. The reward difference is computed using the CPO loss func- tion: log Ha (V=l log (gn(z)) The construction of the Aya preference dataset follows the same methodology as the MT preference data, where we fine-tune the Aya QA dataset via SFT and use the tune model to generate answers for the training data. System-generatde treated as dis-preferred, while original references are considered preferred. As shown Figure [ the open-ended QA task exhibits significantly larger reward differences compared to machine trans- lation. For instance, the maximum reward difference for the smallest 80% of MT preference data is 20, whereas it is approximately 300 for Aya QA. Similarly, the maximum reward difference for the MT preference data is 131, while that for Aya QA nearly tenfold larger.
responses are
Cumulative Probability
0.8
0.6
0.4
0.2
Machine Translation
Ayo QA
0.0
0
200
400
600
800
1000
1200
1400
Difference of Rewards
Figure 5: Cumulative distribution of reward differences between machine translation and open- ended question answering tasks in contrastive preference optimization.
D PROMPTS
In Figure 6
present the prompt used for GPT-4o post-editing during the construction of the preference dataset, as well as the prompt used for X-ALMA in generating translations.
FULL RESULTS
We report translation quality using XCOMET-XL, recent WMT metric shared tasks (Freitag al. 2023 2024 have found high correlation between trained metrics like XCOMET-XL and human preferences. However, those findings are limited to a few languages, and correlation with human judgments has also been shown to degrade for trained metries in out of domain (relative to WMT, i.e. FLORES) settings (Zouhar et al.) 2024). For these reasons we also report the more traditional BLEU metric.
Tables to present the results for each translation direction across language groups the Flores- 200 dataset, while Table 15 shows the full results for the WMT'23 dataset. On the Flores-200 dataset, X-ALMA surpasses all other sourse multilingual models in every translation direction according to COMET-22, and in 97 out of 98 directions according to XCOMET-XL Additionally, ARPO, when compared to SFT, demonstrates superior performance in all translation directions re- ported by COMET-22 and in 95 out of 98 directions according to XCOMET-XI..
20

---

Published as a conference paper at ICLR 2025
GPT-4o Post-Edit Prompt
System:
You are native speaker of both <source language> and <target language>, You are expert post editor of manslations from <source language> into <target language> and a helpful assistant dedicated to improving translation quality. You will be provided with source sentence in <source language> and its translation in <target language>. Your task carefully analyze provided translatio,, and suggest improvements to the translation. Note that you only need to generate refined translation in <turget sentence and do not generate anything else.
sentence and
User:
The source sentence in <source language> is: <source sentence? The translation <target lunguage? is: <X-ALMA trunslated sentence Note that you only need to generate a refined translation in <target language> and do generate anything else.
X-ALMA Translation Chat Template
<>> INST] Translate this from <source language> to <target language>:
<ource language>: <source sentence>
<target language>: 

Figure
Prompts used for GPT-4o post editing and X-ALMA translation generation.
Table 7: Full results for Group I in the Flores test data.
Modela
2at
Co rdo
702
712
BLEE
COMET-12
XCOMET
BLEEU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
T.I.MA--T-B Ioar-
14.4
14.8
14.5
17.3
36.h
71.7
50.2
77.3
68.5
11.9
69.4
STA
NLLB 3.38
38,9
37.4
74.8
90.0
76.8
$9.1
75.2
24.5
$1.6
74.1
LLaMAXI Alpoca 7B
38.1
35.1
73.2
$9.5
76.3
31.2
36.6
75.0
20.4
92.9
72.1
LLLMAXI-Aloon KBI
38.5
35.0
72.3
36.2
$5.5
13.6
35.4
72.4
$1.2
08.5
Ape-
21.5
78.8
$1.6
62.9
$4.1
67.5
86.3
14.5
(enyy 3PTy
60--9LL
Arg. WO--H0
XCOMET
COMET-22 XCOMET
XCOMET
XCOMET
68.5
NLLB-3.38
90.7
88.2
LLJMAX2 7B
87.1
75.5
LLAMAX Alpoca 8B
89.1
86.4
73.4
Asse 101
22.1
358
72.2
76.9
87.5
69.0
31.5
86.0
61.2
36.7
85.0
64.0
Asa-23-BB
26,0
$7.9
78.8
15.7
77.3
60.0
20.8
78.3
90.7
19.7
75.1
58.5
Amn-23-3B
26.5
$7.7
78.2
22.1
92.4
67.6
25.8
$3.7
67.9
25.2
79.6
64.7
KALLAAAA 3PT)
M.5
354
351
956
30
3710"
901
"83"
"00""
X-AMMA (Ouzs)
29.5
39,0
90.4
34.2
90.8
$1.5
47.2
91.8
$1.0
38.7
8966
79.3
02. 760
do nen
720
BLFE
COME:1-12
XC UMFF
HI.RU
COMP1-22
XOOMET
BLFU
COME/132
XOOMET
BL
COMBET-22
XCUMET
T.I.MAXX I-80--
14.5
66.2
$1.9
21.0
66.5
12.5
$6.11
78.6
71.
1.2
43.4
42.0
NLLB--311
40.5
40.3
63.7
34.4
83.0
66.0
25.6
R1.3
64.5
16.3
64.2
42.0
LLLMANSAAluss/
53.5
38.9
76.1
46.0
80.7
79.8
41.4
$8.0
75.5
31.2
84.8
75.0
LLaMAXI Alfoca S11
53.1
39.0
76,0
453
8995
19.6
40.5
53.5
75.6
32.5
85.6
75.3
Apo-
43.2
36.1
55.4
89.2
39.7
18.5
71.9
21.3
82.3
68.4
Apo-
46.9
76.8
80.1
680
46.5
App-29-355
(only
X-ALMA (Ohes)
nO-tWn
Args X30-4
XCOMET
COMET-22
XCOMET
COMET-22 XCOMET
N.LBB3336
62.3
79.1
LLJMAX2-AA 7B
85.5
88.2
788
LLIMAXI Aljuca 8B
85.5
88.3
78.1
87.3
74.4
86.5
$7.9
$6.6
Ava-29-358
33.9
37.5
78.8
45.2
$5.5
76.8
46.9
80.5
70.6
43.2
$7.4
76.6
X-ALMA (onty
M42
891
78.1
XALMA (Ouzs)
33.9
S8.1
79.3
45.7
89.5
79.
98.5
91.6
98.6
45.9
89.4
76,9
21

---

Published as a conference paper at ICLR 2025
Table 8: Full results for Group 2 in the Flores test data.
Modcs
en--cc
50-105
en-tgl
ILUU
COMET 22
XCUMET
BLEU
COMBT 23
XCOMET
B1.0
COMET 22
XCOMET
BLUU
CUMET'22
XCOMET
Liata-X.1
37.5
86.3
17.5
25.0
$3.9
17.2
3012
14.3
14.0
23.3
81.7
72.
NLLBB-333B
43.1
87.8
77.6
23.5
56.5
80.0
35.7
87.3
76.8
31.3
68.5
$0.5
LLaMAX2 Alpaca-BB
37.2
87.2
73.1
23.1
55.5
79.0
31.5
86.5
77.2
25.0
$7.0
78.5
76.6
06-9
77.2
Asp- 101
37.8
24.2
78.0
87.0
78.5
Apo-23-3B
27.8
77.3
53.4
Aow-23-358
27.7
53.2
(oaly
(Ouis)
Avg. #TTHRSS
11.7
75.4
NIL18-3.38
80.4
ELaMAX2-Alpaa--P
51.8
32.3
79.6
ELaMANN3AApaaa-BB
41.5
32.0
78.0
31.3
76.6
23:40
48.4
37.9
31.1
90.3
350
48.6
38.4
ALMI A
19.9
42.2
82.9
X-ALMA (Oon)
50.2
43.5
90.8
40.0
89.4
83.0
BLEU
COMET 22
XCOMET
BLEU
COMET 22
XCOMET
BLBU
COMET 22
XCOMUT
BLEU
CUMET-22
XCOMET
Llama-511
35.7
76.1
Y2.7
25.5
16.2
11.4
9.2
15.
63.3
24.1
71.5
CY.3
NLLBB-3. 3.33
37.9
83.7
70.5
27.1
53.3
76.3
34.7
84.0
71.7
28.5
54.4
73.1
LLaMAXX Alpca-l 7B
43.6
88.4
78.4
29.3
56.5
78.4
36.7
85.0
78.2
31.9
$7.6
78.9
LLaMAXS Alpca Alpac 8D
42.9
88.3
78.1
29.0
56.7
7%.1
38.6
35.0
73.0
31.3
$7.5
78.5
Apa 101
41.1
87.6
75.5
23.8
96.5
75.0
35.5
35.3
72.7
31.2
57.4
78.1
Apo- 23-413
19.5
85.8
75.9
50.5
67.4
79.6
37.3
87.0
16.5
34.1
13
79.0
Ava- 23-358
46.3
88.4
77.8
33.1
$7.7
78.5
41.7
83.5
78.0
36.0
63.3
78.5
X. ALMA (nly SFT)
48.6
89.2
77.8
34,9
57.7
77.8
44.9
89.0
77.9
36,9
53.3
78.5
X. ALMA (Ouns)
48.7
89.6
78.7
33.0
$7.9
79.0
44.6
89.2
78.4
33.5
$8.6
79.3
pt pt--we
Ang. 30X-HRM
XCDMET
XCOMET
RI
COMFT-32
XCDMRT
Limna 3.1
43.5
31.3
14.0
20.1
54.5
01.0
27.1
703
13.3
NELLB 3.38
42.3
85.7
75.1
31.4
53.0
35.7
73.5
E.I.AA2-AApua A8
LLaMAXS-AApaaa-BB
38.1
78,0
79.8
(Oum)
Table 9: Full results for Group 3 in the Flores test data.
Models
en-ePg
en-imi
WET-RST
BLEU
COME:T-22
XCDMET
BI.F)
COMET-22
XCOMET
HL.BU
COMET-12
XCDMET
BLEU
COMPT-22 XOOMET
Liama 3.1
20.6
87.33
14.2
23.3
53.7
14.3
14.44
03.0
43.1
1.4
15.3
75.4
NLLLB 3.38
40.5
77.5
34.4
53.5
17.4
32.2
89-2
77.5
33.8
$9.0
37.1
LLaMAN2 Alpaca 70
33.0
77.1
29.5
$7.9
77.5
25.4
87.9
16.3
8.1
79.1
37.4
LLLMAAFFAlpcaa 4B
12.2
76.8
29.3
51.4
71.0
26-4
87.7
16.4
76.2
73.8
77.5
79.4
Ang: MIH-ARX
XCOMET
Limae31
NLLB-3.18
54.2
ELaMAX2-Alpas
LLaMAN3 8B
23.8
28.1
29.4
13.9
23-150
30.3
17.8
65.
X-ALMA (AMAAAA SPT)
31.8
16.0
804
X-A (Omm)
32.0
98.9
78.9
16.2
90.9
80.7
BLEU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
BLBU
COMET-32
XCOMET
BLEU
COMET-22
XCOMET
Liama 3.1
113
33.7
15.7
1.3
39.7
43.1
14.4
61.11
63.2
6.11
16.6
48.3
NELB 3.3g
37.6
36.0
74.1
57.1
$4.3
71.6
30.7
84.2
73.1
35.8
83.4
71.1
LLaMAX2 ALaMAA22Appcca71 Alpaca770 70
35.1
87.6
77.9
39.5
$7.2
77.2
35.1
96.2
76.7
40.5
81.2
75.38
LLaMAX3-Alpcs
35.2
87.5
77.6
39.8
67.2
772
35.1
96.4
76.8
40.6
87.3
76.6
Appu
32.0
85.4
70.7
53.7
64.3
901
32.7
86.
76.7
35.0
83.0
72.6
A6--21-48
12.6
84.4
71.7
25.0
78.4
63
36
86
76.8
23.0
70.9
66.0
A50-23-158
18.3
85.7
75.5
36.2
64.5
72.4
38.6
87.1
76.7
33.8
85.3
75.0
ALMA only SIT)
43.4
85.4
77,9
45.6
53.2
77.1
38.7
37.0
76.6
46,3
88.4
78.8
X-MMM (Ours)
42.9
8KA
78,5
45.6
$8.4
77.6
36.7
87.2
77.4
64.7
85.4
79.4
LK-188
Arg. 300-
BLEU
L.ama-3.1
37.3
51,2
48.
NE.18-3338
35.2
35.0
72.0
16.8
37.5
LLaMANS Alpaca
37.7
(only
22

---

Published as a conference paper at ICLR 2025
Table 10: Full results for Group 4 in the Flores test data.
Modcs
en-+fr
en-11d
en-tig
en-195
ILUU
COMET 22
XCUMET
BLEE
COMET
XCOMET
B1.0U
COMETT 22
XCOMUT
BLUU
CUMPTT22
XCOMET
Liata-X.1
23.5
85.3
13.7
12.1
75.5
64.5
1.5
47.33
11.2
34.0
66.3
753
NLLBB-3388
51.1
88.3
76.9
46.4
91.2
17.9
17.7
91.6
59.9
41.6
89.1
76.8
LLaMAX2 Alpaca-BB
42.0
86.8
75.7
18.0
99.7
77.2
4.4
66.9
23.3
33.0
19.3
76.7
2.4
34.4
K7.4
73.2
Asp- 101
60.8
Apo-23-3B
31.0
4.4
87.3
79.7
Aow-23-358
57.2
77.4
(oaly
99.7
(Ouis)
Avg. #iTHRRS
25.5
NILL8-3.38
ELaMAX2-Alpaa-TP
54.9
67.9
ELaMAAX3AApaaa-BB
25.0
31.9
70.4
25.9
35B
60.4
27.7
61.3
ALMI ( STT)
11.6
63.9
36.1
75.8
X-ALMA (Oon)
12.0
64.1
79.3
35.6
886
76.5
BLEU
COMET22 22
XCOMET
BLEU
COMET 22
XCOMET
BLIU
COMET 22
XCOMET
BLEU
CUMET-22 22
XCOMET
Llame-511
21.6
YL.
21.9
70.5
152
1.3
41.8
13.1
10.3
63.4
35.7
NLLBB-3. 3.33
38.1
86.6
72.7
54.3
$4.5
68.5
13.5
63.3
43.5
31.4
52.1
65.3
LLaMAXX Alpca-l 7B
42.1
88.8
77.3
40.4
$8.9
78.2
15.4
71.8
56.7
40.2
59.3
77.1
LLaMAXS Alpcca
Alpcc 8D
41.6
85.7
76.5
40.8
59.0
78.2
19.6
76.0
60.6
41.3
59.5
77.0
Apa 101
41.2
88.6
77.0
35.8
$3.4
75.3
21.7
79.8
61.5
39.0
57.5
73.8
Apo- 23-413
85.3
89.8
77.4
64.1
69.5
78.5
1.5
47.0
18.8
80.0
81-3
75.9
Ava- 23-358
47.00
89.5
77.0
45.7
59.5
78.4
533
$4.1
33.0
43.9
63.7
77.0
X. ALMA (nly SFT)
47.8
89.6
77.3
47.3
59.5
78.2
30L1
81.9
63.3
46.9
89.1
77.0
X. ALMA (Ouns)
46.0
89.6
77,8
45.8
90.1
78.
29.2
$1.9
63.4
45.4
$9.5
77.8
Ang. 30X-HRM
XCOMET
XCOMET
Limna 3.1
10.7
65.2
10:3
22.0
71.2
3910
17.
03.6
NELLB 3.38
25.3
71.8
31.5
10.4
29.3
81.1
E. 78
72.6
ALaMAA3-AApaaa-BB
72.6
37.2
(Oum)
Table 11: Full results for Group 5 in the Flores test data.
Mudels
en-ie1
COME-T-22
XCDMET
BLEU
COMET-22
XOOMET
HC.BU
COMET-12
XCDMET
BLEU
COMPT-22 XCOMET
Liama 3.1
27.1
B8.0
13.5
19.1
52.3
10.4
19.3
12.3
7014
13.0
77.2
GUL
NLLLB 3.38
32.1
91.0
77.7
27.4
59.0
76.6
26.4
89.3
78.7
25.2
89.1
77.3
LLaMAN2 Alpaca 70
26.0
89.1
75.3
19.9
56.4
14.5
19.1
87.0
74.9
15.0
57.0
31.0
LLaMAA--AApoca 4B
25.6
88.1
73.4
20.4
66.2
16.2
18.2
35.6
73.7
17.0
86.1
12.8
78.6
27.0
57.2
$2.5
27.0
M0--911
Aug.. MIT--O0K
XCOMET
COMET-32 XCOMET
17.3
62..2
NLLB-3.18
21.6
76.1
ELaMAX2-
18.3
20.8
LLaMAN3
17.2
19.8
72.5
18.3
22.6
76.4
20.7
77.2
14.6
70.6
51.2
23-150
7.9
22.4
19.1
66.8
X-ALMA (AMAAAAA SPT)
29.3
78.5
23,3
78.9
76.1
90.6
80.2
X-A (Omm)
30.8
91.1
79.3
23.2
90.4
79.3
28.8
91.0
80.2
BLEU
COMET-22
XCOMET
BLEU
COMET-2 22
XCOMET
BLBU
COMET 22
XCOMET
BLEU
COMET-22
XCOMET
Liama 3.1
22.1
61.3)
64.5
10.3
$7.4
4330
8.5
31.3
$3.99
3.3
19.2
41.3
NELB 3.3g
29.4
80.1
59.9
33.0
96.1
75.3
14.0
70.1
43.7
12.6
67.1
35.9
LLaMAX2 Alpaca770
37.6
87.9
77.8
24.5
77.2
71.5
32.2
87.5
78.2
29.7
85.1
75.6
LLaMAX: A-paca--
37.5
88.1
77.8
$4.2
67.5
77.5
32.5
87.8
73.6
31.0
86.0
76.4
Appu
35.6
87.6
76.4
53.1
66.5
75.3
70.9
85
74.65
30.2
85.8
76.2
A6--21-48
40.7
88.5
75.
36.1
67.8
77.8
25.0
81
67.2
24.6
80.6
65.6
A60-23-158
42.1
88.5
77.9
30.0
88.
78.0
92.9
86.5
76.6
33.9
K5.A
73.5
ALMA only SIT)
45.3
89.0
75.4
33.0
$7.9
77.0
37.3
35.7
78.9
33,9
$7.1
75.1
X-MMM (Our))
42.6
75,8
37.4
$8.3
78.3
37.6
89.1
79.7
36,7
$7.4
75.7
LV-98E
12 11-
Arg. JUX
BLEU
XCOMET
XCOMET
L.ama-3.1
2,5
43.2
12.8
51.1
49.6
9.9
34.
45.2
NE.18-3338
10.4
20.3
20.0
$2.4
11.6
23.0
74.6
LLaMANS Alpaca
(only
23

---

Published as a conference paper at ICLR 2025
Table 12: Full results for Group 6 in the Flores test data.
Midels
BLEU
C0-75C COMETT22
XCUMET
BLEU
COMBIT 21
50-124
XCOMET
BLBU
Ch-734 COMET 22
XCOMET
BLEU
COMET-22
XOOMET
9.3
65.3
13.1
15.0
$2.0
10.4
10.3
66.1)
25.0
7.0
75.1
48.1
NLLBB-333B
25.0
90.5
79.7
24.1
91.7
$1.1
22.6
87.9
25.1
14.8
54.6
70.2
LLaMAX2 Alpaca-BB
1900
88.4
76.1
18.4
90.2
79.6
28.1
83.9
78.1
10.7
53.0
68.1
17.5
77.
77.3
9.6
78.6
Asp-
18.9
27.3
77.4
72.1
Apo-23-3B
30.7
Aow-23-358
1003
47.6
(oaly
57.6
75.7
(Ours)
36,4
Arg. WiTHAREE
N.LLB-3.38
LLaMAAN2 Alpaca-7B
35.2
20.3
LLLMANNS Alpaca
72.2
Ays-23-38
14.7
23.32
12.8
57.3
16.2
56.0
X-ALMA
15.0
26.5
90.3
X-LMA (Ours)
1538
44.9
78.4
27.5
81.3
XCOMET
COMET-22 122
XCOMET
BL.BU
COMET-32
XCOMET
COMFT-22
XCOMET
12.3
17.7
07.55
57.2
NLLBB-3.1
10.2
17.2
61.0
25.6
54.5
70.3
LLaMAX2 Alpaca-7
31.7
23.4
76.4
76.7
66.8
LLaMAN3 Alpaca 8D
31.6
79.4
24.6
76.9
50.7
51.5
32.5
29.7
25.3
75.5
25.5
$4.5
69.7
23-31
15.4
20.4
26.1
16.8
3.6
60.1
32.9
App- 230-23-150
28.9
74.5
87.3
30.4
88.4
19.4
79.4
65.0
X-ALMA MMA SPT)
18.2
89.3
K7.5
16.0
90.0
79
76.9
58
73.0
28.4
96.8
31.6
XALMA (Ouns)
38.8
89.6
K3.5
36.2
90.5
90.4
28,8
88.5
77.7
29.3
$7.1
72
Aug.
BLEU
COMET-32
XCOMET
BLEU
COMET-22
XCOMET
BLBU
0OMET-32
XCOMET
Llama-5.11
133
65.1
10.4
24.2
78.5
6013
11.3
15.3
$5.4
NLLLB-3.38
26.2
84.9
73.1
16.8
77.1
12.8
17.2
76.1
54.4
LLaMAXX A.22-Al 7B
25.2
86.6
76.4
25.8
$6.5
77.2
26.1
85.5
76.1
LLaMAXX Alpaca 8D
26.3
87.5
77.1
25.9
56.5
77.3
23.9
81.7
74,1
Aga 101
26.5
87.0
76.9
23.1
$4.5
71.
26.8
95.5
75.3
Apo- 23-40
29.4
880
77.3
29.4
67.1
774
21.0
79.9
64.4
Apo- 23--23-35
32.2
88.7
77.8
$3.2
67.6
77.4
28.8
85.0
74.7
X-AI MA (unly SPT)
10.6
88.1
77.0
36.4
67.1
96.8
92.1
85.2
73.4
X-AI MA (Dum)
30.8
88.7
78.1
20.5
$7.6
78.1
32.2
88.7
78.4
Table 13: Full results for Group 7 in the Flores test data.
Models
RLFU
COMET-33
XCDMET
BIFUU
COMFT-22
XCOMET
BL.BI
COMPT-12
XCEMET
BLEU
COMFI-22
XCOMET
115
81.7
55.7
21.7
59.5
13.3
7.5
62.4
27.8
4.3
57.6
47.5
NLLBB-338
24.3
87.2
66.2
34.4
50.9
67.7
17.1
74.3
65.6
16.4
76.5
76.8
FLLMAX2-Alpxa-70
11.0
78.9
40.5
24.8
76.9
62.0
11.1
70.7
62.9
13.8
80.5
$6.3
LLAMAX3 LLLMAXS-Alpaaa-B 4B
13.7
82.7
57.3
23.5
76.6
619
10.1
69.3
63.1
10.7
78.4
$3.03
Apa J01
15.6
83.9
00.5
21.4
75.5
57.7
10.3
69.3
60.2
10.5
77.5
79.1
App
0.4
65.7
64.2
25.0
66.9
0.9
68.7
64.5
1.5
09-2
65.4
App- 23-350
155
51.1
25,0
6
1:3
61.1
56.3
69.3
66.0
X-ALMA (mly SPT)
25.0
34,3
67.6
18,0
75.9
21.5
$4.0
$9.5
(Oum)
Ang: RITHEE
12:1
Alpcaa
15.5
LLaMANS-Apacce.o
App-21-315
350
6.5
ALMA
24.5
X XLMA (Ours)
24.0
24.5
BLEU
COMET-22
XCOMET
BLEU
COMET:22
XCOMET
BLBU
COMET-32
XCOMET
BLEU
COMET-22
XCOMET
Liama-31
19.2
9.2
$3.7
322
45.1
31.1
45.5
45.4
NILB--3.9
42.3
90.2
73.5
58.7
68.9
34.0
87.0
70.1
33.0
80.7
90.9
ELLMAX2-Alpa Alpaca-7B
11.0
42.7
47.9
27.3
$1.7
63.4
23.5
79.9
66.0
26.0
53.4
$6.6
LLMMAXS Alpaca 4B
9.9
66.0
60.7
35.4
58.9
67.9
30.6
87.3
70.1
32.9
89.3
90L3
Apa 101
25.0
82.3
63.0
34,5
$7.5
06.6
30.1
35.2
68.9
31.2
$4.9
$5.1
Apo 23
3.4
33.6
38.5
37.5
89.1
67.9
7.5
68.9
66.2
10,0
77.0
68.1
Apo- 23-358
9.3
63.1
53.0
40.1
69.5
68.2
184
79.9
9
23.5
64.1
$1.2
X-AI MM only SFT)
40.4
90.1
72.5
43.0
69.8
67.7
37.7
88.5
70-4
41.2
90.6
901
X-AI MA (Dum)
40.5
9I.3
72.7
42.7
90.1
68.4
37.7
8R.6
70.8
41.4
90.7
91.1
Ang. 302-4W11
BLEU
COMET-22 XCOMET
BLEU
COMET-22
XCOMET
Limna 3.1
4.9
35.4
4.1
47.5
17.55
NLLB 3.38
31.6
86.0
72.5
36,9
53.3
15
LLaMAN2 Aapcaa2
25.0
$1.1
69.7
20.4
73.8
06.7
LLaMAAK-AAppca-Bl
30.5
85.5
73.1
27.8
63.5
72.4
Ago- Ap2-23-3R
50.4
13.5
Apo-23-158
23.3
(unly SFT)
36.4
(Curs
24

---

Published as a conference paper at ICLR 2025
Table 14: Full results for Group 8 in the Flores data.
Models
BLEU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
BLBB
COMET-32
XCOMET
BLEU
COMET-22
XCOMET
Liama-31
18.9
82.9
20,0
5.6
09.5
50.7
21.4
86.
77.3
20.5
65.1
73.5
NIT1--3.5 A8
27.5
86.3
75.2
14.0
66.9
76.6
32.6
86.5
77.5
90.4
67.8
76.1
LLaMAXX Alpaca 7B
19.7
88.9
73.7
9.4
$2.0
68.2
17.8
83.7
72.4
22.0
85.1
72.7
LLaMAXS Apaca 8D
14.1
82.2
89.1
7.3
90.0
05.5
17.7
84.5
74.0
23.3
96.2
74.7
Apa 101
17.2
88.1
72.8
11.5
$3.5
75.4
19.1
96.4
77.6
20.5
55.4
75.2
Apor 23-40
26.5
87.3
77.5
2.0
75.5
09
25.2
87.7
19.4
27.0
13.1
72.7
23--21-158
27.4
87.1
76.5
3.0
67.2
54.8
25.8
87.6
70
28.0
83.1
72.0
X-AI MA (nnly SPI)
29.1
878
77.5
14.00
R8.2
79.0
28.4
88.5
80.2
30.7
80.6
79.6
X-AI MA (Oum)
76.3
HR.2
78.3
14.0
R8.4
79.3
27.1
86.8
80.8
33,6
89.8
79.5
M0-+Ey
BLEU
COMET
XCOMET
BLEU
COMET
XCOMET
BLBU
COMET 22
XCOMET
BLEU
COMET
XCOMET
Limna 3.1
8.2
77.7
61.8
3.1
53.5
19.4
19:1
31.2
05.3
9.3
51.9
05.1
N.30--333
20.6
73.9
13.2
14.7
29.0
16.1
18.6
59.3
75.6
13.0
86.5
14.7
8.2
54.0
11.9
16:2
85.
10.1
85.1
69.3
LLLMAA -MaMaA--Appaa-3
12.7
7.9
72.5
138
67.5
6.8
74.5
Apo-
75.0
13.6
75.6
Apo-23-38
1.2
26.6
Apo-23-358
74.5
37.1
(unly
27.7
78.3
77.0
(Ouss
XCOMET
LLaMAX2-Alpaa-78
ELLMAX3-Alpaa--BB
12.9
23-3B
13.2
09.0
Apm-23-15B
13.6
$4.4
X-ALMA (mnly SPT)
25.0
89.2
78.9
X-AMMA (Oun)
22.7
89.4
79.3
BLEU
COMET 22
XCUMET
BLEU
COMETT 22
XCOMET
BLBU
COMET 22
XCOMET
BLEU
COMET 22
XCOMET
Llmaa5.1
17.1
50.4
41.2
3.5
46.5
41.3
25.7
723
$6.56
3.7
43.6
25.5
NLLB-333
36.2
85.1
74.3
15.1
77.5
61.7
29.3
83.5
09.0
39.1
95.0
73.2
LLaMAX2-Alpaca-11
36.3
88.8
74.3
18.5
64.5
73.2
2910
84.1
73.7
36.8
85.8
76.3
11aMAX A1aMAX-AAppes-35
35.1
85.8
75.6
7.9
70.0
44.2
35.1
87.6
76.2
30.5
83.2
75.6
Asp- 101
35.0
85.8
74.7
21.5
K5-2
54.2
92.8
87.3
76.1
33.9
96-4
73.3
Aso-23-48
41.5
87.9
76.4
10.5
75.6
57.2
36.8
87.0
76.3
43.2
63.4
76.5
Aso-23-338
43.4
87.6
75.8
17.9
$2.6
69.6
39.6
83.5
76.3
45.6
13.9
76.7
X-ALMA (naly SFT)
41.2
87.5
75.4
25.8
56.7
74.6
37.6
85.1
75.5
44.5
83.3
75.8
X-ALMA (Oun)
40.9
8KD
76.5
25.5
$7.0
75.6
38.0
75.6
44.2
18.9
76.88
T.l 900
ky 1Gn
UR FFO
BLBU
COMETT-22
XCOMET
BLEU
COMET:22
XOOMET
BLBB
COMETT22
XCOMET
BLEU
COMET-22
XOOMET
T.ama-311
16.
$4.0
75.4
72.0
76.9
73.5
77.7
71.5
72.8
BEBU
COMET
XCOMBT
NLLBB-3.3
ELaMAX2
LLaMAN3 8B
Ays- 101
19.7
85.0
74.2
Aya-23-4B
22.4
76.7
90.5
A3n-23-35B
25.4
82.7
69.5
X-AMAA (mnly SFT)
34.8
87.5
75.1
XALMA (Ous)
35.2
880
76.0
25

---

Published as a conference paper at ICLR 2025
Table 15: Full results for all languages in the WM test data.
Models
CA-14C
50-12h
BLEU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
BLBU
COMET-22
XCOMET
BLEU
COMET-22
XCOMET
ALMA-R-13B
31.4
$4.0
66.8
32.3
$5.0
11.3
22.8
$3.5
74.4
Joverdistnutt
vo.2
32.9
85.1
41.9
$5.6
70-2
25.2
$3.3
71,1
33.3
79.7
11.0
'4.8
1.38
3[6
E5.
251
KI3
LLaMAN2 70ppaa
186
59.5
39.8
$1.6
64.6
15.4
10.7
21.1
$1.5
67.81
20.5
55.2
$4.0
19.
11.
81.8
66.7
23.5
81.5
67.6
Agp-
25.4
32.0
14.1
72.0
23.1
69.9
Apo-23-48
19.3
718
Apo-23-158
71.7
(unly SFT)
(Cuns
Avg. WIT-
COMET-22
XCOMET
XCOMET
COMET-32
XCOMET
ALMA-R-136
TowerBistoust78
35.5
3..4
360
8188
64.6
23.3
EL AMMA33-Appaa--BB
19.8
24.4
22.4
21.0
23-3B
26.3
26.5
64.3
28.0
70.5
Apm-23-158
34.0
70.5
79.3
84.1
70.3
X-AL (unly SPT)
27.4
85.3
71.6
31.4
73.7
33.5
85.7
71,0
X-ALMA (Oun)
25.3
85.5
72.2
30.5
86.2
74.1
33.7
86.1
73,0
BLEU
COMET 22
XCUMET
BLEU
COMET 22
XCOMET
BLBU
COMET 22
XCOMET
BLEU
COMET 22
XCOMET
ALMA 13B
42.6
85.5
89.1
23.2
50.5
30.6
33.0
83.3
72.1
Toweritsttngt Y0.2
39.8
81.6
69.3
23.9
60.5
90.4
$4.1
K3.1
72.1
NLLBB-3.38
20.1
65.6
20.1
11.4
67.8
41.6
6.5
65.8
61.5
24.4
76.7
62.2
1. 21ppa--
22.1
760
66.1
20.7
78.
A8
169
79.5
68.5
20.6
81.1
70.3
1. AMAX--Appss-3
25.6
79.4
66.7
23.3
70.3
A9.3
176
80
89.3
20.4
B1.3
70.5
Aso-
34.0
81.6
65.1
13.8
73.7
55.5
13.9
77.3
63.7
28.4
B1.4
70.5
Aso-23-48
32.3
82.1
67.9
23.5
78.8
65.0
193
80.2
RSL6
30.0
61.6
70.8
Asn-23-358
32.7
82.3
68.3
23.5
79.7
69.5
21.3
81.6
69.8
31.7
N3.3
71.1
X-AMM (mll SFT)
42.5
85.3
66.9
23,8
50.3
69.9
20.4
81.6
70.2
33.8
K2.4
71.5
X. ALMA (Ours)
41.7
85.7
69.1
25.1
80.9
71.2
21.0
82.4
71.2
33,0
83.3
72.3
UK 70C
BC 10B
203. JX 70B
COMET-12
XCOMET
BLFU
COMPT-20 XOOMET
COMET-32
XCOMET
LLLMAXX- Alpaca-7
LLaMAN3-AMpuss-BB
Ap9-23-358
32.0
X-ALMA
(Cuss
345
71.5
26

---

Published as a conference paper at ICLR 2025
F EXAMPLES OF OVER-REJECTION
Figure presents examples over-rejection translations from Chinese to English. For each source sentence, we provide translations from the reference, ARPO (implemented on CPO), and CPO, The words where ARPO and CPO differ from the reference are color-highlighted: green indicates that the variation does not affect the meaning, while red indicates a potentially negative impact on translation quality. As shown in Figure / CPO exhibits more stylistic variations than ARPO across all translation examples. Although most of the stylistic changes introduced by CPO are accurate and do not impair meaning, small number are detrimental. Excessive changes in style can result in sub-optimal translations, phenomenon we refer to as 'over-rejection'.
Example
Source: 2: T NE 20.0 Rt ( ai 200 & A A AI dt
American and German lives lost. ARPO: All but one of the British cruisers were sunk, and nearly 200 Americans and Germans lost their lives. cruiser, ships were sunk, nearly
Reference: All the ships sunk, for British
exception
lives.
Example
Source: 2017 # k. EE QVC M.
Reference: late 2017, Siminoff appeared on shopping television channel QVC.
Siminoff appeared the QVC televiston shopping cbannel
CPO: At the end of 2017, Zhukovaky oppeared on the QVC television shopping channel.
Example
Source: U0 88/
Reference: The smaller the Rosbby number, the less acive the star with respect to magnetic reversals. lower Raschy
active CPO: The Rossby number lower, the less active the star in flipping its magnetic field.
flipping its magnetic field.
Example
Source: F5 40 BA TWNE A (CUMEE)) 26-4 T
Reference: Majorcan cuisine, like that of similar zomes in Mediteranean, based on bread, vegetables and meat (specially pork), slive oil throughout.
ARPO: Mallarea's cuisine, like that similar Mediterranaan regions, is hased primarily on berad, vogetables, and menal (particularly pork), and exclusively olive
CPO: The cuisine of Mallorca is similar to that of other Mediterranean regions, with dishes primarily based on bread, vegetables, meat (particularly park), and entirely using
Example
Source: 32 f1 AMB 4 (EM /E
Reference: the performens
joints stage, theatre encouraging
join
ARPO: As result, the actor smoked marijuana and the theater itself encouraged the audience to join in. CPO: As result, actors smoked cannabis on stage, and the theater itself encouraged audience to participate in the act,
Examples over-rejection in Chinese-to-Englinn translation, comparing translations from the reference, ARPO, and CPO. Green highlights indicate acceptable variations, while red highlights show the potentially harmful changes. CPO introduces more stylistic differences than ARPO, with most being correct but some leading to over-rejection. Although most of variations are correct, the phenomenon of excessive stylistic changes leading to non-optimal translations is referred to as 'over-rejection'.
Figure 7:
27


Reviews:
[{"review_id": "Uekh7sLsei", "reviewer": "Reviewer_nWUB", "summary": "The paper presents an approach to multilingual machine translation that focuses on mitigating losses on non-high-resource language pairs. The authors focus on 50 languages ranging from low- to high-resource. The approach consists of performing several levels of continued pretraining and post-training using LoRA modules that group language families together. The training recipe introduces a new preference learning method called ARPO that aims to mitigate \"over-rejection\" of samples that seems to occur in machine translation for pairs of hypotheses that are very similar to each other in which the differences are just a few characters or tokens. Overall the results indicate state-of-the-art results on multilingual MT benchmarks when compared to other open source multilingual LLMs.", "questions": "Questions made above", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is very well written, clear and easy to read. The motivation is solid, timely and important: multilingual LLMs usually do not consistently maintain translation quality when we scale the number of languages. The experimental settings are sound and robust. The preference learning method proposed, ARPO, is very useful and takes into consideration shortcomings of previous methods applied to MT. It is going to be useful for the community working on MT and similar problems.", "weaknesses": "* It's unclear if the method works for non English directions. From what I could understand one could load more than one language module and translate across language families. Could you clarify that?\n* All the evaluation relies on reference-based automated metrics. There's no human evaluation to validate if the findings automated metrics hold", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uvgMDNb7lO", "reviewer": "Reviewer_En5M", "summary": "The paper presents plug-and-play adapter modules to improve language scalability for multilingual machine translation with LLMs. Unlike existing models that massively scale the support for new LPs without delivering high quality on all supported languages, the authors claim that using language-specific (LS) modules (for language groups) can reduce interference between languages during training and improve learning and downstream performance. Only the base model and the corresponding language-specific module are activated for a specific LP during inference. \n\nThe authors train the complete model (base +LS modules) in three pre-training and two post-training stages. For the last post-training stage, they also propose adaptive rejection to reduce over-rejection, a phenomenon that happens when the preferred and dispreferred translations have high overlap.\n\nResults on FLORES shows that their model achieves consistently high-quality translations across language subgroups as compared to  baselines.", "questions": "1. L249-250 Why is parallel text referred to as a pseudo-monolingual dataset?\n\n2. L213-215: what does accurate classification mean here?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors propose plug-and-play adapters to improve language scalability for LLM-based multilingual machine translation and present a recipe for training such models. \n2. The multilingual preference dataset (to be released) can be a valuable resource to the community.\n3. Results on the FLORES benchmark show that the proposed approach results in superior quality than baselines across language subgroups, showing the efficacy of their approach.", "weaknesses": "1. Using language-specific adapters for multilingual MT is not new.  The paper doesn't establish much connection to prior work that has been done before the LLM era to improve multilingual machine translation by incorporating various modular and adaptable approaches. This is important even if the methodologies and technical setups differ. \n\n2. The ARPO loss introduced in Section 4  doesn't address the core issue which is the granularity at which the preference loss is defined and applied. Instead of diluting the signal from dispreferred response with a penalty when the preferred and dispreferred are close, a more meaningful learning signal would be to help the model focus on differences between the two texts at a finer granularity.  This can be especially important where preferences hinge on subtle but critical nuances of translation. For now, ARPO risks being a temporary fix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SFAY6axj8l", "reviewer": "Reviewer_wi3x", "summary": "The paper introduces X-ALMA, a multilingual machine translation model that aims to achieve top-tier performance across 50 diverse languages, regardless of their resource levels. It is friendly to mid- and low-resource languages. The model uses a plug-and-play language-specific module architecture to prevent language conflicts during training and a carefully designed training regimen with novel optimization methods to maximize translation performance. Experimental results demonstrate that X-ALMA outperforms existing multilingual LLMs in every single translation direction on the FLORES-200 and WMT’23 test datasets according to COMET-22. The paper's contributions include the design of X-ALMA's architecture and training methodology.", "questions": "On page 5, in Pre-Training Stage 2: For the 10B used in fine-tuning, are an equal number of tokens taken from each language in the group, or are they taken proportionally?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper introduces a new approach to multilingual machine translation, featuring a plug-and-play module architecture that effectively addresses language conflicts during training. Additionally, the ARPO method represents a creative combination of existing ideas, aiming to optimize translation performance.\nExtensive experimental evaluations demonstrate the effectiveness of the X-ALMA model, achieving superior performance compared to existing open-source multilingual LLMs on multiple language pairs. The paper provides clear descriptions of the background, related work, experimental setup, and analysis of results, making it easy to understand and follow.\nThis work has the potential to make a significant impact on multilingual machine translation, particularly in terms of delivering high-quality translations for resource-limited languages. \nThe paper is well-structured and clearly written, providing readers with a comprehensive understanding of the research work.", "weaknesses": "The paper presents a comparison with the state-of-the-art in LLM-based machine translation. I am curious about how this compares to the state-of-the-art results of encoder-decoder NMT models that are trained on specific languages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)