Asymmetric Conflict and Synergy in Post-training for LLM -based Multilingual Machine Translation (2025)

Abstract: The emergence of Large Language Models (LLMs) has advanced the multilingual machine translation (MMT), yet the Curse of Multilinguality (CoM) remains a major challenge. Existing work in LLM-based MMT typically mitigates this issue via scaling up training and computation budget, which raises a critical question: Is scaling up the training and computation budget truly necessary for high-quality MMT, or can a deeper understanding of CoM provide a more efficient solution? To explore this problem, we analyze the linguistic conflicts and synergy, the underlying mechanism of CoM during post-training phase. We identify an asymmetric phenomenon in linguistic conflicts and synergy: the dominance of conflicts and synergy varies in different translation directions, leading to sub-optimal adaptation in existing post-training methods. We further find that a significant bottleneck in MMT appears to lie in post-training rather than multilingual pre-training, suggesting the need for more effective adaptation strategies. Building on these new insights, we propose a direction-aware training approach, combined with group-wise model merging, to address asymmetry in linguistic conflicts and synergy explicitly. Leveraging this strategy, our method fine-tunes X-ALMA-13B-Pretrain—trained only with multilingual pre-training—achieving comparable performance to XALMA-13B (only SFT) while using only 20B pretraining tokens and 17B parameters—5.5× fewer pretraining-tokens and 1.7x fewer model size—with just 0.85 COMET drop on Flores-200 testsets of 50 languages.

## introduction large language models (llms) have shown remarkable general capabilities @xcite @xcite and have advanced multilingual machine translation @xcite @xcite . for example, aya-101 @xcite expands support to 101 languages and achieves strong performance in multilingual machine translation, while llamax @xcite further pushes performance beyond 100 languages. the common practice behind these successes is the large-scale pretraining, which typically involves monolingual pretraining foot_0 , parallel pretraining, or both-followed by a small-scale, high-quality posttraining phase. however, as llms scale to more languages, they suffer from the issue of curse of multilinguality (com) @xcite , which degrades the translation performance. understanding and mitigating com is not new in the mmt literature. in traditional mmt, existing research has identified critical factors such as resource imbalances, limited model capacity, linguistic similarity, and complex interactions between language pairs, particularly for low-resource languages @xcite @xcite , and proposed solutions including language-specific modules @xcite @xcite , vocabulary optimization @xcite , data sampling techniques @xcite @xcite , and continual learning approach @xcite . based on these studies, recent llm-based mmt research focuses on designing increasingly complex training pipelines and modular architectures. for instance, @xcite proposed a five-stage training pipeline incorporating language-specific modules. however, existing analyses primarily focus on the encoder-decoder paradigm, while current llm-based approaches heavily rely on scaling up model capacity and computational resources, making them prohibitively expensive. this raises a critical question: is scaling up the training and computation budget truly necessary for high-quality mmt, or can a deeper understanding of com in llm-based mmt provides a more efficient solution? in this work, we systematically investigate linguistic conflicts and synergy during post-training phase. we conduct extensive experiments with different settings: across 5 to 50 languages, three pretrained llms -alma-7b-pretrain, alma-13b-pretrain and x-alma-13b-pretrain, three distinct post-training strategies -multilingual training, group multilingual training, and separate training. we observe a consistent pattern: asymmetry in linguistic conflicts and synergy (figure 2 , appendix b.1and b.2 ). for example, in multilingual training, @xmath0, leading to performance degradation, whereas @xmath1, where xx denotes 49 different languages other than english. we further show this asymmetric phenomenon cannot be easily mitigated through existing training approaches, such as group multilingual training ( another key finding of our work is that a simple multilingual pre-training stage can be sufficient to equip foundation models with ideal multilingual capabilities, whereas the bottleneck lies in the post-training stage (dotted lines in figure 2 (g-i)). motivated by these findings, we propose a novel direction-aware training (dat) approach and build an efficient mmt starting from a relatively efficient base model, the x-alma-13b-pretrain-utilizing only simple multilingual pretraining on 20 billion tokens. our approach fully leverages the interactive characteristics of different language directions to reduce conflicts while maximizing synergy. we also present a scalable version of the approach, named datm, which utilizes model merging to further enhance efficiency with only negligible performance degradation. through comprehensive evaluations on flores-200 and wmt23 benchmark, we demonstrate the effectiveness of our approach. notably, as shown in figure 1 , compared to x-alma (only sft) @xcite , our model x-alma-13b-dat maintains comparable performance while having two advantages: 1) utilizing a simple and efficient training recipe -starting from base models with fewer pre-training tokens and employing a post-training stage. 2) parameter-efficient -we consume 1.7x fewer parameters compared to x-alma (only sft). these results demonstrate that simple pre-training combined with dedicated post-training can also achieve good multilingual performance. ## experimental settings in this section, we introduce the basic experimental settings used in section 3 and section 4. ## the phenomenon: asymmetry in linguistic conflicts and synergy in this section, we investigate the phenomenon of asymmetry in linguistic conflicts and synergy in llm-based mmt. we begin by illustrating the phenomenon (section 3.1) and analyzing its distribution across two essential factors: language resources and groups (section 3.2). finally, we show how this phenomenon poses challenges to existing post-training strategies (section 3.3). ## direction-aware training and merging for efficient llm-based mmt in this section, we show how to construct an efficient mmt system by leveraging the insights from section 3, starting from a base model with simple multilingual pre-training.