The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLM s (2025)

Abstract: Large language models (LLMs) still struggle across tasks outside of high-resource languages. In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce. Building on prior work, we first validate that the subsets of model parameters that matter most for mathematical reasoning and multilingual capabilities are distinctly non-overlapping. To exploit this implicit separability between task and target language parameterization, we develop and analyze numerous modular frameworks to improve the composition of the two during fine-tuning. These methods generally employ freezing parameters or post hoc model merging to assign math and language improvement to different key parts of the LLM. In the absence of in-language math data, we demonstrate that the modular approaches successfully improve upon baselines across three languages, four models, and two fine-tuning paradigms (full and LoRA). Furthermore, we identify the most consistently successful modular method to be fine-tuning separate language and math experts and model merging via Layer-Swapping, somewhat surprisingly. We offer possible explanations for this result via recent works on the linearity of task vectors. We further explain this by empirically showing that reverting less useful fine-tuning updates after training often outperforms freezing them from the start.

## introduction post-training large language models (llms) on labeled text data is a critical step in developing models for real-world applications. however, when these llms are fine-tuned for lower-resource languages, significant challenges arise due to the pretrained model's limited capabilities. although in recent years the broader scaling of pretraining and increased investment in additional languages @xcite have led to major improvements, pretrained llms still struggle to understand and generate text in all but a few languages @xcite . this pretraining disparity is further exacerbated by the lack of available high-quality multilingual fine-tuning data @xcite and the significant cost to procure such annotated data (even through machine translation). for many of the capabilities developers target during post-training (e.g., instruction-following, reasoning, or safety) there are only sufficient open-source data available in english, chinese, and a handful of other languages. this motivates the need for better cross-lingual transfer: the generalization of learned capacities from high-resource languages to lower-resource ones @xcite . despite recent releases of massive mixture-ofexpert llms @xcite @xcite , a large majority of modern llms are dense, meaning that all parameters are active during training and inference. however, even within dense llms, recent works have found separability in where and how varying capabilities are represented @xcite . for example, multilingual capabilities are typically concentrated in the top and bottom transformer layers and multi-head attention parameters of an llm @xcite . this notably contrasts mathematical reasoning capabilities being encoded mainly in the middle transformer layers @xcite . in the context of cross-lingual transfer, this functional separation motivates modular approaches to finetuning, which distinct model components can be trained, swapped, or merged @xcite for efficient and flexible multiobjective optimization. in this work, we explore several modular approaches for composing target task and target language capabilities in off-the-shelf dense llms. ## experiments we describe numerous methods that modularize off-the-shelf, dense llms in different ways. we describe separate training as when we conduct separate sft runs on different datasets, albeit starting from the same off-the-shelf model. as previously mentioned, the separately trained checkpoints are then merged via layer-swapping. separate parameters implies that only the partition of parameters allocated (see section 3.3) to that dataset are trained while the rest remain frozen. ## results our experimental setting was designed to replicate a real-world scenario where multilingual llm developers would take a post-trained llm and are in our setting, we find that only training on the language dataset is more effective in improving the target language mgsm score than only on the math dataset (details in appendix a.1). this implies, perhaps, that what our four models need most, is improved swahili, bengali, or telugu abilities as opposed to math improvement. we validate the lack of need for full-parameter training when doing both language adaptation and math sft. once the most useful parameters have been identified for such a skill, as discussed in section 3.3, comparable performance to full sft can be achieved with a fraction of the trainable parameters. beyond potentially contributing to compositionality, this leads to faster and more memoryefficient training. more details on these baselines can be seen in appendix a.1. we do note, however, that in the absence of resource limitations, sft with less trainable parameters converged a bit slower and full fine-tuning still performed best. this is also true for lora, which has much less trainable parameters by nature. a significant result is that all our modular solutions perform statistically-significantly better than the non-modular baselines, as can be seen in within our modular solutions, however, we find numerous surprising results. first, freezing the unused parameters in training experts before layer-swapping does not improve upon full training. as detailed in the last four columns of to validate results further, we also evaluate more expensive continual pretraining (cpt) for qwen2.5 in bengali across the experimental designs and find agreement with our sft results (see details in appendix a.2, a.7). however, we limit discussion of these results because of the small scale of experimental results. we additionally analyze the composability of individual experts under layer-swapping. we define a good merging indicator as an evaluation signal of an expert that correlates with the performance of the merged model. we find that performance on general nlu benchmarks-belebele and global mmlu-is a stronger indicator of a language expert's merge quality than mgsm results in the target language. similarly, mgsm performance in english is a better predictor for a math expert than mgsm in the target language. this is notable because mgsm in the target language is the target task of course, yet results more directly related to the training data tends to be more important for proper task composition. ## discussion given the rejection of our hypothesis that simultaneous fine-tuning would most effectively compose task and language capabilities, we discuss potential explanations for this outcome. train-then-revert vs. freeze-then-train intuition may dictate that fine-tuning parameters and then later reverting part of them should be less effective than simply freezing those parameters from the start. in the former, the fine-tuning is unaware of future edits while the latter provides hard constraints during optimization. however, empirically, we find that across models, trainingthen-resetting outperforms freezing-then-resetting. we display this for our english math fine-tuning in ## conclusions our results demonstrate that imposing modularity into dense llms for cross-lingual transfer is quite effective in low-data scenarios. we empirically validate this with numerous ways to impose such modularity through fine-tuning with frozen parameters or model merging, all of which prove more effective than non-modular baselines. furthermore, we discover the surprising success of layer-swapping over other modular methods that fine-tune task and language together or do not ad hoc revert parameter updates. we conjecture that the success of this ad hoc merging method is because the math and language experts, when represented as task vectors, exhibit a high degree of linearity. as a result, this method benefits from more robust training over all parameters while also leading to effective compositionality. we also empirically demonstrate that the success of layer-swapping is in part due to frozen-parameter fine-tunings underperforming full fine-tunings followed by parameter resets.