F ro M : F robenius Norm-Based Data-Free Adaptive Model Merging (2025)

Abstract: With the development of large language models, fine-tuning has emerged as an effective method to enhance performance in specific scenarios by injecting domain-specific knowledge. In this context, model merging techniques provide a solution for fusing knowledge from multiple fine-tuning models by combining their parameters. However, traditional methods often encounter task interference when merging full fine-tuning models, and this problem becomes even more evident in parameter-efficient fine-tuning scenarios. In this paper, we introduce an improvement to the RegMean method, which indirectly leverages the training data to approximate the outputs of the linear layers before and after merging. We propose an adaptive merging method called FroM, which directly measures the model parameters using the Frobenius norm, without any training data. By introducing an additional hyperparameter for control, FroM outperforms baseline methods across various fine-tuning scenarios, alleviating the task interference problem.

## introduction in recent years, significant advancements have been made in the fields of artificial intelligence and natural language processing, with large language models (llms) emerging as a focal point of attention @xcite . among the various approaches, the pretraining and fine-tuning paradigm has emerged as the foundational framework for llm technology @xcite @xcite . with a growing number of fine-tuned downstream models, model merging has emerged as an effective approach to integrate models trained under different hyperparameter settings or for multitask learning @xcite . moreover, model merging is commonly implemented in a data-free method, making it particularly advantageous for privacy-sensitive applications like federated learning @xcite . most existing model merging methods are based on linear operations such as weighted averaging @xcite and task arithmetic @xcite . weighted averaging combines parameters from multiple fine-tuned models, while task arithmetic enables multi-task learning and forgetting by adding and subtracting task vectors derived from parameter differences. nevertheless, these methods encounter a fundamental challenge: simple linear combinations of parameters often lead to task interference @xcite . this issue is especially evident in parameter-efficient fine-tuning (peft) scenarios @xcite , such as low-rank adaptation (lora) @xcite . compared to full fine-tuning (fft), the sparse structure of low-rank matrices in these methods further aggravates task interference @xcite . figure 1 presents a schematic illustration of the fine-tuning and model merging processes. to address this challenge, we take inspiration from the nonlinear weighted optimization method of regmean @xcite and introduce a novel method, frobenius norm-based data-free adaptive model merging (from). our method improves upon the regmean method, which derives a closedform solution by approximating the linear layer outputs of the models using the inner product matrix of the training data. however, regmean still encounters challenges due to its reliance on access to training data, which may not always be available. to address this limitation, from adopts a data-free approach that directly measures model parameters using the frobenius norm, and employs adaptive coefficients to adjust the discrepancies between task vectors. our approach reduces the reliance on training data, while outperforming baselines in both full fine-tuning and lora scenarios, effectively mitigating the task interference issue. our main contributions are as follows: ## related work fine-tuning of llms. fine-tuning is a method for optimizing pre-trained llms for specific tasks. through fine-tuning, llms not only excel in general tasks but can also meet the requirements of particular application scenarios @xcite @xcite . from the perspective of the amount of parameters updated, fine-tuning can be categorized into two approaches: fft and peft. fft updates all parameters of the pre-trained model, which often yields excellent performance on specific tasks. however, this approach is both time-consuming and computationally intensive. as models become larger and more complex, conducting fft has become more challenging, particularly in environments with limited computational resources @xcite . alternatively, peft focuses on updating a small number of parameters to reduce computational costs and storage requirements, making it more widely used in resource-constrained settings or customized scenarios @xcite . one of the most widely used methods within peft is lora @xcite , which significantly reduces computational and storage costs by fine-tuning low-rank matrices, achieving highly effective finetuning with a limited number of parameters. model merging. @xcite introduce several collaboration strategies for llms, highlighting model merging as a technique that prevents models from getting trapped in local optima and improves multi-task collaboration ability. traditional methods typically rely on weighted averaging of model parameters. @xcite introduce model soups, which apply both a weighted averaging method and a greedy merging strategy. (matena and raffel, 2022) adopt a different approach by utilizing the fisher information matrix for parameterlevel model merging, applying a weighted merging strategy to enhance model integration. focusing on the outputs of linear layers, @xcite propose regmean, a method that indirectly utilizes the training data through the inner product matrices to derive a closed-form solution for merging model parameters. meanwhile, @xcite present a novel paradigm called task arithmetic, in which task vectors capture the ability shift of different fine-tuned models compared to the pretrained model across various tasks. this allows for operations like forgetting and learning through arithmetic manipulation of the task vectors. however, the task vectors of different models may inevitably lead to some degree of task interference. prior research has demonstrated that weight disentanglement is crucial to the success of task arithmetic @xcite . to address this, @xcite introduce ties-merging, a strategy that minimizes redundant parameters and selects signs to reduce interference during model merging. in addition, the dare method proposed by @xcite involves the random dropping of incremental parameters and rescaling operations. by integrating with other model merging algorithms, it enhances the performance of the model after fusion. according to the research by @xcite , existing merging methods are more challenging to apply to lora fine-tuned models compared to fft models. our method builds upon regmean and introduces improvements that eliminate the need for obtaining the inner product matrix of training data. it can also be solved using a closed-form solution, supporting the integration of either fft models or lora adapters. ## experimental setup merging of fft models. we test two different model architectures, each based on the same foundational model, across three distinct fft tasks: instruction-following, mathematical reasoning, and code-generating. first, we adopt the experimental setup of @xcite compute matrix b using equation 7 7: regarding benchmark selection, we follow the setup of @xcite , utilizing alpacaeval @xcite to assess the models' instructionfollowing abilities, gsm8k @xcite and math @xcite to evaluate their mathematical reasoning skills, and hu-maneval @xcite and mbpp @xcite for testing code generation capabilities. further details can be found in appendix b. ## conclusion in this paper, we propose a novel merging approach called from, which utilizes the frobenius norm to quantify weight discrepancies between models without any training data. this method adjusts adaptively through a hyperparameter, and the merged parameters are derived using a closedform solution. for the experimental part, we first test our method on three models based on llama-2-13b, and the results show that from outperforms existing baseline methods. next, while baseline methods struggle to effectively integrate three finetuned models based on qwen2.5-7b, our approach outperforms the baseline methods, demonstrating its effectiveness and robustness. finally, when merging lora adapters, from also shows superior performance over other methods. overall, the proposed from method demonstrates outstanding performance in both fft and lora scenarios, effectively alleviating the task interference issue in model merging and showcasing strong applicability. we hope this work inspires future research in the model merging field, helps address existing task interference issues, and provides a new perspective for tackling these challenges.