EMR-Merging: Tuning-Free High-Performance Model Merging

Authors: chenyu  huang
†  peng
tao  chen
  tong he
xiangyu  yue
wanli  ouyang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 05:39:31

Abstract: The success of pretrain-finetune paradigm brings about the release of numerous model weights. In this case, merging models finetuned on different tasks to enable a single model with multi-task capabilities is gaining increasing attention for its practicability. Existing model merging methods usually suffer from (1) significant performance degradation or (2) requiring tuning by additional data or training. In this paper, we rethink and analyze the existing model merging paradigm. We discover that using a single model's weights can hardly simulate all the models' performance. To tackle this issue, we propose Elect, Mask & Rescale-Merging (EMR-Merging). We first (a) elect a unified model from all the model weights and then (b) generate extremely lightweight task-specific modulators, including masks and rescalers, to align the direction and magnitude between the unified model and each specific model, respectively. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance. We find that EMR-Merging shows outstanding performance compared to existing merging methods under different classical and newly-established settings, including merging different numbers of vision models (up to 30), NLP models, PEFT models, and multi-modal models.

Main paper: 
 introduction: 
 With the rapid development of deep learning, different model architectures [36, 22, 71, 88] are proposed, along with multiple training strategies [89, 86] . Pre-trained models' capabilities are enhanced, thus showing increasing significance [54, 22, 7, 19] . Finetuning models on downstream tasks from a pre-trained model has become a standard paradigm in both NLP and vision fields [20, 51, 19, 22, 5, 87] , which usually leads to improved performance with less labeled data. With the development of open-source repositories such as Huggingface [79] , timm [77] , and torchvision [44] , the number of pre-trained and finetuned checkpoints exponentially rise. However, applying individual models to different tasks results in high storage and deployment costs. Multi-task learning (MTL) partially solves this problem by jointly training a model using multiple datasets [70, 93, 95] , but it suffers from (i) high computational costs and (ii) data unavailability due to privacy [33] . Recently, model merging attempts to solve these drawbacks by combining weights instead of additional training, thus showing vital significance and broad application prospects.A simple strategy of model merging is averaging the model weights [80] , but it usually causes obvious performance degradation, as shown in Fig. 1 . To this end, there are multiple model merging methods proposed to improve the performance of the merged model, which can be roughly divided into three categories: (i) Weighted averaging of model weights include Fisher-Merging [46] and RegMean [33] . They use pre-computed Fisher information matrices [23] and inner-product matrices [33] to tune the coefficients for weighted averaging. (ii) Task vector-based methods that add task vectors together instead of model weights, include Task Arithmetic [30] , Ties-Merging [84] , and AdaMerging [85] . Ties-Merging handles the interference issue and AdaMerging adaptively tunes the merging coefficients. (iii) Pre-processing techniques include DARE [90] . It reduces interference by dropping most elements and rescaling the others in task vectors. Despite the promising results, there are two unresolved problems with the existing model merging methods: (1) The performance gap between the merged model and individual models or MTL is still obvious, as shown in Fig. 1 . (2) The performance improvement of existing methods depends on tuning by data or training, as shown in Tab. 1.Figure 1 : The average accuracy of the multi-task performance of different model merging methods on eight vision tasks. Among all the merging methods, our EMR-MERGING is the only one comparable to the performance of MTL and even individual models. [33] × ✓ × ×Task Arithmetic [30] × ✓ ✓ × Ties-Merging [84] × ✓ ✓ × AdaMerging [85] ×✓ × ✓ EMR-Merging(Ours) × × × ×To boost the performance of model merging, we rethink and analyze the existing model merging paradigm. We discover that the goal of all the existing methods is to obtain a single model applicable to all the N tasks, as follows:EQUATIONwhere [W 1 ..W N ] are the model weights to be merged, M denotes the merging function, and W M is the merged model weight. This paradigm may inevitably lead to a nonnegligible gap between the merged model and each individual model, especially when there are numerous models or models on challenging tasks. We argue that using a single model weight to simulate all the model weights is sub-optimal. To tackle this issue, we propose a brand new merging paradigm: We first extract a unified model weight from all the models' weights, and then we calculate and store significant but lightweight task-specific parts of each model weight. This process can be written as:EQUATIONwhere W uni represents the common and shared part of all model weights and [E 1 ..E N ] denote the task-specific parts of each model weight. M ′ is the revised merging function following our paradigm.We first elect a unified model from all the model weights. The election strategy is choosing the maximum absolute value of each parameter on the specified sign direction to minimize interference and avoid additional tuning. Then we generate additional lightweight task-specific modulators, including masks and rescalers. Their functions are respectively to align the direction and magnitude of the unified model with the original task-specific model. We find that applying the task-specific modulators to the unified model can better approximate the task-specific model, thus improving performance. The detailed process, theoretical and empirical analysis of the proposed method are illustrated in Section 3. By applying our method, the performance of model merging is significantly enhanced and is comparable to MTL or individual models, as shown in Fig. 1 . Meanwhile, EMR-MERGING requires no data, tuning, or any additional training, as shown in Tab. 1.We first demonstrate the effectiveness of the proposed EMR-MERGING under the existing setting of (1) merging Vision Transformer (ViT) [22] models of different sizes on 8 vision tasks, (2) merging parameter-efficient finetuning (PEFT) models on 11 language tasks, and (3) merging GPT-2 [55] models on 7 language tasks. Our method shows significant performance improvement under these settings, even when compared to the strongest baseline. We further validate the method's effectiveness under newly-established and more challenging settings including: (4) merging ViTs on 30 vision During the (b) Inference Procedure, we apply the corresponding mask and rescaler to the unified task vector to obtain a specific task vector. The process of (c)Task-specific Direction and Amplitude Modulation includes obtaining task-specific masks and scalers.tasks, (5) merging RoBERTa [43] models on 8 NLP tasks, and (6) merging BEiT3 [75] models on 5 multi-modal tasks.Our contributions can be summarized as: (1) We propose a novel merging method called EMR-MERGING, which merges task-specific models into a unified model and lightweight task-specific modulators (i.e., masks and rescalers), requiring no data, tuning, or additional training. (2) The proposed EMR-MERGING is simple-but-effective, and its effectiveness is validated on various classical benchmarks and newly-established benchmarks under various vision, NLP, PEFT, and multi-modal settings. (3) We show that the masks and rescalers of EMR-MERGING for aligning task-specific direction and amplitude of task vectors are applicable to most kinds of merging methods. 
related work: 
 Model Merging obtains a model using the existing task-specific model weights instead of training [33, 30, 84, 85, 66, 90, 46] . Simply averaging [80] usually causes severe performance degradation. Various methods are proposed to handle this problem. Fisher-Merging [46] and RegMean [33] use fisher information matrices [23] and inner-product matrices [33] to calculate the merging coefficients for weighted merging. However, they require additional matrices released by model owners or manually computed. Task Arithmetic [30] merges models by adding together task vectors, which is the difference between the finetuned and pre-trained models. Ties-Merging [84] and AdaMerging [85] are based on task vectors. Ties-Merging resolves interference and AdaMerging adaptively learns the merging coefficients. However, the performance of Task Arithmetic and Ties-Merging highly depends on manually tuning the merging coefficients and AdaMerging needs additional training to obtain them. DARE [90] reduces interference by randomly dropping most elements and rescaling the remaining ones in each task vector before merging. However, DARE's performance is only validated under the setting of merging a limited number of tasks and the performance gain is also limited. In addition, all the existing methods merge models into a single one, and have not been verified under experimental settings of more models to merge, models on more difficult tasks, and multi-modal models. In this paper, we propose EMR-MERGING, which requires no tuning while showing impressive performance under various settings.Multi-Task Learning trains a single model using training data from multiple tasks together [70, 93, 95] . MTL typically necessitates access to the labeled data of multiple tasks for training the model from scratch. Though enabling the model multi-task capabilities, MTL suffers from not only (i) the expensive computational cost for training, especially for large models, but also (ii) the limited data availability due to data privacy [85] . In comparison, model merging solves the mentioned problems by combining the model weights without using training data or additional training, thus obtaining a multi-task model while sharply reducing the costs.Supervised Finetuning from pre-trained models on down-stream tasks is becoming a standard paradigm in both NLP and vision fields [20, 51, 19, 22, 5] . Depending on whether all the parameters of models are adjusted, SFT can be divided into conventional full finetuning (FFT) and parameterefficient finetuning (PEFT), which is proposed to reduce the number of trainable parameters for downstream tasks by adjusting the inserted small modules called adapters while keeping the whole model frozen [28, 29, 42] . PEFT is becoming the prevailing method to adapt pre-trained large models because of its efficiency [94] . There are a large number of pre-trained, full finetuned model weights, and PEFT module weights available on public repositories [79, 77, 44] . In this paper, the proposed EMR-MERGING is based on the common pretrain-finetune paradigm and we show the applicability of our method to both full finetuned models and PEFT modules.3 Method 
motivation: 
 Given N tasks [T 1 ..T N ], the goal of model merging is to obtain a model applicable to all the tasks using finetuned models [W 1 ..W N ] from the same pre-trained model W pre on each task. Existing methods focus on merging the models into a single model W M . Please check Appendix C for detailed information on the existing merging methods. However, a single model can hardly represent all the model weights, thus causing severe performance drops. We discover that the combination of a unified task vector and lightweight task-specific modulators can settle this issue to a significant extent by approximating the task-specific vectors better without any additional tuning. The size of proposed task-specific modulators is discussed in Section 4.4, which is much smaller than that of a model.The overall framework of EMR-MERGING is shown in Fig. 2 . We follow the setting of task vectorbased methods [30, 84, 85] and we merge models using task vectors. For task T i , i ∈ [1..N ], the corresponding task vector is defined asτ i = W i -W pre , where τ i ∈ R d .Electing a unified task vector We first create an aggregate elected sign vector γ uni = sgn( N t=1 τ t ) by choosing the sign with the higher total magnitude of each parameter across all relevant task vectors. Then we choose the maximum absolute value of each parameter with the sign consistent with γ uni from all the task vectors and obtain absolute value vector ϵ uni ∈ R d . By combining γ uni and ϵ uni , the unified task vector can be obtained by τ uni = γ uni ⊙ ϵ uni . The electing procedure can reserve the maximum amplitude and sign information shared by the task vectors, thereby maximally reducing interference. The unified task vector τ uni corresponds the W uni in Eq. 2. Before being applied to task T i , the τ uni needs to be modulated in advance by task-specific modulators, which are corresponding to E i in Eq. 2. The generation of task-specific modulators is described below:Task-specific masks. Next, we compare the unified task vector τ uni with each task vector τ i . The taskspecific mask M i = (τ i ⊙ τ uni > 0) for task i sets the elements whose signs are not correspondent with τ uni to zero and the rest to one. The function of the masks is to align the direction of the unified model with the task-specific model. The masks share the same structure with the task-specific models but due to their 1-bit nature, the size of a mask is much smaller than that of a task vector.Task-specific Rescalers Then, for each task, we compute a rescaler parameter to keep the average absolute value of the elements in τ t and M t ⊙ τ uni equal. The function of the rescalers λ i = sum(abs(τi)) sum(abs(Mi⊙τ F )) is to align the parameter magnitude of the unified model with the task-specific model. The significance of rescaling is also reported by DARE [90] , which claims that after dropping most elements in a task vector, rescaling the rest leads to better results compared to not.Before being applied to a task, a task-specific modulation is required to be conducted to the unified task vector. After that, we add it to the pre-trained parameter values W pre . The inference steps of applying the merged model to task t are as follows: Ŵt = W pre + τt , where τt = λ t • M t ⊙ τ uni . It should be noted that during the whole process, no additional tuning is needed, thus requiring no data or additional training. We summarize the algorithm flow in Appendix A.Our goal is to merge model weights by minimizing the distance between the merged model W uni and each individual model W i , where the distance can be calculated by:EQUATION)where τ i refers to the task vector for task T i and τ uni is the unified task vector.Analysis 1: Effectiveness of Masks. After applying the masks M i = (τ i ⊙ τ uni > 0) to the unified model τ uni , the distance Dis M can be formulated as:EQUATIONwhere Dis refers to the distance before applying the masks. Eq. 4 demonstrates that the distance between the merged model and each individual model can be reduced after applying the masks.Analysis 2: Effectiveness of Rescalers. After applying the rescalers λ i = sum(abs(τi)) sum(abs(Mi⊙τ F )) to the masked task vectors M i • τ uni , the distance Dis M,λ is formulated as:EQUATIONEq. 5 demonstrates that the distance between the merged model and each individual model can be minimized after applying the rescalers. Please check Appendix B for detailed proof.In Fig. 3 , we visualize partial results of merging eight ViT-B/32 models on different tasks using t-SNE [69] and Grad-CAM [61] . It can be seen that each procedure of EMR-MERGING can help improve the performance of the merged model and perform closer to individual models. Specifically, a more obvious distinction is shown in t-SNE and a more precise target is focused by Grad-CAM.Please check Section 4.1.1 for experimental details and Appendix E for more visualization results.  
experiment validation: 
 Baseline methods. We compare the proposed EMR-MERGING with: (1) Individual Models, (2) Traditional MTL, (3) Weight Averaging, (4) Fisher Merging [46] , (5) RegMean [33] , (6) Task Arithmetic [30] , (7) Ties-Merging [84] , (8) AdaMerging [85] . For more details about baseline methods, please check Appendix C. Settings. We follow the setting from Task Arithmetic [30] , Ties-Merging [84] , and AdaMerging [85] .We employ ViT-B/32 and ViT-L/14, two variants of CLIP [54] models' visual encoders, as the pre-trained models. The performance of each method is evaluated by eight image classification tasks, including SUN397 [83] , Cars [35] , RESISC45 [10] , EuroSAT [27] , SVHN [91] , GTSRB [65] , MNIST [38] , and DTD [11] . All the datasets are evaluated by accuracy.Results. The experimental results of merging ViT-B/32 and ViT-L/14 on eight tasks are shown in Tab. 2 and Tab. 3. We observe that EMR-MERGING shows significant performance improvement Under this setting, EMR-MERGING performs very close to or even better than traditional MTL, which is normally considered as a reference upper bound for model merging work [85] . For visualized comparison, we provide some visualization results of different merging methods using t-SNE and Grad-CAM in Fig. 5 . It can be seen that among all the merging methods, the visualization results of EMR-MERGING are the closest to individual models, which corresponds to quantitative results. Please check Appendix E for more visualization results.Settings. To further explore the performance of EMR-MERGING, we establish a new benchmark on merging vision models, expanding the number of task-specific models from eight to 30. We employ ViT-B/16 [22] pre-trained on ImageNet-21k [18] as the pre-trained model. The performance is evaluated by image classification datasets including MNIST [38] , CIFAR-10 [36] , Vegetables [1] , Food-101 [6] , Kvasir-v2 [53] , Cars [35] , Intel Images [4] , EuroSAT [27] , Weather [82] , Cats and dogs [15] , MangoLeafBD [2] , Beans [37] , CIFAR-100 [36] , GTSRB [65] , SVHN [91] , Dogs [34] ,Fashion MNIST [81] , Oxford-IIIT-Pet [50] , Landscape Recognition [17] , Flowers Recognition [45] , STL-10 [12] , CUB-200-2011 [73] , EMNIST [13] , DTD [11] , RESISC45 [10] , SUN397 [83] , Kenyan-Food13 [32] , Animal-10N [64] , Garbage Classification [8] , and Fruits-360 [47] , covering tasks from common food classification to disease detection. All of them are evaluated by accuracy.Results. The experimental results are shown in Tab. 4. It can be clearly seen that under this challenging setting of merging 30 models, all the existing methods show significant performance drops compared to individual models. Even RegMean, which performs best among existing methods, still exhibits a performance decay of nearly 25%. However, EMR-MERGING can reduce this value to 3.48%. This shows that the proposed method maintains the performance comparable to individual models when merging vision models even if the number of tasks increases.Settings. We partially follow the setting from DARE [90] . However, instead of merging two or three models at a time, we merge all eight models finetuned on each task. RoBERTa-base [43] model is selected as the pre-trained model. The performance of each method is evaluated by eight tasks from GLUE [74] benchmark, respectively CoLA [76] , SST-2 [63] , MRPC [21] , STS-B [9] , QQP [31] , MNLI [78] , QNLI [56] , and RTE [24] . Among them, CoLA is evaluated by the Matthews correlation coefficient, STS-B is evaluated by the average value of Pearson and Spearman correlation coefficients, and the rest tasks are evaluated by accuracy.Results. The experimental results are shown in Tab. 5. It can be seen that EMR-MERGING outperforms all the existing methods on every task, verifying the applicability of the proposed method to language models. Note that the reported results of Ties-Merging, Task Arithmetic, and RegMean are the best among multiple hyper-parameter settings. Please check Appendix D.4 for more detailed information. It should also be noted that we find that under our setting of merging multiple models, DARE may not help improve the performance. Similar results were also reported by [25] . This may be due to DARE's random dropping strategy can no longer resolve conflicts among task vectors under the setting of merging multiple models. Please check Appendix D.3 for DARE's experimental results.Settings. We follow the setting from FusionBench [68] , a benchmark for model merging. We merge GPT-2 [55] models on seven tasks from GLUE [74] , each with a different head for classification. Under this setting, each task is evaluated by accuracy.Results. The experimental results are shown in Tab. 6. EMR-MERGING outperforms all the merging methods by over 10% and decreases the performance degradation caused by model merging from 12% to 1.6%. This validates the applicability of EMR-MERGING to fully finetuned GPT2-scale language models.Settings. We follow the setting from Ties-Merging [84] . (IA) 3 [42] is a PEFT method that uses learned vectors to scale the base model activations. We use T0-3B [60] as the base model and merge (IA) 3 modules. The performance is evaluated using eleven datasets, including RTE [24] , CB [16] , Winogrande [59] , WiC [52] , WSC [39] , COPA [58] , H-SWAG [92] , Story Cloze [62] , and ANLI [48] from R1 to R3. All the datasets are evaluated by accuracy.Results. The experimental results are shown in Tab. 7. EMR-MERGING outperforms all the merging methods. Compared to methods without validation, EMR-MERGING improves the average accuracy on each task by 2.2%. When compared to methods that require validation data to tune hyper-parameters or compute matrices for weighted merging, EMR-MERGING still improves the average performance by 0.7%, validating the applicability of our method to PEFT models.Settings. We merge BEiT3-base [75] models finetuned on five datasets from different kinds of tasks, respectively ImageNet-1k [18] (Image Classification), VQAv2 [26] (Visual Question Answering), NLVR2 [67] (Visual Reasoning), COCO Captioning [41] (Image Captioning), and COCO Retrieval [41] (Image-Text Retrieval). Among them, COCO Captioning is evaluated by BLEU4 [49] , CIDEr [72] , METEOR [3] , and ROUGE-L [40] . The other tasks are evaluated by accuracy.Results. The experimental results are shown in Tab. 8. It can be seen that EMR-MERGING performs best on all the vision-language tasks regardless of which evaluation metric is applied among all the merging methods, validating the effectiveness of EMR-MERGING in merging multi-modal models. In Fig. 6 , we compare the number of parameters and performance using individual models, Ties-Merging, and EMR-MERGING when merging different numbers of ViT-B/32 models under the setting of no-validation.Compared to other merging methods, EMR-MERGING requires a little additional storage for taskspecific modulators. However, compared to a single 32-bit model, the additional storage caused by a task-specific 1-bit mask equals a binarized network, whose size is 32 times smaller than a single 32-bit model [14] . Additionally, the storage required by a task-specific rescaler, which is a single parameter, is negligible. In Fig. 6 (a), we compare the number of parameters when merging different numbers of models, and we observe that EMR-MERGING's parameter number is slightly more than Ties-Merging but significantly fewer than individual models. Performance. The performance comparison when merging different numbers of models is shown in Fig. 6(b ). Compared to Ties-Merging, the performance of EMR-MERGING is higher and decreases more slowly as the task increases. Note that EMR-MERGING outperforms individual models under the 2-task setting. Similar findings are reported by DARE [90] . More details are shown in Appendix D.5.We perform ablations on all the components of EMR-MERGING as follows.Ablation on Electing procedure. Tab. 9 shows the results of merging eight ViT-B/32 models when the Electing procedure is replaced by other task vector-based merging methods. The effectiveness of our Electing strategy is verified by outperforming the combination of other merging methods with masking and rescaling. Another interesting finding is that as a post-processing procedure, masking and rescaling can help improve the performance of task vector-based merging methods, respectively 6.8%, 5.0%, and 6.6% for Task Arithmetic, Ties-Merging, and AdaMerging++.Ablation on Masking and Rescaling procedures. Then, we further validate the importance of Masking and Rescaling procedures by disabling either or both of them. The results are shown in Tab. 10. It can be seen that simply electing results in a severe performance drop while adding Masking and Rescaling can improve the performance by 27.5% and 12.6%, respectively. Furthermore, compared to separately applying either of these two procedures, jointly applying Masking and Rescaling leads to greater improvement, up to 29.4%. 
conclusion: 
 In this paper, we study on tuning-free and high-performance model merging. We first attribute the severe performance degradation of existing merging methods to that a single model can hardly simulate all the models' performance. Then we propose ELECT, MASK & RESCALE-MERGING (EMR-MERGING), which does not require any data access or additional training for tuning. The effectiveness of EMR-MERGING is validated by comprehensive experiments on various classical benchmarks and newly-established benchmarks under vision, NLP, PEFT, and multi-modal settings. EQUATIONBy applying the masks M i , i ∈ [1..N ], the distance becomes:Dis M = N i=1 ∥τ i -M i ⊙ τ uni ∥ 2 N (9)Furthermore, it can be written as:EQUATION)where abs(•) returns the absolute value of each element in the input. For ease of comparison, the distance without applying M i can be formulated as:Dis = N i=1 ∥Mi ⊙ (abs (τi) -abs (τuni)) ∥ 2 N + N i=1 ∥ (1 -Mi) ⊙ (abs (τi) + abs (τuni)) ∥ 2 N = Dis M + N i=1 ∥ (1 -Mi) ⊙ abs (τuni) ∥ 2 N (11)Thus, we demonstrate that Dis M ≤ Dis, indicating applying task-specific masks can reduce the distance between the merged model and individual models, thus showing effectiveness.Analysis 2: Effectiveness of Rescalers. Suppose we apply a rescaler λ i > 0 to the masked unified task vector M i • τ uni , the distance becomes: To minimize the distance in Eq. 12, we set the first derivative of Dis λ with respect to λ i to 0, thus λ i can be calculated by:Dis M,λ = N i=1 ∥τ i -λ i • M i ⊙ τ uni ∥ 2 N = N i=1 ∥abs (τ i ) -λ i • abs (M i ⊙ τ uni ) ∥ 2 N (12)EQUATIONwhich exactly matches our setting of λ i . This indicates that our setting of rescalers λ i can minimize the distance between the merged model and individual models, which is: Dis M,λ ≤ Dis M , thus showing effectiveness.It is also reflected in Fig. 7 that after Masking and Rescaling, the sign conflicts and L2 distance between the merged model and task-specific models are reduced and the cosine similarity can is improved.  
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the original papers or websites that produced the code package or dataset.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: • Individual Models refer to task-specific models before merging.• Traditional MTL uses datasets from all the tasks to train a single model jointly.• Weight Averaging element-wisely averages all the model weights. Its effectiveness when applied to fine-tuned model weights from the same pre-training has been verified [80, 57, 33] .• Fisher Merging [46] uses Fisher information matrices [23] to calculate the importance of each parameter and weighted merges them based on their importance.• RegMean [33] weighted merges models based on a closed-form solution to the merging problem. When merging K linear model weights W i , where f i (x) = W T i x, i = 1..K, the merging problem can be formulated as:min W K i=1 ∥W T X i -W T i X i ∥ 2, where W is the merged model weights, and X i denotes the input of i th model. The closed-form solution to the problem is: W= ( K i=1 X T i X i ) -1 ( K i=1 X T i X i W i ).Inner-product matrices need to be computed before merging.• Task Arithmetic [30] defines task vectors as the difference between finetuned model weights and the pre-trained model weights. Suppose a model θ i is finetuned from a pre-trained model θ pre , the task vector is τ i = θ i -θ pre . When merging θ 1..K , the merged model isθ M = λ K i=1 τ i + θ pre ,where λ is the merging coefficient. • Ties-Merging [84] (Trim, Elect Sign & Merge) believes that the conflicts among the task vectors severely effect the merged model's performance. Ties-Merging solves this problem by eliminating redundant parameters and resolving symbol conflicts. • AdaMerging [85] uses an unsupervised method to learn the merging coefficients for each task vector (Task-wise AdaMerging) or each layer (Layer-wise AdaMerging). AdaMerg-ing++ is realized by adopting Ties-Merging [84] before learning the merging coefficients.• DARE [90] (Drop and Rescale) validates the extremely redundant properties of language models. As a pre-processing technique, DARE randomly drops most (90% or even 99%) delta parameters (task vectors) before merging to potentially mitigate the interference of parameters among models.We follow the settings in Section 4.1.1 and merge ViT-B/16 models. Tab. 11 shows the accuracy of merging ViT-B/16 models on eight vision tasks. The proposed EMR-MERGING brings about 5.6% performance improvement compared to Adamerging++ [85] , further demonstrating the effectiveness of EMR-MERGING.To further explore the performance of EMR-MERGING, we follow the settings in Section 4.1.1 and add one more task, ImageNet-1K [18] . We merge models on these nine tasks using different merging methods. The results are shown in Tab. 12 and EMR-Merging shows a much more significant improvement compared to existing merging methods (up to 20%).DARE's experimental results when combined with RegMean and Task Arithmetic are shown in Tab. 13. It can be seen that when applied to merge eight models, DARE works on a few tasks under low dropping rate settings but it generally fails. We attribute its failure to the random dropping strategy's unapplicability to merging multiple models. Under the setting of merging two or three models, randomly dropping most parameters in task vectors can significantly reduce interference but conflicts are a lot more difficult to avoid when merging multiple models. In Section 4.4, we showed partial results of merging different number of ViT-B/32 models by Fig. 6 .Here we provide quantified and task-specific performance results in Tab. 15.F Configuration of Fig. 4 and Fig. 7 In Fig. 4 and Fig. 7 , we hope to compare the sign conflicts, L2 distance, and cosine similarity of the merged model weights and individual model weights. To calculate the sign conflicts, we element-wisely compare the merged model weights to each individual model weights and record the ratio of the elements whose signs conflict. We report the average value of the sign conflicts between the merged model and each individual model. To calculate the L2 distance or cosine similarity, we first flatten the merged model weights and each individual model weights as 1-dimension vectors.Then we calculate the L2 distance or cosine similarity between the merged model and each individual model and report the average value.Despite the convincing results, the proposed method suffers from several limitations. On the one hand, compared to existing methods, EMR-MERGING requires a little additional memory to store the light-weight task-specific modulators. On the other hand, as a common limitation of task vector-based methods, EMR-MERGING cannot be generalized to models trained from-scratch because the task vector is based on the pretrain-finetune paradigm.Further Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: The paper introduces a new method for model merging, a currently hot research topic, which considers how to directly fuse multiple independently trained models at the weight level to obtain a single comprehensive model. The proposed approach addresses the significant performance degradation from the multi-task counterpart and requirement of additional data and training in existing methods.

Strengths of the paper and the proposed method include: 
- the paper is well-written.
- method does not require additional training, data, and tuning, which enhances the applicable scenarios of the method. This makes it practical for real-world applications where data availability is limited or privacy concerns restrict access.
- experiments are extensive, including VIT model, language models, and multi-modal models. In particular, the visual model is extended to 30 tasks.
- informative ablation studies are provided.
- the methods are computationally lightweight and straightforward to implement, yet they perform well
- The paper provides a solid theoretical foundation for the effectiveness of the proposed method, including detailed proofs and empirical analyses. This adds robustness to the claims made by the authors.

Regarding the weaknesses identified by the reviewers, the rebuttal and subsequent discussion effectively answered most of these, including the additional computational overhead during inference and comparison to parameter-efficient fine tuning methods.

The discussion about the applicability of the method to quantized networks was quite fruitful, and the proposed changes to the document based on this will add to the depth of the presentation of the method.

The rebuttal effectively addresses the novelty of the proposed method, showing that it is quite different than the methods indicated by the reviewer.

The issue of not being able to do multiple tasks in parallel is moot, since competing methods cannot do this either.

Reviews: review: paper_summary: paper_summary: This paper aims to improve the performance of model merging in the multi-task learning domain. By electing a unified model, identifying a mask and a rescaling factor for each task, the proposed method EMR-Merging is able to significantly improve the merging performance over the previous state-of-the-art, such as Adamerging, Ties-Merging, Task Arithmetic, etc. Besides the CV model merging, EMR-Merging also effectively merges the large language models, approaching the individual fine-tuned performance.
strength_weakness: strength_weakness: 1. I am curious why the proposed way works for the unified model. One specific case is that, if the the average of the task vector is close to zero, but not exactly equal to zero, e.g., 0.0001, which will give us a positive sign. If the magnitude of this weight is large, a positive sign multiplied by this large magnitude weight can lead to a large error. Can the authors elaborate more the intuition behind the unified model, mask, and rescaler? 

2. Can you explain the process of obtaining the mask? I could not find the definition of $\tau_F$ after reading the paper in detail.

3. What is the configuration of Figure 4? More details are needed here. 

4. How to interpret the t-SNE visualization?

5. The distance between figures is over-reduced. It can be confusing and misleading. For instance, figure 5 and figure 4.

6. How expensive to extend EMR-Merging to LLMs, such as LLaMa3?

7. How does the proposed method perform compared to SOTA https://arxiv.org/pdf/2402.00433?
questions: questions: Please see the above weaknesses.
limitations: limitations: Provided.
paper_summary: paper_summary: Model merging directly fuses multiple independently trained models at the weight level to obtain a single comprehensive model, which is a current research hotspot. This paper proposes a new model merging method EMR-MERGING to reduce the performance gap between the merged model and the independent models.
strength_weakness: strength_weakness: - **Loss of parallel ability**: Existing model merging methods obtain a unified model for all tasks, so samples from all tasks can be included in one inference (i.e., a batch of data). However, EMR-MERGING needs to configure an independent model for each task during inference, thus losing the ability of parallel inference. In real scenarios, a batch of data may come from different tasks rather than a single task.
- **Important baseline missing**: It is unclear what the benefits and advantages of EMR-MERGING are compared to "sparse independent models". Ties-Merging shows that removing 90% of the neurons in each independent model has almost no change in performance. So why not just use the sparse version of the task vector instead of using EMR-MERGING?
- **Symbol or spelling error**: 
  - The symbols are not uniform, and the uniform vector is sometimes $T_F$ and sometimes $T_{uni}$.
  - Table 11: “huper-parameter settings” -> “hyper-parameter settings”
  - Sec 4.2.2: “using 1eleven datasets” -> “using eleven datasets”

- Finally, there is a lack of datasets and checkpoints for CV (30 tasks), NLP, and multimodal in the anonymous link. If the authors can release them, it will help the community research.
questions: questions: - Is it possible to select only the maximum value in the Electing operation? Is it also possible to select the average value?
- In Figure 4(c), regarding Cos similarity, TV > Ties-Merging > AdaMerging, but in terms of accuracy, TV < Ties-Merging < AdaMerging. Figure 4(b) is similar. These phenomena seem to be contradictory. How does the author explain this?
limitations: limitations: The authors discuss some limitations of the method: for example, it consumes additional memory compared to other model merging methods, and all rely on the same pre-trained model. However, as in Weaknesses, this method may lose the ability to parallelize inference on multiple tasks, and it is unclear how much benefit it has compared to sparse independent models.
paper_summary: paper_summary: The authors identify two issues in current model merging methods: significant performance degradation from the multi-task counterpart and requirement of additional data and training. To tackle those issues, they propose EMR merging, which first creates a unified task vector, then selects masks based on each task-specific model’s correspondence with the unified task vector, finally rescales it. EMR merging shows strong empirical performance among several benchmarks.
strength_weakness: strength_weakness: 1. The novelty is limited. The three steps are essentially a combination of TIES and DARE, where only the masking step is slightly different from TIES. Due to this high similarity, it is unclear why the proposed method performs so much better than TIES and DARE. It would be better to provide a more disentangled analysis for each of the three steps through ablation studies, so that the readers can have a better understanding where exactly the performance improvement comes from.
2. Several related works are missing [1,2] (both on ArXiv in Feb 2024). It is important to do a thorough comparison, especially when the paper basically claims SoTA results.
3. Not all methods are compared on all benchmarks. For instance, AdaMerging is not compared with on NLP benchmarks. Although the original paper of AdaMerging does not provide results in NLP, I do not see any algorithmic obstacle that prevents it from being implemented in the NLP setting.

[1] Representation Surgery for Multi-Task Model Merging. Yang et al. ICML 2024.

[2] Merging Multi-Task Models via Weight-Ensembling Mixture of Experts. Tang et al. ICML 2024.
questions: questions: 1. What is the sparsity of the masks? I know that it is kind of task/model dependent, but it would be nice to show those results.
2. Do EMR still need the hyperparameter tuning on the scaling factors? Or the one from rescalder is final? Because in DARE, they tune the scaling factors even after they compute the rescaling factors.
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: The paper presents a method called EMR-MERGING (ELECT, MASK & RESCALE-MERGING) for merging models finetuned on different tasks into a single model with multi-task capabilities without the need for additional tuning or training. This method addresses the limitations of existing model merging techniques, which often suffer from performance degradation or require additional data and tuning.
strength_weakness: strength_weakness: - **Additional Computational Overhead:** The proposed method introduces extra computational overhead during inference due to the use of masks and rescalers. This aspect should be discussed in the paper, particularly regarding its impact on the overall efficiency and performance in real-world applications.
- **Storage Considerations for PEFT Models:** Pre-trained models finetuned on downstream tasks using Parameter-Efficient Finetuning (PEFT) methods typically require only the storage of the pre-trained model weights and additional adapter weights. These adapter weights consume significantly less storage space compared to the masks utilized in EMR-MERGING. The authors should address this comparison and discuss the storage implications in detail.
- **Impact of Model Quantization:** With the trend towards quantization of models, reducing their precision from 32-bit to 8-bit, 2-bit, or even 1-bit, the storage overhead of masks becomes increasingly significant. Quantization is an essential step in the development of large models, and as such, the relative storage cost of the masks in EMR-MERGING will be more pronounced in the future. The authors should consider discussing the impact of model quantization on their method and the potential challenges it poses.
- **Limited Dataset Size in Experiments:** The datasets used in the experiments are relatively small, which undermines the persuasive power of the results. For example, in the case of visual tasks, large models like CLIP's ViT-L exhibit strong generalization capabilities and can achieve high zero-shot classification performance on these small datasets. As a result, the performance differences among various model merging methods might not be significant. The authors should consider using larger, more diverse datasets to better demonstrate the effectiveness and robustness of their approach.
questions: questions: NA
limitations: limitations: NA

Comments: title: Thanks for your comments
Rebuttal by Authors
comment: We appreciate your comments, which can help improve our paper's quality greatly and inspire our future work. 

The discussion about the impact of model quantization and PEFT will be included in the revision. 

Thank you for your efforts. You are welcome to discuss with us if you have any other questions.
Thank you for your responses,  I appreciate the effort and detail you have provided in addressing the concerns raised.

I have no additional questions at this time and will determine my final score after conferring with the Area Chair and the other reviewers.
Thank you so much for your review and feedback. We further address your comments regarding Weakness 2 and Weakness 3 as follows:

Answer to Weakness 2: 

1) **Model finetuning and model merging are fundamentally different tasks with different targets**. For model fine-tuning, it consists of full-parameter fine-tuning that pursues fine-tuning performance and parameter-efficient fine-tuning (PEFT) that pursues fine-tuning efficiency, both of which have the same target of customizing models for specific domains, **necessitating labeled datasets and sufficient computational resources for supervised training**. For model merging, under the background that there exists an exponentialy increasing number of pre-trained or finetuned model weights, it targets to take advantage of these existing weights and obtain a single model with multi-task abilities, **without the need for labeled data or supervised training**. 

2) **Model merging can be combined with kinds of model fine-tuning methods**. More specifically, the proposed **EMR-Merging can be applied to not only fully finetuned models but also PEFT models**, as shown in Tab. 6 of our paper. We only need the fine-tuned adapters on each task, and EMR-Merging can merge these adapters into a single adapter with a few masks and rescalers, whose overhead is less than storing all the adapters while achieving performance close to that of multiple individual adapters.

3) **Model finetuning and model merging are both significant techniques**. In this paper, our EMR-Merging focuses on realizing tuning-free and high-performance model merging instead of reducing storage space requirements as much as possible like PEFT. Besides, **model merging and PEFT can be easily combined to achieve further parameter reduction**, which is a potential area for future work. For example, there have been some studies focusing on merging PEFT models [1,2]. 

Answer to Weakness 3:

1) When **compared to multiple individual 8-bit models**, EMR-Merging that consists of one 8-bit model and several masks and scalars can still significantly reduce the parameter numbers. 

2) When **compared to other merging methods**, EMR-Merging shows significantly better performance requring no tuning, demonstrating its greater applicability. 

3) **A potential solution to your concern is using PEFT techniques to train the quantilized models and merge the PEFT modules using EMR-Merging**. This can minimize the number of parameters on multiple tasks, which will be included in our future work.

----
**References**

[1] Parameter efficient multi-task model fusion with partial linearization, ICLR 2024.

[2] Composing Parameter-Efficient Modules with Arithmetic Operations, NeurIPS 2023.
The author seems to have misunderstood my points regarding Weaknesses 2 and 3. I would like to clarify them as follows:
- Weakness 2: A pre-trained model with N adapters can also perform multiple tasks. In this case, I don't quite understand why model fusion is necessary. What advantages does the proposed EMR-MERGING method have over the PEFT approach? PEFT allows for fine-tuning downstream tasks while reducing storage space requirements and achieving satisfactory results. Furthermore, PEFT consumes significantly less storage space compared to storing a mask.
- Weakness 3: In cases of highly quantized models (e.g., 8-bit), EMR-MERGING would occupy 1/8 of the original model’s size, which significantly limits the effectiveness of the method. In contrast, PEFT does not encounter this issue.