AdaMerging: Adaptive Model Merging for Multi-Task Learning
Authors: Enneng Yang
Zhenyi Wang
Li Shen
Shiwei Liu
Guibing Guo
Xingwei Wang
Dacheng Tao
Venue: ICLR 2024 poster (2024)
Keywords: Multi-task Learning
Model Editing
Task Arithmetic
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging) to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data.

Abstract: Multi-task learning (MTL) aims to empower a model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute MTL without necessitating a retraining process using the initial training data. Nevertheless, this direct addition of models often leads to a significant deterioration in the overall performance of the merged model. This decline occurs due to potential conflicts and intricate correlations among the multiple tasks. Consequently, the challenge emerges of how to merge pre-trained models more effectively without using their original training data. This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging). This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data. Specifically, our AdaMerging method operates as an automatic, unsupervised task arithmetic scheme. It leverages entropy minimization on unlabeled test samples from the multi-task setup as a surrogate objective function to iteratively refine the merging coefficients of the multiple models. Our experimental findings across eight tasks demonstrate the efficacy of the AdaMerging scheme we put forth. Compared to the current state-of-the-art (SOTA) task arithmetic merging scheme, AdaMerging showcases a remarkable 11\% improvement in performance. Notably, AdaMerging also exhibits superior generalization capabilities when applied to unseen downstream tasks. Furthermore, it displays a significantly enhanced robustness to data distribution shifts that may occur during the testing phase.

Published as a conference paper at ICLR 2024
ADAMERGING: ADAPTIVE MODEL MERGING FOR MUSTI-TASK LEARNING
Enneng Yang', Zhenyi Wang",, . Li Shen"' Shiwei
Shhiel Liu', Guibing Guo',, Xingwei Wang', Dacheng Tao 8
Northeastern University, China UUniversity of Maryland, USA SJD Explore Academy, China
*University of Oxford, UK "Nanyang Technological University, Singapore
ennengyangest wew.aaallee...mmmmm www..ee..e. .wd..eddu.ccc. zwangl 6 690umd. edu, mathshen1 1:gmma
shiwei. Luemathe ax.ac.uk, {guugb, > Bww..new. www.eeneeeedd.c. dacheng, Lao0gmail,
ABSTRACT
Multi-task learning (MTL) aims to empower model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute MTL without necessitating a retraining process using the initial training data. Nevertheless, this direct addition of models often leads sig- nificant deterioration in the overall performance of the merged model. This decline occurs due potential conflicts and intricate correlations among the multiple tasks. Consequently, the challenge emerges of how to merge pre-trained models more effectively without using their original training data. This paper introduces inno- vative technique called Adaptive Model Merging (AdaMe r rinn). This approach aims to autonomously learn the coefficients for model merging, either in task wisee or layer-wise manner, without relying on the original training data. Specifically, our AdaMerging method operates as an automatic, unsupervised task arithmetic scheme. It leverages entropy minimization on unlabeled test samples from the multi-task setup as a surrogate objective function to iteratively refine the merging coefficients of the multiple models. Our experimental findings across eight tasks demonstrate the efficacy of the AdaMerging scheme we put forth. Compared to the current state-of ofte--of-the-ar art task arithmetic merging scheme, AdaMerging showcases remarkable 1 11% improvement in performance. Notably, AdaMerging also exhibits superior generalization capabilities when applied to unseen downstream tasks. Fur- thermore, it displays significantly enhanced robustness to data distribution shifts that may occur during the testing phase. The code is available AdaMerging.
INTRODUCTION
Multi-task learning (MTL) is technique that enables the transfer of knowledge (Wu al., 2020; Wang et al., 2023; Jiang et al., 2024) among multiple tasks by efficiently sharing model parameters, leading to improvements in overall performance (Caruana, 1997; Liu al., 2019b; Vandenhende et al., 2021) across variety of tasks. Consequently, it has garnered significant attention in fields such as computer vision (Misra et al., 2016; Chen al., 2018: 2020), natural language processing (Collobert & Weston, 2008; Dong et al., 2015), and recommendation systems (Ma et al., 2018; Yang et al., 2023; Song et al.. 2024). In the context of foundation models, there are two key considerations. On the one hand, it is highly inefficient to pursue the traditional MTL approach for large pre-trained models by collecting a large volume of training data due to the high data labeling and computation cost. On the other hand, the advent of pre-trained models' popularity (Qiu et al., 2020) has led to a prevalent practice among downstream tasks. These tasks independently fine-tune the same pre-trained model, such ViT (Dosovitskiy et al., 2021) or BERT (Devlin al., 2019), and subsequently release these fine-tuned models, often without disclosing the specifics of their original training data. Consequently, there has emerged a recent trend in the research community, focused on exploring methodologies for effectively merging multiple independently trained models without relying on their training data for the purpose of MTL (Matena & Raffel, 2022; Jin et al., 2023; Ainsworth al., 2023; Ilharco et al., 2023: Huang et al., 2023; Ortiz-Jimenez et al., 2023; Yadav et al., 2023; Li et al., 2023).
"Corresponding author

---

Published as a conference paper at ICLR 2024
Recently, a novel concept in MTL known as task arithmetic has emerged (Ilharco et al., 2023). Task arithmetic introduces the notion of a "task vector", which can be described as a vector of weights fine-tuned specifically for given task, subtracted from the corresponding pre-trained weights (as illustrated in Fig. 2(a)). Essentially, a task vector serves as a unique representation for a particular task. Research in this area, focusing on methods centered around task vectors (Ilharco et al., 2023; Yadav et al., 2023), has demonstrated that by summing multiple task vectors and integrating them into pre- trained model, a new model can be created with the capability handle multi-task learning effectively (as depicted in Fig. 2(b)). However, despite the promising results, there still exists substantial performance gap between task vector- based MTL methods, such as Task Arithmetic (Ilharco et al., 2023) and Ties-Merging (Yadav et al., 2023), and traditional MTL approaches, as highlighted in Fig. 1. This disparity in performance suggests that further research and refinement are required to bridge the existing gap and unlock the full potential of task vector-based MTI. methodologies.
A critical observation in the analysis of task vector-based MTL methods is the significance of the merging coeffi- cient (denoted as in Fig. 2(b)) associated with the task vector. This coefficient plays a pivotal role in determin- ing the average accuracy of the final MTL model. As illustrated in Fig. 1, partieularly in the cases of Task Arith- metic (indicated by the yellow line) and Ties-Merging (represented by the blue line), an ill-suited merging coef- ficient can lead to a situation where the model struggles to effectively perform MTL. In such scenarios, the average accuracy across multiple tasks becomes unacceptably low. This sensitivity to the merging coefficient may stem from potential conflicts (Guo et al., 2020; Vandenhende et al., 2021) or intricate relationships (Ma et al., 2018; Standley et al., 2020) among the multiple tasks, which make the merging process highly susceptible to the choice of this coefficient. Consequently, one of the primary challenges encountered in task vector-based MTL lies in determining the appropriate task vector merging coefficients that facili- tate the optimal integration of multiple tasks, all without relying on the original training data for each task. Additionally, it is more desirable and flexible to fine-tune different coefficients for different layers within the merged model. However, when dealing with a substantial number of tasks and layers, traditional approaches such as grid search (Liashchynskyi Liashchynskyi. 2019) or combinatorial optimization search (Liu et al., 2020) become impractical for identifying suitable model merging coefficients. Hence, addressing this issue efficiently and effectively remains a challenging research problem in the field of task vector-based MTL.
Average Accuracy (96)
70
60
50
AdaMe (0urs)
Merpingnng
de
WaintArarage
Confficient A
Figure 1: The impact of coefficient on the average accuracy of various MTL methods on eight tasks. Among them, Task Arithmetic (Ilharco et al., 2023) and Ties-Merging (Yadav et al., 2023) based on task vectors achieved the best average accuracy when coefficient À = 0.3, which were 69.1% and 72.9% re- spectively. Traditional MTL and our AdaMerging are 88.9% and 80.1%.
In this paper, our inspiration comes from test-time adaptation schemes aimed at optimizing model generalization when faced with previously unseen test data (Wang et al., 2021; Liang et al., 2023). Building upon these concepts, we introduce an innovative automatic unsupervised multi-task model merging scheme. This scheme leverages the minimization of prediction distribution entropy unlabeled multi-task test data as a surrogate objective to adaptively learn model merging coefficients. The intuitive motivation of entropy minimization is to make the model produce a more deterministic output when faced with a given input, which can lead to a more robust and accurate model. Our approach begins with an analysis of the relationship between entropy and prediction loss across eight tasks. As depicted in Fig. 3(a), our observations reveal that samples with lower entropy also exhibit smaller prediction losses. Furthermore, we calculate the Spearman correlation coefficient (Myers & Sirois, 2004) to quantify the relationship between entropy and prediction loss, as illustrated in Fig. 3(b). The results affirm a positive correlation between entropy and prediction loss, confirming that entropy can serve as a suitable proxy objective for optimization purposes. Subsequently, we put forth two adaptive model merging schemes, collectively referred to as AdaMerg ng. These schemes are designed to automatically learn merging coefficient for each task vector each layer of each task vector, as depicted in Fig. 2(c) and (d). To update these merging coefficients, we employ entropy minimization as proxy objective, thereby enhancing the adaptability and performance of the multi-task model merging process.
2

---

Published as a conference paper at ICLR 2024
a) Task Vectors
b) Task Arithmetic
c) Task-wisc AdaMcrging
d) Layer-wise AdaMerging
0 0sm
Oxn
A
pr
0
Ppe
=0 ---
Bun. pp + A(T, +T,)
8un. Pp (A,T, +.T)))
6Am (8 16, +X 2
Figure 2: (a) Definition of "task vector", the task vector Tp is obtained by subtracting the pre-trained weights pre from the model weights 0g fine-tuned on the data of task k. (b) Task Arithmeric (Ilharco et al., 2023) for MTL, which assigns same merging coefficient À to each task vector T3 (k E {A,, B)). (c) Task-wise AdaMerging for MTL, which learns a distinct merging coefficient λg to each task vector T& (k E {A, B)). (d) Layer-ue worr-wwiee AdaMergin in for MTL, which learns a distinct merging coefficient X/. to each layer (1 E (1, of the task vector Ta (k E {A, B)).
Finally, we conduct comprehensive evaluation to ascertain the superiority of AdaMerging when compared to existing task vector-based methods, revealing its advantages in three key aspects: (i) Significantly Higher MTI. Performance: Our extensive testing across eight task vectors demonstrated that AdaMerging adaptive learning merging coefficient sigmificantly enhances the average accuracy across multiple tasks. For instance, the ViT-B/32, AdaMerging improved approximately 5.0% to 11.0% over Task Arithmetic and Ties-Merging. (ii) Substantially Improved Generalization: Our evaluation two sets of previously unseen downstream tasks underscored AdaMerging's 's superior generalization capabilities, resulting in improvements ranging from 4.4% to 9.1% when compared Task Arithmetic and Ties-Merging. (iii) Robust Test Data Distribution Shifts: In addition to performance gains, AdaMerging exhibited substantially enhanced robustness in multi-tan task testing across seven distribution drifts, with an average improvement of 8.45% compared to Task Arithmetic.
This paper makes four significant contributions: (i) We re-examine existing task based multi- task learning (MTL) methods and unveil the substantial influence of model merging coefficients on the average MTL performance. (ii) We introduce novel approach called AdaMerg r i ng, which autonomously learns merging coefficients in an unsupervised manner. This method can adaptively determine coefficients for different task vectors wwis AdaMerging) or individual layers within different task vectors (Layer-wiss sise AdaMerging), (iii) We establish a strong positive correlation between entropy minimization and loss minimization MTL's test data. This correlation signifies that these metrics can effectively serve as proxy objectives for optimizing the model merging coefficients within AdaMerg. (iv) We conduct comprehensive experiments to validate our method. The results demonstrate its substantial improvements in performance, generalization capabilities, and robustness compared state-of-the-art (SOTA) task vector-based model merging methods.
2 RELATED WORK
Joint Training for Multi-Task Learning. The joint training method gathers training data from multiple tasks to learn these tasks simultaneously (Caruana, 1997) to achieve knowledge transfer (Wu al., 2023). Existing works mainly focus on mitigating task conflicts from architecture (Misra et al., 2016; Sun et al., 2020) or optimization (Sener & Koltun, 2018; Liu et al., 2021) perspective. Architectural-basse methods mitigate task interference by sparsifying (Liu al., 2019a; Ding et al., 2021), branching (Lu et al., 2017; Guo et al., 2020) or modularizing (Ma et al., 2018; Hazimeh et al., methods balance multiple tasks from the perspectives
2021) shared structures. Optimization-base of task training weights (Sener& & Koltun, 2018 8; Liu et al., 2019a), gradient dominance (Chen et al., 2018; He et al., 2022; Yang et al., 2023), and gradient conflicts (Yu et al., 2020; Chen et al., 2020; Liu et al., 2021). However, the conventional approaches for collecting raw data across multiple tasks for joint training face challenges that may render them unsuitable in the era foundation models. This is primarily due to either (i) their computational inefficiency stemming from the high computation cost for updating the pre-trained models or (ii) numerous data owners refrain from releasing valuable or pristiv-sensitive raw data. Instead, they opt to share models fine-tuned on these pre- trained models.
Model Merging for Multi-task Learning. The practice of model merging has emerged as a promising solution enhance model generalization and facilitate MTL. The first type of research
3

---

Published as a conference paper at ICLR 2024
involves merging multiple models, all initially trained on the same task, with the aim of enhancing
the model's overall generalization (Gupta et al., 2020; Cha et al., 2021; Wortsman et al., 2022; Wang et al., 2022) to perform federated learning (Li et al., 2019; Wang al., 2020; Liu et al., 2022). The other type of work attempts to merge models for different tasks to perform MTL. (Matena & Raffel, 2022; Jin et al., 2023; Ainsworth al., 2023; Stoica al., 2023; Ortiz-Jimenez al., 2023; Zhang et al., 2023; Ilharco et al., 2023; Yadav et al., 2023). This paper primarily concentrates on the latter approach. However, simple model averaging alone can significantly deteriorate performance across multiple tasks. Consequently, in recent years, numerous advanced techniques have surfaced to mitigate the performance loss associated with model merging. For example, Fisher Merging (Matena & Raffel, 2022) employs the Fisher information matrix (Fisher, 1922) to measure the importance of individual model parameter. Subsequently, leverages this importance metric to guide the model merging. However, the computation of the Fisher information matrix becomes computationally and memory-intensive when dealing with a large number of model parameters. RegMean (Jin et al., 2023) suggests minimizing the L2 distance between the merged model and each individual model. However, this approach necessitates the precomputation and provision of the inner product matrix for the training dataset. This information may not be accessible if the model owner chooses not to disclose it. In recent developments, Task Arithmetic (Ilharco et al., 2023), introduces the concept of "task vectors". This approach demonstrates that merging task vectors to create a consolidated model can effectively facilitate MTL. Building upon this foundation, PEM Composition (Zhang et al., 2023) extends the task arithmetic framework to incorporate the merging of LoRA (Hu et al., 2021) models. Taking this a step further. Ties-Merging (Yadav et al., 2023) addresses task conflicts within the Task Arithmetic paradigm. accomplishes this by resetting redundant parameters, resolving sign conflicts, and exclusively merging parameters that exhibit sign-consistency Task vector-based studies overlook a critical challenge encountered when dealing with a diverse collection of models, i.e., the coefficients governing the model merging process play pivotal role in achieving optimal merging performance. In contrast, our work specifically emphasizes and addresses this issue to bridge the performance gap.
Overall, our work has three essential differences from existing task vector-based MTL schemes: (i) They share a merging coefficient across all task vectors, limiting the ficxibility of task vector combinations. By contrast, our method adopts different merging coefficients across different tasks or even different layers, substantially enhancing the flexibility of adaptations. (ii) Existing works employ grid-searching the merging coefficients, thus lacking guiding principle and is costly and infeasible when the number of tasks is large, while our work takes entropy minimization as a proxy objective to optimize the merging coefficients efficiently and automatically. (iii) We significantly improve multi-task performance. generalization to unseen tasks, and robustness to test data distribution shifts.
METHODOLOGY
We define the notation and model merging problem in Sec. 3.1, and briefly describe the solution based on task vectors. In S.c.3.2, we introduce the proposed AdaMerging method in detail.
3.1 PRELIMINARIES
Notation: Let fo(x;) -> Ds be a neural network model parameterized by a set of weights 0 ,e,,..,, ). which takes x, E Rd as an input data and outputs the predicted value di E RCC Among them, 0f is the weight of the I-th (1 E 21, ... LJ) layer, L represents the number of layers of the network f. d represents the dimension of the input data F,, and C' represents the number of classes. Without loss of generality, egg. ViT (Dosovitskiy et al., 2021) or BERT (Devlin et al., 2019)), are Opre = (Opres 0 pre:
assume that the weights of a well- pre- traitrainedd model, Obre)-
There are K tasks, and each of them has fine-tuned pre on their own private training data (z,, ys )) N represents the number of training samples for task k. Consequently, the model's 's weights after fine-tuning for task i are recorded as 0% = (0). . .
The model merging problem defined as how to combine weights (0x K to get a new weight 0MTL without necessitating retraining process using the initial task's 's training data, and ensure that fowrr. can perform tasks 1, 1. ... K simultaneously. A straightforward approach is to perform weight averaging, i.c., θ MTTI. = K Σk= 0g. however the performance of this approach usually drops dramatically (Ilharco al., 2023; Yadav al., 2023).
Problem Definition:
4

---

Published as a conference paper at ICLR 2024
Task Arithmetic: A recent research (Ilharco et al., 2023) defines the concept "task vectors" and completes various task arithmetic operations based on task vectors, such as adding multiple task vectors to the pre-trained weight Pre to perform MTL. Specifically, as shown in Fig. 2(a), the task vector T3 w.r. task & is defined as a vector obtained by performing a subtraction operation with the fine-tuned weights 0% and the pre-trained weights Opre. i.e., Tk 0x Oprc- Furthermore, multiple task vectors [Tk]k=1 are added and merged into the pre-trained model, fASTL 8pre X >Σ TR. where the coefficient À represents the importance of model merging. On this basis, Ties- Merging (Yaday et al., 2023) shows that some parameter values in the task vector may be redundant, or the signs of the parameters may conflict, and direct merging will cause performance losses. Based on this assumption, they proposed perform three steps of Trim, Elect Sign and Disjoint Merge on merging task vectors. We combine these steps and abbreviate them as one Φ() operation. Therefore, model merging in Ties-Merging can be expressed 0MTL Opre + A A R=1 (Ti).
Task arithmetic a simple and effective idea. As shown in Fig. 1, task vectors based MTL model merging methods, i.e., Task Arithmetic (blue line), Ties-Merging (yellow line), are significantly better than simple weight averaging scheme (pink line). However, there is still a clear gap between them and the traditional MTL (black line). In addition, task vector- model merging methods are very sensitive to the merging coefficient A. An ill-suited À will cause the performance to be lower than the weighted average,
reach unacceptably low accuracy. When the number of tasks is large, grid searching the merging coefficients for each task vector is expensive. This motivates us to conduct further research narrow the performance gap between task vector-based MTL and traditional MTL.
3.2 ADAPTIVE MODEL MERGING FOR MULTI-TASK LEARNING
In this section, we propose an unsupervised adaptive model merging method for task vectors based MTL, called AdaMerging. It makes the merging coefficient of each task vector learnable (Task-wiss) se AdaMerging ig). Furthermore, different layers task vector can also automatically learn different merging coefficients in AdaMerging (Layer-wiss AdaMerging).
3.2.1 ADAMERGING: ADAPTIVE MODEL MERGING
Task-wise AdaMerging: As shown in Fig. 2(c), our Task-wise AdaMerging assigns a separate merging coefficient λ to each task vector Tk, that is: OUTT = Ppr + Σk= TE. -isk-wise AdaMerging allows task vectors T3 that have a positive transfer to the average MTL performance to occupy a larger proportion in 0 MTL. while task vector Tix that is harmful to MTL will reduce their contribution to the merging weight 0 MTL. thereby improving the average MTL performance.
Layer-wise AdaMergigg: However, Task-wise AdaMerging may not be enough to alleviate the interference of task vectors. In the deep neural network model, the information learned by each layer is different. For example, the lower layer may learn general features, while the higher layers may learn task -ssecipec features (Yosinski et al., 2014). Therefore, when merging task vectors, the TT T2..... T of different layers for each task vector Tę should also have different weights (Tt,
contributions {Ak, Xg..... A } to the final multi-tare task weights fMTL- Based on this, we propose the Layer-wise AdaMerging scheme shown in Fig. 2(d), which is formalized as: 0MTL (0'TT))
(0) + + 1 X.TT) where L represents the number of layers.
AdaMerging++: The above AdaMerging adaptively merges the original task vector T3 in the Task Arithmetic (Ilharco et al., 2023). Naturally, can also adaptively merge the task vector ((Tx) after removing parameter redundant values and sign conflicts in Ties-Merging (Yadav et al., 2023).
We call this variant AdaMerging++, and the corresponding Tisk-wise AdaMerging++ and Layer-
wise AdaMerging++ versions are formalized as dArTI = 6pre + EE_ (Tx) and BMTTI =
{0 ( = =(0 (or + ( respectivell
Now, AtaMergigg/AdaMerrinnttt faces critical challenge, that we only have the task vector of each task without their initial training data. How to optimize merging coefficients {Ax 1 (or (1)) K.L.
) Our inspiration to solve this challenge comes from test-time adaptation (Wang et al.,
5

---

Published as a conference paper at ICLR 2024
Spearman Correlation Coefiiiett
1.00
2.0
Nreage Loss
0.0
0.00
2.5
0.6
1177
1.0
0.4
0:00
0.5
0.54
0.
[00
(010d
(220g
(n50d)
(060.))
(azad
(eang
D10
Coms
Resucss
Eunsser
Sinee
arsra
AlL
Emppy
Figure 3: Correlation of entroyy HY) and avareage loss L(Y, Y) on eight tasks (or datasets). (a) We divided the test samples into eleven groups according to the entropy of the samples, and observed the average prediction loss of the samples in each group. We observe that groups with smaller entropy correspond to smaller average losses. (b) We calculated the Spearman correlation coefficient between entropy and prediction loss on eight tasks (or datasets) and observed a high positive correlation.
2021; Niu et al., 2022; 2023), they adapt the weights of the trained model on unseen test data cope with the distribution shifts on the test data.
3.2.2 ENTROPY OPTIMIZATION
We use entropy minimization on multi-task unlabeled test samples as an optimization surrogate K.L objective function update the merging coefficients { (or (A1 in our AdaMerging.
Entropy Minimization: For sample Hs, the predicted output of a neural network fa( is 9t, the corresponding Shannon entropy (Shannon, I 1948) is expressed as ( = Σ. p (Di.e) log (fr.c). where p(it..) E [0, 1] represents the probability that the input x; is predicted be the e-th class. Previous research on test-time adaptation (Wang et al., 2021; Niu et al., 2023) found that optimizing the model's parameters based entropy minimization (Grandvalet & Bengio, 2004; Roy al., 2022), min H (0:), ), on test samples can make the model adapt to unseen test data distributions.
However, it is unclear whether entropy minimization can be used as an effective surrogate objective function in multi-task model merging. To verify whether entropy minimization can be used as a proxy objective for MTL loss, we performed the analysis on the eight tasks used in the experiment. First, we 1K [( J Jkmi 1, and compute the prediction of N 1K the multi-task model fesTr test data as Ý (Lfowre (a:)) jkm Next, we calculate the loss between the real label Y and the predicted value Ý case-by-case and obtain Ý) {f(vs, ū (o){.% where | represents the total number of test samples for all tasks, and represents a loss function. such as cross-entropy. We also calculate the entropy of each sample on the test set and get / H²)
combine the test data of the eight tasks as (X,
(H \ Finally, we analyze the correlation between entropy HYY) and prediction loss L(Y, Y) from two aspects. (i) We divide the multi-task samples into multiple intervals based on entropy H()) from small to large, such as I {I,, I2,...... } = {(0.0, 0.1]. (0.1 0.2). ........ (1.0, x)) and count the average prediction loss of the samples contained in each interval Ie (f E {1, (1.2.... 11)). As shown in Fig. 3(a), we observe that the average loss corresponding to the interval with small entropy is also smaller. (ii) We also directly calculated the Spearman correlation coefficient (Myers & Sirois, 2004) of entropy HÝ)) and prediction loss L.YY, ÿ). As shown in Fig. 3, we observe that the average correlation between the two on multi-task data (i.e., dark purple "ALL") is as high as 0.87. Therefore, we can conclude that entropy minimization (i.e., min HY))) can serve as an effective surrogate objective for loss minimization (i.e., min L(Y, Y)) on MTL. In Fig. 10 of the Appendix, we further verify that this correlation exists across different training stages of model merging.
Optimization Objective: Based on the above verification, we take entropy minimization as the optimization proxy goal of the model merging coefficient in our AdaMerging/AdaMergnnttt For example, the optimization form of the merging coefficient in Task-wise AdaMerging is:
K
min
Σ
Σ
(fferrr (I+)) where OMTL 0pre~ +
B.

---

Published as a conference paper at ICLR 2024
where B& represents batch of unlabeled test samples sampled in task k. The coefficient {Ax can be updated iteratively by obtaining the gradient through backpropagation. This is trivial with automatic differentiation tools like Pytorch (Paszke et al., 2017). It should be emphasized that, on the one hand, we do not need all test data to be available. Even if only 0.1% or 1% of unlaheled tests are available, method can have significant performance improvements. On the other hand, our extra training time is also very cheap. These results are presented in the appendix.
4 EXPERIMENT
In this section, we introduce the experimental setup in Sec. 4.1 and the experimental results in Sec, 4.2. Due to page limitations, some details and results shown in the Appendix.
4.1 EXPERIMENT SETUP
Datasets and Models: Following Ilharco et al. (2023) and Yadav et al. (2023), we study task vectors based multi-task model merging on eight image classification datasets: SUN397 (Xiao et al., 2016), Cars (Krause et al., 2013), RESISC45 (Cheng et al., 2017), EuroSAT (Helber et al., 2019), SVHN (Yuval, 2011), GTSRB (Stallkamp et al., 2011), 1), MNIST (LeCun, 1998), DTD (Cimpoi et al., 2014). We provide more detailed description of the dataset in the Appendix A. In the main text, we use the Vit-B/32 and ViT-L/14 architectures in CLIP (Radford et al., 2021) as pre-trained models to conduct experiments. We also report the results on the Vit-B/16 architecture in the Appendix B.
Baselines and Metric: Our baselines are mainly divided into two categories, one is non-model merging, i.e., Individual and Traditional MTL; and the other is various advanced model merging methods, such as Weight Averaging, Fisher Merging (Matena & Raffel, 2022), RegMean (Jin et al., 2023), Task Arithmetic (Ilharco et al., 2023) and Ties-Merging (Yadav et al., 2023). Baseline details are provided in Appendix A. Among them, Task Arithmetic and Ties-Merging are task vectors based MTL methods, which are also our most important baselines. In addition, our methods include Task-wise AdaMerging, Task-wise AdaMerging+++ Layer-wise AdaMerging, and Layer-wise AdaMerging++. Unless otherwise specified, our method uses the Layer-wise version. We report the average accuracy (i.e., Avg Acc) of MTL model on the test set of all tasks as an evaluation metric.
4.2 PERFORMANCE, GENERALIZATION, ROBUSTNESS
In this section, we demonstrate the superiority our approach over SOTA methods for merging task vectors by evaluating it from three key perspectives: performance, generalization and robustness.
Significantly Higher MTL Performance. We verify that the proposed AdaMerging method sig- nificantly outperforms existing model merging methods in performance. As shown in Tab. I and Tab. 2, we tested the performance of merging ViT-B/32 and ViT-L/14 on eight tasks, respectively, We have the following observations: (i) Individual and Traditional MTL methods achieve the optimal performance, which are 90.5% and 88.9% under V/T-B/32. However, they all rely on initial training data for multiple tasks. Additionally, independent fine-tuning requires storing model for each task. (ii) Weight Averaging is the simplest model merging solution. Naturally, its performance is also the lowest. Furthermore, Fisher Merging merged models by calculating parameter importance, and RegMean imposed the constraint that the distance between the merged MTL model and a single model close. Both of them perform better compared to the Weight Averaging. (iii) Advanced task vectors
Table 1: Multi-task performance when merging ViT-B/32 models on eight tasks.
Method
SUN397
Cars
RESISC45
EumSAT
SVHN
GTSRB
MNIST
DTD
Avg Ace
Pretrained
62.3
59.7
60.7
45.5
31.4
32.6
48.5
43,8
48.0
Individual
75.3
77.7
96.1
99.7
97.5
98.7
99.7
79.4
90.5
Traditional MTL
73.9
74.4
93.9
98.2
95.8
98.9
99,5
77.9
88.9
Weight Averaging Fisher Merging (Matesa & Raffel, 2022)
65.3
63.4
71.4
71.7
64.2
52.8
87.5
50.1
65.8
68.6
69.2
70.7
66.4
72.9
51.1
87.9
59.9
68.3
RegMean (Jin et al., 2023)
65.3
63.5
75.6
78.6
78.1
67.4
93.7
52.0
71.8
Tisk Arithmetic (Ilharco et 2023)
55.2
54.9
66.7
78.9
80.2
69.7
97.3
50.4
69.1
Ties Merging (Yadav et al., 2023)
59.8
58.6
70.7
79.7
86.2
72.1
98.3
54.2
72.4
Tisk-wise AdaMerging (Ours)
58.0
68.8
85.7
81.1
84.4
92.4
Task-wise AdaMerging*+ (Qurs)
60.8
$6.9
73.1
83.4
87.3
82.4
95.7
50.1
73.7
Layer-wise AdaMerging (Ours)
64.5
68.1
79.2
93.8
87.0
91.9
97.5
59.1
80.1
Layer-wise AdaMerging++ (Ours)
66.6
68.3
82.2
94.2
89.6
89.0
98.3
60.6
81.1
7

---

Published as a conference paper at ICLR 2024
Table 2: MMlti--ask performance when merging ViT-L/14 models on eight tasks.
Method
SUN397
Cars
RESISCA5
EuroAAT
SVHN
GTSRB
MNIST
DTD
Avg Acc
Pretmined
66.8
71.0
59.9
58.4
50.5
76.3
Individual
82.1
92.4
97.4
100
98.1
99.2
99.7
84.1
94,2
Traditional MTL
80.8
96.3
96.3
97.6
99.1
99.6
84.4
Weight Averaging
72.1
81.6
82.6
91.9
78.2
70.7
97.1
62.8
79.6
Fisher Merging (Malena Raffel, 2022)
69.2
87.5
93.5
80.6
74.8
93.3
82.2
RegMean (Jin et al,, 2023)
73,3
81.8
86.1
97.0
88.0
84.2
98.5
60.8
83.7
Arithmetic (Ilharco et al., 2023)
73.9
86.6
94.1
87.9
86.7
98,9
Ties-Merging (Yadav et al., 2023)
76.5
85.0
89.3
95.7
90.3
83,3
99.0
68.8
86.0
AdaMerging (Ours)
79.0
90.3
90.8
96.2
93.4
98.0
99.0
79.9
90.8
AdaMerging++ (Ours)
79.4
90.3
91.6
97.4
93.4
97.5
99.0
79.2
91.0
Table 3: Generalization results on two unseen tasks when merging ViT-B/32 models on six tasks.
Seen Tasks
Unseen Tasks
SUN397
RESISC45
SVHN
GTSRB
Avg Acc
MNIST
EuroSAT
Avg Ace
Task Arithmetic (llharco al., 2023)
63.3
62.4
75.1
57.8
84.6
50.4
70.6
77.2
46.2
61.7
Ties Merging (Yadav et al., 2023)
67.8
66.2
77.2
56.7
77.1
70.9
69.3
75.9
43.3
59.6
AdaMerging (Ouns)
65.2
65.9
88.5
61.1
92.2
91.5
77.4
81.0
56.1
70.0
AdaMerging++ (Ours)
86.3
89.8
68.7
Method
SUN397
Cars
GTSRB
EuroSAT
DTD
MNIST
Avg Acc
RESISCAS
SVHN
Arg Ace
Task Ariifinetic (illurco el., 2023)
75.2
95.7
52.3
51.1
Ties Tees-Merging (Yaday et al., 2023)
68.0
67.1
67.7
78.4
56.5
92.8
71.8
58.7
49.2
53.9
AdaMerging (Ouns)
67.1
67.8
94.8
94.4
59.6
98.2
80.3
50.2
60.9
55.5
ArgaMngg (Ours)
68.9
69.6
91.6
94,3
61.9
98.7
80.8
$2.0
64.9
58.5
based multi-task merging methods (i.e., Task Arithmetic and Ties-Merging) have achieved good performance. For example, Ties-Merging has achieved the performance in ViT-B/32 and ViT-L/14 by 72.4% and 86.0%, respectively. However, there is still a big gap between this and Traditional MTL (i.e., 88.9% and 93.5%, respectively). (iv) Our Task wise AdaMerging and Task-wise AdaMerging++ use unsupervised learnable coefficients to merge task vectors in Task Arithmetic and Ties-Merging respectively, bringing 2% and 1.3% performance improvements respectively ViT-B/32. Thanks to the more fine-grained fusion solution, on ViT-B/32, our Layer-wise AdaMerging and Layer-wise AdaMerging++ bring 11% and 8.7% performance improvements compared Task Arithmetic and Ties-Merging, while on ViT-L/14, our method brought improvements of 6.3% and 5.0%. Our AdaMerging greatly reduces the gap between model merging and traditional MTL solutions.
Substantially Improved Generalization. MTL hopes to transfer the knowledge of old tasks to new tasks and improve the generalization of the MTL model. To this end, we compare the performance of Ada and task rector-based model merging methods (Task Arithmetic and Ties-Merging) on two sets of unseen tasks. Tab. 3, we merge the task vectors corresponding to six tasks and test on two unseen tasks (i.e. their task vectors are not merged). We observe: (i) On the six seen tasks, AdaMerging and AdaMerging++ are significantly better than Task Arithmetic and Ties-Merging. (ii) More importantly, AdaMerging method maintains this superiority on two unseen tasks. For example, on the two tasks of MNIST and EuroSAT, the average performance of AdaMerging and AdaMerging++ improved by 8.3% and 9.1%, respectively, compared with Task Arithmetic and Meri-ng In addition, on the two unseen tasks of RESISC45 and SVHN, the average accuracy improvements of AdaMerging and AdaMerging++ are 4.4% and 5.4%, respectively. These results indicate that our AdaMerging and AdaMerging++ methods
methods generalize better to unseen tasks.
Robust to Test Data Distribution Shifts. Considering that the model provider only releases the fine-tuned model and does not expose the original training data, the model merger's ' test data may differ from the model owner's training data. we tested whether AdaMerging still effective when the test data distribution shifts significantly. Following Hendrycks & Dietterich (2019), we created 7 corruption test data, and examples of corrupted images are shown Fig. in Appendix The results on VIT-B/32 are shown in Tab. 4. On clean test data, AdaMerging has an 8.2% performance improvement compared to Task Arithmetic. On the corruption test datasets of Motion Blur, Impulse Noise, Gaussian Noise, Pixelate. Spatter, Contrast and JPEG Compression. AdaMergin''s performance is 11.2%, 6.7%, 5.8%, 8.9%, 6.7%. 10.1% and 9.8% higher than Task Arithmetic respectively. These evidences fully demonstrate that AdaMerging is more robust to test data distribution shifts.
Summary: Our AdaMerging/ Saaarrginn//aMMMgginnneee allows us to adapt to unlabeled test data of task vectors, unlabeled test data of unseen tasks, unlabeled corruption data in unsupervised way when training model merging coefficients, thereby optimizing the best suitable model merging coefficients are used to obtain model with better performance, generalization robustness.
8

---

Published as a conference paper at ICLR 2024
Table 4: Robustness results when merging ViT-B/32 models on four tasks.
Clean Test Set
Corruption Test Set (Motion Blur
Method
EuroSAT
RESISC45
GTSRB
Avg
EuroSAT RESISC4S
GTSRB
Aug
Task Arithmetic
66.9
94.7
82.6
75.1
79.8
65.3
68.1
80.0
64.2
69.4
AdaMerging (Ours)
73.7
96.1
85.8
96.3
88.0
71.2
74.6
82.7
94.1
80.6
Corruption Test Set (Impulse Noise)
Corruption Test Set (Gaussian Noise)
Method
EuroSAT RESISC4S
GTSRB
Avg
Cars
EuroSAT RESISC45
Arg
Task Arithmetic
62.1
49.1
72.7
40.4
56.1
63.6
55.4
75.9
49.4
61.1
AdaMerging (Ours)
67.2
30.8
75.9
77.5
62.8
69.9
41.2
80.6
76.0
66.9
Coruuttion Test Set (Pixelate)
Comuption Test Set (Spatter)
Method
Cars
EuroSAT RESISC45
GTSRB
Avg Ace
Cars
EuroSAT RESISC45
GTSRB
Arg Ace
Task Arithmetic
2.78
41.5
22.8
66.6
33.4
63.3
60.1
73.9
54.3
62.9
AdaMerging (Ours)
2.49
53.8
22.4
90.6
42.3
69.9
43.6
75.4
89.4
69.6
Computinn Test Set (Contrast)
Corruption Test Set (JPEG Compression)
Method
Cars
EuroSAT RESISC45
GTSRB
Avg Ace
Cars
EuroSAT
RESISC45
GTSRB
Arg Ace
Task Arithmetic
66.0
62.9
75,9
70.6
68.9
66.5
72.3
60.0
AdaMerging (Ours)
71.7
69.8
79.3
95.1
79.0
70.9
75.8
83.6
90.1
80.1
4.3
ADAMERGING ANALYSIS
Task-wise Coefficients. In Tab. 5, we consistently observe that the merging coefficients of each task vector are inconsistent. When the number of tasks relatively large, obviously undesirable to grid search the coefficients of each task, but our AdaMerging avoids this manual search process.
Table 5: Model merging coefficients (Ax ) change with respect training steps ViT-B/32.
Method
SUN397
Cars
RESISC45
EuroSAT
SVHN
GTSRB
MNIST
DTD
Task-wise AdaMerging
0.2202
0.1413
0.2826
0.3284
0.2841
0.4003
0.1978
0612
Task-wise +daMerging++
0.3171
0.1698
0.4235
0.5198
0.4386
0.5803
0.2452
0.2885
Layer-wise Coeflicients. Fig. 4 shows the merging coefficients learned by Layer-wise AdaMerging and AdaMerging++ on ViT-B/32 respectively. We observed that: (i) The coefficients learned by each layer of each task vector are different, which shows that the importance of each layer in the model merging process different. (ii) The coefficients learned by shallow layers are generally smaller than those of deep layers, which indicates that shallow layers rely more on the weights of the pre-trained model rather than the weights provided by task vectors, while the deep layers rely more on the weights provided by the task vectors. This may be since the shallow layer learns general features,
which are cross cross-task, task, while the deep layer learns task-specific features (Yosinski et al., 2014),
SUN397
1.00
Cars
RESISC45
0.75
EuroSAT
SVHN
0.50
GTSRB
MNIST
0.25
DTD
12
16-
20
24
28
32
36
90
24
48
52
56
90
61
68
72-
76
80
88
88
92
96
100
104
109
112
116-
120
124
128
132
136
140
144
148
152.
156
0.00
SUN397
RESISC45
0.75
EuroSAT
SVHN
0.50
GTSR6
MNIST
0.00
16
20
24
28
36
44
48
52
56
64
72
80
" 88
92
100
104
Layer
108
112
116
120
124
128
152
136
140
144
152
156
Figure 4: Learned model merging coefficients {X
Layer-wise
AdaMerging (Above) and
AdaMerging++ (Below) on ViT-B/32. The k-th row represents the k-th task vector, the 1-th column
represents the (-th layer, and the intersection point represents the coefficient X/--
5 CONCLUSION AND FUTURE WORK
Advanced task arithmetic shows that new models built by merging multiple task vectors into a pre- trained model can execute MTL without needing original training data. However, task vector- rector-based MTL methods are very sensitive to the merging coefficient. In this paper, we propose an adaptive model merging scheme (abbreviated as AdaMerging) solve this problem, which takes entropy minimization as a surrogate objective to automatically learn the merging coefficients for each task vector layer. Experimental results show that the proposed AdaMerging is superior to the current SOTA model merging methods in multi-task performance, generalization and robustness. In the future, plan to further explore model merging solutions for different architectures.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
Li Shen is supported by STI 2030-Major Projects (No. 00212D020140S) Enneng Yang and Guibing Guo are supported by the National Natural Science Foundation of China under Grant No. 62032013, the Science and technology projects in Liaoning Province (No. 2023JH3//02000055) and the Fundamental Research Funds for the Central Universities under Grant No. N2317002.
REFERENCES
Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. In ICLR, 2023.
Rich Caruana. Multitask learning, Machine learning, 28:41-75, 1997.
Junbum Cha, Sanghyuk Chun, Kyungjac Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lec, and Sungrae Park. Swad: Domain generalization by sceking flat minima. NeurIPS, 34:22405-22418, 2021.
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In ICML, 794-803.
PMLR, 2018.
Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. In NeurlPS, 2020.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10) 1883,, 2017.
Mircea Cimpoi, Subhransu Maji. lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In CVPR, pp. 3606-3613, 2014.
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML., pp. 160-167, 2008
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT 2019, pp. 4171-4186. Association for Computational Linguistics, 2019.
Ke Ding, Xin Dong, Yong He, Lei Cheng, Chilin Fu, Zhaoxin Huan, Hai Li, Tan Yan, Liang Zhang, Xiaolu Zhang, et al. Mssm: multiple-level sparse sharing model for efficient multi-task learning. In SIGIR, pp. 2237-2241, 2021.
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. Multi-task learning for multiple language translation. In ACL. pp. 1723-1732, 2015.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Ronald A Fisher. On the mathematical foundations of theoretical statistics. Philosophical transactions of the Royal Society of London. Series A, containing papers of mathematical or physical character, 222(594-604)3309--66 1922.
Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. NeurIPS, 17, 2004.
Pengsheng Guo, Chen- Yu Lee, and Daniel Ulbricht. Learning to branch for multi-task learning, In ICML, pp. 3864-3863, PMLR, 2020.
Vipul Gupta, Santiago Akle Serrano, and Dennis DeCoste. Stochastic weight averaging in parallel: Large-batch training that generalizes well. In ICLR. OpenReview.net, 2020.
10

---

Published as a conference paper at ICLR 2024
Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery, Maheswaran Sathiamoorthy, Yihua Chen, Rahul Mazumder, Lichan Hong. and Ed Chi. Dselect-k: Differentiable selection in the mixture of experts with applications to multi-task learning. NeurlPS,
2021.
Yun He, Xue Feng, Cheng Cheng, Geng Ji, Yunsong Guo, and James Caverlee. Metabalance: Improving multi-task recommendations via adapting gradient magnitudes of auxiliary tasks. WWW, pp. 2205-2215, 2022.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2266, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. ICLR, 2019.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 2021.
Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprini arXiv:330.13266, 2023.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In ICLR, 2023.
Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Dapeng Liu, Jie Jiang, and Mingsheng Long. Forkmerge: Mitigating negative transfer in auxiliary-tase learning. Advances in Neural Information Processing Systems, 36, 2024.
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro. and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. ICLR, 2023.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv::442.0 6980, 2014.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV workshops, pp. 554-561, 2013.
Yann LeCun. The mnist database of handwritten digits. http://yaan. lecun. com/exdb/mmist// 1998.
Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu, and Li Shen. Deep model fusion: A survey, arXiv preprint arXiv:2309. 15698, 2023.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In ICLR, 2019.
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribu- tion shifts. arXiv preprint arXiv:2303. /5361, 2023.
Petro Liashchynskyi and Pavlo Liashchynskyi. Grid search, random search, genetic algorithm: big comparison for nas. arXiv preprint 192iv:19/..00059 2019.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. NeurIPS, 34: 11111888781889 2021.
Chang Liu, Chenfei Lou, Runzhong Wang, Alan Yuhan Xi, Li Shen, and Junchi Yan. Deep neural network fusion via graph matching with applications to model ensemble and federated learning. In ICML, pp. 13857-13869, PMLR, 2022.
Jialin Liu, Antoine Moreau, Mike Preuss, Jeremy Rapin, Baptiste Rozicre, Fabicn Teytaud, and Olivier Teytaud. Versatile black-box optimization. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 620-628, 2020.
Shikun Liu, Edward Johns, and Andrew Davison. End-to-end multi-tast task learning with attention. In 1880. Computer Vision Foundation / IEFE, 2019 9a..
CVPR, pp. 1871 11780.
11

---

Published as a conference paper at ICLR 2024
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. Multi-task deep neural networks for natural language understanding. In Anna Korhonen, David R. Traum, and Lluís Märquez (eds.), ACL, pp. 4487-4496. Association for Computational Linguistics, 2019b.
Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully- adaptive feature sharing in multi-task networks with applications in person attribute classification. In CVPR, pp.
2017.
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. Modeling task relationships in multi-task learning with multi-gate mixture- of--eeerss.. In SIGKDD, pp. 1930-1939. ACM, 2018.
Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. NeurlPS, 77033-6117 2022.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In CVPR, pp. 3994-4003. IEEE Computer Society, 2016.
Leann Myers and Maria Sirois. Spearman correlation coefficients, differences between. Encyclope- dia of statistical sciences, 12, 2004.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In ICML, pp. 16888-16905. PMLR, 2022.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In ICLR, 2023.
Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. In NeurlPS, 2023.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10): 1872-1897, 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICMI., pp. 8748-8763. PMLR, 2021.
Subhankar Roy, Martin Trapp, Andrea Pilzer, Juho Kannala, Nicu Sebe, Elisa Ricci, and Amo Solin. Uncertainty-guie source-free domain adaptation. In ECCV, pp. 537-555. Springer, 2022.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In NewrlPS, pp- 525-536, 2018.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379 (733):379-223 423, 1948.
Derun Song, Enneng Yang, Guibing Guo, Li Shen, Linying Jiang, and Xingwei Wang. Multi-scenario and multi-task aware feature interaction for recommendation system. TKDD, 2024.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: multi-class classification competition. In IJCNN, pp. 1453-1460. IEEE, 2011.
Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In ICML, pp. 9120-9132. PMLR, 2020.
George Stoica, Daniel Bolya, Jakob Bjorner, Taylor Hear, and Judy Hoffman. Zipit! merging models from different tasks without training. arXiv preprint arXiv:2305.nm 30500030 2023.
12

---

Published as a conference paper at ICLR 2024
Ximeng Sun, Rameswar Panda, Rogerio Feris, and Kate Saenko. Adashare: Learning what share for efficient deep multi-task learning. NeurIPS, 33:8728 1187788-7400, 2020.
Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool. Multi-task learning for dense prediction tasks: A survey. TPAMI, 44(7): 3614-3633, 2021.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021.
Hongyi Wang, Mikhail Yurochkin, Yuckai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. In ICLR, 2020.
Zhenyi Wang, Xiaoyang Wang, Li Shen, Qiuling Suo, Kaiqiang Song, Dong Yu, Yan Shen, and Mingchen Gao. Meta-lcarning without data via wasserstein distributionally-roooe model fusion. In UAL, pp. 2045-2055. PMLR, 2022.
Zhenyi Wang, Li Shen, Tiehang Duan, Qiuling Suo, Le Fang, Wei Liu, and Mingchen Gao. Distri- butionally robust memory evolution with generalized divergence for continual learning. TPAMI, 2023.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-opess Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korbliihh, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In ICMI., pp. 23965-23998. PMLR, 2022.
Chengyue Wu, Teng Wang, Yixiao Ge, Zeyu Lu, Ruisong Zhou, Ying Shan, and Ping Luo. pi-tuning: Transferring multimodal foundation models with optimal multi-task interpolation. In ICML, pp. 37713-37727. PMLR, 2023.
Sen Wu, Hongyang R. Zhang, and Christopher Ré. Understanding and improving information transfer in multi-task learning. In ICLR. OpenReview.net, 2020.
Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva, Sun database: Exploring a large collection of scene categories. IJCV, 119:3-22, 2016.
Prateck Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Resolving interference when merging models. NeurlPS, 2023.
Enneng Yang, Junwei Pan, Ximei Wang, Haibin Yu, Li Shen, Xihua Chen, Lei Xiao, Jie Jiang, and Guibing Guo. Adatask: A task-aware adaptive learning rate approach to multi-task learning. In AAAI, volume 37, pp. 10745-10753, 2023.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? NeurlPS, 27, 2014.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.
Gradient surgery for multi-task task learning. NeurIPS, 33:5824-5888 2020.
Netzer Yuval. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. Composing parameter-efficien modules with arithmetic operations. arXiv preprint arXiv:2306. 14870, 2023.
13

---

Published as a conference paper at ICLR 2024
A EXPERIMENT SETTINGS
This section provides detailed dataset description, baseline description, and training details.
Dataset Details. Following Task Arithmetic (Ilharco al., 2023), Ties-Merging (Yadav et al., 2023), we study multi-task model merging on eight image classification datasets below.
SUN397 (Xiao et al., 2016) a scene classification dataset, which contains images in 397 classes, with a total of 108,75 754 images, and each class has at least 100 images.
Stanford Cars (Cars) (Krause et al., 2013) is a car classification dataset, which contains 196 classes of cars and total of 16,185 images. Each class in the training set and test set is divided at a ratio of 1:1.
RESISC45 (Cheng et al., 2017) is remote sensing image scene classification data set. It contains 45 classes of scenes and a total of 31,500 images, of which there are approximately 700 images in each class.
EuroSAT (Helber et al., 2019) is satellite image classification dataset containing 27,000 labeled and geo-referenced images in 10 classes.
SVHN (Yuval, 201 1) is a real-world digital classification data set extracted from house numbers in Google Street View images. There are 10 classes in total. The training set contains 73,257 samples, the test set contains 26,032 samples, and 531,131 additional simple samples can be used as additional training data.
GTSRB (Stallkamp et al., 201 I) is traffic sign classification dataset, which contains 43 classes of traffic signs with a total sample size more than 50,000.
MNIST (LeCun, 1998) is a benchmark dataset for image classification. It contains grayscale images of handwritten digits in 10 classes. The number of images in the training and test sets is 60,000 and 10,000 respectively. The number of images in each class is balanced. DTD (Cimpoi et al., 2014) is a texture classification data set, which contains 47 classes, a total of 5,640 images, and each class has approximately 120 images.
Baseline Details. Our experiments involve the following seven comparison methods and four variations of our method.
Individual means that each task uses an independent fine-tuned model, which has no interference between tasks, but cannot perform multiple tasks simultaneously.
Traditional MTL collects the original training data of all tasks together to train a multi-task model. It can be used as a reference upper bound for model merging work.
Weight Averaging is the simplest method of model merging, which directly averages the parameters of multiple models. It can be used as a lower bound for model merging.
Fisher Merging (Matena & Raffel, 2022) calculates the Fisher information matrix (Fisher, 1922) to measure the importance of each parameter when merging models, and model merging is performed according to the guidance of this importance.
RegMean (Jin et al., 2023) imposes a constraint when merging models, that is, the L2 distance between the merged model and a single model is required to be as small as possible. Task Arithmetic (Ilharco et al., 2023) first defines the concept of "task vectors" and merges task vectors into pre-trained model to execute multi-task learning.
Ties-Merging (Yadav et al., 2023) further solves the task conflict problem in Task Arith- metic (Iharco al., 2023). It eliminates redundant parameters and resolves symbol conflicts through three steps: Trim, Elect Sign, and Disjoint Merge.
Tisk-wise AdaMerging (Ours) is based on Task Arithmetic (Ilharco al., 2023), which uses an unsupervised method to automatically learn the merging coefficient of the task vector in Task Arithmetic.
Task-wise Meggiggg (Ours) is based on Ties Ties-Mgrging (Yadav et al., 2023), which uses an unsupervised approach to learn a merging coefficient for each task vector in Ties- Merging.
Layer-wise AdaMerging (Ours) automatically learns a merging coefficient for each layer of each task vector in Task Arithmetic (Ilharco et al., 2023).
Layer-wise AdaMergign++ (Ours) uses an unsupervised approach to learn a merging coefficient for each layer of each task vector in Ties-Merging (Yadav al., 2023).
Implementation Details. For the seven baseline methods, we follow the experimental settings in Task Arithmetic (Ilharco et al., 2023) and Ties-Merging (Yaday al., 2023). In our experiments,
14

---

Published as a conference paper at ICLR 2024
the merging coefficient À of Task Arithmetic and Ties- Ties-Merging is set to 0.3 by default. For our four K.I. variants, initialize all coefficients {> (or (X/ (X() ( to 0.3 by default before learning and then update them unsupervised. We an Adam optimizer (Kingma & Ba, 2014) to update the merging coefficients, with the learning rate set to 0.001, the momentum to (0.9, 0.999), and the batch size 16. To avoid significantly increasing training costs, we only trained 500 iterations update the merging coefficient. Pre-trained models ViT-B/32, ViT-B/16 and ViT-L/14 from CLIP (Radford al., 2021) like Task Arithmetic (Ilharco et al., 2023) and Ties-Merging (Yadav et al., 2023).
B EXPERIMENT RESULTS
B. PERFORMANCE. GENERALIZATION AND ROBUSTNESS
Performance. Tab. shows the average accuracy of merging ViT-B/16 on eight tasks. We can observe that: (i) Ties-Merging alleviates the conflict problem of task vectors in Task Arithmetic, thus achieving a 3.2% performance improvement compared to Task Arithmetic. (ii) Our Task-wise AdaMerging and AdaMerging++ automatically learn merging coefficient for each task vector in Task Arithmetic and Ties Ties-Merging, thus bringing about 2.2% and 1.0% performance improvements, respectively. (iii) Our Layer-wise AdaMerging and AdaMerging++ further adaptively learn merging coefficient for each layer of each task vector in Task Arithmetic and Ties-Merging, ultimately achieving performance improvements of 1 11.1 % and 8.7%. These results further demonstrate the effectiveness of our AdaMerging scheme in multi-task model merging.
Table 6: Multi-task performance when merging ViT-B/16 models on eight tasks.
Method
SUN397
Cars
RESISC45
EuroSAT
SVHN
GTSRB
MNIST
DTD
Arg Ace
Task Arithmetic (Ilharca al., 2023)
61.1
65.9
74.0
76.2
88.0
73.9
98.4
53.0
73.8
Ties Merging (Yadav et al., 2023)
69.1
72.5
80.5
84.0
85.0
71.5
98.1
54.9
77.0
Task-wise AdaMerging (Ours)
86.3
46.9
Task-wise AdnMerging++ (Ours)
67.8
70.2
79.9
89.2
87,5
79.2
98.3
51.9
78.0
Layer-wise AdaMerging (Ours)
70.2
80.7
94.8
91.6
95.8
98.5
66.2
84.9
Layer-wise AdaMerging*+ (Ours)
71.8
80.8
84.1
94.3
91.9
94.5
98.7
69.8
85.7
Generalization, As shown in Tab. 7, we demonstrate the generalization of Layer-wise AdaMerging under the ViT-B/16 architecture. In the unseen test tasks of EuroSAT and MNIST (their corresponding task vectors are not merged), our AdaMerging improved the average accuracy by 2.3% compared Task Arithmetic. On two unseen tasks, RESISC45 and SVHN, the average accuracy increased by 1.1%. This shows that AdaMerging has better generalization properties.
Table 7: Generalization results on two unseen tasks when merging ViT-B/16 models on six tasks.
Socn Tasks
Unseen Tasks
Method
SUN397
Cars
RESISC45
DTD
SVHN
Avg Ace
BaroSAT
MNIST
Avg Ace
Task Arithmetic
68.1
73.0
81.6
59.1
89.1
83.8
75.8
43.9
87.5
65.7
AdaMerging (Ours)
69.1
79.3
90.0
66.2
95.2
94.4
82.4
45,9
90.1
68.0
Method
SUNN997
Cars
GTSRB
FunSAT
DTTD
MNIST
Average
RESISC45
SVHN
Average
Task Arithmetic
69.0
73.8
81.1
87.6
58.2
98.4
78.0
56.0
67.7
61.8
AdaMerging (Ours)
72.9
81.0
97.1
96.4
66.5
99.2
85.5
$2.3
75.6
63.9
Robustness. Tab. 8 shows the robustness test of AdaMerging and Task Arithmetic based on ViT-B/16 on seven corruption test datasets. Fig. 5 shows example of corruption. We can observe that in the test datasets Motion Blur, Impulse Noise, Gaussian Noise, Pixelate, Spatter, Contrast and JPEG Compression where the distribution drifts, the average accuracy of AdaMerging is 9.9%, 8.2%. 7.8%. 6.8%, 12.4%, 9.5% and 9.7% % higher than that of Task Arithmetic, respectively. This shows that our AdaMerging more robust to test data distribution shifts than Task Arithmetic.
B.2 ANALYSIS EXPERIMENT
Task Relationship Analysis. As shown in Fig. 6(a) and (b), show the correlation between pairs of task vectors in VIT-B/32 and ViT-L/14, respectively. We observe a phenomenon consistent with Task Arithmetic (Ilharco et al., 2023), that is, these task vectors are almost orthogonal to each other.
15

---

Published as a conference paper at ICLR 2024
Table 8: Robustness results when merging ViT-B/16 models on four tasks.
Clean Text Set
Corruption Test Set (Motion Blur)
Method
EuroSAT
RESISC45
GISRB
Avg Ace
Cars
EuroSAT RESISC45
GTSRB
Arg
Task Arithmetic
75.3
96.3
85.3
80.5
84.3
73.5
70.9
83.9
72.2
75.1
AdaMerging (Ours)
83.4
97.2
88.6
97.5
91.7
81.3
75.9
87.4
95.6
85.0
Computtinn Test Set (Impulse Noise)
Cormution Test Set (Gaussian Noise)
Method
EuroSAT
RESISC45
GTSRB
Avg Ace
Cars
EuroSAT RESISC45
GTSRB
Avg
Task Arithmetic
70.4
39.5
75.2
54.0
64.8
72.2
60.8
78.5
51.0
65.6
AdaMerging (Ours)
77.6
42.1
81.9
90.2
73.0
79.1
58.9
81.2
74.5
73.4
Comuuttion Test Set (Pixelate) GTSRB
Comuption Test Set (Spatter) GTSRB
Method
EuroSAT RESISC45
Avg Ace
Cars
EuroSAT RESISC45
Avg
Task Arithmetic
03.8
38.0
24.8
71.3
34.5
72.1
58.4
79.9
67.6
AdaMerging (Ours)
04.1
46.4
23.6
91.3
41.3
79.3
60.9
85.8
93.7
80.0
Corruption Test Set (Contrast) GTSRB
Corruption Test Set (JPG Compression)
Method
Cars
EuroSAT RESISC45
Avg Ace
Cars
EuroSAT
RESISC45
GTSRB
Avg Acc
Task Arithmetic
73.4
62.5
81.3
76.9
73.5
75.1
73.1
84.8
64.7
74.4
AdaMerging (Ours)
81.4
68.1
85.8
96.8
83.0
81.9
76.0
87.3
91.0
84.1
Clean
Motion Blur
Contrast
JPEG Compression
Spatter
Pixelate
Impulse Noise
Gaussian Noise
Figure 5: An example of corruption data visualization, in which the corruption image generation method refers to Hendrycks & Dietterich (2019).
In partieular, there are very few task vectors with high similarity between them, such as SVHN and MNIST, because they are both handwritten digit recognition tasks. The orthogonality of task vectors provides good initial conditions for model merging, indicating that they have the potential to be combined into single model, and our results show that this is indeed the case, Further, we merge four groups of task vectors with different correlation degrees, namely (SVHN, MNIST), (SVHN, GTSRB), (SVHN, SUN397), and (SVHN, EuroSAT). The results are shown in Fig. 7. We observe that under task vectors with different degrees of correlation, our AdaMerging technique always effective because it aims to adaptively learn optimal merging coefficients.
Impact the Amount of Available Test Data on Performance. The AdaMerging proposed in this paper requires unlabeled test dataset to perform entropy minimization optimization. Having all test data available may be unrealistic some scenarios. this section, we verify the performance changes of AdaMerging when different amounts (e.g.. 0.1%, 1%, 5%. 100%) of test data available. As shown Fig. 8 and Tab. 9, we observed that even when only 0. % unlabeled test data are available, our AdaMerging and AdaMerging+++ still have performance improvement of 4.9% and
16

---

Published as a conference paper at ICLR 2024
0.5
0.5
0.04
0.04
1.00
0.03
0.02
0.33
0
0.04
0.4
0ar
0.0
0.08
0.03
0.22
0.4
RESISC45
004
0.09
1.02
0.07
0.09
0.04
0.04
0.26
RESEC15
0.03
0.02
1.30
0.05
0.02
0.38
0.09
0.03
0.3
0.3
EUSBAT
EJISAT
SVHN
043
0.08
0.03
0.05
1.60
1109
1.15
0.05
SKHN
0.12
002
0.02
0.04
100
0.08
0.12
012
MNIST
0.0
0.04
0.04
0.04
9.16
0.08
1.07
0.04
MNST
0.02
0.3
0.08
0.04
0.12
0.06
1.00
0.03
0.1
0.04
0.04
0.04
1.00
0.03
0.00
0.2
0.05
0.00
10
SUReRT
Oom
Resocat
073R8
SUGDT
RESSS4R
07EN0
MNNT
010
ViT-B/32
(b) ViT-L/14
Figure 6: Cosine similarity between task vectors on ViT-B/32 and VIT-L/14.
100
100
Ancarrey (%)
85
96
80
Acsllergigg
Achllegging
vidi MVVT
Indivitull O7SRB
vndivitual
AdaMerrggg
Indivitua
Acaleeggggng
100
200
300
500
100
200
300
400
lterations
Iterations
(a) SVHN and MNIST
(b) SVHN and GTSRB
100
Accurasy (8))
90
96
Ascuracy
80
70
Adsllening
100
200
300
400
500
0
100
200
300
400
500
Iterations
Iterations
(c) SVHN and SUN397
(d) SVHN and FuroSAT
Figure 7: Merging of task vectors with different correlations the ViT-B/32 model. Note that when iteration=0, it also represents the performance of Task Arithmetic (Ilharco et al., 2023) (A = 0.3).
5.5%, respectively, compared to Task Arithmetic and Ties Merging, In addition, when 5% the data available, it can almost achieve performance comparable 100% of the data. This shows that our AdaMerging is valuable and can bring significant performance improvements even with a small amount of data.
Table 9: Impact of the amount of available test data performance when merging VIT-B/32 models.
Method
RESISCA5
Arg
Task. Arilhmetic (llbaroo cl al., 2023)
0.00%
55.2
51.9
66.7
78.9
80.2
69.7
97.3
50.1
69.1
Layer-wise AdaMerging (Oun)
0.10%
62.5
59.7
71.2
69.5
89.3
84.2
96.2
57.5
74.0
Layer-wise AdaMerging (Ours)
1.00%
61.9
66.3
81.8
36.
88.6
85.8
97.
52.5
77.5
Layer-wise AdaMerging (Ours)
5.00%
63.7
68.6
79.1
93.3
86.5
91.7
80.1
Layer-wise AdnMerging (Ouns)
79.2
91.9
59.1
Merginggg (Yadav al., 2023)
0.00%
59.8
58.6
70.7
79.7
86.2
72.1
98.3
54.2
72.4
Layer-wise AdaMerging++ (Ours)
0.10%
70.0
66.2
74.6
79.9
89,3
83.6
98.4
61.2
77.9
Layer-wise AdaMerging*+ (Ours)
1.00%
66.9
68.6
8L.1
89.2
$7.1
98.1
61.8
80.6
Laye-wwie AdaMerging-++
98.2
layerwise AdaMergings+ (Ours)
100.5
66.6
68.3
83.2
89.6
89.0
98.3
60.6
81.1
Supervised AdaMerging Analysis. This paper uses unsupervised entropy minimization as proxy objective for supervised cross-entropy loss to optimize model merging coefficients. Therefore, AdaMerging trained with supervised cross-entropy loss should be an upper bound on our unsupervised AdaMerging, As shown in Fig. 9 and Tab. 10, we observe that the performance of unsupervised AdaMerging version is very close to that of the supervised AdaMerging version. For example, the Avg Ace of supervised Task-wise AdaMerging is 71.3%, while the Avg Ace of our unsupervised Task-wise AdaMerging is 71 71.1% %. This also further verifies that it is reasonable for us to use entropy minimization proxy for cross-entropy loss in merging coefficients learning.
17

---

Published as a conference paper at ICLR 2024
Taxk Artimeli
85.0
TES-Mergng
Asstoegigg (X])
62.5
Aversge Accuracy
Aestleggiea (S4)
Avengg Acturnay
755
77.5
75.0
70
72.0
100
200
300
400
500
70.0
0
100
200
300
400
500
Itanatons
(a) Task Arithmetic and AdaMeggig
(b) Ties-Merging and AdaMerging++
Figure 8: Impact of the amount of available test data (e.g., 0.1%, 1%, 5%, 100%) on performance when merging ViT-B/32 models.
Table 10: Performance comparison between supervised and unsupervised versions of AdaMerging.
Method
Label
SUN397
Cars
RESISC4S
EuroSAT
SVHN
GTSRB
MNIST
DTD
Avg Ace
Aritmmetic
2023)
Tusk-wise AdaMerging
Supervised
58.4
56.4
74.8
81.2
81.5
77.4
83.3
52.3
71.3
Task-wise AdaMerging
Unsupervised
58.0
53.2
68.8
85.7
81.1
$4.4
92.4
44.8
71.1
Layer-wise AduMerging
Supervised
66.8
68.4
85.3
88.7
89.8
95.9
65.6
81.6
Layerwise AdaMerging
Unsupervised
91.9
97.5
Mer--ngggnng (Yaday
59.8
58.6
70.7
86.2
72.1
98.3
54.2
724
Morgings
Supervised
59.3
79.1
91.5
73.7
Task-wise AdaMenging++
Unsupervised
60.8
56.9
73.1
83.4
87.3
82.4
95.7
50.1
73.7
Layer-wäse Ada Merging++
Supervised
68.2
69.8
84.8
93.4
89.3
89.1
97.3
64.3
82.0
Layerwisee AdaMerging++
Unsupervised
98.3
Parameter Cost A Analysis. As shown in Tab. 11, our AdaMerging introduces very few coefficients that need to be updated. The total number of parameters of the eight task vectors 907,589.640. and our Task-wise AdaMerging only added parameters, and Layer-wise AdaMerging added 1,248 parameters.
Time Cost Analysis. As shown in Tab. 12, we show the performance that AdaMerging achieve un- der different training costs (based on a single GeForce RTX 3090). We observed that our AdaMerging brought about 2% performance improvement when it took 7.5 minutes longer than Task Arithmetic. When training for 50 minutes, AdaMerging brought an 8% performance improvement. This shows that AdaMering very cost-effective and can bring significant performance improvements with only a small amount of training time.
Merging Coefficients Visualization. Fig. 11 shows the changes during the iteration process of merging coefficient optimization of each task vector in Task-wise AdaMerging and AdaMerging+++ which shown every ten steps. addition, Fig. 12 and Fig. 13 show the merging coefficients of eight task vectors learned by Layer-wise AdaMerging and AdaMerging++ on V/T-B/16 respectively. Finally, Fig. 14 and Fig. 15 show the coefficients learned under ViT-L/14. We can elearly observe that in different layers of different task vectors, the learned merging coefficients are different. Finding the merging coefficients of many layers through grid search is almost impossible.
Visualization of Spearman's Correlation Coefficient Between Entropy and Loss. As shown in Fig. 10, we show the correlation changes unsupervised entropy minimization and supervised cross-entropy loss at different training stages (i.e., the number of iterations are {0, 100, 200, 300, 400,500) respectively). We observe that in the merging coefficients learning process of AdaMerging, entropy minimization and cross- cros-entrrppy loss always have high correlation. Therefore, entropy minimization can be used as surrogate objective to optimize model merging coefficients.
Visualization of Correlation between Entropy and Loss. As shown in Fig. 16, analyze the correlation between the entropy and the model's prediction loss for eight tasks (or datasets) on the initial merged model. As described Sec. 3.2.2, in each dataset, we sort the entropy on the test samples from small to large into eleven groups and observe the average loss of sample prediction within each group. We observe that groups with smaller entropy generally have smaller average losses. Therefore, it is reasonable to take Shannon entropy minimization as an unsupervised optimization surrogate objective for loss (c.g.. cross-entropy) minimization.
18

---

Published as a conference paper at ICLR 2024
TES-Narggs
(N3
85
Aubrlergin Dupenteet
Average Accurnry (8)
85
TaBlwet Asaldergig (uupentted)
Tabkwti Adallerg broupentied
Tedlwas Acaifergigg corupayiie
Adallerg ry Depenteed
Taplwas Acaifeggig oupentred)
Tedowrre Atdleggoo crmppniegg
Average
75
100
200
300
400
500
100
200
300
400
500
Iterations
Iterations
AdaMerging
(b) AdaMerging++
Figure 9: Supervised and Unsupervised AtaMergin//AaaMeriinn+++ merging ViT-B/32 models.
Table 11: Parameter cost of model merging for AdaMering on ViT-B/32.
Method
Task Arithmeric
Merg-ng
Task-wise AdaMerging Layer-wise AdaMerging
Total number model merping parameters (all lask vectors)
907 907_589,6100
907.589.610
907,589.610
907,589.640
Total number of trainable model merging cocflicients
1,248
Table 12: Time cost of model merging for AdaMering ViT-B/32.
Training Time
Base
+7.5 min
+12.5 min
+25 min
+50 min
+100 min
+125 min
Avg Acc of Layer-wise AdaMerging Avg Acc of Layer-wise AdaMerging++
69.1
71.1
72.1
74.5
77.1
79.7
80.1
72.4
74.1
74.8
76.3
78.3
80.5
81.1
0.5
07
2.50
0.6
100
200
500
400
600
0.50
100
200
500
400
900
0.50
100
250
500
400
600
5048
BurSAT
0.0
0.8
0.8
07
SUH-
1.0
0.9
0.0
0.5
0.85
Figure 10: Spearman correlation coefficient between entropy H(Y) and avareage loss L(Y, Y) on eight tasks (or datasets) different stages of training(e.g.. Iteration=(O, 100, 200, 300, 400, 500)), and we observed high positive correlation.
SUN 397
RESISCAS
SVNNN
MM
0.8
SUN
RESISCAS
SVHN
MNIST
Cars
EuroSAT
GTSRB
Cars
EUroSAT
GT5RB
0.6
0.4
0.2
10
50
iteration(x10)
iteration
Figure 11: Model merging coefficients {Ax 1= change with respect training steps ViT-B/32: (a) Task-wise AdaMerging; (b) Task-wise wise AdaMerging+++ Each line represents the change process of the coefficient d of task vector Tç (k E {1, (1,2...,KK) K)).
19

---

Published as a conference paper at ICLR 2024
1.0
SUN397
0.8
Task Vectors
RESISC45
EuroSAT
0.6
SVHN
0.4
GTSRB
MNIST
0.2
DTD
0.0
8
12
16
20
2 28
32
36
40
44
49
52
56
60
64
68
72
80
88
97
96
100
104
108
112
116
120
124
128
132
136
140
144
152
156
Layers
Figure 12: Learned model merging coefficients of Layer-wise AdaMerging on VIT-B/16. The k-th row
represents the k-th task vector, the (-th column represents the 1-th layer, and the intersection point represents the coefficient A/-
1.0
SUN397
Cars
0.8
Task Vectors
RESISC45
EuroSAT
0.6
SVHN
0.4
GTSRB
MNIST
0.2
DTD
0.0
12
16
20
24
28
32
36
40
44
48
52
56
60
64
72
76
80
84
88
92
96
100
104
108
112
116
120
124
128
132
136
140
144
148
152
Layers
Figure 13: Learned model merging coefficients of Layer-wise AdaMerging++ on ViT-B/16. The k-th row
represents the k-th task vector, the /-th column represents the I-th layer, and the intersection point represents the coefficient Al.
SUN397
Cars
0.8
Task Vectors
RESISC45
EuroSAT
0.6
SVHN
0.4
MNIST
0.2
DTD
0.0
14
28
35-
42
56
63
77
84
91
105
11.2
119
126
133
147
154
161
168
175
182
196
203
217
224
231
238
245
252
259
266
273
280
287
294
301
Figure 14: Learned model merging coefficients of Layer-wise AdaMerging VIT-L/14. The k-th row
represents the k-tth task vector, the (-th column represents the 1-th layer, and the intersection point represents the coefficient A/
SUN397
Cars
Task Vectors
EuroSAT
SVHN
0.4
GTSRB
MNIST
0.2
DTD
0.0
1
14
28
35-
42
56
63
77
84
91
98
105
112
119
126-
133-
147
154
T61
168
175
182
196-
203
217
224
238
245
252
266
273
280
287
294
301
Figure 15: Learned model merging coefficients of Layer-wise AdaMerging++ ViT-L/14. The k-th row
represents k-th task vector, the /-th column represents the (-th layer, and intersection point represents the
coefficient Ai.
20

---

Published as a conference paper at ICLR 2024
SUN397
Cars
2.66
2.30
2.0
Average Loss
Average Loss
1.5
1.86
1.61
1.00
1.5
1.26
1:35
1.48
1.0
0.82
0.50
1.0
0.66
0.0
(0-44)
(0.1-.2)
(0.203)
(8.3--4)
(0445))
(0.50))
(0.0-.7)
(0.7-0.))
(.8.4.))
(0.9-200
>1.
(0.01)
(0.1-..)
(.2.33)
(0.9-44)
(0.5-0.) (0445)
(0.80.7)
(.7--.))
(0.003))
(6.0..))
>1.0
Entropy
Entropy
RESISC45
EuroSAT
2:42
1.5
Average Loss
2.0
1.00
1.98
Averaige Loss
1.15
1.20
1.30
1.43
1.52
138
0.1
0.5
0.35
0.5
0.2
(0-04)
[0.1-03]
(0.20.3)
(0.3--4)
(04-05)
(0.50.)
(0.6-.7)
(0.74...g)
(0,--.))
(0.9-1 la
>1.0
(0401)
(0.140.)
(0.2033)
(0.304)
(0.5- (0.40.))
(0.60.)
(0.7-..9)
(0.84.)) (0.6-))) 3)
.1.0
Entropy
Entropy
SVHN
GTSRB
1.88
2.0
1.5
Average Loss
Average Lass
1.5
1.0
0.97
0.71
0.77
1.0
0.68
0.5
0.5
0.56
0.15
0.04
0.05
0.11
0.21
0.0
(940.)
(0.10
(0.34.4] (0.20.3)
(0.40.5)
(9.5-) of
(0.60.7]
(0, af
(0.8-..))
(0.3 of
>10
(64.1)
(0.14.2)
(0.2-.3)
(0.30.4)
(0.40.5)
(0.5-8.))
(0.60.)
(0.8-0.) (0.74..))
(0.9..))
-1.0
Entropy
Entropy
MNIST
DTD
1.21
2.57
2.5
Average Loss
0.84
Average Lasss
2.17
2.28
0.73
2.0
0.19
0.5
155
0.35
0.03
0,11
1.06
(00.4)
(0.10.4)
(0.20.3)
(03-04)
(0.40.5)
(0.50.4)
00.60.77
(0. (0.740.6)
(0.6-.))
0..1-
.1.0
(64.2)
(0.10.2)
(0.24.3)
(0.30.]
(0.40.5)
(0.5...)))
(060..)
(.7.4))
(0
(0.9..))
.1.
Entropy-
Entropy
Figure 16: Correlation analysis of entropy and average loss on eight tasks (or datasets). We can observe that there is high positive correlation between entropy and prediction loss each dataset.
21


Reviews:
[{"review_id": "wq6KyrPLZp", "reviewer": "Reviewer_88YY", "summary": "The paper first points out that multi-task model merging coefficients have a significant impact on the performance of existing merging solutions, and grid search model merging coefficients are unrealistic. Then, this paper proposes a new merging scheme, AdaMerging, based on multi-task entropy minimization to learn the optimal merging coefficients. Finally, the results under various architectures show that the proposed solution is significantly improved compared to existing model merging solutions.", "questions": "1. Is AdaMering in Tables 2, 3, and 4 a Task-wise or a Layer-wise version?\n2. Why is the model merging performance closer to traditional MTL in a larger architecture?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper studies model merging without original data, which is an important research direction.\n2. This paper proposes an unsupervised model merging scheme, which is technically feasible. Experimental results show that the proposed scheme has better multi-task performance, generalization, and robustness.\n3. The paper is well organized and easy to understand, and the proposed solutions are easy to follow and implement.", "weaknesses": "1. In the motivation, the authors need to explain the intuitive motivation for entropy minimization as a proxy objective for loss.\n2. In the experimental analysis, the author needed to explain why AdaMerging has better generalization and robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dCZxwJf75r", "reviewer": "Reviewer_3WDA", "summary": "This paper tackles the problem of **Multi-task learning** in the context of foundation models: While the most common MTL paradigm is to train a single model on multiple tasks jointly, the paper investigates a different direction of merging single-task networks to form a single unified network. Like standard MTL, this approach can be affected by negative interference between tasks, in that naively merging model weights of conflicting tasks leads to poor performance. \nSpecifically, the proposed method builds on task arithmetic: While previous works considers simple uniform averaging of task vectors, this paper proposes `AdaMerging` which automatically learns how to weigh each model when merging the tasks, as well as a per-task *and* per-layer variant. The primary goal is to improve the model performance, as task arithmetic approaches still perform worse than standard MTL training. Finally, a last variant `AdaMerging++` which further integrates some ideas from `TiesMerging` (e.g. by removing redundant parameters and sign conflicts in the task vectors before merging them).\n\nIn practice, we may not have access to the original single task model training data, but only to a set of potentially unlabeled multi-task data. A key insight of the paper is that the entropy of the MTL model predictions is often correlated with the actual loss on the corresponding samples. Consequently, the proposed method directly optimizes the weights of the task vectors $\\lambda$, while minimizing the entropy of the current MTL model's prediction. The proposed `AdaMerging` is evaluated on ViT-backbones from CLIP models, on a suite of computer vision tasks, and compared to task arithmetic methods as well as traditional MTL optimization.", "questions": "- Does the **traditional MTL** baseline use some form of task weighing ? Intuitively I would expect that a \"good\" set of task vector weights might also be useful for reweighing/rebalancing the different tasks losses/data in traditional MTL training; but it would also be an interesting insight if that is not the case\n\n- Do you have insights on how a **supervised variant of `AdaMerging`** would perform ? It would be interesting to understand how much of the current gap with traditional MTL is due to the different supervision assumption, or due the task arithmetic process itself, versus directly finetuning the model weights  on MTL data.\n\n- Minor note:\n  *  in related work: *Task Arthmetic* -> Task Ar**i**thmetic", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "- **Good writing**: The paper is well written and easy to follow, also with good illustrative figures.\n\n- **Interesting research direction**: Task vector arithmetic for foundation model is a novel and interesting take on multi-task learning. Extending it to learning the task vector weights seems like a natural and meaningful direction, and very much in-line with automatic loss/gradient weighing scheme in standard multi-task optimization methods. \n\n- **Good set of ablation experiments**: The results on generalization (unseen tasks and corrupted data), as well as the qualitative visualization of the learned task vectors weights are very insightful.", "weaknesses": "- The conclusion of **Section 3.2.2** seems a bit strong to me from the conducted experiment: the analysis shows that the entropy and loss of a trained MTL model are nicely correlated, but it does not necessarily mean that they yield equally good directions during training: Doing the same analysis at different timesteps during the MTL model training could show whether and how this correlation holds during training.\n\n- **Discrepancy in supervision** : If I understood correctly, the single-task and MTL baselines use standard supervised training schemes; Task merging baselines (e.g. task arithmetic) are data-free methods and only require the task vectors; finally`AdaMerging` learns task arithmetic weights on unlabelled test samples ($B_k$ on page 6). While these are unsupervised, seeing test samples seems unfair compared to other baselines, especially the task arithmetic ones; it also may not be realistic to have access to a whole (unlabelled) test dataset at once (as opposed to e.g. a few-shot setting)\n\n- The experimental evaluation only considers **ViT-based backbones on small/medium computer vision benchmarks**. This is a bit different from the introduction, which focuses on readily available pretrained foundation models for which the original training data may be unknown. Furthermore, this also raises the question of MTL baselines: there is a very large literature on automatically weighing tasks losses/gradients for computer vision tasks, which may be stronger baselines than `traditional MTL`, and increase the performance gap even more (e.g. GradNorm, PCGrad, GradDrop...etc)\n\n\n**Overal summary**: To summarize, my main concerns are mainly *(i)* the fairness of chosen baselines (in terms of supervision in the case of task arithmetic schemes, and in terms of optimization strategies for traditional MTL) and to a lesser degree *(ii)* the strength of the conclusions derived from the analysis in **Section 3.2.2**.\n\n**Post rebuttal summary** I'm increasing my rating from 5 to 6 as the authors have addressed my main concerns, in particular about the additional requirement of test data; and I think/hope a more in-depth discussion of the trade-offs of the newly introduced experimental setting compared to the ones of traditional task merging and traditional MTL would make the contribution even stronger.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L8462947wE", "reviewer": "Reviewer_eRin", "summary": "Merging multiple fine-tuned models without a retraining process along with initial training data has been shown to be feasible, i.e., two existing works, Ties-Merging and Task Arithmetic. However, directly adding models may fail due to potential conflicts and intricate correlations. This paper proposed a new approach to automatically learn the merging weights by minimizing testing-time entropy on unlabeled samples in an unsupervised manner. Instead of the need for initial training data, this paper showed that testing-time entropy can serve as an approximated objective compared to traditional supervised loss. The authors proposed two main ways to learn the merging weights. One is task-wise merging, which learns the coefficient for each task. The other is a more fine-grained version, layer-wise merging, which not only learns the coefficient for each task but also for each layer. In their experiments, they included eight tasks to validate their approach with ViT models. Results have shown improvements in performance (for classification accuracy), generalization capabilities (to unseen tasks), and robustness (to test data distribution shifts) compared to the other two SOTA methods.", "questions": "* The task relationships across 8 tasks included in the paper can be mentioned. I am curious about the performance changes when we have unrelated tasks and high-correlated tasks. \n* In some cases, can the part of the merging weights be negative terms?\n* Does the $\\lambda$ correlate to the performance gain in Table 1 and Table 2? Is there any relationship between weights and performance gain/loss?\n* (minor comment): It will be nice to move Fig 2 earlier for a better understanding.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Originality: Merging multiple fine-tuned models has been shown feasible, but this paper proposed and proved that using testing-time entropy as an objective to learn merging weights is effective and can be automatic. They also suggested that learning weights across different layers is crucial to the success of merging. These show the strength in originality.\n* Quality: The results of the experiments are solid and promising. They closed the performance gap between conventional MTL and task arithmetic-based methods.\n* Clarity: The presentation of this paper is clear. \n* Significance: Developing a single versatile model from ***diverse off-the-shelf fine-tuned models*** is important to LLM communities. Methods proposed by this work can reduce the efforts of collecting initial training data and the need for retraining for merging models. Besides, to merge fine-tuned models without tuning merging weights by grid-search, their automatic method to learn weights via test-time entropy is important to develop the means of model fusion. At last, the results in the paper improved two existing SOTA methods and showed method's strength in several dimensions.", "weaknesses": "* Though we don’t need to train model again via original training data, we still need to access a certain amount of testing data for testing-time-entropy minimization. How the (minimum) amount of testing data can affect the quality of merging weights ($\\lambda$ in the paper) is encouraged to study and present in the paper.\n* In additional to the amount of testing data, the burden/computational needs/computational time to learn $\\lambda$ to converge via unlabeled testing samples is missing and lack of comparison to the other two SOTA methods. This study should be included and enhance the soundness of the proposed method.\n* I didn't see any restriction or regularization on $\\lambda$, especially in the optimization objective in Sec. 3.2.2. Does $\\lambda$ always need to be $\\sum_{i=k}^K \\lambda_i = 1$?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S4wXm2gb0v", "reviewer": "Reviewer_zdU4", "summary": "This paper presents a model merging method in the context of multi-task learning. The key idea is to take a pre-trained model and fine-tune it separately for each task using task-specific data, resulting in task-specific models. Then the paper introduces a novel method for automatically merging the task-specific parameters without the need for retraining. In essence, this work builds upon the foundations of Task Arithmetic [1] and TIES-MERGING [2] but enhances the process by incorporating adaptive task weights.\n\n[1] Ilharco, Gabriel, et al. \"Editing models with task arithmetic.\" ICLR2023.   \n[2] Yadav, Prateek, et al. \"Resolving Interference When Merging Models.\" NIPS2023.", "questions": "- I'm not sure the paper has sufficient novelty to be published in the top-tier conference since the proposed method only goes one step further from Task Arithmetic [1] and TIES-MERGING [2] by incorporating trainable weights for task vectors.The concept seems thin to support an entire paper, with only one page (page 6) dedicated to the novel part. Authors should consider diving deeper into this direction. For example, exploring the underlying reasons for the weight relationships between different tasks and their potential correlation with task relationships could enhance the paper's depth. Additionally, the learned weights could be utilized to guide the training of multi-task models, as seen in Auto-lambda [3].\n\n- Is it really necessary to conduct experiments to show the relationship between Shannon entropy and cross entropy? Actually from the information theory, the two concepts are almost the same thing or we can say cross entropy is derived from Shannon entropy. It's kind of trivial or even unnecessary to do experiments in Figure 3. Besides, the usage of Shannon entropy to train the adaptive weights also limits the method can only be used for classification tasks.\n\n- It's better to show the performance of the pre-trained model on each task as well in Tables 1 and 2. \n\n- Limited application of this kind of work. From Tables 1 and 2, we can clearly see that traditional MTL or we say all-shared MTL can achieve a very high accuracy, not to say SOTA MTL methods like AdaShare [4] and AutoMTL[5]. In practice, machine learning engineers might prefer these alternatives due to their superior performance. Besides, for the model merging direction, it's weird to assume that although we may not be able to get the train data for each task, we can still get the pre-trained weights of the model. Most importantly, those task-specific models even need to be trained from the same pre-trained weights. \n\n[1] Ilharco, Gabriel, et al. \"Editing models with task arithmetic.\" ICLR2023.   \n[2] Yadav, Prateek, et al. \"Resolving Interference When Merging Models.\" NIPS2023.   \n[3] Liu, Shikun, et al. \"Auto-lambda: Disentangling dynamic task relationships.\" TMLR2022.   \n[4] Sun, Ximeng, et al. \"Adashare: Learning what to share for efficient deep multi-task learning.\" NIPS2020.   \n[5] Zhang, Lijun, Xiao Liu, and Hui Guan. \"Automtl: A programming framework for automating efficient multi-task learning.\" NIPS2022.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- The paper is easy to follow. \n- The idea is easy to catch up with.", "weaknesses": "- The novelty remains a question for me. \n- The practical value of this method is not well supported.\nPlease refer to the questions part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)