DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging
Authors: Kotaro Yoshida
Yuji Naraki
Takafumi Horie
Ryotaro Shimizu
Ioannis Mitliagkas
Hiroki Naganuma
Venue: ICLR 2026 Poster (2026)
Keywords: Model Merging
Model Editing
Task Vector
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: We identified previously unrecognized detrimental factors in model merging and introduced DisTaC, a knowledge distillation-based approach designed to mitigate their effects.

Abstract: Model merging has emerged as an efficient and flexible paradigm for multi-task learning, with numerous methods being proposed in recent years. 
However, these state-of-the-art techniques are typically evaluated on benchmark suites that are highly favorable to model merging, and their robustness in more realistic settings remains largely unexplored.
In this work, we first investigate the vulnerabilities of model-merging methods and pinpoint the source-model characteristics that critically underlie them.
Specifically, we identify two factors that are particularly harmful to the merging process: (1) disparities in task vector norms, and (2) the low confidence of the source models. To address this issue, we propose **DisTaC** (**Dis**tillation for **Ta**sk vector **C**onditioning), a novel method that pre-conditions these problematic task vectors before the merge. DisTaC leverages knowledge distillation to adjust a task vector's norm and increase source-model confidence while preserving its essential task-specific knowledge. Our extensive experiments demonstrate that by pre-conditioning task vectors with DisTaC, state-of-the-art merging techniques can successfully integrate models that exhibit these harmful traits, where they would otherwise fail, and achieve significant performance gains.

Published as a conference paper at ICLR 2026
DISTAC: CONDITIONING TASK VECTORS VIA DISTILLATION FOR ROBUST MODEL MERGING
Kotaro Yoshida'
Yuji Naraki
Takafumi Horie
Ryotaro Shimizu
Toannis Mitliagkas
Hiroki Naganuma
Institute Science Tokyo 'Independent Researcher *Kyoto University
ZZOZO Research
Mila Université de Montréal
ABSTRACT
Model merging has emerged as an efficient and flexible paradigm for multi-task learning, with numerous methods being proposed in recent years. However, these state-of-the-ar art techniques are typically evaluated on benchmark suites that are highly favorable to model merging, and their robustness in more realistic set- tings remains largely unexplored. In this work, we first investigate the vulnera- bilities of model-merging methods and pinpoint the source-model characteristics that critically underlie them. Specifically, ticularly harmful to the merging process: (1) disparities in task vector norms, and (2) the low confidence of the source models. To address this issue, pro- pose DisTaC (Distillation for Task vector Conditioning), a novel method that pre-conditions these problematic task vectors before the merge. DisTaC lever- ages knowledge distillation to adjust a task vector's norm and increase source- model confidence while preserving its essential task-specific knowledge. Our ex- tensive experiments demonstrate that by pre-conditioning task vectors with Dis- TaC, state-of-the-art merging techniques can successfully integrate models that exhibit these harmful traits, where they would otherwise fail, and achieve signif- icant performance gains. The source code is available at https: / /github. com/ httm/kttooo8999/Disstte
identify two factors that par-
INTRODUCTION
The recent wave of open-sourcing both large pretrained models (Devlin et al., 2019; Rombach al., 2022: Achiam et al., 2023; Grattafiori et al., 2024) and their fine-tuned downstream variants (Wolf et al., 2019; Taori et al., 2023) has put unprecedented variety of neural networks within easy reach of anyone. This democratization has, in turn, accelerated research on model merging (Wortsman al., 2022b;a; Ilharco al., 2023; Yadav et al., 2023; Akiba al., 2025), techniques that create new, customized models by integrating existing fine-tuned models without the need for additional large-scale training. In particular, a flurry of recent methods aims build multi-task models by merging networks that have been fine-tuned independently for each task, rather than retraining single shared model from scratch (Ilharco al., 2023; Yadav et al., 2023; Ortiz-Jimenez al., 2023; Wang et al., 2024; Yoshida et al., 2025; Gargiulo et al., 2025). Many of these techniques require only minimal extra training or none at all. Compared with conventional multi-task learning (MTL), they offer two key advantages: (i) they eliminate the need to aggregate all task-specifi labeled data in one location, sidestepping data-access constraints, and (ii) they make it easy to add or edit the model's 's skill particular task after deployment (Yang al., 2024a).
On established benchmarks, these approaches have shown promising gains, in some cases approach- ing the performance of traditional MTL (Gargiulo et al., 2025). Yet those benchmarks are built under conditions that are highly idealized for model merging; how robust current merging methods remain in more practical, pessimistic settings is still largely unknown. Bridging this gap prerequisite for real-world application.
"Conresponding author yoshida.k. 0.553 mmisst..c.jp 'Coresponding author naganuma. hirranuaa.hiookie @ mila.quehec.

---

Published as a conference paper at ICLR 2026
17.5
90-
10a
Norm. N0..CC (%)
60-
12.5
80-
10.0
Producton
40-
20-
5.0
10
2.5
7153
Commaat
TSVM
Caro
BTD
Bumsar
GTSRD
MNIST
SUN207
DTD
Eureaa
crsan
RESUSE Resuiigts Sins
Original
R.R Mismatch
IR-10
1R-10
wo LS
(o=0.1)
(c)
Figure 1: Failure Cases of Multi-Task Model Merging. All results were obtained using CLIP with ViT-B-32 backbone the eight vision tasks. (a) Comparison of normalized accuracy after merging models from different fine-tuning configurations averaged over eight vision tasks. The gray bar represents the conventional setting (a uniform learning rate of 10-5 with hard labels). The blue bar indicates the result of merging after training just one task with a learning rate (LR) of 10- The yellow bar shows the result when all tasks were trained with label smoothing (LS). Both the blue and yellow configurations show significant performance degradation compared to the conventional setting. (b) Change in the task vector norm after fine-tuning with different learning rates for the same number of steps across eight vision tasks, The gray bar uses learning rate of 10 10-5, matching the conventional benchmark, while the blue bar uses 10 We observe a 5 to 7-fold difference in the resulting task vector norms. (c) Change in the entropy of the model's predictive probabilities after fine-tuning with or without label smoothing across eight vision tasks. The vertical axis is on a logarithmic scale. Training with label smoothing increases the entropy by three orders of magnitude.
To that end, we first pinpointed where generic multi-task model merging pipelines break down. Our analysis reveals two especially harmful factors: (1) differences in task vector norms and (2) low prediction confidence of source models. Figure 1a illustrates the vulnerability of recent merging methods to these factors using CLIP (Radford et al., 2021) with a ViT-B-32 (Dosovitskiy et al., 2021) backbone on the eight vision tasks defined in Section 5.1: blue bars show the effect of training models with different learning rates, thereby altering task vector norms (see Figure lb), while yellow bars show the effect of label smoothing (Müller et al., 2019) (LS), which reduces model confidence (see Figure 1c). In the plot, the horizontal axis lists the merging methods, and the vertical axis reports the average normalized accuracy (Norm. ACC) across the eight tasks, defined as the post- merge accuracy relative to the pre-merge aceuracy obtained by individual models for each task. In both cases, every method's 'ss performance degrades substantially compared to the standard baseline, represented by the gray bars (a uniform learning rate of 10 with hard labels), with a maximum 24% drop in Norm. ACC.
These failure modes often arise in real-world deployments. For instance, differences in task vec- tor norms stem from varied learning rates, fine-tuning steps, weight decay used during the individual fine-tuning of each task (Devlin et al., 2019; Wightman et al., 2021). Low confidence often results from techniques such LS, Mixup (Zhang et al., 2017), and focal loss (Lin et al., 2017). We therefore contend that models should be pre-conditioned before merging to remove their latent harmfulness. To this end, we propose Distillation for Task-vector Conditioning (DisTaC) a lightweight knowledge distillation (KD) procedure that tackles both issues using only unlabeled data: To correct task vector- -orm disparities, DisTaC first rescales each vector chosen target norm and then restores any performance lost through this scaling by distilling knowledge from the original model. To address low source-model confidence, it trains the student with a higher temper- ature than the teacher (Tam > Tier), so the student ultimately produces lower-entropy predictions, that is, predictions that are more confident.
Algorithm 1 combines these two conditioning steps, allowing them to be carried out in single pass. Because DisTaC leverages the already-trained task vectors as the initialization for KD and relies solely unlabeled data, it incurs minimal computational overhead and imposes only modest practical requirements, yet markedly improves the robustness of existing model merging techniques in challenging scenarios.
Empirically, on eight vision tasks with ViT-B-322 B-32/L-11 backbones, DisTaC increased post-merge ac- curacy by up to 20.8 percentage points and restored the best-performing TSVM merge's normalized
2

---

Published as a conference paper at ICLR 2026
accuracy from 68% to 92% under low-confidence conditions, thereby matching the conventional "ideal" benchmark performance (i.e., merging high-confidence models with uniform task vector norms), all with minimal computational cost. Our contributions
follows:
We identify two failure modes in model merging: (i) the task vector norms of the source models differ (Section 3.1), and (ii) the source models' outputs are low-confidence even well-calibrated (i.e., their predicted probabilities match the true frequency of correctness) (Section 3.2). We provide theoretical explanations and empirical results for each of these phenomena.
We propose DisTaC, distillation method of source model's weights under appropriate conditions (Section 4), and demonstrate that it mitigates aforementioned failure modes (Section 5.2.1). Our DisTaC is a computationally efficient method, as requires only small number of training steps and relies solely on unlabeled data (Section 5.2.2).
From our analysis, we present two guidelines for model merging: (i) when the task vec- tor norms differ, it is better to shrink the larger vector rather than stretch the smaller one (Section 6) and (ii) when the source models have low confidence, more effective to make them overconfident before merging, and then apply calibration method to the merged model (Section 6.2).
PRELIMINARIES
Notation. Let f(::0) RC be a neural network for a C-class classification task, parame- terized by vector Rd. The network maps input vector E X' C RD C-dimensional logit vector. We target a multi-task scenario comprising T supervised tasks. Let 6pm E RR be the parameters of open-source pretrained backbone. For each task E ( T), we obtain model that has already been fine-tuned on the corresponding labeled dataset D, { vt,)) ] 1 i=1
yielding task-specific weights 0g E Rd Each label yt, E 1) is a one-hot vector indicating the ground-truth class.
2.1 MODEL MERGING FOR MULTI-TASE LEARNING
Recent model merging techniques operate on the task vectors (Ilharco et al., 2023) T, := 0: 0 pre and obtain single multi-task model by linearly combining them:
Omd
0 pre
+
Σ PL
where each P. Rdxd is method-specific matrix that mitigates inter-task interference.
In the following, we explain the P; used each merging method.
Uniform averaging: P. Ia.
Task arithmetic (Ilharco et al., 2023): P. de Ia, where de E R.
Tees-ngrging (Yadav et al., 2023): P. X mTies, Id. where X I. mTs.. (0, m determined by the norm each weight parameter to mitigate inter-task conflicts.
Consensus Merging (Wang et al., 2024): P. X McCos, Lai where x₂ E K, Mcoos.t {0, 1)d. The framework is the same as -eeginggg but the binary mask Mc determined in the following steps. First, create the TALL mask MTALL,I which is binary mask of weights where each element set to the norm of T, is larger than the weighted distance between Tr and ΣT Tr- Then, create McCunsd: where each element is set if the corresponding element of MTALLL in at least i tasks, reflecting agreement among the source models regarding the importance.
TSVM (Gargiulo al., 2025) cannot be expressed within the framework of Eq. (1). Instead, it suppresses task interference by whitening the matrices U, and V, obtained from the singular value decomposition of the task vectors T. U,T.V.
3

---

Published as a conference paper at ICLR 2026
2.2 KNOWLEDGE DISTILLATION
Knowledge distillation (KD) is a model compression and transfer paradigm in which a compact student network is trained replicate the behavior of a larger, well-performing teacher network (Hinton et al., 2015). By minimizing a joint loss that combines ground-truth supervision with a soft- target signal derived from the teacher's output distribution, the student acquires the teacher's dark knowledge, namely, fine-grained inter-class relationships encoded in the soft logits, while retaining substantially smaller parameter footprint. Formally, for a given input æ. let ž := f(x: ; Our) E RC and Zstu := f(x; su)) E IRC be the output logits from the teacher and student models, parameterized by Oirr E _d and Osu E Rd, respectively. The KD objective then augments the conventional cross- entropy loss LCE with softened Kullback-Leibler (KL) divergence term:
LXD (1 () LCuE (Zuu: v) + TTuTT KL. (a(Zza//T (uar) (Zau/Tau))
TTuu))
where a denotes the softmax, Tur, Tau 2 is the distillation temperature, and Ç E [0, balances hard versus soft supervision.
3 FAILURE MODES IN MODEL MERGING
3.1 TASK VECTOR NORM DISPARITY
We begin by demonstrating that differences task vector norms can severely impair model merging. In practical fine-tuning, practitioners select diverse hyperparameters, including learning rate, number of training steps, weight decay, and optimizer, each of which influences the distance between the final weights and their initialization, i.e. the norm of the task vector.
To quantify this effect, we fine-tuned CLIP models with Vision Transformers (ViTs) backbones, specifically ViT-B-32, on eight vision tasks as introduced in Section 5.1 with two learning rates, 10-5 (gray) and 10-1 (blue), and plotted the resulting task vector norms in Figure 1b. Across all tasks, observe 5-7x gap between the two settings. Crucially, the difference is not confined to any particular layer: parameter scales diverge consistently throughout the network, as demonstrated in Section E.
Figure la reports the corresponding merge performance. The gray bars denote the baseline where all eight tasks are fine-tumed with 10-5, while the blue bars show the average over eight experiments in each of which one task is replaced with a higher learning rate of 10-4 and the other seven remain unchanged. We measure performance using normalized accuracy. Injecting single high-norm task vector degrades every merging method, with losses of up to 14%. These results confirm that norm discrepancies pose a fundamental obstacle to robust task vector merging.
The detrimental effect of norm disparity on model merging can be explained with a straightforward theoretical analysis formalized as Proposition 1.
Proposition 1. Let T1, E Rdd with ||T2|| > 0, and define å == =||T|I//|TT/l Assume TI 1 T2. For Tmerge Ti T2.
cos( (Tmorga: T2)
21 -8,
cos(Tmergee Tmerges TI)
S 5.
VI + 8
vI + 82
Hence, when 8 < 1, the merge is nearly perfectly aligned with T2 while its aligument with Tı is at most (8))
Empirically, task vectors are observed be approximately orthogonal (Ilharco al., 2023); assum- ing orthogonality, we obtain Proposition 1. The proof is given in Appendix B. This result shows that the merged solution almost entirely inherits the directional characteristics of the high-norm task, while the contribution of the low-norm task vanishes up to O Under the Neural Tangent Kernel approximation (Jacot et al., 2018) A Af(x) the functional shift from pre-trained model is determined exclusively by the task vector's direction. Thus, the geometric dominance of the high-nom vector implies that the merged model functionally mimics the high-norm task while failing to preserve the low-norm task's knowledge, leading vere performance drop. Consequently, such norm disparity can cause severe drop in performance on the low-norm task and thereby degrade the overall effectiveness of the merged model.
fa; Oo + Tmerge) - fax; 0o) R3 Teerge (x; 0o),
4

---

Published as a conference paper at ICLR 2026
3.2 CON-CONFIDENCE SOURCE MODELS
We now show that low confidence constitutes a second, equally damaging failure mode. Para- doxically, models that are well calibrated can be fragile from the perspective of model merging; conversely, we argue that the more overconfident a source model is, the more robust it becomes to merging.
model's decisiveness can be quantified by the entropy of its predictive distribution. Using the same experimental configuration as in Section 3.1, we replaced the learning-rate manipulation with a single change: turning label smoothing on or off. Figure plots the resulting prediction entropies: the gray bars correspond to training without label smoothing, while the yellow bars use = 0.1. The vertical axis is logarithmic; with label smoothing the entropy increases by up to three orders of magnitude.
Figure (yellow bars) shows how this reduced confidence affects merging. all algorithms, the normalized accuracy decreases markedly by up 24% compared to the baseline without smoothing. This degradation exceeds that caused by norm discrepancies how harmful low-confidence source models can be. In short, routine training choices that alter confi- dence (e.g. label smoothing, Mixup, focal loss) can induce large swings in post-merge performance. These phenomena can also be supported from a theoretical perspective. (Appendix C)
the previous section, underscoring
KNOWLEDGE DISTILLATION FOR TASK VECTOR CONDITIONING
Here, we propose Distillation for Task vec- tor Conditioning (DisTaC) KD-based pre- conditioning method that eliminates the harm- ful effects of individual task vectors during model merging, as identified in Section 3.
Algorithm DisTTaC
Require: Pre-trained parameters Ppee. task tor Tr. scaling factor Ke, temperature pair (Ters Tsm), regularization weight 3, unlabeled dataset D. drawn from the distribution of task t, learning rate 7), number of steps K
4.1 TASK VECTOR NORM CONDITIONING
Ensure: Fine-tuned student parameters
1: 0g +- 0pe + KeT:
> Anchor point > Student initialization
2: 0 --0 0n
First, to correct task vector norm disparity, Dis- TaC harmonizes the norms while preserving single-task accuracy. is to adjust the norm by scaling the task vector, ic. replacing t; with RTT, using a scalar scaling
3: for k = 1,2..... K do
naive countermeasure 4: Sample mini-batch B, C DD
5:
L+
6:
for all æ; E B, do
factor K₁- Unfortunately, this constant rescal-
7:
Zier + f(x;; Ppe + T()
ing offers no guarantee of performance reten-
8:
Zuu f(x,; 0)
tion and can severely degrade accuracy relative to the pre-merge model.
9:
Sur + o (
10:
Sau 0 Zstu/ Tasu)
11:
L L + T stu KL( 8au)
We therefore propose to recover the lost perfor- mance through KD: starting from Opee + KyTE, we treat the pre-merge model as the teacher and distill its predictions into the rescaled student using only unlabeled data from the same task as the one underlying solely on unlabeled data, uses soft-target dis- tillation only, i.e., we fix 1 in Eq. 2, omitting the cross-entropy loss entirely.
12:
end for
13:
L
+-
B.
+ 3/100 0
0
1) VoL
Gradient step
15: end for
Since DisTaC relies
Although one might instead fine-tune 0pe + K;T, with labeled examples, obtaining a sufficiently large supervised corpus at merge time is typically impractical. By contrast, access to unlabeled data commonly assumed during model merging (Yang et al., 2024b; Yan et al., 2025; Yoshida et al., 2025), and KD imposes only mild additional requirements.
To prevent the task vector norm from drifting far from Opee + KeT, during KD, we include an & regularizer their difference, shown in Algorithm 1.
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of post-merge accuracy across fine-tuning configurations and the effect of DisTaC. Absolute accuracy is displayed in large font size, whereas normalized accuracy ap- pears in parentheses in smaller font. "Individual" denotes the average performance of the source models their respective tasks, and "MTL" represents the performance of conventional MTL. When the task vector norms diverge (Norm Mismatch) or the source models exhibit low confi- dence (Low Confidence), performance consistently degrades relative to the standard benchmark setting (Original). Under these conditions, DisTaC effectively pre-conditions the source models, achieving performance comparable to Original even in both stringent settings.
Method
Original
Norm Mismatch VIT-B-32
Low Confidence VIT-B-32
ViT-B-32
ViT-L-14
ViT-L-14
VIT-L-14
Pre-trained
47.3
65.1
47.3
65.1
47.3
65.1
Individual
89.9
93.7
89.3
93.3
89.8
94.0
MTT.
87.8
92.6
Task arithmetic Task arithmetic
(0.4
(8.0
63.6
(8.6
51.0(58.3)
66.9
DisTaC
(7.2)
83.9 (89.6)
63.6
(8..)
TIES TIES
74.0
99.0
59.1 (66.4)
(7.0))
54.5 (62.0)
68.3 (73.0)
DisTaC
73.1 (81.0)
84.4 (90.2)
68.7 (77.9)
79.4
Consensus TA
74.8 (828)
85.3 (90.7)
68.8
(2.0
54.6
(8.6
Consensus TA
DisTaC
73.7 (82.2)
84.9 (90.7)
67.7 (76.5)
80.0 (85.8)
EMR-Merging
93.0 (885
(8.0
(9.6)
39.2((45.)
27.4 (30.1)
EMr-Merging
DisTaC
88.1 (97.3)
92.7
(7.3)
92.3 (98.1)
TSVM
(3.3
90.5 (96.3)
72.2 (80.2)
84.5 (90.7)
60.7 (68.4)
71.6
TSVM
DisTaC
82.9 (91.8)
(9.36)
81.5
89.7 (96.2)
Iso-CTS Iso-CTS
81.0(89.)
(9.44)
78. (86.2)
90.8 (96.9)
(2.5
80.8 (86.0)
DisTaC
80.3 (88.9)
90. (96.1)
69.0(78.D
86.1 (91,5)
WUDI-Merging
83.5
91.7 (9.7))
49.2
57.9 (60.8)
(400))
28.0
WUDI-Merging
DisTaC
84.4 (93.2)
91.4
73.8 (83.3)
91.6
4.2 SOURCE MODEL CONFIDENCE CONDITIONING
To mitigate low-confidence issues, DisTaC aims to increase each source model's confidence before merging, thereby rendering the model more robust to the merge. Here the student and the teacher are identical at initialization, i.c. 0, = Ppc + T:. We set the student temperature Tam higher than the teacher temperature Tir so that the student, trained on higher-entropy distribution, is pushed toward a lower-entropy (more confident) output when the temperature is later reset to 1. Consequently, the distilled student becomes more confident than its teacher.
One may worry that the over-confidence harms model reliability in practice. However, standard post- hoc calibration methods (e.g. temperature scaling) can mitigate over-confidence, whereas merging with an underconfident model leads to large performance drops that make the merged model im- practical. detailed discussion appears Section 6.2.
Unified algorithm. The two conditioning strategies above are unified by Algorithm 1. When both norm disparity and low-confidence coexist, they can be mitigated simultaneously by choosing an appropriate sealing factor Ke and temperature pair (Trar, Tan).
5
EXPERIMENT
5.1 SETUP
We conducted experiments in multi-task setting following Ilharco et al. (2023). Specifically, we adopted eight vision tasks: Cars (Krause et al., 2013), DTD (Cimpoi et al., 2014), EuroSAT (Helber et al., 2019), GTSRB (Stallkamp et al., 201 1), MNIST (LeCun, 1998), RESISC45 (Cheng et al., 2017), SUN397 (Xiao et al., 2016), and SVHN (Netzer et al., 2011). Our models applied VIT- B-32 and ViT-L-14 to CLIP. We evaluated post-merge performance using absolute accuracy and normalized accuracy under the two aforementioned failure modes: the case with diverged task vector norms (Norm Mismatch) and the case with low-confidence source models (Low Confidence).

---

Published as a conference paper at ICLR 2026
Relative ACC Teacher (%)
120
2.0
Relative ACC to Teacher (%)
100
-2.0
Prediction Entropy
100
-1.6
1.5
Norm Increstse
80-
1.0
1.2
70-
11.0
60
-0.0
60
0.8
100
200
300
400
500
100
200
300
400
500
DisTac Steps K
DisTac Steps
(a) Norm Mismatch
(b) Low Confidence
Figure 2: Evolution of DisTaC over steps. Results are averaged over the eight vision tasks with ViT-B-32; the eгrоr band shows one standard deviation around the mean. (a) Norm Mismatch: the blue curve plots normalized test accuracy relative to the teacher, and the green curve shows the percentage change in the task vector norm from the DisTaC initialization. Within roughly 100 steps, accuracy recovers to (or exceeds) the teacher's level while the task vector norm remains virtually unchanged from its Ke-adjusted target. (b) Low Confidence: the blue curve again reports nor- malized test accuracy, whereas the orange curve tracks the test prediction entropy. About 100 steps suffice to drive the entropy substantially lower, yet the teacher-level accuracy fully preserved.
The detailed settings for each scenario followed those described in Section 3. We adopted seven merging methods as baselines: task arithmetic (Ilharco et al., 2023), Ties-Merging (TIES) (Yadav al., 2023), Consensus Merging (Consensus TA) (Wang al., 2024), MerR-Merging (Huang al., 2024), TSVM (Gargiulo et al., 2025), Iso-Merging (Iso-CTS) (Marczak et al., 2025), and WUDI- Merging (Cheng al., 2025). For TaC, knowledge distillation was run for 500 steps. In the Norm Mismatch regime we assign task- scaling coefficient individually for each of the eight norm- disp-iipartty configurations: the task vector with the largest f--norm rescaled so that, after scaling, its norm equals the mean norm of the remaining seven task vectors. A neutral temperature pair is then used, (Tier, Tu)) = (10, 10). In the Low Confidence regime we instead fix K₁ = 1 and sharpen the student by adopting more asymmetric temperature pair, (Tirr Tam) (1, 10). More detailed settings can be found Appendix D.
5.2 RESULTS
5.2.
MERGING PERFORMANCE
Table 1 summarizes the results. Absolute accuracy is displayed in larger font, whereas normalized accuracy appears in parentheses in a smaller font. As noted in Section 3, all methods exhibit a substantial and consistent performance decline relative to the conventional configuration (Original) under both failure modes, revealing clear vulnerability (white rows). The rows highlighted in gray show the performance obtained by first applying DisTaC for pre-conditioning and then merging. DisTaC consistently enhances merge performance, yielding gains of up to 35.8% absolute accuracy for ViT-B-32 and 63. 63.6% for ViT-L-14. Moreover, for EMR-Merging which achieves the highest merge performance, DisTaC raises the accuracy under both failure modes to a level comparable with the Original configuration in most cases, indicating that the intended merge performance robustly maintained even in challenging scenarios.
5.2.2 EFFICIENCY OF DISTAC
Here, present how the single-task performance each task, the task vector norm, and the pre- diction entropy change during the KD process DisTaC, as well as the computational cost required for sufficiently thorough training.
Figure/ shows the average over eight vision tasks of the training history when KD by DisTaC is applied to ViT-B-32. The blue curve denotes the test accuracy relative to the teacher's test accuracy, the green curve the task vector norm relative to its value at the initialization point, and the orange curve the test prediction entropy.
7

---

Published as a conference paper at ICLR 2026
First, Figure 2a depicts the training history under the Norm Mismatch setting in Table 1. It achieves performance comparable to, or even surpassing, the teacher model's 's test performance within 500 steps, while the l regularizer of DisTaC keeps the task vector norm roughly 1.1x that of the initialization point, Pre + KrT, at the end of the 500 steps.
Of partieular interest is that DisTaC occasionally surpasses the teacher model's test performance. We identify two factors underlying this phenomenon. The first is the scale given by 6ę. In particular, observed that reducing Kę can sometimes improve generalization performance. That is, the DisTaC initialization point already outperforms the teacher model, and we observed this in every instance in which the teacher model was exceeded. This phenomenon of the student outperforming the teacher is confirmed in (Furlanello et al., 2018), where it has been shown that student can surpass the teacher by repeating KD between identical architectures. Furthermore, in this case, since KD is performed while keeping the student's 's norm smaller than the teacher's, s, it is plausible that a regularization effect similar to weight decay is being exhibited.
Next, Figure 2b presents the training history under the Low Confidence setting in Table 1. Within 500 steps, particularly during the first 100 steps, it achieves substantial reduction in prediction entropy while maintaining test accuracy at level nearly equivalent to that of the teacher model.
DISCUSSION
6.1 STRETCHING VS. SHRINKING TASK VECTORS
When task vectors differ significantly in norm, a natural question arises: Should shorter vee-
120
Relative ACC to = =10 (%)
tors be stretched to match longer ones, should longer vectors be shrunk to match the
100
shorter ones? Our findings support the latter;
we advocate shrinking the longer vectors.
There are several reasons for this. First, it is conceivable that model performance is more ro- bust to scaling down a task vector than scaling it up. Figure 3 shows how test accuracy varies across vision tasks when applying different scaling factors Kę to the task vector, i.e., eval- uating Ope + KrT for K2 E 3.0). Shrinking the task vector (Ke < 1.0) retains performance comparable to or even better than the original fine-tuned model across broad range. In con- trast, stretching beyond Ki = 1.0 degrades ac- curacy, and by Ki 3.0, the model under- performs even the zero-shot bascline across all tasks. A similar trend was also observed for ViT-L-14 (see Section E.3).
40
20
0
Opre
0,
2.5
Cars
DTD RESISC45
EuroSAT
GTSRB
MNIST
Figure 3: Effect of scaling task vectors on test accuracy. For each of the eight vision tasks (VTT- B-32), we evaluate the model Opre + KrT as the scaling factor Kt varies from 0.0 3.0. Model performance is more robust to shrinking the task vector than to stretching harmonizing task vector norms, longer vectors should be shrunk match shorter ones.
suggesting that when
As shown earlier in Figure 1b, real-world fine- tuning pipelines often result in over 5x varia- tion task vector norm due differing learn- ing rates or training durations. In such cases, stretching small-norm vectors to match larger ones risks disrupting the pretrained model's useful representations and therefore undesirable.
Furthermore, Ilharco et al. (2023) observed that merging task vectors with smaller norms tends to yield better performance. A likely explanation that smaller displacements remain within the local linear regime around Ope, where first-order approximations hold more accurately. This also aligns with the NTK perspective discussed Ortiz-Jimenez et al. (2023); Yoshida et al. (2025), under which merging remains valid and weight disentanglement is preserved near the pretrained initialization. Notably, Theorem 3.1 Wei et al. (2025a) demonstrates that the performance gap between the merged model and the fine-tuned model is proportional the product of the learning
8

---

Published as a conference paper at ICLR 2026
rate and the number of fine-tuning steps. This theoretical insight aligns with our claim that shrinking task vectors preferable.
Taken together, these observations strongly suggest that when normalizing task vectors for merging, it is preferable to shrink the longer ones rather than stretch the shorter ones.
6.2
CONFIDENCE RELJABILITY IN MODEL MERGING
100
80
78.0
80
62.4
Norm. ACC (%)
60
58.3
Test ACC (%)
40
20
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.01
0.1
Confidence
0.01
0.1
Perfectly Calibrated
(a) Reliability diagram over different label smoothing (b) Normalized accuracy over diffcrent label smooth- strengths
ing strengths
Figure 4: Impact of label smoothing confidence calibration and merge performance. (a) Av- erage reliability diagram for ViT-B-32 across eight vision tasks under different label-smoothing strengths Without label smoothing (a O, dark purple) the model is strongly overconfident; as a increases to 0.01 the model becomes well-calibrated, and at a 0.1 it turns underconfident. (b) Test normalized accuracy obtained when the corresponding source models are merged. Merge performance decreases monotonically with larger revealing clear trade-off: lower confidence comes the cost of lower accuracy after merging.
As noted in Section 3.2, successful model merging often conflicts with maintaining reliable confi- dence estimates both the source and merged models. Figure illustrates this trade-off by sweeping the label -abel-smoothing strength a used during fine-tuning of the source models.
First, the calibration curves in Figure 4a show that model trained without label smoothing (dark- purple line) is strongly overconfident, which is consistent with the well-known tendency of modern deep networks (Guo al., 2017). As a increases from 0.01 to 0.1 (red yellow), the models become well-calibrated and eventually underconfident, matching the observations of Müller et al. (2019). Figure 4b then reports the normalized accuracy obtained when these source models are merged. Accuracy decreases monotonically with larger a. revealing inverse correlation between subel-smoothing strength and merge performance.
In short, current merging methods perform best when the source models are deliberately overcon- fident. To retain reliable confidence after merging, we therefore advocate applying post-hoc cali- bration, such temperature sealing (Guo al., 2017), to the merged model rather than trying to calibrate the sources beforehand.
LIMITATION
While our main experiments are primarily limited to vision tasks using CLIP, we demonstrate in Appendix 6 the significance of each failure mode and the effectiveness of DisTaC in NLP tasks. However, since our evaluations both domains are exclusively restricted to classification tasks, ex- tending framework generation tasks and other modalities remains highly critical direction

---

Published as a conference paper at ICLR 2026
for future exploration. Additionally, rather than exploring all possible causes of task interference, we specifically focus on the two main failure modes: norm disparity and low source-moder confi- dence. Furthermore, DisTaC assumes access unlabeled data for distillation, which can at times be challenging due to potential security constraints. Nevertheless, we emphasize that DisTaC achieves 96% of ideal performance even when using extremely small datasets data with severe dis- tribution shifts, demonstrating strong robustness in such settings (see Appendix E.5). Furthermore, other approaches, such as Yang al. (2024b); Yan et al. (2025), also rely on the availability of unlabeled data. Despite these limitations, we believe that our experiments directly support our main claims on failure modes and are sufficient to demonstrate the effectiveness of our approach.
CONCLUSION
We presented DisTaC, lightweight and practical pre-conditionin method for task vectors that im- proves the robustness of model merging in multi-task learning. Our analysis identified two major failure modes of norm disparity and low cource-model confidence that frequently occur in real-world merging scenarios. DisTaC addresses both issues simultaneously via KD) on unlabeled data, requir- ing only minimal computational cost and no access to task labels. Through extensive experiments, demonstrated that TaC not only recovers performance degraded by task vector scaling, but also enhances confidence in the source models without sacrificing generalization. Furthermore, we showed that DisTaC enables state-of-te of-th---rtttttttt merging methods to succeed in challenging cases where they would otherwise fail. Our findings highlight the importance of task vector conditioning, and believe that DisTaC provides simple yet powerful tool make model merging more reliable and broadly applicable.
ACKNOWLEDGEMENT
Our deepest gratitude goes out to the anonymous reviewers whose invaluable insights substantially enhanced the quality of this manuscript. This work was supported by RBC Borealis through the RBC Borealis AI Global Fellowship Award, which was awarded to Hiroki Naganuma. The computation resource of this project is supported by "TSUBAME Encouragement Program for Young/Female Users" of Center for Information Infrastructure at Institute of Science Tokyo and by "Joint Us- age/Research Center for Interdisciplinary Large-scale Information Infrastructures" Japan.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint 230.::333000774,, 2023.
Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, pp. 1-10, 2025.
Keshigeyan Chandrasegaran, Ngoc-Trum Tran, Yunqing Zhao, and Ngai-Man Cheung. Revisiting label smoothing and knowledge distillation compatibility: What was missing? In International conference on machine learning, pp. 2890-2916. PMLR, 2022.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Bench- mark and state of the art. Proceedings of the IEEE, 105(10):1865-188) 1883, 2017.
Runxi Cheng, Feng Xiong, Yongxian Wei, Wanyun Zhu, and Chun Yuan. Whoever started the interference should end it: Guiding data-free model merging via task vectors. In International Conference on Machine Learning, 2025.
Leshem Choshen, Elad Venezian, Noam Slonim, and Yoav Katz. Fusing finetuned models for better pretraining, arXiv preprint arXiv:2204.03300 2022.
Mircea Cimpoi. Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEÉE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
10

---

Published as a conference paper at ICLR 2026
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics:: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: //openreview net. forum? d=Yi cbFdNTTy.
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Bomn again neural networks. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1607-1616. PMLR, 10-15 Jul 2018. URL https:/ petpps:/pproceell ngs mmlr. press/v80 fur anel 1 o 8a html.
Antonio Andrea Gargiulo. Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, and Emanuele Rodola. Task singular vectors: Reducing task interference in model merging. In Proceedingy of the Computer Vision and Pattern Recognition Conference, pp. 18695- 18705, 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Zhiwei Hao, Jianyuan Guo, Kai Han, Yehui Tang, Han Hu, Yunhe Wang. and Chang Xu. One-for- all: Bridge the gap between heterogeneous architectures in knowledge distillation. In A. Oh, T. Naumann, A. Globerson, K. Saenko. M. Hardt, and S. Levine (eds.), Advances in Neu- ral Information Processing Systems, volume 36, pp. 79570-79582. Curran Associates, Inc., 2023. URL https://proceett nggs neurips ee/ pappraaperr.a http:/pper____eeeeee files/ pates//ppp//2223//// file/ rb885f199c7a5ccc443333333
.df.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):211722226, 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv 2015.
preprint arXiv:1503.022
Roger Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Chenyu Huang, Peng Ye, Tao Chen, Tong He, Xiangyu Yue, and Wanli Ouyang, Emr-mergree tanrmeeginn: Tuning-free hegh-perffrmance model merging. arXiv preprint 240.: 2024.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2023.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Advances in Neural Information Processing Systems, 31, 2018.
Xisen Jin, Xiang Ren, Daniel Preotiuc-Piete Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights language models. In The Eleventh International Conference on Learning Representations, 2023. URL https: /oweorvveeeeeeeeeeeeeeeeeeee net forum? f
Jangho Kim, Seonguk Park, and Nojun Kwak. Paraphrasing complex network: Network compres- sion via factor transfer. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- -isa-B,, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Cur-
ran Associates, Inc., 2018, URL httttt:ss/ppccceeiinne neurips. cttpo/scccppppppppeeeeeeeee files/ paper/2018/Fr fi e/ 6d9cb http:///eee..t.......e..eaaaeaa.

---

Published as a conference paper at ICLR 2026
Masanari Kimura and Hiroki Naganuma. Geometric insights into focal loss: Reducing curvature for enhanced model calibration. Pattern Recognition Letters, 189:195-200 2025.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference computer vision work- shops, pp. 554-561, 2013.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mmist// 1998.
Lu Li, Tianyu Zhang, Zhiqi Bu, Suyuchen Wang, Huan He, Jie Fu, Yonghui Wu, Jiang Bian, Yong Chen, and Yoshua Bengio. MAP: Low-compute model merging with amortized pareto fronts quadratic approximation. In The Thirteenth International Conference on Learning Representa- tions, 2025. URL https:::p//... /ppp/reeeeeeeeeeeeeeeeeeeeeeeeee net/ forum? ? id=177sRWsvve.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp- 2980-2988, 2017.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence on Learning Representations, 2019. URL https: http////ppeeeeeeeeee/eeee www.tee/tteeeee forum? id= /
Bkg6RiCqY7.
Sihui Luo, Xinchao Wang, Gongfan Fang, Yao Hu, Dapeng Tao, and Mingli Song. Knowledge amalgamation from heterogeneous networks by common feature learning. In Proceedings the 28th International Joini Conference on Artificial Intelligence, IJCAI'19, pp. 3087-3093. AAAI Press, 2019. ISBN 9780999241141.
Daniel Marczak, Simone Magistri, Sebastian Cygert, Bartlomiej Twardowski, Andrew D. Bagdanov, and Joost van de Weijer. No Task Left Behind: Isotropic Model Merging with Common and Task- Specific Subspaces. International Conference on Machine Learning, 2025.
Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703-17716, 2022.
Ranjith Merugu, Bryan Bo Cao. and Shubham Jain. Statsmerging: Statistics-guided model merging
via task specific teacher distillation, 2025. URL
httpaaawwww....aa//////aaaa/a/// 2506. 04567.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances in neural information processing systems, 32, 2019.
Hiroki Naganuma, Kotaro Yoshida, Laura Gomezjurado Gonzalez, Takafumi Horie, Yuji Naraki, and Ryotaro Shimizu. On fairness of task arithmetic: The role of task vectors. arXiv preprint 2rxiv:2505.24262. 2025.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised featur learning. Granada, 2011.
Guillermo Ortiz-Jimenez, Alessandro Favero. and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Sys- tems, 2023. URL https: / wpw.revieeweweee////pee.. net forum? d- 0A9 f2 JZDGW.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image syuthesis with latent diffusion models. In Proceedings the the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter, 2020. URL 01100 10.
org/ abs/1910.
12

---

Published as a conference paper at ICLR 2026
Zhiqiang Shen, Zechun Liu, Dejia Xu, Zitian Chen, Kwang-Ting Cheng, and Marios Savvides. Is label smoothing truly incompatible with knowledge distillation: An empirical study. In Interna- tional Conference Learning Representations, 2021. URL https: http:////ppeeeiiieeeeee nnt// forum? dePObuuGVrGa..
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: a multi-class classification competition. In The 2011 international joint conference on neural networks, pp. 1453-1460. IEEE, 2011.
Hsuan Su, Hua Farn, Fan-Yun Sun, Shang- Tse Chen, and Hung-yi Lee. Task arithmetic can mitigate synthetic-to-real gap in automatic speech recognition. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 8905-8915, Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: 10. 100118663/112224.mmml main. 503. URL https: htt.aaaaaaaaaaa http:///taaaaacooacaccoooo org 2024. amnnp-maing 503/.
Anke Tang, Li Shen, Yong Luo, Nan Yin, Lefei Zhang, and Dacheng Tao. Merging multi-task mod- els via weight-ensembling mixture of experts. In Proceedings of the 41st International Conference on Machine Learning. 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instloction-oolloiinn model. Stanford Center for Research on Foundation Models, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.099: 092888 2023.
Joachim Utans. Weight averaging for neural networks and local resampling schemes. In Proceedings of the AAAI Workshop on Integrating Multiple Learned Models, pp. 133-138, 1996.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: multi-task benchmark and analysis platform for natural language understanding. In International Conference Learning Representations, 2019. URL https: ctttps://openreviiie /oppnreview. net/for forum?
Ke Wang, Nikolaos Dimitriadis, Guillermo Ortiz-Jiménez, François Fleuret, and Pascal Frossard. Localizing task information for improved model merging and compression. In International Con- ference Machine Leurning, 2024.
Yongxian Wei, Runxi Cheng, Weike Jin, Enneng Yang, Li Shen, Lu Hou, Sinan Du, Chun Yuan, Xiaochun Cao, and Dacheng Tao. Unifying multimodal large language model capabilities and modalities via model merging. arXiv preprint 250i:::2005.9892 2025a.
Yongxian Wei, Anke Tang, Shen, Zixuan Hu, Chun Yuan, and Xiaochun Cao. Modeling multi-task model merging adaptive projective gradient descent. In Forty-second International Conference on Machine Learning, 2025b. URL https: http///pppee///eeee / net wtt/forum?? EqoKRSRSPa.
Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training procedure in timm. arXiv preprint arXiv:2/ 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. Huggingface's transformers: State-of-thh-are natural language processing. arXiv preprint 1900::1110 2019.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the International Conference Machine Learning. volume 162, pp. 13965-23999 PMLR, 2022a. URL https: ttttepp:/pproceeiinee /proceedings. mlr. alr.press/vl w2t.1//ttwttta////e html.
13

---

Published as a conference paper at ICLR 2026
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 7959-7971, 2022b.
Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva. Sun database: Exploring a large collection of scene categories. International Journal of Computer Vision, 119; 3-22, 2016.
Yangyang Xu, Yibo Yang, and Lefei Zhang. Multi-task learning with knowledge distillation for the dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 21550-21559, 2023.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-merging: Re- solving interference when merging models. Advances in Neural Information Processing Systems, 2023. URL https: : /wpp//eeeeeeeeeeeeeeeeee/e
Kunda Yan, Min Zhang, Sen Cui, Qu Zikun, Bo Jiang, Feng Liu, and Changshui Zhang. CALM: Consensus-aware localized merging for multi-task learning. In Forty-second International Conference on Machine Learning, 2025, URL https: hatpss://ppeeeveeeee net forum?id-
OgfvSDn73E.
Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in Ilms, milms, and beyond: Methods, theories, applications and opportunities. arXiv preprint 0oXiv:2408.07666 2024a.
Enneng Yang. Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adaptive model merging for multi-task learning. In The Twelfth International Con- ference Learning Representations, 2024b. URL https:/ oppnreviewwweeeee/eeeeepeeeeeeee net forum? id-n22 6NgD3QX.
Kotaro Yoshida, Yuji Naraki, Takafumi Horie, Ryosuke Yamaki, Ryotaro Shimizu, Yuki Saito, Ju- lian McAuley, and Hiroki Naganuma. Mastering task arithmetic: $ tauSjj key indicator for weight disentanglement. In The Thirteenth International Conference on Learning Representa- tions, 2025. URL https::///// https:://ppereeeeeeee
net forum? dd=1iwii 6zbxs.
Yuya Yoshikawa, Ryotaro Shimizu, Takahiro Kawashima, and Yuki Saito. Transferring visual ex- plainability of self-explaining models through task arithmetic. arXiv preprint arXiv:2507.0438 043300 2025.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXi:: 2017.
Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), October 2019.
Zhilu Zhang and Mert Sabuncu. Self-distillation as instance-specific label smoothing. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances Neu- ral Information Processing Systems, volume 33, pp. 2184-2195. Curran Associates, Inc., 2020. URL bttps://proceedings neurips.........pp cc cc/paper............oe. wwwes//appere//// Paper. pdf.
Kaixiang Zheng and EN-HUI YANG. Knowledge distillation based on transformed teacher match- ing. In The Twelfth International Conference on Leurning Representations, 2024. URL https: / //opereeview ew.. net. forum? d-MM3337DDGGGG
Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. A robustly optimized BERT pre-training approach with post-training. In Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Kang Liu, Wanxiang Che, Shizhu He, and Gaoqi Rao (eds.), Proceedings of the 20th Chinese National Conference on Com- putational Linguistics, pp. 2218-122, Huhhot, China, August 2021. Chinese Information Pro- cessing Society of China. URL https: /ttae:://llllaaaa www.caannnooooyoooooooooooooo .rgg or0/2021 cal 1-1 -1. .088..
14

---

Published as a conference paper at ICLR 2026
A RELATED WORK
A.1 MODEL MERGING AND TASK ARITHMETIC
Research on integrating multiple neural network models by performing operations on their param- eters has been widely conducted, starting with Utans (1996). These techniques enable a model to learn diverse tasks with less time and computational resources, and have become increasingly im- portant in recent years as the number of model parameters has grown dramatically. For instance, in early approaches to model merging. models with the same architecture were fine-tuned and then merged by averaging their parameters (Wortsman et al., 2022a; Choshen et al., 2022). More sophis- ticated methods have since been proposed, such as Fisher Merging (Matena & Raffel, 2022), which based on maximizing the posterior probability of the model, and RegMean (Jin et al., 2023), which minimizes the distance between output activations before and after merging. In contrast, task arithmetic (Ilharco et al., 2023) focuses on the task vector, defined as the difference in parameters between fine-tuned model and pre-trained model, and performs addition and subtraction of task vectors in parameter space. This approach offers the advantage of allowing flexible, localized mod- ifications the model and has found applications across diverse tasks (Tang et al., 2024; Su al., 2024; Yoshikawa et al., 2025; Naganuma et al., 2025).
Recent research on task arithmetic has theoretically analyzed the simple addition of task vectors and proposed multiple methods to address its shortcomings. Approaches aimed at improving the prop- erties of task vectors focus on the linearity in fine-tuning (Ortiz-Jimenez et al., 2023: Yoshida al., 2025). These methods, based on the Neural Tangent Kernel (NTK) (Jacot et al., 2018), treat the model's output as linear during fine-tuning in order to reflect vector operations in parameter space onto the model's inputs and outputs. Meanwhile, several studies have been conducted from the perspective of mitigating interference between task vectors. TeES-Megging (Yadav et al., 2023) em- phasizes the removal of redundant elements and the consideration of sign in each vector dimension. AdaMerging (Yang et al., 2024b), on the other hand, automatically adjusts merging coefficients per task and per layer to reduce task interference and enhance robustness through test-time adaptation. Wang et al. (2024) introduced a framework for pinpointing the parameters that carry information shared across tasks and, on that basis, proposed Consensus Merging, which builds task-wise masks that align more closely with inter-task consensus than the masks used in TeES-Merging. While traditional multi-objective optimization can be computationally prohibitive, Li et al. (2025) amor- tized this cost by leveraging quadratic approximations to identify diverse Pareto-optimal merging solutions. More recently, Wei et al. (2025b) reformulated model merging as minimizing the loss gap between the merged model and each task-specific model, introducing DOGE with subspace projection and task-aware scaling. TSVM (Gargiulo et al., 2025) interprets task interference as non- orthogonality among the layer-wise singular vectors of the task vectors; by whitening those singular directions, TSVM further improves merge quality.
Despite these advances, Ilharco et al. (2023) and nearly all follow-up studies on multi-task model merging benchmark their methods under highly idealized settings, leaving real-world failure modes largely unexplored. In this work, we show that (i) discrepancies in task vector norms and (ii) low source- model confidence are key sources of interference. We introduce DisTaC as a simple pre- conditioning step that mitigates both problems before merging.
A.2 KNOWLEDGE DISTILLATION
DisTaC addresses the limitations of existing task arithmetic methods by incorporating knowledge distillation. Knowledge distillation is technique proposed for transferring knowledge from a teacher model to a smaller student model (Hinton et al., 2015). Although initially intended for model compression (Hinton et al., 2015; Kim et al., 2018; Sanh et al., 2020), it has also been ap- plied in contexts such as self-distillation, where repeated distillation between models of the same architecture leads to performance improvement (Furlanello et al., 2018; Zhang et al., 2019; Zhang & Sabuncu, 2020). Among these applications, several studies have explored generating models that can handle multiple tasks by distilling knowledge from single or multiple teacher models (Luo et al., 2019; Hao et al., 2023; Xu et al., 2023). These approaches achieve distillation by mapping the pa- rameters of multiple teacher models into a shared space for the student model. Conversely, it is also possible to distill models with different architectures individually to obtain task vectors, which can
15

---

Published as a conference paper at ICLR 2026
then be merged using task arithmetic (Merugu al., 2025). DisTaC adopts the latter approach and resolves the issue of variability in the norms of task vectors by obtaining them through distillation.
Applying distillation to task arithmetic requires addressing the impact of soft targets. Numerous studies have analyzed the effects of label smoothing in the context of knowledge distillation (Müller al., 2019; Shen al., 2021; Chandrasegaran al., 2022; Zheng & YANG, 2024). In this study, we demonstrate that fine-tuning with soft targets significantly affects the models obtained through model merging, and propose a method to mitigate this effect by increasing the confidence of the student model.
B PROOF FOR PROPOSITION
Let 1-TT|////TTal and assume Ti T2. Then
Tmergell || = || 12 = ||T1|1
||2/| 2 = (1 + 111
For the cosine similarity with T2. we compute
T2 1
Tmerge
cos( Tmergee T2)
I T2ll Using the inequality (1 + 1-822/112
T2
v
882 for O, we obtain the lower bound.
Similarly, for the cosine similarity with T1.
cos( Tmerge: TI)
| Tmerge T V(1+ 82) 1 12
Tmerge TI
8 /1 + 82
Since 8/VI+002 < 6, the claim follows.
Hence, when å < 1, the merged vector is nearly aligned with T2 while its alignment with is suppressed by a factor of (6)).
C
THEORETICAL INSIGHTS INTO TASK VECTOR MERGING FOR MODELS OPTIMIZED WITH DISTINCT OBJECTIVES
This appendix provides step-by-ster derivation of the theoretical results concerning the effect of calibration penalties on the arithmetic merging of task vectors. We demonstrate how calibration can introduce first-order degradation in cross-entropy (CE) performance upon merging, an effect not observed when merging standard CE-trained task vectors.
C.1 NOTATION AND ASSUMPTIONS
We use the following notation. For task i, the standard cross-entropy (CE) objective is
( == E [log Po(y) x=)]
We also consider calibrated objective that augments CE with a generic penalty (,)) weighted by A 0:
JCAL(()) == (0) X, C.)).
For either objective + E {CE, CAL), the task task-specific optimum is denoted
0 = arg min (0).
Throughout, we assume the objectives J.E and J.AL are C2 in a neighborhood of a fixed base pa- rameter 0,. Let H, := 22 denote the CE Hessian at Og and assume H; is positive-definite ensuring that 0, lies in locally convex region of the CE landscape. For notational convenience write the gradients at 0n as
CE (0o)
b, == (Oo).
For example, detailed description of evaluating focal loss be found Kimura & Naganuma (2025).
16

---

Published as a conference paper at ICLR 2026
C.2 THEORETICAL PRELIMINARIES FOR THE MAIN RESULT
C.2.1 DERIVATION OF THE STANDARD TASK VECTOR
The optimal parameter vector d for the standard cross-entropy loss satisfies the first-order opti- mality condition, which states that gradient at this point zero.
V. CE CCE) O.
Using the definition of the task vector, we can write 0, = 0u + -.. Substituting this into the optimality condition yields:
+ (E) 0.
(4)
We now perform first-order Taylor series expansion of the gradient function (.) around the point Og.
(0o + V. (0,) ) + v2 Je (Ga)T:
CE + +(1TT (12))
(5)
Using our established notation for the gradient (g.) and the Hessian (H,) at Og. this becomes:
+ H,T, CE + (IIT, 0.
(6)
For fine-tuning scenarios where the task-specific solution o is close to the pre-trained model On. the norm of the task vector ||T|B||| is small. We can therefore neglect the higher-order terms.
[
H CE x O.
g:
Since H; is assumed to be positive-definit, it is invertible. We can solve for the task vector CE.
H,T CE
Ti
(8)
which gives the well-known result from single Newton-Raphsen step:
CE -H, g:-
C.2.2 DERIVATION OF THE CALIBRATED TASK VECTOR
We now apply the same procedure the calibrated objective function (C)
Gradient and hessian at the base point. First, compute the gradient and Hessian of J (CAL(()
at the base point Og. The gradient is:
V CALL = (J + A,C((O))
(10)
0=00
(. + (C, g: + AA,;
(11)
(12)
Let A, := (l,) be the Hessian the calibration term. The Hessian of the calibrated objective, which we denote by H, is:
H, = C2L = 2 (J (,C,(O))
(13)
8=
V²J,CE(O)) - C.C
(14)
H. + A,A,
(15)
Neumann series expansion of म, 1 To solve for the calibrated task vector T CAL we need the inverse of the calibrated Hessian, H For a small penalty weight di we can approximate this inverse. We begin by factoring out H;:
H H, + A; H; (I H, (M,A)) H. AH A.)
(16)
The inverse is then given by H. (I A We can expand the term (T A using Neumann series (Hom & Johnson, 2012), (T x) (-x(-
17

---

Published as a conference paper at ICLR 2026
which converges if the spectral radius of X is less than 1. Assuming X, is sufficiently small such that 1 A, < 1, we have:
(I - (A.H, A.)²
(17)
=I- HH, A
O
(18)
Substituting this back into the expression for 17;1:
H = A,, A, + ((
(19)
- - +(A)))
(20)
Solving for T CAAL. The calibrated task vector - CAL is found by applying the first-order optimality
condition to J JCAL and linearizing around 0y:
JJ (O (.CCAL(())) + V2 (O)) CAL + C[|TT:AL11 O.
(21)
Using the expressions from BI and ignoring higher-order terms:
gg + A,b:)) + CATLCL x O.
(22)
Solving for CAL gives:
CAL ~ 11, 1 (g. + A,b:). T
(23)
Now, we substitute the approximation for H from equation 20:
Ti CAL (H A,,, A,H, + (( (g, + A.b.)
(24)
(H X, b, X,H, A, g; NH A, A,H, bb.) + O(A))
(25)
- 11, ' X,H,
A,H, + O(
(26)
We recognize the first term as the standard task vector, T -11 111111.. g: The expression becomes:
CAL = =
CE X, b. + A,H, A,H, g: + OAA))
(27)
In many practical scenarios, especially after extensive pre-training, the initial gradient norm Ig. is small. Consequently, the term A; A; A, g,, which is of order O(Aillg:l)). is often negligible compared to the term -, which is (Ass). Under this simplifying assumption, we can define the first-order correction due to calibration as:
6, = X; H, b,
(28)
This allows us to express the calibrated task vector as a simple perturbation of the standard task vector:
CAL T, + O0A
(29)
C.2.3 TASK VECTOR MERGING
We consider merging two task vectors using simple linear combination with positive coefficients a, B > O. We define two types of merged parameters:
f
merge := 0 + at CE + 1TT
(30)
CAL merge : O, + at
CAL + +BBT. CAL
(31)
Taylor expansion of the CE loss for merged vectors. Our goal is to evaluate the CE loss JCE not at its own optimum, but at the merged parameter points. We use second-order Taylor expansion of J CF around On:
CE (0) J (Qu) = g.. -g 0b) + 2
(e
TT,TH.
H. 0b) + o 0b1 (32)
18

---

Published as a conference paper at ICLR 2026
Merging of CE vectors.
Let AOCE
O merge
8g aTi 3T2
The change in CE loss for
task i is:
( merge)
B (0,) g (aTi + :3= O(1
(33)
Let's analyze the first-order term in the expansion. Using g: = -
CB from equation 7:
g (aT) + ) = ag, 7I T + 5g. T
(34)
= =a(- HiT, =a(-HiT, (B) T CE
+ 3( H CE T
72
CE
(35)
-a(Ti CE) H
CE B(; CB) H:T2 CE
(36)
The term for task i itself (i = 1 and analyzing -| or = and analyzing =2 is (
-al -. 122 Since H, positive-definite, this self-term is strictly nega- tive. The cross-term's s sign is indefinite. However, the dominant contribution to the loss change is
typically negative and of order [[[ indicating that merging CE vectors does not increase the loss first order.
Merging of calibrated vectors. loss is:
Let AOCAL OCAL merge
CAL + ST2 CAL, The change in 0u = aTT
CCFOOCAT - -((R(OO) = 8. (AT1
merge)
CAL + BTZ + 011
(37)
We substitute
CAL T +ōz;
g (aT)
CAL + BT CAL) 2 gi (a(T) + 81)
CE + +52)) B(T2
(38)
(AT + BT2
CE
CF. + (g. [ggT5;, 8,) + 3 82)
(39)
Original term. +1
Additional term. ((1[[=])
Let's analyze the additional term introduced by calibration. Using the definitions of g, and 5z:
3, (-H,,, (-,,,,"")
(T H,H,'DJ.
(40)
This term is first-order in x and its sign is not guaranteed to be negative. If this term is positive, it can cause an increase in the CE loss. Since its magnitude is (Al) [I), it can dominate the (1 terms when || is small, leading a net increase in the CE loss.
C.3 MAIN RESULT AND PROOF
Proposition 2. Under the assumptions stated, if vectors {g. 8 are not both zero strictly negative, then there exist merge coefficients a, > such that for at least one task E (1,2).
J.C COCA ) > J. ( (
This difference is first order in the calibration weights >1, A2
Proof.
We analyze the difference in the CE loss for task between the two merging strategies. Let C merge Og and AOCAL mergo
AOCE
Og-
(C merge)
JC
OF (Erge)
= (O,) + g,
CAL
CAL + 0(1 0(1AOCAL(1)) (O)) + g. AE + 001100E8133
g: (AO
AE + 021
(41)
The difference between the merged displacement vectors is:
AOCAL
AE
(AT CAL 3 CAL) (OT CE + 3T
= a(T CAL -(CE) + ( CAL -2 CE
= a + 0()))) + 3(02 + 0(2)))
= aσ 362 O(23).
(42)
Substituting this back, the leading term of the loss difference is:
(O (AL
J.
CE
merge)
agg, 81) + B(gT 6z).
(43)
19

---

Published as a conference paper at ICLR 2026
The terms g, 61 and g. 82 are scalars of order [[1[T]])- Their signs depend on the geometry of the loss landscapes. Unless both scalars are non non-positive for both tasks = 1,2, we can choose positive coefficients a, 3 that result in positive sum for at least task. For instance, if g, ő1 0 for a given i, we can select a small enough B > O relative to (gg 61) + B(g 82) is positive.
such that the total sum
This positive term is of order [[][[T]). It dominates the other terms of order [([ and O when ||| and are sufficiently small, leading net increase in the CE loss for calibrated merging compared to standard merging.
Interpretation This result provides theoretical basis for the observation that merging task vec- tors trained with certain penalties can be detrimental. The calibration penalty introduces a linear per- turbation term ò to the task vector. This term is not necessarily aligned with the descent direction of the cross-entropy loss J. When multiple such vectors added, these misaligned perturbations can combine constructively to push the merged parameter vector into a region of higher CE loss. This increase is of first order in À and can therefore be significant. In contrast, merging pure CE vectors does not introduce such a first-order degradation term.
D EXPERIMENT DETAILS
All experiments were run on NVIDIA A100 GPUs (40 GB memory each). Fine-tuning jobs used four GPUs in parallel, whereas all evaluations were performed on single GPU.
Fine-tuning Details. Our training protocol closely mirrors the public code of Ilharco et al. (2023). For each task, fine-tuned CLIP backbones (ViT-B-32 and ViT-L--44) for 2000 updates using AdamW (Loshchilov & Hutter, 2019) with a weight-decay factor of 0.1. We adopted a cosine- annealed learning-rate schedule preceded by 200 warm-up steps and used a mini-batch size of 128; ViT-L-14 training employed gradient accumulation to match this effective batch size. Following the findings of Ilharco al. (2023), we kept CLIP's text encoder frozen and treated the logits obtained from class-specific prompts (e.g., "a photo of a {classname)] updating only the image encoder during fine-tuning. Regarding the learning rate, we used 10-4 only when training task vectors with large norms in the Norm Mismatch setting, and 10-5 for all other cases. In the Low Confidence setting, the label smoothing strength was set to a = 0.1.
as a fixed classification head,
Merging Details. For all four merging methods adopted in this study, it is necessary to tune the task vector coefficient A Following Ilharco et al. (2023), we imposed unified constraint on all X and searched the range from 0.0 to 1.0 (in increments such as 0.05) based validation accuracy.
Distillation Details. The distillation procedure generally followed the fine-tuning settings de- scribed above, except that the number of steps was 500 and the learning rate was fixed at 10-5 for all cases. The E2 regularizer weight 3 was set to 0.5.
D.1 NORMALIZED ACCURACY
The normalized accuracy for task on its dataset D. is defined as the ratio of the post-merge model's accuracy to the single-task model's accuracy:
normalized accuracy, accuracy(O D:)
accuracy(O,, D.)
where the function accuracy(0., O, D) denotes the accuracy of the model [(; 0) on dataset D.
E
ADDITIONAL RESULTS
E NORM COMPARISON ACROSS LAYERS
Figure (weights) and Figure 6 (biases) visualize how the parameter norm of each ViT-B-32 layer changes when the learning rate raised from 10-5 (gray bars) to 10-4 (blue bars). The scale shift
20

---

Published as a conference paper at ICLR 2026
not confined to few layers; rather, every block exhibits consistent multiplicative increase. In other words, tuning with a larger learning rate stretches the entire task vector almost uniformly, across both weight matrices and bias terms. This layer-wise coherence implies that any merge-time correction must adjust the global scale of the model, not merely a subset of layers.
1.2 Norm
Blk0.s proj
Blkl.c [c
Bikt blkl.c. proj
Blk2. proj
Blk3.c prop
Blk4.c Fc
Bik4.c proj
B1k5. p..p
Blk6. c. p.pro
Blk7.c [c
Bik7.c 1.7.c_prog proj
Blk8. c proj
Blk9 proj
Blk10.c. [c
Blk10.c proj
Bik11.c ppjj pmj
Blk0.c
Blk2.c
Blk3.c
Blk5.o
Blk6.c
Bik8.c.
Blk9.c.
Blk11.c
(a) MLP
1.2 Norm 1
BikkOin proj
Blk0. out proj
Bik1 Lin proj
Blk Lout, pro)
Blk2.in proj
Blk2 out, proj
Bik3.in proj
Blk3 out, proj
Blk4 Lin praj
Blk4. out, proj
Biks..n in proj
Blkk out, proj
Blk6, pro)
Blk6. out proj
Blk7in pruj
Blk7 out proj
Blk8 in proj
Bik8 Blk, pro)
Blk9.in proj
Blkk9 out proj
Blk10.in in proj
Bik1 out proj
BiK1lim proj
blkI 1. Lout, proj
(b) Attention
1.2 Norm
0.5
0.0
Blk0.In 1
Blk0.in 2
Bik 1
Bik1in 2
Blk2.nn 2
B 1
B1k3.In 2
Blk4.In 1
Blk4.In 2
Blk5.In 1
Blk5.In 2
Blk6.nn 2
Blk7.n 1
Blk7.nn 2
Bik8.n 1
Blk9.In 2
Blk10.n 1
Blk11n 1
1ikll.nn 2
In post
Blk2.nn
B1k6In
Blk8.n
Blk9In
Dlk10.n
(c) LayerNorm
LR=10-6
LR=10-4
Figure 5: Layer-wise average task-vector norms for weight parameters in ViT-B-32, averaged
over eight vision tasks. Gray correspond to a fine-tuning learning rate 10-5 blue bars to 10
E.2 OTHER CONFIDENCE CALIBRATION METHOD AND MERGING PERFORMANCE
We assessed two additional caafidence-calibratin label smoothing. For each of the eight vision tasks we fine-tuned ViT-B-33 32 with Mixup or focal loss and then merged the resulting task vectors. For Mixup, the interpolation coefficient was sampled independently at each iteration from the uniform distribution 2/(0, 1). For focal loss, we set the focusing parameter to 7 10. Table reports the outcomes. Like label smoothing, both Mixup and focal loss markedly reduced merge accuracy relative to the Original configuration, confirming that they also raise prediction entropy and thus interfere with model merging. In every case, how- ever, applying DisTaC restored accuracy level par with Original. demonstrating that DisTaC reliably conditions confidence even when the source models were calibrated with Mixup or focal loss.
Bechniques-Misse and focal loss alonggieee
21

---

Published as a conference paper at ICLR 2026
0.3
1.2 Norm
0.2
0.1
Blk0. fo
Blk0.c pro)
Blkl.c fo
Bik1.c proo
Blk2.c fo
Blk2.c, __roo
Blk3. fc
Bik3.c p..rro
Blk4.c, fe
Blkd pro]
Blk5. fc
Blk5. proj
Blk6 fo
Blk6.6 _proj
Blk7.c fc
BIk7 proj
Blk8.e. fe
BIk8 c p_oj
Blk9.c. fc
Bik9.c __roo
Bik Blk10.c proj
Bikll.c fo
Biki 1.c proj
Blk10.o
(a) MLP
1.2 Norm
0.2
0.1
0.0
Blk0.ou pro)
Blk in proj
Bik1 proj
Blk2.in proj
Blk2.oul proj
BIK3 in proj
Bik3.0m proj
Bik4.in pro)
Blkd out proj
proj
Blk5 out proj
Blk6 proj
B1k6 Blk6.out, out, proj
Blk7 in proj
Bik out. proj
Blk8 1...in proj
Bik8 out proj
Blk9 in proj
Blk9 buk, proj
Blk10.in proj
Blk Blk10.ou 0.out, proj
Bikl1 in proj
Biki 1 out proj
(b) Attention
0.15
1.2 Norm
0.10
0.05
0.00
in pre. bias
Blk0 2
Bik1.nn 2
Blk2.nn 2
Blkk3.n 1
Blk3.In 2
Blickin 1
Blk4.In 2
Blk5.n 2
Blk6.n 1
Bik7.n 2
BIk 1
Blk9in 1
Blk10.in 2
Bik1 1.1n 1
Biki 1.1n 2
In bias
Blk00in
Bik1Inn]
Blk2.n
Blk5..n
Blk6.In
Bik7
Blkk.In
Blk9
Blk10.n
(c) LayerNorm
LR=10-66 5 I.R=10
Figure 6: Layer-wise average task-vector norms for bias parameters in VIT-B-32, averaged
over eight vision tasks. Gray bars correspond to fine-tuning learning rate of 10-5 blue bars to 10 10-4
22

---

Published as a conference paper at ICLR 2026
Table Impact of confidence-calibraatie fine-tuning on merge accuracy. Source models (ViT- B-32) are fine-tuned with three popular calibration techniques label smoothing (LS), Mixup, and focal loss-before merging. every case the resulting merge accuracy drops far below the Original benchmark, showing that low-confidence sources hamper model merging. When the same models are first processed with DisTaC, accuracy restored level on par with Original, confirming that DisTaC's confidence conditioning is effective across all three calibration schemes.
Method
Original
LS
Mixup
Focal Loss
Task arithmetic
(78.0)
51.0(58.3)
52.3 (6.55)
(6.59))))
Task arithmetic + DisTac
(3.6
66.8
67.2
TIES TIES
(8.0
54.5 (62.0)
55.5 (63.9)
59.4
Tac
68.7 (77.9)
(78.7)
72.1 (82.4)
Consensus TA
(82.8
54.6
54.8 (63.0)
(689))
Consensus TA
+ DisTac
67.7 (76.5)
(77.8)
71.7 (81.7)
TSVM
(2.4)
60.7 (68.4)
60.9(69.6)
69,3 (79.5)
TSVM
+ DisTac
81.5 (91.8)
80.1
81.8
E.3
IMPACT OF TASK VECTOR SCALING ON VIT-L-14
We carried out the same scaling experiment (see Figure 3) on the larger ViT-L-14 backbone. As shown in Figure 7, the trend matches that of Figure 3: shrinking the task vector (A < 1) leaves single-task accuracy largely unchanged-often even slightly higher-whereas stretching (A > 1) rapidly erodes performance. These results further support the recommendation that, when task- vector norms are mismatched, one should shrink the longer vectors rather than stretch the shorter ones for robust model merging.
100
Relative ACC 101=1.0
80
60
40
20
prc
0,
0.5
1.0
1.5
2.0
2.5
3.0
Cars
DTD
EuroSAT
GTSRB
Figure 7: Effect of scaling task vectors on test accuracy. For each of the eight vision tasks (VIT- L-14), evaluate the model Ppe + Ar the scaling factor À varies from 0.0 to 3.0. Shrinking the task vector (A < 1.0) often preserves or even improves accuracy relative to the fine-tuned model (A 1.0), while stretching the vector (A 1.0) leads to sharp degradation. At A 3.0, performance falls below that of the zero-shot model on all tasks. These results support shrinking long task vectors match shorter ones when resolving norm disparities.
E.4 SCALING ALONE IS INSUFFICIENT TO OVERCOME NORM MISMATCHI
To test whether simple rescaling is sufficient, we revisited the Norm Mismatch scenario and aligned the longest task vector to the mean norm of the remaining vectors before merging. Figure 8 reports the resulting normalized accuracy for ViT B-32 the eight vision tasks: Original (gray), Norm Mismatch after only scaling (light orange), and Norm Mismatch followed by DisTaC (red). The r-axis lists the task whose vector lengthened; "Avg." is the mean over all tasks.
23

---

Published as a conference paper at ICLR 2026
100
80
Norm. ACC
60
40
20
0
Cars
DTD
EuroSAT
GTSRB
MNIST
RESISC45
SVHN
SUN397
Avg.
Original
Norm Mismatch (only scaling) Norm Mismatch (DisTaC)
Figure 8: Normalized merge accuracy for ViT-B-32 on the eight-task benchmark under three conditions. Gray: Original. Light-orange: Norm Mismatch after rescaling the longest task vector the mean norm of the others. Red: same rescaled vectors followed by DisTaC. Simple scaling narrows the gap only slightly, whereas DisTaC fully restores accuracy to the Original level. "Avg." denotes the average across all tasks.
Scaling alone lifts accuracy slightly but still leaves sizeable gap Original. In contrast, apply- ing DisTaC after scaling recovers the lost performance and matches the baseline across every task. As explained in Section 6.1, even shrinking a task vector inevitably hurts its single-task accuracy; DisTaC is therefore essential for restoring that accuracy before merging.
E.5 SENSITIVITY ANALYSIS ON UNLABELED DATA
We conducted experiments to assess DisTaC's sensitivity to data size and quality.
Robustness to Data Size. We first tested Dis DisTaC's performance by varying the number of unlabeled samples per class (100, 200, 300, 400, and 500). Table shows the average relative test accuracy across all tasks, where 100% represents the test accuracy achieved using the full unlabeled dataset (2,490 samples per class on average) for DisTaC. For comparison, we included results for distillation starting directly from the pretrained model
The results demonstrate DisTaC's strong robustness to limited data. DisTaC achieves over 90% of the full-data test performance with just 300 samples per class in both failure modes, and maintains over 80% performance even with 100 samples (reaching 96% in the Norm Mismatch case). Com- pared distillation from the pretrained model, DisTaC exhibits superior robustness. This highlights the methodological benefit of initializing distillation from the already scaled task vector (ppre
Table 3: Relative test accuracy with varying unlabeled data size per class. The baseline (100%)
corresponds to test accuracy using the full unlabeled dataset.
Method
100
200
300
400
500
Norm Mismatch
Seiill-foo-Pretrainnee
71.1
75.7
83.1
88.2
89.0
DisTaC
96.0
96.0
97.3
98.6
99.0
Low Confidence
Distill- -roe-Prete
70.1
73.8
81.2
84.6
87.6
DisTaC
83.9
87.4
90.5
91.0
95.0
24

---

Published as a conference paper at ICLR 2026
Robustness to Data Quality. We next assessed robustness to degraded data quality by introducing dataset shift via Gaussian blur during distillation. This setup simulates roaldworld conditions like variations in weather camera quality. The blur strength is controlled by the kernel size (fixed at 5) and the intensity range miin max), where a larger value indicates stronger corruption. Table shows the relative accuracy against the performance achieved using clean data for distillation.
The analysis confirms DisTaC's high robustness to quality degradation. DisTaC consistently main- tains performance, achieving over 90% of the clean-data performance even under the most severe corruption ma 3). In the challenging Low Confidence case, DisTaC maintains near-perfect accuracy (over 98. 08.5%) regardless of corruption intensity. DisTaC demonstrates superior robustness compared to the baseline, suggesting that utilizing the original fine-tuned model as the teacher ef- fectively filters noise present the unlabeled data.
In conclusion, these experiments confirm that DisTaC possesses sufficient robustness to variations in both unlabeled data size and quality, supporting its effectiveness for real-world applications.
Table 4: Relative test accuracy under Gaussian blur corruption. Ranges [aminn man denote the blur intensity, with larger values indicating stronger corruption.
Method
[0.1, I]
[0.1, 2]
[1, 3]
Norm Mismatch
Seaill-foo-Pretrainne
98.1
95.7
90.7
DisTaC
100.4
96.2
91.7
Low Confidence
Distill-fron Prom-Pretre
98.1
97.3
94.7
DisTaC
99.6
98.5
99.9
E.6 COMPUTATIONAL EFFICIENCY DISTAC
Table 5: Computational cost of DisTaC on ViT-B-32 averaged over 8 tasks.
Metric
Value
Hardware
NVIDIA A100
Batch Size
64 per device
Time per Step
R 0.0064
Total Time (500 steps)
3.2 s
Peak Memory Usage
7.1 GB
To empirically validate the claim that DisTaC is computationally lightweight, measured the train- ing cost using the ViT-B-32 backbone on 2 NVIDIA A100 GPUs As summarized Table 5, the distillation process is extremely efficient. With batch size of 64, the average training time is ap- proximately 0.0064 seconds per step across the eight vision tasks. Consequently, the standard 500- step DisTaC procedure requires only about 3.2 seconds complete (excluding evaluation time). The peak GPU memory usage was recorded at 7.1 GB, which includes the overhead for online teacher inference; this could be further optimized by pre-computing teacher predictions.
E.7 GENERALIZING DISTAC NLP
We conducted experiments using RoBERTa-base (RoBERTa-b)) RoBERTa-large (RoBERTa- (Zhuang et al., 2021), and 1lama2-7b (Touvron al., 2023) to examine whether our claims ex- tend beyond vision tasks to the NLP domain. Following Ilharco et al. (2023), we adopt four GLUE benchmark Wang et al., 2019) tasks: CoLA, MRPC, RTE, and SST-2. In the NLP experiments, we evaluate the same settings as in vision: Norm Mismatch and Low Confidence.
The results are presented in Table 6. In comparison to the original configuration, the normalized score degrades under both Norm Mismatch and Low Confidence settings. In instances of norm mismatch among task vectors, the application of DisTaC effectively reduces interference between task vectors, thereby enhancing the normalized score from that of task arithmetic without DisTaC
25

---

Published as a conference paper at ICLR 2026
Table 6: Comparison of post-merge accuracy across fine-tuning configurations and the effect of DisTaC in NLP. Absolute accuracy is displayed in a large font size, whereas normalized accuracy appears in parentheses in a smaller font. When the task vector norms diverge (Norm Mismatch) or the source models exhibit low confidence (Low Confidence), the normalized score degrades relative to standard benchmark setting (Original). Under these conditions, DisTaC effectively pre-conditions the source models, improving performance in both settings.
Orgnaa RoDlRTT-)]
Confidence RoBERTa
RoDERTa-b
Lluma2-76
RoBERTab
Llumad-7
Task arithmetic
759
56.8a
55,3
75.7ws
Task arihmetic
DisTaC
59.9
64.4
75.0
62.
70.0
73.0059:
Ties mexgang
08.7365
38.3
merging
62.4766
58.9
DiTTaC
648mm
Consensus merging
61.023)
07.9204
Consensus DisTaC
60.5
63.
68.4
62.2
69.5
72.0.54:
(e.g., RoBERTa-large cxhibits an increase from 58.1 to 80.5, an improvement of 22.4 points in the normalized score). Furthermore, when the task vectors exhibit low confidence, the implementation of DisTaC results in an elevation of the normalized score compared to scenarios without DisTaC (e.g., RoBERTa-large: 73.9 82.3, an enhancement of 8.4 points in the normalized score). These findings indicate that (i) the identified failure modes of norm disparity and low confidence we iden- tify arise in both vision and language tasks, and (ii) DisTaC conditioning consistently enhances outcome of merging for CLIP/ViT, Roberta, and Llama. We posit that these results demonstrate the cross-modality generalizability of vision and language. Notably. the recovery is stronger at larger scales (e.g., llama2-7b in Norm Mismatch), suggesting that the method retains its efficacy model capacity expands.
26


Reviews:
[{"review_id": "Y15GSup4zO", "reviewer": "Reviewer_A7va", "summary": "This paper investigates the robustness of multi-task model merging methods under realistic, non-idealized conditions. The authors identify and empirically validate two key factors that degrade the performance of existing merging techniques: (1) disparities in the norms of task vectors, often arising from different fine-tuning hyperparameters, and (2) low prediction confidence of the source models, which can result from calibration techniques like label smoothing. To address these issues, the paper proposes DisTaC, a pre-conditioning step that uses knowledge distillation on unlabeled data. DisTaC adjusts task vector norms and increases model confidence before the merge, thereby making state-of-the-art merging algorithms more robust. Experiments on vision tasks show that DisTaC significantly improves performance in these challenging scenarios, often recovering it to the level of an ideal merge.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper addresses a novel and highly relevant problem. While most research focuses on improving performance on idealized benchmarks, this work astutely investigates the robustness of merging methods in more practical, pessimistic settings, which is a critical step for real-world applicability.\n\n2. The problem formulation is exceptionally clear and well-motivated. The failure modes are demonstrated with convincing empirical evidence (Figure 1), making the motivation for the proposed solution immediately apparent and easy to follow.\n\n3. The paper provides valuable and actionable insights for practitioners. The discussions around \"shrinking vs. stretching\" task vectors and the trade-off between source model calibration and merge performance offer practical guidelines that extend beyond the method itself.", "weaknesses": "1. The empirical evaluation is limited in scope. All experiments are conducted on CLIP/ViT models for vision classification tasks. Given the current prominence of model merging in the context of Large Language Models (LLMs), the absence of any experiments on language tasks is a significant limitation and leaves the generalizability of the findings in question.\n\n2. The investigation of failure modes, while insightful, feels somewhat narrow. The paper focuses on norm disparity and low confidence but does not explore other plausible sources of incompatibility, such as differences in optimizer states, dataset quality/size, or the specific architecture of parameter-efficient fine-tuning (e.g., LoRA rank).\n\n3. The method introduces a dependency on unlabeled data from the task distribution. While the authors argue this is a reasonable requirement, it makes the method less self-contained than truly data-free merging techniques and may be a practical constraint in some scenarios.\n\n4. Proposition 1 formalizes a basic geometric intuition about vector addition but offers little deep insight into the functional consequences within a neural network. The choice of cosine similarity as the primary metric is also debatable, as its connection to actual task performance is not explicitly established.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4K2dSHbmAf", "reviewer": "Reviewer_XSii", "summary": "The paper investigates the robustness limits of state-of-the-art multi-task model merging methods, identifying two critical failure modes: disparities in task vector norms and low confidence in source models. To address these, the authors propose DisTaC—a pre-conditioning framework utilizing knowledge distillation to harmonize task vector norms and boost model confidence before merging. Extensive experiments on eight vision tasks with CLIP backbones demonstrate that DisTaC consistently restores or enhances merging performance under challenging, realistic scenarios where conventional merging fails.", "questions": "1. How does DisTaC's independent conditioning of each task vector affect the final merge performance when tasks are fundamentally antagonistic or highly complementary?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a sharp empirical and theoretical analysis of robustness shortcomings in existing model merging approaches, uncovering how norm disparity and low confidence can produce significant accuracy degradation after merging. \n2. DisTaC is a clear, well-motivated conditioning method, leveraging knowledge distillation with unlabeled data for practical pre-conditioning. By integrating both norm scaling and confidence sharpening, it directly addresses the two key failure modes.", "weaknesses": "1. Since the pre-conditioning step is applied independently to each source model, the total computational cost scales linearly with the number of models. For large-scale ensembles, this cumulative overhead can become prohibitive, undermining the method's primary efficiency advantage over full-scale retraining. The distillation cost also scales with model parameter count, becoming computationally prohibitive for large-scale foundation models (e.g., LLMs). \n2. The study’s empirical validation is confined to vision tasks, leaving its applicability to other modalities, particularly Large Language Models (LLMs), unverified. Given that task vector dynamics in LLMs may be substantially more complex, the method's effectiveness for merging models with diverse reasoning and generative capabilities is not guaranteed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4q8l0p6l8L", "reviewer": "Reviewer_P2H4", "summary": "The paper investigates the robustness of model merging methods. The authors identify two critical failure modes in existing model-merging pipelines:\n\n+ Task vector norm disparities — differences in magnitudes caused by heterogeneous fine-tuning hyperparameters (e.g., learning rates, steps).\n\n+ Low pretrained-model confidence — often resulting from training techniques such as label smoothing, Mixup, or focal loss.\n\nTo mitigate these issues, the authors propose DisTaC, a lightweight rescaling task vectors to a target norm and increasing model confidence by training students with higher temperatures than teachers. Empirical evaluations on eight vision tasks demonstrate that DisTaC restores and often improves post-merge accuracy.", "questions": "Why not provide the results of the Original version when combined with DisTaC in Table 1? It is helpful for us to understand that DisTaC may be harmful for performance in some cases.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The experiments are extensive, including ablation studies, visual analyses (e.g., layer-wise norm shifts in Figs. 5–6), and additional tests with Mixup and focal loss to demonstrate generality.\n\nThe authors go beyond idealized benchmarks and pinpoint practical causes of model-merging failures, which is interesting and meaningful.", "weaknesses": "The experiments are restricted to vision-only tasks using CLIP ViT backbones. No tests are conducted on NLP, speech, or multimodal architectures.\n\nWhile the paper argues that unlabeled data are “readily available,” this assumption may not hold in all domains, such as the security issue, or we can not find a comprehensive set for LLMs.\n\nWhile DisTaC generally improves results, the paper lacks a sensitivity analysis of unlabeled data quality that would clarify when it might fail or overfit.\n\nThe authors assert that DisTaC is computationally lightweight (“500 steps, unlabeled data only”), but do not quantitatively compare its runtime or cost to baseline merges, nor provide results on large-scale LLMs such as Qwen.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zF67W0vapW", "reviewer": "Reviewer_bqXy", "summary": "The authors identify two critical failure modes that are common in practical scenarios but overlooked in idealized benchmarks: (1) Task Vector Norm Disparity, where task vectors have vastly different magnitudes; and (2) Low-Confidence Source Models, where models produce high-entropy predictions, often as a result of calibration techniques like label smoothing.\n\nThe paper proposes a lightweight pre-conditioning method. DisTaC uses KD on unlabeled data to prepare models for merging. To correct norm disparities, it first rescales a task vector to a target norm and then uses KD to distill knowledge from the original, high-performing model, thereby recovering the performance lost during scaling. To correct for low confidence, it distills knowledge using a higher temperature for the student model than the teacher, which encourages the student to produce lower-entropy, more confident predictions.\n\nThe authors demonstrate empirically on eight vision tasks with ViT-B-32 and ViT-L-14 backbones that these failure modes significantly degrade the performance of state-of-the-art merging methods. They then show that applying DisTaC before merging consistently restores performance to the level of the idealized benchmark, with minimal computational overhead. Finally, the paper offers practical guidelines for model merging.", "questions": "1. Figure 3 and the discussion in Section 6.1 suggest that shrinking a task vector can sometimes lead to better performance than the original model. This is a very interesting observation. Could you elaborate on the potential reasons for this? Does this imply a regularization effect, suggesting that the initial fine-tuning may have overshot a more optimal, shorter task vector?\n\n2. Regarding the influence of learning rate and training steps on the effectiveness of model merging, you may refer to the error analysis in Theorem 3.1 of [1], where the impact of the norm of the task vector was also observed. In [2], analysis of the properties of task vectors was conducted.\n\n*[1] Unifying Multimodal Large Language Model Capabilities and Modalities via Model Merging. arXiv preprint arXiv:2505.19892*  \n*[2] MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation. arXiv preprint arXiv:2406.07529*", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper's primary contribution is the identification and diagnosis of two practical failure modes in model merging. While the component techniques (knowledge distillation) are not new, their application to pre-condition task vectors to solve these specific robustness issues is novel.\n\n2. The claims are substantiated by a thorough and well-designed set of experiments. The inclusion of theoretical motivation (Proposition 1 and the analysis in Appendix C) adds rigor and provides a deeper understanding of the observed phenomena. The ablation studies, such as the analysis of shrinking versus stretching vectors in Figure 3, are particularly convincing and support the practical guidelines offered.\n\n3. By addressing the gap between idealized academic benchmarks, the paper provides a crucial step towards making model merging a more practical and reliable tool. DisTaC is a simple, effective, and computationally cheap solution that can be readily adopted by practitioners.", "weaknesses": "* The experiments are confined exclusively to CLIP-based models on vision tasks. The general applicability of DisTaC would be strengthened by demonstrating its effectiveness in other modalities, such as merging fine-tuned LLMs, or on MLLMs.\n\n* The method's reliance on unlabeled data from the task distribution is a potential limitation. The paper would benefit from an analysis of the method's sensitivity to the quantity of this unlabeled data. For instance, how much data is needed for DisTaC to be effective, and how does performance degrade if the unlabeled data is from a slightly shifted distribution?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)