Knowledge is a Region in Weight Space for Fine-tuned Language Models
Authors: Almog Gueta
Elad Venezian
Colin Raffel
Noam Slonim
Yoav Katz
Leshem Choshen
Venue: EMNLP 2023 Findings (2023)
Keywords: Weight space
loss landscape
loss space
finetuning
fine-tune
loss connectivity
basin
minima
TL;DR: There are high performance regions in the weight space

Abstract: Research on neural networks has focused on understanding a single model trained on a single dataset. However, relatively little is known about the relationships between different models, particularly those trained or tested on different datasets. We address this by studying how the weight space and the underlying loss landscape of different models are interconnected.

Specifically, we demonstrate that finetuned models that were optimized for high performance, reside in well-defined regions in weight space, and vice versa -- that any model that resides anywhere in those regions also exhibits high performance. Notably, we show that language models that have been finetuned on the same dataset form a tight cluster in the weight space, while models finetuned on different datasets from the same underlying task form a looser cluster. Moreover, traversing around the region between the models leads to new models that perform comparably or even better than models obtained via finetuning, even on tasks that the original models were not finetuned on.

Our findings provide insight into the relationships between models, demonstrating that a model positioned between two similar models can acquire the knowledge of both. We leverage this and design a method for selecting a better model for efficient finetuning. Specifically, we show that starting from the center of the region is as effective, if not more, than using the pretrained model in 11 out of 12 datasets, resulting in an average accuracy improvement of 3.06.

Knowledge is a Region in Weight Space for Finetuned Language Models
Almog Gueta Technion IIT
Elad Venezian
Colin Raffel
IBM Research eladv@il. ibm. com
UNC Chapel Hill
almoggu@gmail. com
craffe fel@@maill.com com
Noam Slonim
Yoav Katz
Leshem Choshen
IBM Research noams@il, 1im.immm..... com com
IBM Research katz@il. ibm. com
IBM Research
leshem. choshen@il ..l..mm ibm. com
Abstract
General flegion
Research on neural networks has focused on understanding single model trained sin- gle dataset. However, relatively little is known about the relationships between different mod- els. particularly those trained or tested on differ- ent datasets. We address this by studying how the weight space and the underlying loss land- scape of different models are interconnected.
Task Region
Datason
Pro trained
Specifically, we demonstrate that finetuned models that were optimized for high perfor- mance, reside in well-defined regions in weight space, and vice versa that any model that re- sides anywhere those regions also exhibits high performance. Notably, we show that lan- guage models that have been finetuned on the same dataset form tight cluster the weight space, while models finetuned different datasets from the same underlying task form looser cluster. Moreover, traversing around the region between the models leads to new mod- els that perform comparably or even better than models obtained via finetuning, that the original models were not finetuned on.
Figure I: A schematic view of the weight space. Fine- tuning ends up in region determined by the dataset (deep blue) which resides in the task (light blue) and language tasks regions (outer blue). Any combination of finetuned weights is found within the region. Each region is characterized by a low loss on the correspond- ing: dataset, task datasets, or diverse linguistic datasets. Generally, loss is lower inside the region than outside or in its boundaries.
explores the relationship between the weights of different finetuned models and the capabilities they exhibit. We analyze the weight space, where each model is represented by a weight vector 0 E R". For simplicity, we refer to both point in weight space and the neural network itself as a "model".
tasks
Our findings provide insight into the rela- tionships between models, demonstrating that model positioned between similar models can acquire the knowledge of both. We leverage this and design a method selecting better model for efficient finetuning. Specifically, we show that starting from the center of the region than using the pretrained model in II out of 12 datasets, resulting in an average accuracy improvement of 3.06.
We find that distance characterizes models' knowledge and similarity. Particularly, after fine- tuning pretrained model similar datasets, the resulting models close each other in weight space ($2.3). Throughout the paper, we consider 3 granularities ($3.1), showing that (i) models fine- tuned on the same data are closer to each other than to other models; (ii) models finetuned on the same task also cluster together; and (iii) models finetuned on general language tasks are not spread arbitrarily around the pretrained model, but fall in constrained region in space.
effective, not more.
1 Introduction
Models that share the same architecture but differ in their weights can have dramatically different capabilities. As an example, finetuned variants of a pretrained model all share an architecture, yet they are specialized for different tasks. This study
We find that different finetuning runs on the same data tend to converge on similar points in weight space rather than dispersed points. Loosely, those points embed the necessary knowledge to perform
Research done during internship in IBM Research.

---

points in the proximity of finetuned models might also perform the task well. Notably, such points in weight space might not necessarily be reached via finetuning, but rather via spatial transforma- tions. Indeed, we replicate the finding (Entezari et al., 2021, c.f. $8) that models finetuned on the same dataset are linearly connected, i.e., points on the line between the two models attain similar or even lower loss ($5.1). We expand this finding the convex hull between the finetuned models ($5.2), suggesting that knowledge is shared across the region in space. That is, finetuned models define a connected basin of low loss, and every point within it performs well. To show this, we test models sampled from the region and find they even outperform the models achieved by finetun- ing. Moreover, we replicate the findings in all the aforementioned granularities: regions per dataset, task, and general. For each, we observe a low loss across datasets, beyond the loss the individ- ual models optimized. Furthermore, we show in $6 that these regions are relatively tight, in the sense that extrapolating (rather than interpolating) can quickly produce a poorly performing model. the task. This leads to the hypothesis that other
the second experiment type, we compare losses of one group of models to another. Below, we describe the datasets ($2.1), settings ($2.2), and granularity levels of comparison between models ($3.1).
2.1 Datasets
We finetune and evaluate models on 36 datasets. Those datasets can be categorized into a few fami- lies: natural language inference (NLI), Sentiment analysis and Topic classification tasks, Twitler do- main, and a collection of general datasets that cov- ers a wide range of capabilities. We chose classifi- cation datasets for reliable evaluation. The details of each dataset family are found in App. A. We mostly rely on the MNLI (Williams al., 2018b) dataset, the NLI family, and the General group, as case studies, and elaborate on them below:
dataset family contains 12 text classifi- cation datasets from GLUE (Wang et al., 2018) and SuperGLUE (Wang et al., 2019), excluding test- only (AX-b (Wang et al., 2019), AX-g (Poliak al.. 2018) 8)) and regression (STS-B (Cer et al., 2017)) datasets. We further exclude WSC (Levesque et al., 2012) and CoPA (Roemmele et al., 2011) which are small and therefore produce unstable results (e.g., finetuning results were sometimes lower than pretrained model results). The datasets consist of wide range of classification tasks, from sentiment analysis to linguistic acceptability to NLI.
General
Our empirical findings have intriguing implica- tions, suggesting, for example, that the best models may not lie at the edges of the region, but rather closer to its center, while finetuning often yields models at the edge of the region. Motivated by these findings, we demonstrate in $7 that model created by averaging the weights of finetuned models from the same region outperforms the pretrained model on variety of tasks after subsequent finetuning, even on tasks that the original finetuned models were not trained on.
NLI family is composed of natural language inference (NLI) datasets: MNLI (Williams et al., 201 2018a), QNLI Rajpurkar et al. 2016, RTE (Da- gan et al., 2005; Bar-Haim et al., 2006; Giampic- colo et al., 2007; Bentivogli et al., 2009), WNLI (Levesque et al., 2011), ESNLI (Camburu et al., 2018), and adversarial NLI (Nic et al., 2020).
Overall, our work contributes to the growing body of knowledge about the loss landscape, find- ing connectivity in whole bounded region rather than mere linear connectivity, finding connectivity between models not trained on the same task, and finding connectivity in generalization, evaluating models on multiple losses. We also provide initial context to empirical findings about fusing models. We discuss the relations to previous works in $8.
2.2 Training Approaches
We experiment with RoBERTa-base (Liu et al., 2019) as our base pretrained model, except in App. B where we analyze different pretrained mod- els. For finetuning, we follow the standard hyper- parameters (Liu ct al., 2019), with larger batch size of 256 and learning rate of 5e 5. Most ex- periments analyze different seeds, and the same- dataset clustering seeds ($.. Those seeds control randomly initialized weights in the classifi- cation head as well as data shuffling.
2 Experimental Setup
We conduct two main types of experiments. In one we train models with different characteristics (c.g., dataset or task, see $3.1) and examine their representation in weight space using clustering. In

---

mm
parc
Tore
(a) Clustering models by dataset.
(b) Clustering models by dataset family.
Figure 2: Clusters of finetured models different datasets or tasks, projected by t-SNE. We find that both datasets and dataset families correspond to regions in space. In each figure, each model is represented as a dot, where the inner color is the color of the dataset/task the model finetuned with and the outer color is the color of the most common dataset/task in the cluster (representing the cluster label). Datasets/tasks names are shown in legends.
2.3 Clustering Approach
their loss. Unfortunately, the loss is often incompa- rable across datasets or tasks. Hence, we compare by preserving each encoder, and fitting a classifica- tion head to each model for each target dataset.
the clustering experiments, we qualitatively ex- piore whether models trained on similar data end up close together in weight space. We experimented with various distance metrics and clustering algo- rithms. While many metrics worked well, we found that subtracting the pretrained weight values from the finetuned values (referred to as "task vectors" by Ilharco et (2022)) and measuring distance via cosine similarity was conceptually simple, cheap to compute, and provided qualitatively reasonable results compared to more sophisticated methods (Kornblith et al., 2019; Toledo et al., 2022). We also tested Euclidean distance but it did not produce clear clusters. This is likely caused by the weights' norm growth during training (Merrill et al., 2020) that is unrelated the data at hand (SC). This can also explain questions that were previously left open (Qin et al., 2022). As clustering algorithm, we use Spectral Clustering with as many clusters datasets or dataset families (Pedregosa et al., 2011). For visualization, we project the 120M di- mensional weight vectors into 2 dimensions using t-SNE (Van der Maaten & Hinton, 2008).
Specifically, calculate the loss of model we perform the following: First, we remove any ex- isting masked language modeling layers or clas- sification heads and replace them with a new ran- domly initialized classification head. This leaves the rest of the weights i.e., the encoder fw, fixed. We then perform linear probing, i.c., we train only the new classification head on a desired target data Etrain and its labels ytrain- Lastly, we pass the test data Ftest through the model (including the classifier fō on top of it) and report the loss with respect to the labels Y/esst- Formally, for the model few and loss function I, we report the gen- eralized loss (w)) - ([fow( Itest) where fo = arg ming ((fo,. Mtrain). This ap- proach has desirable trait: When considering the task which the model was originally finetuned, our loss lg is equal to the original finetuning loss I. Furthermore, since fitting linear classifier given fixed representation a convex optimization prob- lem, we observe similar results across runs.
3 Methodology: Comparing Models
The generalized loss lg enables comparing mod- els finetuned different datasets. It is hence unde- sirable test only consider loss dataset, but also the average loss on family of datasets. For example, the aver- age loss across all entailment datasets rather than the loss on a particular dataset.
this work, we compare models that share an architecture, but were trained on different data. To do so, we investigate the space of weights w E Rd where each model has weight vector and each point in space represents model. We adopt the typical perspective that the model fo consists of a representation encoder fa followed by task- specific classifier fo. i.e. fo = fo fa == fow (Choshen et al., 2022a; Ram et al., 2022).
of the datasets. We thus
3.1 Levels of Granularity
To study the relationship between weights of simi- larly trained models, we experiment with levels
Ideally, we would compare finetuned models by

---

of granularity for dataset similarity, At each level, we analyze models finetuned on source datasets sharing some traits. In each level's setting, we de- fine an interior group (hereafter In) of datasets that share a trait as well as an exterior group (hereafter Ex) of models not sharing the trait. By default, we report on each group the average loss over all source datasets used for finetuning In models.
els compared models that have been trained on different datasets or tasks. Notably, despite the fact that neural networks implement highly non-linear functions, finetuning similarity is expressed in the Euclidean space of their weights. Moreover, we show in App. &C that the direction in space is de- termined by the type of training data and not by its amount. In App. B, we show that this proximity is contingent on starting from the same base model.
Same-Dataset. In the most specific case, mod- els are similar if they were finetuned on the same dataset. Interior models are finetuned on MNLI (Williams et al.. 2a) and Ex on the rest of the General datasets. We report the loss over MNLI.
Similarity Per Dataset. In the simplest case, for each dataset in the General group, we finetune mod- cls with 20 random seeds and cluster the resulting 280 models into 12 clusters. As seen in Fig. 2(a), for the most part, models finetuned on the same dataset are clustered together. Accordingly, the overall clustering accuracy is 98%, with but 3 clusters perfectly matched.
Same- At this broader granularity, con- sider the group of models trained on the same task. In that case, In contains models finetuned on NLI datasets and Ex contains models finetuned on all other datasets. We report loss over all NLJ datasets, except for ANLI which is not intended for such test purposes. ANLI is made with adversarial examples that cause misclassifications for NLI-trained mod- cls. In initial trials, it showed similar trends, but we omit from the test for good practice.
Similarity Per Task. In this experiment, we show that models finetumed on datasets from the same task are also close in weight space (we discuss same-domain proximity in App. D). As explained in $2.1 we have dataset families for 3 tasks: NLI, Topic, and Sentiment. For each dataset in each family, We finetuned models with 5 random seeds. Then, cluster all models into 3 clusters. As seen in Fig. 2(b), models that were finetuned on the same task family are closer to each other and are clustered together (clustering accuracy of 90%). We report the F1 Score per group in App. D.
General. In the most general case, we consider any model finetuned on any of the General datasets as In. This leaves little to consider as exterior, so we construct Ex by perturbing the pretrained model's weights in random direction. We apply perturbation whose norm is equal to the average task vector norm of In models. Since there is no clear prior sampling a random direction in space, we aim for a prior that prefers points the weight space that represent "reasonable" networks. We use Xavier initialization (Glorot & Bengio, 2010) to de- fine such a prior. The prior is an i.i.d. Gaussian distribution over each weight with zero mean and where variance depends the layer characteristics. This choice reduces the probability of sampling net- works with exploding or vanishing outputs, which would stand as a weak baseline.
Similarity in General. Unlike datasets tasks, not create multiple distinct general groups and can not expect multiple clusters to occur. Therefore, we do not present clustering for this granularity level. However, we can still infer that this general region does not encompass the whole space around the pretrained model, and has a supe- rior loss in general (see $5.2).
4.1 Cause: Data Type, not Size
Supposedly, a confounding factor may explain the above results, wherein the finetuned model moves more with more data. To test this, we finetune models on sub-samples with different sample sizes (200, 400, 800, 1.6K, 3K). For consistency, we take only the 9 datasets from General family that contain at least 3K training samples. We then cluster the finetuned models into k clusters, with & the number of datasets or the number of dataset sizes.
4 Analysis in Weight Space
We start our analysis by showing that the models trained on similar data fall into the same region in weight space i.c., they are clustered together. We leave the inverse claim (i.e. showing that mod- els within the cluster obtain lower loss than the models outside the cluster) to $5.1 and $5.2.
Specifically, we find (see Fig. 2) that finetuning on similar data results in closer weight space mod-
The resulting clusters (App. C) are clustered by data type, not by the amount of data, similar to

---

Fig. 2. Choosing be the number of data-sizes does not cluster by data size either. We conclude that the observed similarity comes from the nature of the data, and not from the size of a given dataset.
Results (Fig. 3) show that interpolated models perform comparably or even better than the models they are created from. We present further results testing the groups on different losses in App. $E and find performance is often best somewhere the interpolation between the two models. We now elaborate on each granularity level separately.
5 Loss in the Region between Models
In $4, we claim that models trained on similar data converge near each other, but is this area to which they converge meaningful? In this section, we show that models falling in the entire region around these clusters correspond to performant models.
Interpolation Per Dataset. We interpolate 5 fine- tuned models on the MNLI dataset (resulting in a total of 10 pairs) and evaluate on MNLL. We report an analogous experiment with SST2 in App. $E. Figure 3(a) shows that the interpolated models per- form well on average and even outperform the orig- inal models they are created from. Similar results were found in other settings (e.g.; Wortsman et al., 2022b) and we discuss those works in $8.
The models we analyzed so far were the outcome of gradient-based optimization process searching for the minimum loss. The locality we observed in weight space indicates that the points found through this procedure are concentrated in rela- tively small regions. We hypothesize that a whole region of low losses (corresponding to performant models) exists between the separate points found during finetuning. For example, the "NLI region" contains MNLI, SNLI and QNLI models but also other points that reflect models that might not have been found through gradient-based optimization on a specific dataset but exhibit the general abilities needed to perform natural language inference.
Interpolation Per Task. We interpolate 5 models finetuned on MNLI with 5 models finetuned on ESNLI, both from the NLI task, resulting in 25 pairs, and evaluate on all NLI test datasets. We replicate the results of the previous experiment and find the interpolated models are performant on all targets on average, can be seen in Fig. 3b).
Interpolation In General. We interpolate 5 mod- cls finctuned on MNLI with 5 models finetuned on SST2, both from the General family, resulting in 25 pairs and evaluate on all General datasets as targets. Fig. 3(C) shows improved performance in this extended group and better performance in the interpolated models than in the finetuned ones.
We test this hypothesis by interpolating pairs of similarly trained models and show in $ 5.1 that the points between the models perform comparably or even better than the original finetuned mod- els. This suggests that indeed there are regions in weight space where all points encode the knowl- edge or behaviour required for a particular task. We expand this claim in $5.2 and show that the whole region that lies between these models (their convex hull) corresponds to models that perform well.
5.2 Comparison between Region losses
Thus far, we showed that models on the line be- tween model pairs perform well. We now extend the analysis to show that models in the whole re- gion between similar models perform well. How- ever, visualizing searching whole multidimen- sional region (the convex hull) is not feasible. In- stead, we sample models in the region and show they outperform their external counterparts.
5.1 Interpolation: Lines Between Model Pairs
In this experiment, we consider the points in weight space between pairs of finetuned models. Given a pair of models, we shift from one model to the other by linearly interpolating between their weights, i.e., we take the model's 's weights W1, W2 E Rd, and consider weighted sums of their weights: W1 a + * (1 a). where a E [0. 1]. We then evaluate each interpolated model both on the datasets the original models were finetuned on, and on additional datasets unseen by the models. We interpolate pairs of different models finetuned on the same dataset, or on two different datasets. We report the average losses produced by repeating the experiment with finetuning using different seeds.
Let In be group of models and In' be the con- vex hull between all the models in In, making each model in In' weighted average of models in In: > [In] a:-w; where |!!m ag = 1 and wi E In. Prac- tically, as In' is infinite, we estimate it by sampling In models uniformly from the region they convey.
We note that weighted averaging in this manner was shown to be practical and work well in many scenarios, either in efficient finetuning ($7 Yadav

---

0.3650
Avg
Avg
0.1525
0.1625
0.220
0.215
0.3275
0.3575
0.1250
0.205
0.1225
0.1525
0.200
0.1200
0.1500
MNLID.1 0.2 0.3 0.4 05 06 0J7 0.8 O.9NNLI
MML. 0.7 5.1.A. D.S D.5 0.7 D. O.RESNLI
S5120.1 0.2 0.6 0.7 08 MNU
(a) Interpolation per dataset.
(b) Interpolation per task.
(c) Interpolation in General.
Figure 3: Losses of linearly interpolated models created between pairs of similar models. The best loss often lies between models. In each figure, the solid line the losses' average during interpolations for different values, the edges of the lines represent the average loss pure finetuned models we interpolate, the Y axis is the average loss value, and the axis the position determined by a. The shade is standard deviation the losses' average.
100
group In
group
100
group
80
80
In
80
Percett
In
6.0
40
40
40
20
0.325 0.150
0.55
0.20
0.25
0.24
0.26
loss
Loss
Loss
(a) Losses in the dataset region
(b) Losses in the task region
(c) Losses in the general region
Figure 4: Loss distributions of 3 groups: In (similarly finetuned models), In (models between models in In), and Ex (baseline models). Fig. 4(a) shows models from MNLI region tested on the MNLI loss. Fig. 4(b) shows models from NLI region tested on NLI losses, Fig. 4(c) shows models from the General region tested on the General losses.
et al., 2023) or full finetuning (Choshen et al., 2022b; Matena & Raffel, 2021, c.f. $8).
Loss Per Dataset. We test the performance models between models finetuned on a dataset. We consider the case where In the group of finetuned models on MNLI and Ex is the group of finetuned models on General datasets. Both groups are evalu- ated the MNLI dataset. We find PB is 100% for In, meaning that all MNLI models outperform MNLI than all the rest of the models. More surpris- ing that the same is true for In', PB of 100% all the models between MNLI models are better than Ex. In fact, in 88% of the times models are also better than In - i.e. models finetuned on MNLI!
We define a metric to compare two groups of models. Given In models group and the exterior models group Ex, we calculate PB as the probabil- ity that an model outperforms an Ex one:
PB
E iEInjEEx
(4g( S (w;))
PB can also be applied In' and Ex.
As loss function, we take the average loss over the source datasets used to create the In models.
Loss Per Task. We compare models from a task region with models from other regions. Here, In are the models finetuned on NLI task and Ex on the rest of the datasets described in $2.1. Both groups evaluated all NIII test datasets. NLI In group models are better in PB 75.3% of the cases, and the In models in 100%. In' also better than with PB = 96.7%.
Testing models from In and In' groups, we find they indeed outperform Ex models on the tasks the In models were trained on. We find this is true in all granularity levels models in the dataset region are better than other models, and more broadly any finetuned model is better than models that have been randomly shifted by the same distance from the pretrained model. Moreover, (as in 1).1) that In' even better than the In. In addition to the bottom- line metric PB, we depict the loss distributions across those models in Fig. 4.
again find
Loss In General. We define in to be finetuned models on General datasets and Ex to be random models as defined in $3.1. Both are evaluated on

---

the General datasets. We find again that In is better than Ex (PB 89.8%) but worse than In' (PB- 90%) which is also better than Ex (PB = 100%). To conclude, we consistently see that the region between finetuned models not only provide models that are better than the baseline but also provides models that are better than the finetuned models defining the edges of region.
0.30
0.20
-10
327
6 Region Edges
Figure 5: Losses of linearly extrapolated models created from pairs of models finetuned on MNLI. The solid line the average losses, the vertical dashed lines indicate the average loss of the pure models extrapolate (a or a = 1), and the X axis is the position (meaning the and (1 a:) values used in the extrapolation). The shade is the standard deviation across runs.
Above, we have shown that there are spacial re- gions that specify learnt generalizations. We now look for the boundaries of those regions, where loss is no longer similarly low. To do that we traverse in the opposite way to the interpolation. We also test the edges going from the center of the region other directions in App. F.
the Same-Dataset region, as has indeed been used in practice (c.f. $8; Wortsman et al., 2022b,a).
6.1 Extrapolation: Lines Between Models
We provide another implication of our findings. If indeed models in In' share partial information with models from In, this aggregated information may be general and useful for other tasks. In prac- tice, there are two common uses for trained model, either for the immediate classification of unseen examples or as a starting point for further training. We focus on the later use as a low loss directly indicates it could be useful in that setting.
In Section 5.1, we took pairs of models and found that the linear path between them passes through region of low loss. We now continue this path and check how far in the opposite directions (i.e. away from the model being interpolated to) do we need to move in order for the loss to rise. We reproduce the interpolations settings of $5.1, but apply linear extrapolation, i.e., test & values out of range [0,1]. We make 10 steps in logarithmic advances from 1 to 32 and similarly from 0 to -31. Figure depicts the results for the Same-Datased gramularity level. We provide more detailed results in App. G. We find that for all granularity levels ex- trapolation rapidly reaches bad performance. This implies the converged models are near the edge of the loss basin. We further observe that the region has relatively flat base and steep cliffs, implying that the regions we find are small basins and not e.g. subspace. In a sense, we discover bounded region that characterizes the loss region (of e.g.. MNLI dataset) where the models within have a low loss and the models beyond have a high loss.
We hypothesize that points in the region could be better for finetuning than finetuning the pretrained model itself. As there are endless possibilities of points in the region with no preference to any spe- cifie, we practically pick the centroid of the region, i.e. the average between models in In. The cen- troid point is equally influenced by each model defining the region, and without further informa- tion may be stronger than arbitrary points in the region (see App. $E), but also be suboptimal (see $5.1, App. ŞE).
For subsequent training, we employ parameter- efficient finetuning. Specifically, BitFit (Ben Zaken et al., 2022), one of the most parameter-efficien methods, which has been shown to attain strong performance. Changing only small subset of the weights reduces the complex effects of training dynamics and eases attributing improvements to the initialization weights. We avoid giving unfair advantage our method and for each target dataset choose the centroid of all models excluding ones finetuned on the target dataset itself.
7 Practical Takes
Our work has several practical implications. First, we observed ($5.2) that models inside the region (In') often superior to the finetuned models defining the region (In). Practically, one can aver- age models from the same region and cautiously expect the resulting model to perform better. This model can be used without further finetuning, in
We find (Fig. 6 and App. H) that starting from

---

the centroid results in better performing model than starting from a pretrained model, by 4.04% on average. The centroid is better in almost all cases, outperforning the pretrained in 9 cases, matching the results in 2, and underperforming in I case. Efficient finetuming is especially interesting in the scenario of scarce data (App. H). We hence replicate the results in few-shot scenario limiting the training examples K. The general trend is replicated, only that improvements reach as high as 34% improvement and above 10.66% on average.
linear path moves. Intriguingly, we have observed that these low-loss regions are unique for each specific dataset or task, whereas Juneja et al. (2022) has reported the existence of multiple basins per each. We also generalize this finding to models that were not trained on the same data and are tested on different data. Qin et al. (2022) is the only work know to compare models trained on different tasks. However, they report random chance perfor- mance in this case. To enable meaningful model comparison, we proposed the generalized loss ($3).
Our results also support and provide some pre- liminary explanations of recent practical findings. Some works show that starting from finetuned model helps when finetuning on a different target dataset (Choshen ct al., 2022a; Phang et al., 2018), which may be related to the fact that the initial finetuning stage moves the model into the general "language" region (or, even better, the region of space corresponding to the target task). Moreover, growing literature has shown improvements from averaging two more finetuned models. Some of those average models trained on the same dataset (Wortsman et al., 2022b,a), which we show picks a model from inside the dataset region. Others show that averages between models can improve models from tasks that they were not trained on (Choshen et al., 2022b; Matena & Raffel, 2021), which agrees with our more general findings. Ilharco et al. (2022) further suggests that some attributes can be added to the model by moving in certain directions in the weight space. parallel work, Ram' al. (2022) considers two finetuning stages before averaging. Lu et al. (2022) and Talman et al. (2023) propose optimization methods featuring Stochastic Weight Averaging (SWA). Our results may indicate that the success of such methods may be partly attributed to its tendency to fall within region, rather than its borders. More recent work considers iterative model averaging, where in each iteration multiple models are trained in parallel from the same initial point and then averaged to aggregate their knowl- edge. Such a procedure has been demonstrated both for self-supervised pretraining (Li et al., 2022) and as a supervised pretraining, similar to mas- sively multitask learning scenario Yohnya et al., 2022). Future work could focus on under- standing how those processes move through the weight space and whether they move to areas of loss space outside of the region corresponding to a single iteration of averaging finetuned models.
091
00P
A/c
-2
6
10
Accurasy Dilfenence
Figure 6: Centroid model gains over the pretrained. Models efficiently finetuned BitFit) over target datasets,
8 Explaining previous results
A great deal of prior work considered the con- nectivity between models, i.e. whether the path in weight space between two networks has a low loss throughout. Early work demonstrated that models trained on the same dataset have such path but that the path not necessarily linear (Garipov et al., 2018; Frankle et al., 2020). This non-lincarity was often explained by the fact that networks can represent the same function after their weights are permuted (Ainsworth et al., 2022; Jordan et al., 2022; Chen et al., 1993; Hecht-Nielsen, 1990). Taking into account these symmetries and/or using the same initialization was then shown produce a linear path of low loss (McMahan al., 2017; Entezari et al., 2021). Benton et al. (2021) even considered simplexes of low loss, rather than linear paths. In addition, Mirzadeh et al. (2020) showed that multitask learning converges to a point with low loss for both tasks, and in parallel work Qin et al. (2022) showed that the minima connected for two datasets of the same task. We generalize those notions in the context of finetuned models. Specifically, we confirm that indeed there is a linear path between two models, but further that there is a whole region with low loss through which the

---

9 Conclusion and Discussion
Combining all of our results together conveys a con- sistent message: There are regions in weight space corresponding to good performance on dataset, task, or in general. From $2.3 we can conclude that performant models are centered in certain areas (or more specifically basins) in weight space. We find in $5.1 that these form one basin rather than multiple nearby points falling into multiple basins and, in $5.2, that this basin is a convex region and not simply a line between two points. Finally, the extrapolations in $6 show those areas do not ex- ceed far beyond the finetuned models. Moreover, results suggest that models found via finetun- ing typically lie on the boundaries of these regions and often suboptimal, prompting future work in exploring the limitations of gradient-based training.
10 Limitations
We discuss limitations where relevant throughout the work, but also provide this section for general discussion of limitations.
Our work was only evaluated finetuning pre- trained model, and hence may not hold in general when randomly initializing. They also focused on English classification data.
While our results were very robust when refer- ring to tasks, we did not find many groups of datasets of distinet domains to test on and got mixed results in those aspects. We discuss the results in App. D.
The scope of our experiments is broad in some aspects it less so others. While our exper- iments included thousands of finetuned models, trained on 36 datasets and also evaluated on 36 datasets. We did not replicate it on many pretrained models as well.

---

References
Chen, A. M., Lu, H.-m. and Hecht-Nielsen, On the geometry of feedforward neural network error surfaces. Neural Computation, 5(6):100927,, 1993. doi: 10.1 1a.116/2neco.1900 1993.5.6.910.
Ainsworth, S. K., Hayase, J., and Srinivasa, S. Git re-basin: Merging models modulo permutation sym- metries. arXiv preprint (tti::222904888300 2022.
Choshen, L., Venezian, E.,. Rohnac S., Slonim, N., and Katz, Y. Where to start? analyzing the poten- tial value of intermediate models. arXiv preprint arXiv:2211.00107 2022a.
Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampic- colo, D., and Magnini, B. The second paseal recog- nising textual entailment challenge. 2006.
Barbieri, F. Camacho-Collados J., Ronzano, F., Espinosa-Anke, L., Ballesteros, M., Basile, V., Patti, V., and Saggion, H. SemFval-22018 Task 2: Mul- tilingual Emoji Prediction. In Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-22018) New Orleans, LA, United States, 2018. Association for Computational Linguisties.
Choshen, L.., Venezian, E., Slonim, N., and Katz, Y. Fusing finetuned models for better pretraining. arXiv preprint 2XXi:2224403344 20226.
Clark, C.. Lee, K. Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technolo- gies, Volume I (Long and Short Papers), pp. 2924- 2936, Minneapolis, Minnesota, June 2019. Associa- tion for Computational Linguisties. doi: 18.18/53/ v1/N19- 1300. URL https: //tclanthocoogy....ooooo org/ 119--
Barbieri, F., Camacho-Collados, J., Espinosa Anke, L.. and Neves, L. TweetEval: Unified bench- mark and comparative evaluation for tweet classi- fication. In Findings of the Association for Com- putational Linguistics: EMNLP 2020, pp. 1644- 1650, Online, November 2020, Association for Com- putational Linguistics. doi: 10.18653/v 1/2020. findings-emnlp. 148. URL https://aclaantholog. org/2820. findings- emm.p.. ..pp..... 148.
Dagan, I., Glickman, O., and Magnini, B. The pascal recognising textual entailment challenge. In MLCW. 2005.
Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F. M., Rosso, P., and Sanguinetti, M. SemBval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 54-63, Minneapolis, Min- nesota, USA, June 2019. Association for Computa- tional tional Linguisties. doi: 10. 18653/v 1/S 1007 URL. https://aclannhhology/ wwg/s1g-//////// 2007.
de Marneffe, M.-C., Simons, M., and Tonhauser, J. The CommitmemlBank: Investigating projec- tion in naturally occurring discourse. 2019. To appear in Proceedings of Sinn und Bedeutung 23. Data can be found at https: http/gtttbb.... www.ett...oooeeeeeeeeeeeeeeeooee com/mcdm/ Commi tmentBank/.
Dolan, W. B. and Brockett, C. Automatically con- structing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP200S), 2005. URL https: // /aallanhhlogy Sallanttology.org/Ieess. org.rrr--500...
Ben Zaken, E., Goldberg. Y., and Ravfogel, BidFit: Simple parameter-efficient fine-tuning for bransformer-based masked language-models. In Pro- ceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pp. 1-9, Dublin, Ireland, May 2022. Association for Computational Linguistics. URL https:
Don- Yehiya, S., Venezian, E., Raffel, C., Slonim, N.. Katz, Y., and Choshen, L. Cold fusion: Collaborative descent for distributed multitask finetuning. 2022.
doi: 10. 11111655//11222222clllhhoo //aclanthology htcplloggg.rgggg//// wwwp//2...c.ee//oo/.///...... ac1-22ortlssst1
Elazar, Y., Kassner, N., Ravfogel, S., Feder, A., Ravichander, A., Mosbach, M., Belinkov, Y., Schütze, H., and Goldberg, Y. Measuring causal effects of data statistics on language model's 'factual' predictions. 2022.
Bentivogli, L.,, Clark, P., Dagan, I,, and Giampiccolo, D. The sixth pascal recognizing textual entailment challenge. In TAC, 2009.
Benton, G., Maddox, W., Lotfi, S., and Wilson, A. G. G. Loss surface simplexes for mode connecting volumes and fast ensembling. International Conference on Machine Learning, pp. 769-779. PMLR, 2021.
Entezari, R., Sedghi, H., Saukh, O., and Neyshabur, B. The role of permutation invariance in linear mode connectivity of neural networks. arXiv preprint arXiv:2) 10.0696, 2021.
Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., and Blunsom, P. e-snli: Natural language inference with natural language explanations. In NeurlPS, 2018.
Frankle, J., Dziugaite, G. K., Roy, D., and Carbin, M. Linear mode connectivity and the lottery ticket hy- pothesis. In III, H. D. and Singh, A. (eds.). Pro- ceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 3259-3269. PMLR, 13-18 Jul 2020. URL press/v119/ httpv/1199//aaaae///1 html.
Cer, M., Diab, M. T., Agirre, E., Lopez-Gazpio. and Specia, L.. Semeval-2017 task I: Semantic tex- tual similarity multilingual and crosslingual focused evaluation. In International Workshop on Semantic Evaluation, 2017
mlr.

---

Garipov, T., Izmailov, P., Podoprikhin, D., Vetrov, D. P.. and Wilson, A. G. Loss surfaces, mode connectivity. and fast ensembling of dnns. Advances in neural information processing systems, 31, 2018.
Conference on the Principles of Knowledge Repre- sentation and Reasoning, 2012. URL http://dl. acm. org/ //cgg//ieee criiatiion.... cwwmid=mmmmdeece
303 3: 3031 1989.
Levesque, H. J., Davis, E., and Morgenstern, L. The winograd schema challenge. KR, 2011.
Giampiccolo, D., Magnini, B., Dagan, L,, and Dolan, W. B. The third pascal recognizing textual entailment challenge. In ACL-PASCAL@AAII 2007.
Li, M.. Gururangan, S., Dettmers. T., Lewis, M., Althoff, T., Smith, N. A., and Zettlemoyer. L. Branch-train- merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2008.03305 2022.
Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Teh, Y. W. and Titterington, M. (eds.), Proceedings of the Thirteenth International Conference on Artifi- cial Intelligence and Statistics, volume 9 of Proceed- ings of Machine Learning Research, pp. 249-2422555 256, Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR. URL https: / http//poceeennn..e//e mlr prrss// "9/glorot10a aatorrtttahttm..... htmll
Li, X. and Roth, D. Learning question classifiers. In COLING 2002: The 19th International Conference on Computatioonal Linguistics, 2002. URL https: //aclanthology, www./og-gggo// 1150.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D.. Levy, O., Lewis, M., Zettlemoyer, L.., and Stoyanov, V. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1107. 1692, 2019.
He, R. and McAuley, J. Ups and downs:
Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web. pp. 507-
Lu, P., Kobyzev, I., Rezagholizadeh, M., Rashid, A., Gh- odsi, A., and Langlais, Improving generalization of pre-trained language models via stochastic weight averaging. arXiv preprint (2.iv:22122.5955.. 2022.
517, 2016.
Hecht-Nielsen, R. of feedforward network weight spaces. In ECKMILLER, R. (ed.), Advanced Neural Com-
On the algebraic structure
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng. A. Y., and Potts, C. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computa- tional Linguistics: Human Language Technologies. pp. 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http: /ttp:///www. http://wwwwlwweebe.. aclweb. org/ org./antholooey//1111155
puters, pp. 129-135. Hooth-Holland, Amsterdam, 1990. ISBN 978-0-444- 88400-88. doi: https: V/doi.org/10. 10. URL
httpp://wwwt wcwenceeieeettccoocmmoo com/ 94.
Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururan- gan, S., Schmidt, I.., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022.
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., and Takala, P. Good debt or bad debt: Detecting seman- tic orientations in economic texts. Journal of the Association for Information Science and Technology. 65(4):782-796, 2014.
Jordan, K., Sedghi, H., Saukh, O., Entezari, R., and Neyshabur, B. Repair: Renormalizing permuted activations for interpolation repair. arXiv preprint arXiv:2211.0844 2022.
Matena, M. and Raffel, C. with fisher- weighted averaging. arXiv preprint arXiv:2111.0mm 11.09832, 2021.
Merging models
Juneja, J., Bansal, R., Cho, K., Sedoc, J., and Saphra, N. Linear connectivity reveals generalization strategies. arXiv preprint arXiv:2205. /2411, 2022.
McMahan, B., Moore, F.,, Ramage,
D., Hampson, S., learn-
and y Arcas, B. A. Communication ing of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273-1282. PMLR, 2017.
Khashabi, D.,, Chaturvedi, S., Roth, M., Upadhyay, S., and Roth, D. Looking beyond the surface: chal- lenge set for reading comprehension over multiple sentences. In Proceedings of the Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies (NAACI-III.T)) Association for Computational Linguistics, 2018. URL. ht tptps// /ww.. aclweb. org wrrg/anthology/apeeee hatpp//ppps//////// 8/1N111//111111 1023/.
Merrill, W., Ramanujan, V., Goldherg, Y., Schwartz, R., and Smith, N. A. Parameter norm growth during training of transformers. 2020.
Mirzadeh, S. I., Farajtabar, M., Gorur, D., Pascanu, R., and Ghasemzadeh, H. Linear mode connectivity in multitask and continual learning, arXiv preprint arXiv:2010.044 04495, 2020.
Kornblith, S., Norouzi, M., Lee, H.. and Hinton, G. Similarity of neural network representations revisited. In International Conference on Machine Learning, pp. 3519.3529. PMLR, 2019.
Mohammad, S. M. and Bravo-Marquez, F. Emotion intensities in tweets. In Proceedings of the sixth joint conference on lexical and computational semantics (*Sem), Vancouver, Canada, 2017.
Levesque, H., Davis, E. and Morgenstern, L. The Wino- grad schema challenge. In Thirleenth International

---

Nie. Y.. Williams, A., Dinan, E., Bansal, M., Weston, J., and Kiela, D. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4885-4901 Online, July 2020, Association for Computational Linguistics. doi: 10. 18.11/66// 1/2020..11222/01/// wwwp:/0000.--mmii.4444 441. URL https: / //aclanthology org wwwg./200..2/ acl acl-main..eem 441.
Ram'e, A.. Ahuja, K., Zhang. J., Cord, M.. Bottou, L., and Lopez-Paz. D. Recycling diverse mod- els for out-of-distribution generalization. ArXiv, abs/2212.10445 2022.
Roemmele, M., Bejan, C. A., and Gordon, A. Choice of plausible alternatives: An evaluation of common- sense causal reasoning. In 2011 AAAT Spring Sympo- sium Series, 2011.
Pang, B. and Lee, L. Seeing stars: Exploiting class relationships for sentiment categorization with re- spect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computa- tional Linguistics (ACT. (05), PP. 115-12,, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics. doi: 11111////1991111991119 nttps://aclanthollog. org/p05- 1015.
Rosenthal, S., Farra, N., and Nakov, P. SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceed- ings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 502-518, Vancou- ver, Canada, August 2017. Association for Computa- tional Linguistics. doi: 10. 1/815// https: https://aclanttoloogy wwwpllooo.ooog...g/o wrgg:/7-2//////
URL
URL
Pedregosa, F., Varoquaux, G., Gramfort, A.. Michel, V. Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournaeaa, D.. Brucher, M., Perrot, M., and Duchesnay. E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12: 2825-2830, 2011.
Scherer, K. R. and Wallbott, H. G. Evidence for univer- sality and cultural variation of differential emotion response patterning. Journal of personality and so- cial psychology. 66(2):310, 1994.
Sheng, E. and Uthus, D. Investigating societal biases in a poetry composition system. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pp. 93-106, Barcelona. Spain (Online), December 2020. Association for Compu- tational Linguistics. URL. org/2020. 9e0.00..eblpp11..
Phang, J., Févry, T., and Bowman, S. R. Sentence encoders on stilts: Supplementary training on inter- mediate labeled-date data tasks. ArXiv, abs/1811.01000 2018.
Pilehvar, M. T. and Camacho-Collados, J. WiC: The word-in-context dataset for evaluating context- sensitive meaning representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies (NAACL-HLI) Asso- ciation for Computational I. Linguistics, 2019. URL https: //ar:v///xxvvv/ www.voog//aaaa.... wttg/br//////////// 89121.
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A.. and Potts, C. Recursive deep mod- els for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference Empirical Methods in Natural Language Processing. pp. 1631-1642, Seattle, Washington, USA, Octo- ber 2013. Association for Computational Linguisties. URL
Drg/
Poliak, A., Haldar, A., Rudinger, R., Hu, J. E., Pavlick, E., White, A. S., and Durme, B. V. Collecting diverse natural language inference problems for sentence rep- resentation evaluation. In Conference on Empirical Methods in Natural Language Processing, 2018.
Talman, A., Celikkanat, H., Virpioja. S., Heinonen, M., and Tiedemann. J. Uncertainty-awar natural lan- guage inference with stochastic weight averaging. arXiv preprint arXiv:2304.0477 230400426.. 2023.
Toledo, A., Venezian, E., and Slonim, N. Revisiting sequential information bottleneck: New implementa- tion and evaluation. Entropy, 24(8):1133 2022.
Qin, Y. Qian, C., J., Chen, W., Lin, Y., Han, X., Liu, Z., Sun, M.. and Zhou, J. Exploring mode connectiv- ity for pre-trained language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pp. 6726-6744 0726-6746, Abu Dhabi, United Arab Emirates, December 2022, Asso- ciation for Computational Linguistics. URL https: //aclanthology wrg./022.cc... 2022. w0222.emll.mmmmmmmmmmmmmmm2....1 smnlp-main
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9 (1)), 2008
Van Hee, C., Lefever, E., and Hoste, V. SemEval- 2018 task 3: Irony detection in English tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pp. 39-50, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/S18. 1005. URL https: httpp:://aclanthology.. Stg.////....///1-///////
Rajpurkar, P., Zhang. J.. Lopyrev, K., and Liang, P. SQuAD: 100,000+ questions for machine compre- hension of text. In Proceedings of the 2016 Con- ference on Empirical Methods in Natural Language Processing, pp. 2383-2392, Austin, Texas, Novem- ber 2016. Association for Computational Linguis- tics. doi: 10. 10D1666//1/D16 1264. URL https: / //aclanthology wwwelloo..oooggggo wtgo 1264.
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O.. and Bowman, S. GLUE: A multi-task benchmark

---

and analysis platform for natural language under- standing. In Proceedings of the 2018 EMNLP Work- shop BlackboxNI P: Analyzing and Interpreting Neu- ral Networks for NLP, pp. 353-355, Brussels, Bel- gium, November 2018. Association for Computa- tional Linguistics. doi: (0.18653/vI// URL https://aclaantholggg. org//18-544.
Wang, A., Pruksachatkun, Nangia, N., Singh, A.. Michael, J., Hill, F., Levy. O., and Bowman, S. R. Superglue: A stickier benchmark for peuproe language understanding systems. NeurIPS, 2019,
Warstadt, A., Singh, A., and Bowman, S. R. Neural network acceptabilityy judgments. Transactions of the Association for Computational Linguistics, 7:625- 641, 2019. doi: 10.1162/tacl a 00290. URL https: //aclanthology -
Williams, A.. Nangia, N., and Bowman, S. A broad- coverage challenge corpus for sentence understand- ing through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume (Long Pa- pers). pp. 11-2-1 122, New Orleans, Louisiana, June 2018 8a. Association for Computational Linguis- tics. doi: 10. 18853/111/N8.. 1101. URL https: / //aclanthology wwg/N18---- 1101.
Williams, A.. Nangia, N., and Bowman, S. A broad- coverage challenge corpus for sentence understand- ing through inference. In Proceedings of the Con- ference of the North American Chapter of the As- sociation for Computational Linguisties: Human Language Technologies (NAACL-HLT), Association for Computational Linguisties. 2018b. URL http; //aclweb. org/ a
Wortsman, M., Gururangan, S., Li. S., Farhadi, A., Schmidt, I.., Rabbat, M., Morcos, A. lo-fi: dis- tributed fine-tuning without communication. arXiv preprint arXiv:2210.1 1948, 2022a.
Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., and Schmidt, L. Model soups: averaging weights of multiple fine- tuned models improves accuracy without increasing inference time. 2022b.
Yadav, P., Tam, D., Choshen, I.., Raffel, C., and Bansal, M. Resolving interference when merging models. ArXiv, abs/2306.00 0100060..770.. 708, 2023. URL https://api. semanticscholar Irg/CCrp II:rPpuuID:299640053 259064039.
Zampieri, M.. Malmasi, S. Nakov, P., Rosenthal, S., Farra, N., and Kumar, Predicting the Type and Target of Offensive Posts in Social Media. In Pro- ceedings of NAACI, 2019.
Zhang, X., Zhao, J., and LeCun, Y. Character-level con- volutional networks for text classification. Advances in neural information processing systems, 28, 2015.

---

A Dataset List
Most datasets could be downloaded from huggingface datasets. We explicitly state the download link when relevant. As we used groups of datasets we report here the full list of datasets they contain.
General: CoLA (Warstadt et al., 2019), SST2 (Socher et al., 2013), MRPC (Dolan & Brockett, 2005), QQP (data. www.-uuuoraa... cttpeaaatoooi/ttttuueee
(usllasseeuusstii--arrs))) MNLI (Williams et al., 2018a), QNLI Rajpurkar al. 2016, RTE (Dagan et al., 2005; Bar-Haim et al., 2006; Giampiccolo al., 2007: Bentivogli et al., 2009), WNLI (Levesque al., 201 1) BoolQ (Clark al., 2019), CB (de Marneffe et al., 2019), CoPA (Roemmele et al., 20,1), MULTIRC (Khashabi et al., 2018), WIC (Pilehvar & Camacho-Colladde 2019)
NLI datasets: MNLI (Williams et al., 20 8a), QNLI Rajpurkar et al. 2016, RTE (Dagan et al., 2005; et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009), WNLI (Levesque et al., 201 1), ESNLI (Camburu et al., 2018), adversarial NLI (Nic et al., 2020).
Twitter domain datasets (collected by TweetEval (Barbieri al., 2020)) Emolnt (Mohammad & Bravo- Marquez, 2017), Emoji (Barbieri al., 2018), Irony (Van Hee et al., 2018), OffenseEval (Zampieri et al., 2019), HatEval (Basile al., 2019), Sentiment Analysis (Rosenthal et al., 2017)
Sentiment Analysis: Poem Sentiment (Sheng & Uthus, 2020), IMDB (Maas et al., 2011), Rotten Tomatoes (Pang & Lee, 2005). SST 5bins (Socher et al., 2013), SST2 (Socher et al., 2013), Amazon reviews (He & McAuley, 2016) ,Financial Phrasebank (Malo et al., 2014)
Topic Classification: AG news (Zhang et al., 2015), ISEAR (Scherer & Wallbott, 1994). Yahoo answers (Zhang et al., 2015), DBpedia (Zhang al., 2015), 20 newsgroup (Zhang et al., 2015), TREC in both
fine-grained and coarse-grained labels (Li & Roth, 2002)
B Similarity Per Dataset, when Starting from different Pretrained Models
After seeing in $2.3 the repeated behavior on several granularity levels, we were curious whether we could receive the same behavior on larger granularity level models starting from different pretrained RoBERTa models, and finetuned on the same datasets. In this experiment, we employ two pretrained RoBERTa models, the original RoBERTa- base and the re-implementation of RoBERTa- base created by Elazar et al. (2022). We finetune each one on the same datasets, from the General family. Results show that the models get clustered according to the pretrained model they were created from, regardless to the finetuning they went through. This might arise from the low distances moved from the initialization. pretraining changes the model's weights much more than finetuning. Therefore, since start from different pretrained models, the resulted finetuned models are more similar to the pretrained model they started from.
As the results on both pretrained models are comparable, we deduce that there not one unique basin or region for each ability, but many. However, around a starting point it seems there are distinct regions within reach.
C Cause: Data Type, not Size
We provide the clustering and visualize with t-SNE in Fig. 7. We see that the clustering and the data type agree in all but one of the cases.
We provide in Fig. 8 a detailed view of the similarities between each pair of models by dataset and amount of data seen in training. We find that with relatively little data, the direction in space is already determined, i.e., similar datasets go to similar direction even with limited amount of data.
Similarity Per Task and Domain
As noted in 2. the datasets we use can be separated into specific four dataset families in addition to the general group: NLI. Sentiment analysis, Topic, and Twitter. while the first three are characterized by their task, the last group characterized by the domain the dataset it contained. As one can see Fig. and although the clustering shows good separation between task groups, it struggles to separate the Twitter domain group models from the other groups. Separating the space into 4 clusters and labeling them in a

---

CoLA
SST2
MRPC
QQP
1-SNET y
QNLI
STSB
BooiQ
MultiCC
WIC
t-SNE
Figure 7: Clusters of finetuned models on different datasets, with increasing train set sizes, projected by 1-SNE, Each model is represented as dot, where inner color is the color of the dataset the model was finetuned with, and outer color is the color of the most common dataset in the cluster (representing the cluster label). Datasets names are shown in legend.
Experiment/class
Twitter
NLI
Topic
Sentiment
Avg
Fl Cluster Tasks and Domain
30
100
61
71
63
FI Cluster Tasks
100
87
83
90
Table 1: Fi Score Classification performance by cluster majority, columns, model group names, rows the clustering settings, with and without the domain group (Twitter).
I-to-1 mapping to maximize accuracy, we find 31 f-score on the Twitter cluster and 62,71,1 on the Topic, Sentiment and NLI groups respectively.
A possible explanation may be that the domain feature is orthogonal the task feature, in the sense that some datasets should be assigned to two groups at the same time (for example TweetEval Sentiment Analysis (Rosenthal et al., 2017) is part of the Twitter domain group, as well as the Sentiment analysis task group). This gives place to two competing hypotheses that we leave unanswered. Either the regions of domains overlap with regions of tasks; or, even if less likely, domains are not separated into regions in space, unlike tasks.
E Interpolation Between Models
We provide more comprehensive interpolation experiment. In we show the interpolation between pairs of models and report the loss each of the datasets used to create the pair of models, as well as the average reported in the main paper.
In Fig. 10, one can see not only the interpolation between models in In, but interpolation between the centroids. We take the average of all the models in group from which interpolate (c.g., all centroid. We then repeat on the other group and interpolate between those
MNLI models) and set centroids instead of interpolating between actual finetuned models. We find that although now we are interpolating between two points that were both not the outcome of traditional ways optimization, we find comparable and often even lower losses than before. This also motivates the practical experiments reported in $7.

---

Figure 8: Cosine similarity between models trained on different datasets, with varying data sizes (blocks). The diagonal per block is blurred at the beginning of training, but with still a small amount of data models are highly similar to models trained on similar data. We do not observe similarity between models of similar size.
F Loss Region Outside of Models in Other Directions
After seeing that reach outside of regions by performing Ainear-extrapolatinn mance of models when move away to different directions. To test it, start with several models of the same region, calculate their centroid by averaging their weights, and gradually move away from this centroid according to the same procedure as in section 3. We move away from the centroid towards one two directions: towards the origin of the axis, or towards random directions. We evaluate on the same datasets the In models were finetuned on.
test the perfor-
Figure 11 shows the results for the first and third granularity levels.
A detailed analysis for each level follows.
Outside of the Dataset Region. We compare the performance of three types of models: finetuned models on MNLI, models moving from the centroid of MNLI models to the origin, and models moving from it to random directions.
Results show that when the distance of the generated models from the centroid is similar the distance of the finetuned models (radius <. 1). the generated models perform as well the finetuned models, meaning we are still inside the MNLI region and all models share the knowledge needed for the MNLI target task. It also implies the directions which finetuned models vary are not special, most changes around the center are equally acceptable.
When the distance increases and we move farther away from the centroid, the performance of the randomly generated models decreases significantly, indicating the end of the region. A surprising finding is that this happens random directions. but not when going towards the origin. The performance in that case is similar the performance of the finetuned models, when the models farther from the centroid then the finetuned models. While did not expect this phenomenon have an explanation it, we report as an avenue for future work.
Outside of the Finetuning Region. We compare the performance of three types of models: finetuned models on datasets from the General family, models starting from the centroid of those models towards the origin towards random directions. Each time, we evaluate all above models on single dataset from the General family, separating the performance of the model finetuned on the target dataset (called source model), to the rest of finetuned models (called non-source models), resulting in total four types models in the comparison, including the two types generated models starting from the centroid.

---

TWEET
t-SNE
NLI
SENT
t-SNE
Figure Clusters finetuned models, trained on datasets groups, distinct by and domain. The models projected by t-SNE, where each model is represented as dot, where the inner color is the color of the task/domain the model was finetuned with and the outer color is the color of the most common task/domain in the cluster (representing the cluster label). We find that tasks are can easily distinguished, while is hard to separate Twitter domain models.
We average the performance of each type on target datasets we evaluate on, and show the results Figure (b)... We can see that the source model outperforms all other models. small distances from the centroid, the non-source models underperform the generated models, and for large distances outperform the generated models going towards random directions. The generated models going towards the origin outperform the two above types of models, for all distances. These results suggest that when staying close enough to the centroid, roaming from the centroid to different directions might be superior to a finetuned model other datasets then the target dataset perform better than generated models going towards random directions, since the last are probably outside of the region. Worth noticing, the standard deviation of the last meaningfully larger than the rest of the models, and of the one of generated models in the Dataset granularity level.
different dataset. However, when distancing far from the centroid, finetuned models
Extrapolation Between Models
Fig. 12 presents the same behaviour for all three granularity levels- extrapolation rapidly reaches bad performances.
We provide a more comprehensive extrapolation experiment showing each time the extrapolation with the loss of each of the datasets used to create the pair of models, and the average reported in the main paper. We find (see Fig. 13(b) that despite all of our datasets called and framed as natural language inference, WNLI (Levesque et al., 201 1) behaves differently and might be considered not strictly a part of the region. This may also explain the long tail in Fig. 4(b).
H Efficient Finetuning
We provide in this section the full results of efficiently finetuning. We provide the full results of the regular finetuning Table and the few-shot setting in Table 3 and Fig. 14.

---

MNLI
SST2-MNLI
SST2-SST2
AVG
0224
R2233
Average Loss
avg tasks
0.21
0.20
0-100E
0.19
0.30
02911
0.3043
Loss on MNLI
N 10
25
N = 10
avg
0.20
0.1218
1289
1.221
Loss on SST2
0.12
N
10
SST2
0-1162
avg tasks
0.10
0015
00750
P.0649 p.0677
0.06
(0.1)
(1.0)
[0.1)
(1,0)
(0,1)
(1,9)
Position
Position
Position
(a) Interpolation Per Dataset
Figure 10: Losses linearly interpolated models created between pairs of similar models. each figure, solid line is the losses' average during interpolations for different values, the edges of the lines represent the pure finetuned models we interpolated, Y axis is the average loss value, X axis is the position determined by a, N is the number of pairs interpolated between. The minimum average loss during the interpolation is noted and the shade is the standard deviation of the losses average. The purple line provides the average loss of the interpolation between centroids of models.
dataset name boolg
cb
cola
muli
mipe
multire
qali
rte
ss12
wic
wali
mean
Pretrain
62.17
50.36
69.13
53.17
68.38
57.20
64.88
74.49
50.40
78.78
55.14
54.08
61.51
Fuse
62.23
56.79
69.49
63.01
69.46
57.14
73.77
79.91
61.59
84.91
55.52
52.68
65.54
Gain
0.06
6.43
0.36
9.85
1.08
-0.06
8.89
5.42
11.19
6.12
0.38
-1.1
4.03
Table 2: Gains of efficient finetuning starting from the centroid the pretrained model. In columns names of datasets (mean is their average) and in rows the choice of base model and their difference, the gain.
Outside Dataset Region
Outside of Finetuning Region
Figure 1I: Performance of the finetuned and the generated models from the centroid to the origin and to random directions, with respect to the distance from the region. each graph, axis is the accuracy, axis is the radius (which is the values used for generating the models. Only relevant for the constant lines), the solid lines present the average accuracy of the generated models, the dashed lines present the average accuracy of the finetuned models (a constant value), and the shade is the standard deviation of the accuracies average. Models" groups legend.

---

6.5
0.2
330
030
6.8
(a) Extrapolation per dataset.
(b) Extrapolation per task.
(c) Extrapolation General.
Losses of linearly extrapolated models created from pairs of similar models. In each figure, the solid line the average losses during extrapolations for different a values, the vertical dashed lines indicate the average loss of the pure models extrapolate (a 0 position (meaning the and (1 - a') values used in the extrapolation). The shade is the standard deviation of the losses' average across runs.
Figure 12:
the Y axis is the average loss value, and the X axis is the
dataset name
boolq
cb
cola
mnli
mrpc
multire
quli
rte
sst2
wic
wali
mean
Pretrain
62.17
50.36
69.13
34.04
68.38
57.20
50.72
63.18
48.52
50.92
49.91
54.08
54.88
Fuse
62.23
56.79
69.49
63.01
69.46
57.14
73.77
79.91
61.59
84.91
55.52
52.68
65.54
Gain
0.06
6.43
0.36
28.97
1.08
-0.06
23.04
16.74
13.07
33.99
5.61
-1.41
Table 3: Gains of efficient finetuning with up to IK examples, starting from the centroid or the pretrained model. In columns names of datasets (mean is their average) and in rows the choice of base model and their difference, the gain.

---

10.00
Puutoo
P000000
(a) Extrapolation Per Task and mixed
VILESSUU
VINU NELL
-155
119.
-22
12.7
2x7
107
160
12.00
165
(b) Extrapolation Per Domain
Figure 13: Losses of linearly extrapolation models created between pairs of similar models. In each figure, the solid line the average losses during extrapolations for different a values, the vertical dashed lines indicate the average loss of the pure models (meaning the a and (1 a) values used the extrapolation), N is the number of pairs we extrapolated between, the values top of the line the loss at the edges and at the minimum average loss during the extrapolation, and the shade is the standard deviation of the losses average. Each Column represents extrapolation between different types models and each row evaluates those same models and their extrapolations on a different target tasks.
extrapolate (a 0 or 1), Y axis is the average loss value, X axis is the position

---

SST2
QNLI
RTE
WIC
CoLA
BoolQ
MuluRC
5
10
15
20
25
30
35
Accuracy Difference
Figure 14: Losses pretrained and centroid models on several target datasets, where both models were efficiently finetuned using BitFit in a few-shot scenario limiting training data to 1K.


Reviews:
[{"review_id": "4BHM59bERP", "reviewer": "Reviewer_UZHb", "summary": "", "questions": "See **Reasons To Reject**", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your support, as well as the clear and accurate summary that implies that our submission was well-understood.\n\nWe agree regarding the benefit of a deeper theoretical exploration of why these weight space clusters exist and how they facilitate knowledge transfer. At this point, we can only demonstrate the existence of these regions in hopes of motivating future work towards understanding why these regions exist and how they correspond to knowledge transfer. We consider our work as an important first step in identifying this characteristic that will motivate future work on exploring it deeper.\n\nThank you for your style improvements, we will fix those to ensure a consistent style.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper offers a unique and intriguing perspective by investigating the relationships between different models in the weight space and loss landscape. This sheds light on the transferability of knowledge between models and provides a basis for improving fine-tuning processes.\n\nThe authors provide solid empirical evidence to support their claims. Through experiments and analyses, they demonstrate the existence of clustered regions in the weight space, the improved performance of models within these clusters, and the potential for generating high-performing models through traversal within these regions.\n\nThe proposed fine-tuning method based on starting from the cluster center is a practical contribution. It offers a systematic approach to leveraging the knowledge contained within the weight space clusters.\n", "reasons_to_reject": "While the empirical evidence is compelling, the paper could benefit from a deeper theoretical exploration of why these weight space clusters exist and how they facilitate knowledge transfer.\n", "questions_to_authors": "See **Reasons To Reject**", "paper_topic_and_main_contributions": "This paper investigates the relationships between different models, particularly those trained or tested on different datasets. While most previous research has concentrated on understanding individual models trained on specific datasets, this study delves into the connections between various models trained or tested on distinct datasets. The findings highlight that a model positioned between two similar models can assimilate the knowledge of both, enabling the authors to propose an efficient fine-tuning method that starts from the center of the cluster, outperforming the traditional approach in terms of accuracy.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "It is advisable to use consistent annotation and font style, e.g. line 415-417 $ln$ and $\\text{ln}$"}, "extra_text": {}}, {"review_id": "u1ZDXVDFXS", "reviewer": "Reviewer_JyVr", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the positive review and for finding the paper as interesting and useful.\n\nRegarding your questions:\n1. The main message of the paper is as you stated at the beginning, but this is not a method paper. We do show that our findings have practical implications but do not presume to say this is the central contribution or the main point of the paper. We believe this is also the source of confusion. \n\n    * One way to exploit this finding (well-performing models form a tight cluster in weight space) is to select the center of the cluster as a starting model for fine-tuning, as you mentioned in the paper’s contributions. Additionally, since models inside the region share partial information with other models in the region (and outperform even the edges of the region), using any model from the region as a starting model should benefit the user. Another possible way to leverage our insights is to average models from the same region and expect the resulting model to perform better in the Same-Dataset region, without further fine-tuning.\n\n    * Guaranteeing that this finding will always be true is hard. While our experiments included thousands of fine-tuned models, trained on 36 datasets and also evaluated on 36 datasets, and we observed the same behaviour of tight clusters in all of them, we did not replicate it on many pretrained models yet. We believe future research could continue to investigate this behaviour and possibly find a promise or conditions for it.\n\n2. We agree that it is beneficial to provide a list of fine-tuning practices and elaborate on practical implications in Section 7. We included several practices and demonstrated a possible practice for subsequent training and its results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) Very interesting perspective on weight space analysis, clustering/visual analysis look very explanatory.\n2) Observation that well-performing models are centered in certain areas of the weight space might be useful for potentially selecting the final model (as opposed to standard evaluation).", "reasons_to_reject": "1) The main message of the paper is still not very clear - how can we exploit this finding (well-performing models form a tight cluster in weight space)? Is this always true?\n2) It would be good to provide a list of useful fine-tuning practices based on authors' findings.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors study how the weight space and the underlying loss landscape of different models are interconnected. Based on their findings, the authors propose a better method for selecting a model for fine-tuning.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "39HjFUvoWW", "reviewer": "Reviewer_xwZQ", "summary": "", "questions": "1. The experimental dataset of the paper is mainly concentrated on English classification data, can the similarities and differences between fine-tuned models be studied on datasets of other languages and fields?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the positive feedback and for finding many reasons for accepting the paper.\n\nRegarding your question about other languages and fields:\nWe assume they behave in a similar way to the English classification datasets, as it is unlikely that the similarities and differences between fine-tuned models are language- or field-specific. However, in order to deeply research the relationship between the weight space and loss landscape, we chose to focus on a specific language and field and leave other datasets for future work. It is worth noting that the datasets we studied cover a wide range of tasks, and we observed the same similarities and differences in all of them. This also supports the belief that this will be the case for other datasets. While not strictly applying the same methodology, an ongoing work of ours tries to shed more light on multilingual model weights. We see that as a separate branch of work.\n\nRegarding your comment about further research on how to use these findings to improve the performance of the models and explain these findings, we definitely agree. We consider our work as an important first step in researching these findings and hope it will motivate future work on explaining them and finding more ways to exploit them. We would like to point out that we elaborated on several practical improvements and demonstrated one of them in Section 7.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper conducts in-depth research on the relationship between the weight space and loss landscape of fine-tuned language models, providing new ideas and methods for the study of deep learning.\n2. By fine-tuning 36 datasets, the authors discover the similarities and differences between fine-tuned models and propose a new method to visualize the similarities between fine-tuned models.\n3. The paper proposes a new method to evaluate the performance of fine-tuned models, which can more accurately assess the performance of fine-tuned models. Besides, this paper explores the limitations of fine-tuned models, providing new ideas and methods for the study of deep learning.", "reasons_to_reject": "1. Although the experimental results of the paper show that the similarities and differences between fine-tuned models have an important impact on the performance of the models, further research is needed on how to use these findings to improve the performance of the models and explain these findings.\n", "questions_to_authors": "1. The experimental dataset of the paper is mainly concentrated on English classification data, can the similarities and differences between fine-tuned models be studied on datasets of other languages and fields?", "paper_topic_and_main_contributions": "This paper explores the relationship between the weight space and loss landscape of fine-tuned language models, and how to navigate the region between models to achieve better performance. The authors studied the weight space and loss landscape of fine-tuned models by training on 36 datasets and found similarities and differences between the fine-tuned models. Additionally, they proposed a new method to visualize the similarities between fine-tuned models and discussed the limitations of fine-tuning techniques in deep learning. The theme of this paper is fine-tuning techniques in deep learning, with a focus on the relationship between the weight space and loss landscape of fine-tuned models.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QTUcLw6FBG", "reviewer": "Reviewer_TEP2", "summary": "", "questions": "1. This paper discussed a lot of the weight space of full parameters. Why do you compare your centroid initialization with pre-training initialization for 'efficient fine-tuning [1]'? What is the performance comparison of your fuse method and regular finetuning (full-parameter)?\n\n2. What is the point if you have to fine-tune the full parameters of several language models first to get the centroid and then perform efficient fine-tuning [1]?\n\n3. How do you handle the trainable parameters of layer-norm layers when you do interpolation/extrapolation of different models?\n\n[1] BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language models. Zaken et al  ACL22", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the review and deep reading, and for pointing out that the paper “shows interesting findings to this community”. In general, we are of one thought that the main part of the paper is the analysis, and the last practical part is mostly an example of how this might be beneficial, We also mention other cases where this explains previous practical findings.\n\nRegarding your questions: \n1. Regarding the two questions about weight space of full parameters: \n    * Motivation for using the centroid (also asked about in reason to reject  #2) - Thank you, we will elaborate on it in the camera-ready version of the paper. Essentially, our findings suggest that certain regions perform well, and it is not clear what in them performs better (but not the edges or outside them). As there are endless possibilities of points in this region, we presumed the centroid is a bit less of an arbitrary choice than others, it might also avoid leaning towards any specific model and for us was the intuitive choice without adding further information. Therefore, for efficient fine-tuning we selected the centroid as a single, practical and efficient initialization point. \n    * Regarding the comparison to efficient\\regular fine-tuning - \nWhen considering (efficient) fine-tuning, we see the choice of fine-tuning method and initialization as two complementary choices. The user chooses the fine-tuning method, and we provide a preferable initialization to fine-tune. \nIn terms of comparing to a finetuned model -- even without fine-tuning, the fused models already outperform the finetuned models on the task. We show in sections 5.1 and 5.2 that the fused models (In’) perform comparably or even better than the regular fine-tuned ones (In) which they are created from, even though the In’ models are not fine-tuned by themselves. \nThis implies that full fine-tuning would be even better, as previous papers indeed show. We cited them in sections 5.2 and 8, but we will emphasize this in the camera-ready version.\n\n2. Regarding efficiently fine-tuning finetuned models- We fine-tuned the full parameters of several language models for the purposes of the research, to get a set of models that we know their training settings and procedure, and to avoid possible confounds. However, in a practical setting one can take existing fine-tuned models, for example from Hugging Face or users of the same service, and use them to define the region. Thus, the only computation done would be to average (CPU) and efficiently finetune. \n\n3. Regarding the layer-norm parameters- In practice, we had no issues with them and therefore we did not need to handle them separately. We assume that their norm is similar enough, as they started from the same pre-trained model and did not diverge too much. Some papers deal with fusing norm-layers (cited in the related work), but it is always in the context of training from scratch where this issue is more critical.\n\nRegarding missing citations - thank you for noting that, we will add these papers in the camera-ready version. \n\nRegarding the BitFit results - it is an interesting point we did not note while writing the paper as we focused on comparing the two BitFit cases to each other.\n\n\nWe are sorry it got a bit long. We have tried to address all your concerns in a way that respects the depth of your questions, and believe that our paper has improved considerably by emphasizing those points in our writing.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper investigated an important problem: understanding relationships between fine-tuned models in weight spaces.\n2. They conducted experiments on extensive datasets and analyzed the interpolation and extrapolation of language model encoders.\n3. Their findings are well-motivated and thoroughly demonstrated through detailed experiments and visualizations. For example, the comparison between interpolation and extrapolation in the weight space significantly shows the properties of different regions.", "reasons_to_reject": "1. The baseline results in Table 2 are pretty poor compared to both full-parameter and partial-parameter fine-tuning. For instance, MNLI: (53.17), QNLI: ( 64.88), QQP: (74.49), MRPC: (68.38), and SST2: (78.78). Although BitFit [1] use BERT models, the performance gap is quite large, In their paper, BitFit can achieve on-par performance as full-parameter fine-tuning. Could you explain it? \n\n2. The motivation to use the centroid of fine-tuned full-parameters models as the initial point of efficient fine-tuning is missing.\n\nOverall, I think this is a solid empirical analysis work studying the weight space of the fine-tuned language models, the authors provide enough analysis and show interesting findings to this community. It seems they want to propose a method based on their findings, however, I think this part is not well-motivated and the experiments are weak. Besides, several related works are missing.", "questions_to_authors": "1. This paper discussed a lot of the weight space of full parameters. Why do you compare your centroid initialization with pre-training initialization for 'efficient fine-tuning [1]'? What is the performance comparison of your fuse method and regular finetuning (full-parameter)?\n\n2. What is the point if you have to fine-tune the full parameters of several language models first to get the centroid and then perform efficient fine-tuning [1]?\n\n3. How do you handle the trainable parameters of layer-norm layers when you do interpolation/extrapolation of different models?\n\n[1] BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language models. Zaken et al  ACL22", "paper_topic_and_main_contributions": "This work explores the relationship between fine-tuned language models in weight space empirically. They found that models fine-tuned on the same dataset cluster together in the weight space and models fine-tuned on the same task also form clusters though looser ones. Models fine-tuned on any language tasks seem to reside in a constrained region compared to random perturbations of pre-trained weights.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Linear Connectivity Reveals Generalization Strategies. Juneja et al. ICLR23.\n[2] Improving generalization of pre-trained language models via stochastic weight averaging. Peng et al. EMNLP22.\n[3] Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. Talman et al. NoDaLiDa23."}}]

Author rebuttal:

Meta-review: This paper studies how the weight space and underlying loss landscape of different models are interconnected. The authors have provided insightful findings as summarized well by one of the reviewers that "They found that models fine-tuned on the same dataset cluster together in the weight space and models fine-tuned on the same task also form clusters though looser ones. Models fine-tuned on any language tasks seem to reside in a constrained region compared to random perturbations of pre-trained weights." Based on these analysis, the authors propose to create a better model by picking center of a region in the weight space, resulting in superior results. As summarized by the reviewers and myself, the pros and cons are:

### Pros:
1. This paper provides very novel and insightful empirical analysis as well as discussion on the weight space and loss landscape of different fine-tuned models. This is relatively new yet important direction for knowledge transfer.  
2. "The authors provide solid empirical evidence to support their claims. Through experiments and analyses, they demonstrate the existence of clustered regions in the weight space, the improved performance of models within these clusters, and the potential for generating high-performing models through traversal within these regions."

### Cons:
1. "how to use these findings to improve the performance of the models and explain these findings."  
2. Baseline results in Table 2 are poor.

Most of the reviewers agree that this paper is sound and exciting. Even though that this paper does not demonstrate the practical application of the findings in a comprehensive way, I think this analysis itself is interesting and insightful enough to stand as a nice paper alone.
Decision: Accept-Findings