Merging Text Transformer Models from Different Initializations
Authors: Neha Verma
Maha Elbayad
Venue: Accepted by TMLR ()

Abstract: Recent work on one-shot permutation-based model merging has shown impressive low- or zero-barrier mode connectivity between models from completely different initializations. 
However, this line of work has not yet extended to the Transformer architecture, despite its dominant popularity in the language domain.
Therefore, in this work, we investigate the extent to which separate Transformer minima learn similar features, and propose a model merging technique to investigate the relationship between these minima in the loss landscape. 
The specifics of the architecture, like its residual connections, multi-headed attention, and discrete, sequential input, require specific interventions in order to compute model permutations that remain within the same functional equivalence class.
In merging these models with our method, we consistently find lower loss barriers between minima compared to model averaging for several models trained on a masked-language modeling task or fine-tuned on a language understanding benchmark. 
Our results show that the minima of these models are less sharp and isolated than previously understood, and provide a basis for future work on merging separately trained Transformer models.

Published in Transactions on Machine Learning Research (11/2024)
Merging Text Transformer Models from Different Initializations
Neha Verma
nvvera7 @ jhu. edu
Center for Language and Speech Processing, Johns Hopkins University
Maha Elbayad
elbayadm meta.com
Meta
Reviewed on OpenReview:
Abstract
Recent work on permutation-on model merging has shown impressive low- or zero- barrier mode connectivity between models from completely different initializations. However, this line of work has not yet extended the Transformer architecture, despite its dominant popularity in the language domain. Therefore, in this work, we investigate the extent which separate Transformer minima learn similar features, and propose model merging technique to investigate the relationship between these minima in the loss landscape. The specifics of the architecture, like its residual connections, multi-headed attention, and discrete, sequential input, require specific interventions in order to compute model permutations that remain within the same functional equivalence class. merging these models with our method, we consistently find lower loss barriers between minima compared model averaging, across models trained on a masked- lankuugg modeling task or fine-tuned on a language understanding benchmark. Our results show that the minima of these models are less sharp and isolated than previously understood, and provide basis for future work merging separately trained Transformer models.
1 Introduction
The geometry of loss landscapes is the subject of extensive prior work attempting to understand the behavior and properties of different minima (Kawaguchi] [016 Li et al. 2018 Du et al. 2019). Prior work understanding the loss landscapes of deep neural networks has found different types of geometric paths of low loss between converged models, demonstrating a degree of connectedness between these separately trained minima (Freeman & Bruna 2017] Garipov et 2018 Tatro et al. 2020). These findings have restructured our understanding the relationship between different minima loss space by uncovering entire low-loss regions in which several minima may be co-located.
A related line of work finds specifically linear paths between differently initialized models, where the linear inter- polations of these minima have low loss like either endpoints (Entezari et al. 2022] Ainsworth et al. 2023). This connectivity is feasible by transforming the weights of the models, without changing their function, in order to compare them within a more similar loss space. The types of transformations exploit inherent model symmetries that are achievable via permuting model features, where the same underlying function may have numerous valid parameter configurations. So far, this work has mostly covered simple architectures like multi-layer perceptrons or VGGNets (Sioonyan & Zisserman) 2015). Some work has included ResNets well (He et al. 2016), and show that only wider ResNets can be merged with much smaller loss than standard versions (Ainsworth al. 2023 Stoica al. 2024).
While this research has led to important conclusions about loss landscapes between separately trained models, the Transformer architecture has been largely unexplored in terms of understanding its permutation symmetries and loss landscape geometry. Prior work has emphasized the importance of understanding loss landscape geometry, as a better understanding of loss geometry can lead improvements in optimization techniques, ensembling, and model merging techniques (Garipov al. 2018 Ainsworth al. 2023). There has been some prior work in merging Transformer architectures (Jin et al. 2023 Imfeld al. 2023), but these either do not account for models from separate initializations,

---

Published in Transactions on Machine Learning Research (11/2024)
do not create valid permutation mappings that transform model into another member of its symmetric equivalence group (Chen et al. 1993).
Therefore, in this work, we explore the extent to which separately trained Transformer models learn similar representa- tions, and then propose permutation based merging method to align representations from these separate models. With this, we can study the extent of the connectivity between their minima in loss space in order to extend our understanding of loss landscape geometry this important and popular architecture. We specifically investigate their connectivity through the lens of permutation sermutation-invariaat linear mode connectivity (Entezari et al.. 2022). Our permutation based merging method builds upon prior work from Ainsworth et al. (2023), but focuses necessary interventions needed for the Transformer architecture, such as merging Multi- Meati- Attention, and the inputs/outputs of each residual connection.
Our contributions are the following:
1. We introduce a new model merging algorithm based on model permutations that combines Transformers from separate initializations.
2. We demonstrate reduced loss barriers between masked language models trained from completely separate initializations compared to vanilla merging.
3. We extend our approach to fine-tuned models and show consistently smaller loss barriers between models compared to vanilla merging.
In Section 2 we discuss related work to place our contributions in context of the existing literature. In Section 3 we describe our method for computing the permutations needed to align two Transformer minima in order to compare them loss space. We outline the models, data, and evaluation used to test our approach in Section Finally, we discuss our findings and analysis in Section
2 Related Work
Loss Landscape & Mode Connectivity. Deep neural networks are typically trained by optimizing a loss function with an SGD variant. Loss landscapes of these networks have been shown to contain infinitely many global minimizers that equally reachable via SGD (Kawaguchi 2016: Du al. 2019). Overparameterization is of the reasons behind the abundance of minima leading different functions that behave similarly the training data (Nguyen al. 2019 Simsek et al. 2021 Liu et al., 2022). Permutation and scaling invariances also lead to functionally identical minima that differ in the weight space (Ttro et al. 2020: Entezari et al. 2022).
Prior work has established that the optima of loss functions are in fact connected by simple curves over which training and test accuracy are nearly constant (no loss barrier) (Freeman & Bruna 2017: Garipov et al. 2018 Draxler et al., 2018). This phenomenon is referred to mode connectiviry. Entezari al. (2022) conjectured that if the permutation invariances of neural networks are taken into account, these optima are linearly mode connected, i.c. the linear path connecting these two models has no loss barrier. This is linear mode connectivity.
Interpolating Models Empirically, linear interpolation between neural network weights has become an important tool. In the context of fine-tuning the same large trained model, averaging models enabled state-of the art accuracy on ImageNet (Wortsman et al., 2022). Wortsman et al. (2022); Rame et al. (2022) established that if fine-tuned models lie in a single low error basin, then weight averaging performs similarly to ensembling. It is however not guaranteed that finetuned models (starting from the same initialization) will reside in the same loss basin.
Prior work linear interpolation-basel model merging has focused on improving the algorithms used to bring the hidden units of two networks into alignment, in order to reduce the barrier to interpolation between them. Sinnhh & Jaggi (2020) develop a strong optimal transport-based method which allows linear interpolation between pair of ResNet networks (He et al. 2016). Entezari et al. (2022) use an approach based on simulated annealing (Zhan et al., 2016) in order to find permutations such that wide ML.ss trained on MNIST can be linearly interpolated with a barrier of nearly zero. Ainsworth et al. (2023) develop several permutation-basee algorithms for MLPs and ResNets, and demonstrate
We release our code h httpp:://////. //gi thub. cow/nvernal /ewer/eigggggg reneggnng--e text- transformers
2

---

Published in Transactions on Machine Learning Research (11/2024)
PT
WiQ
PARA
PT
Wx
PAIR
Wo
LN
W
W2
a
LN
P
Pies
P
Pro
PFF
Pas
pT
Wv
MHA
Figure 1: Example of a Transformer layer with its parameters outlined in blue boxes, specific hidden states as circles, and the flow of operations indicated with arrows. ① indicates the dot- dot-product operation, and Q indicates addition. LN refers to LayerNorm modules. We include permutation and inverse permutation matrices at each weight matrix to indicate the proposed operations from our method. Pres refers to the residual permutation, РMH refers to the attention (MHA) permutation, and Ppp aligns the cor-ffrward layers.
zero-barrier linear mode connectivity on widened ResNets. Stoica et al. (2024) extend this work and allow for feature merges to happen within each model as well, removing internal feature redundancies and improving model merging outcomes.
3 Proposed Transformer Merging Method
In this section, we deseribe the components of our method that address how to permute specific portions a Transformer model 0g in order bring them into alignment with separately trained model, A Extending permutation based model merging Transformers is non-trivial they have more complicated connections than simpler MLP-like architectures. We discuss how find permutation matrices computed from feature correlations, and then we specifically address the parameters involved in multi- headi- attention, residual connections, and pre- and post-net parameters. We diagram a Transformer layer in Figure and the proposed permutation operations that we describe in detail throughout this section.
3.1 Computing Correlation and Permutation Matrices
Given two models trained on the same data but from from separate initializations, namely 0 and Qg, we compute post-activation features for each layer or sublayer parameter W. C in order to determine which features might correspond between models (Ainsworth et al. 20233] Stoica et al. 2024). our setting, features are computed at the token level, and all special non-vocabulary and non-padding non-padding tokens always included (such as [SEP], [CLS]). At given layer sublayer, compute d-dimensional activations across tokens from both models X XB E Raxd and then determine feature relatedness via cross-correlation computed as the following:
E ({(XA (4,,) 1T,,)T(XX T MXx
C corr(X Xn)
xx, xx
where are feature standard deviations, and μ feature means. We choose to standardize and mean-center the features before comparing them because the magnitude of feature values can vary greatly in some pre-trained text Transformers (Puccetti et al. (022)). Then, given feature cross-correlations C E ladxd we compute the optimal permutation as the following:
n² arg C2,(()
where म (1,2, d} + (1, d} a permutation mapping. This optimization problem finds the feature correspondences between the two models that lead the highest total correlation captured. This problem an instance of the assignment problem and can be solved using the Jonker- Volgenant algorithm (Crouse 2016 Tatro et al. 2020 Ainsworth et al. 2023
3

---

Published in Transactions on Machine Learning Research (11/2024)
After converting the map nT to its corresponding permutation matrix P. can apply P to the original weight matrix WBB C BB so that the order of the layer's features most closely resembles that of W. C a
W < PW
(3)
We then apply PT P-1 the next layer in order to unpermute the new ordering in model pp:
W B' 4+1
WB PT.
(4)
After applying all computed permutations to 0g to get 0r. the final merged model is computed as A0 + (1 A)0B for some À E [0, 1].
3.2 Multi-Headed Attention
In finding corresponding neurons in the Multi-Headed Attention (MHA) parameters, namely the key (Wx), query (WQ), value (Wv). and linear layer (Wo) weights, we propose several methods to compute potential permutation matrices.
For each of the key, query, and value weights, the full parameter W Rdantaxdactt is logically partitioned into h attention heads each of output dimension d2 dmoda/ h (Vaswani et al. 2017). In order to apply a permutation to these full weight matrices and maintain the functional equivalence of the overall model, permutations must operate on each attention head separately, and not permute features between attention heads. This is because the final hidden vector from MHA reflects a concatenation of the result from each head, which are computed separately with weights K. Wo Wv. for head
P
P1
P2
P2
P3
P3
Pall
P1
P1
Ps
Monotonic
Heads Permutation
Ignoring
Head Aligmment
Heads
Figure 2: Example permutation matrices resulting from different strategies for attention head alignment. Each P. reflects permutations for features within attention heads.
Since our models are trained from distinct initializations, the correspondence of their attention heads may differ in addition to the correspondence of features within each head. This distinction diagrammed in the first permutations of Figure 2 We collect features from just after the attention computation and before the linear layer following attention. These features are the aforementioned concatenations. We first compute C = corr X A. X 12) and then partition this correlation matrix by heads into d₃ x dy correlation matrices, for each potential attention head pair. Let C be the block of the correlation matrix corresponding to head pair (j, k). We then compute the optimal permutation for each unique head pair, and store its head-internal permutation and cost from the following:
Cjkk
cost(j, k) max
(,)()))
(5)
We then compute the outer head correspondence permutation with assignment problem:
Touter arg max cost(j,
(6)
The outer permutation dictates the subset of previously computed inner permutations used in the final permutation, and the order in which concatenate them, resulting in our 2-staged MHA permutation. We show this approach
4

---

Published in Transactions on Machine Learning Research (11/2024)
Algorithm Multi-Headed Attention Permutation
Input: Correlation Matrix C, number of heads, h
for = to h do
for j j= i to h do
Tij. costso, j) LineaSSumAssignmennn (C*)
end for end for
Totter
LineaSum/Asigaameen(nn
Tfinal
= concat( (T somm
Output: Thoal
in Algorithm method.
1
We note that LineuSumAAsignneent refers to the specific assignment problem we encounter our
The resulting permutation matrix PMTA applies following: MHA can apply Wo described previously, but apply РмНA to each of Wv. WK, and WQ- We note that the (Wq æ, WK, a) dot-product attention operation multiplies away the head dimension du, meaning we do not necessarily have to use the same permutation for Wv and (Wq, Wk} However, we still have to use the same outer permutation for all three matrices, as the attention weights resulting from (Wq, æ. WK; æ) are still head-specific. In experimentation, we do not find a notable difference hetween using separate permutation for (WQ, W K and using the same permutation for Wo: WK and Wv. Therefore,
we consider only the latter case for simplicity.
We show an example of output from our proposed algorithm in Figure 2 well some alternative approaches to permuting MHA weights. The first permutation matrix shows the resulting block-diagonal structure of assuming that heads are aligned between different minima. The second matrix shows example from our method, and the third matrix disregards head structure and allows permuting features across heads. We note that ignoring heads will not lead to a valid permutation π where fa; 0) = f(:; क (()), but we still include it for experimental comparisons.
3.3 Residual Connections
Each Transformer layer, assuming no cross-attention, contains two residual connections. This means all layers, from the embedding layer to the output layer, is linked via consecutive residual connections. This diverges even from ResNets, which generally contain skip-connections every 2 or layers (He et al. 2016).
We diagram the residual connections of Transformer layer and their relationships model parameters in Figure The first connection skips past the multi-headed attention sublayer, and the second connection skips past the feed-forward sublayers, as diagrammed. The connections can be formulated the following, where LN refers to LayerNorm:
- IN(WOMHA(z) (z) a:), x' LN(Ws + xx))
As seen in the equations, the input and output of both sublayers added to create output, which implies that if a permutation operation is applied to the output state, the permutation needs to be the same for both addends.
We note that the addends are normalized via the LayerNorm module, and any permutation to the output would need to permute the features of the LayerNorm module as well. We apply the permutation to the weights of LN. Because LN is not full weight matrix, and maintains the same feature ordering as its input, we must apply the permutation to the addends of the residual connections as well (Stoica al. 2024).
This permutation would be computed from features War and W
5

---

Published in Transactions on Machine Learning Research (11/2024)
Now, ignoring the parameters from the LayerNorm module for a moment, that applying the permutation to the output of the second residual connection leads to the following:
Pæjj = P (W,ReLU( ((aa")) + a²,)
PW, + Par
= PWyRRLU ( + P(WoMHA(Z)) + +
(8)
We see that due to the residual structure, any permutation applied to second feed-forward weight parameter, W2, must also be applied to MHA, or more specifically the Wo matrix. To unpermute these features, we apply pT where the permuted x and arj states become inputs, which are Wi and (We, Wv, WK), respectively.
Because the input to each layer must be permuted Px, and the output of each layer is also permuted Pæy, we can see that the entire Transformer architecture uses the same {P, PT) matrices for all the weights involved in residual connections. This is unlike our other proposed permutations for multi-headed attention which are specific to each Transformer layer. Requiring the same permutation throughout the model reduces the degrees of freedom available in finding an optimal permutation, which is an important result demonstrating potential difficulty of aligning Transformers.
At the ends the models, namely the embedding layer(s) and output layer(s), we also apply these transformations, as the input to the first Transformer block and the output of the last Transformer block are permuted. We apply this to the embedding weights, including positional and any special token embeddings, and the final layer, we apply pT to the weight matrix immediately following the last Transformer block LayerNorm. This is usually pooling dense layer, depending the model task.
Because of the multiple potential features that could contribute to the computation of the residual permutations, namely both x and æ; across all layers, we consider several strategies for learning these mappings. We test 4 approaches: First refers to only obtaining features from the features immediately following the embedding layer(s). Last refers to only obtaining features from just after the final Transformer layer's LayerNorm. All refers to concatenating all features syy and xy from all Transformer layers, and separate refers to computing {P. PP' pairs per Transformer layer, one for wyj. and the other for a. The separate approach also does not lead valid permutation like Ignore Heads in Section 3.2 but also include it for experimental comparisons.
3.4 Feed-Forward and Output Layers
Unlike the residual stream or Multi-Headed attention, Feed- Feed-Forward sub-layers require no special attention in order to permute them into new space. We simply compute correlations from the features after the computation of the first Foed-Forward layer (W1), and compute P, PT separately for each Transformer layer. We apply these permutations described in Section..1
W + PW
W < W.PT
As many models have task-specific output layers like classification heads or masked- language modeling heads, there is also another permutation able to be computed between pooling/dense layer and the actual model head, which tends to also be linear layer. This permutation mapping would proceed as normal, like the feed-forward example, but we do not include it in our experimentation as we see no notable differences in its inclusion.
4 Experimental Settings
4.1 Models
We investigate Transformer encoder- based masked language models in this work. Specifically, we consider different BERT models, seeds 1 through 5, from the MultiBERT's reproductions (Devlin et al.) 20099] /Sellam al. [202T). Each of these models a batch ordering. These properties are the type of SGD-related variation we seek study between different minima. All
checkpoint, but trained with different random initialization and random

---

Published in Transactions on Machine Learning Research (11/2024)
models the same original BERT vocabulary and tokenizer. All of our reported experiments include mean and standard error across the 10 unique pairings resulting from our 5 different models.
For classification tasks, we fine-tune each of the MultBERTSs models with a randomly initialized classification head, including pooling layer and classification layer weights. We keep the head initializations the same across models.
We report vanilla averaging as our main baseline for comparison, computed as Oavg (0, + 0B).
4.2 Tasks and Datasets
We focus on two different tasks to test our method. We use the masked language modeling task to test our base method, as this is the main task that BERT and many other pretrained Transformer encoded models are trained with. We also consider fine-tuning these models for classification tasks, and then comparing them in their fine-tuned state. We use the General Language Understaning Evaluation (GLUE) benchmark (Wang al. 2018 for our classification tasks, and exclude WNLI as in Devlin al. (2019). GLUE is set of 8 diverse natural language understanding classification tasks. We report scores on GLUE test sets from our reproductions in Table 4 in Appendix A
For our experiments on masked language modeling, we use the validation set of the Wikitext-109 103 benchmark as our evaluation data Merity et al. 6 31 For computing model activations, we extract a random sample of just over million sentences of the Books corpus Zhu et al. (2015). For majority of our experiments, we sample 100k sentences and use this subset to compute features, unless stated otherwise. We take diverse sample across different genres among the books available. We choose the Books corpus it is part of the original pre-training data from BERT (Devlin al. 2019).
For GLUE experiments, we use the full training data for each of the tasks to compute features, and the full validation sets to compute losses. The amount of data available for each task varies, and statistics are also reported in Table in Appendix A
4.3 Evaluation
To compute loss barriers, we compute several interpolations of A and p,. as A0 + - - Specifically, we use 21 samples evenly spaced between \ = O and À = 1, inclusive. We use the definition of loss-barrier as Frankle & Carbin () defined as the maximum difference between the loss of an interpolation and the average loss of the base models:
max C(A0A + (1- x)02) ] (C(AA) + L(OB)).
(10)
To compute MLL haty/seeud-ppeplexiii we use masking probability of 0.15 across block sizes of 128 tokens.
For N masked samples in text W, we compute pseudo-perplexity as:
Pseudo-PPL (W; 0) = 2 loga po Po(w; W-
(1))
We report loss on GLUE tasks as normal, defined by each task, across the entire validation set.
5 Results and Analysis
5.1 By component
We report results on the 10 MultiBERT's merges after merging different sets of Transformer components described in Section 3. We show averaging with no permutations, merging all feed-forward sublayers. merging all multi-headed attention sublayers, and merging all feed-forward sublayers and all multi-headee attention sublayers. Aligning and merging either the feed- feed-forward sublayers or the attention sublayers clearly leads to perplexity reduction over the baseline, and their combination leads stark reduction, of almost 7x the original perplexity À 0.5. We do not include permuting
results across the range of interpolations, displayed Figure 3 We report vanilla
"e obtain the wikitext-183-rawwv1 version. available from http:///wws/..ggeeeaeeeeee/tssts
4 Referred as linear interpolation instability this work.

---

Published in Transactions on Machine Learning Research (11/2024)
No permutationns
MHA permutations only
10000
permutations only
FF-MHHA permutation
8000
Pseudo-PPLJ 6000
4000
2000
Model
Model B
Interpalation weight
Figure Pseudo-perpee perepeeeeryy scores of BERTs, trained on the masked language modeling task, combined using our method. Curves differ by which components they merge. Results across 10 merges shown with standard error regions shaded around each curve. Each additional merged component leads to further barrier reduction.
parameters involved in residual connections (Section 3.3) and the output projection (Section 3.4) here as they do not outperform merging only feed-forward and attention sublayers. We discuss the residual connections further Section
5.3
The consistently reduced barrier between minima indicates that these different models connected with lower loss path than seen without considering these models within more similar loss space. We note that we do not observe linear or convex loss path between these models, as sometimes observed in previous work on MLPs, ResNets, and VGGs (Tatro et al.) 2020; Entezari et al., 2022 Ainsworth et al., 2023). In this line of prior work, the same data is generally used to train models, compute activations for alignment and merging, and compute loss barriers. Due to the extensive pretraining data of these masked language models, and the limited alignment and evaluation data we use, we do not test for linear mode connectivity in the same manner. Instead, the stark loss reduction seems indicate that minima connected with barrier least as high as what we report.
Forward Merged
Attention, Merged
Average Feature Correlation
0.25
Feed-forwerd, Pre-merge
Attention, Pre-merge
0.20
0.15
0.10
0.05
0.00
Layer
Figure 4: Average feature correlations between layers from different MultiBERTs, We report correlations for both Feed-Forward and MHA features. Both components see much higher average correlation after applying their respective component permutations. Values are averaged over 10 merges, with standard error regions shaded.
In understanding the extent to which these different Transformers learn similar representations throughout the model, compute the average feature correlations between all 10 masked language model pairs. We report correlations for attention and feed-forward layer features before and after applying our method. Individual correlations computed
8

---

Published in Transactions on Machine Learning Research (11/2024)
between features of the same index. We show these values in Figure We find that the average feature correlations of aligned models are significantly higher than those of the original models, which demonstrates some success of our alignment procedure, but still no higher than 0.3. Previous work has reported higher average correlations on different architectures like ResNets (Tatro et al. 2020), but in the image domain. Some pre-trained transformers are also known to sparsely activated and able to pruned heavily (Li al. 2023 Dalvi et 2020), which may also lead to lower average feature correlations.
5.2 Multi-headed attention
Table Loss Barriers merged MultiBERT's with feed-forward and attention components merged. Methods describe which MHA algorithm was applied. Loss barriers are the largest difference between an interpolation and the average of the individual model losses. Maintaining head structure in the permutation while allowing different head correspondences between models is the most optimal permutation.
Method
Loss Barrier
Std. Err.
Vunilla Attention Avg.
4.31
0.21
Monotonic Head Alignment
4.13
0.20
Ignore-Heads
3.97
0.25
Head-Perm
3.71
0.23
Before Permutation
After Permutation
0.6
0.5
Absolute Correlation
Model B Index
0.4
400
0.2
600
0.1
200 400 600
200 400
600
Model Index
Model Index
Figure 5: Visualization of correlation matrices between features before and after permuting. These features are from the seventh multi-headed attention layer from 2 different MultiBERTs models. On the left, 12 attention head boundaries are elearly visible, and highly correlated regions do not necessarily correspond to the same attention head indices, supporting two-stage permutation method. On the right, the two-stage permutation method outcome can be via the dark diagonal line, and its surrounding block diagonal pattern.
We report loss barriers for our Head-Pernin PeaddPeermttaionn multi-headed attention approach as compared some alternatives also described in Section 3.2 in Table I These results reflect permuting both attention parameters as well as beed-forward parameters. We see that our proposed Peradd-ertttation approach for the attention sub-layer outperforms simple attention averaging, as well as approaches ignoring the multi-headed structure of the weight parameters (Ignore-Heads), and not allowing for different head correspondences across different models (Monotonic). We also show an example correlation matrix between the first heade- attention layer from 2 different MultiBETts models Figure 5 The correlation matrix shows clear attention head boundaries, as well as scattered pattern that supports our proposed technique that does assume any monotonically ordered head correspondence.
5.3 Residual Stream
As described Section 3.3 many of the permutation operations in the Transformer architecture are shared due to its repeated Add&Norm components. This linkage results in huge reduction in the number of permutation symmetries of

---

and attention Published in Transactions on Machine Learning Research (11/2024)
the Transformer architecture, and therefore the number of valid permutations of parameters involved in the residual stream. We report the loss barriers after applying our permutation alignment to only the parameters involved in the residual connections in 2 We find that an identity permutation performs significantly better than all other proposed approaches. Even the separate permutation approach does not outperform using the identity matrix Ia as the residual merge. This approach does not even create a valid Transformer symmetry, but has many more degrees of freedom than the other proposed approaches.
Table 2:
Loss Barriers of merged MultiBERTs with only residual components merged. Methods describe which features were used to compute pernutations, Loss barriers are the largest difference between an interpolation and the average of the individal model losses. As there only one {P, }T} pair for the entire residual stream, identity permutations outperform our learned approaches.
Method
Loss Barrier
+
Std. Err.
Identity
4.95
0.38
First
7.58
0.19
Last
7.41
0.18
7.34
0.22
Separate
9.38
0.49
5.4 Amount of Data
We report loss barriers on the masked language modeling task while varying the amount of sentences used to compute correlations. All previous experiments reported 100k sentences. We combine layers in this setting. Results are shown in Figure 6 Values are fairly similar across data amounts, suggesting that there is no strong relationship between the amount of data used and the overall loss barrier. We do see, however, small variations between different data amounts, which are likely due to the content of the different sets of sentences used to create the correlation matrices. We believe that more than the quantity of tokens, the type and quality of text used for computing correlations likely matters more. We leave further investigation into specific data sources for aligning text-based Transformer models for future work.
4.75
Loss barter
4.25
3.75
3.50
3.00
50k of Sentences
Figure 6: Loss barriers on the masked language modeling task with different data amounts used for computing features. Standard error regions are shaded. The amount of data does not have strong directional relationship with the size of the loss barrier in this setting.
5.5 GLUE Results
We investigate loss barriers between fine-tuned BERT models across different GLUE tasks. We compare loss barriers from vanilla averaging those obtained after applying our method combine the models. Different from our result on masked language modeling, include residual permutations find it has slight decrease in loss barriers over
10

---

Published in Transactions on Machine Learning Research (11/2024)
just feed-f and attention permutations. We use the First approach to compute residual permutations. We report these results in Table 3 We additionally include loss barrier curves for all 8 tasks in Appendix B in Figure 7
Table 3: Loss barriers for both vanilla averaging of fine-tuned BERT models, and barriers for BERT models merged with our method. In this setting, we merge forward, attention, and residual components. Residual mergers are conducted using the First strategy. 6 of 8 tasks see loss barrier reduction in using our method.
Vanilla averaging
Ours
Barrier
Error
Barrier +
Error
MNLI-mm
0.61
0.03
0.72
0.08
QQP
1.37
0.09
1.20
0.11
QNLI
0.64
0.04
0.77
0.06
SST-2
0.42
0.04
0.36
0.07
CoLA
1.31
0.14
1.11
0.13
STS-B
5.15
0.44
4.24
0.35
MRPC
2.74
0.08
1.93
0.11
RTE
0.53
0.04
0.41
0.05
While we are able to observe lower loss barriers between minima, the trends of loss reduction across interpolations is inconsistent, especially compared to the masked language modeling setting. For example, while the maximum loss of vanilla averaging higher than the maximum loss of our approach, there are still other interpolation weights where the loss of our approach instead higher than vanilla averaging. We also note an interesting pattern of lower loss than either parent model for several tasks around 0.15 and again some vanilla merges between these fine-tuned models have different behavior than their pretrained alternatives, seen in Figure 7 with the presence of "M" like loss curve shapes between minima. While our method can extend to this setting to find a lower loss path between fine-tuned minima, it is unclear which kind of data is necessary to observe the lowest loss path. We leave further investigation into connecting fine-tuned models, and further understanding of their pre-merging connectivity in loss space, for future work.
0.85 Additionally, the loss barrier curves of
6 Discussion and Conclusion
By considering the set of functionally equivalent Transformers reachable using permutation mappings, we can con- sistently find linear paths between models with lower loss than the path obtained via vanilla interpolation. This conclusion about the connectedness between these models has implications of Transformer loss space and the sharpness of their minima; in comparing minima using our method, they are far less isolated than previously understood. This understanding of the geometric properties of minima can have implications in how we design optimization methods, ensembles of models, and additional merging techniques. For example, it is widely contested whether sharp minima can generalize as well as flat minima across many deep learning models Keskar al, (006)): Dinh al, (0017). In our work, we show that it is necessary to consider permutation invariances of Transformer models when characterizing the geometric properties of their minima.
understanding of the "smoothness"
As we take only first attempt at connecting separately trained Transformers along lower loss path, there is much room for future work in understanding Transformer loss landscapes. A deeper understanding of the relationships between fine-tuned models, Transformer width and loss barriers, and the data needed to compute more informative correlations is needed in order to further characterize the relationship between Transformer minima.
Broader Impact Statement
While is already difficult to meaningfully characterize the capabilities of a trained model, is even more difficult to know what an interpolated model's capabilities are, with respect to their parent models. Our work focuses on the losses of these interpolated models rather of their performance in other aspects. Outside of loss, it unclear how the performance of these models changes when combined. Thorough testing of interpolated models should be investigated before actual use.
11

---

Published in Transactions on Machine Learning Research (11/2024)
References
Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation
symmetries. In The Eleventh International Conference on Learning Representations, 2023. URL https://
An Mei Chen, Haw-minn Lu, and Robert Hecht-Nielsen. On the geometry of feedforward neural network error surfaces. Neural Compalation, 5(6):910-927, 1993. doi:
1993.5.6.910
David F. Crouse. On implementing 2d rectangular assignment algorithms. IEEE Transactions Aerospace and doi: 10.1109/TAES. 201.1...5.....1.........
Electronic Systems, 52(4):1679-1696 2016.
Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. Analyzing redundancy in pretrained transformer models. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 4908-4922 Online, November 2020. Association for Computational Linguistics. doi: 10. 11111833111220200 12..66651122200..m/mmmmaa......
URL
398
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre- trtinrnning of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.). Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pupers). pp. 4171-4186 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/NI 1423. URL https::/accanntoology org/
N19- 1423
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In International Conference on Machine Learning, pp. 1019-1028. PMLR, 2017.
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially barriers in neural network energy landscape. In International conference on machine learning, pp. 1309-1311 PMLR, 2018.
Simon S. Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. Gradient descent provably optimizes paaremaaaetteeeeiize neural networks. In International Conference on Learning Representations, 2019. URL htttps:://peeeeeieww.onr net/
Forum?id=S1eK3ii099
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation invariance in linear
mode connectivity of neural networks. In International Conference on Learning Representations, 2022. URL
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. International Conference on Learning Representations, 2018.
C Daniel Freeman and Joan Bruna. Topology and geometry of half-rectified network optimization. In 5th International Conference on Learning Representations, ICLR 2017, 2017.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew Gordon Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in Neural Information Processing Systems, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Moritz Imfeld, Jacopo Graldi, Marco Giordano, Thomas Hofmann, Sotiris Anagnostidis, and Sidak Pal Singh. Trans- former fusion with optimal transport, 2023.
Xisen Jin, Xiang Ren, Daniel Preotiuc-Piee Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. In The Eleventh International Conference on Learning Representations, 2023. URL hhtps: //openreview. mett/forrumm.c-FCCnhuuRa..
12

---

Published in Transactions on Machine Learning Research (11/2024)
Kenji Kawaguchi. Deep learning without poor local minima. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Cur- ran Associates, Inc., 2016. URL https://proceedinng, ienceeiinggsneuris..... wwtppppper..ilessppprrr/////e/ea 72fc399265c712244655113 1a33aa110022233ppppppaeo pdff
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference Learning Representations, 2016.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 31, 2018.
Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, et al. The lazy neuron phenomenon: On emergence of activation sparsity in transformers. In Conference on Parsimony and Learning (Recent Spotlight Track), 2023.
Chaoyue Liu, Libin Zhu, and Mikhail Belkin, Loss landscapes and optimization in over-parameterized non non-linear systems and neural networks. Applied and Computational Harmonic Analysis, 59:85-116, 2022.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2016.
Quynh Nguyen, Mahesh Chandra Mukkamala, and Matthias Hein. On the loss landscape of a class deep neural networks with no bad local valleys. In International Conference on Learning Representations, 2019. URL
Giovanni Puccetti, Anna Rogers, Aleksandr Drozd, and Felice Orlettr Outlier dimensions that disrupt transformers are driven by frequency. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 1286-1304, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/ wwwpnnhhooogy.org/22222 Findings-emnlp....
URL https://
Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, patrick gallinari, and Matthieu generalization. In Alice H. Oh, Alekh Agarwal, Danielle
Cord. Diverse weight averaging for Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. https
Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf, and Ryan Cotterell, All roads lead rome? exploring the invariance of transformers' representations, arXiv preprint 1:Xi::2300514555, 2023.
Thibault Sellam, Steve Yadlowsky, Ian Tenney, Jason Wei, Naomi Saphra, Alexander D' Amour, Tal Linzen, Jasmijn Bastings, Iulia Raluca Ture, Jacob Eisenstein, et al. The multiberts: Bert reproductions for robustness analysis. International Conference on Learning Representattinss 2021.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
Berfin Simsek, François Ged, Arthur Jacot, Francesco Spadaro, Clement Hongler, Wulfram Gerstner, and Johanni Brea. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 9722-9732. PMLR, 18-24 Jul 2021. URL https: //proceedings. mlr http:/inns.....pprs//eeeee 39/simsek21a. l
Sidak Pal Singh and Martin Jaggi. Model fusion via optimal transport. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 22045 22055, Curran Associates, Inc., 2020. URL http:::/pproeeeiing.nurs......pp
fi Filpr//aane0555544444ceeeeeeee
pdf
13

---

Published in Transactions on Machine Learning Research (11/2024)
George Stoica, Daniel Bolya, Jakob Bjomer, Taylor Hearn, and Judy Hoffman. Zipit! merging models from different tasks without training. In The Twelfth International Conference Learning Representations, 2024. URI. https
//operreviee.ne///orrmmm
Norman Tatro, Pin- Yu Chen, Payel Das, Igor Melnyk, Prasanna Sattigeri, and Rongjie Lai. Optimizing mode connectivity via neuron alignment. Advances in Neural Information Processing Systems, 33, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Tal Linzen, Grzegorz Chrupata, and Afra Alishahi (eds.). Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355, Brussels, Belgium, November 2018. Association for Computational Linguistics. 8-44
doi: 10. N0.18853/v1// 1/W13 18-5446. URL URL httpps://acaanntoooogy.oooo
Mitchell Wortsman, Gabriel Ilharco. Samir Ya Gadre, Rebecca Roelofs, Raphael GontijoLLppes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korblith, et al. Model soups: averaging weights multiple fine-tuned models improves accuracy without increasing inference time. International Conference on Machine Learning, pp. 23965-23998, PMLR, 2022.
Shi-hua Zhan, Juan Lin, Ze-jun Zhang, Yi-wen Zhong, et al. List-based simulated annealing algorithm for traveling salesman problem. Computational intelligence and neuroscience, 2016, 2016.
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer vision, pp. 19-27, 2015.
A GLUE Reproduction
We report the average values of our GLUE reproductions across the MultiBERT's models here. We train MNLI- mismatched, QQP, QNLI, SST-2, CoLA, STS-B, and RTE tasks for 3 epochs, and we train MRPC for 5 epochs. We follow all other hyperparameters of the reproduction implemented in Ren et al. (023).
Table 4: Results on GLUE for both the original BERT model Devlin et al. (201)), and our reproduction across MultBERTTss models 1-5. We report the average and the standard deviation across 5 models. Accuracy is reported for MNLI-mm, QNLI, SST-2, and RTE. Fl scores are reported for QQP and MNLI. Matthews correleation is reported for CoLA, and Spearman-r correlation reported for STS-B.
Task
MNLI-mm
QQP
QNLI
SST-2
CoLA
STS-B
MRPC
RTE
Training instances
392k
363k
108k
67k
8.5k
5.7k
3.7k
2.Sk
Validation instances
9.8k
40.4k
5.5k
0.9k
1k
1.5k
0.4k
0.3k
BERTBAST
$3.4
71.2
90.5
93.5
52.1
85.8
88.9
66.4
Our Reproduction
84.320.002
87.3
91.040.003
91.72.0002
58.340.011
89.240..03
90.7.0.010
66.2+9.063
B GLUE Loss Plots
We plot loss barriers for vanilla merging and our method across 8 GLUE tasks. For a majority of the tasks, the loss barrier of interpolations is smaller than that of vanilla merging.
14

---

Published in Transactions on Machine Learning Research (11/2024)
mnii
co
vanila
2.0
varilaa
1.8
1.4
LOSS
1.0
1.0
0.6
Interpolation weight A
Interpolation weight À
qnli
rte
1.2
vanile
vanila
1.1
1.1
Loss
0.9
0.6
0.8
0.5
Interpolation weight À
Interpolation weight À
sst2
stsb
vanile
Loss
3
2
Interpolation weight A
Interpolation weight À
aqp
mrpc
vanils
3.0
M
1.4
2.5
Loss
2.0
0.8
1.5
Interpolation weight A
Interpolation weight À
Figure 7: Loss barriers on 8 GLUE tasks for both the vanilla interpolation and merging strategy.
15


Reviews:
[{"review_id": "y8oQSycr2Z", "reviewer": "Reviewer_MkbX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- Novel contribution in extending permutation-based model merging techniques to Transformer, addressing a 2-staged approach for multi-headed attention.\n- Comprehensive empirical evaluation across multiple settings, including masked language modeling and fine-tuned models on GLUE tasks.\n- Provides insights into the geometry of Transformer loss landscapes, suggesting implications for optimization methods and ensemble techniques.\n- Clear methodology and detailed ablation studies examine different components of the proposed approach.\n\n**Weaknesses:**\n\n- Limited theoretical analysis or guarantees for the proposed method, relying primarily on empirical results – why post-activation features are used?\n- The approach does not consistently outperform vanilla averaging for all GLUE tasks, and the reasons for this variability are not fully explored.\n- The paper does not extensively compare the proposed method to other recent model merging techniques beyond vanilla averaging.\n- While lower loss barriers are demonstrated, the practical implications or benefits of this finding (e.g., for downstream tasks or model compression) are not thoroughly explored.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the loss of landscape connectivity between separately trained (different initializations) Transformer models. The authors propose a model merging technique that aligns representations from separate models using permutation-based methods for different components of the Transformer architecture, including multi-headed attention, residual connections and feed-forward layers. The main merging technique is as follows; compute correlation matrices between features of the two models for different layers and components, and find optimal permutations that align the features using the Jonker-Volgenant algorithm, then apply these permutations to specific components (e.g., multi-head attention) of one model to align it with the other. The authors demonstrate that their method consistently finds lower loss barriers between model minima compared to simple model averaging, both for pre-trained masked language models and models fine-tuned on GLUE tasks. The results suggest that Transformer minima are less isolated in the loss landscape than previously understood when considering permutation symmetries. In their experiments, the authors show that merging both feed-forward and attention sublayers leads to a significant reduction in perplexity, almost 7 times lower than the original. They also find that the amount of data used to compute correlations for alignment does not strongly affect the loss barrier. For fine-tuned models on GLUE tasks, the method reduces loss barriers in 6 out of 8 tasks compared to vanilla averaging, though the trends are less consistent than in the masked language modeling setting.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "N/A", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wkEubmj8Hi", "reviewer": "Reviewer_S6LB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nIt provides a simple and intuitive way to merge transformer weights. The experimental results show that it provides lower loss barrier values compared to simpler merging methods.\n\nIt explains the method in detail. It will help readers to reproduce the results.\n\nWeaknesses:\n\nThere is no comparison with prior studies merging transformers e.g. Imfeld et al. (2023). The conditions may not be exactly the same, but still the method should be still able to be used for comparisons. \n\nIt is not shown that the merged model can be actually useful. I think it is a good empirical result that the loss barrier can be lowered by tailored permutation-based merging methods, but if the merged model is not actually useful, the value of such merging is questionable. Imfeld et al. (2023) shows the merged model can be better after finetuning.\n\nIt is a natural extension of existing permutation-based methods to transformers and the results are not surprising and may be less novel. \n\nIt does not have theoretical discussions. Why are transformers considered as special?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a permutation-based merging method for transformers and give analyses on the loss barrier in the setting of linear mode connectivity. The merging method considers the specific structure of transformers such as multi-head attention and outperforms the vanilla interpolation method and naive interpolation methods that do not consider the structure. The experimental results provide valuable insights on the landscape of the loss values of transformers e.g. the minima may be far less isolated than previously understood.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would love to see comparisons with existing methods, at least ones mentioned in the paper if appropriate.\n\nIt would be good to see finetuning or other experiments that show better performance for the merged model.\n\nIt would be good to have discussions on why model merging has been challenging for transformers. The model is of course more complicated than MLPs and CNNs, but the components are basically matrix multiplications and it does not seem that it is \"theoretically\" more challenging.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MF7GNtgJIS", "reviewer": "Reviewer_3kxD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\nThe paper is overall well written and easy to follow. The methods part is described precisely enough to understand the merging procedure properly.\n\nThe proposed method is simple and effective. The results (at least partially) underline that.\n\n**Weaknesses**\n\nTo me it appears unclear why in two out of eight cases (Table 3) the presented permutation-based merging performs worse than vanilla averaging. I think that this should be addressed in more detail. What happens in these cases? How do the correlations look like? Is it a failure of the permutation search?\n\nOne debatable point of criticism of the paper is its incremental nature. The fundamental methodological framework is based on Ainsworth et al. (2023) (cf. also Yamada 2023). In the present work, however, this method has been adapted to work with transformers in a somewhat straightforward manner. Although it is very reasonable how this has been accomplished, it should be discussed whether this addition brings enough innovative novelty for publication in TMLR.\n\nMasanori Yamada, Tomoya Yamashita, Shin'ya Yamaguchi, Daiki Chijiwa. Revisiting Permutation Symmetry for Merging Models between Different Datasets. 2023. https://arxiv.org/pdf/2306.05641", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the merging of separately trained transformer models to understand their loss landscape connectivity. It introduces a permutation-based model merging algorithm tailored for transformers, addressing specific architectural challenges like multi-head attention and residual connections. The results show that this method consistently reduces loss barriers between models, indicating that transformer minima are less isolated than previously thought, and suggesting that similar representations are learned even from different initializations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors should add more details on the permutation operation. It would make the paper more accessible to know how the \"optimal\" permutation is found. \n\nFigure 3 should be more self-explaining.", "broader_impact_concerns": "Model merging is an important aspect when looking at effective distributed learning - especially for LLMs (which are currently mostly transformer-based). From that perspective, the contribution might be very useful in practice. However, it should be clarified better, why it fails in some cases and whether these cases could be identified on-the-fly without checking plain averaging for comparison.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores the merging of separately trained transformer models to analyze their loss landscape connectivity. It introduces a permutation-based merging method that accounts for transformer-specific structures like multi-head attention and residual connections. The method outperforms standard interpolation techniques by reducing the loss barriers between models, providing evidence that transformer minima are less isolated than previously believed. The results suggest that despite different initializations, transformers learn similar representations, offering new insights into the structure and connectivity of their loss landscapes.

All reviewers vote for acceptance, though they raised some concerns. However, the concerns are primarily about the novelty of the method. As per TMLR's policy, novelty alone cannot be grounds for rejection, and the method is well evaluated. Additionally, the authors responded to the reviewers' comments effectively. Therefore, the paper can be accepted by TMLR.
Decision: Accept (poster)