Linear Connectivity Reveals Generalization Strategies
Authors: Jeevesh Juneja
Rachit Bansal
Kyunghyun Cho
João Sedoc
Naomi Saphra
Venue: ICLR 2023 poster (2023)
Keywords: loss landscapes
OOD generalization
NLI
text classification
loss surfaces
transfer learning
challenge sets
NLP

Abstract: In the mode connectivity literature, it is widely accepted that there are common circumstances in which two neural networks, trained similarly on the same data, will maintain loss when interpolated in the weight space. In particular, transfer learning is presumed to ensure the necessary conditions for linear mode connectivity across training runs. In contrast to existing results from image classification, we find that among text classifiers (trained on MNLI, QQP, and CoLA), some pairs of finetuned models have large barriers of increasing loss on the linear paths between them. On each task, we find distinct clusters of models which are linearly connected on the test loss surface, but are disconnected from models outside the cluster---models that occupy separate basins on the surface. By measuring performance on specially-crafted diagnostic datasets, we find that these clusters correspond to different generalization strategies. For example, on MNLI, one cluster behaves like a bag of words model under domain shift, while another cluster uses syntactic heuristics. Our work demonstrates how the geometry of the loss surface can guide models towards different heuristic functions in standard finetuning settings.

Published as a conference paper at ICLR 2023
LINEAR CONNECTIVITY REVEALS GENERALIZATION STRATEGIES
Jeevesh Juneja', Rachit Bansal', Kyunghyun Cho², João Sedoc", Naomi Saphra² Delhi Technological University. "New York University jeeveshjune ja jm@ com, nsaphraenyu.edu
ABSTRACT
In the mode connectivity literature, it is widely accepted that there are common circumstances in which two neural networks, trained similarly on the same data, will maintain loss when interpolated in the weight space. In particular, transfer learning is presumed to ensure necessary conditions for linear mode connectivity across training runs. In contrast to existing results from image classification, we find that among text classifiers (trained on MNLI, QQP, and CoLA), some pairs finetuned models have large barriers of increasing loss the linear paths between them. On each task, we find distinct clusters of models which are linearly connected the test loss surface, but are disconnected from models outside the cluster- models that occupy separate basins on the surface. By measuring performance on specially-crafted diagnostic datasets, find that these clusters correspond different generalization strategies. For example, on MNLI, one cluster behaves like bag of words model under domain shift, while another cluster uses syntactic heuristics. Our work demonstrates how the geometry of the loss surface can guide models towards different heuristic functions in standard finetuning settings.
INTRODUCTION
Recent work on the geometry of loss landscapes has repeatedly demonstrated tendency for fully trained models fall into single linearly-connesee
basin of the loss surface across different
training runs (Entezari et al., 2021 Frankle et al.) 2020 Neyshabur et al. [002)). This observation has been presented as a fundamental inductive bias of SGD (Ainsworth et al.) 2022). and linear mode connectivity (LMC) has been linked to in in-domain generalization behavior (Frankle et al. [2000] Neyshabur et al. 2020). However, these results have relied exclusively on a single task: image classification. In fact, methods relying assumptions of LMC can fail when applied outside of image classification tasks (Wortsman et al.] 2022), but other settings such as NLP nonetheless remain neglected in the mode connectivity literature.
In this work, we study LMC in several text classification tasks, repeatedly finding counterexamples where multiple basins accessible during training by varying data order and classifier head initialization. Furthermore, we link model's 's basin membership to a real consequence: behavior under distribution shift.
In NLP, generalization behavior is often described by precise rules and heuristics, as when language model observes plural subject noun and thus prefers the following verb be pluralized (the dogs play rather than plays). We can measure model's 's adherence to particular rule through the use of diagnostic or challenge sets. Previous studies of model behavior on distt--iistiiuttion (OOD) linguistic structures show that identically trained finetuned models can exhibit variation in their generalization to diagnostic sets (MeCoy et al., 2020 Zhou et al., 2020). For example, many models perform well in-domain (ID) data, but diagnostic sets reveal that some of them deploy generalization strategies that fail to incorporate position information (McCoy et al. 2019) or are otherwise brittle.
These different generalization behaviors have never been linked to the geometry of the loss surface. In order to explore how barriers in the loss surface expose model's generalization strategy, will consider a variety of text classification tasks. We focus on Natural Language Inference (NLI; Williams et al. 2018 Consortium et al. 1996), as well as paraphrase and grammatical acceptability

---

Published as a conference paper at ICLR 2023
tasks. Using standard finetuning methods, find that in all three tasks, models that perform similarly on the same diagnostic sets are linearly connected without barriers on the ID loss surface, but they tend to be disconnected from models with different generalization behavior.
Our code and models are public D Our main contributions are:
contrast with existing work computer vision (Nyshhaur et al. 2020), we find that transfer learning can lead to different basins over different finetuning runs (Section 3 We develop metric
for model similarity based on LMC, the convexity gap (Section 4 and an accompanying method for clustering models into basins (Section44.1)
We align the basins to specific generalization behaviors (Section 4. In NLI (Section 2.1). they correspond to preference for either syntactic or lexical overlap heuristics. On a paraphrase task (Section 2.2 they split on behavior under word order permutation. On a linguistic acceptability task, they reveal the ability to classify unseen linguistic phenomena (Appendix
We find that basins trap a portion of finetuning runs, which become increasingly disconnected from the other models as they train (Section (.2).. Connections between models in the early stages of training may thus predict final heuristics.
IDENTIFYING GENERALIZATION STRATEGIES
Finetuning standard GLUE WWng et al. 2018 datasets often leads models that perform similarly on in-domain (ID) test sets (Sellam et al. 2021). In this paper, to evaluate the functional differences between these models, we will measure generalization to OOD domains. We therefore study the variation of performance on existing diagnostic datasets. We call models with poor performance on the diagnostic set heuristic models while those with high performance are generalizing models. We study three tasks with diagnostic sets: NLJ, paraphrase, and grammaticality (the latter in Appendix (A]
All models are initialized from trained with Google's s original trainer The only difference between models trained on particular dataset is the random seed which determines both the initialization of the linear classification head and the data order; we do not deliberately introduce preferences for different generalization strategies.
P with linear classification head and
The
2.1 NATURAL LANGUAGE INFERENCE
NLI is a common testbed for NLP models. This binary classification task poses challenge in modeling both syntax and semantics. The input to an NLI model is a pair of sentences such as: (Premise: The dog scared the cat. Hypothesis: The cat scared by the dog.} Here, the label is positive or entailment, because the hypothesis can be inferred from the premise. If the hypothesis were, "The dog was scared by the cat". the label would be negative non-entailment. We the MNLI Williams et al. 2018) corpus, and inspect losses on the ID "matched" validation set.
NLI models often "cheat" by relying on heuristics, such as overlap between individual lexical items or between syntactic constituents shared by the premise and hypothesis. If model relies on lexical overlap, both the entailed and non-entailed examples above might be given positive labels, because all three sentences contain "scared", "dog". and "cat". McCoy et al. (2019) responded to these shortcuts by creating HANS, a diagnostic set of sentence pairs that violate such heuristics:
Lexical overlap (HANS-LO):
Entails any hypothesis containing the same words as the premise. Subsequence: Entails any hypothesis containing contiguous sequences of words from the premise. Constituent: Entails any hypothesis containing syntactic subtrees from the premise.
Unless otherwise specified, heuristics, so higher accuracy on HANS-LO indicates less reliance on lexical overlap heuristics.
the non-entailing HANS subsets for measuring reliance
!Code:
ht http:////ggatuuuaacccccc//altat/
Models:
at ttaps
10 18/ ///////ased...eee/ee/e 1-12
768 108_A-122213
httpp::/ggithe com http:///.w...oooooeeeeeeeeee//// QQP models are trained with Google's recom-
mended default hyperparameters (details in Appendix P). MNLI models are those provided by McCoy et al.
2020
2

---

Published as a conference paper at ICLR 2023
type gen- gengen.-
type gen-to-bouu.
type bcur-to-heur.
0.50
N 20
N 25
N
ITND tait
10sses
0.45
0:40
3.00
20
ONVSNVII mp
losses
2.00
1.00,00
position
position
(a) MNLI (top) and HANS-LO (bottom) loss variation during linear interpolation.
type gen gongem.-gem.
type gen--to-beur:
type
heur-to-heur.
25
N
ITNW upp
0.84
0105
0.82
T OTSSWILL - Dsp
0.40
0055
0.20
(0,1)
position
position
position
(b) MNLI (top) and HANS-LO (bottom) accuracy variation during linear interpolation.
Figure 1: [Loss surface on the line joining pairs of models trained on MNLL.] Performance during linear interpolation between pairs models taken from the best (gen.) and worst (heur.) in HANS-LO accuracy. Heuristic models tend to be poorly connected to generalizing models, although they are well connected to each other. N indicates number of model pairs. Position on the x-axis indicates the value of a during interpolation.
2.2 PARAPIRASE
Quora Question Pairs (QQP: Wang et al., (017)) is a common paraphrase corpus containing 400k pairs of questions, annotated with a binary value to indicate whether they are duplicates. We use the PAWS-QQP (Zhang et al. 2019) diagnostic set to identify different generalization behaviors. PAWS-QQP contains QQP sentences that have been permuted to potentially create different meanings. In other words, they are pairs that may violate a lexical overlap heuristic.
3 LINEAR MODE CONNECTIVITY
Models discovered by SGD are generally connected by paths over which the loss is main- tained (Draxler et al. 2019 Garipov al. 2018), but if we limit such paths to be linear, con-
nectivity is no longer guaranteed. We may still find, however, that two parameter settings A and 8R. which achieve equal loss, can be connected by linear interpolation (Nagarajan and Kolter 2019
Goodfellow et al. 2015) without any increase in loss. In other words, loss L(O.;; Xmmin Virain)
L(OA; main: Vurain) and L(Oa; X Xurai: Yumin) S L(OB; Xumin: Virain) in each parameter setting 0g de-
fined by any scalar 0 <
0. - =0 + (1-a)0n
Frankle al. (2020) characterized LMC between models initialized from single pruned network, finding that the trained models achieved the same performance as the entire un un-pruned network only when they were linearly connected across different SGD seeds. Their results suggest that performance
3

---

Published as a conference paper at ICLR 2023
H2
0,0052
0.5717
00.581
Ho
0,0
0,0
0.
0.5045
-0.5
-0.5
-0.-
0.4710
-1.5
(4) generalized models
(b) heuristic models
(c) generalized and heuristic models
Figure 2: Loss surfaces on the unique plane through 3 models. Colorscale indicates the MNLI matched validation set loss surface, planes passing through various kinds of finetuned MNLI models. The points Go... and Ho... denote the generalizing and heuristic models respectively. The planes are plotted with scale where unit is the same the length of the bottom edge of the triangle formed by the three points defining the plane. For example, the plane containing points Go, G1, G2 plotted with the scale unit len(G)G()). in both and directions.
closely tied to the LMC of the models in question. This implied result is further supported by Entezari et al. (2021). ). which found a larger barrier between models when they exhibited higher test error. Neyshabur
(2020) even suggested that LMC is a crucial component of transfer learning, finding that models initialized from the same pretrained model are linearly connected, whereas models trained from scratch exhibit barriers even when initialized from the same random weights.
Our results complicate the narrative around LMC (Fig. I(a). While we find that models with high performance on OOD data were indeed linearly connected to each other, models with low performance were also linearly connected to each other. It seems that the heuristic and generalizing models occupy two different linear basins, with barriers in the ID loss surface between models in each of these two basins.
HANS performance during interpolation: It is clear from Fig. I(a) that, when interpolating between heuristic models, OOD HANS-LO loss significantly improves further from the end points. This finding implics that the heuristic basin does contain models that have better generalization than those found by SGD. In contrast, the generalizing basin shows only a slight improvement in OOD loss during interpolation, even though the improvement in ID test loss is heuristic basin. However, we can see from ( that low losses on interpolated models do not always translate consistently higher accuracy, although interpolated models that fall barriers of elevated test loss do lead to lower ID accuracy. These results suggest a substantial divergence in model calibration between interpolations of different heuristics.
substantial than in the
Connections over dimensions: To understand the loss topography better, we present planar views of three models using as in [Benton et al. (2021). In the plane covering the heuristic and generalizing models, large central barrier intrudes on the connecting edge between heuristic and generalizing models (Fig. On the other hand, the planes passing through only heuristic or only generalizing models each occupy planes that exhibit a smaller central barrier. Visibly, this barrier smallest for perimeter composed of generalizing models and largest for the mixed perimeter. These topographies motivate the following notion of an e-convex basin.
3.1 CONVEX BASINS
Inspired by Entezari al. 2021). h,, define notion of basin in order to formalize types of behavior during linear interpolation. contrast to their work, however, our goal is not to identify a largely stable region (the bottom of basin). Instead, interested in whether set of models
4

---

Published as a conference paper at ICLR 2023
pearsonr -0,674
pearsonr -0.242
04
0.5
cluster-]
Accuracy HANS-LO
Acc. on HANS 1,0
04
0.4
03-
0.3
0.2
0.1-
-502
||W
-0.51
cill
0.03
||
3.51
C2ll
0.2
2.4
2.5
26
27 E-sharpness
2.5
29
30
35
Figure 3: Using features of the ID loss landscape to predict reliance of MNLI models on lexical overlap heuristics. Least squares fit shown. (a) HANS-LO accuracy difference between distances from the centroids C1, C2 of the CG-based clusters. (b) HANS-LO accuracy vs e-sharpness.
connected to the same low-loss points by linear paths of non- non-increasing loss, thus sharing optima that may be accessible by SGD. This motivation yields the following definition.
For resolution 6 2 0, we define an 6-convex basin as a convex set S, such that, for any set of points W1, .-. W2 E S and any set of coefficients 01 ... On 2 O where E. 0g relaxed form of Jensen's ' inequality holds:
T2
a ag ((wk
kgwkk I c
In particular, we say that set of trained models 0I,... 6n form an c-convex basin if the region inside their convex hull is an e-convex basin. In the case where O, we can equivalently claim that the behavior of L in the region is convex.
This definition satisfies our stated motivation because any two points within an 6-convex basin are connected linearly with monotonically (within c) decaying loss to the same minima within that basin. While such a linear path does not strictly describe a typical SGD optimization trajectory, such linear interpolations are frequently used to analyze optimization behaviors like module criti- cality (Neyshabur et al. 2020; Chatterji et al. 2020). One justification of this practice that much of the oscillation and complexity of the training trajectory is constrained to short directions of the solution manifold (Jastrzebski et al., 2020) Xing et al.) 2018 Ma et al.) 2022). while large scale views of the landscape describe smoother and more linear trajectories (Goodfellow al., 2015 Fort and Jastrzebski] 2019). We use this definition of e-convexity to describe linearly connected points as existing the same basin. However, pairwise -on-increasing linear connections are necessary but not sufficient condition for all convex combinations to be non-increasing, as seen in Fig. 2
THE CONVEXITY GAP
One possibility we argue against is that the increasing loss between models with different heuristics is actually effect of sharper minima in the heuristic case. There is significant body of work on the controversial (Dinh et al., 2017 Kaur et al., 2022) association between wider minima and generalization in models (Li al. [2018] Keskar et al. 2017 Hochreiter and Schmidhuber) (197). Prior work shows that minima forced to memorize a training set without generalizing on a test set exist in very sharp basins (Huang et al. 2020), but these basins are discovered only by directly pessimizing on the test set while optimizing on the train set. is less clear whether width is predictive of generalization across models trained conventionally.
We do not find that sharpness is a good predictor of the quality of model's generalization strategy (Fig. (bb). On the contrary, find that the basin in which model sits is far more predictive of
5

---

Published as a conference paper at ICLR 2023
0152
0.15
0.012
0124
0.14
0.041
HANS wonetttutt Ace-
HANS-LO val Vel
0.07
0.12
0.109
0.12
0243
0.125
0.08
0.08
0.303
177
0.552
0.190
0.44
0.04
0444
0.1
0.45
6,128
LND
0,50
0.00
0.003
CL003
0,10
0.00
LO
val
Aee
constiluen val
Figure 4: Color indicates CG distance (on ID MNLI validation loss) between the x- and y-axis NLI models. Models are sorted by increasing performance on (a) HANS-LO. (b)
generalization behavior than optimum width is. To identify these discrete basins, we define metric based on linear mode connectivity, the convexity gap (CG), and use to perform spectral clustering.
Entezari et al. (2021) define a barrier's height BH on linear path from 01 to 0g as: BH 02) = sup[C(a0) + (1 - a 02) - (ax((0,) + (1 0x) ((2)))
a a [0
(3)
We define the convexity gap on linear path from 01 to 0g as the maximum possible barrier height on any sub-segment of the linear path joining 01 and 02. Mathematically,
CG 02) = sup BH{ + - 702, 301 + (1 - 3)02) Y. B E [0 v.3
(4)
Under this definition, an c-convex basin has a convexity gap of at most e within the basin (proof in Appendix B. In Appendix [ we compare our metric area under the curve (AUC) and BH. These alternatives exhibit weaker clusters, though Euclidean distance has similar behavior to CG.
4.1 CLUSTERING
The basins that form from this distance metric are visible based on connected sets of models in the distance heatmap (Fig. 4(a). To quantify basin membership into a prediction of HANS performance, we perform spectral clustering, with the distances between points defined CG-distance on the ID loss surface. Using the difference between distances from each cluster centroid ||w cil and || C2 we see a significantly larger correlation with HANS performance (Fig, 3(a)), compared to baseline of the model's e-sharpness (Fig. 3(b). Furthermore, linear regression model offers significantly better performance based on spectral clustering membership than sharpness
In Fig. 4(b) perform well on constituent overlap diagnostic sets tend to fall in the basin containing models biased towards lexical overlap. This behavior characterizes the two basins: the larger basin syntax-aware (tending to acquire heuristics that require awareness constituent structure), while the smaller basin is centax--nawwan (acquiring heuristics that rely only unordered sets of words) [
the heuristic that defines the larger cluster: constituent overlap. Models that
Distributions of the clusters: We find (Fig. 5) that CG-based cluster membership accounts for some of the heavy tail of performance HANS-LO, supporting the claim that the convex basins the loss surface differentiate generalization strategies. However, the distribution of ID performance also differs significantly between clusters, so we may choose to relate basin membership ID behavior instead of OOD. We discuss this alternative framing further in Appendix H
Here, w, C1. C2 are in spectral embedding space.
'The connection between cluster membership and generalization performance is continuous. That is, those
members of the generalizing cluster which are eloser the cluster boundary hehave little more like heuristic models than the other models that cluster
WWe reject alternative hypothesis, that the relevant difference is an awareness of word position rather than syntax, because reliance subsequence overlap poorly predicted by basin memhership (Appendix C

---

Published as a conference paper at ICLR 2023
15
Count
Count
10-
Count
0,310
0315
0.10 Acc. HANS-subuquunnee
0TO
Acc. on MNLI
Asc. on HANS-LO
Acc. HANS-oesttuenl
Figure 5: Histogram of accuracy scores of the MNLI models. Cluster that relies on lexical overlap heuristics in orange; cluster that generalizes to HANS-LO is blue.
0.419
pearsonr
-0.642
0,645
0.30
cluster-O
0.422
025
0.440
cluster-1
PAWS val
0.20
on PAWS
0.435
0.416
0.15
0.430
0.438
0.425
0.440
0.05
0.420
0.444
0.00
0.434
0.43
0,43
0.860
0.00
0.01
0.02
|| C1ll - 1| - C2ll
PAWS val f
Figure 6: Results from CG (over QQP validation loss) for QQP models. (a) CG heatmap with QQP models sorted in order of increasing performance PAWS-QQP. (b) Among QQP models, cluster membership is highly predictive of PAWS-QQP performance. Least squares shown.
QQP: On QQP, we also find distinct clusters of linearly connected models (Fig. 6(a) As NLI, find that cluster membership predicts generalization on PAWS-QQP (Fig. 6(b)
4.2 GENERALIZATION BASINS TRAP TRAINING TRAJECTORIES
At this point we have aligned basins with particular generalization behaviors, but is possible that the heuristic models just need to train for longer switch to the generalizing basin. We find that this not the case (Fig. 7 Heuristic models that fall closer to the cluster boundary may drift towards the larger cluster later in training. However, models that more central to the cluster actually become increasingly solidified in their cluster membership later in training.
These results have important implications. First, they confirm that these basins constitute distinct local minima which can trap optimization trajectories. Additionally, our findings suggest that early on in training, conclusion is crucial for any future work towards practical methods of directing or detecting desirable generalization strategies during training (Jastrzebski al. 202T.
detect whether final model will follow particular heuristic. The latter
5 RELATED WORK
Connectivity of the Loss Landscape: Draxler et al. (2019) and Garipov et al. (2018) demonstrated that pairs of trained models are generally connected to each other by nonlinear paths of constant train and test loss. Garipov et al. (2018) and, later, Fort and Jastrzebskil (2019) conceptualized these paths as high dimensional volumes connecting entire sets of solutions. Goodfellow et al. (2015) explored linear connections between model pairs by interpolating between them, while Entezari et al. (2021) and Ainswwoth et al. (2022) suggested that in general, models that seem linearly disconnected may be connected after permuting their neurons. Our work explores models that linearly connected
7

---

Published as a conference paper at ICLR 2023
Movement in CG metric space with training
0.445
cluster-O
cluster-1
switching clusters
0.440
Final f7 on PAWS
0.435
0.430
0.425
-0.02
-0.01
0.00
0.01
0.02
1|w - -ill 1|w C2ll
Figure 7: Movement between clusters during finetuning QQP. Three checkpoints(15000 25000 and end of training(341 10) steps of training) for each model are considered. The arrows point from one checkpoint, to next checkpoint of the same model. Clustering uses the CG metric the QQP validation loss surface. Heuristic models near the cluster boundary move towards the generalizing cluster, but central models in both clusters move closer to their respective centroids.
without permutation, and is most closely linked with Neyshabur et al. (2020). which found that models initialized from shared pretrained checkpoint are connected. Frankle (2020) also found that pruned initial weights can lead to trained models that interpolate with low loss, the pruned model generalizes well. These results, unlike ours, all focus on image classification. Wortsman et al.) (2022) considers a similar setting to ours by finetuning with a linear classifier head, and develops weight ensembles that depend on assumptions of LMC. They find that such ensembling methods are effective in image classification settings, but less so in text classification.
Generalization: Diagnostic challenge sets common method for evaluating generalization behavior in number of modalities. Imagenet has number of corresponding diagnostic sets, from naturally-hain examples (Koh et 2021 Hendrycks et al. [202T) to perturbed images (Hendrycks) and Dietterich 2019) to sketched renderings of objects (Wang et al. 2019). In NLP, diagnostic sets frequently focus on compositionality and grammatical behavior such as inflection agreement (Kim / et 2019, [Sennrich) 2017). Diagnostic sets in NLP can be based on natural adversarial data (Gulor
dava al.. 2018) Linzen et al., 2016), constructed by perturbations and rephrases (Sanchez et al.
2018: Glockner et al.) 2018), generated artificially from templates (Rudinger al. [0088] Zhao ctt al. 2018). NLI models in particular often fail on diagnostic sets with permuted word order (Kim al. 2018; Naik et al.) 2018 McCoy et al. 2019), exposing a lack of syntactic behavior. Because we focus on text classification, we are able to interpret behavior on available rule-based diagnostic datasets in order differentiate between the mechanical heuristics used by different models.
Function Diversity: Initializing from a single pretrained model can produce a range of in-domain
generalization behaviors (Devlin et al. 2019 Phang et al. 2020 Sellam al. 2021 Mosbach et al.) 2021). However, variation on performance in diagnostic sets is even more substantial, whether those sets include tests of social biases (Sellam et al.) 2021) unusual paraphrases (MeCoy et al.) 2020; Zhou et al., 2020). [enton et al. (2021) found that a wide variety of decision boundaries were expressed within low-loss volume, and Somepalli et al. (2022) further found that there diversity in
8

---

Published as a conference paper at ICLR 2023
boundaries during OOD generalization, observed when inputs are far off the data manifold. Our work contributes to the literature on how diverse functions can have similar loss by linking sets of identically trained models to different OOD generalization behavior. We also expand beyond visualizing decision boundaries by focusing on how basins align with specific mechanistic interpretable strategies.
6 DISCUSSION AND FUTURE WORK
Our results stand in stark contrast to findings in image classification. Once neuron alignmen is accounted for, the linear mode connectivity literature (Entezari et al., 2021 Wortsman al. 2022 Frankle et al. 2020 Ainsworth et al., 2022 Neyshabur et al. (0200) points overwhelmingly to the claim that SGD consistently finds the same basin in every training run. There are several reasons to be skeptical about the generality of this view outside of image classification. First, image classifiers may not rely much on global structure, instead shallowly composing local textures and curves together (Olah et al. 2018). In contrast, accurate syntactic parses are applied globally, as a single tree consistently describes the composition of meaning across sentence. Second. existing results are in settings that are well-established to be nearly linear early in training (Nakkiran et al., 2019). Early basin selection may thus occur in a nearly convex optimization space, explaining why different runs consistently find the same basin. However, future work might dig further into image classification, exploring whether it is possible to identify different basins that avoid particular spurious correlations.
Within NLP, we have many more challenge sets for detecting spurious correlations (Warstadt et al. tasks also provide challenge sets (Kim and consider biases towards subject-verb number agreement or syntactic tests (Linzen et al. 2016; Gulordava et al. 2018; Ravfogel et al. 2019). While we identify three tasks with discrete basins, MNLI and QQP have a similar format (a binary label on the relationship between two sentences) and CoLA outlier models are extremely rare. A greater diversity of tasks could reveal how consistently basins align with the generalization strategies explored in existing diagnostic sets, or even identify more faithful distinctions between basin behaviors.
2020). Beyond classification, sequence-to-sequeee
Linzen 2020). In language modeling,
Future research may also focus on evaluating the strength and nature of the prior distribution over basins. Transfer learning priors range from the high entropy distribution provided by training from scratch those pretrained models which seem to consistently select a single basin (Neyshabur et al., 2020). Possible influences on basin selection, and therefore on generalization strategies, may include length of pretraining (Warstadt et al. 2020, see Appendix J). data scheduling, and architecture selection (Somepalli et al.) (022)) The strength of a prior towards particular basins may be not only linked to training procedure (Appendix K) but also related to the availability of features the
pretrained representations (Lovering al., 202 Hermann and Lampinen] 2020 Shah et al. 2020)) This aspect of transfer learning is a particularly ripe area for theoretical work in optimization.
Another future direction is to detect early in training whether a basin has a desirable generalization strategy. For example, can we predict if a Transformer will use position information or act like a bag-of-words model? It is promising that CG on training loss is predictive of OOD generalization (Appendix G.. so measurements can be directly on the training set. Furthermore, a model tends to stay in the same cluster late in training (Section 4.2). final heuristics may be predicted early.
The split between generalization strategies can potentially explain results from the bimodality of CoLA models (Mosbach al. 202T) to wide variance NLI diagnostic sets McCoy al. (2020). Because weight averaging can find parameter settings that fall on a barrier, we may even explain why weight averaging, which tends to perform well on vision tasks, fails in text classifiers (Wortsman et al.) 2022). Future work that distinguishes generalization strategy basins could improve the performance of such weight ensembling methods.
CONCLUSIONS
In one view of transfer learning, a pretrained model may select prior over generalization strategies by favoring a particular basin. Neyshabur et al. (2020) found that models initialized from the same
We used the methods of Ainswortt et al. 2022 to align neurons within each head across different training runs, finding alignment without any permutation. Therefore, our basins are likely to be separate even under neuron permutation (though there may be permutations we did not find), in contrast to their findings on CIFAR.

---

Published as a conference paper at ICLR 2023
pretrained weights are linearly connected, suggesting that basin selection is key component of transfer learning. By considering performance on NLP tasks instead of image classification, we find that pretrained model may not commit exclusively single basin, but instead favor small set of them. Furthermore, we find that linear connectivity can indicate shared generalization strategies under domain shift, evidenced by results NLI, paraphrase, and linguistic acceptability tasks.
ACKNOWLEDGEMENTS
This work was supported by Samsung Advanced Institute of Technology (under the project Next Generation Deep Learning: From Pattern Recognition to Al) and NSF Award 1922658 NRT-HDR: FUTURE Foundations, Translation, and Responsibility for Data Science.
This collaboration was facilitated by ML Collective.
REFERENCES
K. Ainsworth, J. Hayase, and S. Srinivasa. Git Re-Basin: Merging Models modulo Permutation Symmetries, Sept. 2022. URL http://arxiv org org/ abs/ 2209.04836 arXiv:2209004836 [cs].
G. W. Benton, W. J. Maddox, S. Lotfi, and A. G. Wilson. Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling. arXiv:2102.1 13042 [cs, star], Feb. 2021. URL http? arXiv: 2102.13042.
//arxiv.
2200. 13042
N. Chatterji, B. Neyshabur, and H. Sedghi. The intriguing role of module criticality in the general- ization of deep networks. In International Conference on Learning Representations, 2020. URL https::/cpeereete
net 2e
F. Consortium, R. Cooper, D. Crouch, J. Eijck. C. Fox, J. Genabith, Jaspars, Kamp, D. Milward, M. Pinkal, M. Poesio, S. Pulman, T. Briscoe, H. Maier, and K. Konrad. Using the framework, 1996.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805, 2019.
L. Dinh, R. Pascanu, S. Bengio, and Y. Bengio. Sharp Minima Can Generalize For Deep Nets. 0rXiv:1703.0493: [cs], May 2017. URL 1703.04933.
arXiv:
Draxler, K. Veschgini, M. Salmhofer, and F. A. Hamprecht. Essentially No Barriers in Neural Network Energy Landscape. arXiv:/803. 00885 [cs, stat], Feb. 2019. URL http:://rxix arXiv: 1803.00885.
org/abs /wwaab///83... 00885
R. Entezari, H. Sedghi, O. Saukh, and B. Neyshabur. The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks. arXiv:2110.06296 [cs], Oct. 2021. htthtpp //axxiv.org/ab//211... 06296 arXiv: 2110.06296.
S. Fort and S. Jastrzebski. Large Scale Structure of Neural Network Loss Land- scapes. Advances Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL
neurips htt.:/r.pps.cccppppe//////////// 3800dleee4d0f2222aaa000111aaaaa2 html
J. Frankle, G. K. Dziugaite, D. Roy, and M. Carbin. Linear Mode Connectivity and the Lottery Ticket Hypothesis. In Proceedings of the 37th International Conference on Machine Learning, pages 3259-3269. PMLR, Nov. 2020. URL htttps::/proceediinngg .ml...r.................. www.m.......///ee. ISSN: 2640-3498.
Frankle20a. html
T. Garipov, P. Izmailov. D. Podoprikhin, D. Vetrov, and A. G. Wilson. Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs. arXiv:1802. 10026 [cs, stat), Oct. 2018. URL http://arxiv. www.vorrggbbb.. 10026
http:///arxv..orr//b////2/
arXiv: 1802,10026
10

---

Published as a conference paper at ICLR 2023
M. Glockner, V. Shwartz, and Y. Goldberg. Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of the 56th Anmual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 650-655 Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 10.116653/1//11--2100 URL https
J. Goodfellow, O. Vinyals, and A. M. Saxe. Qualitatively problems. arXiv:1412.6544 [es, stat), May 2015. URL
neural network wwt.iiiiattiioittttioooo
6544 arXiv: 1412.6544.
K. Gulordava, P. Bojanowski, E. Grave, T. Linzen, and M. Baroni. Colorless green green recurrent networks dream bierarchically. arXiv: 11138 [cs], Mar. 2018. URL http::/arxiv.
1803.11138 arXiv: 1803.11133
D. Hendrycks and T. Dietterich. Benchmarking neural network robustness common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. CVPR, 2021.
K. L. Hermann and A. K. Lampinen. What shapes feature representations? Exploring datasets, architectures, and training. arXiv:2006. 12433 [es, stat], Oct. 2020. URL httt::/ abs 12433 arXiv: 2006. 12433.
S. Hochreiter and J. Schmidhuber. Flat minima. Neural Computation, 9:1-42, 1997.
W. R. Huang, Z. Emam, M. Goldblum, L. Fowl, J. K. Terry, F. Huang, and T. Goldstein. Understanding Generalization through Visualizations. arXiv: /90660029999 [cs, stat], Nov. 2020. URL |RLhttp:
03291
arXiv: 1906.03291.
S. Jastrzebski, M. Szymczak, S. Fort, D. Arpit, Tabor, K. Cho, and K. Geras. The Break-Even Point on Optimization Trajectories of Deep Neural Networks. arXiv:2002.0955 [cs, stat], Feb. 2020. URL http:: wrt/ 2002. 09572 arXiv: 2002.09572.
S. Jastrzebski, D. Arpit, O. Astrand, G. B. Kerg. H. Wang, C. Xiong, R. Socher, K. Cho, and K. J. Geras. Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization. In Pro- ceedings of the 38th International Conference on Machine Learning, pages 4772 PMLR, July 2021. URL lbttps:/pprocedd wtt.igg........ss....... /vtsss/vv1//// (as.:999/aatttttttaaaaea htmml ISSN: 2640-3498.
mlr
S. Kaur, J. Cohen, and Z. C. Lipton. On the Maximum Hessian Eigenvalue and Generalization. arXiv:2206116633 June 2022. doi: 10.48550/arXiw.2266.666 10654. URL abs/2206. 0654v2
N. Keskar, D. Mudigere, Nocedal, M. Smelyanskiy, and P. Tang. On large-batch training for deep learning: Generalization gap and sharp minima. ArXiv, abs/1609.04836, 2017.
Kim, C. Malon, and A. Kadav. Teaching syntax by adversarial distraction. Proceedings of the First Workshop Fact Extraction and VERification (FEVER), pages 79-84, Brussels, Belgium, Nov. 2018, Association for Computational Linguistics. doi: 10.18653/w1/118-53 URL https:://///m.
N. Kim and T. Linzen. COGS: A Compositional Generalization Challenge Based on Semantic Interpretation. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 9087-9105, Online, Nov. 2020. Association for Computational Linguistics. doi: 10.186533v1/220 ww1.00memmllmmmim..mmm URL Intpt:t::/// ttttpp:://ccaanholooyy orgg/ 2020. emnlp main. 731
N. Kim, R. Patel, A. Poliak, P. Xia, A. Wang. T. McCoy, I. Tenney, A. Ross, Linzen, B. Van Durme, S. R. Bowman, and E. Pavlick. Probing what different NLP tasks teach machines about function word comprehension. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 235-249, Minncapolis. Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/S19. 19-16/9999911....
URL https: // ac wecannhhooogy
org/ S1 902
11

---

Published as a conference paper at ICLR 2023
P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga,
R. L. Phillips. I. Gao, T. Lee. E. David, Stavness, W. Guo, B. A. Earnshaw, I. S. Haque, S. Beery, J. Leskovee, A. Kundaje. E. Pierson, S. Levine, C. Finn, and P. Liang. WILDS:
A Benchmark of in-the- Wild Distribution Shifts. arXiv:2012.0742/ [cs), July 2021. URL htttp://axxv.. wwtp abs 2012 07421
arXiv: 2012.07421.
A. Kumar, A. Raghunathan, R. M. Jones, I. Ma, and P. Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution. In International Conference on Learning Represen- tations, 2022. URL httphhtttt///peeeeeeeee
H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein. Visualizing the Loss Landscape of Neural Nets. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 3389. Curran Associates, Inc., 2018. URL http:: http:/ppaarrs.iis......p www.. weriips.cc/appeee
T. Linzen, E. Dupoux, and Y. Goldberg. Assessing the Ability LSTMs Learn Syntax-Sensitive Dependencies. arXiv:1611.011 666.01368 [cs], Nov. 2016. URL http::/taxxi..oo//// wtt.:/awww...gga..aaaaaaaa 01368 arXiv: 1611.01368.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: robustly optimized bert pretraining approach, 2019. URL. https
11692
I. Loshchilov and Hutter. Decoupled weight decay regularization, 2017. URL org/abs/117 1111.000111111111111111111111111
C. Lovering, R. Jha, T. Linzen, and E. Pavlick. Predicting inductive biases of pre-trained models. In ICLR, 2021.
C. Ma, L. Wu, and L. Ying. The Multiscale Structure of Neural Network Loss Functions: The Effect on Optimization and Origin. arXiv:2204. //36 [cs], Apr. 2022. URL abs/2204. 11326
arXiv: 2204.11326.
R. T. McCoy, J. Min, and T. Linzen. Berts of feather do not generalize together: Large variability in generalization across models with similar test set performance. ArXiv, abs/1911.02996 2020.
T. McCoy, E. Pavlick, and T. Linzen. Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428-3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 111886/111111911331 URL https::/aclanttollog.
org/P19 111999111 1134
Miller, R. Taori, A. Raghunathan, S. Sagawa, W. Koh, Shankar, Liang, Y. Carmon, and I. Schmidt. Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In in-Distribution Generalization, Oct. 2021. htt.::://aaa...o////aa.a/ arXiv:2107.04649 [cs, stat].
M. Mosbach, M. Andriushchenko, and D. Klakow. On the Stability of Fine-tuning BERT: Mis- conceptions, Explanations, and Strong Baselines. arXiv:2006. 04884 [es, star], Mar. 2021. URL htttp://arxise wwt.:oggg/////o...o http://bs//000..000
arXiv: 2006.04884.
V. Nagarajan and Z. Kolter. Uniform convergence may be unable to explain generalization in deep learning. In Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019. URL httptttt///prcccciiinoo neurips..e.p.p.... http://iinn...uu.....o...aaapp//
cctppper/ppppppcccc/cc 9/hash/ 7bb- Abstractst......... htm]
A. Naik, A. Ravichander, N. Sadeh, C. Rose, and G. Neubig. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference Computational Linguistics, pages 3340-2353, Santa Fe, New Mexico, USA. Aug. 2018. Association for Computational Linguisties. htt.httts:///acaataaat ogy ogw.oggoc...oooooooooooo
12

---

Published as a conference paper at ICLR 2023
P. Nakkiran, G. Kaplun, D. Kalimeris, T. Yang, B. L.. Edelman, F. Zhang, and B. Barak. SGD on Neural Networks Learns Functions of Increasing Complexity. arXiv:1905. 1600 [cs, stat], May 2019. URL/tttp://axxtt wwwp:xivi.ii...vvoooooo... abs 111.1.1001..16
arXiv: 1905.11604.
B. Neyshabur, H. Sedghi, and C. Zhang. What is being transferred in transfer learning? In Advances in Neural Information Processing Systems, volume 33, pages 512-523, Cur- Associates, Inc., 2020. URL httpss:/ .nipaperrs.iiss............s..ss nips hctpppppee/222///aalll
C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev, The Building Blocks of Interpretabilitty. Distill, 30))=10, Mar. 2018. ISSN 2476-0757. doi: 10.23915/ distill.00010. URL
Phang, J. Park, and J. Geras. Investigating and Simplifying Masking- Saliency Methods for Model Interpretability. arXiv:2010.0911 09750 [cs]. Oct. 2020. URL
abs 2010 09750 arXiv: 2010.09750.
Ravfogel, Y. Goldberg, and Linzen. Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages. arXiv:/903 1003v:999300444 004300000 [cs], Mar. 2019. URL httpttt:://////t http::t/ttt:/t// arXiv: 1903.06400.
org/abs 903. 06400
R. Rudinger, Naradowsky, B. Leonard, and B. Van Durme. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for
Computational Linguistics: Human Language Technologies, Volume (Short Pupers), pages 8-14, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi:
1..18653/11/11--1 20022. URL hatpps://aclaathhll.gs.. ottp:///tttooooooooo//o.. org/N18- 2002
I. Sanchez, J. Mitchell, and S. Riedel. Behavior analysis of NLI models:
Uncovering the influence of three factors on robustness. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Pupers), pages 1975-1985, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18- 1179. URL/httpa:/ ac
1twp:/loyy..g//1--1111111 1179
Sellam, S. Yadlowsky, Tenney, J. Wei, N. Saphra, A. D' Amour, T. Linzen, Bastings, I. R. Ture, J. Eisenstein, D. Das, and E. Pavlick. The MultBERTs: BERT Reproductions for Robustness Anal- ysis. In ICLR, Sept. 2021. URL /ttp::://ppeeerrreiie http:/eeteeeettooommooooo-o-0000
R. Sennrich. How grammatical is character-level neural machine translation? assessing MT quality with contrastive translation pairs. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 376-382, Valencia, Spain, Apr. 2017. Association for Computational Linguistics. URL htttsss/
oxq/E17-2060
H. Shah, K. Tamuly, Raghunathan, P. Jain, and P. Netrapalli. The Pitfalls of Simplicity Bias in Neural Networks. 1xXiv:2006007710 [cs, star], Oct. 2020. URL [RR.httpp//arxlw.or///// 2006. 07710 arXiv: 2006.07710.
G. Somepalli. L. Fowl, A. Bansal, Yeh-Chiang, Y. Dar, R. Baraniuk. M. Goldblum, and T. Goldstein. Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective. arXiv:2203.00 08/00/224444 [cs], Mar. 2022. URL http:
//arxiv. www.lvoorrgaaae
2203222233.. 08124 arXiv: 2203.08124.
A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. ArXiv, 0ts/1804407461, 2018
H. Wang, S. Ge, Z. Lipton, and E. P. Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pages 10506-10518, 2019.
Z. Wang, W. Hamza, and R. Florian. Bilateral multi-perspective matching for natural language sentences. ArXiv, abs/1702,00 /2//1702.03114, 2017.
A. Warstadt, A. Singh, and S. R. Bowman. Neural network acceptability judgments. arXiv preprint arXiv: 1ooc. 2018.
13

---

Published as a conference paper at ICLR 2023
A. Warstadt, Y. Zhang, X. Li, H. Liu, and S. R. Bowman. Learning Which Features Matter: RoBERTa
Acquires a Preference for Linguistic Generalizations (Eventually). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 217-235.
Online, Nov. 2020. Association for Computational Linguistics. URL https: www. ac wweb
org/anthol wwloog//22// emn 1ammminmmm..
A. Williams, N. Nangia, and Bowman. A broad coverae challenge corpus for sentence un- derstanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Laguage Technologies, Vol- ume 1 (Long Papers), pages 11 1112-1122, Association for Computational Linguistics, 2015 8. URL attp://acle
org www.annthoooyyllaa 11c1
T. Wolf, L. Debut, V. Sanh, Chaumond, C. Delangue, A. Moi, Cistac, Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. Transformers: State-of-the-art natural lan- guage processing. In Proceedings of the 2020 Conference on Empirical Methods in Natu- ral Language Processing: System Demonstrations, pages 38-45, Online, Oct. 2020. Associ- ation for Computational Linguistics. doi: 10.18653/v 10.18653 172000.emmlp-demmo.... URL https
7/ //aclancholoyy.ooo ptq/2020..o...ooo emn pomnpp-demss...
M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, and L. Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. arXiv:2203.055 05482 [cs], Mar. 2022. URL http::pppp/xxxx.oo////oo///o//// abs /203 05482
arXiv: 2203.05482.
C. Xing, D. Arpit, C. Tsirigotis, and Y. Bengio. A Walk with SGD. arXiv: [cs, stat), May 2018.
08770
arXiv: 1802.08770.
Y. Zhang. Baldridge, and L. He. PAWS: Paraphrase Adversaries from Word Scrambling. In Proc. of NAACL, 2019.
J. Zhao, T. Wang, M. Yatskar, V. Ordonez, and K.-W. Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Short Papers), pages 15-20, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.186531 N0.18653/v1ll 11/33//1/1112// 2003. URL/ttttps:///cllattoooogg .rg. 2003
X. Zhou, Y. Nic, H. Tan, and M. Bansal. The Curse of Performance Instability in Analysis Datasets: Consequences. Source, and Suggestions. arXiv:2004. 13606 [cs]. Nov. 2020. URL http:
//axxiv.org/abs/
13606
arXiv: 2004.13606.
14

---

Published as a conference paper at ICLR 2023
A LINGUISTIC ACCEPTABILITY
The Corpus of Linguistic Acceptability (CoLA; Warstadt al. 2018 is of acceptable and unacceptable English sentences collected from the linguistics literature. Linguistics has a longstanding practice of studying minimal changes that render sentences ungrammatical: one CoLA example is the pair of sentences "Betsy buttered the toast" (acceptable) and "Betsy buttered at the toast" (unacceptable).
CoLA includes an ID val/test set, where the examples are taken from the same linguistics papers that the training set uses. However, it also includes an OOD diagnostic val/test set. The diagnostic sets are taken from a different set of linguistics papers, so in order for model to perform well on CoLA-OOD, must transfer a general ability to recognize unacceptable English sentences, rather than simply learning the set of acceptability rules described in the ID sources.
A.1 EXPERIMENTAL DETAILS
We found that default settings on the HuggingFace (Wolf et al. 2020) training scrip resulted in more pronounced barriers between models, compared to the Google script used for NLI and QQP |Because our goal is to study the relationship between barriers and generalization, we therefore chose use Huggingface for our CoLA analysis. Like in our other experiments, we kept the default hyperparameters, which differ slightly from the Google script. The CoLA models were trained for epochs with learning rate of 2 10-5, a batch size of 32 samples, and no weight decay. This script uses the Adam W (Loshchilov and Hutter (017) optimizer too, with a linear learning rate decay schedule but no warm-up.
A.2 CLUSTERING
nearsonr 0.367
0040
0.524
0.35
CaLA-OOD val MCC
0,529
0,00
MCC on CoLA-OOD
0.56
0105
0.5c
0,540
0005
0.551
0.05
chuste-0
0.50
049
0.515
1:40
0.58
000
eluster1
-0.00
-0.55
0.00
0.10
OOD val MCC
|| - cyll ||w
C2ll
Figure 8: Results from CG (over CoLA in-domain validation loss) for CoLA models. (a) CG heatmap on CoLA models, sorted by OOD validation. (b) A scatter-plos of cluster membership versus performance on CoLA-OOD validation set.
In CoLA, there are very few barriers between finetuned models. A single model out of the 48 finetuned accounted for all substantial interpolation convexity gaps (Fig 8(a)). thus forming its own one-point cluster when using CG as distance metric for spectral clustering. This outlier model outperformed all others on OOD generalization (Fig. 8(b)). suggesting that CoLA is another task where models with different generalization behavior are disconnected.
text-classification
_Lax_gguue py
'The major difference between scripts appears to be the lack of different initializations of classification head between huggingface runs. That is, different data order is the only source of SGD noise in HuggingFace runs. However, it is likely that the presence absence of second cluster during the sweep is due to random chance, given that only single model of 48 falling into the outlier cluster these results.
15

---

Published as a conference paper at ICLR 2023
B THEORETICAL RESULT ON CONVEXITY GAPS
Theorem 1. An e-basin will have (G w2) S for every pair of models W1, w2 on its surface.
Proof. Recall the definition convexity gap as the maximum value of the barrier height of any segment f1, 02 along the interpolation between W1 and W2 from Equation I
CG (wi, w2)
sup ( + (1 - u2, 9w1 (1 - ()) v.8
7.8 [0,
(5)
As are in an e-basin, the defining inequality from Equation 2holld V0,, 0g e-convex basin
an0 s
(00))
(6)
Applying this for n 2:
L(a,01 + (1 01 )22 (a, ((01) + (=1)((())))) S
v01,02 E 6-basin
Hence the supremum of the quantity on LHS is also Sceing the definition BH from Equation we immediately see that:
3
BH (1, 02) S
V01, 82 E c-basin
(8)
As CG is the supremum of BH over elements within the e convex basin only, we have:
CG(w1, W2) sc
Vw1, W2 e-basin
C HANS PERFORMANCE ON SUBSEQUENCE HEURISTICS
Models that perform poorly subsequence heuristics tend to fall in the bag-of-words basin. However, the clusters are less pronounced than for either constituent or lexical overlap heuristics (Fig. 9
0.009
0.16
HANS subsequence val Acc.
0.026
0.14
0.12
0.037
0.10
0.043
0.08
0.052
0.06
0.062
0.04
0.079
0.140
0.009
0.026
0.037
0..43
0.052
0.062
0.079
0.140
HANS- SuNSS-uuseeeeence val Acc.
Figure 9: CG (over in-domain MNLI validation loss) for MNLI model pairs, sorted by increasing
performance on HANS -uss-uuueeeee
16

---

Published as a conference paper at ICLR 2023
D FLEXIBLE MODE CONNECTIVITY
While our results strongly suggest that linear mode connectivity breaks down between heuristic and generalizing models, they should nonetheless be connected, at least by nonlinear path of constant training loss (Draxler et al.) 2019, |enton et al. (021). We consider the behavior of HANS generalization along this path, which maintains near-constant ID loss. These paths were easy to identify with Riemannian curves and segmented lines (Garipov et al. 2018), and exhibited poor
HANS-LO generalization the further along the path the parameters fall, shown in Fig. I0
The results here show an inversion of the pattern exhibited on HANS-LO loss in the linear interpolation connections to exhibit peaks ID loss but no corresponding spikes in OOD loss, here we find constant-loss paths which exhibit peaks in HANS-LO loss. It appears that gradient descent finds generalizing models spite of the fact that they exist in constant-loss multicimensional volume which exhibits a wealth of heuristic models.
case: whereas Fig. [ shows ceuristic-to-genraalll
Loss on segmented line (6 sub-epoch training)
4.0
MNLI
HANS-LO
3.5
All HANS
3.0
NLL Loss
2.5
2.0
bend:
bend:
bend:
hend: 4
1.5
1.0
0.5
20
40
Distance along the segmented line
Figure 10: Loss along the segmented curve (after 6 subepochs) between heuristic and generalizing NLI model. sub-epoch equivalent to 120 training steps with a batch size of 1 128, and learning rate of x 10-5
E CORRELATIONS BETWEEN PERFORMANCE ON HANS SUBSETS
As provided in Fig. 11 performance on HANS-LO non-entailing, which reflects an aversion to the lexical overlap heuristic, positively correlated with ID accuracy. However, in the case of HANS-constiuennt model that successfully averts the constituent overlap heuristic is likely to perform poorly ID and on most diagnostic sets. We therefore conjecture that models which avoid constituent overlap heuristics are likely failing acquire syntax, rather than effectively acquiring better generalization strategies.
Performance on the subsequence heuristic diagnostic set is positively correlated with performance on both LO and constituent heuristic diagnostic sets, as it includes aspects of both other heuristics. For every diagnostic set heuristic, performance on the non-entailing subset (i.e., examples that violate the heuristic) is negatively correlated with performance on the entailing subset (i.c., examples that adhere to the heuristic).
17

---

Published as a conference paper at ICLR 2023
1.0
mnli
0.0073
0.24
0.006
0.49
0.35
-0.33
0.8
subseg ent 0.0073
-0.39
0.59
-0.28
0.08
-0.15
0.6
subseq
0.24
-0.53
-0.078
0.16
0.4
Data split
LO ent- 0.006
0.59
-0.53
-0.41
0.11
-0.26
LO
0.49
-0.28
0.78
-0.41
-0.081
const, 0.35
-0.46
-0.2
-0.4
suberg ems
rubsee non-ead
Lo ent-
const ent-
cuast, pon-enl-
LO non nonoo
Data Split
Figure 11: Correlation between accuracy various ID and OOD diagnostic NLI test sets. MNLI refers the MNLI validation set. Other sets were taken from HANS validation sets for subsequence, constituent, and lexical overlap heuristics. Both entailing and non-entailing subsets are included.
EXPERIMENTAL DETAILS
F.1 FINETUNING THE MODELS
The QQP models were trained for epochs, with learning rate of 2 10 batch size of 32 samples and weight decay of 0.01 from the bert-ber bert-base-uncaset 10 pre-trained checkpoint using the google script This script uses the AdamW (Loshchilov and Hutter, (017)) optimizer with warm-up during the first 10% of the training followed by linear schedule for weight decay. Because these hyperparameters are the recommended defaults for BERT training, they were also used by McCoy et al. (2020) to train the MNLI model set we analyze.
F.2 INTERPOLATIONS
In order evaluate the interpolation between a pair of models, we sample random set of 512 samples from the target dataset and then test each of the interpolated models this same sample. For an interpolation between two fine-tuned models 0, and 02, we evaluate the models with interpolation coefficients (Eqn 1] intervals of To the line joining 0 and fg. The resulting plots shown in Fig. (a) and Fig. ((b) We calculate the convexity gap (G,, is using the loss values of these interpolated models.
http://///..rrree..opppppppppppp 768 A- 12.zipp
1/20 1__10_11/uccceeeeee__ uncased. 1.-
https://github. hww.:ttwbb.ommmooooomoo http:/.comooogllloo
18

---

Published as a conference paper at ICLR 2023
In order to gauge the effect of number of samples, repeated interpolation experiments MNLI using only 256 samples (instead of 512). Using a smaller sample reduced the magnitude of correlation coefficient in Fig. ((a) to -0.65, but the cluster assignments remain the same.
F.3 MODEL EVALUATIONS
When evaluating the performance of a finetuned model (e.g., to sort the models on the linear connectivity heatmaps, or select the top and bottom models for Fig. [(a) and Fig. [(b),. we use the whole dataset available. These include the entirety of:
All non-entailing samples given heuristic from the test split of HANS. split of PAWS-QQP.
The
The entire QQP and MNLI validation sets.
As Feather-BERT's have been finetuned on MNLI, to evaluate them on HANS tasks, we would need to convert the logits for the three MNLI classes (entailments neutral, cont contradicti
to logits for the two HANS classes(ent ling and zon-entailing) We convert these la- bels by taking the logit for non-entailing class as the maximum the logits the MNLI contradi ct and neutral classes, and taking the logit for ent ailaiiiing llln class of HANS as the logit of entail ing class of MNLI.
F.4 COMPUTING SHARPNESS
For calculating e-sharpness of model, we use the definition of Keskar al. (2017):
$2. A) maXyec, (f(x + Ay)) f(x) 100
(10)
+ f(x)
where ff the loss function, x E R²" are the original parameters of the model, A E "xx is matrix that restricts the calculation of e- sharpnesss to subspace of the parameter space, and
C. = {= E TRP ((((A (((((AAt))))))) + S 2, S c(((At)) + 1)vi E [p]
(11)
Following the defaults in Keskar al. (2017), we take Mn Using 32768 samples from MNLI val 1 diat on on matched, we evaluate the initial loss fx) We use the same samples to compute
the maximum loss maxgec, f(x + Ay), using an SGD optimizer with learning rate of 8 x 10-5, batch size of 32, and accumulating gradient over 4 batches. In order to compute maximum loss, perform a total of 8192 gradient updates, each followed by clamping of the weights within C.. We set to 10-5
F.4.1 COMPUTATIONAL RESOURCES
Finetuming 100 QQP models cost around 500 GPU-hours and 48 CoLA models cost around 10 GPU- hours. Each 100 x 100 interpolation and evaluation consumed 114 GPU hours; these were performed for both QQP (at three stages during finetuning, for Fig- 7 and MNLI. The 48 48 interpolation for CoLA cost about 28 GPU-hours. The experiments add up to a total cost of approximately 994 GPU-hours mix of NVIDIA RTX8000 and V100 nodes.
G TRAINING LOSS
Even without any validation test set to analyze, can use the training set alone to make predictions about a model's generalization strategies. Fig, [2]shows a strong relation between training loss convexity gaps and performance HANS-LO. Future work can develop methods for predicting generalization early based entirely on training behavior.
H IN-DOMAIN GENERALIZATION
well known that OOD generalization is strongly correlated with ID performance general Miller al. 2021) Let therefore consider the skeptical position that basin membership does not
19

---

Published as a conference paper at ICLR 2023
pearsonr 0.695
0052
cluster
cluste--0
HANS-LO accuracy
0129
030
uucurscy on HANS-LO
02077
025
6243
020
0.7-
0.2-
0414
001
0.1
0.545
(1:40
0.552
1155
0.00
-002
||W
-100
C1ll
0.50
1|w
Call
692
SO accuracy
(a) Clusters based convexity gaps in the MNLI training loss surface. Best squares fit shown.
(b) Color indicates convexity gap. Models are sorted according OOD performance.
Figure 12: Relationship between CG the MNLI training loss surface and accuracy the HANS- LO diagnostic set.
train
test
JAC²
0
PC; -M02
maX f(w)
00
MNLI
MNLI
0.844
0.002
0.839
0.002
2.50
1.00
MNLI
HANS-LO
acc
0.301
0.110
0.096
0,033
1.86
1.30
QQP
QQP
F1
0.879
0.001
0.872
0.002
7.00
5.00
QQP
PAWS-QOS
F1
0.436
0.005
0.424
0.004
2.40
0,60
CoLA
CoLA ID
MCC
0.601
0,017
0.600
N/A
0.06
0.06
CoLA
CoLA OOD
MCC
0.537
0.015
0.576
N/A
-2.60
-2.60
Table 1: Cluster mean, standard deviation, and distribution overlap for all tasks on both ID and OOD performance under metric f. C₁ is the larger cluster on each task. On CoLA, C2 contains only single model.
directly relate to OOD generalization strategies, but instead OOD performance is mediating the effect of general model quality in-domain. While ID (on MNLI) and OOD (on HANS) performance are certainly positively correlated (Appendix ED. we find that all 12 of the models in the syntax-unawan cluster fall at least 1.3 standard deviations below the mean of the syntax-aware cluster's performance on HANS-LO (Table [ and only standard deviation below the mean MNLI validation. The partitions are also less clear in the ID metrics (Fig. 13 compared to lexical overlap. Therefore, considered binary class, basin membership is more predictive of OOD behavior than of ID NLI. Furthermore, the fact that performance on HANS- MANS-constituent is negatively correlated with ID performance (Appendix ED makes it clear that the underlying difference between these clusters is awatax-awareness, not overall model quality.
CoLA presents even stronger evidence for interpreting basin membership as related to OOD gener- alization over ID. The single outlier model falling outside of the main basin is a full 2.6 standard deviations outside mean on OOD performance, but only 0.6 on ID performance.
However, we should not dismiss the relationship between ID accuracy and cluster membership. Compared the diagnostic set PAWS, the validation set is more predictive CG on QQP (Fig. [4p. In both MNLI and QQP, the relationship between ID and centroid distance also seems more linear than OOD. Based on these findings, the human-defined heuristics of the PAWS dataset may not be the correct characterization of the difference in generalization behaviors between different basins. We leave the discovery of better characterizations of difference in strategy between basins to future work.
One caveat is the fact that ID evaluation and ID convexity gap measurements are performed on the same validation set, because the test sets not public for these datasets. Therefore, ID performance
20

---

Published as a conference paper at ICLR 2023
0.737
0,87
-016
0.349
clustee-0 cluster
0.41
0.14
0.345
0.842
MNLI Avc.
Acc. 08 MNLI
0.344
0.10
0.842
0.65
0.440
014
0.945
0.838
0,02
0.34
0.941
0.945
0.00
-002
-0.01
0.00
011
0.80
call 1|
MNLI
(a) Clusters based convexity gaps in the MNLI validation loss surface. Best squares shown.
(b) Color indicates convexity gap. Models are sorted according ID performance.
Figure 13: Relationship between CG on the MNLI in-domain validation loss surface and MNLI in-domain validation accuracy.
pearsone -0.859
0.365
035
clusta-O cluster-1
0.360
930
0.578
025
Q
val
f 00
0.979
0,899
0.10
0.880
000
0.863
0,881
0.814
1107%
0,87%
0,880
0055
0.00
001
|| Cyll
||W C2ll
QQP
(a) Clusters are based on convexity gaps in QQP validation loss surface. Best squares fit shown.
(b) Color indicates convexity gap. Models on axes are sorted according to ID performance.
Figure 14: Relationship between CG on the QQP in-domain validation loss surface and QQP in- domain validation accuracy.
exposed during the clustering process, casting some doubt on alignment between the clusters and ID performance.
I ALTERNATIVE BARRIER MEASUREMENTS
In Fig. 15 we show NLI model clustering using the original barrier height metric (Equation 3 from Entezari et al. (202T). can see that the Pearson's correlation coefficient is -0.49, which is far lower in magnitude than correlation in the CG-metric space.
We also show the results for another metric, viz. area under the interpolation curve (AUC), in Fig. [6 Although this shows the same Pearson's 's correlation coefficient, the clusters much less crisp the heatmap. In order to compute AUC, we first subtract the lowest point on the curve from all points, and then compute the under the shifted curve.
Finally, we consider the effect of Euclidean distance (Fig. 18 and The clustering effect is extremely strong and predictive of generalization behavior. is worth considering the possibility that basins trap the models a way that forces them to have larger Euclidean distances, but those Fuclidean distances the property that most determines the generalization strategy of the model.
21

---

Published as a conference paper at ICLR 2023
pearsonr -0.499
0.12
clustee-O
clusta-i
016
HANS-LO val Aec.
Acc. on HANS-LO
005
0.243
0.5
02-
0,300
6.1
0.352
0.44
0,414
000
0.549
6053
6.128
02-43
0,50
||W - C1ll - ||w
Call
HANS LO val Acc.
Figure 15: Relationship between BH on the MNLI validation loss surface and HANS-LO accuracy.
-0.674
0,022
cluste-O cluster-1
0.124
HANS-LO Aec
0,207
12
Acc. on HANS-LO
0.4-
0.249
0.3
06
0.2
02
0.8
0,545
0.22
343
3,00
8.33
9.545
-0.88
-0.02
-0.01
0.50
0.51
0.02
083
||W
C1ll
1|W
c2ll
Aec.
Figure 16: Relationship between AUC on the MNLI validation loss surface and HANS-LO accuracy.
pearsonr -0,600
0.022
cluster-O
0.128
cluster-1
0.207
Acc. on HANS-LO
0.4
vANS-LOOvl
0.243
0.700
0,352
21
0:414
0.555
0.129
220.007
20.243
6.70
0,81
0.55
||W -
W|
C2
LO val
Age.
Figure 17: Relationship between Euclidean distance the MNLI validation loss surface and HANS- LO accuracy.
22

---

Published as a conference paper at ICLR 2023
-0.852
0.855
24
0.814
O0P
QOP val
0.876
o0
0.874
21
0.873
0.870
cluster-O
0.880
cluster-1
0697
0.779
0.000
0,90
19
15
To
0.5
96
05
1.5
[|w
cill 1|w
call
1046
QQP
val
Figure 18: Relationship between Euclidean distance on the QQP validation loss surface and PAWS- QQP accuracy.
J ROBERTA RESULTS
BERT shows consistent clustering behavior, consistently aligned with generalization performance, across each text classification task measured. RoBERTa (Liu et al. 2019) uses the same architecture but trained longer and with slightly different objective and hyperparameters. Despite this similarity, RoBERTa shows significantly better generalization on HANS-LA (Fig. [96b) and does nor exhibit clear clustering behavior (Fig. [9(a). These results suggest that the simple basin selection behavior we observe in BERT may indicate that model can still benefit from longer periods of pretraining.
cluster-0
0.65
cluster-1
HANS-LO Voval Acc.
700
0.12
Count
0712
4.11
4
0.10
0.734
0.99
0.806
6.88
0/08
OUTSR
0,744
6,70
0.70
0.75
0.80
HANS-LO LO val Acc.
Acc. on HANS-LO
(a) Heatmap of CG for finetuned RoBERTa models, sorted by HANS-LO performance. No significant clustering effects are visible, but the scale of barriers similar that among the heavily clustered BERT models.
(b) Histogram on HANS-I LO performance for fine- tuned RoBERTa models, with blue and orange in- dicating the respective models found in each of the clusters formed by k-means clustering the CG spectral space.
Figure 19: CG results on of MNLI models finetuned from RoBERTa.
K RELATION TO FINETUNING DYNAMICS
2022 analytically proved that finetuning creates distorting effect by rescaling existing features rather than learning new features. To remedy this distortion, they proposed that rather than finetuning the entire network once, first train the linear classifier head in isolation and then finetune the rest of the network. This procedure is called linear probing then full fine-tur tuning (L.--T).. We find that L.--TT reduces the range of convexity gaps, and accompanying clustering
Kumar et al.
23

---

Published as a conference paper at ICLR 2023
0.135
15.
0224
HANS-LO vol Ace.
9,0008
Count
10
0,0006
0.315
5
0,0002
0.505
0.22
0.30
0338
0,52
0415
0,505
0,0000
0
0.2
0.4
Acc. on HANS-LO
LO val Ace.
(a) Heatmap of CG for BERT MNLI models trained by LP-FT, sorted by HANS-constiuee constttntt performance. This approach dramatically reduces the non convex- ity of interpolation, compared to full finetuning.
(b) Histogram HANS-LO LO performance for LP- FT BERT models, with blue and orange indicating the respective models found in each of the two clus- ters formed by k-means clustering on the CG spec- tral space.
Figure 20: CG results on a set of MNLI models, initialized from BERT, that have been trained by LP-FT (Kumar et al. 2022).
effects, between models (Fig. 20 p. This change is accompanied by a slight shift in the distribution of behavior HANS-LO (Fig. 20(b)-
These findings are to be expected, as training the linear layer with the rest of the model frozen is a convex optimization problem, and therefore each training should be settling into the same basin to start. These results imply that avoiding certain basins may be part of the advantage of LP-FT traditional finetuning. Although the resulting models still contain slight outlier, the strong multi-cluster behavior is no longer evident. It also presents potential disadvantage of LP-FT for ensembling. however, as the resulting models are more homogenous and therefore ensembling can provide limited benefits.
L THE RELATIONSHIP BETWEEN EUCLIDEAN DISTANCE AND CG
Interpolations in MNU
Interpalations in QQP
245
24
Eustdenn
23-
21
9.50
0.02
0.04
0:06
0.08
0.10
0.2
0.1
0.16
0.30
3.05
0.10
8.15
0.20
0.25
0.50
0.55
Convectit Caps
Convesiity Gaps
Figure 21: Relationship between Euclidean distance and CG. Line of best fit shown separately for connections between and within basin clusters found in the validation loss CG distance space. (a) MNLI models (b) QQP models
One extreme possibility is that these are not distinct basins, but instead that every model falls on the perimeter of single radially symmetric barrier where every cross section is a nearly identical shape. The CG would then be determined entirely by Euclidean distance. Although clearly loss surface geometry causes Euclidean distances to emerge rather than vice versa, we also see from Fig, that the relationship between these two metrics is different within between basin clusters. Although
24

---

Published as a conference paper at ICLR 2023
Upe
bye Ki-25
Sye deriblle
oye temilla
lapa
porbl
parao
pottas
(a) bert-tiny
(b) bert-small
mal
N
EAt 18P
pudlkis
parii
palia
(c) bert-base-uncased
Ioy- gra-teges
oye gn
toe tocooeii
pwiko
peoki
p-lm
polls
(d) bert-large
Figure 22: [Loss surface the line joining pairs of models trained on MNLLJ Performance during linear interpolation between pairs of models taken from the best (gen.) and worst (heur.) in HANS-LO accuracy and average models (middle). Overall the trend is towards better connectivity as the model size increases.
there is no stark division between connections within and between basins, in general Euclidean distance is more responsive to (i.e., has greater slope with respect to) the size of the barrier within clusters. Therefore placement of a model within a basin might determine Euclidean distance in different manner than placement with respect to other basins.
M SCALING LAWS OF LINEAR MODE CONNECTIVITY
We explore the effect of model size linear connectivity of finetuned models in Figure 22 As the scale increases we see the loss landscape changes from one of complete disconnectivitty(Figuee (2()) to one of complete conmectivty(Figure 22(d).. This is in line with the results of Ainsworth al. (2022), which show increasing connectivity for wider ResNet models.
We also observe that the region closer to the finetuned models becomes low-loss(Figure 22(b) first, and only as we increase the scale further, does the non-convexity disappears and the region between models become low-loss too. is though the basins in which the finetuned models land get progressively closer to each other with increasing model size.
25


Reviews:
[{"review_id": "z7uyZL34_V", "reviewer": "Reviewer_bPGJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies linear connectivity from a perspective of out-of-domain generalization for NLP tasks. Experiments show that 'generalizing' and 'heuristic' models (which are defined based on their out-of-domain performance) are separated in terms of what 'basins' they belong to. Further analysis along with clustering techniques show a connection between models' behaviors and their position in the loss surface.", "summary_of_the_review": "To reiterate from 'strengths and weaknesses':\n\n- Novel and interesting observations.\n- Well-defined metrics.\n- Lacking some information on experimental setup.\n- Limited implications.", "strength_and_weaknesses": "The empirical observations are interesting and show novel connections between a model's behavior (in terms of its out-of-domain generalization and preferences).\n\nThe work proposes new metrics that seem intuitive and are suitable for what they're designed to capture, like the convexity gap, epsilon-basin.\n\nThe main weakness, in my opinion, is that there seems to be lacking information regarding the experiments. Most, if not all, experiments rely on having 'heuristic' and 'generalizing' models, but I couldn't find details on how these models are obtained. Are generalizing and heuristic models trained the exact same way and only differ in the random seed? If different training settings were used, then a detailed discussion on it would be very valuable.\n\nAlthough the paper shows interesting observations, it's unclear whether these have meaningful implications and/or applicability, and are somewhat restricted to the NLP tasks explored.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "nP2iOC_YHu", "reviewer": "Reviewer_XZas", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies linear mode connectivity (LMC) for language tasks. Usually, LMC is studied on the training and test loss landscape, both of which come from the same distribution. [Neyshabur et al.](https://arxiv.org/pdf/2008.11687.pdf) study LMC for several fine-tuning tasks, and show that the LMC can be observed on the fine-tuning landscapes when initializing from the same pre-trained model. However, LMC isn't observed when initializing from random initialization. [Neyshabur et al.](https://arxiv.org/pdf/2008.11687.pdf) developed this observation to understand \"what is being transferred in transfer learning.\" They make the case that pretraining helps by allowing LMC on the target landscape and biasing the optimizer trajectories to converge to a shared basin. \n\nIn this paper, the authors take this further, and instead of just studying LMC on the target task's landscape, they also consider an out-of-domain diagnostic task. The authors show that while fine-tuned models behave well on the target task, only some of them behave well on the diagnostic task. The models which do well (generalizing models) and do not do well (heuristic models) on the diagnostic tasks form separate clusters on the target landscape. Each of these clusters is mode connected, but they are not connected to each other.  The paper then suggests a notion of \"distance\" in the model space called \"convexity-gap,\" which leads to meaningful clustering of the models in terms of their functional behavior. Finally, the paper shows that as training proceeds, different models get closer to their clusters, suggesting that early diagnostic tests of models can predict their final generalization performance. ", "summary_of_the_review": "I think the results and discussion in section three are interesting, but I have some concerns about the experimental procedure. I also don't like the writing of the paper, as the authors are not the most explicit about some important details. But most importantly, I believe the metric proposed in section four needs to be investigated further. The experiments don't depict any clear advantage of the metric over simpler measures. Overall, this reduces the novelty of the work. I encourage the authors to make the writing changes, develop more on their metric, add some extensions mentioned in section six, and then resolve future directions to improve the paper. \n\nOn an orthogonal note, I am also curious if the authors have explored the connections to neural mode collapse? ", "strength_and_weaknesses": "The paper asks an interesting question: how does the functional behavior differ between different modes split from a pre-trained model? It discusses several valid and interpretable diagnostic sets for doing this. If the experiments in the paper were performed correctly (see my apprehensions later), it would offer novel insight about LMC.\n\n\n #### **Missing important experimental details**\nMy main issue with the paper is that it is not very precise with the different experimental details. In some cases, it just leads to a non-smooth reading of the paper, but in other cases, it leads to significant confusion about what is being done in the experiments.\n\n1. How were the plots in figure 2 obtained? Are we looking at two random directions, two principal component directions, or something else altogether? One could miss high-dimensional barriers by looking at random directions; non-convex functions can look convex, etc. \n\n2. Since several data sets/tasks are involved, it is crucial to be specific about the loss landscape on which LMC is observed or not observed. I could infer which landscape is being plotted and which models are connected on which task, but only after carefully checking the paper and the appendix. This should be more clearly specified in the figure captions and should not be a matter of confusion. For reference, see section 2 in [this paper](https://arxiv.org/pdf/2008.11687.pdf), and note how non-colloquial markers are used throughout the paper, to be specific.  \n\n3. Regarding footnote 3, were different endpoints obtained using different initializations for the linear layer? This seems to be the case since the same seed is used for initialization and data permutation. If so, this is different from the usual LMC setup, where the branches are split from the same training trunk after several training steps from the same initialization. This is a crucial point, as models training from different initializations don't usually show LMC. Most importantly, **this could be why different fine-tuning runs converge to different basins**.\n\n\n#### **Convexity Gap**\n\nI am less convinced about the convexity gap metric. It is easy to look at figures and enforce one's interpretations. Based on the discussion in the paper, it is unclear if CG dominates other strategies. The clustering in Figure 18 looks comparable to CG. The authors should discuss this in the main paper. In light of this, it is clear that there are clusters as they are visible under all metrics, but it is not clear if CG is the best way to obtain clusters.  Similarly, the discussion in appendix H is critical and can't be pushed to the appendix. It is not clear to me, based on the appendix, that the authors successfully refute the skepticism that clustering is indicative of target domain performance, and the differences can be heightened by looking at diagnostic tasks. Overall, this discussion is not mature yet and requires further investigation. \n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "P2SLpJspdU", "reviewer": "Reviewer_b2ea", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper observes barriers in the loss landscape between fine-tune models which have discovered different \"generalization strategies\" in the context of text classification. In contrast, models with the same generalization strategies have no barrier when interpolating linearly between the two models. ", "summary_of_the_review": "The paper investigates an interesting phenomenon and does so in a clear and reproducible manner. While additional experiments would strengthen the investigations, I believe the paper will be of interest to the community and should be accepted.", "strength_and_weaknesses": "Strengths:\n- The phenomenon uncovered by this paper is thoroughly investigated and will be of great interest to the community.\n- The work is very well contextualized among related investigations.\n\nWeaknesses:\n- The experiments are already very thorough, so while it is not necessary I think the paper would be much stronger with the three following experiments.\n1. A lot of times in this paper the NLP world is contrasted with the vision world. However, a common technique for fine-tuning in the vision world is to first fine-tune a linear classifier, then unfreeze and fine-tune all params [1]. What happens when you first train a linear probe then use this as the common initial head when fine-tuning pairs - do you still see different generalization strategies? Relatedly, what happens when you start with a T5 or T0 so that you are starting from an already good model, instead of with a new classification head so that performance is initially not good.\n2. In vision, LMC also breaks when fine-tuning if you do so with a very high LR. What is the effect of LR in these experiments?\n3. Instead of interpolating, what happens when you ensemble the heuristic and generalized models?\n\n[1] https://arxiv.org/abs/2202.10054\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "Am8q0N6NRm7", "reviewer": "Reviewer_rtsz", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work explores the linear mode connectivity (LMC) feature of neural networks in the context of language models. The authors observe a contrast with the literature, which claims that (LMC) is nearly ubiquitous in neural networks. The authors note that the empirical findings based on these claims are biased toward the image domain and demonstrate this is not necessarily the case in natural language settings. Furthermore, the authors also find that out-of-distribution generalization is not associated with the sharpness of the minima but the density of models in a particular linearly connected basin. ", "summary_of_the_review": "Overall, this work appears to analyze LMC from the perspective of generalization on OOD data and find previously unknown connectivity patterns and energy barriers. While there are more questions created in the analysis than answers (see above), this work is a good contribution to the LMC story and why the phenomenon might not be as straightforward as previously thought.", "strength_and_weaknesses": "Strengths:\n\n- Demonstrates that the LMC phenomenon is not universal across the natural language modality.\n- Useful observations in the association of LMC with generalization\n- Clustering analysis using the CG metric appears to be novel and an alternative to sharpness.\n\nWeaknesses:\n\n- Not much investigation of the architecture dependence. For instance, Ainsworth 2022 showed that the LMC phenomenon depended on the neural network's architecture and width.\n- Unclear how pre-training initialized LMC and LMC with random initialization modulo permutations are the same minima.\n- Since the authors demonstrate a lack of connectivity on out-of-distribution data, other hyperparameters may impact results. Ainsworth et al. 2022 noted that the LMC phenomenon does not occur until training has converged to some degree. (See Figure 3 at https://arxiv.org/pdf/2209.04836.pdf). Convergence on in-distribution may not indicate convergence on out-of-distribution. How do the authors define the training criterion to ensure the out-of-distribution has sufficiently \"converged\"?\n- Doesn't the clustering analysis of the CG metric require the diagnostic dataset, whereas sharpness does not? If this is the case, the claim about sharpness being inferior seems overstated. While the overall results of the paper would indicate otherwise, has the clustering of the CG approach been tried on MNLI and QQP?\n\nComments:\n\n- Absence of evidence is not evidence of absence. The LMC hypothesis is not easily refuted empirically because a permutation search procedure's inability to find a viable permutation for LMC does not mean a permutation does not exist. This is why I do not consider it a weakness of this work, but it would be good for the authors to comment on this issue in the manuscript.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster