Input Space Mode Connectivity in Deep Neural Networks
Authors: Jakub Vrabel
Ori Shem-Ur
Yaron Oz
David Krueger
Venue: ICLR 2025 Poster (2025)
Keywords: mode connectivity
input space
deep learning
adversarial detection
interpretability
percolation theory
Primary area: interpretability and explainable AI
TL;DR: We extend the concept of loss landscape mode connectivity to the input space of deep neural networks.

Abstract: We extend the concept of loss landscape mode connectivity to the input space of deep neural networks. Mode connectivity was originally studied within parameter space, where it describes the existence of low-loss paths between different solutions (loss minimizers) obtained through gradient descent. We present theoretical and empirical evidence of its presence in the input space of deep networks, thereby highlighting the broader nature of the phenomenon. We observe that different input images with similar predictions are generally connected, and for trained models, the path tends to be simple, with only a small deviation from being a linear path. Our methodology utilizes real, interpolated, and synthetic inputs created using the input optimization technique for feature visualization. We conjecture that input space mode connectivity in high-dimensional spaces is a geometric effect that takes place even in untrained models and can be explained through percolation theory. We exploit mode connectivity to obtain new insights about adversarial examples and demonstrate its potential for adversarial detection. Additionally, we discuss applications for the interpretability of deep networks.

Published as a conference paper at ICLR 2025
INPUT SPACE MODE CONNECTIVITY IN DEEP NEURAL NETWORKS
Jakub Vrabel * CEITEC, Brno University of Technology
Ori Shem-Ur Tel Aviv University
Yaron Oz Tel Aviv University
David Krueger
Mila, University of Montreal
ABSTRACT
We extend the concept of loss landscape mode connectivity to the input space deep neural networks. Mode connectivity originally studied within parameter space, where it describes the existence of low-loss paths between different solu- tions (loss minimizers) obtained through gradient descent. We present theoretical and empirical evidence of its presence in the input space of deep networks, thereby highlighting the broader nature of the phenomenon. We observe that different in- put images with similar predictions are generally connected, and for trained mod- els, the path tends to be simple, with only a small deviation from being a linear path. Our methodology utilizes real, interpolated, and synthetic inputs created us- ing the input optimization technique for feature visualization. We conjecture that input space mode connectivity in high-dimensional spaces is geometric effect that takes place even in untrained models and can be explained through percola- tion theory. We exploit mode connectivity obtain new insights about adversarial examples and demonstrate its potential for adversarial detection. Additionally, we diseuss applications for the interpretability of deep networks.
INTRODUCTION
The high-dimensional nature of the parameter space in deep neural networks (DNNs) gives rise to the intriguing phenomenon of loss landscape mode connectivity. This concept reveals that differ- ent solutions of the model, achievable through stochastic gradient descent (SGD) training, are not isolated but rather interconnected by simple paths of low loss (Garipov et al.) 2018 Draxler et al.) 2018). This observation challenges the classical view of the loss landscape, which is traditionally perceived as containing many equivalent, discrete minima [Kawaguchi] 2016).
Connections between the modes can be as simple as linear in specific cases (Nagarajan & Kolter, 2019), while they are often quadratic or piece-wise linear in the majority of cases. Linear mode connectivity (LMC) can be associated with the consistency of mechanisms employed by models for predictions (Lubana al. 2023) (e.g., background vs. object in image classification models). Interestingly, LMC relates to the concept of lottery tickets and their stability to SGD noise after a few initial training iterations (Frankle et al. 2020).
Mode connectivity can be utilized for enhanced ensembling strategies to create more diverse ensem- bles (Garipov al. 20118). This approach can significantly boost the accuracy and generalization capabilities of models, especially in handling out-of cuttof-distribution data (Fort et al./ 22020). Further- more, parameter space mode connectivity can be employed repair backdoored or error-injected models (Zhao et al. 2020p.
The exact causes for mode connectivity within neural networks remain elusive. Architectural sym- metries (node permutation) are known contributors (Zhao et al.) 2023) but do not explain the full picture. Commonly used loss functions also significantly contribute the degeneracy of the pa-
"Correspondence to: jakub, vrabe crrabel@ceitec. vutbr.cc cz. Work partially performed during a research visit the University of Cambridge.

---

Published as a conference paper at ICLR 2025
rameter space. Our work further explores the hypothesis that mode connectivity is more general phenomenon of high dimensional geometry, partially studied in Fort & Jastrzebski (2019).
As we demonstrate, the concept of mode connectivity can be generalized to the input space of neural network. In contrast to the originally studied parameter space, where data are fixed and model (parameters) variable, we employ inverted setup (fixed model, variable data). We utilize real, virtual (interpolated), and synthetic inputs, which are obtained by input optimization (Olah) al. 2017). Considering the cross-entropy loss, we prepare and compute several examples of optimal imputs (natural images, high-frequency patterns, and adversarial examples) and show that they are mode connected in vision models for classification. Insights about the connectivity reveal the potential for adversarial detection and interpretability of DNNs. We propose that the existence of input space mode connectivity can be explained through high-dimensional percolation, which is purely geometrical phenomenon. We also note that trained networks tend to contain simple paths with only small deviations from linear paths for many real input pairs, and we speculate that this could be an aspect of the implicit regularization of deep learning.
The main motivations for studying input space mode connectivity are: 1) advancing the scientific understanding of deep learning, i.e., bridging the gap between theory and practice; 2) providing new insights into interpretability by understanding the nature of adversarial examples and poten- tial for exploring the class-optimal data manifold; and 3) enhancing adversarie robustness through adversarial detection.
Contributions
We extend the concept of mode connectivity from the original parameter space to the input space of (vision) DNNs.
We observe fundamental differences in connectivity between loss-minimizing inputs across various setups (real vs. real images, real adversarial attacks, synthetic images) that allow us to distinguish between them.
We propose a novel method for detecting adversarial examples that outperforms baselines for advanced attacks (deepfool (Moosavi-Deztoool et al. 2016) and Carlini- Carlini-Wagner (C&W)
(Carlini & Wagner 2017)).
We conjecture that arbitrary inputs with the same prediction are mode connected in ran- domly initialized neural networks. We provide a proof sketch of this conjecture, based on percolation theory (Duminil-Copin, 2017), along with empirical evidence.
RELATED WORK
Prior work of Jacobsen et al. 2019) studied the phenomenon of excessive input invariance in neural networks. This research later inspired Balasubramanian et al. (023) to explore a specific form of input space connectivity between real inputs and so-called invariance- based adversarial examples (referred to blind spots). Remarkably, these blind spots can closely resemble images from dif- ferent classes with entirely different semantics than the reference image, yet they maintain the same model prediction confidence along the linear interpolant paths the reference. In contrast, we study the connectivity between almost any pair of inputs that have similar model outputs. The major dif- ference from (Balasubramanian et al. 2023) is that invariance-based adversarial examples are (from the model's perspective) out-of-distributie inputs. Moreover, we empirically and theoretically demonstrate that (loss-based] input space mode connectivity exists in general, including untrained networks.
pathological samples, whereas we consider all possible
Interpolating pairs of real input samples of different classes was proposed as an advanced data aug- mentation procedure called mixup (Zhang et al.) (018)). Mixup was shown to improve generalization and adversarial robustness, but mode connectivity was not discussed.
Our work is directly motivated and inspired by seminal research on parameter space mode connec- tivity (Garipov et al.) 2018 Draxler et al., 2018). We leverage various established methods and analytical tools to explore connectivity in the input space. Notably, we adapted a simplified algo- rithm to bypass loss barriers from Fort & Jastrzebski (2019). In the same work, the authors proposed phenomenological model of the high-dimensional loss landscape that allows analysis of the train- ing dynamics and mode connectivity. Yunis et al.) (2022) identified a high- nigh-dimensional convex hull of low loss between multiple loss minimizers, which is relevant the discussion about the optimal
2

---

Published as a conference paper at ICLR 2025
imput manifold. Simsek et al. (2021) show that originally discrete minima can be connected by adding a single neuron to each layer, highlighting the significance of space dimensionality and generality of the phenomenon.
We also build on the feature visualization by optimization technique OOla et al. 2017 to create optimal inputs and prepare adversarial attacks. For the discussion about adversarial detection, follow and adapt the benchmark from Harder et al. 20021)
Loss minimizer&
Intermediate point B crozs cntrooyyyyy....o
Loss minimizer cross entropy: O
cross entropy:
Updated Intermediate point cross entropy 1. 1.77-06
Cross-entrapy Low
Normalized di fference B-B
0.10
0.00
0.02
05
15
interpolntion conffiiincto
Figure 1: Barrier between two input modes. Two examples (A and C) that minimize loss for the class jackfruit shown. The interpolated point B exhibits maximal loss. Left bottom: Cross-entropy loss for interpolated virtual points between A and C; a diagram illustrating the interpolation paths and subsequent optimization of point B, constrained within the orthogonal hyperplane vector AC. Middle bottom: The optimized point B' minimizes the loss and is nearly indistinguishable from the original point Right bottom: Normalized difference pattern B'-B, where the maximum pixel value of the difference pattern is 1-5% of the maximum pixel in the original image.
3 METHODOLOGY
3.1 MODE CONNECTIVITY (FROM PARAMETER TO INPUT SPACE)
We define a parameter space mode as a set parameters 0 that minimizes the loss L (e.g.. cross- entropy for classification task) for given dataset. That is
= argmin C(f(D;
(I)
where f(: is a neural network output, and D dataset that contains both data samples X and ground truth y. Loss minimizers in the parameter space are usually obtained through SGD, with diversity arising simply from using different initializations and/or exploiting other sources of randomness such as the sequence in which examples are presented to the model during SGD-based training (i.c., SGD noise).
Analogically, the input space mode for selected class ys is single data example
I; argmin L( (x; 0), y.)..
where in contrast to the parameter space, the model is fixed (pretrained). Input modes could be either real data examples with loss below a certain small threshold 8 or obtained through an optimization process (see Section 3.2
3

---

Published as a conference paper at ICLR 2025
After observing two (or potentially more) modes Z and ze, we linearly interpolate them create virtual points (a) as
x(ax) = ox; + a))
where a E [0, 1] is uniformly sampled. In general, a loss barrier exists between the two modes (Fig. Ieft bottom). The barrier can be bypassed via optimization of the highest-loss point TB. which results in modes connected by simple (partially linear) paths of low loss (as described in Section 4.1). The optimization follows the procedure described in Section 3.2 In the following sections, we use a simplified notation A A for points/images, and A-c denotes a path between A and C.
3.2 FEATURE VISUALIZATION BY OPTIMIZATION (FVO)
FVO is technique for post-hoc interpretability of DNNs (Olah et al. 2017). It starts from small noise 24 at the input of a network that forward propagated to the layer of interest. We aim to find imputs that activate selected neuron (or neurons) within the layer of interest. Neurons can be either maximized or activated to a specific pattern through a loss function. The input is iteratively updated w.r.t. the gradient of the loss, similarly to the standard gradient descent training. Note that this optimization process does not produce an unique result, and so multiple distinct examples can be generated and their (input space) mode connectivity studied (see App. AA.
The optimization procedure can be easily adapted for real images. We start from the maximal loss intermediate point В and optimize until reach B', which minimizes the loss (see left bottom diagram in FT. The optimization is constrained within the orthogonal hyperplane to the vector AC, representing a simplified method for connecting modes inspired by Fort & Jastrzebski (2019).
EXPERIMENTS
In this section, we provide empirical evidence for the input space mode connectivity in pretrained vision models and discuss potential applications of the phenomenon. First, show how the two modes can be connected through the optimization of the loss barrier. Second, we study and exploit the connectivity between adversarial attacks and natural images to create algorithm for adversarial detection. Third, we show that the connectivity exists for untrained (randomly initialized) networks, which validates our theoretical result in Section 5 Last, (in Appendix A we discuss the potential for interpretability of DNNs.
We used cross-entropy loss for all experiments unless explicitly stated otherwise. Experiments have minimal compute requirements and were carried out on a regular PC (equipped with GTX 1650 4GB) or cloud services (T4 16GB). Sections 4.2 and Appendix (A Jse GoogLeNe((Imageee))
subsection 4.2.1 uses VGG-16(CIFAR10/100))) section 4.3 uses ResNet18.
4.1 INPUT SPACE CONNECTIVITY
Vision models trained for classification tasks provide an ideal setup to observe input space mode connectivity. To demonstrate this phenomenon in a realistic and practically relevant setting, we used Googl LeNet (Inception vI) (Szegedy et al. 2014) that was pretrained on the ImageNet (Russakovsky al. 2015) and is available from the Py Torch torchvision library (Paskke et al.) 2019). First, we selected representative low-loss examples from the validation dataset, which have cross-entropy loss in the order of le-4 or lower. These examples were linearly interpolated with 1,000 uniformly dis- tributed points. After identifying the interpolated point of maximum loss B, we started a constrained optimization process, encompassing 1024 iterations, which resulted in a good loss minimizer B' (Fig. [] The Adam optimizer (Kingma & Ba 2017) was used with learning rate of 0.005. We added regularization to the optimization with two additional terms: deviation penalty PASE between B and B and high-freed fregurreyeency term Par that penalizes changes in adjacent pixels (see definitions in App. B). The weights for these terms, AMSE 0.1 for the deviation, and Ant ranging from le
to 5e 6 for the high-frequency term, were determined heuristically, varying by class and setup.
4

---

Published as a conference paper at ICLR 2025
created by merging the linear interpolations between A and B', and between B' and C. We sampled 500 intermediate points equally from both segments the new path, disregarding the fact that the barrier typically does not lic exactly at the midpoint of A-*C. The A-B''>e path exhibits low loss while connecting two input modes (Fig. 2 Although small secondary barriers are still present in the A- -C path, we neglect them far they below a certain threshold (8 0.001 considered here). By repeating the optimization process on new segments independently, we can easily bypass secondary barriers. More examples and qualitatively consistent results from Vision Transformer (trained on ImageNet), and ResNet18 18 (He et al. 2015),
A new path
MLP, and CNN (trained on CIFAR-10 10 (Krizhevsky 2009)) are shown in the Appendix G
Imggs
cano-ontirsp
Crpo-srtropy Kos
30009
Crosa-entiog Ioso
0.08
0.22
0.06
0.0
30099
0.44
0.02
040
30005
0.00
30099
0.2
0.4
0.6
0.8
1.0
03
Interporation comfficient (o)
Interpolation confficient (Oar)
Interpolation confficiett Ittee
Figure 2: Bypassing the barrier. After optimizing the intermediate point B, we obtained the updated point B', which linearly connects both and C. Note that small secondary barriers, below the threshold of 5 0.001 loss, are disregarded.
along the two studied paths closely resemble each other in corresponding pairs, as shown in Figure 3 In fact, their difference is the identical pattern depicted in the right bottom of Figure scaled in intensity proportionally to the interpolation factor a. This likely due to the process we use find B', which resembles the way gradient descent is used to find adversarial examples, the differences being that we restrict the search to a hyperplane and do require to remain close to
Figure 3: Images along two paths. We sampled and compared images along the updated path A-+B'>C (dashed rectangle) and the original linear path A-C (bottom row). These paths are illustrated in the diagram in Figure [ The high-loss intermediate point B and its optimized version B' are denoted by the blue vertical rectangle.
4.2 ADVERSARIAL ATTACKS
In this section, we demonstrate fundamental difference in the connectivity between real-real and real-adversarial image pairs. We subsequently leverage this insight for adversarial detection.
Adversarial examples in vision models are images that are generated by perturbing source image way that either minimizes the loss for selected target class (in targeted attacks) or maximizes the loss for the true class (in untargeted attacks), causing the model to misclassify the image. The source image and the optimized adversarial example are often almost indistinguishable to a human observer. The process of generating and employing adversarial images to deceive the model is known as an adversarial attack (see Fig. 4
We find that adversarial examples, even when significantly different in terms of human visual per- ception, are mode-connected to "true" real images (correct class minimizers). Using ImageNet and GoogLeNet, picked low loss example from selected class (here golf ball) the mode A.
5

---

Published as a conference paper at ICLR 2025
Then, source image K from different class (revolver) was optimized obtain adversarial example K' (K' minimizes the same class as the A). We used Adam (learning rate 0.005), 1024 iterations, cross-entropy loss with penalizations (AMSE 0.1 for image deviation term PuSE and Ant = le- 7 for high-freq. penalty Pur. see definitions in App. B). From here, the mode connecting procedure is identical to the previously described: 1) Detect the maximal loss point B, 2) Optimize B to get 3) Interpolate over the new path A-B' -K. However, after a single round of this iterative procedure, non-nelligible secondary barriers emerge in paths A A-B and B' B', which exceed the loss threshold 5 by a considerable margin (as depicted in Fig. [). Secondary barriers can be suc- cessfully bypassed by repeated application of the procedure to both segments of the A-+B'+ path individually. As result, the final low-loss path will necessarily be more complex than for real-real modes addressed Section I. Our hypothesis is that adversarial examples "live" closer to the boundary of the class-optimal manifold (defined in App. due to their origin in images of different class, and therefore have weaker connectivity to real examples of the predicted class.
LOGS PTPIIOAAA A
Source Irage
Adversata soample C
XXC
XXX
Figure 4: Adversarial attack. From the left: i) real input A that minimizes the loss for class golf ball, ii) source input K from different class, iii) optimized pattern added to the K, and iv) adversarial example K' that minimizes the loss for the same class as A. The network predicts K" as a golf ball.
Intermediste cross-ntiagyy 4.820
entropy
B
15.0
12.5
8
5.0
02
3.0
0.2 0.8
1.5
colficectil
1.012
XXX
0.28
11:100
10.000
0.22
03
92
0.6
0.6 1.0
2.0
0.2
2.4
06
0.1 reycl
3.0
Figure 5: Barriers and path complexity. Left: Maximal loss point B the primary barrier (between the real mode A and adversarial attack K'). Right: Interpolated paths between points from Figure 4 The primary barrier was bypassed through the optimized point B'. Secondary barriers (with lower loss) emerged on the new path A-B' K and can be further bypassed by repeated application of the procedure to both segments of the new path (omitted here). This suggests that the path between a real mode A and an adversarial attack K' is more complex than that between two real images.
In addition to the complexity of the paths, we observed significant differences in the barrier heights and shapes between two key scenarios: barriers between real-real modes and real-adversarial modes. To assess the statistical significance of this effect, we conducted the following experiment. Seven unique pairs of images per class were selected from the validation subset of ImageNet, en- compassing 1,000 classes in total. Two pairs from each class, having the highest loss difference between the inputs, were excluded. The remaining 5,000 pairs were interpolated with 250 steps in between, and loss curves were generated. In the adversarial branch, pairs were formed by combining one example from the selected class with one from randomly chosen different class. Adversarial examples were created in the same way as described above with the following settings: learning rate 0.005, AMSE = 0.1, Ant = le 8, and 512 iterations. The subsequent procedure mirrored the one described earlier, yielding another set of 5,000 filtered pairs (5 per class). Descriptive statistics for the maximum barrier height and the gap (the difference between the peak height and the loss value of the higher mode/image) are depicted in Figure

---

Published as a conference paper at ICLR 2025
Real-adversarial mode pairs exhibit significantly higher barrier max, with mean and median 5.82 and 5.99, respectively. In contrast, real-real pairs have mean 2.27 and median 1.79. The effect even more pronounced for barrier gaps, where real-adversarial have mean 5.19 and median 5.30, but real-real have mean 1.00 and median only 0.47. The described be- havior was utilized to design a new method for ad- versarial detection, which relies solely on the shape of the loss curves on linear interpolant paths.
Barrier height statistics
37.5
Adversarial attack
Within-class input
23.0
12.5-
Loss value
4.2.1 ADVERSARIAL DETECTION
0.-
Barrier max
Bonter
Enhancing the adversarial robustness of models essential across various domains, particularly in safety-critical applications (Pang et al. 2018). How- ever, the majority of defenses (e.g., adversarial train- ing) are not sufficient against advanced attacks and can be easily fooled by adaptive attacks (Tramer et al. 20200. Instead of maintaining model accuracy against adversarial examples, an alternative is to de- tect attacks and invalidate predictions for such inputs (Han et al. 2023 Harder al. (02T).
Figure 6: Barrier height statistics. Box- plots depict the maximum barrier heights and gaps (i.c., the difference between the bar- rier height and loss value of the higher-loss mode) for selected pairs from the ImageNet validation dataset. The barriers for real-real and real-adversarial example pairs are com- pared. A total of 5,000 representative pairs (5 per class) were utilized for each scenario.
We propose a simple algorithm for adversarial detection that leverages the relative lack of linear connectivity in natural-ad adverssrall image pairs (as shown in Figure First, we compute the (linear- path) loss curves for all training images T,. Each loss curve is obtained from an input pair consisting of Ie (a training image) and Tiemplate.y, (a low-loss reference image from the same class, yi = argmax (x,, f)), which is preselected for each class. Second, concatenate the loss curves with logits f( 0), sorted in descending order, which provide information about prediction confidence. The second step slightly enhances performance against simpler attacks, though it can be fully ablated for attacks like deepfool and C&W). Third, resulting feature vectors (loss curves+logits) are then used train a classifier. Test images are processed through the same pipeline and the resulting loss curves are classified by the model.
We implemented the algorithm an existing benchmark (Harder et 2021 that uses VGG-16 (Simonyan & Zisserman 2015) model and CIFAR datasets. The benchmark employs rep- resentative selection of adversarial attacks (FGSM (Goodfellow al., 2015). BIM (Kurakin al.) 2017). PGD (Madry et al.) 2019). deepfool (Moosavi-Dezfooli et al. 2016), and C&W (Carlini Wagner 2017)) all their untargeted forms. Interestingly, our method outperformed baselines for more advanced attacks (deepfool and the particularly challenging C&W) but underperformed on simpler (FGSM, BIM, PGD). This is a consequence of the nature of untargeted attacks, which generally aim alter the prediction by maximizing the loss for the correct class. However, unlike FGSM, BIM, and PGD, both deepfool and Carlini & Wagner (C&W) also minimize the loss for a specific incorrect class, making the misclassification more controlled. Our algorithm relies the existence of a distinct loss barrier, which is less pronounced in simpler attacks but more defined in advanced ones like deepfool and C&W. Complete results are provided in Table Note that we used a classifier (KNN) that was optimized jointly for all attacks on validation data (in contrast to attack-depended classifiers and hyperparameters in the benchmark). Additionally, unlike the best- performing baseline, our method does not require access to the feature maps.
Note that the proposed detection method could fail against adaptive attacks, as common for the majority of current defenses (Bryniarski et al. 2021). We did not test this scenario due to the non- differentiability of our current version of the detection algorithm and will address it in future work.
4.3 THE EXISTENCE OF CONNECTIVITY IN UNTRAINED MODELS
Finally, we show that modes, represented by class-optimal inputs, are connected even in the case of untrained, randomly initialized models. This experiment was inspired by findings in Section 3 but present it here, preceding the section, to maintain the consistency of the manuscript.

---

Published as a conference paper at ICLR 2025
Table I: (adapted from (Harder et al. ()2T)) Comparison of detection methods. Accuracy/AUC (%).
The attacks are applied on the CIFAR- CIFAR-I0/100 test set and the VGG-16 NET. Attacks are described in Appendix D
Dataser
Deeptool
CIFAR-10
LID
86.47003
85.6/93.3
80.4790.0
18.1785.3
Mahalanobis
95.67 98.8
97.3/99.3
96.07 98.6
76.1 84.6
76.9 84.6
InputMFS
98.1 79.7
93.5/97.
58.0/600. 93.6/97.
54.7 56.1
LayerMPS
100
/
LayerPFS
97.0/ 99.9
98.0/99.9
99.9
86.1/92.2
83,8/93.3
connectivity (ours)
69.8
95.2/98.6
91.8/97.4
91.4//6.9
CIFRR 100
LID
72.9/811
85.5
79.07 86.9
61,8/67.2 64.9
Mahalanebis
90.5 96.3
73.5/81.3
76.3 82.1
89.2/95.3
89.0/941 94.7
InputMFS
98.4 95.5
89.1/94.1
90.9/95.1
53.3 54.6
LayerMPS
99.5 100
97.1/99.5
97.0 99.7
83.8/9100
87.1/93.0
LayerPFS
96.9 99.3
90.3/96.7
99.6/97.6
78.8 18,8
79.1 84.0
Mode connectivity (ours)
67.3 72.8
94.3
88.97 /5.2
86.8794. 94.0
92.9/97.3
As there are no natural datasets for untrained models, we computed class-optimal synthetic inputs using the FVO (see Section f.2 definition, and Appendix [A for analogical results from the FVO on a trained model). The optimization procedure failed to converge GoogLeNet, likely due to the high input dimension (3, 224, 224). Using the ResNet18, adapted for CIFAR 10 (input shape (3, 32, 32), output (10)), starting from Gaussian noise (0, 0.01), we were able to find inputs with losses below the specified threshold 0.0005. Adam optimizer with learning rate 0.05 and weight decay le-7 used for maximum of 4096 iterations until reaching the desired loss. A diversity of inputs was achieved through high-frequency penalization with a weight Aar = 2.5e - 7 of one the inputs in each pair.
The two modes were linearly interpolated, and the loss curve is shown in Fig. 7 left. After two round of the standard mode-connecting procedure, obtained four-segment piece-wise linear -CC that has loss below 5 = 0.001 everywhere. Note that points B's and B2 were obtained by optimizing the secondary barriers Bş1 and B All the loss curves and barriers are depicted in Fig. 7 In Appendix F.T we also show how the linear connectivity (primary loss curve, i.e., loss on the linear interpolant path) changes during the training.
path. A A-BB's1 -BB's--B'' >B's2 -
Primary
Secondary Curves
Terthary Less Curves
0:10
0.004
Sentrry
0.0009
0.08-
0.0008
$ 0.05
0.003
0.002
0.0008
0.001
0.0005
0.00
0.4
0.0
0.7
0.6
0.A
Primary Interpolation confiicient
Primary Interpolation coefiicient
Primary Inter anlaoon confficient
Figure 7: Input connectivity untrained model. Loss curves for synthetic optimal inputs computed for an untrained model. Left: Primary loss curve, connecting the two optimal inputs. Middle: Secondary loss curves, connecting two original inputs through the intermediate point B' (optimized from the primary barrier). Right: Tertiary loss curves, four-segment path connecting two original inputs through intermediate points B's B', and B'sz (optimized first secondary barrier, optimized primary barrier, optimized second secondary barrier. respectively).
TOWARDS A THEORETICAL EXPLANATION OF INPUT SPACE CONNECTIVITY
This section establishes the theoretical foundations underlying the phenomenon of input space mode connectivity. We begin by introducing some definitions.
We denote our input and output spaces as = Rd and = Bd, {u e Rd respectively. A path connecting x00) E
| 1 S Rd,
x(1) E X is defined as a continuous function
'The first round optimizes the harrier the primary linear path, the second round optimizes secondary barriers the two linear segments and results in four-segment path.
8

---

Published as a conference paper at ICLR 2025
[0,1] X. We define network that maps the input space the output space as any function within the framework of the tensor program formalism Yang (0099 2020): Yang & Littwin (202T), featuring Lipschitz -bouude point wise nonlineartties. Specifically, for parameter vec- tor 0, such network accepts inputs and returns outputs:
f(: X -YY.
(4)
Given any 8 > O, network f. parameter vector 0, and a loss function L: Y Y - R+, we say that path 2x-) is d-connected around Y for this loss function if and only if:
Va E [0. 1] (([(((();; 0), s
(5)
with arbitrarily large probability.
A detailed description of our network class, and the requirements from the loss function is provided in appendix E.1 Further assumptions and limitations are discussed in appendix H.1
5.1 GEOMETRIC MODE CONNECTIVITY
We present the concept of geometric mode connectivity, suggesting that almost all inputs on which neural network makes similar predictions tend to be connected, dx grows infinity. We find this is the case empirically for both trained and untrained networks. We believe the latter finding can be proven as a consequence of high-dimensional geometry, and formalize it in the conjecture:
Conjecture 5.1 (Geometric Input Space Connectivity).
Given a subset of the input space X' C X, a network f(; (): at initialization, and a loss function C. whose specifications are provided in Appendix [E.]] the following holds: For arbitrarily small O < 8' < 6, any two inputs go, E X', selected independently of 0 and with similar predictions are almost always connected as dx + 00:
(2o, Rn are 8-connected & (J (ro; 0), (z1; 58 -1
(6)
where 8 O (6), and zo. x1,0 are chosen randomly as described in appendix E.1
We justify the conjecture for L y') = - ||y- and the case where the network's outputs are numbers between zero and one, f(::6) : X - [0, 1]. The generalization to more general cases is straightforward.
We include a detailed proof sketch for this conjecture in appendix E which we believe provides significant insight into the phenomenon of mode connectivity but note that it is not fully complete and rigorous proof:
(i First, in appendix the Lipschitz continuity of the network's activation functions, and its semi-linear structure. Using this property, we show that for every 0 < 5 (assuming for simplicity that 1 E N), there exists 0 < s. such that if we divide the input space into E-sized hypercubes, then for every cube inside the lattice, all the points' outputs (f (I; 8)) belong to one of the following overlapping intervals:
demonstrate that the network is Lipschitz continuous, by leveraging
Y88 = 10,
35 2 2
[0, 8),
16, 26.2))...... 6.1]
6,
We can then label each input by the interval (or intervals), that corresponds to the output of all the inputs in the cube. We show that if there exists sufficiently small then for any two points To. I1 E X' that satisfy the conjecture's 's condition (C (f (my; .f (x11 9)) I 8'), their respective cubes share the same interval. This is chosen the length of our graph hypercubes.
(ii) Next, we demonstrate in appendix E.3 that to find 8-connected path between the two inputs, it suffices to find a connected path of cubes between the cube of the first input and the cube of the second, with the same lnter.
Assuming the cube's 's intervals are drawn randomly and independently from each other with similar probability p ze ö, we encounter a classic high-dimensional percolation problem Duuminll Copin 2017): Meester & Roy (199),, where the cubes represent the points in the graph, and they

---

Published as a conference paper at ICLR 2025
connected if they share the same interval. It is well known that in such case, the probability that two points are connected grows rapidly with the dimension, as shown in the conjecture 5.1
It is important note that the assumption of independent probability is not realistic, there strong correlations among nearby inputs of wide neural networks, even at initialization. This is why we decided not consider statement [5. as theorem, but rather conjecture. However, we expect that in general, assuming independence only underestimates the true connectivity, as the correlation between outputs of nearby inputs in neural networks at initialization tends to be positive.
5.2 PATH SIMPLICITY BIAS
As discussed above, justified conjecture 5.1 only for neural networks at initialization, which raised the question, whether this property persists after training. In practice, we observed that not only do the inputs tend to remain connected, but they also tend to be connected approximately linearly, with only a relatively small barrier. Furthermore, by combining only a few linear paths, almost completely eliminate this barrier. This tendency is strong enough to be useful in adversarial detection, as we shown in section 4.2 implicil regularization.
We speculate that this phenomenon results from
Implicit regularization refers to an inherent tendency of learning system to prefer simpler hypoth- esis functions Soudry et al. (2018); Neyshabur (2017); Barrett & Dherin (2020); Gidel et al. (2019); Belkin et al.) 2018); Jacot et al. (2018) Lee et al. (2019); Mulayoff et al. (2021). We propose that one kind of simplicity preferred by deep learning is generally not to change between similar inputs if possible. This tendency, combined with geometric considerations, could explain the observed
behavior. In appendix F we present some ideas on the mechanisms responsible for this tendency; however, more work is needed to substantiate this.
CONCLUSION AND FUTURE WORK
We demonstrated mode connectivity in the input space of deep networks trained for image classifi- cation, extending the original concept beyond parameter space. This phenomenon was shown with both real and synthetic images various controlled setups. For tested examples, we were always able to find low-loss paths between any two modes. We explored potential applications of this nectivity, particularly for the adversarial detection and interpretability of DNNs. Our findings reveal that natural inputs can be distinguished from adversarial attacks by the height of the loss barrier the linear interpolant path between modes. While for real-real mode pairs, the loss barrier is small even negligible, avvesarvesllial pairs often have high and complex barriers. By exploring con- nections between different types of class-optimal inputs we obtained a novel perspective on DNN interpretability. The new evidence for mode connectivity beyond the parameter space supports the hypothesis that mode connectivity is an intrinsic property of high-dimensional geometry and can be studied through percolation theory. This hypothesis, however, requires additional investigation to fully understand its implications.
Future work could formalize the theory behind our conjecture of geometric mode connectivity and identify minimal requirements for its manifestation. Another direction is to develop a theory and conduct experiments to better understand implicit regularization the input space. Practical appli- cations include optimizing the proposed naive adversarial detection algorithm and leveraging input space connectivity insights for more effective adversarial training, e.g., sampling adversarial exam- ples along low-loss paths for model retraining, analogous to parameter space connectivity in deep ensembles (Fort et al. 2020). Investigating the geometric properties of the input space loss landscape and exploring the motivations ford may also enhance model interpretability.
ACKNOWLEDGMENTS
J.V. acknowledges the support of the Technology Agency of the Czech Republic under project FWI 1020175. The work of Y.O. is supported in part by Israeli Science Foundation excellence center, the US-Israel Binational Science Foundation, and the Israel Ministry of Science.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Sriram Balasubramanian, Gaurang Sriramanan, Vinu Sankar Sadasivan, and Soheil Feizi. Exploring geometry of blind spots in vision models. In Teirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: http:///ppereeeee.ee wwweerreeie..eetfrrriiee wtt.oorrmmmmmuuuuho.oo 1
David GT Barrett and Benoit Dherin. arXiv:2009. / /62, 2020.
Implicit gradient regularization.
arXiv preprint
Mikhail Belkin, Siyuan Ma, and Soumik Mandal. To understand deep learning we need to under- stand kernel learning. In International Conference on Machine Learning, pp. 541-549. PMLR, 2018.
Jean Bricmont, Joel Lebowitz, and Christian Maes. Percolation in strongly correlated systems: the massless gaussian field. Journal of statistical physics, 48: 1249-1268, 1987.
Oliver Bryniarski, Nabeel Hingun, Pedro Pachuca, Vincent Wang, and Nicholas Carlini. Evading adversarial example detection defenses with orthogonal projected gradient descent, 2021. URL
https://a //rxiv. org/ alp//////2155 15023
A. L. R. Bug, S. A. Safran, Gray S. Grest, and Itzhak Webman. Do interactions raise or lower percolation threshold? Phys. Rev. Lett, 55:1896-1899, Oct 1985. doi: 10.1 10.1 1896. URL http::
aps.
55.16
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017
IEEE Symposium Security and Privacy (SP), pp. 39-57 57, 2017. doi: 10.1109/SP.017.99
Christopher Chalhoub, Alexander Drewitz, Alexis Prévost, and Pierre-François Rodriguez. Univer- sality classes for percolation models with long-range correlations, 2024.
Xiangning Chen, Cho-Jui Hsich, and Boqing Gong. When vision transformers outperform resnets without pre-training or strong data augmentations, 2022. URL www.ttts://axxiv.or/////
01548
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1309-1318. PMLR, 10-15 Jul 2018. URL ctttps:/pproceedine
mlr.
Hugo Duminil-Copin. Sixty years of percolation, 2017.
Stanislav Fort and Stanislaw Jastrzebski. Large scale structure of neural network loss land- scapes. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpesr/ppooceed.....aaaoos..... file/ F111///002..aa000000222222222222
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanann Deep ensembles: A loss landscape per- spective, 2020. httpppps///a..vvooo///////////// 02757. pdf
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In Hal Daumé III and Aarti Singh (eds.), Pro- ceedings of the 37th International Conference Machine Learning, volume 119 of Proceed- ings of Machine Learning Research, pp. 3259-3269. PMLR, 13-18 Jul 2020. URL https: //proceedings.oo .m wttp:p/////////ralleeee.e/
/
html
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wil- son. Loss surfaces, mode connectivity, and fast ensembling of dnns. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL ht httpsssss//pppoeennnn.eeeeeiiiie fi Fiee0000005717700000000444666333

---

Published as a conference paper at ICLR 2025
Gauthier Gidel, Francis Bach, and Simon Lacoste-Julient Implicit regularization of discrete gradient dynamics in linear neural networks. Advances in Neural Information Processing Systems, 32, 2019.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015. URL htttp://aaxxx wwt.po/gt////oooo..oooooot..e.tt 6572. pdf
Sicong Han, Chenhao Lin, Chao Shen, Qian Wang, and Xiaohong Guan. Interpreting adversarial examples in deep learning: A review. ACM Comput. Surv., 55( jul 2023. ISSN 0360-0300.
doi: 10. 14//5/949999 URL httthttt:://dd http:p::///oii.rr/1//./. wwt.
Paula Harder, Franz-Josef Pfreundt, Margret Keuper, and Janis Keuper. Spectraldefense: Detecting adversarial attacks on cnns in the fourier domain. In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021. doi: 10.1 109/UCNNN5238..201199999
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CoRR, abs/ 15100155,, 2015. htt.:/ttt::///xxiiv
033.5
Joern-Henrik Jacohsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. In International Conference on Learning Representattons, 2019.
URL https::/gpenrevieww htt.::ee////ooo forum id BkfbpsåcF
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernl: Convergence and gen-
cralization in neural networks. Advances in neural information processing systems, 31, 2018.
Kenji Kawaguchi. Deep learning without poor local minima. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https::/ppoceeednngggg neur.ps
f
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL
https://arxiv: org/ pdtf//df//// 1412. 6980.pdl
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world, 2017.
Jachoon Lee. Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl- Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019.
Ekdeep Singh Lubana, Eric J Bigelow, Robert Dick, David Krueger, and Hidenori Tanaka. Mechanistic mode connectivity. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 22965-23004. PMLR, 23-29 Jul 2023. URL http::://ppoccedngg......
htmll
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, 2019.
Ronald Meester and Rahul Roy. Continuum percolation, volume 119. Cambridge University Press, 1996.
Seyed-Mohsen Moosavi-Dezfoo Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2574-2582, 2016. doi: 00.119//VPPR.221..288.
Rotem Mulayoff, Tomer Michaeli, and Daniel Soudry. The implicit bias of minima stability: A view from function space. Advances in Neural Information Processing Systems, 34:17749-1776 2021.
12

---

Published as a conference paper at ICLR 2025
Vaishnavh Nagarajan and J. Zico Kolter. Uniform convergence may be unable to explain generaliza- tion in deep learning. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-But E. Fox, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. tt http:htttpp.pppooeeeedepeeeeeeee ngs nww.ggs..eurriee ccpppppprrcppppppepppppp les/ http://20011111116/6666622222333
Behnam Neyshabur. Implicit regularization in deep learning. arXiv preprint 2017.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Dis- till, 2017. doi: 10.2391 Seature-visual liaaliiziion
URL at.ps://dist.11.pbb//
Tianyu Pang. Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversar- ial examples. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedinggg itwp://eeeeddnnn..eeee......... http:///w....aacccooo..eeaaea633
files/
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary VeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buu Buc, E. Fox, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL httptttss:
new.reneerrggo cc/paper files/ Paw.r/00--ppppppp.....
Fierre-François Rodriguez and Alain-Sol Sznitman. Phase transition and level-set percolation for the gaussian free field. Communications in Mathematical Physics, 320(2)5571 601, December 2012. ISSN 1432-0916. doi: 10.10077/400220-01-166999 URL http:ttpp:/// http:://xxddiii.ooo.oo..o httpp::p///xx..o..o////....
1007 1007/5000 1007/=00220-000 012 1649-y
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV,, (1TT)),115333 115222211-252 2015. doi: 10.1007/111223-015-000
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, 2015.
Berfin Simsek, François Ged, Arthur Jacot, Francesco Spadaro, Clement Hongler, Wulfram Gerst- ner, and Johanni Brea. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learn- ing Research, pp. 9722 9732, PMLR, 18 24 Jul 2021. URL http:tttt:///poeeeeee .mw.iiinn..mm.......mmmm.......m mmlr. press/vi 39/ wtt.://sssss222mmm
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The im- plicit bias of gradient descent on separable data. Journal of Machine Learning Research, 19(70): 1-57, 2018.
Dietrich Stauffer and Ammon Aharony. Introduction to percolation theory. Taylor & Francis, 2018.
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers, 2022.
URL
htRl::ttt:://axxx http:///axxvvvvooo httpo//aab///0.....eg 10270
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions, 2014. Utt.https://aaat wwtp:/iiwvogg/df//////// 409. 4849.4884.55505 pdf.
13

---

Published as a conference paper at ICLR 2025
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1633-1645. Cur- ran Associates, Inc., 2020.
wgwp:/occeddng.g......o. neur ps. wwtpppppwwppppppppppppppppppeeee files/ (tt.://000...111111eeeeeee666010 2rr..-02222338010......3333881r0
Greg Yang. Wide feedforward recurrent neural networks of any architecture are gaussian pro- cesses. Advances in Neural Information Processing Systems, 32, 2019.
Greg Yang. Tensor programs ii: Neural tangent kernel for any architecture. arXiv preprint arXiv:2006. 14548, 2020.
Greg Yang and Etai Littwin. Tensor programs iib: Architectural universality of neural tangent kernel training dynamics. In International Conference on Machine Learning, pp. 11762-11772. PMLR, 2021.
David Yunis, Kumar Kshitij Patel, Pedro Henrique Pamplona Savarese, Gal Vardi, Jonathan Frankle, Matthew Walter, Karen Livescu, and Michael Maire. On convexity and linear mode connectivity in neural networks. In OPT 2022: Optimization for Machine Learning (NeurIPS 2022 Workshop), 2022. URL
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empiri- risk minimization, 2018.
Bo Zhao, Nima Dehmamy, Robin Walters, and Rose Yu. Understanding mode connectivity via parameter space symmetry. In UniReps: the First Workshop on Unifying Representations in Neural Models, 2023. URL https: https::/opeezeviiesse wwt. fott:/oorummccca222111111.
Pu Zhao, Pu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridging mode connectivity in loss landscapes and adversarial robustness. In International Conference on Learn- ing Representations, 2020. URL/attpps:/opeerriiiee net. Fo
A TOWARDS OPTIMAL INPUT MANIFOLD
Here, we apply input space mode connectivity for the interpretability of DNNs. Synthetic images offer controlled environment for examining input space mode connectivity. Here, images were generated using the feature visualization by optimization technique on GoogLeNet trained on Im- ageNet. Starting from Gaussian noise (0, 1). we optimized surrogate objective function (dot product X sqrt of cosine similarity x 1/2). instead of directly optimizing for cross-entropy. We used this heuristic approach generate synthetic inputs that better match human visual perception, as cross cross-entropy optimization, for unclear reasons, is less effective for this purpose. Through var- ied regularizations and transformations during optimization, we obtained two distinct modes, each exhibiting virtually zero cross-entrope loss, as illustrated in Figure g Mode A exhibits partial re- semblance to natural objects its class (golf ball). Mode contains only high-frequency patterns or noise, lacking obvious semantic structure, and was chosen intentionally for this purpose. The bottom row of Figure g demonstrates that even such different types of synthetic modes can be con- nected. Moreover, since the modes exhibited effectively zero loss, there no secondary barriers after optimizing the maximal loss intermediate point B (almost up to the numerical precision).
It is evident that continuum of optimal inputs exists for given class within the model. By em- ploying a systematic approach to find and connect these optimal modes, the class-optimal man- ifold can be partially explored and analyzed. Formally,
define the class-optimal manifold as Me.5 {x E X' [()a(z)), ye) < 6}. This approach would provide a novel perspective on DNN interpretability, especially as extends the concept of feature visualization by optimization Olah et al. 2017), beyond discrete optimal inputs or activation atlases.
B FURHTER DETAILS ON THE OPTIMIZATION PROCEDURE
In this section, we define the regularization terms used in the optimization procedure described and used in Section E The optimization employs the Adam optimizer the cross-entropy loss with
14

---

Published as a conference paper at ICLR 2025
Loss minimizer
Intermedate point B
Updated intermediane point cross entrapy: 3.50e-37
Loss minimizer cone-ertrapy 0.0000
0000
Crass-ntropy loss
Crosaeentruyy laus
Ciss-antropy loss
0006
2.0
2.0000
B'
1.3
0:03
2.0000
1.0
0.0
porpodation conficant Rayl
0.5
0.0
A
8'
3.0000
0.0
0.2
0.4
0.6
0.0
1.0
1.0 3,0000
Interpolation coefficien:
0.0
0.2
poopoddeo confiee Hard
1.0
Figure 8: Synthetic modes. Synthetic images generated through input optimization from Gaussian noise. Top row: Modes A and C, interpolated high-loss point B and its optimized counterpart B Bottom row: The primary loss barrier was bypassed by a single round of optimization. The new path segments A-+B' and BC are barrier-free up several orders lower loss threshold.
two additional penalty terms: the deviation penalty PUSE and the high-frequency penalty Pur. with corresponding strengths AMSE and Aar. respectively.
The deviation penalty term is the Mean Squared Error (MSE) between the starting point (input tensor) and the resulting point (optimized tensor):
PASE à
Σ
(x;
(x; x²)²²
where n2 represents the pixel value in the starting tensor, xf is the corresponding pixel value in the optimized tensor, and d is the total number of pixels in the tensor. This term controls the allowed deviation by adjusting its weight in the total loss.
The high-frequency penalty term is defined as:
Par
-
+1 I(i,5))
+ Σ 1T( + 1,5) I(i,)),, 2.
where ji,,j) represents the pixel value at position (i,j) in the input tensor. The first term penalizes abrupt changes along the horizontal direction, while the second term does so along the vertical direction.
C PERCOLATION THEORY
Percolation theory. originally developed to describe the structure of a porous material, turned into widely studied model in statistical physics and graph theory (Duminil-Copin) (017).
Let G be an N x N grid, where at each cell, one inserts ball with (Bernoulli distribution) probability p and leaves empty with probability the probability that for any two nonempty cells A and B in G. there is a path of nonempty cells that connects the two. When p is small, the probability is zero, while when p + 1, there is almost certainly connecting path. The latter phase case is called percolation. In the limit N a sharp transition at a critical probability = Pe, that separates the two phases. Figure shows site percolation in two dimensions near the critical probability.
p. Site percolation theory considers the question: what
there is
The percolation model has a straightforward generalization to higher dimensions, as well as to more than the two labels (empty and nonempty), and to general probability distribution for choosing the
15

---

Published as a conference paper at ICLR 2025
labels. In particular, the choice of labels between different cells can be correlated. The structure of the correlation affects the transition to percolation. Intuitively, one expects that higher correlations imply higher probability of having connecting paths. Correlated percolation is an active field of research in physics and mathematics (see e.g. (Bug et al. 1985 Bricmont et al. 1987 Rodriguez & Sznitman
2012
Chalhoub et al. 2024
P-060
Figure 9: An illustration of site percolation two dimensions near the critical probability Chalhoub al. (2024) (the critical probability for two-dimensional site percolation is around 0.59). In this illustration, an "infinite" cluster can be observed. However, it is notable that a significant number of points do not belong to it.
In the framework of our work, the grid is defined by the s-cells the data space, while the labels are the intervals. Since the dimension of the grid is very large, the percolation probability Pc. which inversely proportional the dimension, very small. Hence, our analysis initialization is at the percolation phase.
We provide a dictionary (see Table and graph theory communities.
2 for the terms used in percolation theory in statistical physics
Table 2: Dictionary for percolation theory terms used in statistical physics and graph theory.
statistical physics
graph theory
site
vertex (node) odge
bond
cluster
connected component
infinite cluster
infinite connected component
percolation threshold
critical probability
order parameter
size of the largest component connected subgraph (not infinite)
lattice animal
D ADVERSARIAL DETECTION BENCHMARK
Here, we briefly overview attacks employed in the benchmark by Harder et al.) (2021).
Fast Gradient Sign Method (FGSM): A rapid untargeted attack method that adjusts an input I; by adding small perturbation the direction of the loss function's 's gradient:
Fi,adv + sign( V.. L(xi, v))
(10)
16

---

Published as a conference paper at ICLR 2025
Basic Iterative Method (BIM): An iterative enhancement of FGSM, applying small, peated perturbations, ensuring each update remains within an e-bounded range:
(n+1)) Clip....e (n) + e sign( V (l u)))
(11)
2,adv
Projected Gradient Descent (PGD): A generalization of BIM that begins from a random point near 2; within the c-ball.
Deepfool (DF): Iteratively modifies I; to cross the closest decision boundary by minimiz- ing the perturbation required to alter the model's classification.
Carlini & Wagner WC&W): A powerful optimization-ban baaed attack that minimizes the L2 norm of the perturbation while ensuring misclassification by optimizing loss function de- signed to reduce the model's confidence in the correct class. The attack solves the following objective:
min (tanh(rr,.u)) + +) Dz + + ( + +1))
(12)
where f(2) is a function that encourages the logits to either lower the true class's score (untargeted) raise the target class's score (targeted). The parameter c is optimized to balance minimizing the perturbation and ensuring successful misclassification. The C&W attack can be applied in both targeted and untargeted forms.
E CONJECTURE 5.1 PROOF SKETCH
E.1 ASSUMPTIONS AND GENERALIZATIONS
E.1.1 THE NETWORK
We justify conjecture 5. for fully connected neural networks defined by set of L E N layers, characterized by L weight matrices and L bias vectors, denoted as { L...., L}. The first layer is defined as:
pt p1.0 x +
(13)
Each subsequent layer = ..... T. satisfies:
gil ( +
(14)
where o is Lipschitz-bounded activation function that acts on the vector in pointwise manner.
We then define the network's prediction using a non-linear final activation şp : + [0, 1] such that:
p
(15)
The generalization to other networks within the tensor programs formalism Yang 2019 2020 Yang
& Littwin (202I) (and then using 9)) is straightforward, as long the nonlineanties Lipschitz. The classes of networks described by this formalism is very diverse, and includes most of the relevant types of neural networks, such fully connected neural networks, recurrent neural networks, long short-term memory units, gated recurrent units, convolutional neural networks, residual connections, batch normalization, graph neural networks, and attention mechanisms.
The reason that we can generalize our result to any network in this class is that in our justification, we only use the mimi-linear structure of fully connected neural networks, and, by definition, any network in this class can be described as a composition of global linear operations and pointwise non-linear functions.
1.2 INITIALIZATION AND INPUT DISTRIBUTION
As will see, we can always divide the input space into sufficiently small hypercubes. Therefore, is not crucial how we initialize our network, as long as it remains well-defined as d, - cc.
The only requirement is that for every 0 p and X', the probability for having every label in YA is at least ö, where ă O()), except for subset of intervals, whose combined probability is
17

---

Published as a conference paper at ICLR 2025
less than 1 p. Therefore, p is the probability for the entire conjecture to hold and can be arbitrarily close to
We conjecture that this holds for most well-normalized initializations and networks, and any finite, arbitrarily large 0 < R-radius balls:
x' By = ff E T2 | S R
(16)
E.1.3 THE Loss FUNCTION
We work with C (y. l/ w'1l- However, the generalisation any loss function that satisfies C: Y x RT. such as it is Lipschitz continues in both arguments, and for every Y:
C Cy,, 9) 0
(17)
is straightforward.
E.2 DIVIDING THE INPUT SPACE
Now that we have provided all of the necessary definitions, we can proceed justify the conjecture. We will begin by demonstrating that small changes in the input space generate small changes in the network's output.
Lemma E.1 (Lipschitz Continuity of the Network).
neural network, as described above, is almost always Lipschitz continuous at initialization. This means that for any probability arbitrarily close to one, 0 < < 1, there exists some 0 < M, such that for every x, Ax X: If + Ax; 0) - f(z; 0)1 - M ||
(18)
An immediate consequence of this lemma is that for the situation described in conjecture [5.1] for any probability arbitrarily close to one, 0 < < 1, there exists some O < E such that for every x, Avr E X, if ||| E, then: [| O); (a;; 0) < 8- - S.
8'
(19)
This result implies that we can divide our input space into O(2N) classes, as in equation 7 This means that if we divide our input space into cubes of size Va. not only does every input in each cube belong to the same class, but also that if two different inputs Io, T1 E X satisfy (2 0) f(11; 0)ll < 8', then their cubes share the same interval.
Proof of Lemma E.1
We prove the lemma by induction. We start with the induction base the first layer. Since ø is Lipschitz continuous, we know that there exists some < m such that for every r, Ar E R:
10(r Ar) - w()) & m |All
(20)
which means that for every x, Ax E Rdd::
de
de
|ς0( Ar) - -(x)] Σ (ø(a, Ar;)) o(z;))² S Σ (mAr,)² 1/11/11 (21)
Thus, for every x, Az:
|ςδ( + Ax) - x(x))1 S 3mlAAn m|
(22)
Multiplying by 01.0 and adding 01, we get:
f' + Az) - (z) 0 o(x Az) + 01 (81.0 ((z) + 0t) 02.0( -A) x(x)). (23)
Using the subordinate norm the matrix, we find:
| (xr + Ar) f (z)] | | Ar) <(+))l - ||000 m ||
(24)
Which means that for every p 1, we can bound the change f (a + Az) f (x) by:
| + Ar) (x)] m ||
(25)
We can continue this process by induction, showing that the same holds for every layer, which completes proof.
18

---

Published as a conference paper at ICLR 2025
Remark E.1.
should be noted that the bound we found using the subordinate norm was sufficient for our needs, however, it is far from being the optimal one.
This is not a problem for our work, as all that we require is any Lipschitz bound, and then we can take to be arbitrarily small. However, if one wishes find better bound, which also depends on the distance between inputs, or to investigate neural networks in the infinite limit appropriately, more care will be needed.
E.3 FINDING THE PATH
To find now a d-connected path between the two inputs, all we need to do is to find a connected path of cubes between the cube of the first input to the cube of the second with the same intervals.
Assuming that the cube's intervals are drawn randomly and independently from each other with similar probability p (5), we encounter a classic high-dimensional percolation problem, where the cubes represent the points in the graph, and they are connected if they share the same interval. It is well known that such cases, the probability that two points are connected grows rapidly with the dimension, as shown in the conjecture. Specifically, if we ask what is the probability that a certain point will be part the infinite cluster, we know that in the high-dimensional limit, we Stauffer can neglect closed Stauffrr & Aharony (0118) This is known as a "mean field approximation"
The probability that one input is not part of the infinite cluster is the probability that all nearby cubes connected it are also not part of the infinite cluster. Thus, the probability that point is part of the infinite cluster P satisfies:
1 1 P = (1 - S S + -
(26)
Defining d,p, we find a lower bound for P by solving the equation:
P 1 1-
(27)
Defining Q = 1 - P. we get:
1 e
=1 -1- 1- 10
(28)
e - (1+
Which implies:
4
Q
+ Q(g²Q²) 0 Ole -dz),
(29)
qea
thus completing our justification.
F (APPROXIMATE) LINEAR CONNECTIVITY
The next phenomenon wish to address is the tendency of trained networks exhibit linear paths. Intuitively, this is not particularly surprising. For any system to generalize effectively, it must exhibit a preference for simpler hypothesis functions. In the context of overparameterized models, this preference is often referred to as regularization, and when not explicitly imposed, it is termed "implicit regularization". A reasonable form of simplicity involves not varying drastically between similar inputs.
Extensive research has been conducted on implicit regularization from the perspective parameter space. However, its understanding from the input space perspective remains limited. A notable study Mulayoff et al. (202I) demonstrated that at equilibrium, the second derivative of neural networks with respect to the inputs is bounded and can be minimized as the learning rate increases.
Another possible approach is to consider the Neural Tangent Kernel (NTK) limit. It is known that, in the infinite-widh width limit, neural networks exhibit linear-like behavior Jacot al. (201): Lee et al. 2019). In this limit, their evolution over time can be described by a kernel, two-point matrix function ((2, x'), which generally tends to be larger where the two inputs
x' are similar.
In this regime, for gradient descent, the final prediction can be viewed as an "averaged sum" Lee et al. (2019) of the data, where for every X, more weight is assigned label of inputs that were
19

---

Published as a conference paper at ICLR 2025
closer to (according to the kernel). And it is reasonable to assume that a linear combination of two inputs will be closer to the original inputs than to any other input, that could partially explain the linear mode connectivity.
Another avenue that could be promising is to consider f(azo + 1 0), and decompose it as faao, 0) + f(( 2 0) plus an additional term, whose magnitude should be shown to be small. both f(oz0. 0) and f((1 - a 0) align in the same direction, we would obtain the correct label after applying the softmax function.
is important to emphasize that the ideas presented here are speculative, and only represent our preliminary conjectures. We have not yet validated these concepts, and more rigorous research is needed to confirm or refute these notions.
F.1 CONNECTIVITY EVOLUTION THROUGHOUT TRAINING
Following the observation of connectivity in untrained models (Sec. 4.3) and hypothesizing the role of implicit regularization in trained models, we studied the temporal evolution of the connectivity over training batches and epochs. To maintain comparability for the early stages of training, where the model performs poorly on real data, we again employed synthetic optimal inputs. The optimiza- tion process was adapted from Sec. (ResNet188 input shape (3, 32, 32), output shape 10, starting from Gaussian noise (0, 0.01), Adam optimizer with learning rate 0.05 and weight decay le-7, maximum of 4096 iterations or until reaching the desired loss), with difference of using the loss threshold 5 = 0.005 for convenience and faster convergence. A diversity of inputs was achieved through figh-frequency penalization with weight AV 2.5e 7 of one the inputs in each pair. We prepared 5 pairs for each class and computed primary loss curves. Averaged loss curves (over all pairs, classes and model stages) for the first 30 batches are shown in Fig. T0 and similarly for the first 30 epochs in Fig. 11
14
14
Figure 10: Batch evolution of the averaged loss barriers. Batches are indexed starting from one; zero means an untrained model. Shaded area and error bars show standard deviation.
134
Figure 11: Epoch evolution of the averaged loss barriers Epochs are indexed starting from one; zero means untrained model. Shaded area and error bars show standard deviation.
While we observe the increase of connectivity (decrease of loss barriers) in the early phases of training, it saturates and variance begins grow some point. There are likely more competing effects taking part over the training. It is important to note that the barrier height and complexity depend on the losses of the interpolated inputs.
20

---

Published as a conference paper at ICLR 2025
G EXTRA RESULTS FOR THE SECTION 4.1
Here, we present additional examples of the input space mode connectivity on pairs drawn from the validation datasets of ImageNet and CIFARI0. We used the following models: GoogLeNet (see
Figure 12). Vision Transformer (Figure T3 ResNet18 (Figure 14), MLP (Figure 15 and CNN
(Figure 16
The exact architectures that were employed are the following:
1. GoogLeNet pretrained model inception v1.
2. VIT vit_base pease.patchheee 6.224. ssamiink from timm library pretrained on ImageNet-1K k using Sharpness Aware Minimization Chen et al. (2022)
3. ResNet18 custom trained on CIFAR10.
4. MLPP custom trained CIFARIO. The architecture consisted of 5-layer fully- connected layers with input dimension 3072 (32x32x3 images), hidden layers sizes 4096, 4096, 2048, 1024, and output dimension 10. We used ReLU activations and dropout (p=0.1) after each hidden layer, trained for 100 epochs with Adam optimizer (I==0.001, weight.eecay=le-6).
5. MLP custom trained on CIFARIO. The architecture consisted of six layers (4 convo- lutional, 2 fully-comnected), with convolutional layers of sizes 64, 128, 256, 512 (3x3 kernels), followed by fully-connected layers of sizes 1024 and 10. We used batch nor- malization, ReLU activations, max pooling (2, 2), and dropout (p=0.1) after each layer, trained for 100 epochs with Adam optimizer (Ir=0.001, weight.decay=le-6)
Note that the loss barrier can be negligibly small for some pairs even before the initial round of barrier optimization.
E331
020
Atons earee
Ciaa aD40
Sutee
0.301
040 535 553
Figure 12: GoogLeNet(ImageN) loss curves. Representative examples of connectivity in a trained GoogLeNet for selected classes. Each row contains a pair of same-class images. The primary loss curve is obtained from linear interpolation A-B, and secondary loss curve A-BB' +C Note that the highest loss value in the secondary curves least one magnitude lower than in the primary curves.
21

---

Published as a conference paper at ICLR 2025
10.0
50.30
Secnedey Ornee
30300
$0:00
$09000
000000
Figure 13: Vision Transformer(mmageNe)) loss curves. Representative examples of connectivity in a trained ViT for selected classes. Each row contains a pair of same-class images. The primary loss curve obtained from linear interpolation A-B,, and secondary loss curve A-B'' -C. Note that the highest loss value in the secondary curves is at least one magnitude lower than in the primary curves.
G.1 COMMENTARY ON THE VIT RESULTS
The results presented in Figure 13 valid for standard transformer models without advanced regu- larization strategies or other training tricks. For more recent models employing data augmentations, regularization, and extensive pretraining (such as Steiner al. (2022)), the lowest observed loss for validation images was approximately 0.05. These loss values exceed the threshold for our definition of a mode, and consequently, their connectivity qualitatively different. In our non-extensive ex- periments, were able to connect these higher-loss examples as well, but the number of required segments varied, typically three or more.
G.2 SMALL BARRIERS CLOSE TO NUMERICAL PRECISION
In some cases, for the custom- custom-trained MLP and CNN models, the loss barriers are extremely small. After a single round our optimization procedure, the loss values approach zero within numerical precision (see Figure 17
Specific cases exhibit "true" linear connectivity (see Figure 18), which not generally observed in advanced models (beyond the MLP and CNN architectures). Our hypothesis is that this behavior results from train-test data leakage in CIFARIO and the memorization of these data samples. This phenomenon will be studied more extensively in future work.
H LIMITATIONS
H.1 SECTION 5.1
The main limitation, other than the basic assumptions discussed in section E.I our problem as a standard model percolation, assuming no correlation between different sites. However, this assumption does not hold for neural networks, even initialization. We argue,
is that treated
22

---

Published as a conference paper at ICLR 2025
05
33
Jem k-877
Elow tober
stlaee
000000
Figure 14: ResNec18(CIFAR1O) loss curves. Representative examples of connectivity in trained 8 for selected classes. Each row contains a pair of same-class images. The primary loss curve obtained from linear interpolation A A-BB, and secondary loss curve A-B'- Note that the highest loss value in the secondary curves is at least one magnitude lower than in the primary curves.
however, that since correlations in neural networks positive, the actual scenario should be more favorable. Nevertheless, diligent investigation and exact proofs are necessary to confirm this.
H.2
SECTION 5.2
The ideas discussed in that section are speculative and represent our initial hypotheses. These con- cepts have not yet been validated, and further rigorous research is required to either confirm or disprove them.
H.3 SECTION
Experiments were carried out common vision models. The potential generalization of the concept to other data modalities was not tested. Our approach requires a continuous input space up to standard numerical precision. Therefore, technical adjustments would be necessary for applications beyond vision models, such as language models that use tokenization.
23

---

Published as a conference paper at ICLR 2025
80015
Seceedary
0.000
00000
soaces
53
C900
cauce
cacan
Figure 15: MLP(CIFARIO) loss curves. Representative examples of connectivity in a custom- trained MLP for selected classes, Each row contains a pair of same-class images. The primary loss curve is obtained from linear interpolation A-B, and secondary loss curve A- Note that the highest loss value in the secondary curves is at least one magnitude lower than in the primary curves.
Altew 2000
1mm 8030
20
Emur ane
1mw arn
ncw
035 153
Figure 16: CNN(CIFAR10) loss curves. Representative examples of connectivity in a custom- trained CNN for selected classes. Each row contains a pair same-class images. The primary loss curve is obtained from linear interpolation A-BB and secondary loss curve A-+B' -C. Note that the highest loss value in the secondary curves is at least one magnitude lower than in the primary curves.
24

---

Published as a conference paper at ICLR 2025
Figure 17: MLP (CIFARIO) loss curve example with a small barrier. Many example pairs have loss barriers so small that, after a single round of our optimization procedure, the loss values approach zero within numerical precision.
Figure 18: MIP (CIFAR10) loss curve example with zero barrier. Some example pairs have loss barriers exactly equal to zero and, therefore, exhibit true linear connectivity.
25


Reviews:
[{"review_id": "ycbUiDn0Fn", "reviewer": "Reviewer_KNNq", "summary": "The authors identify an interesting phenomenon, namely input mode connectivity, where samples with similar predictions could be approximately linearly interpolated such that the interpolated sample remains a low loss.", "questions": "1. Can we repeat the produce of finding A->B'-C for each segment recursively to obtain A->E'->D'->B'->F'->G'-C (maybe a longer path), such that there is no essential barrier existing?\n2. How can we use the input mode connectivity to give a picture of the decision boundary of DNNs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This topic is interesting. Investigating the model connectivity in the input space could help us to shape the decision boundary of DNNs.\n2. The insight of mode connectivity is indeed an intrinsic property of high-dimensional geometry is important, as it might be able to explain various phenomena lied in the field of model connectivity, such as wide neural networks are easier to satisfy mode connectivity after accounting for permutation invariance.\n3. The potential application towards adversarial examples is insightful, which might explain the existence of adversarial examples.", "weaknesses": "1. The major issue of this work is that the investigation is not in-depth enough. For example, in Fig. 3, the path A->B'->C and A->B->C look similar but they differ significantly in terms of mode connectivity. \n    - How should we quantify such differences? or why the small difference B'-B (as shown in right bottom of Fig. 1) is significant in terms of model connectivity? \n    - Here is another example, in the adversarial example part, why real-adversarial pair shows a large barrier than real-real pair? An intuitive explanation is at least expected.\n\nThese are all important questions and represent the motivation why we are interested in investigating the mode connectivity in input space.\n\n2. Their theory cannot explain the phenomenon they discovered. Their conjecture is only able to explain the mode connectivity for untrained NNs with infinity large input dimension. However, in their experiments, two real images with similar predictions are usually not connected unless another intermediate point is found, say B'. Clearly, their theory cannot explain the realistic scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qc5XecKFQE", "reviewer": "Reviewer_hRew", "summary": "This paper studies input space mode connectivity, where instead of studying the loss surface as a function of the parameters, the authors consider varying the inputs (for a fixed set of parameters). More specifically, the authors investigate paths connecting two sets of inputs and the resulting behaviour of the loss, in complete duality to the well-known mode connectivity in parameter space. Several choices for “modes” in this context are explored: (1) Validation data points that achieve very low loss, (2) a validation data point and an adversarial example optimized towards the same class but based on a datapoint of different class and (3) synthetic “optimal\" points optimized to maximise a given logit. For all these cases, the authors show how simple piecewise linear paths suffice to connect such points, while limiting the barrier to very small values. Adversarial examples exhibit larger barriers in a significant manner, allowing a detector to leverage this difference to classify whether an image is adversarial or not.", "questions": "1. Do you have any intuition whether results would fundamentally change if other architectures were considered? E.g. [1] find that mode connectivity in the parameter-sense is influenced by the choice of architecture, i.e. there is different behaviour for vision transformers or multi-layer perceptrons.\n2. I’m a bit confused regarding the adversarial example detector; Are you comparing loss barriers after a single iteration of your algorithm in both cases, or after two iterations in case of the adversarial setting? I thought that in both cases the barriers became very small? \n\n[1] Disentangling Linear Mode-Connectivity, Altintas et al., 2023", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The role of the input when it comes to loss behaviour is somewhat understudied and the authors develop new ideas in this direction while keeping things very analagous to the results observed for parameter loss landscapes.\n2. The authors give further credibility to their results by mathematically proving them in an idealized setting assuming independence. While this is not realistic, I do find the argument of the authors convincing that correlations in this case will most likely help connectivity.", "weaknesses": "1. The biggest weakness is the lack of motivation presented in this paper for input space connectivity. Why is this an interesting quantity to study? In case of the parameter loss landscape where this notion originated, the motivation was from an optimisation point of view; is SGD attracted to a convex region of the parameter space? Does it find isolated minima or are there entire regions of low loss? ﻿There might be good motivations for input space connectivity as well (I’m not an expert in this area) but the paper does not do a good job at presenting it in its current form. In general I also have no intuition whether it is surprising that real-real images can be connected with two segments or not, etc. \n\n2. I like the idea of using the difference in barrier between real-real and real-adversarial inputs, but as usual in adversarial robustness, I think that new threat models need to be investigated when taking this idea into account. I.e. can one now develop adversarial examples that are designed to mimick the barrier of real examples, thus fooling the new classifier? I don’t expect the authors to necessarily develop such an algorithm but this possibility should at least be discussed in the paper.\n\n3. I also have a hard time interpreting the adversarial example results. It is not that surprising that it requires more segments to connect things properly compared to the real-real scenario (the image is still very different after all). How does this compare to simply two images coming from different classes and their barriers in between? \n\n4. The writing of the paper is not very satisfying. The notation is rather sloppy (e.g. \"0.1*MSE for image deviation and 1e-7*high-frequency penalty\"),  optimization details are listed without defining them (e.g. high-frequency penalty). The actual algorithm to obtain a piece-wise linear curve is never properly defined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vUAx8ubXwl", "reviewer": "Reviewer_rBxt", "summary": "This paper presents an interesting phenomenon, that given two input points $A,C$ which are classified as the same class by the model (i.e. have similar output), there exists a point $B'$, such that the linear interpolation between $A,B'$ and $B',C$ are all classifed as the same class. The authors refer to this phenomenon as \"input space mode connectivity\", despite they are not actually linear connected. The authors also presented a analysis under a very strong and unrealistic condition, conjecturing that this phenomenon is intrinsic to high-dimensional spaces.", "questions": "- I don't understand the algorithm presented in Section 4.2.1. In my understanding, the claim is that if one of the endpoints is from an adversarial attack, then the method of optimizing $B$ to find a connected (piecewise linear) path will not work. However, the algorithm of detecting adversarial attack is to use the interpolation loss curve and logits as the input and do a classification. I don't see how this proposed algorithm is related to the finding.\n- The wording of Section 4.3 is too confusing that I totally can not understand. What does \"natural datasets for untrained models\" mean? What does \"starting from Gaussian noise\" mean (start from it and do what?) What does \"high-frequency penalization\" mean?\n- In Conjecture 5.1, why does the condition has \"for any probability $0 < p < 1$\", but $p$ is never used in the statement? What does \"almost always connected\" mean?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This phenomenon is interesting and potentially valuable for understanding the behaviour of deep neural networks and the geometry of high-dimensional spaces.\n- This finding has a practical useness that it can be used to detect adversarial attacks.", "weaknesses": "- The presentation is kind of confusing. See Questions for concrete issues.\n- In my understanding, the finding in this paper is not really linear connectivity as the path found by the proposed method is actually a piecewise lienear path with 2 pieces. This makes the title and introduction kind of misleading. \n- The experiments are only on a few image classification tasks and models. It is not clear if this phenomenon is general enough. \n- It seems that the thoretical explanation presented in Section 5.2 (even if we omit the too strong assumption of randomized labeling of each grid) only explains why there can exist a connected path, but does not guarantee that the path is linear, which is inconsistent with Conjecture 5.1.\n- In Conjecture 5.1, why do you need to assume \"a subset of input space $X' \\subseteq X$\"? Does the conjecture hold even if $X'$ itself is unconnected? Moreover, there is a statement \"two random inputs $x_0, x_1 \\in X'$\". What is the distribution of the randomness? If it is uniform, then there must be extra constraints on $X'$ (such as compactness), since not every subset of a Euclidean space has a uniform distribution. (For example, you can not draw two points uniformly from a 2d plane).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)