Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks

Authors: xin-chun  li
jin-lin  tang
bo  zhang
lan  li
de-chuan  zhan
asymmetric  valley
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 06:42:34

Abstract: Exploring the loss landscape offers insights into the inherent principles of deep neural networks (DNNs). Recent work suggests an additional asymmetry of the valley beyond the flat and sharp ones, yet without thoroughly examining its causes or implications. Our study methodically explores the factors affecting the symmetry of DNN valleys, encompassing (1) the dataset, network architecture, initialization, and hyperparameters that influence the convergence point; and (2) the magnitude and direction of the noise for 1D visualization. Our major observation shows that the {\it degree of sign consistency} between the noise and the convergence point is a critical indicator of valley symmetry. Theoretical insights from the aspects of ReLU activation and softmax function could explain the interesting phenomenon. Our discovery propels novel understanding and applications in the scenario of Model Fusion: (1) the efficacy of interpolating separate models significantly correlates with their sign consistency ratio, and (2) imposing sign alignment during federated learning emerges as an innovative approach for model parameter alignment.

Main paper: 
 introduction: 
 The massive number of parameters and complex structure of deep neural networks (DNNs) have catalyzed extensive research to mine their underlying mechanics [54, 33, 41] . Visualizing and exploring the loss surfaces of DNNs is the most intuitive way [42, 18] , which has ignited many interesting findings, such as the monotonic linear interpolation [22, 19, 64] and linear mode connectivity [22, 14, 21, 15, 63, 2] . Loss landscape visualization has also been applied to show the optimization trajectory [52, 32, 34] , understand the effectiveness of Batch Normalization [31, 60] , BERT [11, 23] , deep ensemble [17, 29, 21] , and so on.Perturbation analysis around the local minima of DNNs [7, 61] , i.e., the shape of the valleys they reside in, is a very popular research topic. The concept of flat minima was originally proposed by [27] , who defines the size of the connected region around the minima where the loss remains relatively unchanged as flatness. Subsequent studies debate whether the flat or sharp minima could reflect the generalization ability [37, 42, 35, 56, 16, 12, 40, 3] . The previous works constrain the valley shape to be symmetric, while recent work points out that not all DNN valleys are flat or sharp, and there also exist asymmetric valleys [24] , which has not been systematically studied as far as we know.This paper in-depth analyzes the factors that may affect the valley symmetry of DNNs. Previous work's analysis of valley shape primarily utilizes the 1D interpolation of θ f + λ , where θ f represents the minima solution and denotes a random noise. As shown in Fig. 1 , we believe that the valley symmetry depends both on the convergence solution and noise, with each of them being influenced by some factors. The most significant innovation in our research is considering the effect of noise direction on valley visualization, as previous work has simply taken the Gaussian noise. Specifically, we start by comprehensively and carefully determining the visualization method to plot the valley shape, which could influence the conclusion significantly [30, 42, 40] . We finally take the Norm-Scaled (NS) visualization method [30] that normalizes the noise and further scales it to θ f for better determining the plot range of λ, where the direction of raw noise is not changed. Then, after investigating 7 common noise directions and 6 special ones, we conclude: the degree of sign consistency between the noise and the convergence solution should be a determining factor for asymmetry. This phenomenon is basically insensitive to the utilized datasets. Next, we focus on the impact of the network architecture with or without Batch Normalization (BN) [31] , indicating that the BN initialization also impacts the valley symmetry. Finally, different hyperparameters lead to solutions with various valley widths but show no asymmetry consistently.Aside from empirical observations, theoretical insights for our interesting findings are provided. We first declare that adding sign-consistent noise to parameters may have a larger probability of keeping activating the neurons or keeping the overwhelming score in classification tasks. Then, we show that the trace of the Hessian matrix along the sign-consistent direction is smaller, implying a flatter region. The above findings inspire applications in the fields of model fusion [46, 66, 32, 2] . This paper first explains why model aggregation based on pre-trained models often leads to performance improvements, i.e., the success of model soups [66] , and then proposes constraining the sign of DNN parameters in federated learning [53, 70, 49] to facilitate aggregation.Our novel contributions can be summarized as (1) exploring the valley shape under different noise directions that have not been studied yet; (2) proposing that the flat region could be expanded along the direction that has a higher sign consistency with the convergence solution;(3) pointing out the influence of BN and its initialization on valley symmetry; (4) presenting theoretical insights to explain our interesting finding; (5) explaining and inspiring effective algorithms in model fusion. 
related works: 
 Exploring the Valley Shape of DNNs. The valley around a minima solution has originally been viewed as flat or sharp [27] , and the large batch size training may lead to sharp minima with poor generalization [37] . However, [12] declares that two solutions that are scale-invariant in performance may lie in regions with significantly different flatness. [42] verifies the findings of [12] by applying varying weight decay to small and large batch training, leading to results contrary to [37] . With the filter-normalized plots, [42] again observes that sharpness correlates well with generalization error. Later work also shows that the sharpness calculation should be dependent on the parameter scale [40] or the dataset [3] . Whether valley width reflects the generalization is still inconclusive. The proposal of asymmetric valley [24] further throws this debate into limbo, as valleys around minima could be flat on one side but sharp on the other side, which makes it more difficult to define flatness. This paper thoroughly studies the causes and implications of the unexplained asymmetry phenomenon.Exploiting the Valley Shape of DNNs. Exploring the valley shape of DNNs could help us better understand the inherent principles of DNNs. The work [42] utilizes 2D surface plots to show that the residual connections in ResNet [26] could prevent the explosion of non-convexity when networks get deep, and [60] attributes the success of BN to its effects of making landscape significantly more smooth. The asymmetric valley [24] provides a sounding explanation for the intriguing phenomenon in stochastic weight averaging [32] . Additionally, studying the valley shape could also benefit the proposal of effective optimization algorithms, e.g., the entropy-based SGD [6] , and the (adaptive) sharpness-aware minimization [16, 40] . Penalizing the gradient could also lead to solutions around flat regions [71] . We also apply the findings in this paper to the area of model fusion.Model Fusion. Directly averaging two independent models may encounter a barrier due to training randomness or permutation invariance [4, 22, 2, 50] . However, if the two models are generated from separate training of a common pre-trained model, the model fusion may perform better than individual models, i.e., the model soups [57, 66] . A recent work [68] finds that resolving sign conflicts when merging multiple task-specific models is neccessary, which is most related to our current work. We will explain the success of model soups based on the relation between the asymmetric valley and the sign consistency of model parameters. Popular federated learning algorithms also take the parameter averaging process to fuse the individual models updated on isolated clients [53, 51] . A huge challenge is the Non-Independent and Identical Data distributions of data islands (Non-I.I.D. data) [70, 28] , which could make local models too diverged to merge. Multiple regularization methods are proposed to align parameters before model fusion [45, 1, 44, 36] . We propose an effective regularization method that focuses on the sign of parameters, which is inspired by our interesting findings. 
basic notations and preliminaries: 
 Our major tool is plotting the 1D error curve of DNNs following the formula θ f + λ (Fig. 2 (A) ). θ f denotes the converged model, and denotes a noise vector sampled from a specific distribution. More about the visualization of DNN loss landscape could be found in [42, 48] .The previous studies focus on studying the valley shape under different θ f , and aim to mine the shape's relation to generalization. To mitigate the influence of parameter scales and make the visualization fairly comparable between different θ f , [42] proposes the filter normalization method to properly visualize the loss landscape. The processing of the noise is i,j ← i,j i,j θ i,j f , where i is the index of layer and j is the index of filter. This way normalizes each filter in the noise to have the same norm of the corresponding filter in the converged point θ f . Further, [40] proposes a proper definition of sharpness (i.e., adaptive sharpness) based on the filter normalization, extending it to all parameters and formally defining:EQUATION, . . . , w q , where f j with 1 ≤ j ≤ m denotes the j-th convolution filter in θ f and n j is the number of parameters it owns. w j with 1 ≤ j ≤ q denotes the j-th parameter that is not included in any filters. I is a vector with all values as one. Then, the adaptive noise T θ f is utilized to study the sharpness of θ f .Different from the previous studies, we aim to explore the valley shape of a fixed θ f under different and λ. First, the direction of 1D interpolation in previous works is limited to the Gaussian noise, while we study the impact of different noise types. Second, setting λ positive or negative could obtain a valley with different flatness, i.e., the asymmetric valley [24] . Hence, under the fixed θ f , we do not need to rectify the noise direction filter-wisely. We take the visualization way used in [30] , which only normalizes the noise and then re-scales it to the norm of θ f , i.e., ← θ f . The utilized Norm-Scaled (NS) noise is shown in Fig. 2 (C ). Compared with Filter NS noise (Fig. 2 (B )), this way does not change the direction of the noise and shows the original valley shape along the direction . Additionally, to plot 1D error curves in the same figure, we fix λ in the range of [-1, 1]. Another scale factor s is added to control the visualized width. Overall, we use the following way to plot 1D error curves:θ f + λ * s * θ f , where λ ∈ [-1, 1], and we set s = 1.0 by default. The Frobenius norm is used. Notably, we utilize the NS noise by default and use the Filter NS noise when comparing the valley shape under different converged points, e.g., the studying of BN initialization in Sect. 4.2.1. 0.0 0.5 1.0 Test Error NS ε ∼ G(0, 1) s=0.2 s=1.0 s=2.0 NS ε ∼ U(-1, 1) s=0.2 s=1.0 s=2.0 NS ε ∼ {-1, 0, 1} s=0.2 s=1.0 s=2.0 NS ε ∼ G(1, 1) s=0.2 s=1.0 s=2.0 NS ε ∼ U(0, 1) s=0.2 s=1.0 s=2.0 NS ε ∼ {0, 1}Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
experimental findings: 
 This section presents the major findings about factors that respectively impact the noise and converged points. Experimental details and more verification results are in Appendix A and B.We train VGG16 [62] with BN [31] on CIFAR10 [38] for 200 epochs and obtain the converged model. For the given θ f , we plot the 1D error curves under the following 7 common noise types: (1) G(0, 1): Gaussian noise with mean as 0 and std as 1; Then, a fantastic idea motivates us to change the sign of the noise. The detail of this motivation is provided in Appendix A. 4 . Specifically, we use the following method to replace the sign of with that of θ f : ← * sign(θ f ), where ⋅ returns the absolute value element-wisely and sign(⋅) returns 1 or -1 based on whether the element is positive or negative. The corresponding results of the 7 common noises become completely asymmetric, which are plotted in the second row of Fig. 3 . Furthermore, the valleys all follow the tendency that the positive direction is flat while the negative direction is sharp. Hence, we propose our major finding: the sign consistency between noise and converged model determines the asymmetry and the valley is flatter along the noise direction with a larger sign consistency. The finding is formulated as: L(θ f + aη) < L(θ f -aη), where η = * sign(θ f ) denotes the sign-consistent noise, and a > 0 is a constant. L(⋅) is the loss function, which could be the prediction error or cross-entropy loss. The following three experimental studies could further verify this interesting finding.The Manual Construction of Noise Direction. We element-wise sample the noise from G(0, 1), and then manually change its elements' sign with a given ratio r ∈ {0.0, 0.1, . . . , 1.0}. For example, r = 0.5 means that we sample 50% elements in the noise and change their sign to the same as θ f . Then, we plot the average test error of the positive interpolations and negative interpolations, i.e., E λ [Error(θ f + λNS( ))] with λ ∈ [0, 1] and λ ∈ [-1, 0], respectively. Fig. 4 plots the average test errors on two groups of networks and datasets. The test errors of positive and negative directions are nearly equal when r = 0%. As r becomes larger, the average test error of the positive direction monotonically decreases while the negative one increases, implying that the valley shape becomes more and more asymmetric.The Investigation of 6 Special Noise Directions. We then investigate several special noise directions including (1) the initialization before training, i.e., 1 = θ 0 ; (2) the converged model itself, i.e., 2 = θ f ;(3) 3 = sign(θ f ); (4) 4 = sign(θ f -µ); (5) 5 = sgp(θ f ); (6) 6 = sgp(θ f -µ).Here, µ denotes the mean value for each parameter group, e.g., the mean value of "conv1.weight". sgp(⋅) returns 1 or 0 based on whether the element is positive or not. The visualization results are provided in Fig. 5 . The first four directions lead to asymmetry, while the last two do not. First, the elements in 2 and 3 surely have the same sign with θ f , which leads to an asymmetric valley. Because the mean values of most parameters are near zero, 4 performs likely as 3 . 5 and 6 only have the same sign with the positive parameters in θ f , and applies zero to negative parameters in θ f , which shows no asymmetry. The most interesting result is the 1 = θ 0 , whose elements may be centered around zero according to the Kaiming initialization [25] . However, the BN initialization is asymmetric, which leads to asymmetric curves (Sect. 4.2.1). The results of VGG11 without BN on CIFAR100 [38] show no asymmetry when = θ 0 (Appendix B, Fig. 17 ).The Finding Holds for ImageNet and Various Parameter Groups. We extend the findings to large-scale datasets and apply noise only to specific parameter groups. Specifically, we use the pre-trained models (e.g., ResNeXt101 [67] ) downloaded from "torchvision" 1 . Because these models are pre-trained on ImageNet [10] , we could directly use them to verify our findings without additional training. Multiple parameter groups are considered as follows: (1) "ALL" denotes the whole parameters; (2) "CLF" denotes the weights in the final classifier layer; (3) "FEAT" denotes the weights in the layers aside from the final classifier; (4) "LAYER1" denotes parameters in the first several blocks; (5) "CONV" denotes all convolution parameters; (6) "BN" denotes all of the BN parameters. As shown in Fig. 6 , applying sign-consistent noise could lead to asymmetric valleys. Notably, this holds for both the metrics of CE loss and prediction error.Then we focus on studying the effects of BN and its initialization, then present the results under various hyperparameters. ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 LR=0.003 ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 BS=32 ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 BS=2048 ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 WD=1e-3 ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 WD=1e-5 ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ 0.0 0.5 1.0 Test Error ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0The default initialization method of BN is setting w as ones and b as zeros [31, 9] . Hence, the w, i.e., the "BN.weight", may be asymmetric after convergence. We take three ways to initialize the BN weights, including (1) the elements are all ones; (2) the elements are sampled from U (0, 1); (3) the values sampled from G(0, 0.1). We train three models based on these three types of BN initialization. In Fig. 7 , the first row shows the initial and converged parameter distribution of a specific BN weight. The traditional initialization leads to converged BN weights with all positive values, which are nearly centered around 0.2. The uniform initialization between 0 and 1 also leads to positive converged weights. The symmetric Gaussian initialization leads to converged values symmetric around 0. Then, we plot the valley shapes under the noise direction ∈ {0, 1}. Because this part involves a comparison among different convergence points, we plot the results by both the NS noise and Filter NS noise. As vividly displayed in Fig. 7 , the first two initialization ways encounter obvious asymmetry while the Gaussian initialization shows nearly perfect symmetry. If we carefully analyze the sign consistency ratio of them, we could easily explain this phenomenon. The noise direction ∈ {0, 1} has a larger overlap with the first two initialization methods because the converged BN weights are all positive, while it has a lower overlap with the initialization from G(0, 0.1). This implies that the traditional BN initialization will lead to nearly all positive converged BN weights, which may influence the valley symmetry.Previously, [37] shows that the batch size could influence the valley width, and further [42] advocates that weight decay could also play a role in the valley width. Different from them, we aim to study whether these hyperparameters influence the valley symmetry. We train VGG16 networks on CIFAR10 with various hyperparameters. We use the SGD optimizer with a momentum value Then, we explore the interpolation between two solutions under different hyperparameters, which are studied in [37, 42, 22] . We take the batch size of 32 and 2048 as an example and denote the converged solution as θ f1 and θ f2 . The test error curve of (1 -λ)θ f1 + λθ f2 is plotted, with λ ∈ [-1, 2]. Aside from the interpolation curve, we also plot the parameter distributions of θ f1 and θ f2 . The parameters are divided into five groups, including "BN Weight", "CLF Weight", "CLF Bias", "Other Weight", and "Other Bias". "CLF" denotes the last classification layer, and "Other" denotes other layers aside from the BN layers and the last classification layer. To simplify the figures, we only plot the mean and standard deviation of the parameters, denoted as µ p and σ p . Fig. 8 shows the parameter distributions and the interpolation curve. The interpolation curve shows that the small batch training (i.e., λ = 0.0) lies in a sharper and nearly symmetric valley, while the large batch training (i.e., λ = 1.0) lies in a flatter but asymmetric valley. Small batch training (i.e., θ f1 ) leads to parameters with smaller mean and std values. This is because the utilized weight decay is 0.0005, which makes the parameter scale smaller due to longer training [42] . Then, we explain the different results of valley symmetry. The interpolation formula could be re-written as θ f1 + λ(θ f2 -θ f1 ) and θ f2 + (λ -1)(θ f2 -θ f1 ), which respectively shows the 1D interpolation centered around θ f1 and θ f2 . If we let = θ f2 -θ f1 , then we could plot the sign consistency ratio (i.e., how many parameters have the same sign) of θ f1 and , and θ f2 and . The results are in Fig. 8 , where we provide the values of the five parameter groups.Obviously, θ f2 is more consistent in the sign values, which shows a flatter region towards the positive direction. In contrast, the sign consistency ratio of θ f1 is smaller, which only shows slight asymmetry. 
theoretical insights to explain the finding: 
 This section provides theoretical insights from the aspects of ReLU activation and softmax function to explain the interesting phenomenon. The forward process of DNNs contains amounts of calculation of W T h, e.g., the fully connected and convolution layer. W denotes the weight matrix and h is a hidden representation. A special case is the final classification layer with the softmax function. Given h ∈ R d , the ground-truth label y ∈ [C], and the weight matrix W ∈ R C×d , the softened probability vector isp = softmax(W h). The cross-entropy (CE) loss function is L(W ) = -log p y . The gradient of w c is g wc = -(I{c = y} -p c )h, with c ∈ [C]and I{⋅} being the indication function. This implies that the update direction of W lies in the subspace spanned by hidden representations, which also holds for intermediate layers and convolution layers [59] . After adequate training steps, the parameters that activate the ReLU function or correspond to the ground-truth label should correlate well with their inputs. We show a demo classification weight learned on the "sklearn.digits"2 . Fig. 10 shows the pattern change of w + λ * sign(w) with λ ∈ [-1.0, 1.0]. Clearly, the weight under λ = 0.0 correlates well with the input sample (i.e., Digit 0 in Fig. 10 ). Setting λ > 0.0 will almost keep the pattern, while λ < 0.0 destroys it significantly. That is, w + λ * sign(w) with λ > 0.0 may keep providing a high score for the target class, while setting λ < 0.0 may decrease the score.For the ReLU activation, it also holds that w + λ * sign(w) will have a higher probability of keeping activating the neurons when λ > 0.0. To simplify the analysis, we assume the learned w equals a * h + δ, where a is a constant and δ is a random Gaussian vector. Then we could easily verify that (w + λ * sign(w)) T h will have a higher probability of keeping activating neurons under a positive λ than the negative one. The details and simulation results are in Appendix D.2. If the neuron outputs are only simply scaled by a factor, it will not affect the relative scores of the final classification. For example, the inequation of w T 1 h > w T 2 h will not change if h is scaled by a positive factor, while it does not hold for h whose values are not activated, i.e., h = 0.Then we provide a further analysis via analyzing the Hessian matrix of the softmax weights. Specifically, the Hessian of L(W ) = -log p y w.r.t. W is H = (diag(p) -pp T ) ⊗ hh T , where ⊗ denotes the Kronecker product. The trace of H is tr(H) = tr(diag(p) -pp T ) * tr(hh T ). The first part could be calculated as ∑ c p c (1 -p c ), where c is the class index. According to the above analysis, adding sign-consistent noise to w y could enlarge the score of w T y h, which may make the p y larger and p c≠y smaller [47] . That is, the predicted probability vector tends to be a one-hot vector when adding sign-consistent noise, and ∑ c p c (1 -p c ) will be near zero. Hence, the trace of the Hessian matrix is smaller along the sign-consistent direction. Since softmax is convex, and the eigenvalues of the Hessian are all positive, a smaller trace means smaller eigenvalues, which makes the loss curve flatter. The empirical observation can be found in Appendix D.3.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: We provide core codes and a demo code to reproduce the observed phenomena in our paper. We do not provide codes with external links. The demo code is in Code 1 and Code 2.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
applications to model fusion 6.1 explaining the success of model soups: 
 Commonly, the interpolation of two independently found DNN solutions may encounter a barrier [22, 63] . Surprisingly, if these two models are updated from the same pre-trained model, then the barrier will disappear, and the linear interpolation brings a positive improvement [57, 66] . The common explanation follows that the pre-trained model may possess less instability when compared to the random initialization models [20] . We guess that the sign consistency ratio may influence the parameter fusion performance of the two models. Perhaps, the sign of model parameters updated based on the pre-trained model remain nearly unchanged during the process of fine-tuning.We experimentally verify our guess on two datasets, i.e., training VGG16BN with completely random initialization on CIFAR10, and training ResNet18 with pre-trained initialization from PyTorch on Flowers [58] . The datasets are first uniformly split into two partitions, and then two models with corresponding initialization are separately trained or fine-tuned for 50 epochs. The checkpoints in the {1, 2, 3, 5, 10, 20, 30, 50}-th epoch are stored. For checkpoints in the specific epoch, we denote the two models on the two partitions as θ A and θ B , respectively. The interpolation accuracy of (1 -λ)θ A + λθ B on the test set is plotted in Fig. 11 . The interpolation curve of VGG16BN on CIFAR10 indeed encounters a significant barrier, especially when the epoch is larger, e.g., E = 50. In contrast, the interpolation surpasses the individual models on Flowers, which is attributed to the pre-trained ResNet18. As an explanation, we calculate the sign consistency ratio between θ A and θ I , θ B and θ I , and θ A and θ B , denoted as "SSR-IA", "SSR-IB", and "SSR-AB", respectively. θ I means the initialization model. We also plot the gap of model interpolation and individual models when λ = 0.5, i.e., Acc(0.5θ A + 0.5θ B ) -0.5(Acc(θ A ) + Acc(θ B )). The right of Fig. 11 clearly shows that the sign consistency ratio could almost perfectly reflect the tendency of the interpolation gap. Notably, the sign consistency ratio of models on Flowers is higher than 0.95, which means that fine-tuning the pre-trained model does not change the parameter signs a lot, which facilitates the following parameter interpolation. The previous work [68] points out that disagreement on the sign of a given parameter's values across models is a major source of interference during model merging. Although our finding is similar to the previous work, the motivation and specific explanation differs a lot.Traditional machine learning models will encounter the challenges posed by "isolated data islands", e.g., the Non-I.I.D. data [53, 28] . FedAvg [53] , as the most standard federated learning (FL) method, utilizes the parameter server architecture [43] , and fuses collaborative models from local nodes without centralizing users' data. Specifically, the local clients receive the global model from the server and update it respectively on their devices using private data, and the server periodically averages these models for multiple communication rounds. That is, FedAvg takes the simple parameter averaging to fuse local models. However, due to the Non-I.I.D. data and permutation [50, 65, 69] , the local models could become too diverged to effectively merge. Numerous efforts are paid to regularize the local models so as not to go too far away from the global model, such as the proximal term proposed by FedProx [45] , the contrastive regularization proposed by MOON [44] , the dynamic regularization of FedDyn [1] , and the position-aware neurons proposed by FedPAN [50] . Inspired by our finding, we propose to regularize the sign change when updating local models, i.e.,EQUATIONwhere t denotes the communication round and k is the index of client. L k ce is the common crossentropy loss of the k-th client, and θ t is the global model received from the server. θ k t ← θ t is the local model to be updated, and the loss could regularize the sign of θ k t to be close to that of θ t . Because obtaining the sign of parameters is not a continuous function, we therefore apply a sigmoid function to parameters as an approximation. We name this method FedSign and list its pseudo-code in the Appendix C. Compared with other regularization methods, our proposed FedSign is well-motivated because of our finding that interpolating sign-consistent models may lead to a flatter loss region.Experimental studies are verified on CIFAR10 [38] and CINIC10 [8] that are commonly utilized in previous works [69, 50] . Decentralizing the training data of these datasets by a Dirichlet distribution could simulate the Non-I.I.D. scenes as in real-world FL. The Dirichlet alpha α is utilized to control the Non-I.I.D. level, with a smaller α representing a more rigorous heterogeneity between clients' data. We set α ∈ {10.0, 1.0, 0.5} respectively. The number of clients is 100, and the total communication round is 200. During each round, a random set of 10% clients participate in FL and every client takes 5 epochs update on their individual data. After all communication rounds, we evaluate the model performance on a global test set on the server (i.e., the original test set of corresponding datasets). We select our hyperparameter γ from {0.001, 0.01, 0.1} and report the best results. For α = 10.0, i.e., a relatively I.I.D. scene, a smaller γ is better. In contrast, γ = 0.1 or γ = 0.01 will be more proper for α = 0.5. The performance comparison results are listed in Tab. 1. For FeaAvg and our proposed FedSign, we rerun the experimental studies five times and list the standard deviation of accuracies, showing that the accuracy doesn't fluctuate very much. FedSign could surpass the compared methods, which shows the positive effects of regularizing the sign change in FL.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Experimental settings and details are provided in the Appendix A.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
limitations and future works: 
 Although we provide theoretical insights to explain the interesting phenomenon, no formal proofs are provided to show the conditions and scopes that lead to asymmetric valleys. Additionally, this phenomenon is only investigated in the image classification tasks. Future research includes providing formal theoretical foundations for our findings and verifying them on more tasks. According to the analysis in Sect. 5, we advocate that this phenomenon is more likely to be applicable to DNNs that contain both the ReLU and softmax.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: Our paper presents interesting phenomena about DNNs, which do not need error bars. However, we provide multiple groups of experimental observations to verify that the phenomena are common across various conditions. For example, multiple groups of experimental studies are provided to support our finding, e.g., the plots in Appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
conclusion: 
 We explore and exploit the asymmetric valley of DNNs via numerous experimental studies and theoretical analyses. We systematically examine various factors influencing valley symmetry, highlighting the significant role of sign consistency between noise direction and the converged model. The findings offer valuable insights into practical implications, enhancing the understanding of model fusion. A novel regularization method is proposed for better model averaging in federated learning. take 12 layers in the transformer encoder and use 8 heads in each multi-head self-attention block. We set the embedding dimension as 128 to reduce the computation burden. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Our work has no societal impact because we focus on studying the basic properties of DNNs.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper does not pose such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA] Justification: The dataset and pre-trained used in the work are publicly available, and we were unable to find the license for the dataset we used.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: We do not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: We provide the training details for obtaining converged models. We investigate the following pairs of datasets and networks. For the series of Fig. 3 and Fig. 5 , we train VGG16BN on CIFAR10, ResNet20 on SVHN, VGG11 with no BN on CIFAR100, AlexNet on Food101, and ViT on Food101. For VGG16BN, ResNet20, VGG11 with no BN, and AlexNet, we use the SGD optimizer with a momentum of 0.9. We set the learning rate as 0.03. We use a cosine annealing way to decay the learning rate across 200 training epochs. The default weight decay is 0.0005, and the default batch size is 256. For training ViT, we use the AdamW optimizer with a learning rate of 0.0001. The batch size is 256, and the weight decay is 0.0005. We also take the cosine annealing way to decay the learning rate.The equation of the BN layer is X = w X-m X √ v X +η + b, where X ∈ R C×d denotes the feature map with C channels, and m X ∈ R C and v X ∈ R C are channel-wise mean and variance values of the feature map. In practice, w, b ∈ R C are learnable parameters, while m X and v X are running statistics that are calculated during the forward pass. When interpolating θ f + λ with BN layers, we should clear these running statistics after interpolating model parameters and feed the interpolated model to the dataset for another forward pass to calculate proper data distributions. The forward-again process is also utilized in previous works [32, 66] .For Fig. 7 , we only change the initialization method of BN layers and keep the other hyperparameters not changed. For the series of Fig. 8 , we only change a specific hyperparameter including the learning rate, batch size, or weight decay. Specifically, the learning rate is varied in {0.1, 0.003}, and the batch size is varied in {32, 2048}, and the weight decay is varied in {0.001, 0.00001}.The asymmetric valley is initially proposed by [24] , while it does not propose the inherent principles behind the phenomenon. It only points out that adding asymmetric noise (e.g., ∼ {0, 1}) to DNNs may result in an asymmetric valley. The symmetric noise around zero may not show such patterns (e.g., ∼ {-1, 0, 1}). This inspires us to plot valleys along three types of symmetric noise directions and four types of asymmetric noise directions, and the results are shown in Fig. 3 . Indeed, the last four types of noise are not symmetric around zero and show slight asymmetric valleys. However, this is not so obvious. Notably, the utilized network in Fig. 3 has BN layers, i.e., VGG16 with BN.Then, we try to apply ∼ {0, 1} to DNNs without BN, i.e., VGG11 without BN. The valleys become symmetric as shown in the first row of Fig. 13 . Hence, this makes us consider the effect of BN layers. Fortunately, we find that the traditional BN initialization will initialize the values in "BN.weight" as 1, and the converged BN weights are all positive. The initial findings could be summarized as:• If DNNs have BN, and the parameters are perturbed by noise with symmetric values around zero, the valleys are symmetric. This is shown as the top first three plots in Fig. 3 .• If DNNs have BN, and the parameters are perturbed by noise with asymmetric values around zero, the valleys are slightly asymmetric. This is shown as the top last four plots in Fig. 3 .• If DNNs do not have BN, and the parameters are perturbed by noise with symmetric values around zero, the valleys are symmetric. This is shown as the top first three plots in Fig. 13 .• If DNNs do not have BN, and the parameters are perturbed by noise with asymmetric values around zero, the valleys are symmetric. This is shown as the top last four plots in Fig. 13 .That is, only the second case shows slightly asymmetric valleys, where the noise (e.g., ∈ {0, 1}) has a large sign consistency with the BN weights (e.g., > 0.0). As shown in Sect. 4.2.1, if we replace the initialization of BN weights with a random Gaussian initialization, then the plotted valley becomes symmetric because the converged BN weights are symmetric around zero again (Fig. 7 ).That is, the converged "BN.weight" are all positive values under the common BN initialization. If we perturb them by asymmetric noise (e.g., ∈ {0, 1}, ∈ G(1, 1), ∈ U (0, 1), or ∈ {1}) that has a larger sign consistency with "BN.weight", the plotted valleys are asymmetric. This provides an explanation for the asymmetric valley found by [24] .From the above summary, we guess that the sign consistency between parameters and the noise may lead to asymmetric valleys. Hence, a fantastic idea motivates us to change the sign of the noise to the same as θ, which leads to obvious asymmetric valleys (the bottom row of Fig. 3 and Fig. 13 ).In this section, we provide more experimental verification results to make our work more solid. We list the supplemented results by the corresponding experimental studies in the body.B.1 More Results as Fig. 3 Fig. 3 shows the valley shape under 7 common noise types. The results are plotted via training VGG16 with BN on CIFAR10. We list more results to verify the finding is common across various networks and architectures, including: (1) ResNet20 on SVHN (Fig. 12 ); (2) VGG11 without BN on CIFAR100 (Fig. 13 ). Obviously, these results are indeed similar, which verifies again that the sign consistency ratio matters a lot in the valley symmetry.We also extend the findings to the large-scale dataset and popular network architectures including:(1) AlexNet on Food101 (Fig. 14 ); (2) ViT on Food101 (Fig. 15 ). Applying noise with a higher sign consistency also leads to asymmetric valleys. However, the large models are relatively stable to some extent, and the asymmetry is not as obvious as the results on the previous datasets and networks.B.2 More Results as Fig. 5 Fig. 5 shows the valley shape under 6 special noise types. The results are plotted via training VGG16 with BN on CIFAR10. We list more results to verify the finding is nearly common across various networks and architectures, including: (1) ResNet20 on SVHN (Fig. 16 ); (2) VGG11 without BN on CIFAR100 (Fig. 17 ). Obviously, these results are indeed similar. An exceptional case is the first sub-figure in Fig. 17 , i.e., using the initialization as noise leads to a symmetric valley, while the valley in Fig. 5 and Fig. 16 is asymmetric. This is because of the initialization of BN parameters, where the latter two utilize BN weights as all ones. However, Fig. 17 takes VGG11 without BN layers, which shows no asymmetry.We also extend the findings to the large-scale dataset and popular network architectures including:(1) AlexNet on Food101 (Fig. 18 ); (2) ViT on Food101 (Fig. 19 ). Similar to the previous results, although the asymmetry is not as obvious as the results on the previous datasets and networks, the curves still show slight asymmetry when the sign consistency is high.B.3 More Results as Fig. 6 Fig. 6 shows the valley shape investigated on ImageNet with pre-trained ResNeXt101. We also provide similar results with the pre-trained ResNet50 as in Fig. 20 . Fig. 7 shows the impact of BN and its initialization on the valley symmetry. The results are plotted via training VGG16 with BN on CIFAR10. We list more results to verify the finding is common across various networks and architectures, including (1) ResNet20 on SVHN (Fig. 21 ); (2) ResNet20 on CIFAR100 (Fig. 22 ). Obviously, these results are indeed similar, which shows that the original BN initialization may lead to an asymmetric valley. Replacing the original initialization with the symmetric Gaussian distribution could lead to symmetric valleys.B.5 More Results as Fig. 9 Fig. 9 shows the impact of various hyperparameters on valley symmetry. The results are plotted via training VGG16 with BN on CIFAR10. We list more results to verify the finding is common across various architectures, i.e., ResNet20 on CIFAR10 (Fig. 23 ). This also verifies that the hyperparameters have less impact on the valley symmetry.B.6 More Results as Fig. 8 Fig. 8 studies the interpolation between two models that trained under different hyperparameters, e.g., learning rate (LR), batch size (BS), and weight decay (WD). The body only shows the impact of batch size on training VGG16BN. Then, we list more results to verify the finding is common across various architectures, including: (1) the impact of learning rate on training VGG16BN on CIFAR10 (Fig. 24 ); ( 2 Test Error the impact of batch size on training ResNet20 on CIFAR10 (Fig. 26 ). From these figures, we could again observe that the parameter scale influences the valley width, while the sign consistency ratio matters to the valley symmetry.ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0 -1.0 0.0 1.0 λ ε ← |ε|*sign(θf) ε ∼ G(0, 1), s=0.5 ε ∼ G(0, 1), s=1.0 ε ∼ G(0, 1), s=2.0Additionally, we could further obtain the following conclusion: a larger learning rate (e.g., 0.1), a smaller batch size (e.g., 32), or a larger weight decay (e.g., 0.001) could lead to better performances, which are shown in these figures with a lower test error when compared the opposite hyperparameter. However, their parameter scales are relatively smaller than opposite ones, making their valley width sharper. And commonly, a larger parameter scale of θ f1 will let the sign of θ f1 -θ f2 conform to θ f1 more, which leads to a flatter region along the positive direction of θ f2 . 1: for global round t = 0, 1, 2, . . . , T do 2:S t ← sample max(Q ⋅ K, 1) clients 3:for k ∈ S t do 4:θk t ← ClientProcedure(k, θ t ) 5:end for 6: for each batch with B samples from D k do 4:EQUATIONCalculate the loss as in Eq. 1, update θ t using, e.g., SGD with momentum 5:end for 6: end for 7: Return: the updated model θk tWe provide a pseudo code of the application to federated learning, i.e., FedSign. Our proposed FedSign takes a novel regularization method to limit the change of local models' signs, whose goal is for better model fusion on the server. The pseudo-code is listed as in Algo. 1. Here, T denotes the number of communication rounds, K is the total number of clients, Q is a client participating ratio in each round, E is the local update epochs of participated clients, B denotes the batch size, and D k denotes the private data of the k-th client. In our experimental studies, we set T = 200, K = 100, Q = 10%, E = 5, and B = 64. After all communication rounds, we obtain the final aggregated model θ T +1 , and then we test its accuracy on the global test set to evaluate the aggregation performance of FedSign.Our major finding is formulated as: L(θ f + aη) < L(θ f -aη), where η = * sign(θ f ) denotes the sign-consistent noise, and a > 0 is a constant. L(⋅) is the loss function. This finding holds for several settings, including (1) the cases when L(⋅) is the prediction error or cross-entropy loss, which are verified in Fig. 6 and Fig. 20;  (2) the cases of applying noise to whole parameters, only to softmax classification layer, or other layers, which are verified in Fig. 6 and Fig. 20 .We present theoretical analysis from several possible aspects and finally attribute this finding to the properties of ReLU activation and softmax classification.First, we guess that the sign(θ f ) may have a correlation to the gradient. Specifically, if = ∇ θ f L(θ f ), then given a very smaller λ, we may have the following relationship:EQUATIONwhere the first and third terms denote the gradient ascent step and gradient descent step, respectively. If we set = ∇ θ f L(θ f ) and λ ∈ [-1, 1], the plotted curve may be asymmetric. However, we frustratingly find that sign(θ f ) are almost orthogonal to the ∇ θ f L. On one hand, the converged model θ f has very small gradient values, and elements in ∇ θ f L are nearly zero. On the other hand, adding sign(θ f ) to θ f cannot obtain a lower error than θ f itself, which is not the same as the gradient descent direction. This implies that the inherent properties of them are not the same. Hence, we try other possible explanations.The asymmetry may be related to the inherent asymmetry of the ReLU activation function. According to the analysis in Sect. 5, the gradient of parameters lies in the subspace spanned by the corresponding inputs. Hence, the converged parameters may correlate with their inputs to some extent. Fig. 10 shows the pattern of learned classification weight on the "sklearn.digits". The learned classification weight w shows a pattern of "0" as shown in the figure. However, it is hard for us to provide a concrete expression about the correlation of weight with its corresponding inputs, even for the simplest softmax classification layer. Hence, we assume that the learned w equals to a * h + δ, where a is a constant and δ is a random Gaussian vector. h denotes the hidden representation. And then we simulate the distribution of (w + λ * sign(w)) T h by the following Python code 1. Specifically, we set h ∈ R d as a vector sampled from the distribution of G(0, 1), and set a = 0.1. We take δ as a Gaussian vector sampled from the distribution of G(0, 1). Then we sample N = 10000 groups of δ and plot the distribution of (w + λ * sign(w)) T h under λ ∈ {-1.0, -0.5, 0.0, 0.5, 1.0}. The distributions are shown in Fig. 27 . Obviously, with an increasing λ, the distribution is shifted right. In other words, a negative λ may decrease the value of (w + λ * sign(w)) T h. In ReLU activation, the negative values are not activated, making the loss error change a lot.Listing 1: Simulate ReLU 1 import numpy as np 2 from matplotlib import pyplot as plt Aside from the simulation demo, we also provide the hidden activations for ResNet20 trained on SVHN. The results are shown in Fig. 28 . We plot the activation confusion matrix of θ f +λ * sign(θ f ) with λ ∈ [-1.0, 1.0]. For each λ, we obtain the hidden features extracted by θ f + λ * sign(θ f ). We then compare the features with the original features extracted by θ f and plot the activation confusion matrix. The sum of diagonal values represents the outputs that the original model and the interpolated model commonly activate or do not activate. The value in "[]" shows the sum of diagonal values. Obviously, the value is larger when λ = a than that of λ = -a, with a ∈ {0.2, 0.4, 0.6, 0.8, 1.0}.To conclude, for the ReLU activation, adding sign-consistent noise to parameters will have a higher probability of activating the neurons. If the neuron outputs are only simply scaled by a factor, it will not affect the relative scores of the final classification. For example, the inequation of w T 1 h > w T 2 h will not change if h is scaled by a positive factor, while it does not hold for h whose values are not activated, i.e., h = 0.Given h ∈ R d , the ground-truth label y ∈ [C], and the weight matrix W ∈ R C×d , the softened probability vector is p = softmax(W h). 29 . The test error and cross-entropy loss in the second row show asymmetry, because the second row takes the sign-consistent noise. More specifically, with a positive λ = a, the average trace of P λ is smaller than under a negative λ = -a. Therefore, the trace of H λ=a is smaller than that of H λ=-a . A smaller trace of Hessian means a flatter loss region.The analysis from ReLU and softmax explain the observed phenomenon in this paper, i.e., the sign-consistent noise leads to valley asymmetry and the positive direction is flatter. However, the theoretical insights are not formal proofs, which are future works. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.https://pytorch.org/vision/stable/models.htmlhttps://scikit-learn.org/https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits. html 


Decision: The paper shows under what conditions a minima of the loss surface is asymmetric along a given direction, i.e. when it significantly deviates from a locally symmetric (usually parabolic) shape.

Two reviewers voted for recommendation (7 and 5 scores), while one voted for rejection (score of 4). The discussion phase has not led to reviewers reconciling their opinions.

The discover condition under which a minima is asymmetric seems rather natural and simple, which I see as a strength of the paper.

The applications to model fusion and federated learning are intriguing, as also highlighted by QEpC and underscore the relevance of the studied concepts. For example, whether or not the minima is asymmetric along the interpolation line has a meaningful impact on how easy or hard it is to fuse models (”fine-tuning the pre-trained model does not change the parameter signs a lot, which facilitates the following parameter interpolation”)

Jp1U who voted for rejection remarked that the paper lacks theoretical results (e.g. lack of bounds of the degree of asymmetry). I think that while the paper does emphasizes empirical results and intuitions, it is not in itself a reason to reject the paper.

All in all, I think the paper, while borderline, does clear the bar for acceptance and it is my pleasure to recommend it for this year's program.

Reviews: review: paper_summary: paper_summary: This paper investigates the characteristics of the asymmetric valley in deep neural networks (DNNs) for classification. The authors perform a perturbation analysis around the local minima, considering the direction of the injected noise. The asymmetric valley demonstrates that DNNs exhibit smaller fluctuations with sign-aligned noise, while opposite noise dramatically increases the test error. Other interesting findings include: (1) Batch normalization initialization and hyperparameter choices affect the occurrence of asymmetry. (2)The ReLU activation and softmax function might explain the phenomenon. The authors also leverage the findings to explain the success of model fusion and propose a sign-regularized method for federated learning.
strength_weakness: strength_weakness: 1.	In Section 6.1, the authors claim that pre-training is important for the success of model fusion. However, the compared models have different architectures (VGG16BN vs. ResNet18) and are trained on different datasets (CIFAR10 vs. Flowers). This experimental setting weakens the credibility of the conclusion.
2.	Regarding the conclusion of Fig. 8, “The interpolation curve shows that the small batch training (i.e., λ = 0.0) lies in a sharper and nearly symmetric valley, while the large batch training (i.e., λ = 1.0) lies in a flatter but asymmetric valley,” I find this problematic. Fig. 9 already shows that the valley is asymmetric for BS=32, while Fig. 8 concludes that BS=32 is symmetric. For the possible explanation for the lower part of Fig. 8, my understanding is that BS=2048 has a more biased value distribution, while BS=32 is more zero-centered. Therefore, the same strength of interpolated noise affects BS=32 more. Besides, the degree of asymmetry is not well-defined in the paper, the conclusion based on this comparison might be problematic.
3.	The conclusion from Fig. 8 also conflicts with the statement in related work that “large batch size training may lead to sharp minima with poor generalization.”
questions: questions: See weaknesses.
limitations: limitations: The discussion doesn't include transformer architecture.
paper_summary: paper_summary: This paper explores the factors affecting the symmetry of DNN valleys, encompassing (1) the dataset, network architecture, initialization, and hyperparameters that influence the convergence point; and (2) the magnitude and direction of the noise for 1D visualization. The major contribution is the observation that the degree of sign consistency between the perturbation and the convergence point is a critical indicator of valley symmetry. Theoretical insights from the aspects of ReLU activation and softmax function are provided to explain the asymmetry of DNN valleys. Imposing sign alignment in federated learning is proposed for model parameter alignment.
strength_weakness: strength_weakness: 1. The asymmetry of DNN valleys is not unexpected, and has been studied in [24]. 
2. The observation that sign consistency between the noise and the convergence point affects the valley asymmetry, taking the ReLU activation and softmax function into account, is intuitive to a large degree and hence deos not emerge as a significant contribution. 
3. Lack of theoretical analysis, such as the bound of degree of asymmetry caused by sign consistency.
4. Although the application of sign consistency in federated learning is provided, the readers may be more interested in the implications of valley asymmetry to local search based optimization methods, such as SGD, SAM etc.
questions: questions: 1. Fig.1 is not clear enough.
2. There are two "then"s in line 177.
limitations: limitations: Lack of theoretical analysis and the discussion on the implications of valley asymmetry to local search based optimization methods.
paper_summary: paper_summary: In this work, the phenomenon of asymmetric valleys in deep neural networks (DNNs) minima, first observed and described in [1], is systematically examined. By studying different types of (random) directions around a given minimum, the authors discovered that the degree of sign consistency between the chosen direction and the minimum point is a key factor for asymmetry. Besides thorough empirical evaluation, theoretical reasoning is provided to explain this phenomenon. The findings are also practically applied in model fusion.

[1] Haowei He, Gao Huang, and Yang Yuan. Asymmetric valleys: Beyond sharp and flat local minima. In Advances in Neural Information Processing Systems 32, pages 2549–2560, 2019.
strength_weakness: strength_weakness: I see no concrete substantial weaknesses in this work. I list some of my questions and suggestions in the following section. 

To me, the main shortcoming is weak theoretical support of the claims made in this work. However, the authors acknowledge this, and I hope that future work will address this disadvantage in some way. 

Also, the experiment in Sec. 6.2 about practical implication in federated learning appears to me a little hastily conducted. Some more details would be appreciated (see specific questions below).

The text could be slightly polished to eliminate typos and small inaccuracies.
questions: questions: 1. The Investigation of 6 Special Noise Directions:  
Empirically, noise directions that have the same sign with the positive parameters and applies zero to negative parameters ($\epsilon_5$ and $\epsilon_6$) do not show asymmetry. I wonder, why so? Do the authors have any intuition on that?  
To me, that’s a little counterintuitive, since these directions specifically target the sensitive BN.weight parameters (according to the results of Sec. 4.2.1), don’t they? Also, does the same lack of asymmetry is exhibited by the negative-sign noise directions?
2. The Finding Holds for ImageNet and Various Parameter Groups:  
    a) What type of noise is applied here?  
    b) It is confusing that BN parameters seem like the most “robust to asymmetry” (according to Fig. 6) in light of other experimental results of this paper, underlining the specific role of BN parameters in asymmetry. Even the original work [1] claimed that “directions on BN parameters are more asymmetric”. Could the authors elaborate more on that?
3. Line 186: centered around 0.5?
4. Section 4.2.1:  
Is the noise applied to all parameters or to BN parameters only? If to all, it is indeed remarkable how the initial distribution of BN weights, which take a negligible part in total model parameters, may influence the asymmetry of certain directions (getting back to Question 2b)!
5. Section 5: Did the authors train a plain linear classifier or a neural network in the sklearn.digits experiment?
6. Section 6.2:  
Eq. (1): I might be wrong, but it seems that the regularizer is maximized when the signs of $\theta_t$ and $\theta_t^k$ are aligned, which is undesired, isn’t it?  
The experiments here require stds, as the results presented in Table 1 could lack statistical significance.  
Also, what are the hyperparameters of the baselines, how were they chosen? Were they validated just as $\gamma$ hyperparameter in FedSign? 
7. I suppose that [2], where sign disagreement is also taken into account in model merging, could be a valuable related work.

[1] Haowei He, Gao Huang, and Yang Yuan. Asymmetric valleys: Beyond sharp and flat local minima. In Advances in Neural Information Processing Systems 32, pages 2549–2560, 2019.

[2] Yadav, Prateek, et al. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems 37 (2023).
limitations: limitations: The authors discuss the limitations in Section 7.

Comments: title: Reviewer's response
comment: Thanks for your further responses! Your suggestions are indeed valuable for our future work.

Factually, our article **has already had some preliminary discussions** about your suggestions and concerns.

########

Suggestion 1: About the theoretical analysis and the bound of the degree of asymmetry.

Answer:

(a) We have provided theoretical insights from the Hessian matrix of softmax classification in lines 251-260 and Appendix D.3.

(b) About the degree of asymmetry, we utilized the average error on the left and right sides as a specific metric. This is shown in Fig. 4.

Providing a formal theoretical analysis is the future work **as we stated in the Limitations section**.

########

Suggestion 2: About the application to local search based optimization.

Answer:

(a) The related optimization works (e.g., Sharpness Aware Minimization [16] and Adaptive-SAM [40]) are all **based on the assumption that "flat minima leads to good generalization"**. Our explanation about asymmetry further makes this assumption get into dispute because different noise directions lead to valleys with different shapes, **making the calculation of flatness/sharpness uncertain and non-unique**. Hence, we do not focus on local search based optimization.

(b) On the other hand, our application to federated learning **could also be viewed as a better optimization strategy**, which is to **enhance the ability of model fusion during the local training procedure** in federated learning.

(c) If we really want to apply the findings of this paper to local search based optimization, we can replace the Gaussian noise disturbance used in traditional flatness/sharpness measures with a sign-consistent Gaussian noise, and **design an appropriate flatness/sharpness measure**, and then evaluate whether this appropriate flatness/sharpness measure is related to generalization.
If there is a certain correlation, it may inspire further optimization design. Moreover, this is not a simple verification process and requires a significant amount of future work.

To conclude, considering that **a lot of work has been done on explaining, analyzing, and utilizing asymmetric valleys in this paper**, we authors believe that the workload of the suggestions made by the reviewer can be studied as separate work in the future.

We hope these responses can further address your concerns.
Thank the authors for their replies. I still think this paper can be improved in such aspects as theoretical analysis and insights on its possible applications to local search based optimization. The authors stated theoretical insight is one of the main differences w.r.t literature [24], so I think a formal analysis behind the theoretical intuitions would be important, and the bound of degree of asymmetry, i.e., how large can the asymmetry be and the factors affecting it, is not addressed in the replies. Sharpness-aware minimization etc. utilize the sharpness to improve performance, so I would like to see a discussion on how the asymmetry could possibly be utilized to facilitate optimization or how it affects the optimization. This is also not addressed. Considering these, I  would like to maintain my score.
I thank the authors very much for the detailed and interesting commenting on my questions!

I would like to remain my score unchanged and vote for acceptance.