WHY FLATNESS DOES AND DOES NOT CORRELATE WITH GENERALIZATION FOR DEEP NEURAL NETWORKS 
Authors: Shuofeng Zhang
Isaac Reid
Guillermo Valle-Perez
Ard A. Louis
Venue: ICLR 2022 Submitted (2022)
Keywords: flatness
Bayesian learning
generalization theory

Abstract: The intuition that local flatness of the loss landscape is correlated with better generalization for deep neural networks (DNNs) has been explored for decades, spawning many different flatness measures.  Recently, this link with generalization has been called into question by a demonstration that many measures of flatness are vulnerable to parameter re-scaling which arbitrarily changes their value without changing neural network outputs.  Here we show that, in addition,  some popular variants of SGD such as Adam and Entropy-SGD, can also break the flatness-generalization correlation.  As an alternative to flatness measures,  we use a function based picture and propose using the log of  Bayesian prior upon initialization, $\log P(f)$, as a predictor of the generalization when a DNN converges on function $f$ after training to zero error.   The prior is directly proportional to the Bayesian posterior for functions that give zero error on a test set. For the case of image classification, we show that $\log P(f)$ is a significantly more robust predictor of generalization than flatness measures are. Whilst local flatness measures fail under parameter re-scaling, the prior/posterior, which is global quantity,  remains invariant under re-scaling. Moreover, the correlation with generalization as a function of data complexity remains good for different variants of SGD.

Under review as a conference paper at ICLR 2022
WHY FLATNESS DOES AND DOES NOT CORRE- LATE WITH GENERALIZATION FOR DEEP NEU- RAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
The intuition that local flatness of the loss landscape is correlated with better generalization for deep neural networks (DNNs) has been explored for decades, spawning many different flatness measures. Recently, this link with generalization has been called into question by demonstration that many measures of flatness are vulnerable to parameter reascaiing which arbitrarily changes their value without changing neural network outputs. Here we show that, in addition, some popular variants of SGD such Adam and Entropy-SGID, can also break the flatness- generalization correlation. As an alternative to flatness measures, we use a function based picture and propose using the log of Bayesian prior upon initialization, log P(f), as predictor of the generalization when a DNN converges on function f after training zero error. The prior is directly proportional
the Bayesian posterior for functions that give zero eтror on training set. For the case of image classification, we show that log P(f) is a significantly more robust predictor of generalization than flatness measures are. Whilst local flatness measures fail under parameter re-scaling, the prior/posterior, which is global quantity, remains invariant under re-scaling. Moreover, the correlation with generalization function of data complexity remains good for different variants of SGD.
INTRODUCTION
Among the most important theoretical questions in the field of deep learning are: 1) What charac- terizes functions that exhibit good generalization?, and Why do overparameterized deep neural networks (DNNs) converge to this small subset of functions that do not overfit? Perhaps the most popular hypothesis is that good generalization performance is linked to flat minima. In pioneering works (Hinton & van Camp 1993: Hochreiter & Schmidhuber 1997). the minimum description length (MDL) principle (Rissanen 1978) was invoked to argue that since flatter minima require less information to describe, they should generalize better than sharp minima. Most measures of flatness approximate the local curvature of the loss surface, typically defining flatter minima to be those with smaller values of the Hessian eigenvalues (Keskar al. 2016 Wu et al. 2017 Zhang al. 2018
Sagun et al. 2016: Yao et al. 2018).
Another commonly held belief is that stochastic gradient descent (SGD) is itself biased towards flatter minima, and that this inductive bias helps explain why DNNs generalize so well (Keskar et al.
Jastrzebski et al.) 2018, Wu et al. 2017]; Zhhng al. 2018 Yao et al., 2018 Wei & Schwab 2019 Maddox et al. 2020). For example Keskar et al. (0166) developed a measure of flatness that they found correlated lated with improved generalization performance when decreasing batch size, suggesting that SGD is itself biased towards flatter minima. We note that others (Goyal et al. [017] Hoffer et al. 20177; Smith et al.] [2017] Mingard et al. (201aa) have argued that the effect of batch size can be compensated by changes in learning rate, complicating some conclusions from Keskar et al. (2016). Nevertheless, the argument that SGD somehow itself biased towards flat minima remains widespread in the literature.
2016
In an important critique of local flatness measures, Dinh et al. (2017) pointed out that DNNs with ReLU activation can be re- parameterized through simple parameter-rescalingg transformation.
Ta (W1, W2) (ow1,a-1wa)

---

Under review as a conference paper at ICLR 2022
where W1 are the weights between imput layer and single hidden layer, and wa are the weights between this hidden layer and the outputs. This transformation can extended to any architecture having at least one single rectified network layer. The function that the DNN represents, and thus how it generalizes, is invariant under parameter-rescaling transformations, but the derivatives w.r.t. parameters, and therefore many flatness measures used in the literature, be changed arbitrarily. Ergo, the correlation between flatness and generalization can be arbitrarily changed.
Several recent studies have attempted find "scale invariant" flatness metrics (Petzka al. 2019 Rangamanii et al. 2019; Tsuzuku et al.) 2019). The main idea is to multiply layer-wise Hessian eigenvalues by factor [i which renders the metric immune layer-wise oe-parameterization. While these new metrics look promising experimentally. they are only scale-invariant when the scaling layer-wise. Other methods of rescaling (e.g. neuron-wise rescaling) can still change the metrics, so this general problem remains unsolved.
1.1 MAIN CONTRIBUTIONS
1. For series of classic image classification tasks (MNIST and CIFAR-10) we show that flatness measures change substantially as function of epochs. Parameter re-scaling can arbitrarily change flatness, but quickly recovers to more typical value under further training. We also demonstrate that some variants of SGD exhibit significantly worse correlation of flatness with generalization than found for vanilla SGD. In other words popular measures of flatness sometimes do and sometimes do not correlate with generalization. This mixed performance problematizes a widely held intuition that DNNs generalize well fundamentally because SGD or its variants are themselves biased towards flat minima.
2.
We next study the correlation of the Bayesian prior P(f) with the generalization performance of a DNN that converges to that function f. This prior is the weighted probability of obtaining function f upon random sampling of parameters. Motivated by a theoretical argument derived from non- non-uniform convergence generalization bound, show empirically that log P(f) correlates robustly with test error, even when local flatness measures miserably fail, for example upon parameter re-scaling. For discrete input/output problems (such as classification), P(J) can also be interpreted as the weighted "volume" of parameters that map to f. Intuitively, we expect local flatness measures to typically be smaller (flatter) for systems with larger volumes. Nevertheless, there may also be regions of parameter space where local derivatives and flatness measures vary substantially, even on average they correlate with the volume. Thus flatness measures can be viewed as (imperfect) local measures of a more robust predictor of generalization, the volume/prior P(f).
DEFINITIONS AND NOTATION
2.1 SUPERVISED LEARNING
For a typical supervised learning problem, the inputs live in an input domain X', and the outputs belong to an output space V. For a data distribution on the set of output pairs X' y, the training set S is a sample of m input-output pairs sampled i.i.d. from D, S = [I;; vs) }m ~ Dmm DNN on an input x; is denoted as lr. Typically a
where 2; E X' and y/ E y. The output DNN is parameterized by a vector w and trained by minimizing a loss function L, 3 x ) R. which measures differences between the output ý E y and the ground truth output y E Y/ by assigning score L(ğ. y) which is typically zero when they match, and positive when they don't match. Minimizing the loss typically involves using an optimization algorithm such as SGD on training set S. The generalization performance of the DNN, which is theoretically defined over the underlying (typically unknown) data distribution DD but is practically measured on a test ser E {(2%, | E For classification problems, the generalization error is practically measured as <(E) TET EntEr L[g + v/]. where 1 is the standard indicator function which is one when its input true, and zero otherwise.
2

---

Under review as a conference paper at ICLR 2022
2.2 FLATNESS MEASURES
Perhaps the most natural way to measure the flatness of minima is to consider the eigenvalue distribu- tion of the Hessian Hi 8 ((w)/w,;üu, once the learning process has converged (typically to a zero training error solution). Here for simplicity we use L(w) instead of L( y) as ÿ parameterized by W. Sharp minima are characterized by a significant number of large positive eigenvalues in the Hessian, while flat minima are dominated by small eigenvalues. Some care must be used in this interpretation because it is widely thought that DNNs converge to stationary points that are not true minima, leading to negative eigenvalues and complicating their use in measures of flatness. Typically, only subset of the positive eigenvalues are used (Wu et al. 2017 Zhang et al. 2018). Hessians typically very expensive calculate. For this reason, Keskar al. (2016) introduced computationally more tractable measure called "sharpness":
Definition 2.1 (Sharpness). Given parameters w' within box in parameter space Cg with sides of length Ç > 0, centered around a minimum of interest at parameters w, the sharpness of the loss L(w) at w is defined as:
sharpness: == maxw EC2 (L(w') L(w)) 100.
L(w)
In the limit of small Ç, the sharpness relates the spectral norm of the Hessian (Dinh al. 2017
[I((" 100.
sharpness x 2(1 I(w))
The general concept of flatness be defined 1/ sharrpesss, and that is how we will interpret this measure in the rest of this paper.
2.3 FUNCTIONS AND THE BAYESIAN PRIOR
We first clarify how we represent functions in the rest of paper using the notion of restriction of functions. A more detailed explanation can be found in appendix [ as example:
Here we use binary classification
Definition 2.2 (Restriction of functions to C?), (Shalev-Shwartz & DeviD 2014) Consider a parameterized supervised model, and let the input space be X' and the output space be Y/ noting y {0, 1} in binary classification setting. The space of functions the model can express is (potentially uncountably infinite) set F S ylæi, Let C fci..... C. C X. The restriction of F to C' is the set of functions from to y that can derived from functions in F:
Fo = {(f (ci) ..... (cm)) e F}
where we represent each function from C' to y as vector in yIC),
For example, for binary classification, if we restrict the functions to S + E, then each function in Fs+B represented binary string of length |S| + EE. In the rest of paper, we simply refer to "functions" when we actually mean the restriction of functions to S + E, except for the Boolean system in section S.I where no restriction is needed. See appendix (ariixx(oooo [or a thorough explanation.
For discrete functions, we next define the prior probability P(f) as Definition 2.3 (Prior of a function). Given a prior parameter distribution (ww)) over the parameters, the prior of function f can be defined as:
P(f) / 1[M(w) (W)...
where 1 is an indicator (ommction1[[rrp = 1 if its argument is true or 0 otherwise; is the parameter-
function map whose formal definition is in appendix B Note that P(f) could also be interpreted as a weighted volume (ff) over parameter space. If P.. (w) (w) is the distribution at initialization, the P(f)
the prior probability of obtaining the function at initialization. We normally use this parameter distribution when interpreting P(f)-
3

---

Under review as a conference paper at ICLR 2022
Vsco(f)
fi
f2
fa
L(W)
Zero Erro
V(h)
W
Figure 1: Schematic loss landscape for three functions that have zero-error the training set. illustrates how the relative sizes of the volumes of their basins of attraction VSGD (ft) correlate with the volumes (fi)) (or equivalently their priors P(f.)) of the basins, and that, average, larger V(fi) or P(ft) implies flatter functions, even if flatness can vary locally. Note that the loss L(w) can vary within region where the DNN achieves can be seen in (Mingard et al. 2021b), but here it is more clear that local flatness can be misleading. Note the "error" means the classification error, which is different from the loss.
classification
on S. A similar schematie plot
Remark. Definition 2.3 works in the situation where the space X' and are discrete, where P(f) has prior probability mass interpretation. This is enough for most image classification tasks. Nevertheless, we can easily extend this definition to continuous setting, where we can also define a prior density over functions upon random initialization, with the help of Gaussian Process (GP) (Rasmussen) 2003). For the GP prior see appendix DD However, in this work, we focus exclusively on the classification setting, with discrete inputs and outputs.
2.4 LINK BETWEEN THE PRIOR AND THE BAYESIAN POSTERIOR
Due to their high expressivity, DNNs are typically trained to zero training error on the training set S. In this case the Bayesian picture simplifies because if functions are conditioned on zero error on S, this leads to a simple 0-1 likelihood P(S(f), indicating whether the data is consistent with the function. Bayesian inference can be used to calculate Bayesian posterior probability PB(f(S) for each by conditioning on the data according to Bayes rule. Formally, if S {(x; 9;)) corresponds to the set of training pairs, then
P(F)/P(S) if, f(x;) = Pb(f(S) (o otherwise.
where P(J) is the Bayesian prior and P(S) is called the marginal likelihood or Bayesian evidence. If we define, the training set neutral space N's as all parameters that lead to functions that give zero training error on S, then P(S) JNS Pw (w) (w)dw. In other words, it is the total prior probability of all functions compatible with the training set S (Valle-Pérez et al. 2018 Mingard et al. 2021a). Since P(S) is constant for a given S, (f())) x P(J) for all consistent with that S.
3 THE CORRELATION BETWEEN THE PRIOR AND GENERALIZATION
This link between the prior and the posterior is important, because it was empirically found in an extensive set of experiments by (Mingard et al. 2021a) that, for popular architectures and data sets,
PB(J(S) PSGD(J\SS)
where Ban(f1S) is the probability that a DNN trained with SGD converges function f. when trained to zero error on S. In other words, to first order, SGD appears to find functions with a probability predicted by the Bayesian posterior, and thus with probabilities directly proportional to
P(f). The authors traced this behaviour to the geometry of the loss landscape, as follows. Some general observations from algorithmic information theory (AIT) (Valle-Péren et al. 2018) well
4

---

Under review as a conference paper at ICLR 2022
as direct calculations (Mingard et al. 2019) predict that the priors of functions should vary over many orders of magnitude. When this is the case, it is reasonable to expect that the probabilities by which an optimizer finds different functions is affected by these large differences. This is related to a mechanism identified previously in evolutionary dynamics, where it is called the arrival of the frequent (Schaper & Louis 2014). We illustrate this principle in fig. [ where we intuitively use the language of "volumes". We expect that the relative sizes of the basins of attraction Vs (f), defined as the of initial parameters for which a DNN converges to a certain function f. proportional, to first order, to those of the priors P(f) (or equivalently the "volumes") To second order there are, of course, many other features of a search method and landscape that affect what functions a DNN converges on, but when the volumes/priors vary by so many orders of magnitude then we expect that to first order (sa)( Rs PB(J x P(J)= V(').
Given that the P(f)) of function helps predict how likely SGD is to converge on that function, we can next ask how P(f) correlates with generalization. Perhaps the simplest argument is that if DNNs trained to zero error are known to generalize well on unseen data, then the probability of converging on functions that generalize well must be high. The P(f) of these functions must be larger than the priors of functions that do not generalize well. Can we do better than this rather simplistic argument? One way forward is empirical. [Minaard et al. (2021a) showed that log (P correlates quite tightly with generalization error. These authors also made a theoretical argument based the Poisson-Binon Binomial nature of the error distribution to explain this log-linear relationship, but this approach needs further work.
One of the best overall performing predictors in the literature for generalization performance on classification tasks the marginal likelihood PAC-Bayes bound from Valle-Pérez et al.] (2018); Valle-Pérez & Louis (202)). It is non-vacuous, relatively tight, and can capture important trends in generalization performance with training set size (learning curves), data complexity, and architecture choice (see also (Liu et al. (021)). However, the prediction uses the marginal likelihood P(S) defined through a sum over all functions that produce zero error on the training set. Here we are interested in the generalization properties of single functions. One way around is to use a simple nonuniform bound which to the best of our knowledge was first published in McAllester (1998) as preliminary theory to the full PAC-Bayes theorems. For any countable function space F, any distribution P. and for any selection of training set S of size m under probability distribution D, it can be proven that for all functions| of that give zero training error:
In
+ In 1
YD, Ps Dm
{5, S
P(f)
21-8
m
for 8 (0, 1). Here we consider space Fs, E of functions with all possible outputs on the inputs of a specific E and zero error on a specific S; s.) is the error measured on E + S, which as the error on S O, equals the error on the test set E. This error will converge to the true generalization егror on all possible inputs as [E] increases. Valle-Pérez & Louis (2020) showed this bound has an optimal average generalization error when P(ff) mimics the probability distribution over functions of the learning algorithm. If PSCD(f) se PB(f(S) x P(f), then the best performance of the bound is approximately when P(f) in eq. (4) is the Bayesian prior P(f). Thus this upper bound on ES scales as log (P())).
4 FLATNESS, PRIORS AND GENERALIZATION
The intuition that larger P(f) correlates with greater flatness is common in the literature, see e.g. |Hochriterr & Schmidhuber (197)); Wu et al. (017)), where the intuition is also expressed in terms of volumes. If volume/P(f (f) correlates with generalization, we expect flatness should too. Nevertheless, local flatness may still vary significantly across a volume. For example Izmailov et al. (2018) show explicitly that even in the same basin of attraction, there can be flatter and sharper regions. We illustrate this point schematically in fig. I where one function clearly has larger volume and on average smaller derivatives of the loss w.r.t. the parameters than the others, and so is flatter on average. But, there are also local areas within the zero-error region where this correlation does not hold. One of the main hypotheses we will test in this paper is that the correlation between flatness and generalization can be broken even when the generalization--rion correlation remains robust.
5

---

Under review as a conference paper at ICLR 2022
#outllers moutliers or
loshafsherrpeesss
2.
P01
The correlation between flatness and the Bayesian prior for the 7 Boolean system. The functions are defined on the full space 128 possible inputs. The priors P()) are shown for the 1000 most frequently found functions by SGD from random initialization for a two hidden layer FCN, and correlate well with log(flatness). The function with the largest prior, which is the most "flat" is the trivial one of all Os or all 1s. An additional feature is two offset bands caused by a discontinuity of Boolean functions. Most functions shown are mainly Os or mainly 1s, and the two bands correspond to different number of outliers (e.g. Is when the majority is Os).
Figure 2:
EXPERIMENTAL RESULTS
5.1 PRIOR/VOLUME FLATNESS CORRELATION FOR BOOLEAN SYSTEM
We first study model system for Boolean functions of size n 7, which is small enough to directly
measure the prior by sampling Pérle-Pérez et al. 2018). There are 2 = 128 possible binary inputs. Since each input is mapped single binary output, there are 2 3.4 10 possible functions It is only practically possible to sample the prior P(f) because it is highly biased (Valle-Pérez et al. 2018 M0018Mingard et al., 2019), meaning a subset of functions have priors much higher than average. For a fully connected network (FCN) with two hidden layers of 40 ReI.U units each (which was found to be sufficiently expressive to represent almost all possible functions) we empirically determined P(f) using 10t random samples of the weights over initial Gaussian parameter distribution (w) with standard deviation ow = 1.0 and σか = 0.1.
We also trained our network with SGD using the same initialization and recorded the top-1000 most commonly appearing output functions with zero training error on all 128 outputs, and then evaluated the sharpness/fatnnss using definition 2.1 with an 10-4 4 For the maximization process in calculating sharpness/fatness, we ran SGD for 10 epochs and make sure the max value ceases to change. As fig. 2 demonstrates, the flatness and prior correlate relatively well; fig. [S8 in the appendix shows a very similar correlation for the spectral norm of the Hessian. Note that since we are studying the function on the complete input space, is not meaningful to speak of correlation with generalization. However, since for this system the prior P(f) is known to correlate with generalization (Mingard al., 2021a), the correlation in fig. 2 also implies that these flatness measures will correlate with generalization, at least for these high P(f) functions.
5.2 PRIORS, FLATNESS AND GENERALIZATION FOR MNIST AND CIFAR-10
We next study the correlation between generalization, flatness and log P(f) on the real world datasets MNIST (LeCun et al., 1998) and CIFAR- 10 (Krizhevsky et al. 2009). Because we need run many different experiments, and measurements of the prior and flatness are computationally expensive, we simplify the problem by binarizing MNIST (one class 0-4, the other 5-9) and CIFAR-10 10 (we only study two categories out of ten: cars and cats), Also, our training sets relatively small (500/5000 for MNIST/CIFAR-ID respectively) but we have checked that our overall results are not affected by these more computationally convenient choices. In appendix fig. S24 we show results for MNIST with [S| = 10000.
We use two DNN architectures: relatively small vanilla two hidden- layer FCN with 784 inputs and 40 ReLU units in each hidden layer each, and also Resnet-50 (He et al. 2016), a 50-layer deep convolutional neural network, which is much closer to state of the art (SOTA) system.

---

Under review as a conference paper at ICLR 2022
(a)
(b)
(c)
Glnaraa
FONMNNSST
CN-
(d)
(c)
Figure 3: The correlation between log P(f), sharpness and generalization accuracy MNIST and CIFAR-10. For MNIST |S|=500, =10=0, for CIFAR-10 |S) =S]00 |E]=0000. The attack set size |A| varies from 0 [S] and generates functions with different generalization performance. (a)-(c) depicts the correlation between generalization and log P(f) for FCN on MNIST, FCN on CIFAR-10 10 and Resnet-50 on CIFAR- 10, respectively. (d)-(f) show the correlation between generalization and flatness for FCN on MNIST, FCN on CIFAR-10, and Resnet50 on CIFAR-10. respectively. In this experiment, all DNNs are trained with vanilla SGD.
We measure the flatness cross-entropy (CE) loss the epoch where SGD first obtains training error. Because the Hessian is so expensive to calculate, we mainly use the sharpness/faatnes measure (definition 2.1 which is proportional the Frobenius norm of the Hessian. The final error is measured in the standard way, after applying sigmoid to the last layer to binarize the outputs.
To measure the prior, use the GPs which these networks reduce the limit of infinite width (Lee) et al. 2017; Matthews et al. 2018 Novak et al. (2018b). As demonstrated in Mingard et al. (2021a), GPs can be used approximate the Bayesian posteriors (B())S) for finite width networks. For further details, we refer to the original papers above and to appendix D
In order to generate functions / with zero emror on the training set S, but with diverse generalization performance, we use the attack-set trick from Wu et al. (2017). In addition to training on S, we add an attack set made up incorrectly labelled data. We train on both S and A, so that the error on zero but the generalization performance on test set E is reduced. The larger A is w.r.t. S, the worse the generalization performance. As can be seen in fig. (()-(c), this process allows us to significantly vary the generalization performance. The correlation between log P(f) and generalization error is excellent over this range, as expected from our arguments section
Figs 3 df) show that the correlation between flatness and generalization is much more scattered than for log P(f). In appendix G we also show the direct correlation hetween log P(f) and flatness which closely resembles fig. d) (d-(() because V(J) and correlate so tightly.
5.3 THE EFFECT OF OPTIMIZER CHOICE ON FLATNESS
We then test the effect of changing the optimizer from the vanilla SGD we used in fig. 3 We use Adam (Kingma& & Ba 2014), and entropy-SGD (Chaudhari et al.) 2019) which includes explicit term to maximize the flatness. Both SGD variants show good optimization performance for the standard default Tensorflow hyperparameters does not significantly vary from plain SGD, and this is reflected the priors the functions that they find. More importantly, fig. shows that the generalization siaatataatinonsssessss flatness correlation can be broken by
Their generalization performance, however,
7

---

Under review as a conference paper at ICLR 2022
Conmollaation
FCNMANIST/
FORIMNISTI Ad5m
FCRCIFAA-11
Entropy 500
Adom
lagu///0
(a)
(b)
(c)
FCNIMMSST
FCNIMNISI
FOWCIIAAR-10
Entropy
@gplcharpiisat
logistharpp heseessesess
(d)
(c)
Figure SGD-variants break the flatness ceatess-gennraaliaation correlation, but not the log P())-
generalization correlation. The figures show generalization v.s. log P(J) or flatness for the FCN
trained on (a) and (d) MNIST with Entropy- (b) and (e) - MNIST with Adam; (c) and (f)
CIFAR-10 with Adam. for the same S and E as in fig. 3 Note that the correlation with the prior is virtually identical vanilla SGD, but that the correlation with flatness measures changes significantly.
using these optimizers, whereas the log gennealalali correlation remains intact. A similar breakdown of the correlation persists upon overtraining and can also be seen for flatness measures that use Hessian eigenvalues (fig. S16 to fig. (21).
Changing optimizers or changing hyperparameters can, of course, alter the generalization perfor- mance by small amounts, which may be critically important in practical applications. Nevertheless, as demonstrated in Mingard al. (2021a), the overall effect of hyperparameter or optimizer changes usually quite small on these scales. The large differences in flatness generated simply by changing the optimizer suggests that flatness measures may not always reliably capture the effects hyperpa- rameter optimizer changes. Note that find less deterioration when comparing SGD Adam for Resnet50 on CIFAR-10, (fig. S22 The exact nature of these effects remains subtle.
5.4 TEMPORAL BEHAVIOR OF SHARPNESS AND log P(J)
In the experiments above, the flatness and log P()) metrics calculated at the epoch where the system first reaches 100% training accuracy. fig. we measure the prior and the flatness for each epoch for our FCN, trained on MNIST (with no attack set). Zero training error reached epoch 140, and we overtrain for further 1000 epochs. From initialization, both the sharpness measure from Definition 2. and log P(J) reduce until zero-training error is reached. Subsequently, log P(f) stays constant, but the CE loss continues to decrease, expected for such classification problems. This leads to reduction in the sharpness measure (greater flatness) even though the function, its prior, and the training error don't t change. This demonstrates that flatness is relative concept that depends, for example, on the duration of training. f and [ST7 we show for an FCN on MNIST that the quality of flatness flatness-genera generalization correlations are largely unaffected by overtraining, for both SGD and Adam respectively, even though the absolute values the sharpness change substantially.
One of the strong critiques of flatness is that re-parameterizations such the parameter-rescallnng transformation defined in eq. ( can arbitrarily change local flatness measures (Dinh et al. 2017). Fig. [5 shows that darameter-rescallee reduction in flatness). As demonstrated in the inset, the prior is initially invariant upon parameter- rescaling because f(w) is unchanged. However, parameter-rescaling can drive the system unusual
indeed leads to a spike in the sharpness measure (a strong
8

---

Under review as a conference paper at ICLR 2022
Baycsian prior
0.0
600
-03
(og)oP(n
-60
sharpeass
-10
-100
-120
-140
-25
-160
200 400
600
800
1000
Epochs
Figure 5: How flatness evolves with epochs. At each epoch we calculate the sharpness measure from Definition 2.1 (sharpness is the inverse of flatness) and the prior for our FCN on MNIST with [S| 500. The green dashed line denotes epoch 140 where zero-training error is reached and post- training starts. The red dashed line denotes epoch 639 where purameter-rescaliig takes place with 3.9. Upon parameter-rescalingg rescaling. the sharpness increases markedly, but then quickly decreases again. The inset shows that the prior is initially unchanged after parameter-rescaling. However, large gradients mean that subsequent SGD steps, the function (and its prior) changes, before recovering to (nearly) the same function and log P(f).
parts of the volume with steep gradients in the loss function, which mean that SGD falls off the zero training error manifold. log P(f) goes up because is more likely to randomly fall onto large V(f) functions. However, the system soon relaxes to essentially the same function and log P(f). In fig. STI we show that it is possible prior changing. In each case, the sharpness measure rapidly decays after the spike, suggesting that parameter-rescaling brings the system into parameter region that "unnatural".
obtain spike in the sharpness measure without the
DISCUSSION AND FUTURE WORK
The notion that flatness correlates with generalization is widely believed in the community, but the evidential basis for this hypothesis has always been mixed. Here we performed extensive empirical work showing that flatness can indeed correlate with generalization. However, this correlation not always tight, and can be easily broken by changing the optimizer, or by parameter-rescaling By contrast, the P(J) which directly proportional the Bayesian posterior (J())) for functions that give zero error the training set, a much more robust predictor of generalization.
While the generalization performance a DNN can be successfully predicted by the marginal likelihood PAC-Bayes bound (Valle-Pérez et al. 2018 Valle-Pérez & Louis 2020), no such tight bound exists (to our knowledge) linking generalization and the Bayesian prior or posterior at the level of individual functions. Further theoretical work in this direction is needed. Moreover, it natural to further extend current work towards linking flatness and the prior to other quantities which correlate with generalization such as frequency (Rahaman et al. 2018 Xu et al. 2019), the sensitivity to changes in the inputs (Arpit et al.) 2017] Novak et al.] 2018a) Improvements to the GP approximations we use are important technical goal. P(f) can be expensive calculate, so finding reliable local approximations related to flatness may still be a worthy endeavour. Finally, our main result that log P(f) correlates well with generalization still requires proper theoretical underpinning, notwithstanding the bound in (4). Such explanations will need to include not just networks and the algorithms, but also the data (Zdeborová 2020). We refer readers to appendix A for more discusion on related works.

---

http:////wweeeeeeeeeeeeeeeeeeeee Under review as a conference paper at ICLR 2022
REFERENCES
Mauricio A Alvarez. Lorenzo Rosasco, and Neil D Lawrence. Kernels for vector- valued functions: review. arXiv preprint arXiv: 6251, 2011.
Devansh Arpit, Stanistaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, al. A closer look memorization in deep networks. arXiv preprint arXiv:/706.0553 2017.
Peter Bartlet, Dylan Foster, Matus Telgarsky. Spertrally-normalizzed margin bounds for neural networks. In Advances in Neural Information Processing Systems, pp. 6240-6249, 2017.
Leo Breiman. Reflections after refereeing papers for nips. In The Mathematics of Generalization, pp. 11-15. Addison-Wesley. 1995.
Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal scientific computing, 16(5):1190-1) 1995.
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):124018 2019.
Youngmin Cho and Lawrence Saul. Kernel methods for deep learning. Advances neural information processing systems, 22:342 350, 2009.
Giacomo De Palma, Bobak Toussi Kiani, and Seth Lloyd. Random deep neural networks are biased towards simple functions. arXiv preprint arXiv: 1812. 10156, 2018.
Kamaludin Dingle. Steffen Schaper, and Ard A Louis. The structure of the senootye-phhentyyee map strongly constrains the evolution of non-coding ma. Interface focus, 5(6):20150533, 2015.
Kamaludin Dingle, Chico Q Camargo, and Louis. Input-output output maps are strongly biased towards simple outputs. Nature Communications, 91): 2018.
Kamaludin Dingle, Guillermo Valle Pérez, and Ard A Louis. Generic predictions of output probability based on complexities of imputs and outputs. Scientific Reports, 10(1):1-9, 2020.
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Proceedings of the 34th International Conference on Machine Learning-Voir Volume 70, pp. 1019- 1028. JMLR. org, 2017.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(Jul::121-21150 2011.
Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep
(stochastic) neural networks with many more parameters than training data. In Proceedings of the Thiry Conference on Uncertainty Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017, 2017. URL UtRLhttp::// //att:://aaa...... crg. 1atai1017.. http:/aaac.coaaa......oooooeeeeo
Adrià Garriga-Alonso, Carl Edward Rasmussen, and Laurence Aitchison. Deep convolutional networks shallow gaussian processes. In International Conference Learning Representations, 2019. URL
Sebastian Goldt, Mare Mézard, Florent Krzakala, and Lenka Zıleborová. Modelling influence of data structure on learning in neural networks. arXiv preprint arXiv: 15500, 2019.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint 2017.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Geoffrey E Hinton and Drew van Camp. Keeping neural networks simple. In International Conference on Artificial Neural Networks, pp. 11-18. Springer, 1993.
Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural Computation, 9(1):1-42, 1997.
10

---

Under review as a conference paper at ICLR 2022
Elad Hoffer, Ita Hubara, and Daniel Soudry. Train longer, generalize better: closing the generalization gap in large batch training neural networks. In Advances neural information processing systems, pp. 1731-1741, 2017.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint urXiv1100.03166. 2015.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson, Averaging weights leads to wider optima and better generalization. arXiv preprint
2018.
Stanislaw Jastrzebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Finding flatter minima with sgd. In ICLR (Workshop). 2018
Yiding Jiang. Dilip Krishnan, Hossein Mohahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. arXiv preprint arXiv: /8/0.00/// 2018.
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. arXiv preprint arXiv: 1912.0278, 2019.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint 1609 2016.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint ar:.:: 1422:::412.66980 2014.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-basee learning applied to document recognition. Proceedings of the IEEE, 86(11 (6(11):2278-2322 1998.
Jachoon Lec, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl- Dickstein. Deep neural networks as gaussian processes. arXiv preprint 1rXiv:1711.00// 2017.
L.A. Levin. Laws of information conservation (nongrowth) and aspects of the foundation of probability theory. Problemy Peredachi Informatsii, 10(3):30-33 35, 1974.
M. Li and P.M.B. Vitanyi. An introduction to Kolnogorov complexity and its applications. Springer- New York Inc, 2008
Qianli Liao, Brando Miranda, Andrzej Banburski, Jack Hidary, and Tomaso Poggio. A surprising linear relationship predicts test performance in deep networks. arXiv preprint arXiv:1807.09659 2018.
Henry W Lin, Max Tegmark, and David Rolnick. Why does deep and cheap learning work well? Journal of Statistical Physics, 1223-12722222 2017.
Yang Liu, Jeremy Bernstein, Markus Meister, and Yisong Yue. Learning by turning: Neural architecture aware optimisation. arXiv preprint #xXi:2200200227, 2021.
David JC. Mackay. Introduction to gaussian processes. NATO ASI series. Series F: computer and system sciences. pp. 163,,6,,7 1998.
Wesley J Maddox, Gregory Benton, and Andrew Gordon Wilson. Rethinking parameter counting in deep models: Effective dimensionality revisited. arXiv preprint arXiv:2003.0211 2020.
Susanna Manrubia, José A Cuesta, Jacobo Aguirre, Schastian E. Ahnert, Lee Altenherg, Alejandro V Cano, Pablo Catalán, Ramon Diaz- Uriarte. Santiago Elena, Juan Antonio Gareía-Martin, et al. From genotypes to organisms: ofate-of-the-ar and perspectives of a cornerstone in evolutionary dynamics. arXiv preprint arXiv:2002.00360 2020.
Alexander G de G Matthews, Mark Rowland, Jiri Hron, Richard E Turner, and Zoubin Ghahramani. Gaussian process behaviour in wide deep neural networks. arXiv preprint 1804v: 1e.111844211/1 11711, 2018
David A McAllester. Some pac-bayesian thcorems. In Proceedings of the eleventh annual conference on Computational learning theory, pp. 230-234. ACM, 1998.
Chris Mingard, Joar Skalse, Guillermo Valle-Péres David Martinez-Ran Rubio, Vladimir Mikulik, and Ard A Louis. Neural networks are priori biased towards boolean functions with low entropy. arXiv preprint 19Xiv: 1152,, 2019.
11

---

Under review as a conference paper at ICLR 2022
Chris Mingard, Guillermo Valle-Pérez, Joar Skalse, and Ard A Louis. Is sgd bayesian sampler? well, almost. Journal of Machine Learning Research, 22((9):1-66 2021a.
Chris Mingard, Guillermo Valle-Pérez. Joar Skalse, and Ard A Louis. Is sgd bayesian sampler? well, almost. Journal of Machine Learning Research, 22(79):1-64, 2021h.
Radford M Neal. Priors for infinite networks (tech. rep. erg-tr-94-1). University of Toronto, 1994.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. A pac-bayesian approach to ppectrally-normalle margin bounds for neural networks. arXiv preprint a7X1v:1707.09564, 2017a.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In Advances in Neural Information Processing Systems, pp. 5949-5958, 2017b
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv:/802.0000 /802.08600 2018a.
Roman Novak, Lechao Xiao, Jaehoon Lee. Yasaman Bahri, Greg Yang, Jiri Hron, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl- Dickstein. Bayesian deep convolutional networks with many channels are gaussian processes. arXiv preprint 1rX1v:1810.05148, 2018b.
Henning Petzka, Linara Adilova, Michael Kamp, and Cristian Sminchisescu. A www.rnettrrriattoiiiiiiiaa flatness measure for deep neural networks. arXiv preprint arXiv: 1912.0009100 2019.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. arXiv preprint arXiv: 0888064474, 2018.
Akshay Rangamani, Nam H Nguyen, Abhishek Kumar, Dzung Phan, Sang H Chin, and Trac D Tran. A scale invariant flatness measure for deep network minima. arXiv preprint arXiv: / /902.02434 2019.
Carl Edward Rasmussen, Gaussian processes in machine learning. In Summer School on Machine Learning, pp- 63-71. Springer, 2003.
Jorma Rissanen, Modeling by shortest data description. Automatica, 14(5):465-471, 1978.
David E. Rumelhart, Geoffrey E. Hinton, and Ronald Williams. Learning representations by back-propagating errors. Nature, 323(6088) 1986.
Levent Sagun, Leon Bottou, and Yann LeCun. Eigenvalues of the hessian in deep learning: Singularity and beyond. arXiv preprint arXiv:1611.07470 2016.
Steffen Schaper Ard Louis. The arrival the frequent: how bias in manotye--phenotye maps can steer populations to local optima. PloS one, 9(2)x866335, 2014,
Shai Shwarzz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
Samuel Smith, Picter-Jan Kindermans, Chris Ying, and Quoe V Le. Don't decay the learning rate, increase the batch size. arXiv preprint arX.v:/711.00489, 2017.
Stefano Spigler, Mario Geiger, and Matthieu Wyart. Asymptotic learning curves of kernel methods: empirical data vs teacher-student paradigm. arXiv preprint arXiv:1905. 10843, 2019.
Mingyue Tan. Expectation propagation of gaussian process classification and its application to gene expression analysis. 01 2008
Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2):26-31, 2012.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Normalized flat minima: Exploring scale invariant definition of flat minima neural networks using pac-bayesian analysis. arXiv preprint wwXiv:1901.000 2019.
Guillermo Valle-Pérez and Ard A Louis. Generalization bounds for deep learning. arXiv preprint arXiv:2012.04115. 2020.
Guillermo Péree-Pérez, Chico Camargo, and Ard Louis. Deep learning generalizes because the parameter- function map biased towards simple functions. arXiv preprint arXiv:/805.00 0055228 2018.
Mingwei Wei and David J Schwab. How noise affects the hessian spectrum in overparameterized neural networks. 2019.
arXiv preprint
12

---

Under review as a conference paper at ICLR 2022
Lei Wu, Zhanxing Zhu, et al. Towards understanding generalization of deep learning: Perspective of loss landscapes, arXiv preprint arXi:: 100.99999, 2017.
Zhi-Qin John Xu, Yaoyu Zhang. Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint 1rXXi::901.00523 2019.
Greg Yang. Wide feedforward recurrent neural networks of any architecture gaussian processes. In H. Wal- lach, H. Larochelle, A. Beygelzimer, d'Alché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems, volume 32, Curran Associates, Inc., 2019. URL ht tps http:tttttpooeeeeeeeeeeeo /ppoccede/g/coogg
Greg Yang and Hadi Salman. A fine-grained spectral perspective on neural networks. arXiv preprint arXiv:/907, 100,09990,,1099, 2019.
Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, and Michael W Mahoney. Hessian-basee analysis of large batch training and robustness to adversaries. In Advances in Neural Information Processing Systems, pp. 4949-4959, 2018.
Lenka Zdeborová. Understanding deep learning is also job for physicists. Nature Physics, 16(6):602-00) 604, 2020.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv: 1.1003500, 2016.
Yao Zhang, Andrew M Saxe, Madhu S Advani, and Alpha A Lee. Energy-entnn entropy competition and the effectiveness of stochastic gradient descent in machine learning. Molecular Physics, 116621-23):314- 3223, 2018
13

---

Under review as a conference paper at ICLR 2022
A MORE RELATED WORK
A.1 PRELIMINARIES: TWO KINDS OF QUESTIONS GENERALIZATION AND TWO TYPES OF INDUCTIVE BIAS
In this supplementary section we expand on our briefer discussion of related work in the Introduction of the main paper. The question of why and how DNNs generalize in the overparameterized regime has generated a vast literature. To organize our discussion, we follow (Mingard al. 202 and first distinguish two kinds of questions about generalization overparameterized DNNs:
1) The question of over perr-prraeterized generalization: ized regime, where classical learning theory suggests they should heavily overfit.
Why do DNNs generalize at all the overparameter-
2) The question of fine-tuned generalization: Given that a DNN already generalizes reasonably well, how can detailed architecture choice. optimizer choice, and hyperparameter tuning further improve generalization?
Question 2) is the main focus of large tranche of the litcrature on generalization, and for good reason. In order to build SOTA DNNs, even a few pereent accuracy improvement (taking image classification as an example) is important in practice. Improved generalization performance can be achieved in many ways, including local adjustments of the DNNs structure (e-g. convolutional layers, pooling layers, shorteut connections etc.), hyperparameter tuning (learning rate, batch size etc.), or choosing different optimizers (e.g. vanilla SGD versus entropySGD (Chaudhari et al. 2019) or Adam (Kingma & Ba 2014). In this paper, however, we are primarily interested in question 1). As pointed out, for example famously in Zhang et al. (2016). but also by many researchers before that DNNs can be proven to be highly expressive, that the number of hypotheses that fit training data set S. but generalize poorly, typically many orders of magnitude larger than the number that can actually generalize. And yet in practice DNNs do not tend to overfit much, and can generalize well, which implics that DNNs must have some kind of inductive bias hypotheses that generalize well on unseen data.
& Ben-David 2014) toward
Following the framework of Minnardd et al. 2021a) a), we use the language of functions (rather than that of hypotheses, see also appendix
distinguish two major potential types of inductive bias.
A) The inductive bias upon upon random sampling parameters over a parameter distribution (w). In other words, given a DNN architecture, loss function etc. and a measure over parameters P... (ww( (which can be taken to be the initial parameter distribution for optimizer, but is more general), this bias occurs when certain types of functions more likely appear upon random sampling of parameters than others. This inductive bias can expressed in terms of prior over functions P(f), or in terms of a posterior Pe (f() when the functions are conditioned, for example, on obtaining zero eтror on training set S.
B) The inductive bias induced by optimizers during a training procedure. In other words, given an inductive bias upon initialization (from A), does the training procedure induce a further inductive bias on what functions a DNN expresses? One way of measuring this second form of inductive bias is to calculate the probability (fp((f])) that an DNN trained to zero error on training set S with optimizer opt (typically variant of SGD) expresses function f. and then compare it to the Bayesian posterior probability (J()))) that this function obtains upon random sampling of parameters (Mingard et al. 2021a). In principle Ps (J(()) expresses the inductive hias of type A), any differences between Popt (f[S) and Po(f(S) could he due to inductive biases of type B).
These two sources of inductive bias can be relevant to both questions above about generalization. We emphasize that our taxonomy of two questions about generalization, and two types of inductive bias is just one way of parsing these issues. We make these first order distinctions to help clarify our discussion of the literature, and are aware that there other ways of teasing out these distinctions.
A.2 RELATED WORK ON FLATNESS
The concept "flatness" of the loss function of DNNs can be traced back to Hinton & van Camp (1933) and Hochreiter & Schmidhuber ( (197). Although these authors did not provide a completely formal mathematical detinition of flatness, Hochreiter & Schmidhuber (1997) described flat minima as "a large connected region parameter space where the loss remains approximately constant", which requires lower precision to specify than sharp minima. They linked this idea to the minimum description length (MDL) principle (Kissanen] 1978). which says that the best performing model is the one with shortest description length, to argue that flatter minima should generalize better than sharp minima. More generally, flatness can be interpreted as complexity control of the hypotheses class introduced by algorithmic choices.
For example, Leo Breiman, included the question of overparameterised generalization in DNN back in in 1995 as one of the main issues raised by his reflections on 20 years of referceing for NEURIPS (Breiman T995), ).
14

---

Under review as a conference paper at ICLR 2022
The first thing note is that flatness is a property of the functions that a DNN converges on. In other words, the basic argument above that flatter functions will generalize better, which can be relevant to both questions and 2) above.
It is different question to ask whether a certain way of finding functions (say by optimizing a DNN to zero error on training set) will generate inductive bias towards flatter functions. In Hochreiter & Schmidhuber (197). the authors proposed an algorithm to bias towards flatter minima by minimizing the training loss while maximizing the log volume a connected region of the parameter space. This idea is similar to the recent suggestion of entropy- (Chaudhari et al. 2019), where the authors also introduced an extra regularization to bias the optimizer into wider valleys by maximizing the "local entropy".
In an influential paper, Keskar et al. 2016) reported that the solutions found by SGD with small batch sizes generalize hetter than those found with larger batch sizes, and showed that this behaviour correlated with a measure of "sharpness" (sensitivity of the training loss to perturbations in the parameters). Sharpness can be viewed as a measure which is the inverse of the flatness introduced by Hinton & van Camp ( (1993) and Hochreiter & Schmidhuber (1997). This work helped to popularize the notion that SGD itself plays an important role in providing inductive bias, since differences in generalization performance and in sharpness correlated with batch size. In follow-on papers others have showed that the correlation with batch size is more complex, as some of the improvements can be mimicked by changing learning rates or number of optimization steps for example, see (Hoffer et al. 2017 Goyal et al. [017]] [Smith et al. 2017 Neyshabur et al. 2017b). Nevertheless, these changes generalization as a function of optimizer hyperparameters are important things to understand because they are fundamentally type B) inductive bias. Because the changes in generalization performance in these papers lend to be relatively small, they mainly impinge on question 2) for fine-tuned generalization. Whether these observed effects are relevant for question 1) is unclear from this literature.
Another strand of work on flatness has been through the lens of generalization bounds. For example, Neyshabur et al. (2017b) showed that sharpness by itself is not sufficient for ensuring generalization, but can be combined, through PAC-Bayes analysis, with the norm of the weights to obtain an appropriate complexity measure. The connection between sharpness and the AACBsays framework was also investigated by (Dzuugaite & Roy (2017). who numerically optimized the overall PAC- Bayes generalization bound over a series of multivariate Gaussian distributions (different choices of perturbations and priors) which describe the KL- <L-divergence term appearing in the second term in the combined generalization bound by Neyshabur (2017h). For more discussion of this literature on bounds and flatness, see also the recent review (Valle-Pérez & Louis] 2020). |Rhamaa al. (2018) also draw a connection to flatness through the lens Fourier analysis, showing that DNNs typically learn low frequency components faster than high frequency components. This frequency argument is related to the input-output sensitivity picture, which is systematically investigated in Novak et al. (2018a).
There is also another wide-spread belief that SGD trained DNNs are implicitly biased towards having small parameters norms or large margin, intuitively inspired by classical ridge regression and SVMs. |Barlett al. (0177) presented a margin- based generalization bound that depends on spectral and L2,1 norm of the layer-wise weight matrices of DNNs. Neyshabur et al. (2017a) later proved similar nermraal-orrallized margin bound using BaC-Bay approach rather than the complex covering number argument used in Bartlett et al. (2117). Liao et al. 2018) further strengthen the theoretical arguments that an appropriate measure of complexity for DNNs should be based on product norm by showing the linear relationship between training/testing CE loss of normalized networks. Jiang et al. (2018) also empirically studied the role of margin bounds.
In a recent important large-scale empirical work on different complexity measures by Jiang et al. (019), 40 different complexity measures are tested when varying 7 different hyperparameter types over two image classification datasets. They do not introduce random labels that data complexity is not thoroughly investigated. Among these measures, the authors found that sharpness-bas based measures outperform their peers, and in particular outperform norm-basee based measures. It is worth noting that their definition of "worst case" sharpness is similar to definition 2. but normalized by weights, so they are not directly comparable. In fact. their definition of worst case sharpness in the PAC-Bayes picture is more close to the works by PPetzka (2019); Rangamani al. (2019) y Tsuzuku et al. 2019) which focus finding scale-invariant flatness measure. Indeed enhanced performance are reported in these works. However, these measures are only ceale-invariant when the scaling is layer-wise. Other methods of re-scaling (e.g. neuron-wise re-scaling) can still change the metrics. Moreover, the scope of Jiang et al. (2019) is concentrated on the practical side (e.g. inductive bias of type B) and does not consider data complexity, which we believe is a key ingredient understanding the inductive bias needed to explain question on generalization.
Finally, in another influential paper, Dinh et al. (2017) showed that many measures of flatness, including the sharpness used in Keskar et al. (2016), can be made to vary arbitrarily by re-scale parameters while keeping the function unchanged. This work has called into question the use of local flatness measures as reliable guides to generalization, and stimulated lot of follow on studies, including the present paper where we explicitly study how parameter- rescalllngr affects measures of flatness as a function of epochs.
15

---

Under review as a conference paper at ICLR 2022
A.3 RELATED WORK ON THE INFINITE- WIDTH LIMIT
A series of important recent extensions of the seminal proof in Neal 1994) that single-layer DNN with random iid weights is equivalent to a GP (Mackay 1998) in the infinit-wwdtth limit- to multiple layers and architectures (NNGPs) have recently appeared (Lee et al. 2017 Matthews et al. 2018| Novak et al. [0018) Glrriga-Alonso et al. 2019 Yang 2019). These studies on NNGPs have used this correspondence to effectively perform a very good approximation to exact Bayesian inference in DNNs. When they have compared NNGPs to SGD-trained DNNs the generalization performances have generally shown remarkably close agreement. These facts require rethinking the role SGD plays in question 1) about generalization, given that NNGPs can already generalize remarkably well without SGD at all.
A.4 RELATIONSHIP TO PREVIOUS PAPERS USING THE FUNCTION PICTURE
The work in this paper builds on a series of recent papers that have explored the function based picture in random neural networks. We briefly review these works to clarify their connection to the current paper.
Firstly, in Valle-Pérez et al. (2018) the authors demonstrated empirically that upon random sampling of parameters, DNNs are highly biased towards functions with low complexity. This behaviour does not depend very much on (w) for range of initial distributions typically used in the literature. Note that this behaviour does start to deviate from what was found in (Valle-Pére Pérez et al. 2018). when the system enters a chaotic phase. which can he reached with for tanh or erf non-lincarities and for Pu (w(w) with a relatively large variance (Yang) & Salman 2019). They show more specifically that the bias towards simple functions is consistent with the "simplicity bias" from Dingle al. (2018 (020), which was inspired by the coding theorem from AIT (Li & Vitanyi, 2008), first derived by Levin (1974) . The idea simplicity bias in DNNs states that the parameter- lunction map is sufficiently biased, then the probability of the DNN producing a function f on input data drops exponentially with increasing Kolmogorov complexity K K() of the function f. In other words. high p(O) functions have low (J), and high K(J K()) functions have low P(J). A key insight from (Dingle et al., 2018 2020) is that K()) can he approximated by an appropriate measure k(f) and still be used to make predictions on P(f). even if the true K(f) is formally incomputable. Recently Mingard et al. (2019) and Palma et al. (018)) gave two separate non-AlT based thcoretical justifications for the existence of simplicity bias DNNs. In other words, this of work suggests that DNNs have an intrinsic bias towards simple functions upon random sampling of parameters, and in our taxonomy, that is bias of type A).
If simplicity bias in DNNs matches "natural" data distributions, then, at least upon random sampling of parameters, this should help facilitate good generalization. Indeed, it has been shown that data such MNIST or CIFAR-10 10 is relatively simple (Lin et al. 2017 Goldt et al., 2019 Spigler et al. 2019). suggesting that an inductive bias toward simplicity will assist with good generalization.
A second paper upon which the current one builds is (Mingard et al. 2021a), la), where extensive empirical test (for a range of architectures (FCN, CNN, LSTM), datasets (MNIST, Fashion-MNIST CIFAR- 10, ionosphere, IMDb moviereview dataset), and SGD variants (vanilla SGD, Adam, Adagrad, RMSprop, Adadelta), as well as for different batch sizes and learning rates) were done of the hypothesis that:
Pope (f)) = Pa(f(S).
(5)
Here Popx((()) the probability that an optimizer (SGD or one of its variants) converges upon a function after training to zero training error on a training set S. By training over many different parameter initializations, Pope( (f()) can be calculated. Similarly, the Bayesian posterior probability PB (f(S) is defined as the probability that upon random sampling of parameters, DNN expresses function f. conditioned on zero error on S. The functions were, in the current paper, restriction to a given training set S and test set E. Since the systems always had zero error on the training set, functions could be compared by what they produced on the test set (for example, the set of labels on the images for image classification). It was found that the hypothesis (A.4) held remarkably well to first order, for wide range of systems. At first sight this similarity is surprising, given that the procedures to generate P Popt(] (f()) (training with an optimizer such SGD) is completely different from those for (f()S (where GP techniques and direct sampling were used), which knows nothing of optimizers at all. The fact that these two probabilitics are so similar suggests that any inductive bias of type B), which would be a bias beyond what is already present in Ps [f] is relatively small. While this conclusion does not imply that there are no induced biases of type B), and clearly there are since hyperparameter tuning affects fine-tuned generalization, it does suggest that the main source of inductive bias needed to explain 1), the question of why DNNs generalize the first place, found in the inductive biases of type A), which are already there in Ps(S))) In (Mingard et al. 202 the authors propose that, for highly biased priors P(f). that SGD is dominated by the large differences in basin size for the different functions f. and so finds functions with prohabilities dominated by the initial distribution. A similar effect was seen in evolutionary systems (Schaper & Louis 2014 Dingle al. 2015) where it was called the arrival of the frequent.
16

---

Under review as a conference paper at ICLR 2022
In addition, in (Mingard al. 2021a), la), the authors observed for one system that log(Pp( scaled linearly with the generalization error on E for wide range errors. This preliminary result provided inspiration for the current paper where we directly study the correlation between the prior P(f) and the generalization error.
The third main function based paper that we build upon is (Valle-Pérez & Louis 2020) which provides a comprehensive analysis of generalization bounds. In particular, it studies in some detail the Marginal Likelihood PAC-Bayes bound, first presented in Valle-Pérez et al. (018), which is predicts a direct link between the generalization error and the log of the marginal likelihood P(S). P(S) can be interpreted as the total prior probability that a function is found with zero error on the training set S, upon random sampling of parameters of the DNN. The performance of the bound was tested for challenges such as varying amounts of data complexity, different kinds of architectures, and different amounts of training data (learning curves). For each challenge it works remarkably well, and our knowledge good performance of this bound, which is agnostic about optimizers, suggest that large part of the answer to question 1) can be found in the inductive bias of type A), e.g. that found upon initialization. The bound is not accurate enough to explain smaller effects relevant for fine-tuning generalization. which can originate from other sources such as difference in optimizer hyperparameters. These conclusions approach in this paper, where we use the prior P(f) (which knows nothing about SGD) and show that it also correlates with predicted test error for DNNS trained with SGD and its variants. We do propose a simpler bound that is consistent with the observed scaling, but more work is needed to get anywhere near the rigour found in Valle-Pérez & Louis 2020) for the full marginal likelihood bound.
other bound has been tested this comprehensively. Again, the
consistent with the different
that in all three of these papers, GPs are used calculate marginal likelihoods, posteriors, and priors. Technical details of how to use GPs can be found clearly explained there.
Finally,
The current paper builds on this body work and uses some of the techniques described therein, but it is distinct. Firstly, our measurements on flatness are new, and our claim that the prior P(f) correlates with generalization, while indirectly present in (Mingard et al. 2021a) was not developed there at all as that paper focuses on the posterior Ps [S), and did tackling& different question (namely how much extra inductive bias comes from using SGD over the inductive bias already present in the Bayesian posterior). The attack set trick means that P(S) does not change, while clearly the generalization error (or expected test error) does change, so the marginal likelihood bound is not predictive here.
the attack set trick vary functions that consistent with S, and
B CRAMMTTERFFUNCTION MAP AND NEUTRAL SPACE
The link between the parameters of a DNN and the function it expresses is formally described by the parameter- function map:
Definition B.1 Paraacceer-uunction map). Consider the model defined in definition 2.2 if the model takes parameters within set W C R", then the parametcr-function map M is defined
M : W
w + fw
where fw denotes the function parameterized by w.
The funnttte--uuction map, introduced in (Valle- Pérez al. 2018), serves as bridge between parameter searching algorithm (e.g. SGD) and the behaviour of DNN in function space. In this context we can also define the:
Definition B.2 (Neutral space). For a model defined in Definition B.I and given function f. the neutral space N2 C W is defined as
Ny (w E W M(w) f].
The nomenclature comes from genotype phennoyp-pe maps in the evolutionary literature (Manrubia et al. 2020). where the space is typically discrete, and neutral set refers all genotypes that map to the same phenotype. In this context, the Bayesian prior P(f) can interpreted as the probabilistic volume the corresponding neutral space.
C CLARIFICATION ON DEFINITION OF FUNCTIONS AND PRIOR
The discussion of "functions" represented by DNNs can be confusing without careful definition. fig. S6 we list four different interpretations of "functions" commonly seen in literature which also are directly related to our work. These interpretations cover both regression and classification settings. Let X be an arbitrary input domain and )' he the output space. According to different interpretations of the function represented by DNN, y will he different, for the same choice of X and DNN.
17

---

Under review as a conference paper at ICLR 2022
Regression
Classification (Discrete setting)
(Continuous setting)
f DXN The function represented by DNNs before last activation
f ACT The function represented by DNNs after last activation
LAB
Final activation
Coarse
The function that outputs lahels for all possible inputs
graining
Restriction
Restriction
RRS The function restricted finite set of clata samples
Figure S6: The diagram of different definitions for functions represented by DNNs.
Definition (SDNN). Consider a DNN whose input domain is Then fDNN belongs a class of functions FENN which define the mapping between X' to the pre-activation of the last layer of DNN, which lives in R
JONN FDNN X Rd
d is the width of the last layer of DNN.
In standard GP terminology, fDNN is also called latent function about in regression problems.
Rasmussen
2003).
This is the function we care
context supervised leamiin, we have to make some assumptions about the characteristics JDNN. as otherwise we would not know how to choose between functions which are all consistent with the training sample but might have hugely different generalization ability. This kind of assumptions are called inductive bias. One common approach of deseribing the inductive bias is give prior probability distribution to FDNN, where higher probabilities are given to functions that we consider be more likely. For DNNs, FDnn a of functions over an general) uncountably infinite domain X. There are several approaches define probability distributions over such sets. GP represent one approach, which generalizes Gaussian distributions to function spaces. If we ask only for the properties of the functions to C fa...... .. C. C X' (see definition 2.2). then inference with GP, reduces to inference with a standard multidimensional Gaussian distribution. This is an important property of GP called consistency, which helps making computations with GP feasible. As shown in appendix D we can readily compute with this GP prior over JDNN as long it is restricted on a finite data set. Later in definition C.4 restricted function fRES.
finite number of points, i.e. restriction of FDNn
we will formally define the
In classification tasks, typically get data sample from X V. where without loss of generality has the form y = (1...., k) where is the number of classes. simplicity, we further assume binary classification where y (0, 1} Note in the scope of binary classification we have the last layer width of d 1. To grant the outputs of the function represented by DNN a prohability interpretation, we need the outputs lie in the interval (0, 1). One way of doing is to "squash" the outputs of fDNN to (0, 1) by using final activation, typically logistic sigmoid function X(z)
+ exp(-2)) Subsequently we have the definition fACT fig. [S6]
Definition (fACT). Consider the setting and fDNN defined in definition
where 1, and logistic
activation A(z) = (1 +1++a (xxp(=)))- -1 Then fACT is defined as
fACT == jDNN À >: - (0, 1)
where denotes function composition. we also define space of fACT
FACT [fACT for every fDNN E FDNN)
In real life classification datasets, typically do not have access to the probability of an input classified as certain label, but the labels instead. When discuss functions represented by DNNs classification, we
18

---

Under review as a conference paper at ICLR 2022
usually mean the coarse-grained version of /ACT E FACT, meaning we group all oulputs 1 if the probability of predicting the inputs as being label "1" is greater or equal than 0.5, and otherwise. Mathematically, we define fLAB as: Definition C.3 (fLAD). Consider the setting and fACT defined in definition C.2 and a threshold function
1 if 2 0.5
(2)) 0 otherwise
Then we define fLAB and the space FLAB as:
SLAR JACT OT 10.
FLAB {LLA for every fACT E FACT)
The definition C.3 allows us to describe the function represented by DNN in binary classification as binary string consisting of "0" and "1", whose length is equal to the size of input domain set |/ As explained earlier, classification we also want to put prior over FLAB and use this prior as our belief about the task before seeing any data.
Finally, as we mentioned above. to make computations tractable, we restrict the domain to a finite set of inputs. We use the definition of restriction in definition 2.2 to formally define the "functions" we and practically use in our paper:
Definition C.4 (fRES). Consider a DNN whose input domain is with a last layer width 1. Let C= [c1y..., Cm C X' be any finite subset of X' with cardinality m E N. The restriction of function to C is denoted as J and is defined as the space of all functions from C' to
space F E (FDNN, realizable by functions in F. We denote with fRES elements of their corresponding spaces of restricted functions. Specifically, in regression:
fRES E FEEN CC -
and binary classification:
fRES F C + 10,
Note that in definition C.4 we only consider scalar outputs the regression setting. For multiple-outpu functions, one approach consider d GPs and compute the combined kernel (Alvarez al. (011T).
statistical learning theory, the function spaces JONN and FLAB are also called hypotheses classes, with their elements called hypotheses (Shalev-Shwartz & Ben-David 2014). It is important to note that our definition of prior and its calculation is based on the restriction of the hypotheses class to the concatenation of training set and test set S + E. Mathematically, this means the prior of a function P(f) calculated in the paper precisely P( P(JRES), except for the Boolean system section [.] where the input domain discrete and small enough to enumerate (this can also be thought of as the trivial restriction). As explained above, this restriction is inevitable if we want to compute the prior over FDNN or FLAB- A simple example on MNIST (LeC'un et al., 1998) can also help to gain intuition of the necessity of such restriction, where all inputs would include the set of 28x28 integer matrices whose entries take values from 0-255, which gives 256 possible inputs. This indicates that for real-life data distributions the number of all possible inputs is hyner-astronomicall large, not infinite. Nevertheless, In some cases, such as the Boolean system described in Valle-Pérez et al. (2018) and treated in section 5.1 there is no need for such restriction because it is feasible to enumerate all possible inputs: there are only Boolean units which give 2 128 possible data sample. However, even in such cases, the number of possible functions is still large (2128 1)
D GP APPROXIMATION OF THE PRIOR
In this section, we sketch out how we calculated the prior of function P()) (Valle-Pérez al. 2018 Mingard
al. 2021a). As those papers, we use GP. which have been shown to equivalent DNNs the limit of infinite layer width (Neal 1994 Lee et al.] Matthews et al.] 2018) Tan 2008 Rasmussen (0003). These neural network GPs (NNGPs) have been shown to accurately approximate the prior over functions P(f) of
finite-width Bayesian DNNs (Valle- et al. 2018 Matthews et al. 2018 Mingard et al. 2021a).
For the NNGPs, GP prior is placed on the pre-activations of the last layer of the neural network (before a final non-linearity, e.g. softmax, is applied), meaning that for any finite inputs set x = {xı,...,, 2, }. the
random output vector (pre-activations) z = [a (x1) ....,, (Imm) )) has Gaussian distribution. Note that in this paper, the the last layer has single activation since we only focus on binary classification. This setting corresponding to the definition of function restriction is definition C.4] with z E R". Without loss of generality, we can assume such process has a zero mean. The prior probability of the outputs can be calculated as:
1
exp (-1--' 2T -
P(z)
(6)
(2Th)
T
19

---

Under review as a conference paper at ICLR 2022
Σ is the covariance matrix (often called kernel), whose entries are defined as (xr,, = (1a())), =(x,). Neal (1994) gave the basic form of kernel Σ in single hidden layer case, where Σ depends the variance of weights and biases (o and σb). In DNNs with multiple hidden layers, the kercel for layer can be calculated recursively
by induction, assuming the layer / - is a GP (Lee et al. [011)] Matthews et al. 2018). The kernel for fully connected ReLUUacttivted networks has well known arc-cosine kernel analytical form (Cho & Saul 2009) which we used in all FCNs in our work.
ResNet50, the analytical form of GP kernel intractable. Instead, we use Monte Carlo empirical kernel
(Novak et al. (2)18b,, and apply one step of the fully connected GP recurrence relation (Lee et al. (017)) taking advantage of the fact that the last layer of ResNet50 is fully connected. Mathematically, the empirical kernel can be expressed as:
M
E. (x1,
Ow
(os))
(Nem. (1z)) c..
Mn
m= c=
where (Nomm. (z)) is the activation of c-th neuron in the last hidden layer (L is the total number of layers) for the network parameterized by the m-th sampling of parameters Wm. M is the number of total Monte Carlo sampling, is the width of the final hidden layer, and dw. 0% are the weights and biases variance respectively. In (IS) |Ei).
our experiments, M set to
After calculating P(z) with the corresponding kernel, the prior over (coarse- gaaaseeg restriction functions P(f) can be calculated through likelihond P(f(=), which in our case is just a Heaviside function representing a hard sign nonlinearity. As aan-aassian likelihood produces an intractable P(f). we used Expectation Propagation (EP) algorithm for the approximation of P(f) (Rasmussen 2003). This same EP approximation
was used Mingard al. (2021a) where is discussed further. We represent the function f by the input- pairs on the concatenation of training set and test set S + E.
E COMPARING FLATNESS METRICS
-05
-10
-1.5
-2.0
-3.0
-6.5
-5.5
logiolsharpness)
Figure S7: The direct correlation between sharpness and spectral norm of Hessian for the 1000 most frequently found functions found after SGD runs for a two hidden layer FCN, in the n = 7 Boolcan system (Same system as in fig.
As mentioned in section 2.2 of the main text, the sharpness metric in definition 2.1 can be directly linked to spectral norm of the Hessian by considering the second order Taylor expansion of L(w) around critical point powers of ( (Dinh et al. 2017). We empirically confirm this relationship by showing in fig. [S7 the direct correlation between sharpness and spectral norm of Hessian, as well as in fig. (Shthe correlation between Hessian spectral norm and prior in Boolean system described in section 5.
In addition to the spectral norm, another widely used flatness measure is the product of subset of the positive Hessian eigenvalues, typically say the product of the top-50 largest eigenvalues (Wu al. 2017 Zhang al. 2018). We measured the correlation of these Hessian-base generalization for the FCN/NNIST system in fig S9 Since they correlate well with the sharpness, these flatness measures show very similar correlations with generalization as sharpness does in fig. and fig. 4 In other words. the Hessian-hased flatness metrics also capture the loose correlation with generalization when the neural network trained by SGD and the deterioration of this correlation when we change the optimizer Adam.
flatness metrics with sharpness well as with
20

---

Under review as a conference paper at ICLR 2022
-1.0
(A
-1.
-2.0
-2.3
-3.0
1og1oP(f)
Figure S8: The correlation between prior and flatness in Boolean system where the flatness is measured by spectral norm of Hessian, for the 1000 most frequently occurring functions found by SGD runs with hidden layer FCN. The system is the same Boolean system as fig. except that we use different metric of flatness.
1000 Dovv-TA AM
Japul
Angursharpreess
(a)
(b)
(c)
(d)
Career pererataaion 150
4ppu
Ingusttapeess
copu/Shorpeees
(e)
(f)
gg)
(h)
Figure S9: Two Hessian- Ressian-based flatness metrics show analogous behavior to sharpness defined in
(definition 2.1 The architecture and dataset are FCN/MNIST, with training set size S = 500, and test set size E 1000; which are the same settings fig. (d) and fig. (e). Optimizer: SGD (a) (b): The correlation between Hessian-based flatness metrics and generalization. (c) (d): Sharpness and Hessian-basse based flatness metrics correlate well with one another. Optimizer: Adam (e) (f): The correlation between Hessian- flatness metrics and generalization breaks down, just does for sharpness in fig. (g) - (h): Sharpness and Hessian based flatness metries correlate well with one another, even though they don't correlate well with generalization.
Another detail worth noting is that Keskar et al. (2016) used the L-BFGS- B algorithm (Byrd et al. 1995) to perform the maximization of L(w) in Cc. which is the box boundary around the minimum of interest:
C = (Aw E R": - (Jws] 1) & Aw; < <(wa)]+ + 1) Vi (1,2,...,.))
(8)
However, as quasi-Nou -Neton method, L-BFGS-B is scalable when there are tens of millions of parameters modern DNNs. To make Keskar-sharpness applicable for large DNNs (e.g. ResNet50), we use vanilla SGD for the maximization instead. The hyperparameters for the sharpness calculation are listed in table [ Note that the entries batch size, learning rate and number of epechs all refer to the SGD optimizer which does the maximization in the sharpness calculation process. The number of epochs chosen such that the value of
21

---

Under review as a conference paper at ICLR 2022
Table I: Hyperparameters for sharpness calculation
DATA SET
ARCHITECTURE
Box SIZE (()
BATCH SIZE
LEARNING RATE
NUMBER EPOCHS
BOOLEAN
FCN
10-4
16
10-3
10
MNIST
FCN
10-4
32
10
100
CIFARIO CIFARIO
FCN RESNET50
10
128
5 10
100
10
128
10
100
loss function found each maximization step converges. example the convergence sharpness shown S10 As a check, we also compared our SGD -ha-sharsssss with the original L- L-BFGS B-sharpness, finding similar results.
040600
0.248
0.0344
L(w)
0.338
200
1000
1200
1400
1600
Iterations
Figure S10: The max value of loss function L(w) at each iteration in the process of maximization, when calculating the sharpness using SGD instead of L-BFGS-B. The plot shows the max loss value found by SGD in the box limit Cę will converge after given number of epochs. For this plot the hyperparameters are listed in the second line of table (MNIST).
F IMPLEMENTING PARAMETER RE-SCALING
this section we describe detail implement the alpha scaling DNNs first proposed by Dinh 2017). The widely used rectified linear activation (ReLU) function
&reak max(æ; 0)
exhibits the so-called "non-negative homogeneity" property:
V(2,0) E R (1ea((2)) = axprea(2)
The action of a L-layered deep forward neural network can be written as:
Orea (dheat (-.. &reat (xr W1 + ..)) Wt-1 + b(-1) We + by
in which
the input vector
Wz the weight matrix the L-th layer
br is the bias vector of the L-th layer
simplify notation, we have not included the final activation function, which may take any form (softmax or sigmoid etc.) without modification of proceeding arguments. Generalizing the original arguments from Dinh et al. (2017) slightly to include bias terms, we exploit the non-negative homogeneity of the ReLU function to find that so-called "a-scaling" of one of the layers will not change its hehaviour. Explicitly applying this to the i-th layer yields:
(éme (x aW; + abi)) Wit = (drec (x W. b:)) W
(9)
22

---

Under review as a conference paper at ICLR 2022
-115
Bayesian prior
-120
-125
-0.6
www.ssaappeeee.c (pness)
(og1oP(0
-130
-0.8
-135
starpness
-140
-12
-145
1.4
-150
-16
-155
200 400
600
800
1000
Epochs
Figure S11: The effect of alpha scaling on prior and sharpness. At each epoch we calculate the sharpness and the prior for our FCN on MNIST system with |S| = 500. The green dashed line denotes where zero-training error is reached and post-training starts. The red dashed line denotes the epoch where a-scaling takes place with a = 5.0. Here the value of a is not big enough to "knock" the optimizer out of the neutral space, upon alpha scaling, in contrast to fig. [5] As expected, we observe no change in prior upon alpha scaling (note that prior can change on overtraining if a slightly different function is found by SGD). The sharpness shows a larger peak upon alpha-scaling, as expected. See appendix H
Clearly, the transformation described by (W;, b. W.+1) + (aW., ob, 1W++) will lead to an observationally equivalent network (that is, a network whose output is identical for any given input, even the weight and bias terms differ).
Since the scaling transformation does not change the function, it does not change the prior of the function. However, for large enough o, as shown for example fig. [5] we see that SGD can be "knocked" out of the current neutral space because of the large gradients that are induced by the a scaling. This typically lcads to the prior suddenly surging up, because the random nature of the perturbation means that the system more likely to land on large volume functions. However, we always observe that the prior then drops back down quite quickly as SGD reaches zero training error again. On the other hand, shown in fig. [STT when the value of smaller it does not knock SGD out of the neutral space, and so the prior does not change at Nevertheless, the sharpness still exhibits a strong spike due to the the alpha scaling.
Although not in the scope of this work, it is worth noting that the alpha scaling process in Convolutional Neural Networks (CNNs) with batch normalization (loffe & Szegedy 2015) layer(s) is somewhat different. Because batch normalization layer will eliminate all affine transformations applied on its inputs, one can arbitrarily alpha scale the layers before a batch normalization layer without needing to of compensate following layer. provided the scaling is linear.
G FLATNESS AND PRIOR CORRELATION
the main text, we showed the correlation the Bayesian prior and sharpness with generalization fig. and fig. 4 Here, in fig. [S12] we show the direct correlation of the prior and sharpness. As expected from the figures in the main text. sharpness correlates with prior roughly as it does with generalization i.e. reasonably for vanilla SGD but badly for entropy-SGD (Chaudhari et al. 2019) Adam (Kingma & Ba 2014). We note that, as shown fig. 59 sharpness also correlates relatively well with the spectral norm of the Hessian and log product of its 50 largest cigenvalues for all the optimizers. So the correlation of flatness with prior/generalization does not depend much on which particular flatness measure is used.
Overall, it is perhaps unsurprising that local measure such as flatness varies in how well it approximates the global prior. What is unexpected (at least us) is that Adam and Entropy-SGD break the correlation for this data set. appendix [.2] we show that this correlation also breaks down other more complex optimizers. but, interestingly, not for full-batch SGD. Further empirical and theoretical work is needed to understand this phenomenon. For example, is the optimizer dependence of the correlation between flatness and prior general property of the optimizer, is specific to certain architectures and datasets? One hint that these results may have complex dependencies architecture and dataset comes from observation that for ResNet50 on
23

---

Under review as a conference paper at ICLR 2022
Cifar10, see less difference between SGD and Adam than for the FCN MNIST. More work needed here.
-20
ResNee500CIAAR
SGD
-20
FCNICIFAR-10
-12
CNMMNISTSSC
SGD
Moudtharppss poossi
o0gy/Shurpeessl
(a)
(b)
(c)
Evite opyy SEI
FCRUMMIST
FORCIFAR-1
poharppeees
40guttharppesst
(c)
Figure S12: The direct correlation between prior P(f) and sharpness over different datasets and optimizers. The correlation between prior and sharpness closely resembles the correlation between sharpness and generalization, mainly because prior and generalization are very closely correlated, as seen in our experiments (fig.) fig.
H TEMPORAL BEHAVIOR OF SHARPNESS
When using sharpness in definition 2.1 the metric of flatness, there several caveats. First is the hyper- parameters (see table []: the value sharpness is only meaningful under specified hyperparameters, and in different experiments the sharpnesses are only comparable when the hyperparameters are the same. This renders sharpness less convenient to use (but still much more efficient than Hcssian calculation). Second is the time evolving behavior of sharpness: For the classification problems we study, and for CE loss, it can continue to change even when the function (and hence generalization) unchanged.
Before reaching zero training error, gradients can be large, and the behavior of sharpness (definition 2.1 can be unstable under changes of box size Ç. This effect is likely the cause some unusual fluctuations the sharpness that can be observed in fig. 5|and fig. [ST[ST] aroundd epoch 100, In fig. [S13 show that this artefact disappears for larger Ç. Similarly, when the gradients are big (typical in training), sharpness may no longer link to spectral of Hessian very well.
fig. S14 we first train the FCN to zero error, then "alpha scale" after 500 epochs, and then keep post-training for another 5000 epochs, much longer than fig. The behaviour of sharpness and prior upon "alpha scaling" (not surprisingly) follows our discussion in section 5.4 What is interesting to see here is that after enough overtraining, the effect of the alpha scaling spike appears to disappear, and the overall curve looks like continuation of the curve prior alpha scaling. What this suggests is that alpha-scaling brings the system an area of parameter space that somehow "unnatural", Again, this a topic that deserves further investigation in the future.
Finally, we show the temporal behavior of bassian-based flatness measure in fig. S15 Because of the large memory cost when calculating the Hessian, use smaller FCN on MNIST, with the first hidden layer having 10 units. We find that the Hessian based flatness exhibit similar temporal behavior sharpness.
24

---

Under review as a conference paper at ICLR 2022
Sandept
Log (eveeaarpees)
Log Logrode pinees)
(og) W TATIT (UT0OO (CC
100uAn
lng
30
022
Epochs
Epochs
Epochs
(a)
(b)
(c)
Figure S13: Different temporal behavior of sharpness, prior and accuracy when using different
box size The dataset is MNIST with |s| 500 and [|] 100. The architecture is FCN. SGD optimizer is used. Scaling parameter a 5.0. Green and red dashed line denote reaching zero training error and alpha scaling, respectively. (a) Ç 10-3, are quantitative differences between the values of Ç used, qualitatively we observe similar behaviour.
, (b) 10-1 (c) 10-5 While there
Bayasian prior
40
650
700
Sharpnesss
1091oP(0
-80
sharpness
-120
-160
1000
2000
3000
4000
5000
Epochs
Figure S14: The temporal behavior of sharpness and prior after 5000 epochs of reaching zero training error. The dataset is MNIST with |S| = 500 and EE| = 100. The architecture is FCN, SGD optimizer used. The magnitude of scaling a 6.0.
1 THE CORRELATION BETWEEN GENERALIZATION PRIOR, AND SHARPNESS UPON OVERTRAINING
As shown in fig. 5 the main text, and further discussed in appendix H flatness measures keep decreasing upon overtraining even when function itself does not change. In this section, we revisit correlation between prior, flatness and generalization at different numbers of overtraining epochs, i.c. after reaching zero training As can be seen in fig. to to fig. S21 overtraining does not meaningfully affect the correlation between sharpness, prior, and generalization When the optimizer SGD, the flatness, matter if is measured by sharpness Hessian based metrics, correlates well with prior and (hence) generalization across difference overtraining epochs; whereas when using Adam, the poor correlation also persist in overtraining.
observed the epoch where error first reached in fig. and fig. 4
J FURTHER EXPERIMENTS
J.1 RESNET50 TRAINED WITH ADAM
When training ResNet50 on CIFAR-IO. 10, we use training set [S] 5000, attack set size |A| 5000, test set size |E| 2000. In each experiment, we mix the whole training set with different size of subset attack set. The size of [A] ranges (0, 500, 1000, 1500, 5000). For each subset of attack set we sample 5 times. When training ResNet50 with Adam, we empirically found it is hard to train the neural net to zero training error with attack set size [A] 2500. So only show the results for those functions found with |A] < 2500. In
25

---

Under review as a conference paper at ICLR 2022
-10
log1o (
-30
50
-40
-70
Epochs
Figure S15: The temporal behavior of one Hessian based flatness metric. The dataset is MNIST with |S| 500 and [E] 100. The architecture is smaller FCN (784-10-40-1) the optimizer is SGD. The green dashed line denotes the epoch where the system reaches zero training error. No alpha scaling applied here. The Hessian based flatness metric shows similar temporal behaviour the sharpness measure.
fig. show the results of correlation between sharpness and prior with generalization with limited data. prior, as usual, correlates tightly with generalization, while flatness- generaazaaniiiiiiiionn correlation is much more scattered, although is slightly better than the correlation seen for the FCN on MNIST, and closer to the hehaviour we observed for SGD in the main text.
J.2 MORE SGD-VARIAN OPTIMIZERS
In fig. S25 we provide further empirical results for the impact of choice of optimizer on the sharpness- generalization correlation by studying three common used SGD variants: Adagrad (Duchi et al. (0TT)) Momentum (Rumelhart al., 1986) (momentum=0.9) and RMSProp (Tieleman & Hinton (012). well as full batch gradient descent. Interestingly, full hatch gradient descent (or simply gradient descent) shows behaviour that is quite similar to vanilla SGD. By contrast, for the other three optimizers, the correlation between sharpness and generalization breaks down, whereas the correlation between prior and generalization remains intact, much observed in the main text for Adam and Entropy-SGD...
J.3 LARGER TRAINING SET
In order to rule any potential training size effect main argument of the flatness, prior and generalization relationship, we further performed experiments on MNIST with 10k training examples. Larger training sets are hard because of the GP-EP calculation of the prior scales badly with size. The results are shown in fig. [524 It is clear that the correlations hetween sharpness, prior and generalization follow the same pattern as we see in fig. in which there are only ||] 500, |E 1000 images. If anything, the correlation with prior is tighter.
26

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
90
325
80
1o (010P((f
-350
-425
0.0
Overtraining epochs: 200
Generalization (%)
Generalization (%)
90
-325
80
1og1oP ((
-350
60
-400
5.0
50
425
0.5
1.0
1.5
-400
-350
0.5
1.0
1.5
Overtraining epochs: 300
Generalization (%)
Generalization (%)
90
-325
80
ao
log P(f
-350
70
-375
-400
0.5
1.0
1.5
-400
-350
0.5
1.0
1.5
Overtraining epochs: 400
Generalization (%)
Generalization (%)
90
344
80
-360
log Lo
-360
60
-400
-420
50
-440
-350
Overtraining epochs: 500
Generalization (%)
90
Generalization (%)
90
-325
80
a0
1og ((f)
-350
70
60
50
-42
1.0
1.5
-400
-350
1.0
1.5
-0ogiossharpess)
log1oP(f)
(0ggo(ssaappesss)
Figure S16: The correlation between sharpness, prior and generalization upon overtraining. The dataset is MNIST (|S) 500. E 1000), the optimizer SGD. For the range of (100-500) overtraining epoch tested here, the overall values of sharpness drop with overtraining. By contrast, the priors remain largely the same. For each quantity, the correlations remain remarkably similar with overtraining.
27

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
90
-340
80
-360
log 10
-360
-400
60
420
50
440
-400
-350
Overtraining epochs: 200
Generalization (%)
Generalization (%)
90
80
70
70
log 10
60
60
50
50
440
Overtraining epochs: 300
Generalization (%)
Generalization (%)
80
360
70
380
log
60
60
-400
-42
50
50
-100
-350
Overtraining epochs: 400
Generalization (%)
90
-340
Generalization
80
10 ((A)
-360
-380
400
420
440
-400
-350
6
Overtraining epochs: 500
Generalization (%)
Generalization (%)
90
-325
80
log 10 (f)
-350
70
335
60
4000
-350
10010
l0g10P(f)
M00losShappness:
Figure S17: The correlation between sharpness, prior and generalization when over-trained (keep training after reaching zero training error). The dataset is MNIST (IS) 500, | 1000), the optimizer Adam. The correlations are similar across different overtraining epochs.
28

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
325
log
-350
70
70
375
440
-400
-350
-2
Overtraining epochs: 200
Generalization (%)
90
Generalization (%)
90
325
log 1oP
-350
60
60
-400
425
-400
-350
Overtraining epochs:
Generalization (%)
Generalization (%)
80
80
log
3350
60
60
400
-1.5
-0.5
-350
-1.5
-0.5
Overtraining epochs: 400
Generalization (%)
90
Generalization (%)
90
325
350
log
400
-455
-15
-1.
-0.5
0.0
-400
-350
-1.5
-1.0
-0.5
0.0
Overtraining epochs: 500
Generalization (%)
Generalization (%)
325
log10P(n
-350
-375
60
60
-400
425
1.5
-1.0
-0.5
-400
-350
1.5
-1.0
-0.5
(10g10(13)
log1oP(f)
(10g10(1))
Figure S18: The correlation between Hessian spectral norm, prior and generalization when over- trained (keep training after reaching zero training error). The dataset is MNIST ([S) 500, |E| 1000), the optimizer SGD. The correlations are similar across different overtraining epochs.
29

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
80
log1oP(f)
70
60
-400
50
50
50
-400
-350
50
Overtraining epochs:
Generalization (%)
Generalization (%)
90
80
log1oP(f)
-350
50
100
4400
-350
50
100
Overtraining epochs: 300
Generalization (%)
Generalization (%)
80
log1oP(n
-350
60
-400
50
50
20
40
60
-400
-350
0
20
40
60
Overtraining epochs: 400
Generalization (%)
Generalization (%)
90
-325
80
oo
-350
70
375
60
60
400
500
50
25
50
75
-400
-350
Overtraining epochs:
Generalization (%)
Generalization (%)
80
log1oP(f)
70
60
60
-400
5.0
50
-350
50
log1oP(1)
-log10( [1/ i=1 1
-log1o
( 1
Figure S19: The correlation between Hessian based flatness (product the top 50 largest Hessian eigenvalues), prior and generalization when over-trained (keep training after reaching zero training error). The dataset MNIST ((S) 500, | 1000), the optimizer is SGD. The correlations are similar across different overtraining epochs.
30

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
-325
80
l0
-350
375
400
-25
400
-350
Overtraining epochs: 200
Generalization (%)
Generalization (%)
-32
log 1o
337
4400
-45
50
50
-400
Overtraining epochs:
Generalization (%)
Generalization (%)
325
log1oP
-350
440
425
-350
Overtraining epochs: 400
Generalization (%)
Generalization (%)
-350
1og
-375
-400
-425
-400
-350
Overtraining epochs: 500
Generalization (%)
Generalization (%)
325
80
-35
log10P
-375
-400
-42
3
-400
-350
(10100112)
log10P(f)
-10g10(11))
Figure S20: The correlation between Hessian spectral norm, prior and generalization when over- trained (keep training after reaching zero training error). The dataset is MNIST ([S) 500, |E| 1000), the optimizer Adam. The correlations are similar across different overtraining epochs.
31

---

Under review as a conference paper at ICLR 2022
Overtraining epochs: 100
Generalization (%)
Generalization (%)
90
80
80
oo
-350
70
70
60
-400
50
100
200
-400
-350
100
200
Overtraining epochs:
Generalization (%)
Generalization (%)
log
-350
60
50
50
100
4400
-350
Overtraining epochs: 300
Generalization (%)
Generalization (%)
335
80
log 1o (()
-35
70
375
60
60
-400
50
50
442
100
150
200
-400
-350
100
150
200
Overtraining epochs: 400
Generalization (%)
90
Generalization (%)
90
-325
60
lognoP(f)
-375
60
-400
50
50
-42
100
150
200
-400
-350
100
150
200
Overtraining epochs: 500
Generalization (%)
Generalization (%)
80
log1oP(f)
-350
70
60
60
-400
50
50
-400
50
log1oP(f)
-log10( [
-log1ol [ 1=1
1=1 1
Figure S21: The correlation between Hessian based flatness (product the top 50 largest Hessian eigenvalues), prior and generalization when over-trained (keep training after reaching zero training error). The dataset is MNIST (IS) = 500, |F] = 1000), the optimizer is Adam. The correlations are similar across different overtraining epochs.
32

---

Under review as a conference paper at ICLR 2022
95
Generalizationn (96)
Generaliization (%)
03
23
ResNet50/CIFAR-10//
ResNet50/CIFAR-100
Adam
Adam
-10
-15
0
-05
0.0
0.5
-1000
-1750
-1700
-100
-300
loginssharpness
logıePIn
(a)
(b)
Figure $22: The correlation between generalization and (a) sharpness (b) prior for ResNet50 with | 5000, EE 2000, and |A| ranging from ( to 2500, all on CIFAR-10.
33

---

Under review as a conference paper at ICLR 2022
-30
Cener ceneralizston (h)
-20
Gam www.ettaaaooooooo
Full-batch GD
Full-batch GD
Fulli- balli-bachh GD
Atguu//harpness
leguP10
-30
Ceneraliin allzztnnn (50
Carrer altaation (ha)
5 Der0e
Adagrad
Adagrad
Adagrad
-05
Aoguu/sharpness
leguP10
-bggu/sharrreess
-30
Cenersiigtioon 250
log
Momentum
Momentum
Momentum
22
Abgrulsherpresss
AoguP110
Generoligation (50
Garrer Ceneralsation www.taaaannnnmmmmm (ha
RMSProp
RMSProp
RMSProp
Aognulsherpneess
AoguPP0
Figure $23: More results the correlation between sharpness, prior and generalization when using other SGD-variant optimizers. The dataset is MNIST, JS| 500, E' 1000. The architecture is FCN. The optimizers are full-batch gradient descent, Adagrad, Momentum (momentum=0.9) and RMSProp. All correlations are measured upon reaching zero training error.
34

---

Under review as a conference paper at ICLR 2022
wtw.shhopeesst
(b)
(c)
-150
1008
40Qushharpnesu
(d)
(e)
Figure S24: The correlation between sharpness, prior and generalization on MNIST with |S| 10000, EE| 1000. The attack set size ranges from 1000 9000. The architecture is FCN. (a)-(c): The FCN is trained with SGD; (d)-(f): The FCN is trained with Adam.
35


Reviews:
[{"review_id": "joHMT_cOgJF", "reviewer": "Reviewer_hYg1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "First, its extremely easy to determine the authors. They self-cite collectively over 50 times if you include the publications where the authors are co-authored. Furthermore, they have the current and the former NeurIPS version of this paper posted on arxiv, where the original title was slightly different and with a less ambiguous conclusion -- \"Why Flatness Correlates With Generalization For Deep Neural Networks\". This isn't against ICLRs rules, and this review is not to judge either arxiv version, but since v1 is publicly available, I have considered it among the relevant past work for the currently submitted paper. \n\nGeneral Concerns (no specific order):\n\n\t- The name \"neutral space of f\" has a much more commonly known name among the mathematics/neural network community, its called \"the fiber of f\". See https://en.wikipedia.org/wiki/Fiber_(mathematics).\n\n\t- Section 2.4, it is stated that \"DNNs are typically trained to zero training error on the training set\". As a practitioner myself, I'm not sure who/where this information is coming from. DNNs are typically *not* trained to zero training error as that almost always results in overfitting and an over-usage of time/computational resources.\n\n\t- Figure 1, it shows the loss function has negative error at certain weights (w). This seems to contradict what was said about loss functions in the paragraph immediately prior, that the loss function is zero or positive. What does negative error mean?\n\n\t- I find it slightly disingenuous to place central definitions in the appendices to achieve the page limit. Gamifying the page limit isn't the purpose of the page limit. Expecting reviewers to read 35 pages in the time allotted isn't considerate.\n\n        - Appendix B where the \"parameter-function\" map is defined, it states that it \"was first introduced in [Valle-Pérez et al., 2018]\". The map from the parameter space which takes a parameter and maps it to a function is so ubiquitous its questionable to assign any single person as its creator. My cynical side questions if this was an attempt at self-citation.\n\n\t- Minor: The text makes the jump from L(y,y') to L(w) without stating that y is parameterized by w. Its not a big deal, but the less familiar reader may get confused at an early stage with this jump.\n\n\t- Section 3 states \"Perhaps the simplest argument is that if DNNs trained to zero error are known to generalize well on unseen data...\". That's a very big \"if\". In practice, DNNs trained to zero error are expected to not generalize well on unseen data.\n\n\t- Section 3 states \"Mingard et al. (2021) showed that log (P_B(f|S)) correlates quite tightly with generalization error.\" If this is the case, then its not clear to me if the second aspect of this work is a novel contribution. If it is, its not clear to me how. \n\n\t- Section 5.1 states that the DNN being used \"was found to be sufficiently expressive to represent almost all possible functions\". The ReLU network is shaped 7 - 40 - 40 - 2, and its claimed that this is capable of representing \"almost all of the possible\" 3.4 x 10^34 functions. I'd like proof. For example, consider proposition 3 from the well known work on ReLU linear regions https://arxiv.org/abs/1402.1869, where even the most conservative upper bound for the number of functions computable by a ReLU network with N hidden neurons is 2^N = 2^80 = O(10^24). This is 10 orders of magnitude short for this case. The lower bound on the maximum (Theorem 4) is even more pessimistic, and both of these bounds have been shown to be overoptimistic in later works (e.g. https://arxiv.org/abs/1711.02114) taking advantage of the fact that zero-activations of ReLUs collapse the dimension of the signal passing through the network. Ten orders of magnitude is hard to imagine.\n\n\t- Section 5.1, the authors take 10^8 initial points. That is 26 orders of magnitude short of how many functions are computable, or 0.000000000000000000000001% of the possible functions would've been sampled. It is likely that even less were sampled since these points were chosen from essentially a bounded sphere in parameter space (i.e. finite samples from a thin-tailed Gaussian distribution). Its difficult to back up any empirical claim when there's 10^34 functions.\n\n\t- Critical: Section 5.2 it is stated that \"In order to generate functions f with zero error on the training set S, but with diverse generalization performance, we use the attack-set trick from Wu et al. (2017).\" This seems like a critical flaw in that the training process is not SGD, and therefore the distribution of parameters could be substantially different than that of using SGD. This feels like the evidence only supports the different claim that \"for this attack dataset trick, these claims about flatness (volume) measures hold\".\n\n\t- Section 5.3 states \"Changing optimizers or changing hyperparameters can, of course, alter the generalization performance by small amounts, which may be critically important in practical applications\". True and False, if \"small\" was replaced by \"large\", I'd say true. To be facetious, one could choose hyperparameters extreme enough where generalization is arbitrarily poor. Since hyperparameters can alter the level of generalization from arbitrarily poor to state-of-the-art for some models, this is an important point. Perhaps its present somewhere in the 35 pages and I do not see it, but I'm unable to find much comparison for this at all.\n\n\t- Figure 5, what is \"alpha scaling\"? Not introduced as far as I can find.\n\n\t- In the discussion, it states \"evidential basis for this hypothesis has always been mixed\". It seems like the first main contribution of this work adds to the pile of mixed results. Another excerpt, \"Here we performed extensive empirical work showing that flatness can indeed correlate with generalization. However, this correlation is not always tight, and can be easily broken by changing the optimizer, or by parameter-rescaling\". What I take away from this is that sometimes its correlated, sometimes its not, and there's no clear pattern at this point. Seeing as this is the main title of this paper, it doesn't feel like a novel result at all. It seems like more supporting evidence that we really don't know how flatness measures correlate with generalization. I could be wrong, clarification would be helpful.\n\nTypsetting:\n\n\t- Many figures are hard to read, with ticks have <6 pt font, or otherwise appear to have errors in typesetting, graphs sitting overtop of words, or maybe labels were manipulated on MS paint or the alike. Overall there's very little consistency across figures.\n\n\t- Figure size varies substantially, legends are seemingly randomly placed, sometimes larger than the labels sometimes not. Figures are different sizes even within the same 8-subfigure figure.\n\n\t- SOTA, state of the art (or also written state-of-the-art in this same paper), doesn't require an abbreviation since its used once. Well really, its introduced twice, along with its abbreviation twice. Regardless, not needed. Same goes for cross entropy (CE), introduced without ever using it again. Gaussian Processes (GPs) is used multiple times, but also introduced multiple times, and its not introduced at the first opportunity either. Seems unorganized.\n\n\t- The use of w.r.t., Just write it out instead of bringing in needless abbreviations.\n\n\t- Quotation style fluctuates often. Look closely at the frequently changing quotation marks.\n\n\t- fig. vs. Fig., Appendix vs. appendix, Eq.(4). vs. Eq. (4). Figs vs. fig. for multiple subplots. Consistency and generally speaking, upper case \"Fig.\" and \"Eq.\" tend to look more professional.\n\n\t- General equation typesetting, spacing, periods before a sentence with an equation ends.\n\n\t- Commas, spaces, just simple typos that could be caught by any spell check.\n\n\t- fig. 2 vs. fig. S3, I don't get the S's. For example, in Section 5.1, it states the figure is in the appendix but its linked to regular Fig. 3. Sometimes figures with \"S#\" link to the main text, sometimes not. Check this.\n\n\t- \"The function \"with\" the largest prior...\". Missing the word \"with\".\n", "summary_of_the_paper": "This paper shows that for certain variants of SGD, the commonly held assertion that flat minima imply good generalization may not hold. The overall conclusion from this aspect of the paper is that popular measures of flatness sometimes do, and sometimes do not, correlate with generalization. The authors continue on to propose an alternative measure, the Bayesian prior, which they claim correlates well with generalization. In essence, this measure captures the volume of parameter space which when a networks weights are initialized within the volume's region, it converge to a pre-specified function with low training error. After covering the definitions and related measures, they perform experiments on MNIST, CIFAR-10, and discuss how the choice of optimizer affects the claims.", "summary_of_the_review": "Overall, its very long, there are a few critical concerns about the soundness of the claims, there are some typesetting issues. The figures could use work, a wall of dozens of hard-to-read graphs at the end aren't helping the focus. The conclusion for me is quite ambiguous, I'm not sure what to take away from this paper in terms of if flatness makes sense to use as a predictor of generalization. The title doesn't entirely reflect what's being claimed in the paper, since much of the paper focuses on the Bayesian prior.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Ttbds-GFbHW", "reviewer": "Reviewer_ohft", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strength**: This paper is relatively well written and easy to follow. It studies an interesting and important question that characterizes the generalization performance of a trained neural network. It proposes a new criterion and demonstrates the effectiveness in standard datasets.\n\n**Weakness**: I think the paper is still a bit weak in a number of aspects:\n\n1. Optimizers plays an important role in deciding the final form of a trained model, but the architecture and parameterization are also fundamentally important. It would be great if in addition to the optimizer comparison experiments, the paper could add experiments to compare *across* different architectures. There is also a belief that overparamerization contribute to better flatness (and generalization). So it would be good to see how the number of model parameters correlate with flatness and the Bayesian prior as well.\n\n2. There is a new optimizer called Sharpness-aware Minimization ( https://arxiv.org/abs/2010.01412 ) that minimize the loss sharpness during optimization and are shown to greatly improve the generalization performance in some tasks. It would be interesting to add SAM to the existing optimizer comparison experiments. \n\n3. Although not strictly required, developing a theoretical link between the proposed criterion and generalization would make the paper more coherent and solid. \n\n4. GPs are used to approximate the Bayesian posteriors in this paper. How accurate is this approximation? Could the paper provide a synthetic study (like in 5.1) where the groundtruth can be measured by direct sampling and compare the accuracy of GP based estimation?\n\n5. In the correlation figures (e.g. Fig. 3), the generalization is shown as the y axis. I wonder are all the networks trained to 0 training error? If not, what would be the plot like when test accuracy is plotted in the y axis?\n\n\n------------------\nAfter rebuttal: Thanks for the clarifications. I intend to keep my rating.", "summary_of_the_paper": "This paper studies the correlation between flatness and generalization in deep neural networks and show that, consists with some previous studies, the correlation could sometimes be broken. As an alternative, it propose a new measure based on the Bayesian prior upon initialization, and empirically demonstrate this criterion could maintain a positive correlation with generalization even in the case where flatness breaks.", "summary_of_the_review": "This paper propose a new criterion to measure generalization and empirically show that it correlate with generalization, even in cases where flatness fails to maintain the correlation. It would be more solid if direct theoretical link could be shown. It is OK for the paper to be primarily empirical studies, but in this case, a more systematic set of experiments could help strengthen the paper. While it is impossible to enumerate all possible comparison cases, a number of canonical settings like different parameterizations (see above for details) are very important.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Q4kUXZMiCpi", "reviewer": "Reviewer_r1hF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The empirical observation that Adam and Entropy-SGD could break the correlation between flatness and generalization is interesting.\n- The empirical observation that the Bayesian prior is linked with generalization, reinforces the ideas presented in the Paper (Mingard et. al., 2021). This is an interesting area to explore, especially to connect empirical observations with theoretical findings.\n- Throughout the paper, there are some nice explanations/intuitions given.\n- For each figure presented in the main paper, there are similar experiments to provide sanity checks.\n\nWeaknesses:\n- The novelty of the paper is very limited:\n    - Previous work already suggests that care must be taken when interpreting local measures of flatness. In [1] it has been shown that local flatness alone is not enough, and we need global well-connectivity as well. \n    -The results regarding how re-scaling would break the correlation between generalization and flatness are also studied before in the literature (Dinh et al., 2017). \n    -The link between the Bayesian prior and generalization is studied in (Mingard et. al., 2021). More precisely, in (Mingard et. al., 2021), the authors show that $P_B (f | S)∝$ generalization error. As for a given $S, P_B (f | S)∝ P (f)$, then it is implicit that $P (f) ∝ $ generalization error, which is the main result of this submission. \n- The paper needs to be fully proofread. There are many typos throughout the paper (missing \"a\", \"the\", MINST instead of MNIST, the first sentence of Section 5.3 is incomplete, etc.). The flow of the paper also needs major adjustments. The paragraphs are not coherently written. Also, a lot of the sentences could be summarized/re-written. Some example suggestions:\n   - Definition 2.2 is not needed. \n   - In contribution number 2, explanations should be avoided. Instead only the main results should be stated. From the sentence \"For discrete .... \" the text should be moved to the proper section.\n   - In some parts of the text, some unusual terms are used. Examples: Section 2.1, \"the inputs live in \", \"the observed output y\".\n   - The parameter vector w is not defined in Section 2.1, and in Section 2.2 the loss function is a function of w; which is a mismatch with the notation used in Section 2.1 to define the loss function.\n   - Section 4 could be combined with Section 3.\n   - There are repetitions in Section 3 and some paragraphs do not follow well from one another. \n   - As there are no theoretical results, Section 2 could be significantly shortened. \n\n   All these would help with the flow of the ideas and would help the reader to stay focused on the main outcomes of the paper. In the current shape, the main outcomes are in the shadows: the results of the paper only come after page 6.\n\n- As the paper is an empirical study, it is lacking important ablation studies. The scope of the empirical results should be expanded (for example be similar to the scope of empirical results in [1]). One example case study is to compare different initial random seeds: are the networks with higher volume (higher Bayesian prior) also the networks with higher test accuracy? \n\n- It would be better to move the last paragraph of the related work (which is currently in the appendix) to the main paper, and discuss exactly the new contributions in light of the three mentioned prior work. It is implicit that if the Bayesian posterior is related to generalization (which is studied thoroughly in (Mingard et al., 2021)), then the Bayesian prior is also related to generalization. Stating this result is not a new claim. Although I believe that empirically showing this is interesting, for this paper to be an empirical study, many more ablation studies and settings should be studied to make the claim practically convincing.\n\n\nMinor comments:\n- There is a (rather new) study on neural network loss landscapes [1], where the authors show that the best accuracy is achieved when not only the model has converged to a locally flat region but also to a globally well-connected region. It would be interesting to compare the Bayesian prior with the combination of global connectivity and local flatness. What is the difference between global connectivity and Bayesian prior? Is there a relation between the two? Can we say that globally well-connected regions also have a large \"volume\"?\n- Similar to Definitions 2.1 and 2.2 which are properly cited, definition 2.3 should also be cited from (Mingard et al., 2021).\n- Figure 1 is a replica of figure 6.a of (Mingard et al., 2021). Proper credit should be given.\n- Throughout the paper, either only refer to flatness, or only refer to sharpness. The inconsistency makes the results/text less readable.\n- The results in Figure 2 are not the main focus of the paper. Instead, the main focus is to find scenarios where the correlation between the two and therefore the correlation between flatness and generalization breaks. So, there is no need to present Figure 2 in the main paper. \n\n[1] Yang et al., Taxonomizing local versus global structure in neural network loss landscapes, 2021\n\n\n\n\n", "summary_of_the_paper": "This paper studies the relation between generalization and the flatness of the loss landscapes. The authors show that the correlation between local flatness and generalization is broken in the case of using Adam or Entropy-SGD or by performing parameter rescaling. They show that the Bayesian prior of DNNs (which are trained until zero training error) could on the other hand predict generalization better than the local flatness measures. This is motivated by the work of (Mingard et. al., 2021), where it is shown that the Bayesian posterior function correlates well with the probability function of DNNs that reach zero training error using SGD. ", "summary_of_the_review": "Overall, although the two observations that the Bayesian prior is empirically linked with generalization and that when using Adam or Entropy-SGD flatness is no longer linked to generalization are interesting, the contribution of the paper is not enough. The novelty of the work is very limited and the empirical results are insufficient.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "JU7G2uDUHYY", "reviewer": "Reviewer_z9TP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "There are several recent works, which looked into the connection between generalization and the sharpness of the local minimum. Here, 'sharpness\" is often defined using the spectrum of the Hessian (in addition to some normalization terms in some cases). However, previous measures of sharpness are not invariant under parameter rescaling, which would produce the same solution but with a different level of sharpness. \n\nThe authors in this work propose an alternative measure: the log of the prior of the function as a predictor of generalization. The motivation behind this comes from recent lines of work which look into the relation between generalization and the posterior probability p(f | S). The authors note that the posterior is related to the prior p(f) by a constant term since neural networks are expressive enough to have zero training error. Hence, p(f) might serve as a good predictor of generalization. Here, the prior on the 0-1 function f is the total probability mass of the weights that produce that function over a fixed dataset (training + test), where the probability measure over the weights is computed using GP. The authors demonstrate experimentally on MNIST and CIFAR10 that there is indeed a strong correlation between p(f) and generalization. In addition, because this is defined over 0-1 functions, it is invariant to parameter rescaling. \n\nIn general, the paper is well-written and provide a nice historical review. However, it is not clear where the novelty/contribution is. As mentioned by the authors, previous works have already shown that p(f|S) is a good predictor of generalization. The only addition in this work is replacing p(f|S) with p(f). But, because p(f) is a constant multiple of p(f|S), claiming that p(f) correlates well with generalization is the same as claiming that p(f|S) correlates with generalization, which is what was previously established. \n\nIn addition, I do have concerns with the way p(f) is computed. In this paper, the authors use SGD as a way of sampling function f. If there exists good functions that are not often found by SGD, the space of functions studied by the paper would be a distorted version of the reality. For example, some works on reinitialization (e.g. https://arxiv.org/abs/2109.00267) suggest that there exists flatter local minima that are only reached by SGD if a subset of the parameters are reinitialized frequently. In other words, a good local minimum would be surrounded by sharper local minima that generalize less, but because the flatter local minima are surrounded by the sharper ones, SGD would settle on the sharper local minima. This is fixed using reinitialization. If one uses SGD to sample the functions f, many functions could be missing. \n\nOther comments. \n\nQuestions:\n- How is sharpness in Definition 2.1 computed in the experiments? \n\nMinor issues:\n- In Definition 2.1, it should be mentioned that w is a stationary point, otherwise the connection to the Hessian is wrong. \n- In Definition 2.2, the actual definition is on a separate paragraph. \n- The definition of p(S) can be misleading. It is a function of S, but not \"probability\" of the sample S. I suggest using a different notation (e.g. g(S) instead of p(S)). \n- The statement in Page 4 related to Figure 1 is not really justified. One can draw a picture in which that statement does not hold (i.e. that the volume of f is not correlated with p(f). It is just an intuition but it can be wrong and it is not clear to me why it is more likely to be true. Within the main text, the authors use a more acceptable argument about p(f) having different orders of magnitude but that's different from the claim in Figure 1. \n- Please move Figure 2 to the top of the page. \n- I am curious to know how SGD with momentum and the recent Sharpness-Aware Minimization (SAM) would perform in Figure 4. Have the authors experimented with them? \n\n\nTypos:\n- Page 6, \"The function the largest prior\". \n- Page 9: \"needed.Morever\" --> needs a full stop.", "summary_of_the_paper": "The paper proposes using the log of the prior of the function as a predictor of generalization. They provide arguments for why this is better than using sharpness of the local minima, and support their arguments with empirical results on small datasets. ", "summary_of_the_review": "The paper is well written. However, the main contribution in the paper seems to be identical to previous works. I would appreciate it if the authors elaborate on what their contribution is compared to the works of Valle-Pérez & Louis (2020) and Mingard 2021. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5DIv5GG0rs0", "reviewer": "Reviewer_ZCFx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths:\n\n1. In the specific settings considered, the correlation between generalization and the proposed Bayesian prior is extremely strong, and robustly outperforms the flatness measure.\n\n1. Appendix contains many sub-settings considering different measures of flatness and different training durations, all corroborating the findings in the main text.\n\n1. Disentangling the influence of architecture and optimization on generalization is an important area of research.\t\t\n\n## Weaknesses:\n\nThe main weakness of the paper is that all measurements are done under a somewhat exotic setup, that IMO does not resemble a realistic situation where one would do model selection based on some generalization measure. For this reason I find strong claims about the Bayesian prior being a more robust predictor than flatness (e.g. in the abstract and in the conclusion) unsubstantiated. Precisely:\n\n1. The authors use attack sets to generate models with 0 training loss of varying generalization. I suspect this might produce a very specific distribution of resulting functions. A much more natural and realistic setup would be to just sample many different hyper-parameters like learning rate, batch size, learning rate decay, momentum, number of epochs, weight decay etc. By pushing them to extremes, I am pretty sure it is possible to obtain 100%-train accurate yet poorly-generalizing models. And even if they all happen to generalize well, then that resulting smaller range of performance would be the realistic scale at which practitioners and researchers would want to see a correlation between generalization and the proposed proxy. Another more realistic parameter to vary would be just the training set size, since training on/obtaining more data could be costly, and one might want to estimate generalization scaling as a function of training set size. But an attack set setting is something that I find much less practically relevant, and worry that if we were to consider a practical setting as above, the correlation might not be there (since the manifold of models trained with varying attack sets and the more practically-relevant manifold of models trained with varying hyperparameters are likely very different). Please correct me if I'm wrong, or missing something.\n    \n1. IIUC the Bayesian prior should only be compared between the same models due to the assumption that $P(S)$ is constant (which is otherwise intractable). Therefore $P(f)$ cannot be used for model selection between different architectures or priors on the weights. Only the training set and the training procedure can vary. In contrast, flatness can be compared between any NNs with the same loss function, which is another reason why I think the claim about prior being a superior measure is too strong.\n    \n1. Finally, I think it's realistic to expect that there are manifolds of models where prior does not correlate with generalization, and it would be nice if the authors proactively considered / discussed such settings, to set more realistic expectations for the reader. For example, I suspect that in a regression / MSE loss setting, you could obtain a model that fits the training set perfectly, but then falls off to 0 rapidly everywhere else and doesn't generalize at all. Yet this model will have the highest prior possible, due to the prior being a zero-centered Gaussian. In a classification setting, perhaps constructing an attack set differently could also lead to different behavior (e.g. suppose I only misclassify one class incorrectly, i.e. return a constant class label for the whole attack set - would the prior still correlate with generalization then?). It's fair to leave this sort of investigation for future work, but without such I again find the central claim of the paper too strong.\n    \n\n## Questions:\n\n1. Is the sharpness measured on the training set, or the training and attack set jointly, and do you think it matters? Have you, or any prior work considered measuring sharpness on a separate validation set, and do you think it would make sharpness perform better?\n\n2. Have you considered similar experiments in an MSE / regression setting, with continuous-valued outputs? In this setting it would be simpler and cheaper to measure $P(f)$ in closed-form with NNGP, and I'm curious how MSE and accuracy would correlate with the prior / flatness in this case.\n\n## Minor:\n\n1. Abstract: \"zero error on a test set\" -> \"training set\"?\n1. Section 2.1: the loss is defined on the space of outputs $\\mathcal{Y}$, but is usually instead noted as $L(\\bf{w})$. This is inconsistent with definition but also makes it hard to understand if it's the train / train + attack / validation set loss.\n1. Page 5, line 2: $P_{SGD}(f|S)$?\n1. Figure 2 caption: \"The function _with_ the largest prior\"?\n1. Figure 2, regarding two bands: could you add different colors for them?\n1. Typos in section A.1, e.g. \"Wwe\",  \"type B>\".\n", "summary_of_the_paper": "In binary image classification setting, the paper shows that when using an attack set (augmenting the training set with additional intentionally mislabeled datapoints) to vary the generalization performance of a fixed neural network (NN) architecture, the (approximation of) Bayesian prior $P(f)$ of the outputs of the trained NN on the test set correlates _very_ well with generalization. The effect persists when using different optimizers, and different number of training epochs.\n\nIn contrast, the popular measure of flatness (approximation of the spectral norm of the Hessian of the training loss) correlates worse with generalization in these settings when using SGD, and does not correlate with it at all when using variants like Adam or Entropy-SGD.\n\nThe authors provide some preliminary explanation of the effect and propose the Bayesian prior as a more reliable generalization proxy.\n", "summary_of_the_review": "# Post-rebuttal update\n\nThe authors have answered my questions and clarified that we may have some differences in interpreting their results, but I don't think it's significant enough for me to change the original score.\n\n# Original Review\n\nThe paper presents an interesting setting in which the approximation to the Bayesian prior $P(f)$ correlates remarkably well with generalization (while flatness does not). \n\nAs I detail in my review, I find the setting to be somewhat constrained and unrealistic, and therefore the claim that the prior \"is a significantly more robust predictor of generalization than flatness\" is too strong, especially since there is no solid theoretical or qualitative explanation for why exactly $P(f)$ works.\n\nNonetheless, I still find this to be a novel and meaningful result, and look forward to future research investigating / explaining this effect further, so am inclined to accept (assuming the authors soften their claims or convince me that my concerns are overblown).\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes the use of Bayesian prior upon initialization for predicting generalization performance of a neural network, and empirically shows that it can outperform flatness-based measures. Understanding the underlying reasons that control generalization performance on neural networks is of great theoretical and practical importance, and reviewers find efforts in this direction valuable. However, they believe the submission in current state is not ready for publication.

Specifically, ZCFx believes the setup considered in the paper does not resemble a realistic situation, which makes claims about the Bayesian prior being a more robust predictor than flatness unsubstantiated. ZCFx appreicates authors' response and clarifications, but finds the concerns unresolved. ohft believes the paper is weak in a certain aspects, such as comparing across different architectures (including number of parameters), and comparing with SAM optimizer whose goal is to find flat minima and has shown to greatly improve the generalization performance. ohft acknowledged reading authors' response but the response did not help with changing the score of the paper. r1hF has some reservations about the novelty of the work and the limited experiments, which remained unresolved. r1hF suggests that the authors revise the paper to emphasize on the author's contribution in light of the previous work.

Based on reviewers' feedback, I suggest authors to resubmit after revising the draft to address the issues raised above.
Decision: Reject