On the Implicit Bias Towards Depth Minimization in Deep Neural Networks
Authors: Tomer Galanti
Liane Galanti
Ido Ben-Shaul
Venue: Submitted to ICLR 2023 (2023)

Abstract: Recent results in the literature suggest that the penultimate (second-to-last) layer representations of neural networks that are trained for classification exhibit a clustering property called neural collapse (NC). We study the implicit bias of stochastic gradient descent (SGD) in favor of low-depth solutions when training deep neural networks. We characterize a notion of effective depth that measures the first layer for which sample embeddings are separable using the nearest-class center classifier. Furthermore, we hypothesize and empirically show that SGD implicitly selects neural networks of small effective depths. 

Secondly, while neural collapse emerges even when generalization should be impossible - we argue that the \emph{degree of separability} in the intermediate layers is related to generalization. We derive a generalization bound based on comparing the effective depth of the network with the minimal depth required to fit the same dataset with partially corrupted labels. Remarkably, this bound provides non-trivial estimations of the test performance. Finally, we empirically show that the effective depth of a trained neural network monotonically increases when increasing the number of random labels in data.

Under review as a conference paper at ICLR 2023
ON THE IMPLICIT BIAS TOWARDS DEPTH MINIMIZA- TION IN DEEP NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Recent results the literature suggest that the penultimate (second-to-last) layer representations of neural networks that trained for classification exhibit clus- tering property called neural collapse (NC). We study the implicit bias of stochas- tie gradient descent (SGD) to favor of low-depth solutions when training deep neu- ral networks. We characterize a notion of effective depth that measures the first layer for which sample embeddings are separable using the nearest-class center classifier. Furthermore, we hypothesize and empirically show that SGD implicitly selects neural networks of small effective depths.
Secondly, while neural collapse emerges even when generalization should be im- possible argue that the degree of separability in the intermediate layers is related to generalization. We derive a generalization bound based on compar- ing the effective depth of the network with the minimal depth required to fit the same dataset with partially corrupted labels. Remarkably, this bound provides non-trivial estimations of the test performance and is independent of the depth.
INTRODUCTION
Deep learning systems have steadily advanced the state of the art in a wide range of benchmarks, demonstrating impressive performance in tasks ranging from image classification (Taigman et al.)
2014 Zhai et al. 2021), language processing (Devlin et al. 2019: Brown et al. 2020 open-ended environments (Silver et al. 2016 Arulkumaran et al. 2019) to coding (Chen et al. 2021.
Recent research indicates that deep neural networks generalize well, part because the number of parameters exceeds the number of training samples (Belkin al.) 2018 Belkin 2021 Advani
& Saxe 2017] Belkin et al. 2019). However, it has been shown that in this case deep learning models can precisely interpolate arbitrary training labels (Zhang al. 2017) (also known as the "interpolation regime"). Therefore, understanding interpolation learning appears to be a critical step toward better theoretical understanding of deep learning's successes.
Traditional generalization bounds [Vapnik 1998 Shalev-Shwartz & Ben-David 2014 Mohri et al. 2012 Bartlett & Mendelson 2003) are based uniform convergence. In this approach, instead of directly analyzing the population error of learning algorithm, a uniform convergence-tyee argu- ment would control the worst-case generalization gap (distance between train and test errors) over class of predictors containing the outputs of the learning algorithm. Typically, this is done because for many algorithms is difficult to exactly characterize the learned predictor.
Nagarajan & Kolter (2019), however, raised significant questions about the applicability of typical uniform convergence arguments certain interpolation learning regimes. They described theoretical settings in which an interpolation learning algorithm generalizes well but any uniform convergence bound cannot identify that. Following their work, Bartlett& Long 2021) Zhou al. (2020) Negrea al. 22000) Yang al.] (2021) all demonstrated the failure of forms of uniform convergence in various interpolation learning setups.
Contributions. Because of the inherent limitations of uniform convergence bounds, in this paper we pursue a novel approach for measuring generalization in deep learning that is not based on uniform convergence. Instead, complexity is small compared to the complexity of a network required to fit the same dataset with partially random labels. In other words, even if a trained network has complexity greater than the
bound suggests that the model performs well at test time if its

---

Under review as a conference paper at ICLR 2023
Dataset
MNIST CONN-1-50
Fastion MNNST
CIEARLO CONV-L-100
Archiiectun:
CONV-Z-100
Depth (5)
10
12
15
10
12
15
16
18
20
Test
0,0075
000074
0.0074
0.0996
0,9996
0,0966
0,2659
0,2553
0,2688
P
0.1
0.1
0.1
0.2
0.2
0.2
0.4
0.4
0.4
Our bound
0,1
0,1
0.2
0.2
0.2
0,66
0.66
0.53
Burten Mendehon
2003
X911es14
1.7ce17
2.13e122
3.13ge177
9.
22
1.76ee233
2,6822425
3.75
2015
54624405
7.523:407
6.9974407
2.636c408
4.833c408
2.275:400
5.06c:409
Frohenins
Nevslalur 45
2015
1348e406
8.1942405
2.216e407
2.456:008
2.335p440
1.583.409
1.9676009
1,422+10
3.036+111
Spec
Barten a
117
2361e405
6,4124005
9 0.566440
4.7664406
3.516e406
17666-06
1.19c407
1.49c408
1.272e+10
Syoc Fasb
Neydmbbur
3848e400
1.11994434
1.538c404
4.02294404
2.584e404
2.543e404
9.4833e404
1.01c406
1.033+408
Table 1: Comparing our bound with baseline bounds in the literature for networks of varying depths. Our error bound is reported in the fourth row, and the baseline bounds are reported in the bottom rectangle. While the test error is universally bounded by 1, the baseline bounds are much larger than 1, and therefore, are meaningless. In contrast, our bound achieves relatively tight estimations of the test error and unlike the baseline bounds, our bound is fairly unaffected by the network's depth.
number of training samples, it may be less complex than a model that fits partially random labels. As result, in such cases, our bound may provide a non-trivial estimate of the test error.
To formally describe our notion of complexity. we employ the notion of nearest class-center (NCC) separability. This property asserts that the feature embeddings associated with training samples belonging to the same class are separable according to the nearest class-center decision rule. While original results (Papyan et al., 2020) observed NCC separability at the penultimate layer of trained networks, recent results (Ben-Shaul & Dekel 2022) observed NCC separability also in intermediate layers. In this work, we introduce the notion of "effective depth' of neural networks that regards to the lowest layer for which its features are NCC separable (see Sec. 3.2)
We make multiple important observations regarding effective depths. (i) We empirically show that the effective depth of trained networks monotonically increases when increasing the amount of ran- dom labels in data. (ii) We observe that when training sufficiently deep networks, they converge to (approximately) the same effective depth Lo. Furthermore, as we show in Tab. [ unlike traditional generalization bounds, our bound is empirically non-vacuous and independent of depth.
1.1 RELATED WORK
Neural collapse and generalization. Our work is closely related to the recent line of work on Neural collapse (Papyan et al. 2220; Han et al. (2022). Neural collapse identifies training dynamics of deep networks for standard classification tasks, where the feature embeddings associated with training samples belonging the same class tend to concentrate around their means.
While several papers analyzed the emergence of neural collapse from a theoretical standpoint (e.g.. (Zhu et al. 2021 Rangamani et al.) [022] [u & Steinerberger 2020 Fang et al. [021]] EErgen & PPllanci 2021). its specific role in deep learning and its potential relationship with generalization still unclear. Recent work (Galanti al. 2022a Xu et al. 20222] Galanti et al. (022b) studied the
conditions for when class features variation collapse generalizes from the train samples, to both test
samples and new classes and in the transfer learning setting.
We focus on the following (independent) question in this work: Is neural collapse a good indicator of how well network generalizes? As a counter-argument, Zhu et al. (202T) provided empirical evidence that neural collapse occurs even when training the network with random labels. As a result, the presence of neural collapse cannot indicate whether or not the network generalizes. This experiment, however, does not rule out the possibility of an indirect relationship between neural collapse and generalization. We contend that the degree of separability in the intermediate layers is related to generalization.
Emergence of structure in deep networks. While various papers Papyan 2020); Tirer & Bruna
(2022; Galanti et al. (2022a): Bhn-Shaul & Dekel (2022); Cohen et al. (2018); Alain & Bengio
& Dekel (022): Shwartz-2i Ziv & Tishby 2017) investigated certain geometrical properties within intermediate layers (c.g., clustering and separability). this paper is the first to demonstrate that deep neural networks tend to converge to minimal effective depth that is independent of the network's depth. Even though one can de-
2017); Montavon et al. (2011); Papyan et al. (2017):
2

---

Under review as a conference paper at ICLR 2023
rive "efffective depths" from the experiments of Cohen al. (2018), show that when training sufficiently deep networks they converge to (approximately) the same effective depth.
PROBLEM SETUP
In this section we describe the learning setting we use in the theory and experiments. We consider the problem of training model for standard multi-class classification. Formally, the target task is defined by a distribution and D/c is a label space with cardinality C. To simplify the presentation, we use one-hot encoding for the label space, that is, the labels are represented by the unit vectors IRC and Vc == fee = 1,..., C'} where ee E RC is the cth standard unit vector in RC; with slight abuse of notation, allow ourselves to write y = c instead
over samples (x., y) X' Vc. where X' C Rdd is the instance space,
= ec. For a pair (z.y) v) distributed by P, we denote by P. the class conditional distribution of given y c (i.e., Pc(+) P(z E
c)>.
classifier hw : X + RC assigns soft label to an imput point x E X, and its performance on the distribution P measured by the expected risk
Lp(hw) == (x),
where l: RC Vcc 10, oo) is non-negative loss function (e.g., L2 cross-entropy losses).
We typically do not have direct access to the full population distribution P. Therefore, we generally aim to learn classifier, h, using some balanced training data S := {( u)) w)) U i Mei 1 (B(m) of m C mo samples consisting me independent and identically distributed (i.i.d.) samples drawn from Po for each e E [C]. Specifically, we intend to find W that minimizes the regularized empirical risk
(swh)) Σ ((hw(as) 1/1.
(1)
where the regularization controls the complexity of the function hw and typically helps re- ducing overfitting Finally, the performance of the trained model is evaluated using the train and test error rates; errs(hw) == Σ I[arg max, hw (x;)s * yil and erp((hw) B(x.))> (U[arg max, /w(xx)))))))))))))))())) & yll. where I: (True, False) + {0, 1} the indicator function.
Neural networks. In this work, the classifier hw is a neural network, decomposed into a set of parametric layers. Formally, we write hw == ew, o fi, siv, E {g' R + RP } are parametric functions and ew, E (e' RP - RC a linear function. For example, gw. could be a standard linear or convolutional layer, a residual block or a pooling layer. Here, an element-wise ReLU activation function. With slight abuse of notation, we omit specifying the sub-scripted weights, fs := g o ... og² and h == hw.
ew. o 9we ... sw. where
We optimize our models to minimize the regularized empirical risk (h) by applying SGD for a certain number of iterations T with coefficient À > O. Specifically, we initialize the weights Wo = 2 of using a standard initialization procedure and at each iteration, we update W < W₁ μr V T.g (h+), where f4 > 0 is the learning rate at the f'th iteration and the subset of size B is selected uniformly at random. Throughout the paper, denote by h the output of the learning algorithm starting from the initialization Wo 7. When is irrelevant or obvious from context, will simply write hi hg eg Ss-
Optimization.
Š Sf
3 NEURAL COLLAPSE AND GENERALIZATION
In this section we theoretically explore the relationship between neural collapse and generalization. We start by introducing neural collapse, NCC separability, and effective depth neural networks. Then, connect these notions with the test-time performance of neural networks.
3.1 NEAREST CLASS-CENTER SEPARABILITY
Neural collapse identifies training dynamics of deep networks for standard classification tasks, in which the features of the penultimate layer associated with training samples belonging to the same
3

---

Under review as a conference paper at ICLR 2023
class tend to concentrate around their collapse, (NC2) the class means of the embeddings collapse to the vertices of a simplex equiangular tight frame, (NC3) the last-layer classifiers collapse to the class means up to scaling and (NC4) the classifier's s decision collapses to simply choosing whichever class has the closest train class mean, while maintaining zero classification error.
This includes (NC1) features variability
In this paper we focus on a weak form of NC4 we call "nearest class-center separability" (NCC separability). Formally, suppose we have a dataset S = -E=1S of samples and a mapping f Rdd + RR,, the features of f are NCC separable (w.r.t. 5) if for all E [m], we have h(x;)) yi- where h(z) := arg min [C] (z) - (- To measure the degree of NCC separability of a feature map f. we use the train and test classification error rates of the NCC classifier on top of the given layer, errs (h) and (h)
Essentially, NC4 asserts that during training, the feature embeddings in the penultimate layer be- come separable and the classifier h itself converges the 'nearest class-center classifier' h.
3.2 EFFECTIVE DEPTHS AND GENERALIZATION
In this section we study the effective depths of neural networks and their connection with general- ization. To formally define this notion, we focus on neural networks whose I. top-most layers are of the same size. We observe that neural networks trained for standard classification exhibit an implicit bias towards depth minimization.
Observation 1 (Minimal depth hypothesis). Suppose we have a dataset S. There exists an integer Lo 2 1, such that, if we train a neural network of any depth L Z La for cross-entropy minimization on S using SGD with weight decay, the learned features I6 become (approximately) NCC separable for all I E (Lo.....L. L).
We note that if the Lo 'th layer of fr. exhibits NCC separability, we could correctly classify the sam- ples already in the Lo 'th layer of fo using a linear classifier (i.c., the nearest class-center classifier). Therefore, intuitively its depth is effectively upper bounded by Lo. The notion of effective depth of neural network is formally defined as follows.
Definition 1 (e-effective depth). Suppose we have dataset S and a neural network h g" o g² with g² R" + IRP,, , g : RP: + IR++ and linear classifier e -RP++1 + RCC Let (x( = arg min (l) - 1 (S.)ll- The e-effective depth d'g(h) of the network h is the minimal value [L], such that, s S e (and d'g (h) if such E [L] is non-existent).
To avoid confusion, we note that the e-effective depth a property of a neural network and not of the function it implements. That is, a function can be implemented by two different architectures of different effective depths. While our empirical observations in Sec. 4 suggest that the optimizer learns neural networks of low-depths, it is not necessarily the lowest depth that allows NCC sepa- rability. As a next step, we define the e-minimal NCC depth. Intuitively, the NCC depth of a given architecture is the minimal value L E N. for which there exists neural network of depth L whose features are NCC separable. As we will show, the relationship between the e-effective depth of a neural network and the c-minimal NCC depth is connected with generalization.
Definition 2 (e-Minimal NCC depth). Suppose we have dataset S UZ and a neural network architecture f g" ....o g" with g¹ : R + R and g' E g C l :Ro + R } for all = 2, ... L. The e-minimal NCC depth of G is the minimal depth L for which there exist parameters W (W.), such that, f == fly gWv o... gw, satisfies errs(h) S where h(x) == arg min [If((z) - (S []. We denote the e-minimal NCC depth by (9,
To study the performance of given model, we consider the following setup. Let S = {x+ ) ) and S2 = [ar] )²) be two balanced datasets. We think of them as two splits of the training dataset S. We assume that the classifier h3, is trained on S1 and we use S2 to evaluate its perfor- mance. We denote by X,- { and Y, fm the instances and labels in Sy-
To formally state our bound, we make two technical assumptions. The first is that the misclassified labels that h22 produces over the samples X₂ = Ug_ M
mmo are distributed uniformly.
4

---

Under review as a conference paper at ICLR 2023
Definition 3 (6m--um uniform mistakes). We say that the mistakes of learning algorithm A (S1, 7) + his are 6m -niiformm if with probability 2 1 5m over the selection of S1 ~ PD(m), the values and indices of the mistaken labels of h2, over X2 are uniformly distributed (as a function r().
The above definition provides two conditions regarding the learning algorithm. It assumes that with a high probability (over the selection of $1, S2), h3,, makes the same number of mistakes on S2 across all initializations ?. In addition, it assumes that the mistakes are distributed uniformly across the samples in S2 and their (incorrect) values may be violated in practice, the train error typically has a small variance and the mistakes are almost distributed uniformly when the classes are non-hierarchical (e.g.. CIFARIO, MNIST).
also distributed uniformly. While these assumptions
For the second assumption, we consider the following term. Let p E (0,1/2), a E (0, 1), we denote
8 m.p.a == IP So. [3 (1+ Ln (G,
S U.) > 1y,1dmin I S1 U Sa)]]
(2)
where y2 = 10 and Y,- = { are uniformly selected to be sets of labels that disagree with Y2 on pm and gm values (resp.) and $2 and $2 are datasets obtained by replacing the labels of S2 with Y2 and Y/2 (resp.). We assume that pmpp. is small. Meaning, with high probability, the minimal depth to fit (2 p)m correct labels and pm random labels is upper bounded by the expected minimal depth to fit (2 m)m correct labels and gm random labels for any q 2 (1 + a ). To understand this assumption, we note that in both cases, the model has to fit least m correct labels and pm (or qm) random labels. However, we typically need to increase the capacity of the model in order fit extended amounts of random labels (see Figs. 3
Following the setting above, we are prepared to formulate our generalization bound.
Proposition 1. Let m E N. p E 10,, a E (0, and c E (0. 1). Assume that the error of the learning algorithm is om -uniform. Assume that S2 PB(m) Let N2,, be the output of the learning algorithm given access to a dataset S and initialization 7, Then,
Es, EyTT,/erTp(ht (hg,)) I S. B (,,0,,03,) 2 dimin (9.
(3)
+ a)p 6 1 + 6m..0.0
where Y, (i.)) is uniformly selected to be a set of labels that disagrees with Y2 on pm values.
The above proposition provides an upper bound on the expected test error of the classifier h22 which is the term that we would like to bound. The proposition assumes that the mistakes h3, generates on X2 are distributed uniformly (with probability 2 1 - 6.h)) To account the likelihood that this assumption fails, our bound includes the term 6%,, which is assumed to be small.
Informally, the bound suggests the following idea to evaluate the performance of h2, We start with an initial guess Pm = p E 1/2) of the test error of h2g- Using this guess, compare its e-effective depth with the e-minimal NCC depth (9, S U required to NCC separate the samples in S₁ U S2, where Š2 is the result of randomly relabeling Pmm of Sy 's labels. Intuitively, if the mistakes of N22 are uniformly distributed and its eeffective depth smaller than 4in Si U
52), then, we expect h2, to make at most Pm mistakes S2. Therefore, in a sense, the choice of Pm serves as a 'guess' whether the effective depth of a model trained with S, is likely to be smaller than the e-minimal NCC depth required to NCC separate the samples Sj U S.--
Next, we interpret each term separately. The term [d's (h2, )] depends on the complexity of the classification problem and the implicit bias of SGD to favor networks of small e-effective depths. In the worst case, if SGD does not minimize the e-effective depth or the labels in S₁ are random (and m is sufficiently large), we expect 1', (h3, )] L. On the other hand, dim. (G, S1 U US2) measures the complexity of a task that involves fitting a dataset of size 2m samples, where (2 Pm. pmmm > m of the labels are correct and pmm m are random labels. By decreasing Pm. we expect d d (g, S U2) to decrease, making the first term in the bound larger. In addition, if h of a fixed width, it is impossible to fit an increasing amount of random labels without increasing the depth. Therefore, when Pm m - oc, the dataset S U Sg becomes increasingly harder to fit, and
f" is neural network
00
5

---

Under review as a conference paper at ICLR 2023
expect S1 U S2) to tend to infinity. If E, (,,, (h,,)) )] is bounded as function of L and m and if pm - 0o, we obtain that [dE,, (h2, ) 2 min (g, S1 U U2)] = 0 and together m=000
m=+00
with Pm 0, we have Ess, (es) )1 - on + p.p.a + 0m (1).
As a side note, computing the expectation over Si, S2 in the bound is impossible, due to the limited access of the training data. However, instead, we empirically estimate this term using set of l: pairs (S), Si) of m samples, yielding additional term that scales as (L/VV)) to the bound (see Prop in the appendix).
3.3 COMPARING PROP. WITH STANDARD GENERALIZATION BOUNDS
Classic bounds (e.g., [Vannik (998)) are based on bounding the test error with the sum between the train error together with term O (C(())/)) where C(H) measures the complexity (c.g., VC dimension) of the class H. (c.g., neural networks) and m is the number of training samples. However, as discussed in Sec. T these bounds are vacuous in overparameterized learning regimes (e.g., training Re on CIFARIO classification). For instance, for VC-dimension based bounds (Vapnik) 1998). C(H) equals the VC-dimension of the class H which scales with the number of trainable parameters for ReLU networks (Bartlett et al. 2019). For example, even though the ResNet-50 architecture generalizes well when trained on CIFARIO, it has over 23 million parameters compared to the m 50000 training samples in the dataset.
More recently, Neyshabur et al. (0155: : Bartlett et al. 2017 ) Golowich et al. 2017 Neyshabur
et al. 2018 suggested generalization bounds for neural networks that weakly depend on uniform convergence. In these bounds, the class-complexity C(H) is replaced with the individual complexity C(hw) of the function we learn. For example, Golowich et al. 2017) proposed bounds that scale with C(hw) = p² L. where L is the depth of hw and p measures the product of the norms of its weight matrices. However, Nagarajan & Kolter (2019) showed that in certain cases unregularizes least squares can generalize well even when its norm p scales ((vm) and the bound becomes ,n, (1). Furthermore, these bounds tend to be very large in practice (see Tab. 8 in (Neyshabur et al., 2019) and Tab. D and are negatively correlated with the test performance (Tiang et al. | 2020). In ad- dition, if the network's 's weight matrices' norms are larger than 1, quantities like p grow exponentially
when L is varied. As shown in Tab. this is empirically the case.
Our Prop. Offfes a different way to measure generalization. Since this bound is not based on uni- form convergence, it does not require that the network's complexity would be small in comparison to m, rather, the bound guarantees generalization if the network's effective size is smaller than that of a network that fits partially random labels. For instance, when the optimizer has a strong bias towards minimizing the effective depth, E, 14% (h3, )] ~ d (1) which is by definition upper bounded by minnnG (G, U$2). We note that din (G. S U.))) grows to infinity as m + oo (since the network needs to memorize m - do random labels). On the other hand, & (9, S1) is bounded by the depth of a network that approximates the target function y up to an approximation error e (which typically exists due to universal approximation arguments). Therefore, for sufficiently large m, we expect to have d Mini (G. Si U$²) > d min (G,S) Si). As we empirically see in Sec. the effective depths of SGD-trained networks are usually small.
Unlike previous bounds, our bound has the advantage of being fairly independent of L. Namely, when the minimal depth hypothesis (Obs. [ holds, we expect E,dd', (h2,) )1 to be unaffected by the depth L of h3, (as long L 2 Lo). Since (i S1 U S2) is by definition independent of L, expect Id', (h3,)] 2 di (g, S₁ U 52)] to be independent of L (when L 2 Lu). In Tab. we empirically validate that our bound does not grow when increasing L.
4 EXPERIMENTS
In this section, we experimentally analyze the emergence of neural collapse in the intermediate layers of neural networks. First, we validate the "Minimal Depth Hypothesis" (Obs. []] Following that, we look at how corrupted labels affect the extent of intermediate layer NCC separability and the e-effective depth. We show that as the number of corrupted labels in the data increases, so does

---

Under review as a conference paper at ICLR 2023
NCC
0.6G-
0.6
700
200
320
463
500
100
300
300
430
500
100
200
363
403
500
Epoch
Epoth
Epoch
luyers
layers
08-
NCC NCC
Accurracy
0.4
0.4
1497
0.0
200
200
220
402
200
420
3.0
300
200
502
402
300
Epoch
Epoch
Epoch
layers
layers
layers
Figure 1: Intermediate neural collapse of CONV-L-400 trained on CIFAR10. We plot the NCC train accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
Eltectre Dapth
Effectee
CIFARIO, &
Fashion MNIST. & MLP- MLP-L-100 P-1-100
Fishion MNIST. À
CONY- 400
Came- T-100
Etfedivv Drenth
Effective Deeth
Efrect
CIEARIO, 20
Fushion MINIST,
Fashion MINIST, & CONV L-100
CONV Z.
MLP 1-0
Figure 2: Averaged e-effective depths over the last few epochs. We plot the e-effective depth (y-axis) as function of (x-axis). Each line specifies the e-effective depth of neural network of a certain depth L. We show the averaged e-effective depth over the last k = 1, 20 epochs across initializations. The network's architecture, dataset and & are specified below each plot.
the e-efeectvve depth. Finally, using the bound in Prop. we provide non-trivial estimates of the test error. In Tab. [ empirically compare bound with relevant baselines and show that, unlike other bounds, it achieves non-vacuous estimations of the test error. Throughout the experiments, we used T8sla-k80 GPUs for several hundred runs. Each run took between 5-20 hours. For additional experiments, see Appendix A The plots are best viewed when zoomed in.
4.1 SETUP
Training process. We consider k-class classification problems (e.g., CIFARIO) and train multi- layered neural networks h f4 g oz* Rn Rº on the corresponding training dataset S. The models are trained with SGD for cross-entropy loss minimization between its logits batch size 128, learning rate schedule
and the one-hot encodings of the labels. We consistently with an initial learning rate 0.1, decayed three times by a factor of 0.1 at epochs 60, 120, and 160, momentum 0.9 and weight decay 5e-4. Each model is trained for 500 epochs.

---

Under review as a conference paper at ICLR 2023
1.0
NCC train acc.
Accuroy
6.66-
0.1
0.6
0.2
100
205
300
100
520
150
203
500
520
330
2.0
620
500
Epoch
Epoch
Epoch
2.0
0.0
0.5
NCC test aeC
Accurao)
0.2
0.2
420
Epoch
Epach
Epoch
0% nouse
25% noise
75% noise
Figure 3: Intermediate neural collapse of CONV-10-400 trained CIFAR10 with partially corrupted labels. We plot the NCC train/test accuracy rates of the various layers of a network trained with certain amount of corrupted labels (see titles).
Datasct
MNIST CONV-10-50
Fashion MNIST CONV-10-100
CIFARIO
CIFARIO
Architecture
CONV-I6-100
CONVRES 10:30
0.0075
0.0996
0.2676
0.29
0.05
0.075
0.
0.08
0.15
0.2
044
0.45
0.S
D4
Bound
0.75
0.72
Table 2: Estimating the bound in Prop.
[ We used e 0.005 to measure the effective depths.
Architectures. We used three types architectures: (a) MLP-L-H with L fully-connected layers of width H, (b) CONV-L-H with L x 3 convolutional layers with padding 1, stride 1 and H output channels and (c) a residual convolutional network CONVRES- with residual blocks with two 3 convolutional layers. In each network the layers are interlaced with batch normalization layers and ReLU activations. For more details see Appendix A.
Datasets. We consider various datasets: MNIST, Fashion MNIST, and CIFAR10. For CIFAR10 we used random cropping, random horizontal flips, and random rotations (by 15k degrees for uniformly sampled from (24)). All datasets were standardized.
4.2 RESULTS
Intermediate neural collapse. To study the bias towards minimal depth, we trained a set of CONV-T.-400 networks CIFAR10 with varying depths. In each plot at Fig. [ we report the train NCC classification accuracy rates for each intermediate layer of a network of a certain depth. We make multiple interesting observations: (i) For networks with 8 or higher hidden layers, the eighth and higher layers exhibit NCC train accuracy of approximately 100%, and therefore, are effectively of depth 7. (ii) We observe that neural collapse strengthens when increasing the network's depth, on both train and test data. (iii) The embeddings of the top layers become NCC separable approximately the same epoch. (iv) The degree of NCC separability of intermediate layer converges as function of L. The results of this experiment are substantially extended and repeated with different architectures and datasets in the appendix (see Figs. 13 In these experiments we report the NCC train and test accuracy rates along with additional measures of neural collapse when varying the depth. Specifically, in Figs. and 5 we report the results with CONVRES-L-500,
The effect of the depth on the e-effective depth. In Obs. | we claimed that the e-effective depth insensitive to the actual depth of the network (once it exceeds a certain threshold). To validate conducted the following experiments. We trained models on MNIST, Fashion
this hypothesis MNIST and CIFAR1O with varying depth L. In Fig. 2 we plotted the averaged e-effective depths of each network's 's last k 1, 20 epochs function of We also average the results across
8

---

Under review as a conference paper at ICLR 2023
different weight initializations and plot them along with error bar standard deviations. As can be seen, the e-effective depth is almost unaffected by the choice of L for a given e. Remarkably, for each e, the averaged effective depth varies very little across the various networks. Differently said, the c-effective depths of two trained deep networks of different depths are more or less the same, validating our Minimal Depth Hypothesis.
NCC separability with partially corrupted labels. Simply put, Prop. D compares the depths required to fit correct labels and partially corrupt labels. To better understand the effect of corrupted labels on the complexity of the task, we compare the e-effective depths of models trained with varying amounts of corrupted labels. Namely. we study the degree of NCC separability in the intermediate layers of neural networks that are trained with varying amounts of corrupted labels.
For this experiment we trained instances of CONV-10-400 for CIFARIO classification with 0%. 10% and 75% corrupted labels (e.g., uniformly distributed random labels). We plot the degrees of NCC separation on the train and test sets, 1- em and 1 (h.) across the intermediate layers of the neural networks during the optimization procedure.
As can be seen in Fig- when increasing the amount of random labels, the degree of NCC sepa- rability across the intermediate layers tend to decrease. For example, when training with 2 25% corrupted labels, the sixth layer's NCC accuracy rate drops lower than 98%, in comparison with training without corrupted labels that gives us > 98% accuracy. In particular, the c-effective depth 0.02 (see Def. This experiment is
of the former network is 6 while the latter's is 5, when extended and repeated in a variety of settings in Figs. 14 18
Estimating the bound in equation 3 We estimate the bound in equation 3 for multiple architec- tures and datasets. In each case we used E <= 0.005 by default and employed different 'guesses' p (see Tab. () depending on the complexity of the learning task. We report an estimation of the ex- pected test error of the models, [ of p. For concrete technical details, see Appendix A
)] and an estimation of the bound for each selection
As can be seen, for appropriate selections of p. we obtained non-trivial estimates to the test perfor- mance of the models, which are almost unheard of when it comes standard bounds for deep neural networks. As expected, if the guess p is overoptimistic (e.g., close to Es., perrp(h3, )D, then, the first term in the bound tends to be large compared to Es (h3, )]-
Comparing our bound with standard generalization bounds. Since the e-effective depth of sufficiently deep neural networks is insensitive to depth (see Fig. 2 we expect the bound to be insensitive to depth as well. We estimate the bound in equation 3 for CONV-L-50 trained on MNIST and CONV-L-100 trained on Fashion MNIST and CIFARIO with L = 10, 12, 15 for the first two and with L = 15, 18, 20 for CIFARIO. As shown in Tab. [ we obtain similar bounds for each depth. Finally, we compare
bound several baseline generalization bounds for deep networks to show that it outperforms traditional generalization bounds. We used the implementation of [Nesshabrr al. (2019) to compute the bounds. While our bound is empirically non-vacuous and fairly independent of depth, the traditional bounds are extremely vacuous and rapidly grow when increasing the depth.
CONCLUSIONS
Understanding the ability of SGD to generalize well when training overpurameterized neural net- work is attributed as one of the major open problems in deep learning theory (Zhang et al. 2017). In this paper we offer a new angle to study the role of depth in deep learning and the connection between neural collapse and generalization.
We characterize a notion of effective depth that measures the lowest layer that enjoys NCC separa- bility. We introduce novel generalization bound that measures the likelihood in which the effective depth of a trained neural network is (strictly) smaller than the minimal depth required to achieve NCC separability with partially corrupted labels. This criterion, as demonstrated empirically, is good predictor of generalization. Furthermore, we characterize and empirically demonstrate that when sufficiently deep networks are trained, they converge to the same effective depth, implying that our bound is fairly constant when the depth is varied.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Madhu S. Advani and Andrew M. Saxe. High-dimensione dynamics of generalization error in neural networks, 2017. URL https: httpss///arxvvi/ werexiv.ogg/aaa.. http//bs/1111.... 03667
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. ArXiv, abs/1610.01644, 2017.
Kai Arulkumaran, Antoine Cully, and Julian Togelius. Alphastar: An evolutionary computation perspective, 2019.
org./ abs /19p. 01724 cite 1rxiv:190..01722
Peter L. Bartlett and Philip M. Long. Failures of model-dependent generalization bounds for least- norm interpolation. Journal of Machine Learning Research, 22((04):1 2021. URL http:
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res., 3(null):463-44) 44244466,,442 mar 2003. ISSN 1532-4435.
Peter L. Bartlett, Dylan J. Foster, and Matus Telgarsky. Spectrally-normaaiize margin bounds for neural networks. In Proceedings of the 31st International Conference on Neural Information Red Hook, NY, USA, 2017. Curran Associates
Processing Systems, NIPS' 17, pp. Inc. ISBN 9781510860964.
Peter L. Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mchrabian. Nearly-tight ve-dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning html
Research, 20(63):1-17, 2019. /tRp:t httpa//...raappppee///////////
Mikhail Belkin. Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation. Acta Numerica, 30:203 248, 2021.
Mikhail Belkin, Siyuan Ma, and Soumik Mandal. To understand deep learning we need to un- derstand kernel learning. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learn- ing Research, pp. 541-549. PMLR, 10-15 Jul 2018. URL htttps://proceeddngg mlrr.
press/ watsss//vB/bekkii11m81 html
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconeiling modern machine- learning practice and the classical bias- variance trade-off. Proceedings of the National Academy of Sciences,
2019. doi: f0.1073/pnas.190000 1903070116. URL https:
/ /ww. pnas ow.. /oi /abs/ /10 107.3/1as...... 1903070116
Ido Ben-Shaul and Shai Dekel. Sparsity-probe: Analysis tool for deep learning models. ArXiv,
Abs/2105.068499 2021.
Ido Ben-Shaul and Shai Dekel. Nearest class-center simplification through intermediate layers. CoRR, abs/2201.08924, 2022. htLlhttps//aaxiii wwtpa//w.rggaa//////eea.aa abs//bbls............... 2201.2211111 02.088924
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Hlerbert-Voss, Gretchen Krueger. Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde Oliveira Pinto, Jared Kaplan, Harri Edwards. Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis. Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec
10

---

Under review as a conference paper at ICLR 2023
Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Me- Grew, Dario Amodei, Sam McCandlish, llya Sutskever, and Wojciech Zaremba. Evaluating large language models trained code, 2021.
Gilad Cohen, Guillermo Sapiro, and Raja Giryes. Dnn or k-nn: That is the generalize vs. memorize question, 2018. URL / www.aaiwwwg..aaaaaaaaaaa www./oggg......ooooo...
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume I (Long and Short Papers). Association for Computational Linguistics, jun 2019.
Tolga Ergen and Mert Pilanci. Revealing the structure of deep neural networks via convex duality. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 3004-3014. PMLR, 18 24 Jul 2021.
Cong Fang, Hangfeng He, Qi Long, and Weijie J. Su. Exploring deep neural networks via layer- peeled model: Minority collapse imbalanced training. Proceedings of the National Academy of Sciences, 118(43), 2021.
Tomer Galanti, András György, and Marcus Hutter. On the role of neural collapse in transfer learning. International Conference on Learning Representations, 2022a. URL https: //openzeview.... wttp:/teee/ooommmaaaddddododdd..
Tomer Galanti, András György, and Marcus Hutter. Improved generalization bounds for transfer learning via neural collapse. ICML Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward, 2022b.
Noah Golowich, Alexander Rakhlin, and Ohad Shamir. Size-independent sample complexity of neural networks. Information and Inference: Journal of the IMA, 9, 12 2017. doi: 10.1093/ imaiai/iaz007.
X.Y. Han, Vardan Papyan, and David L. Donoho. Neural collapse under MSE loss: Proximity to and dynamics on the central path. International Conference on Learning Representations, 2022. URL wttpthttss:///oeeeeetht
net wwt. ftt://orumm?? wdd.:wmwmmdwwwwwwwww:: R3
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantas- tic generalization measures and where find them. In International Conference on Learning Representations, 2020. URL httpttts/////rreeeetttt
Jianfeng Lu and Stefan Steinerberger. abs/2012.08465,, 2020.
Neural collapse with cross-entropy loss. CoRR,
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. The MIT Press, 2012. ISBN 026201825X.
Grégoire Montavon, Mikio L. Braun, and Klaus-Robert Müller. Kernel analysis of deep networks. J. Mach. Learn. Res., 12:2563-2581, 2011.
Vaishnavh Nagarajan and J. Zico Kolter. Uniform convergence may be unable to explain generaliza- tion in deep learning. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc. E. Fox, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019. URL htttpppps//pppcceeiigi......//sp neuripss http:a//ww....cppppe//////////1. 03e5%7c2277333dddebbbbbbbob
Papero--Paprr.e pdf
Jeffrey Negrea, Gintare Karolina Dziugaite, and Daniel M. Roy. In defense of uniform convergence: Generalization via derandomization with an application to interpolating predictors. Proceed- ings of the 37th International Conference Machine Learning, ICML.20. '0. JMLR.org. 2020.

---

Under review as a conference paper at ICLR 2023
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. In Peter Grünwald, Elad Hazan, and Satyen Kale (eds.). Proceedings of The 28th Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pp- 1376-1401, Paris, France, 03-06 Jul 2015. PMIR. URL.
ngg mmlr.
3reas//440/Neeshabrr155...
Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. PAC-bayesian approach to spectrally-normalizedd margin bounds for neural networks. In International Conference on Learn- ing Representations, 2018. BtLLhtttss///oeeeeeett http://iwweeettefrrreeeeeeeeeeee www.:eeefooommmissseeeeee.
____C
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro, To- wards understanding the role of over-parametrizztion in generalization of neural networks. ArXiv, abs/1805. 2019.
Vardan Papyan. Traces of class/cross-class structure pervade deep learning spectra. Journal of Machine Learning Reseurch, 21(252):1-64, 2020.
20-933. html
Vardan Papyan, Yaniv Romano, and Michael Elad. Convolutional neural networks analyzed via convolutional sparse coding. J. Mach. Learn. Res., 18:83:1-83:52, 2017.
Vardan Papyan, X. Y. Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40); 24652-24663, 2020.
Akshay Rangamani, Mengjia Xu, Andrzej Banburski, Qianli Liao, Tomer Galanti, and Tomaso Poggio. Dynamics and neural collapse in deep classifiers trained with the square loss. Technical report, Center for Brains, Minds and Machines (CBMM), 2022.
Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning From Theory to Algorithms. Cambridge University Press, 2014. ISBN 978-1-10-705713-5.
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. ArXiv, abss/7733.008 2017.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schritwieserr, loannis Antonoglou, Veda Panneershelvam, Mare Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529:484 489, 2016. ISSN 0028-0836. doi: 10.1038/nature1690
Yaniv Taigman, Ming Yang, Marc' Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance face verification. In Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
Tom Tirer and Joan Bruna. Extended unconstrained features model for exploring deep neural col- lapse, 2022. URL htttttttp///aiii/// org/ http//bbb/222000///
Vladimir N. Vapnik. Statistical Learning Theory. Wilcy-Interscience,, 1998.
Chenfeng Xu, Shijia Yang. Tomer Galanti, Bichen Wu, Xiangyu Yue, Bohan Zhai Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Image2point: 3d plout-cloud understanding with 2d image pretrained models. In IEEE European Conference Computer Vision (ECCV), 2022.
Zitong Yang, Yu Bai, and Song Mei. Exact gap between generalization error and uniform conver- gence in random feature models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11715 PMLR, 18-24 Jul 2021. URI. https: /pwcc////eeeee/ooooooooooccooe.c ngs. mmlr. press/v139/yane html
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers, 2021.
12

---

Under review as a conference paper at ICLR 2023
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. Internationaa Conference on Learning Rep- resentations, 2017.
Lijia Zhou, Danica J. Sutherland, and Nati Srebro. On uniform convergence and low-norm inter- polation learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6867-6877, Curran Asso- ciates, Inc., 2020. URL /ttEpp/s/prcoeednnss..eeeer
cc wctppp/paer/22//1/1eeee.
Zhihui Zhu, Tianyu DING, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A geometric analysis of neural collapse with unconstrained features. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL httptttsst/////// /ttttrs// ew net/ fo rum?i d=KROD.AA
13

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL EXPERIMENTS AND DETAILS
A. ARCHITECTURES
In this section we give a detailed description of the architectures used in the experiments.
The first architecture is a convolutional network, denoted by CONV-L-H. The network starts with stack of 2 x convolutional layer with stride 2, batch normalization, a convolution of the same structure, batch normalization, and ReLU. Following that we have set of L stacks of blocks g (z) = (C)))) where C, is 3 x 3 convolutional layers with H channels, stride and padding 1, B, is a batch normalization layer, and a is the ReLU activation. The last layer is linear. When computing the effective depth, the i" intermediate layer refers to the output of the i'th block of g'.
The second architecture is an MLP, denoted by MLP-L-H consisting of L hidden layers, where each layer g (a) = o (x))) contains a linear layer T; of output width H, followed by batch normalization B, and ReLU activation σ.. The last layer is linear.
The third architecture is a convolutional residual network denoted by CONVRES-L-HL The network starts with stack of a 2 2 convolutional layer with stride 2, batch normalization, convolution of the same structure, batch normalization, and ReLU. Following that we have a set of L residual blocks, where each block computes g²(x)) = ox + BC (C,
where each C² is 3 x 3 convolutional layer with H channels, stride 1 and padding 1, Bd is a batch normalization layer and a is the ReLU activation. The last layer is linear.
A.2 ESTIMATING THE GENERALIZATION BOUND
In this section we describe how we empirically estimate the bound in Prop. Estimating the bound. We would like to estimate the first term in the bound,
P S. 8 B.,1,,
&f (9, U U.)
(4)
According to Prop. 2 in order to estimate this term we need to generate i.idd triplets (S), Stj, Yj). Since have limixted access training data, we variation of cross-validation and generate k1 = 5 i.i.d. disjoint splits (S). Sb) of the training data S. For each one of these pairs, we generate k2 = 3 corrupted labelings Y We denote by S the set obtained by replacing the labels of St with y. and 5% := S U S.y.
As a first step, we would like to estimate E, 14, (b34)] for each E [kı]. For this purpose, we randomly select Ti = different initializations 71...., 7T, and for each one, train the model 2t using the training protocol described in Sec. 4.1. Once trained, we compute d's, (h for each
t [Ti] (see Def. 1) and approximate E, sy (bggg) )1 using d; := +, E dz; (1
As next step, would like evaluate T[d; 2 din (G ). We notice that d; 2 4im ( U $:) if and only if there is a d,-layered neural network f get ....o o for which err (h) S c, where i((z)) == arg mjcy [l (Sa))- In general, computing this Boolean value is computationally hard. Therefore, to estimate this Boolean value, we simply train a (d; + 11-layered network h = e f and check whether its penultimate layer is c-NCC separable, i.e., err (h) S
where h(x) = arg min llf()) (Sa)l- If SGD implicitly optimizes neural networks to maximize NCC separability as observed in (Papyan et al., 2020) (and also in this paper), we should expect to obtain e-NCC separability in the penultimate layer if that is possible with a dlyleyered network. Since training might be non-optimal, to obtain a robust estimation, we train T3 5 models h4 = e, o ft of depth d, + 1 and pick the one with the best NCC separability in its penultimate layer. Namely, we replace (9, (9.5)) 5 5dj) with min d (he) and estimate I[d, 2 J (9.5g) using
Iid, 2 min 2z (h1)).
14

---

Under review as a conference paper at ICLR 2023
Our final estimation is the following
d
sg (h+) rE[Ta]
min
S
P
Bylds, (h3,)) L (G.S
(5)
In order estimate the bound we assume that 8 and 63 m.pp. negligible constants and that 1. The estimation of the bound is given by the sum the left hand side in equation55am p.
Estimating the mean test error. To estimate the mean test error, Es ( }}, typi- cally done in machine learning, we replace the population distribution P with the test set Stest and we replace the expectation over S1 and y with averages across the k 5 random selec- tions of (Si) k1 and 7i random selections of (x0)21- Namely, we compute the following ii
DhI 1 Ei err (b)g) ~ (erpnerrpp())))) kj
Ti
A.3 NEURAL COLLAPSE
To obtain comprehensive analysis of collapse across layers, we also estimate the degree of NC1.
To evaluate NC1, we follow the process suggested by Galanti et al. (2022a). which is a simplified version of the original approach of Papyan et al. 2020)) For a feature map f R R and two (class-conditional) distributions Q1, Q2 over X' variance (CDNV) to be
C R we define their class-distance normalized
V2(1) = Var (Q1) Var (Q2)
21]py(Q1) - (Q2)
where pa un( == Ez- and by Var,(Q) == (x-o(ll((z)) Mu (Q) the mean and variance of u(x)) for Q. Essentially, this quantity measures what extent the feature vectors of samples from Q1 and Q2 are separated and clustered in space.
To demonstrate the gradual evolution of collapse across the layers, for each sub-architecture = .... o (z) we consider the train and test class features variations Avgezer [Vr: (Se, Se)) and Avg.ge [V (Po, P. The population distribution of each class, Po, is replaced with the test samples of that class.
As shown by Galanti et al. 2022a). this definition is essentially the same as that of Papyan et al. (2020). Furthermore, they showed that the NCC classification error rate can be upper bounded in terms of the CDNV. However, the NCC еттог can be zero in cases where the CDNV is larger than zero. For example, if the two classes are uniformly distributed over the 1-radius circles around the points (-1, 0) and (1,0) 0) in R.? then they are perfectly NCC separable while the CDNV between the two distributions is 0.25.
Auxiliary experiments on the effective depth. In Figs. 713 we plot the CDNV and the NCC accuracy rates of neural networks with varying numbers of hidden layers evaluated on the train and test data. Each curve stands for different layer within the network. As can be seen, in all cases, for networks deeper than a threshold we obtain (near perfect) NCC separability in all of the top layers. Furthermore, the degree of neural collapse seems improve with the network's depth.
Auxiliary experiments with noisy labels. In Figs. 14 I8 plot the results of the same experiment, with different networks and datasets (see captions). As be seen, the effective NCC depth of a neural network tends to increase as we train with increasing amounts of corrupted labels.
we repeat the experiment in Fig. 3 and
'The definition he extended finite St, Sq X by defining V7(8,, (,(U(S))
15

---

Under review as a conference paper at ICLR 2023
10
NCC
6.6
6--
Aucuracy
3.6
3.0
100
300
200
430
200
500
Epoch
Epoch
Spoch
buyers
layers
layers
tCC
300
100
320
400
500
100
300
305
430
500
100
200
303
402
300
Eporh
Epoch
Eporh
16
1.0
1.0
NCC test
Accurngy
0.66
Accurecy
0.6
Accuracy
3.6
02
0.
02-
layers
layens
layers
1.0
NCC 0.-
200
430
560
Epoch
Epoch
Epoch
12 Tayers
16 Tayers
20 layers
Figure 4: Intermediate neural collapse of CONVRES-L-500 trained on CIFAR1O. We plot the NCC train and test accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
16

---

Under review as a conference paper at ICLR 2023
CDNV Train
1900
1000
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC train accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
CDNV Test
1pon
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC test accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
Figure 5: Intermediate neural collapse of CONVRES-L-500 trained on CIEARIO. See Fig. the main text for details.
17

---

Under review as a conference paper at ICLR 2023
NCC
6.6
6--
Aucuraay
3.6
0.98
100
300
693
200
430
200
500
Epoch
Epoch
Spoch
layers
layers
laymrs
tCC
300
100
320
400
500
100
300
305
430
500
100
200
302
402
300
Eporh
Epoch
Eporc
16
1.0
1.0
NCC test
Accurnty
0.6
Accurecyy
0.6
Accuracy
3.6
02
0.
0.-
Epoch
layers
layens
layers
1.0
NCC 0.6
200
430
560
Epoch
Epoch
Epoch
12 Tayers
16 Teyers
20 layers
Figure 6: Intermediate neural collapse of CONV-L-400 trained on CIFAR1O. We plot the NCC train and test accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
18

---

Under review as a conference paper at ICLR 2023
CDNV Train
1900
1000
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC train accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
CDNV Test
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC test accuracy
244
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
Figure 7: Intermediate neural collapse of CONV-L-400 trained on CIFARIO. See Fig. main text for details.
in the
19

---

Under review as a conference paper at ICLR 2023
10-
NCC
6.66
acturayy
6--
Aucurayy
3.6
0.98
3.00
100
300
693
200
430
200
500
Epoch
Epoch
Epoch
luyers
layers
layers
tCC
300
100
320
400
500
100
300
305
430
500
100
200
302
402
300
Eporh
Epoch
Eporh
16
1.0
NCC test
Accurnty
0.66
Accurecy
0.
Accurayy
3.6
02
02
02-
Epoch
layers
layens
layers
1.0
0.-
NCC
200
430
560
Epoch
Epoch
12 Tayers
16 leyers
20 layers
Figure 8: Intermediate neural collapse of MLP-L-300 trained on CIFAR1O. We plot the NCC train and test accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
20

---

Under review as a conference paper at ICLR 2023
CDNV Train
1po0
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC train accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
CDNV Test
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC test accuracy
20114
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
Figure 9: Intermediate neural collapse of MLP-L-300 trained on CIFAR10. See Fig. main text for details.
in the
21

---

Under review as a conference paper at ICLR 2023
NCC
6.6
aaturecy
6--
Aucuray
3.6
0.98
300
100
300
200
430
250
500
Epoch
Epoch
Epoch
luyers
layerss
layers
tCC
300
100
320
400
500
100
200
305
430
500
100
200
302
402
300
Eporh
Epoch
Eporh
16
1.0
NCC test
Accurnty
0.66
Accurecy
6.
Accuracy
3.6
02
02
32-
Epoch
Epoch
Spoch
layers
loyers
layers
NCC 0.0
1ear
100
200
430
560
Epoch
12 Tayers
16 Teyers
20 layers
Figure 10: Intermediate neural collapse of CONV-L-50 trained on MNIST. We plot the NCC train and test accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
22

---

Under review as a conference paper at ICLR 2023
CDNV Train
0.
1900
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC train accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
CDNV Test
3--
Epon
3 layers
4 layers
layers
8 layers
10 layers
layers
14 layers
layers
18 layers
layers
NCC test accuracy
layers
4 layers
layers
8 layers
10 layers
12 layers
14 layers
16 layers
18 layers
20 layers
Figure II: Intermediate neural collapse of CONV-L-50 trained on MNIST, See Fig. main text for details.
in the
23

---

Under review as a conference paper at ICLR 2023
NCC
6.6
Arturayy
6-6
Aucurayy
3.6
0.98
106
100
300
693
200
430
250
500
Epoch
Epoch
Epoch
luyers
layers
layers
tCC
1497
300
300
320
400
500
100
300
305
430
500
100
200
302
402
300
Eporh
Epoch
Eporh
16
1.0
NCC test
Accurnty
0.66
Accurecy
0.6
Accuracy
3.6
02
02
32-
Epoch
layers
layens
layers
1.0
0.0
NCC
Year
200
430
560
Epoch
Epoch
12 Tayers
16 leyers
18 layers
Figure 12:
Intermediate neural collapse of MLP-L-100 trained on Fashion MNIST. We plot the NCC train and test accuracy rates of neural networks with varying numbers of hidden layers. Each curve stands for different layer within the network.
24

---

Under review as a conference paper at ICLR 2023
CDNV Train
1900
2 hidden layers
hidden layers
4 hidden layers
6 hidden layers
8 hidden layers
10 hidden layers
hidden layers
hidden layers
16 hidden layers
hidden layers
NCC train accuracy
2 hidden layers
hidden layers
4 hidden layers
6 hidden layers
hidden layers
10 hidden layers
12 hidden layers
14 hidden layers
16 hidden layers
18 hidden layers
CDNV Test
2 hidden layers
hidden layors
4 hidden layers
6 hidden layers
8 hidden layers
10 hidden layers
hidden layers
hidden layers
16 hidden layers
hidden layers
NCC test accuracy
2 hidden layers
hidden layers
4 hidden layers
6 hidden layers
hidden layers
10 hidden layers
12 hidden layers
14 hidden layers
16 hidden layers
18 hidden layers
Figure 13: Intermediate neural collapse of MLP-L-100 trained on Fashion MNIST. See Fig. in the main text for details.
25

---

Under review as a conference paper at ICLR 2023
CUNV -rain
004
Iayer
Tayer
layer
Tayer
layer
NCC train ac0
0% moise
10% moise
25%
50%
75% moise
CLNV- Test
Ingerr 3
layer 4
Tayer
Tayer 8
layer 10
NCC
25%
Figure 14: Intermediate neural collapse of CONV-10-400 trained on CIFAR1O with partially corrupted labels. In the first (third) row, we plot the CDNV on the train (test) data for intermediate layers of networks trained with varying amounts of corrupted labels (see legend). In the second (fourth) row, we plot the NCC accuracy rates of the various layers of a network trained with certain amount of corrupted labels (see titles).
CDNY Train
layer
layer
layer 6
layer
layer 10
NCC 500
0% noise
10% noise
25% noise
50% noise
75% % noise
CDNV Tost
layer 3
layer 4
layer 6
layer
layer 10
NCC NCC.tes tes soc
0% noise
10% noise
25% noise
50% noise
75% noise
Figure 15: Intermediate neural collapse of CONVRES-10-500 trained CIFAR10 with noisy
labels. See Fig 3 in the main text for details.
26

---

Under review as a conference paper at ICLR 2023
CDNY Irain
layer 3
layer 4
layer 6
layer
layer 10
NCC NCCssanac
Eran
0% noise
10% noise
25 noise
50% noise
75% noise
CDNV Ien
layer 3
layer
layer 6
layer
layer 10
NCC test
5.
0% noise
10% noise
25% noise
50% noise
75% noise
Figure 16: Intermediate neural collapse of MLP-10-500 trained on CIFARIO with noisy labels.
See Fig in the main text for details.
CDNY Irain
Eoch
Lacc layer 8
Gxc layer 10
layer 4
layer
layer
NCC tran ucc T
0% noise
5% noise
10% noise
20% noise
30% noise
CDNV Test
Sare layer
layer 4
layer 8
layer
layer 10
2 5.1
NOC pos
0% noise
5% noise
10% noise
20% noise
30% noise
Figure 17: Intermediate neural collapse of CONV-10-100 trained on Fashion MNIST with noisy labels. See Fig. in the main text for details.
27

---

Under review as a conference paper at ICLR 2023
CDNV Train
layer
layer
layer
NCC NCC aoc
0% moise
1% moise
2% noise
5% moioe
10% noise
CDNY Test
layer
layer 4
layer
layer
Layer 10
NCC ao0
0% aolse
1% noise
2% noise
0% dolse
10% noise
Figure 18: Intermediate neural collapse of CONV-10-50 trained MNIST with partially rupted labels. See Fig. 14 for details.
28

---

Under review as a conference paper at ICLR 2023
B PROOFS
Proposition 1. Let m E N. (0,1/2), E (0, and (0. 1). Assume that the evror of the learning algorithm is on -n-ifoffrm. Assume that S2 ~ Pn(m). Let his, be the output of the learning algorithm given access to dataset Si and initialization 7, Then,
E, (1 (k3,) $ S E. 1, (hg, 2 iiin (9, (1 U
(3)
+ + + 6n + +8t pmpa
where Y2 [Y:]= is uniformly selected to be a set of labels that disagrees with Y2 on pm values.
Proof. Let S1 = {(r) u))} and S2 ( =m be two balanced datasets. Let e 0, 0.p.0 and q = (1+aa p. Let Y-2 and Y/ be a uniformly selected set of labels that disagree with Y/ on pm and qm randomly selected labels (resp.). We denote by S2 and S2 the relabeling of S2 with the labels in Y/2 and in 82 (resp.). We define four different events.
A {(S1, S2, Y2) 3 2 (1 a) p: (9, U U$2) > Ey, 1d (9, S1 U S2]1}
A = [(S), S2) the mistakes of li3, are not uniform over S2}
A3 S2, Y2) (S), S2 Y2) & A U A2 and B, (h3,, ) d min (9, S1 52) (6)
A4 = {(S), S2, Y2) (Si, S2, Y2) $ A U A2 and E, (12,)] 2 ( (9, S1 U 5,)
B1 [(S), S2, Y2)
I,', (h2,))
min (9, S1 12))
By the law of total expectation
Es, lerr ( = ss By (h2))
Σ P[A:] R [Ry (h2, )] A4.]
P[A] - P[A2] S. E. ferrs, (h3, )] As] + IP[
where the last inequality follows from errs, (13,) S 1, P[A,] < and A4 B1.
We would like to upper bound each one of the above terms. First, we notice that since the mistakes of the network are 8,, -uniform, P[A2] I 6t.- In addition, by definition P[A,] < 82 m.p.a.
As a next step, we upper bound s. [ (13,)) As]. Assume that S2, V2) A3- Hence, (S1, Y2) & A U A2. Then, the mistakes of h2g over S2 uniformly distributed (with respect to the selection of 7). Assume by contradiction that errs, (h3, ) > (1 + a) p with non- zero probability over the selection of y. Then, since the mistakes of h3, over Sy are uniformly distributed, errs, (h3,) ) > (1 + a) p for all initializations y. Therefore, we have
Ey, d (9, Si U S2)] < E, (42,) <
U.²),
where the first inequality follows from the definition of d (G, S₁ U 32) and the second one by the assumption that (S,, Y2) Ag. However, this inequality contradicts the fact that S2, Y2) & A1. Therefore, we conclude that in this case, lerss (h2,) ) S (1 + a) p and Es..,,,, lerrs, (h2,)) ) As < (1+a)p.
Let m N, pE (0,1/2), a (0, and (0, Assume that the error of the learning algorithm is 8,, -uniform. Let S1, S2, St. ~ PB(m) (for i [k]). Let Y (m be a set of labels that disagrees with Yū on uniformly selected pm labels and St is a relabeling of S2 with the labels in Yz. Let h3, be the output of the learning algorithu given access to a dataset S and initialization Then, with probability at least 1 8 over the selection of (S)) Sh V
Proposition
29

---

Under review as a conference paper at ICLR 2023
have
Es, (erppe/erhhhhhge
k Σ By 5: (3)))1 2 (9, Si
1
log(2/6)
(1 + a:) p 8m + 6 m.p.a
2h:
Proof. By Prop.
D have
Es, By (k3,))
S2.8 Eylds, (13,)) )1 2 Liin (9, (1 + ()) Pm 8 + 52 mpp
S1 U22)
We define i.i.d. random variables
V = Bylds (13g)) 2 d (9, S 8;}
(8)
Therefore, we can rewrite,
P
S Ey (hz,))
Lmi ( E[]
By Hoeffding's inequality,
I
V
[[]
2 c S
(10)
By choosing
Vlog(1 /lgg(/22)//K/ obtain that with probability at least 1 - ô, we have
V S
Σ V. + (10g(128)//2
(11)
When combined with Prop.
we obtain the desired bound.
30


Reviews:
[{"review_id": "r6IzIObmg9", "reviewer": "Reviewer_nZgi", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new generalization bound for deep neural networks based on the effective depth. The idea is motivated by the empirical observation that neural networks’ representations exhibit neural collapsing. The authors define the notions of effective depth and minimal depth of a neural network, and establish a generalization bound based on the comparison between them. The paper also provides empirical estimation of the bound which is shown to be tighter than previous bounds.", "summary_of_the_review": "I think this paper provides a novel perspective of generalization bounds from the angle of neural collapsing. Although the theory is not complicated, it's idea is novel as far as I can tell, and could be of interest to many theoreticians in the ML community. Thus I would be happy to see this work getting accepted.", "strength_and_weaknesses": "Strength:\n- The idea of establishing a generalization bound based on the neural collapsing phenomenon is novel and interesting.\n- The experiments are comprehensive and details are provided.\n\nWeakness:\n- It's unclear whether some of the technical settings are necessary. For example, I don't see how the specific structures of Conv-L-H and MLP-L-H come into play in the theory.\n- I somehow feel the first term in the bound could be pretty large, hence making the bound vacuous. The authors provide some experiments to justify that the term is small, however, the way they compute d_min is by relying on the implicit bias of SGD, which is really not reliable since SGD may not be able to find really the best shallow network that fits the data. I wonder we can find a smaller d_min with the following algorithm: given some target depth d, we train a (d-1)-layer NN classifier to achieve less than eps error, then we embed the last linear layer into an additional layer by add a bunch of 0 weights. Would this lead to a d-layer NN with epsilon error? If so I feel this might give us a smaller depth than train d+1 layer NN and then chop off the last linear layer.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "adBlg-VGke", "reviewer": "Reviewer_JHSQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for evaluation the generalising capabilities of neural networks through neural collapse", "summary_of_the_review": "I have mixed feelings. I am up for new ideas, but the paper does not do a good job on:\n\n 1. Showing that is better measure than other measures. Instead of repeating over and over that \"classical measures\" provide \"vacuous bounds\", how about using that space for and in-depth analysis and comparison with each one of those measures?. and \n\n2. I do not think authors spent the time necessary on the figures so that the reader would not need a microscope to see what is plotted there.", "strength_and_weaknesses": "Strengths:\n- I kind of like the idea of evaluating neural collapse for measuring generalisation.\n- Mathematical analysis seems interesting (although I did not check it in detail).\n\nWeakneses:\n- I am not convinced that this is better measure than what is already present as there is real no proof of that. In the introduction and other parts it states \"In many practical settings [...] making the bounds vacuous\". This is repeated several times, but I failed to see an in-depth analysis of those \"practical settings\" and what are those \"vacuous bounds\". One finds something at the end of section 3 but is the same repetition about vacuous bounds and sqrt(m), and not an in-depth analysis in how it compares to the other \"classical measures\". It all becomes very vague and repetitive when addressing the advantages of the presented measure when compared to those \"classical\" ones.\n- All plots are awful as they are barely visible. Labels must be font size 2.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PW3ELKElX16", "reviewer": "Reviewer_VKrE", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to understand the generalization of deep neural networks. It characterizes the effective depth as the first layer for which sample embeddings are separable using the nearest-class center classifier. Then a generalization bound based on the effective depth is suggested as an indicator of generalization performance. Some experiments are conducted to validate the effectiveness of this bound in estimating the generalization. \n\n\n", "summary_of_the_review": "This work points to a new angle of understanding the generalization performance -- how neural collapse in the middle layer affects generalization. The neural collapse in the middle layer is further quantified by the \"effective depth\". The only concern that comes from the reviewer is the gap between theory and practice. It would be nice to see some experiential results on more \"advanced\" neural networks. ", "strength_and_weaknesses": "Strength:\n- Several insightful phenomena are observed (or validated): the network collapses until very late. The collapse in the middle layers matters for generalizations. \n- A theoretical generalization bound in effective depth is proposed, the bound can serve as a good indicator for MLPs and ConvNets.\n\nWeakness:\n- The experiments are only done for ConvNets and MLPs, how does this estimation work for more practically used architectures such as ResNets or Transformers? \n- The plots are very difficult to read because they are way too small.\n- The author did not provide enough support on why the current bound is better than what have defined before. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject