Towards Understanding the Condensation of Neural Networks at Initial Training
Authors: Hanxu Zhou
Qixuan Zhou
Tao Luo
Yaoyu Zhang
Zhi-Qin John Xu
Venue: NeurIPS 2022 Accept (2022)
Keywords: neural networks
training
condensation dynamics
implicit regularization

Abstract: Empirical works show that for ReLU neural networks (NNs) with small initialization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) condense onto isolated orientations. The condensation dynamics implies that the training implicitly regularizes a NN towards one with much smaller effective size. In this work, we illustrate the formation of the condensation in multi-layer fully connected NNs and show that the maximal number of condensed orientations in the initial training stage is twice the multiplicity of the activation function, where ``multiplicity'' indicates the multiple roots of activation function at origin. Our theoretical analysis confirms experiments for two cases, one is for the activation function of multiplicity one with arbitrary dimension input, which contains many common activation functions, and the other is for the layer with one-dimensional input and arbitrary multiplicity. This work makes a step towards understanding how small initialization leads NNs to condensation at the initial training stage.

Towards Understanding the Condensation of Neural Networks at Initial Training
Hanxu Zhou', Qixuan Zhou', Tao Luo! Yaoyu Zhang 3 Zhi-Qiin John Xul; School of Mathematical Sciences, Institute of Natural Sciences, MOE-LSC and Qing Yuan Research Institute, Shanghai Jiao Tong University Saaa-hhangaai, Shanghai Artificial Intelligence Laboratory 3 Shanghai Center for Brain Science and Brain-Inspired Technology
Abstract
Empirical works show that for ReLU neural networks (NNs) with small initial- ization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) condense onto isolated orientations. The condensation dynamics implies that the training implicitly regularizes a NN towards one with much smaller effective size. In this work, we illustrate the formation of the condensation in multi-layer fully connected NNs and show that the maximal number of condensed orientations the initial training stage is twice the multiplicity of the activation function, where "multiplicity" indicates the multiple roots of activation function at origin. Our theoretical analysis confirms experiments for two cases, one is for the activation function of multiplicity one with arbitrary dimension input, which contains many common activation functions, and the other is for the layer with one-dimensional input and arbitrary multiplicity. This work makes a step towards understanding how small initialization leads NNs condensation the initial training stage.
1 Introduction
The question why over-parameterizee neural networks (NNs) often show good generalization attracts much attention (Breiman [995] Zhang et al. 2021). Luo et al. (202T) found that when initialization is small, the input weights of hidden neurons in two-layer ReLU NNs (the input weight or the feature of hidden neuron consists of the weight from input layer to the hidden neuron and its bias term) condense onto isolated orientations during the training. As illustrated in the cartoon example in Fig. I the condensation transforms a large network, which is often over- over- parameterized, to the one of only few effective neurons, leading to output function with low complexity. Since, in most cases, the complexity bounds the generalization error (Bartlett and Mendelson, 2002), the study of condensation could provide insight to how over- paraamttteee NNs are implicitly regularized to achieve good generalization performance in practice.
Small initialization leads NNs rich non-linearity during the training (Mei et al. 2019 Rotskoff and Vanden-Eijnden 2018 Chizat and Bach 2018 Sirignano and Spiliopoulos 2020). For example, in over-parameterized regime, small initialization can achieve low generalization error (Advnni et al. 2020 Irrespective of network width, small initialization can make two-layer ReLU NNs converge to solution with maximum margin (Phuong and Lampert 2020). Small initialization also enables neural networks to learn features actively (Lyu et al., [021] Luo et al. (202)). ). The gradual increment of the condensed orientations is consistent with many previous works, which show that the network
"Corresponding author: eeyy.sjuu@sjt..ed "Corresponding author: xuzhiqin @ sjtt.edu.ccn
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

output evolves from simple to complex during the training process (Xu et al. 2020; Rahaman et al. 2019; Arpit et al. 2017). The initial condensation resets the network simple state, which brings out the simple to complex training process. Condensation is an important phenomenon that reflects the feature learning process. Therefore, it is important to understand how condensation emerges during the training with small initialization.
Illustration of Condensation
Initialization
After training
Effective network
Figure I: Illustration of condensation. The color and its intensity of a line indicate the strength of the weight. Initially, weights are random. Soon after training, the weights from an input node to all hidden neurons are the same, i.e., condensation. Multiple hidden neurons can be replaced by an effective neuron with low complexity, which has the same input weight as original hidden neurons and the output weight as the summation of all output weights of original hidden neurons.
The dynamic behavior of the training at the initial state is important for the whole training process, because it largely determines the training dynamics of neural network and the region it ends up in (Fort et al. 2020 Hu et al. 2020). which impacts the characteristics of the neural networks in the final stage of training (Luo et al. 2021 Jiang et al. 2019 Li et al. 2018). For two-layer ReLU NNs, several works have studied the mechanism underlying the condensation at the initial training stage when the initialization of parameters goes to zero (Maennel et al. 2018 Pellegrini and Biroli 2020). However, still remains unclear that for NNs of more general activation functions, how the condensation emerges at the initial training stage.
In this work, we show that the condensation at the initial stage closely related to the multiplicity at = 0, which means the derivative of activation at æ 0 is zero up to the (p 1odh--rrde and is non-zero for the p-th order. Many common activation functions, e.g., tanh (2). s1)moid(z). softplus(x)) x), Gelu(z), Swich(z), etc, are all multiplicity p 1. and tanh (z) and x tanh(2) have multiplicity two and three, respectively. Our contribution is summarized as follows:
Our extensive experiments suggest that the maximal number of condensed orientations in the initial training is twice the multiplicity the activation function used in general NNs.
We present theory for the initial condensation with small initialization for two cases, one is for the activation function of multiplicity one with arbitrary dimension input, and the other is for the layer with one-dimensional input and arbitrary multiplicity. As many common activation functions are multiplicity 1, our theory would be of interest to general readers.
Related works
Luo et al. 2021 systematically study the effect of initialization for two-layer ReLU NN with infinite width by establishing a phase diagram, which shows three distinct regimes, i.e., linear regime (similar
to the lazy regime) (Jacot et al. 2018 Arora et al. 2019 Zhang et al. 2020 E. et al. 2020 Chizat and Bach 2019), critical regime (Mei et al. 2019
Rotskoff and Vanden Mijden-Fijnden [2018] Chizat and
Bach 2018 Sirignano and Spiliopoulos 2020 and condensed regime (non-liness regime), based on

---

the relative change of input weights as the width approaches infinity, which tends to 0, O(1) and +oc, respectively. Zhhu et al. (2022) make a step towards drawing phase diagram for three-layer ReLU NNs with infinite width, revealing the possibility of completely different dynamics (linear, critical and condensed dynamics) emerging within a deep NN for its different layers. As shown in Luo et al. (202T), two-layer ReLU NNs with infinite width do not condense in the neural tangent kernel (NTK) regime, slightly condense in the mean-field regime, and clearly condense in the non-linear regime. However, in Luo al. (2021), it is not clear how general the condensation phenomenon is when other activation functions are used and why there is condensation.
Zhang et al. (202 b); Fukumizu and Amari 2200); Fukumizu et al. (2019); Simsek et al. (2021) propose a general Embedding Principle of loss landscape of DNNs that unravels a hierarchical structure of the loss landscape of NNs, i.e., loss landscape of an DNN contains all critical points of all the narrower DNNs. The embedding principle shows that a large DNN can experience critical points where the DNN condenses and its output is the same as that of a much smaller DNN. However, the embedding principle does not explain how the training can take the DNN to such critical points.
The condensation is consistent with previous works that suggest that NNs may learn data from simple complex patterns (Arpit et al. 2017: Xu et al.) 2019 Rahaman al. 2019; Xu al. 2020 Jin et al.
2020, Kalimeris et al. 2019). For example, an implicit bias of frequency principle is widely observed
that NNs often learn the target function from low to high frequency (Xu et al. 2019 Rahaman et
2019, Xu et al. 2020), which has been utilized to understand various phenomena (Ma et al., 2020
Xu and Zhou| 2021) and inspired algorithm design (Liu et al. 2020 Cai et al. 2020) Tancik et al.
2020
Li et al. 2020 2021).
3
Preliminary: Neural networks and initial stage
two-layer NN is
fo(x) = _ (w x):
(I)
j=1
where (-) is the activation function, Wj (00 bj) Rd+1 is the neuron feature including the input weight and bias terms, and æ (æ, 1) d is the concatenation of the input sample and scalar 1, 9 is the set of all parameters, i.e., {aj, wj y For simplicity, we call wy input weight or weight and x input sample.
L-layer NN can be recursively defined by setting the output of the previous layer as the input to the current hidden layer, i.e..
x 00| (x, 1),
a! (a(Wl")) 20) 1), alll = (a (o(WW))) 1), for E (,,3, L} f(0, x) = a x1l & fe(x),
(2)
where WI4 (WIl, b [m)) and my represents the dimension of the I-th hidden layer. For simplicity, we also call each row of Will input weight weight and !!!--1) input the 1-th hidden layer. The target function is denoted as f*(x). The training loss function is the mean squared error
1 1
n
R(0)=
2n Σ (fe (xx)))
(3)
Without loss of generality, assume that the output is miedimmesionnl for theoretical analysis, because, for high- high-dimensiooal cases, we only need to sum up the components directly. For summation, it does not affect the results of our theories. We consider the gradient flow training
θ = -
(4)
To ensure that the training is close to the gradient flow, all the learning rates used in this paper are sufficiently small. We characterize the activation function by the following definition.
Definition (multiplicity p). Suppose that axx) satisfies the following condition, there exists a
p E N such that the s-th order derivative (e) (0) = for = 1,2, ... :P - 1, and o (0) # O. then we say has multiplicity p.
3

---

Remark 3.1. Here are some examples. tanh(z), sigmoid and softplus(2) have multiplicity 1. x ta)h(2)) has multiplicity p
We illustrate small initialization and initial stage as follows
Small initialization: W ~ (1)) and Wilk!-- o(1) for all I's and a!!-1),
We want to remark that it dose not make sense to define the initial stage by the number of training steps, because the training is affected by many factors, such as the learning rate. Therefore, in our experiments, the epochs we use show phenomena can cover wide range. Alternativel,, we consider the initial stage as follows.
Initial stage: the period when the leading-orde order Taylor expansion w.r. activated neurons is still valid for theoretical analysis of Sec. 5)
As the parameters of neural network evolve, the loss value will also decay accordingly, which is easy to be directly observed. Therefore, we propose an intuitive defimition of the initial stage of training by the size of loss in this article, that is the stage before the value of loss function decays to 70% of its initial value. Such a definition is reasonable, for generally a loss could decay to 1% of its initial value or even lower. The loss of the all experiments in the article can be found in Appendix
A.3 and they do meet the definition of the initial stage here.
Cosine similarity: The cosine similarity between two vectors u and is defined as
TTv
D(u, v) (uTu))(((TT)))11
(5)
4 Initial condensation of input weights
In this section, would empirically show how the condensation differs among NNs with activation function of different multiplicities in the order a practical example, multidimensional synthetic data, and 1-d input synthetic data, followed by theoretical analysis in the next section.
4.1 Experimental setup
For synthetic dataset: Throughout this work, we use fully-connected neural network with size, d- m- -m--out- The input dimension d is determined by the training data. The output dimension is dout = 1. The number of hidden neurons m is specified in each experiment. All parameters are initialized by Gaussian distribution N (O, var). The total data size is n. The training method is Adam with full batch, learning rate Ir and MSE loss. We sample the training data uniformly from sub-domain of R.d. The sampling range and the target function are chosen randomly to show the generality of our experiments.
For CIFARIO dataset: We use Resnet like neural network, which has been described in Fig. 2 and the input dimension is d = 32 x 32 x 3. The output dimension dout = 10. All parameters initialized by Gaussian distribution (O, var). The total data size is The training method is Adam with batch size 128, learning rate Ir and cross-entropy loss.
4.2 A practical example
The condensation of the weights of between the fully-connected (FC) layers of Resnet 8-like neural network on CIFAR10 10 is shown in Fig. 2 whose activation functions for FC layers are tanh(z), (1) softplus(x) and x tanh(z). indicated by the corresponding sub-captions. respectively. As shown in Fig. 2(a), for activation function tanh(r), the color indicates cosine similarity D(u, v) of two hidden neurons' weights, whose indexes are indicated by the abscissa and the ordinate, respectively. If the neurons are in the same beige block, D(u, v) ~ 1 (navy-blue block, D(u, v) -)), their input weights have the same (opposite) direction. Input weights of hidden neurons in Fig. a condense at two opposite directions, i.e., one line. Similarly, weights of hidden neurons for NNs with (1r and softplus(x) (Fig. b.. c)), which are frequently used and have multiplicity one, condense at one direction. As the multplicity increases, NNs with tanh x (Fig. d)) condense at two different lines. These experiments suggest that the condensation is closely related to the multiplicity of the activation function.

---

In these experiments, we find that the performance of the Resnet18-1e 8ike network with tanh FC layers with small initialization is similar the one with common initialization in Appendix A.4
D011.1
PlA
DlA
1023
50
(a) tanh(2)
(b) ())
(c) softplus(z)
(d) tanh(3)
Figure 2: Condensation of Resnet fesnet18-like neural networks CIFARIO. Each network consists of the convolution part of resnet18 8 and iully-connected (FC) layers with size 1024-10244 and softmax. The color in figures indicates the cosine similarity of normalized imput weights of two neurons in the first FC layer, whose indexes are indicated by the abscissa and the ordinate, respectively. The convolution part is equipped with ReLU activation and initialized by Glorot normal distribution (Glorot and Bengio 2010). The activation functions are tanh(z), sigmoid(x)) softplus(1) and ttanh(x) for FC layers in (a), (b), (c), and (d), separately. The numbers of steps selected in the sub-pictures are epoch 20, epoch 30, epoch 30 and epoch 61, respectively. The learning rate is 3 x 10 1 x 10
10- and 5 10 separately The FC layers are initialized by N(0, m (.)
and Adam optimizer with cross-entropy loss and batch size 128 are used for all experiments.
4.3 Multidimensional synthetic data
(a) tanh(w) (x)
(b) æ tanh(x)
(c) x² tann(())
(d) sigmoid(x)
(e) softplus(x))
Figure 3: Condensation two-layer NNs. The color indicates D(u, v) of two hidden neurons' input weights epoch 100, whose indexes indicated by the abscissa and the ordinate, respectively. neurons are in the same beige block, D(u, v) (navy-blue block, D(u, v) ~ -1), their input weights have the same (opposite) direction. The activation functions indicated by the sub-captions. The training data is 80 points sampled from 3 3n + 1), where each de is uniformly sampled from [-4,2] 2).
80, d 5, 50, dout var 0.005?. 1 10 3,8 10 10-4,22 4,25 25 x 10-4 for (a-c), (d) and (e), respectively.
For convenience experiments, we then use synthetic data to perform extensive experiments to study the relation between the condensation and the multiplicity of the activation function.
We use two-layer fully-connected NNs with size 5-50-1 to 5-dimensional function Σ 3.5 (5zz + 1), where = (ar 22, Z2, ... ,25) R and each as is uniformly sampled from 2). As shown in Fig. aa. for activation function tanh(r)) input weights of hidden neurons condense at two opposite directions, i... one line. As the multiplicity increases, NNs with tanh(x) (Fig. 3(b) and x² tanh (Fig. 3(c) condense at two and three different lines, respectively. For activation function sigmoid( in Fig. d( and softplus(z) in Fig. 3(e), NNs also condense at two opposite directions.
training data sampled from
For lulti-layer NNs with different activation functions, we show that the condensation for all hidden layers is similar to the two-layer NNs. In deep networks, residual connection is often introduced to overcome the vanishing gradient. To show the generality of condensation, we perform an experiment of six-layer NNs with residual connections. To show the difference of various activation functions, we set the activation functions for hidden layer hidden layer x2 tanh(x)) tanh(x), sigmoid(x)) tanh(æ) and softplus(z)) respectively. The structure of the residual is
5

---

(a) layer
(b) layer 2
(c) layer 3
(d) layer 4
(c) layer 5
Figure 4: Condensation of six-layer NNs with residual connections. The activation functions for hidden layer to hidden layer are x² tanh(#). æ tanh(x), (x). tanh(z) and softplus(z)) respectively. The numbers of steps selected in the sub-pictures 900, epoch 1400 and epoch 1400, respectively, while the NN is only trained once. The color indicates D(u, v) of two hidden neurons' input weights, whose indexes are indicated by the abscissa and the ordinate, respectively. The training data is 80 points sampled from a 3-dimensional function Σx=1 4sin( 12 + 1), where each Fy. is uniformly sampled from [-4, 2]. 80, = 3, m 18, dout = var = 0.012, Ir 4 10-5
epoch 1000, epoch 900, epoch
h++(()) = o (Wih + bx) + (2)), where (x) the output of the (-th layer. As shown in Fig. input weights condense at three, two, one, one and one lines for hidden layer to hidden layer 5, respectively. Note that residual connections are not necessary. We show an experiment of the same structure in Fig. 4 but without residual connections in Appendix A.5
Through these experiments, we conjecture that the maximal number of condensed orientations at initial training is twice the multiplicity of the activation function used. To understand the mechanism of the initial condensation, we turn experiments of 1-d input and two-layer NNs, which can be clearly visualized in the next subsection.
4.4
1-d input and two-layer NN
For 1-d data, visualize the evolution of the two-layer NN output and each weight, which confirms the connection between the condensation and the multiplicity the activation function.
0.5
y 0.0
y 0.0
0.0
-0.5
-0.5
-0.5
-1.0
-1.0
-1.0
-1.0
-0.5
0.0
005
1.0
1.5 -1.0
-0.5
0.0
x
X
(a) tanh(#)
(b) tanh(x)
x tanh(x)
Figure 5: The outputs of two-layer NNs epoch 1000 with activation function tanh(z), tanh(x). and x tanh(x) are displayed, respectively. The training data is 40 points uniformly sampled from sin (3x) + sin(62) (6z)/2 with x E [-1, 1.5). illustrated by green dots. The blue solid lines are the NN outputs at test points, while the red dashed auxiliary lines are the first, second, third and first order polynomial fittings of the test points for (a, b, c), respectively. Parameters are n 40, 1, m 100, don 1, var 0.005 Ir x 10-1
We display the outputs at initial training in Fig. 5 Due to the small magnitude of parameters, an activation function with multiplicity p can be well approximated by p-th order polynominal, thus, the NN output can also be approximated by p-th order polynominal. As shown in Fig. [ the NN outputs with activation function Lanh(x), x tanh(xr) and x² tanh(z)) overlap well with the auxiliary of linear, quadratic and cubic polynominal curve, respectively in the beginning. This experiment, although simple, but convincingly shows that NN does not always learn linear function the initial training stage and the complexity of such learning depends the activation function.

---

0.15
02
0.30
0.0
0.0
-0.15
-0.2
-0.2
-0.15
0.00
0.15
-0.2
0.0
0.2 -0.2
0.0
0.2
(a) tanh(x)
(b) x tanh(x)
(c) x² tanh(x)
Figure 6: The direction field for imput weight w == (w, b) of the dynamics in 4.4 at epoch 200. All settings the same as Fig. []] Around the original point, the field has one, two, three stables lines, on which an input weight would keep its direction, for tanh(x), x tanh(z), and x tanh(x), respectively. We also display the value of each weight by the green dots and the corresponding directions by the orange arrows.
We visualize the direction field for input weight w, == (wj, bj), following the gradient flow,
aj c,o (wy D.)a.,,
n
where e; : fe(x;) (xx) Since we only care about the direction of w and 0g is scalar at each epoch, visualize úij by wjjaj. For simplicity, we do not distinguish uz./a, and wy if there is no ambiguity. When we compute wi for different j's, e;2; for (i = 1. .2) is independent with j. Then, at each epoch, for set of {e,, D we can consider the following direction field
expio' (w wz))
72
When w is set as Wj, we can obtain wy As shown in Fig, 6 around the original point, the field has one, two, three stables lines, on which neuron would keep its direction, for tanh(r), x tanh(z). and x² fanh(x). respectively. We also display the input weight of each neuron on the field by the green dots and their corresponding velocity directions by the orange arrows. Similarly to the high-dimensional cases, NNs with multiplicity activation functions condense at different lines for p 1, 2,3. Therefore, It is reasonable conjecture that the maximal number of condensed orientations twice the multiplicity of the activation function used.
Taken together, we have empirically shown that the multiplicity of the activation function is key factor that determines the complexity of the initial output and condensation. To facilitate the understanding of the evolution of condensation in the initial stage, we show several steps during the initial stage of each example in Appendix
A A.6
Analysis of the initial condensation of input weights
In this section, would present preliminary analysis to understand how the multiplicity of the activation function affects the initial condensation. At each training step, we consider the velocity field of weights each hidden layer neural networks.
j Considering a network with hidden layers, we use row vector w! to represent the weight from
the (k 1)-th layer to the j-th neuron the k-th layer. Since condensation is always accompany with small initialization, together with the initial stage defined in Sec. 3 we make the following assumptions,
Assumption Small initialization infers that w [ ~ o(1) applies for all k's and j's. Assumption During the initial stage of condensation, Taylor expansion of each corresponding activated neurons holds, i.c.
(7) (7 1)!
o' ( = o'(0) +
W x 11 + !
(6)
7

---

Suppose the activation function has multiplicity p. i.e. (0) = 0 for 1.2, P 1,, and (fr) (0) * 0. Then, together with Assumption we have
( (0) (p-11 1}! (w/ll -
o' d'(W]
(ki xf ~
(7)
For each l: and wi satisfies the following dynamics, (see Appendix A.2
f u w. i ii (ut u)u
r
[R] where w can represent w W for all k's and j's, | is the amplitude, and w/r.
For convenience, define operator P satisfying Pw ui w-) To specify the condensation for theoretical analysis, we make the following definition,
Condensation: the weight evolves towards direction which will not change in the direction field and is defined as follows,
= Pw i u(i u) 0.
Since ib u a scalar, í is parallel with u. u is a unit vector, therefore, we have u = + || Remark (An intuitive explanation for condensation). In this work, we consider NNs with sufficiently || O(e), where is small quantity, then dynamics (8) will show that O(F) ~ (ww) and (()) O(F)/O(e). Here O(+) ~ ((w) refers that the evolution of the same order every component of and is the same for ( O(r)/O(e). Therefore, the orientation u would vary much more quickly than the amplitude r. By the dynamics for w (Eq.
small parameters. Suppose
10 and the Taylor approximation with multiplicity p (Eq.
is easy to find that the solutions for
Eq. aare finite. Then, taken together, the orientation would converge rapidly into certain directions, leading to condensation.
In the following, we study the case of (i) p 1 and (ii) m (the dimension of input of the k-th layer equals one), and reach the following informal proposition,
Proposition 1. Suppose that Assumption and holds. Consider the leading-ordes Taylor expansion of Eq. 9 as initialization towards zero. either p = 1 or m = 1 holds, then the maximal number of roots for Eq. gis twice of the multiplicity of the activation function.
Proof. Case 1: p =
By gradient flow, we can obtain the dynamics for wh!h] (see Appendix A.2
A.2e))
T w.
(f(0, - v:) [diag o' (W (4 1K- ( a) 1K-IT
(10)
1=1
where E' -Wil diag (aag(o((W!l)). [ -.)- for I (2., L), Elaol E*E++1 ...
and at represents the neurons of the k-th layer generated by the i-th sample.
For fixed step, we only consider the gradient of loss w.r.t. w! According to our assumption p 1, we have o' o'(0) لf 0. Suppose that parameters are small and denote f(0, xx) ys. By Taylor expansion,
leading 23
Qw = - (0)) [++1 a));
e 11=11
Pw
1 (diag(o' a));
n
[- u)u
c
0,
8

---

where operator Q is the leading-ordes approximation of operator P, and here E( 1:7] is indepen- dent with i, because diag diag'o'(W'!) (W! = diag(o'(0)}, Since diag(o' (0)} = cI,c # 0 by assumption, and, WLOG, we assume a + then
Qu - 4>
e,a k- 11
lk eyay
u u.
We have
Disi C
[-
Di C
[k 11
| e,ay 12
2|
or u
I> e R8
112
This calculation shows that for layer k; the input weights for any hidden neuron j have the same two stable directions. Together with the analysis before, i.e., when parameters are sufficiently small, the orientation u would move much more quickly than the amplitude r, all input weights would move towards the same direction or the opposite direction, i.e., condensation on a line, under small initialization.
Case 2: the k-th layer with one-dimensional input, Le., M;-1 =
By the definition of the multiplicity p. we have
(00(()) (p - 1)! (w x;) + ol(w 2;))-1).
(w x3)
where (.)--1 and o (p) operate component here. Then up to the leading order in terms of the magnitude of 0, we have (see Appendix A.2
Pw
leading order
Qw ==
1 - [(=
Σ
e
l| 1]
Tw
ot 1//--1l--1) [diag]
1).
(p)
E ( a)l}}
(p- 1)!
c,a |-- 1| ( T z!! |t- ( |diag (P)
+[(
| (E 1:1] a)); u)u).
(Eik-I:Lla))
(p- 1)1
WLOG, we also assume + O. And by definition, w ru, we have
Qu = 0 + u= + e
[1-1] 1]
(w (w
[k-1 y-11
1 D e:a; e [R-1] (u zz 1k- 1k--11 1/p-011
Σ c
[k- Tuxx
[k- 1111112
1 Ei c,
|k- 1] tutz!
[k y0 112
n
1) and ma-1 = 1 ), we denote u (u,, T2)TT E n2 and x; [t-1] ((x, [1 ), (z) k ) E R²² then,
Since mi-1 1 + 1 = (for x 11 = (o(Wik-
E (uz (2 [+ + uz( | 1| |22 -- | |k 11
81 - à.
Et (2) (- uz
| 11)2) -- ee( [k- -112
u2
We obtain the equation for ú,
Σ (a(a " + (a. [k ((=="))2) (e))-lec(x)) [ = à Σ (a(a)) (a(a)"")))
! + "2)"))))))-_)) 1)
Since it is an univariate p-th order equation, á 21 has at most p complex roots. Because u is a unit vector, u at most has p pairs of values, in which each pair are opposite.
Taken together, our theoretical analysis is consistent with our experiments, that is, the maximal number of condensed orientations is twice the multiplicity of the activation function used when parameters are small. As many commonly used activation functions are either multiplicity p = 1 or ReLU-like, our thcoretical analysis is widely applied and sheds light on practical training.

---

6 Discussion
In this work, we have shown that the characteristic of the activation function, i.e., multiplicity, is key factor to understanding the complexity of NN output and the weight condensation at initial training. The condensation restricts the NN to effectively low-capacity at the initial training stage, even for finite-width NNs. In the initial stage, the Taylor expansion upto the leading order indicates that the activation function can be approximated by a homogeneous polynomial. Thus, even the amplitudes of weights within a layer are not identical, the network can be reduced approximately. During the training. the NN increases its capacity to better fit the data, leading to a potential explanation for their good generalization in practical problems. This work also serves starting point for further studying the condensation for multiple-layer neural networks throughout the training process.
For general multiplicity with high-dimensional input data, the theoretical analysis for the initial condensation is a very difficult problem, which is equivalent to counting the number of the roots of a high-order multivariate polynomial with special structure originated from NNs. Training data can also affect the condensation but not the maximal number of condensed orientations. When data is simple, such as low frequency, the number of the condensed orientations can be less, some experiments of MNIST and CIFAR100 can be found in Appendix A.7
Acknowledgments and Disclosure of Funding
This work is sponsored by the National Key R&D Program of China Grant No. 2022YFA 1002200 (Z.
X., T. L., Y. Z.). the Shanghai Sailing Program, the Natural Science Foundation of Shanghai Grant No. (Z. X.), the National Natural Science Foundation China Grant No. 62002221 I (Z. X.), the National Natural Science Foundation of China Grant No. 12101401 (T. L.), Shanghai Municipal Science and Technology Key Project No. 22JC1401500 (T. L.), the National Natural Science Foundation of China Grant No. 12101402 (Y. Z.), Shanghai Municipal of Science and Technology Project Grant No. 20JC1419500 (Y.Z.). the Lingang Laboratory Grant No. BALLL--S-2022000 202202-08 (Y.Z.), Shanghai Municipal of Science and Technology Major Project No. 2021SHZDZX0102 and the HPC of School of Mathematical Sciences and the Student Innovation Center, and the Siyuan- cluster supported by the Center for High Performance Computing Shanghai Jiao Tong University, Key Laboratory of Marine Intelligent Equipment and System, Ministry of Education, P.R. China.
References
L. Breiman, Reflections after refereeing papers for nips, The Mathematics of Generalization XX (1995) 11-15.
C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals, Understanding deep learning (still) requires rethinking generalization, Communications of the ACM 64 (2021) 107-115.
T. Luo, Z.-Q. J. Xu, Z. Ma, Y. Zhang, Phase diagram for two-layer relu neural networks at infinite- width limit, Journal of Machine Learning Research 22 (2021) 1-47.
P. L.. Bartlett, S. Mendelson, Rademacher and gaussian complexities Risk bounds and structural results, Journal of Machine Learning Research 3 (2002) 463-482.
S. Mei, T. Misiakiewicz, A. Montanari, Mean-fich field theory of two-layers neural networks: dimension- free bounds and kernel limit, in: Conference Learning Theory, PMLR, 2019, pp. 2388-2464
G. M. Rotskoff, E. Vanden Vanden-Eijnden, Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018. pp. 7146-7155.
I. Chizat, F. Bach, On the global convergence of gradient descent for weeepparamtterized models using optimal transport, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 3040-3050.
Sirignano, K. Spiliopoulos, Mean field analysis of neural networks: central limit theorem, Stochastic Processes and their Applications 130 (2020) 1820-1852.
10

---

M. S. Advani, A. M. Saxe, H. Sompolinsky, High-dimensional dynamics of generalization error in neural networks, Neural Networks 132 (2020) 428 4466.
M. Phuong, C. H. Lampert, The inductive bias of relu networks on orthogonally separable data, in: International Conference on Learning Representations, 2020.
K. Lyu, Z. Li, R. Wang, S. Arora, Gradient descent on two-layer nets: Margin maximization and simplicity bias, Advances in Neural Information Processing Systems 34 (2021).
Z.-Q. J. Xu, Y. Zhang, T. Luo, Xiao, Z. Ma, Frequency principle: Fourier analysis sheds light on deep neural networks, Communications in Computational Physics 28 (2020)
N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, Courville, On the spectral bias of neural networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 5301-5310.
D. Arpit, S. Jastrzebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio, et al., A closer look at memorization in deep networks, in: International Conference on Machine Learning, PMLR, 2017, pp. 233-242.
S. Fort, G. K. Dziugaite, M. Paul, S. Kharaghani, D. M. Roy, S. Ganguli, Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel, Advances in Neural Information Processing Systems 33 (2020) 5850-5861.
W. Hu, L. Xiao, B. Adlam, J. Pennington, The surprising simplicity of the early-time learn- ing dynamics of neural networks, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Bal- can, H. Lin (Eds.), Advances in Neural Information Processing Systems 33: Annual Con- ference Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL: http::ttttt///occeeedeeeeo neurips wtwpsccc/pppaaae httpa/rrrppppcccapp/////////////
Abst htm.a.tttmmmlll. html
Y. Jiang, B. Neyshabur, H. Mobahi, D. Krishnan, S. Bengio, Fantastic generalization measures and where to find them, arXiv preprint arXiv: (2019).
H. Li, Z. Xu, G. Taylor, C. Studer, T. Goldstein, Visualizing the loss landscape of neural nets, Advances in neural information processing systems 31 (2018).
H. Maennel, O. Bousquet, S. Gelly, Gradient descent quantizes relu network features, arXiv preprint arXiv:1803.08367 (2018).
F. Pellegrini, G. Biroli, An analytic theory of shallow networks dynamics for hinge loss classification, Advances in Neural Information Processing Systems 33 (2020).
A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: convergence and generalization in neural networks, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 8580-8589.
S. Arora, S. S. Du, W. Hu, Z. Li, R. R. Salakhutdinov, R. Wang. On exact computation with an infinitely wide neural net, Advances in Neural Information Processing Systems 32 (2019) 8141-8150.
Y. Zhang, Z.-Q. J. Xu, T. Luo, Z. Ma, type of generalization error induced by initialization in deep neural networks, in: Mathematical and Scientific Machine Learning, PMLR, 2020, pp. 144.
W. E, C. Ma. L. Wu, A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics, Science China Mathematics (2020) 1-24.
L. Chizat, F. Bach, A note on lazy training in supervised differentiable programming, in: 32nd Conf. Neural Information Processing Systems (NeurIPS 2018), 2019.
H. Zhou, Q Zhou, Z. Jin, T. Luo, Y. Zhang, 7.-Q. J. Xu, Empirical phase diagram for three-layer neural networks with infinite width, arXiv preprint 12110 (2022).

---

Y. Zhang, Z. Zhang, T. Luo, Z.-Q. J. Xu, Embedding principle of loss landscape of deep neural networks, arXiv preprint arXiv:2105.14573 (2021a).
Y. Zhang, Y. Li, Z. Zhang, T. Luo, Z.-Q. Xu, Embedding principle: a hierarchical structure of loss landscape of deep neural networks, arXiv preprint arXiv:2111. (2021b).
K. Fukumizu, S.-i. Amari, Local minima and plateaus in hierarchical structures of multilayer perceptrons, Neural networks 13 (2000) 317-327.
K. Fukumizu, S. Yamaguchi, Y.-. Mototake, M. Tanaka, Semi-flat minima and saddle points by embedding neural networks overparameterization, Advances neural information processing systems 32 (2019).
B. Simsek, F. Ged, A. Jacot, F. Spadaro, C. Hongler, W. Gerstner, J. Brea, Geometry of the loss landscape overparameterized neural networks: Symmetries and invariances, in: International Conference on Machine Learning, PMLR, 2021, pp. 9722-9732.
Z.-Q. J. Xu, Y. Zhang, Y. Xiao, Training behavior of deep neural network in frequency domain, International Conference on Neural Information Processing (2019) 264-274.
N. Rahaman, D. Arpit, A. Baratin, F. Draxler, M. Lin, F. A. Hamprecht, Y. Bengio, A. Courville, On the spectral bias of deep neural networks, International Conference Machine Learning (2019).
Jin, L. Lu, Tang, G. E. Karniadakis, Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness, Neural Networks 130 (2020) 85-99.
D. Kalimeris, G. Kaplun, P. Nakkiran, B. Edelman, T. Yang, B. Barak, H. Zhang, Sgd on neural networks learns functions of increasing complexity, Advances in Neural Information Processing Systems 32 (2019) 3496-3506.
C. Ma, L. Wu, E. Weinan, The slow deterioration of the generalization error of the random feature model, in: Mathematical and Scientific Machine Learning. PMLR, 2020, pp. 373-389.
Z.-Q. J. Xu, H. Zhou, Deep frequency principle towards understanding why deeper learning is faster, in: Proceedings of the AAAI Conference Artificial Intelligence, volume 35, 2021.
Z. Liu, W. Cai, Z.-Q. J. Xu, Multi-scale deep neural network (mscalednn) for solving poisson- boltzmann equation in complex domains, Communications in Computational Physics 28 (2020) 1970-2001.
W. Cai, X. Li, L. Liu, A phase shift deep neural network for high frequency approximation and wave problems, SIAM Journal on Scientific Computing 42 (2020) A3285-A3312.
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovicch Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, Barron, R. Ng. Fourier features lct networks learn high frequency functions in low dimensional
domains, in: Advances in Neural Information Processing Systems, volume 33, Curran Associates, Inc., 2020, pp. 7537-7547.
X.-A. Li, Z. Xu, I.. Zhang, A multi-scale dnn algorithm for nonlinear elliptic equations with multiple scales, Communications in Computational Physies 28 (2020) 1886-1906.
X.-A. Li, Z.-Q J. Xu. L. Zhang, Subspace decomposition based dnn algorithm for elliptic-tyre type scale pdes, arXiv preprint arXiv:21 (2021).
X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings the thirteenth international conference on artificial intelligence and statistics, 2010, pp. 249-256
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
12

---

Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A]. You are strongly encouraged include justification your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license the code and datasets? [Yes] See Section ??.
Did you include the license to the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] In Sec. and Abstract.
(b) Did you describe the limitations of your work? [N/A]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
In Sec. 3 and
5
(b) Did you include complete proofs of all theoretical results? [Yes] In Sec. and
Appendix(A.2
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or URLY? [Yes] In the supplemental material.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] In Sec. and the captions of each figure.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "sY8IQ7Cvl7n", "reviewer": "Reviewer_Mpot", "summary": "The paper studies the condensation phenomenon qualitatively in neural networks with various activation functions. The number of condensed directions can be at most twice the multiplicity of the activation function, which is shown empirically and some theoretical calculations are given to back up. \n\n", "questions": "Can the authors comment on the difference between Figures 2 and 4 for the sigmoidal and softmax regarding the number of condensed directions? tanh is an odd function, so it makes sense to get $u$ and $-u$, but it is less clear for the other activation functions. \n\nline 64: missing citations! The so-called embedding principle is identified first by Fukumizu and Amari [1] for the addition of one neuron, then generalized in Fukumizu et al. [2] and even more the construction and scaling of such manifolds of critical points is studied in Simsek et al. [3] from the perspective of permutation-symmetry. \n\nline 101: dose -> does\n\nline 103: \"can across\" unclear what is meant in this sentence\n\nline 184: \"It is reasonable\" -> \"it is reasonable\"\n\nline 220: \"suppose we only consider the leading term\" -> this is in no way a theorem statement, please remove the theorem format from the calculations. \n\nI'm willing to increase my score if my concerns are addressed. \n\n[1] Fukumizu, Kenji, and Shun-ichi Amari. \"Local minima and plateaus in hierarchical structures of multilayer perceptrons.\" Neural networks 13.3 (2000): 317-327.\n\n[2] Fukumizu, Kenji, et al. \"Semi-flat minima and saddle points by embedding neural networks to overparameterization.\" Advances in neural information processing systems 32 (2019).\n\n[3] Şimşek, Berfin, et al. \"Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances.\" arXiv preprint arXiv:2105.12221 (2021).", "limitations": "Yes. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: As indicated by the authors, condensation is analyzed before for example by Maennel et al. The paper generalizes the analysis for multiple activation functions. \n\nQuality: Various experiments are presented to study the phenomenon in models from ResNets to two-layer models, to a six-layer network with skip connections. The problem is with Theorem 5.1, although the calculations are informative, they should not be called theorems as they are not rigorous. \n\nClarity: Writing is coherent, but some individual sentences have slight English issues. The figures are nice and illustrative. \n\nSignificance: I think it is interesting for the NeurIps community to understand the initial training dynamics in neural networks. This can yield new inductive biases and training algorithms; also relevant for pruning, compressed networks etc. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "k6mCMaVoQHZ", "reviewer": "Reviewer_vTes", "summary": "The authors studied the empirical phenomenon of condensation during NN training, where the weight vectors of individual hidden neurons in a two-hidden-layer network concentrate in a few directions. The authors demonstrated empirically that the number of concentrated directions is related to the neuronal nonlinearity and specifically its multiplicity. The author then supplied a theoretical analysis under some heuristic assumptions of how this relationship arises.", "questions": "I would appreciate if the authors can better elucidate how the very condensed stage that the authors study is related to expressive, fully trained NNs.", "limitations": "See above.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: the writing is very clear. Concepts, assumptions and definitions are all well-defined, and the empirical and theoretical claims are clearly stated. \nThe empirical results connecting the number of condensed directions and the multiplicity of the nonlinearity seem very convincing, at least in the settings that the authors explored. \n\nWeakness: My main critique is that I'm not sure that the studied regime is an interesting one. If I am interpreting Fig.2 correctly, the authors are suggesting that when using the tanh nonlinearity, the network condenses to an effective network with just two hidden neurons in this initial stage. The trained networks in practical settings will need much bigger effective sizes to fit training sets, and it is not clear to me how behaviors in this early, condensed regime are related to generalization and other phenomena of interest later on in training. The authors appear to suggest that condensation is important because condensation = smaller effective networks = fewer parameters = less overfitting. But first of all fewer parameters does not equate less overfitting, given what we now understand about various double descent phenomena. Second, while the fully trained networks may have effective sizes smaller than their real sizes, the effective size is almost certainly not as small as 2, as suggested by the tanh analysis.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FDhP1UmfTi", "reviewer": "Reviewer_qmpQ", "summary": "Understanding how the weights of a neural network converge during training is an important task in deep learning. In this paper, the authors empirically investigate this problem for deep neural networks at the initial stage of training. Through a set of experiments on both synthetic and realistic data sets, they show that for small enough initialization, the orientation of the weight vectors to hidden neurons converge in a maximal number of directions as given by twice the multiplicity of the activation function. Here, the multiplicity is defined as the integer $p$ for which the derivative of the activation function at $x=0$ is zero up to the order $(p-1)$, and is non-zero for the $p$-th order. \n\nThen, they provide some theoretical support for their experiments by presenting a preliminary analysis, in which they consider a Taylor approximation of the activation function at zero and show that, in the limit of vanishing initialization, convergence of orientations at early stage of training basically implies the alignment along a fixed number of directions as specified by the multiplicity. This analysis is done for activation functions with multiplicity 1 and high dimensional data, and for general multiplicity with 1 dimensional data.", "questions": "Comments and questions:\n- There is some gap between the theoretical and empirical evidence: the theoretical analysis of the paper is done for gradient flow, but all the experiments seem to be done with Adam. How do the results look like if you use gradient descent (full batch size) with a small learning rate?\n- It is mentioned in the introduction that condensation of orientation implies networks with small complexity. I agree that if the hidden neurons have identical directions then one can use the homogeneity of ReLU to rewrite the output and obtain a narrower network. However, this seems not possible for smooth activations. Thus, what are exactly the implications of condensation studied in your paper? What kind of generalization bounds can be proved for condensed weights?\n- If my understanding is correct, the proof of Theorem 5.1 requires an implicit assumption, that is, the weight orientation must converge, and the convergence must occur near the initialization so that all the weights are still very small and the Taylor approximation is valid. However, I feel that this assumption is quite strong. It is not even clear a priori why the direction of the weight vectors should converge for this setting.", "limitations": "Limitations are discussed above. I do not see any societal impact.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality**\nPrevious work [Luo et al 2022] empirically observe that for a two-layer ReLU network with small initialization, the weights of hidden neurons condense on certain directions. However, it was not studied why it happens, and how it behaves for other activation functions. This paper fills this gap by studying the phenomenon for deep nets with smooth activation, and providing some intuitive explanation for why it happens.\n\n**Quality**\nThe paper is easy to understand. However, there is not much discussion on the motivation and significance of their finding. For example, why it is important to study vanishing initialization if it is not even used in practice, how the behavior of the network at the initial stage of training could be informative for generalization where we only care about the final solution (the later stage of training).\n\n**Clarity**\nThe writing is mostly clear.\n\n**Significance**\nI find the connection between the maximal number of condensed orientations and the multiplicity of activation interesting. I hope that this observation could be helpful for future researches in understanding implicit bias of gradient descent in the early stage of training. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept