Temporary feature collapse phenomenon in early learning of MLPs
Authors: Dongrui Liu
Shaobo Wang
Jie Ren
Kangrui Wang
Sheng Yin
Huiqi Deng
Quanshi Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: Neural Networks
Deep Learning Theory
Multi-Layer Perceptrons

Abstract: In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs). We discover and explain the reason for the feature collapse phenomenon in the first phase, i.e., the diversity of features over different samples keeps decreasing in the first phase, until samples of different categories share almost the same feature, which hurts the optimization of MLPs. We explain such a phenomenon in terms of the learning dynamics of MLPs. Furthermore, we theoretically analyze the reason why four typical operations can alleviate the feature collapse. The code has been attached with the submission.

Under review as a conference paper at ICLR 2023
TEMPORARY FEATURE COLLAPSE PHENOMENON IN EARLY LEARNING OF MLPs
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we focus typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs). We discover and explain the reason for the fea- ture collapse phenomenon in the first phase, ie., the diversity of features over different samples keeps decreasing in the first phase, until samples of different categories share almost the same feature, which hurts the optimization of MLPs. We find that such phenomena usually occur when MLPs are difficult to be trained. We explain such a phenomenon in terms the learning dynamics of MI.ss. Fur- thermore, we theoretically analyze the reason why four typical operations can alleviate the feature collapse. The code has been attached with the submission.
INTRODUCTION
It has been widely observed that in initialized neural networks, especially when the network is deep, the loss decrease is likely to have two phases during early epochs of learning, egg., phenomena observed in (Saxe et al. 2013) (iisselli et al. [209 Stevens al., 2020). As Figure shows, the first phase is usually relatively short, in which the training loss does not decrease or decreases very slowly. Then, in the second phase, the training loss suddenly begins decrease fast.
In particular, as Figure [b)) shows, the length of the first phase increases along with the network complexity. In some extreme cases when deep neural networks (DNNs) are very deep, the loss minimization gets stuck, which can be considered as strong first phase with an infinite length, namely, searning-stickingg problem. In fact, the learning-sticking problem is quite common in practice. Jepkoech et al. () and and[Stevens et al. al. (2020) empirically observed the learning-sticking problem without any theoretical analysis. People usually owed the learning-sticking problem to the over-parameterizze settings of DNNs or the optimization ability of DNNs.
However, we discover and attempt to further theoretically explain a new, quite common, yet counter- intuitive phenomenon in the first phase (the learning-sticking phase). That is, as Figure [b) shows, features of different categories become increusingly similar to each other. In some cases, the feature diversity keeps decreasing even until all samples of different categories share almost the same feature in the first phase. We can consider this as the temporary feature collapse (TFC). This TFC happens in various DNNs, including multi-layer perceptrons (MIPs), convolutional neural networks, and recurrent neural networks (see both Figure 2 and Appendix [B). DNNs trained with different loss functions and different learning rates may all exhibit TFC phenomena. The TFC phenomenon usually happens in the early epochs of the training process, especially when the DNN difficult optimize. According our analysis, when the DNN is very deep, when the task is difficult, when the variance of initial weights is small, and when the DNN is trained without momentum or batch normalization layers, the DNN is more likely exhibit the TFC phenomenon.
Based on our theoretical analysis, we discover set of conditions that strengthen the TFC phe- nomenon. Then, we can easily control such conditions by applying typical operations iee. batch normalization, momentum, L2 regularization, and network initialization. Specifically, we investi- gate the learning dynamics the MLP. Moreover, we theoretically explain that these conditions make the training of DNNs more likely to perform like a "self-enhanced system" towards the TFC phenomenon in early iterations. comparison, (Glorot & Bengio 2010 Saxe et al. 2013) inves- tigated the influence of initialization methods the find that the effectiveness initialization methods is probably owing to the large variance of initial weights, which avoids the TFC phenomenon during the -tteiinggttikiiggg phase.
problem. To this end,

---

Under review as a conference paper at ICLR 2023
2.45
3-layer
5-layer
7-layer
9-layer
20-layer
the leaming-sticking
(a)
1.2
problen considered
first with
infinie length
iteration
10
10"
Feature
values
diverse
Festure
valles
0,8
almost the same features
0.1
features
0.4
512 Dimensiors
10
512
312
Dimessions
(b)
5
cemparary Rosture cullage
3
and inining
iteration
the first phase
the second phose
Figure 1: (a) The first phase (learning iterations before the dotted line) gets increasing length and finally becomes the learning-sticking problem (purple curve), when the DNN has more layers. (b) Samples of different categories share almost the same features at the end of the first phase. We can consider this as a TFC phenomenon. We visualize the learning dynamics of an intermediaae--lyee feature in 9-layer MLP. We select 10 salient dimensions illustrate the feature similarity.
Fortunately, we discover that, when we use four typical operations to alleviate the TFC phenomenon, the searning-sticking problem can also be solved. Although previous studies have provided insightful analysis for these well-known operations, e.g., batch normalization and network initialization, we are the first to establish the relationship between the TFC phenomenon and these typical operations. This provides theoretical guidance for the design of DNNs.
More crucially, the TFC phenomenon with the MLP counter-intuitive, and has been neglected for a long time. The investigation of the learning dynamics of the TFC phenomenon would be useful for explaining complex optimization behaviors of DNNs and of considcrable value.
Contributions of this study can be summarized as follows. (1) We discover the common TFC phe- nomenon in early learning of the MLP, which has been ignored for long time. (2) We explain
this phenomenon from the perspective of learning dynamics. (3) We explain why four types of operations can alleviate the TFC phenomenon.
DISCOVERING THE TFC PHENOMENON
It has been widely observed that the loss decrease of DNNs is likely have two phases (Saxe et al., 2013 Simsekli et al.) 2019 Stevens et al., 2020). As Figure [T) shows, the training loss does not decrease significantly in the first phase, and the training loss suddenly begins to decrease in the second phase. In this paper, we discover a new and counter-intuitive phenomenon in the first phase that both the diversity of intermediate-layer features over different samples and the diversity of feature gradients keep decreasing, until samples of different categories share almost the same feature in the first phase. We consider this as a TFC phenomenon.
We consider an MLP with L concatenated linear layers, each being followed by ReLU layer. Only the last linear layer is followed by softmax operation. Let w E Rixd denote the weight matrix of the I-th linear layer with h neurons (1 & & L), and w(()) has been learned for t iterations. Given an input sample æ, the layer-wise forward propagation in the 1-th layer represented as
p(l) ReLU(W p(" = D ((("(( r!!-")
where F Rd denotes the output feature of the 1-th layer after the 1-th iteration. D. denotes a diagonal matrix, which represents gating states in the ReLU layer, and D' E (0. 1}.
Thus, the TFC phenomenon shown as follows. Given two input samples x1 and æ2, the cosine similarity of features cos( l,, ((O) lx.), and the cosine similarity of gradients cos(F!!)) la ((0)) laz) keep increasing, which demonstrates the phenomenon. Here, (:0 denotes the gradient of the loss the feature p(0) Besides, the increasing trend of feature similarity only exists in the first phase.
The TFC phenomenon is widely shared by different DNNs learned for different tasks. In early epochs (or iterations) of the training process, observed such TFC phenomena MLPs, VGG-11
2

---

Under review as a conference paper at ICLR 2023
Layer
Layer
Layer
Layer
Layer 6
Layer?
(c)
9-layer MLPs with
512 neunoms<laee
9-layer MLPPs
9-hayer MLPs with
512 neuronsllaer
128 neurons-laeer
VGG-11
Revised LSTM
CTFAR-10
sinslritty
of prasfients of foutures
CIFAR-10
Tiny ImugeNeet
Commercial
CIFAR-10
Thy ImageNer
CoLA
SST-2
Cootio
Cosiae
Tiny ImageNeet
simillaity
(5)
Cosine
iteration
ierration
ineration
iteration
ileration
iuration
Figure 2: The TFC phenomenon. (a) Cosine similarity of features between samples in different categories ex (1)( 12, 1=)] keeps increasing in the first phase (left to the dotted line),
until the second phase. The low cosine similarity indicates the high diversity. (b) Cosine similarity
of feature gradients between different samples of a category Ex.a'Ex. .cos( (F( lx. F((0 [=.)] keeps in-
creasing in the first phase until the second phase, where X, denotes samples of the category (c)
Cosine similarity of weight changes between weight vectors in layer ExE cos( (Aw Aw("). lx)) keeps increasing in the first phase. Please see Appendixx for results on more DNNs.
The common direction of weight changes be decomposed from two perspectives.
Figure 3:
(a)
Weuss,
cirection
Weakk1
W
Wo-22
Perspective
Perspective 2]
Weights of neu- rons are changed towards com-
Wa-11
Formulated in Equ4
Formulated in Theorem
Wi-q2
direction
W
We-a2
Wowz2
Wo-12
neuron 2
These perspectives enhance each other.
Explained in Lemma 3 and Thooram
mon direction. (b) The logic of ex- plaining the TFC phenomenon.
Such enhuncement explains
direction
Wew43
WeEaR
Wc-23
M
neuron 3
decrease of feature diversity
Discussed Section
69)
(b)
Simonyan Zisserman 2014), and the revised long short-term memory (LSTM) on different types of data, including image data (MNIST (LeCCnn et al. 1998), CIFAR-10 (Krizhessky et al.) 2009), and the Tiny ImageNet dataset (Le & Yang 2015)). tabular data (two UCI datasets census income and TV news (Asuncion & Newman (007)). ). and natural language data (CoLA (Warstadt et al. 2019), SST-2 (Socher et al. 22133), and AGNews (Del Corso et al. (005)). We also tested MLPs with different loss functions, with Leaky ReLU layers (Maas al. 2013). with different learning rates, and with different batch sizes. Figure b)) shows TFC phenomena on these DNNs, and please Appendix for results on more DNNs.
Besides, the learning-sticking problem can be considered as an extreme long first phase. As T shows, the length of the first phase increases along with the network complexity (depth). In extreme cases when DNNs are very deep or the task is difficult, the first phase reaches an infinite length, and the learning gets stuck (please see Appendix Ci more discussions).
3 EXPLAINING THE DYNAMICS OF THE TFC PHENOMENON
In this section, we aim to investigate dynamics of network parameters in early epochs, so as to ex- plain the condition that may boost the likelihood of the TFC phenomenon. In Section 3.] find that the decreasing diversity of feature gradients over different samples is owing to the phenomenon that different neurons in a layer are optimized towards common direction in the first phase. There- fore, we propose two perspectives to illustrate the significance of the common direction. Then, in Section 3.2] we compare these two perspectives to analyze learning dynamics, and we find that the significance of the common direction may be enhanced, just like a "self--nhance" system." Finally, the self-enhanced common direction can explain the TFC phenomenon. The overall logic of the explanation is illustrated in Figure (b). In Section 3 3 we explain the reason why four types of operations can alleviate the TFC phenomenon based on our analysis.
3.1 TWO PERSPECTIVES TO ANALYZE THE COMMON DIRECTION OF LEARNING EFFECTS
In the beginning, let us first focus on the conjecture that the decreasing diversity of feature gradi- ents over different samples be explained by the phenomenon that different neurons in layer
3

---

Under review as a conference paper at ICLR 2023
optimized towards common direction in the first phase. For example, as Figure a) shows, the beginning of the learning, different neurons are originally optimized towards different directions,
but then gradients of different neurons gradually change to a similar direction. Let E:0)) denote the
gradient of the loss w.r.t. the feature F(!) at the I-th layer. Then, according to Eq. 1. the back
propagation of feature gradients F! R² at the I-th layer can be written as
F A0-1 -w!"" D(" p())
(2)
The emergence of common direction of weight changes means that gradients of the d weight vectors in w [w][]] wi,.: ( Rdxh, i.., dLoss /ww gradually become approximately collinear. According to Remark[] we can explain why the enhancement of such common direction
decreases the diversity of feature gradients.
Remark 1. Let us assume that different weight vectors [w] V222.
reel
UT have a dominating
common direction CO E IR.. Then, we can represent wi, (i) = 8,C + where 3. E R; e: R denotes small residual; 3 = [83 B2. Ba],, and e (21, ea] Then, we have
)リ-1) (C")) D. p(0) 3 eD
( p!D)
Remark I well explains the rationale for the above conjecture. That is, if /Lsss/Ow( on different samples are roughly collinear to each other, then such a collinearity would make feature gradi- ents F( of different samples similar to each other. Specifically, during the learning process, if the DNN keeps optimizing wr(h" along the common direction (+0) for a long time, which keeps strengthening the value (* T Df p then feature gradients of different samples are grad- ually pushed towards the same direction B. In other words, as long different weight vectors are optimized towards the same dominating direction, then feature gradients p.( ((!-1) are pushed in the same direction B..
Therefore, the first core task of proving the decreasing diversity of feature gradients is to explain the existence of the common optimization direction shared by different weight vectors. Thus, we propose two perspectives to illustrate how different weight vectors w are changed AW; along common direction during the learning process. By comparing these two perspectives, we can further explain the reason why the significance of the common direction will be further boosted, just like "self-enhanced system." Such
system" will be proven in Section 3.2
This perspective focuses on the influence of the common direction (t()) of the weight change in 1-th layer. For clarity, we omit the superscript (l) to simplify the notation in the following paragraphs in Section 3.1 i.e., Aw!y, AW(", and ((0)) can be simplified by Awt.,, AW, and C, respectively. Let AW, [Awr.1, Aws,,, ... Awraa denote weight changes of weight vectors the 1-th layer. We decompose AW; into the component along a common direction C and a component along other directions as follows.
Perspective 1.
AW. = AV.CT + Des:
(4)
where AV, = [Avr, Av.2. av E Rd denotes the coefficient vector for weight changes of different weight vectors along the common direction C. Specifically, Ae, is relatively small "noise" term, which is orthogonal to C, i.e., =
Lemma 1. (Proof in Appendix(E] F For the decomposition AW,==VVC + given weight changes over different samples AW, we can compute the common direction C by minimizing the fitting error De, when use A to approximate AwT, over different samples across different iterations.
Le., ming, (ErejTannTad BoEE |A ) s.d. DErl= = AW,
AVi Thus, we obtain
AV, AWT C and As,= AW, AW, CC C. S.I. As,C = O. Such settings minimize ||
CTC
Lemma 2. (We can also decompose the weight ((!)) into the component along the common
rection C and the component in other directions. Proof is in Appendix Given the weight W.T w
and the common direction C, the decomposition W.T = V.CT + can be conducted as V. = c and e, W.T W,T CCT C s.t. E,C 0. Such settings minimize ||εe||e.
We conduct experiments to verify the strength of the primary common direction C. To this end, let us focus on the average weight change over different samples AW, Fzex Then,
4

---

Under review as a conference paper at ICLR 2023
Layer 2
Tayer
Layer
Layer 5
Layer
Layer?
Lager
Layer
Layer
Io-4
los3
1e-3
163
le3
1e.4
1e.3
Ie3
led]
lod
Cat cnoggry
Track catigon
1,6
0.0
1.333
0.0
2245
12343
12345
00
5
12345 direction
12345
2243
12345
12345
direction
direction
directionn
direction
Figure 4: The strength of different common directions in the CIFAR- 10 dataset. We trained 9-layer MLPs, where each layer of the MLP had 512 neurons. We illustrated results on the two categories with the highest training accuracies. s; |C,AV; || measures the strength of weight changes along the i-th common direction, where AV; = E.[AV..], The strength of the primary direction was much greater than the strength of other directions. Please see Appendix D for more results on the MNIST dataset and the Tiny ImageNet dataset.
Table I: Strength of components of weight changes along the common direction and other directions. We trained 9-layer MLPs on the CIFAR-10 dataset and the Tiny ImageNet dataset, respectively. Each layer of the MLP had 512 neurons. The strength of the primary common direction was much greater than those of other directions. Appendix
[ provides results on the MNIST dataset and do not decrease monotonically.
Appendix H.2 explains the phenomenon that s!9, S2 and s
Caregory Sipue
Car
Trick
CIAAR-D 10
Layar
Layer
Layer4
Layer
Layer
Layer
Layer
Layer
Layer
Layer
154.0gm
176.5gas
201.6g187
253.6
277.4ga54
160.9
208.1
223.6gaaa
248.4gma
281.5g20.4
St.
115g11
13.0gaa
11.0g17
16.1g/
8.000s
15.0ga
14.0g
14.3
14.2g7
10:0g11
S10
127717
11991a
10.9a1a
11.9.a
88111
14.4114
15.1aa
113314
120usa
12.
S
11.0g1
14.4
12.5,e
13.9
86
143,es
12.4
128
18.1
9.7
Category
Plagpoee
Bottle
Thny ImugeNer
Layer
3
Layer
L
Layer
Layer
Laver
Layer
Layer
6
Prizu
07.8ast
1439
108.9as1
250.8
202.
254.aam
27688:1..
345.2114
4402
10.0g0
9,500
14.4g4
2199:
88110
10.3,14
11.2
122,1
11.9gaa
13.3
7.500
7.9
9.7a12
92:1a
83300
10.4a11
11:6a.
18.811
10.0eas
10.6
7.lnn
113,10
17.0.3
16.6,15
11.6.14
1877:
10.7
10.81.
198,
decompose AW, into components along five common directions as AW, CIAVI. CAV 2. +C&VV + AEj,, where Ci=C' is termed the primary common direction. C22 C2, C4 and Cs rep- resent the second, third, forth, and fifth common directions, respectively, C2 C2, Ca. Ca, and C. are orthogonal to each other. C, and AV.. e are computed based on Lemma I when we remove the first (i 1) components along the direction C,... C from the AW, Figure 4 shows that the strength of the primary common component C approximately ten times greater than the strength of the secondary common component C2AV, Please see Appendix G for more discussions.
Perspective based on feature gradients f((+1), We decompose the weight change by considering the influence of the common direction of the upper layer C(++1, In order to distinguish variables
belonging to different layers, we add the superscript (I) back to AW(", AVV"), and Ae to denote the layer in the following paragraphs.
Theorem 1. (Proof in Appendix (H.)) The weight change made by sample can be decomposed into (h + terms after the 1-th iteration as follows.
AW (
AW(l) primary.c e
Σ
AW(l.)
1:(0) 11) T)
+ +
noise.t
denotes the component along the primary common direction, and AW(Lk) mine, = D. Ac((+1) r( re(t1)" /| 12 de- notes the component along the k-th common direction in the noise term. et ((41,))= Eusle where the SVD of E! (1+1) E BBxh' is given as ((+1)) = uEVT (h S h'), and Eus denotes the k-th singular value E R. Ei (1+1) = Ex (t+IN)) Us and Vy denote the k-th column of the matrix and V, respectively. Besides, we have Vk. E (1,2,....,)) ,,,,,, Uh C(41) = Con- sequently, we have 1.00 = D(0 (0+D)a((4)) F(O) ! 12 E R², and aa = el (((())))))( p!D ( Rixd
where AW!I) : V C(++)) CO+DAV AV F.ll p(1-1)) /IIP!"
Given weight changes !w made by a sample x, the primary term (! prmaay, represents the com- ponent of weight changes along the common direction c(++)) The k-th noise term (W())) noise, repre-
sents the component along the k-th direction Ms. which is orthogonal to g0+1)
5

---

Under review as a conference paper at ICLR 2023
We conduct experiments to verify the significant strength of the component along the common direction ((++1), We compute the average strength of the component along C(++1) over all samples in X as (()) poimary = Tan
plmlay, Similarly, the strength of the component along
the k-th noise direction is computed as S/=== Eng(Tan: Ez x (A
| illustrates that the strength of the primary component 5(() prinury is more than ten times greater than the strength of components along noise directions 5 ,5 and 5
noise, Iz llr).. Table
Discussion about comparing with the sum of all other directions' significance. According to Table T it seems that the sum of strengths of components along other directions is also large. However, dif- ferent directions decomposed by the above method are orthogonal to each other. Therefore, weight changes along different directions are independent, and their strengths cannot be summed up. Thus, we can directly compare the strength of the component of weight changes along each direction to verify the significant strength of the primary direction.
3.2 EXPLAINING THE ENHANCEMENT OF THE SIGNIFICANCE OF THE COMMON DIRECTION
The previous subsection owes the decreasing diversity of feature gradients to the typical common optimization direction shared by different weight vectors. In the current subsection, we explain that the common optimization direction phenomenon is very likely to be further enhanced, just like "self-enhanced system." The self-enhancement of the optimization direction will explain the decreasing diversity. Specifically, the overall logic of this subsection has three steps. In Step 1, we explain the phenomenon that the significance of the common direction can be enhanced by training samples in a certain category in very early epochs. In Step 2, we extend the analysis of the enhancement of the common direction to a more generic case, i.e., explaining the enhancement caused by training samples from different categories. In Step 3, we further explain that the self- enhancement of the common direction decreases the diversity of features and feature gradients, i.e. explaining the TFC phenomenon.
Before explaining the enhancement of the significance of the common direction, let us first clar- ify assumptions in the proof. (1) The direct proof of the emergence of "self-enhanced system" from the very beginning of training an initialized MIP is difficult. Instead, we explain that the self-enhancement of the common direction probably started under the background assumption that features of different samples have been pushed a little bit towards a specific common direction. (2) The MLP usually first learns few categories, instead of simultaneously learning all categories. Ex- perimental results in Figure 7 and Appendix havve verified the trustworthiness of this assumption.
According to Eq. and Eq. 5) weight changes made by the sample æ can be given as
Perspective 1: AW (( AV."
+ AE Perspective 2: AW!" (((") O
(6)
By comparing the above two perspectives, we discover an interesting potential that the common direction ( is similar = and the feature F((-1) is similar to the vector =AV(
Inspired by this, we aim to prove the seff-enhancement of the significance of the common direc- tion, by explaining the intuition that the feature F(!-1) and the vector v.!)) become more and more similar to each other in the first phase. As the first step of the proof, Theorem 2 shows that if we only consider training samples x X. in the same category c, then features r:" of samples in this category would become increasingly similar to each other. On the other hand, such training samples have similar training effects, i.e., pushing weights of different neurons v,() all towards the average feature a.Exex, !A! (A( []].
Step 1: Explaining the significance of the common direction is enhanced by all training samples in a certain category. Specifically, let us first consider the aforementioned background assumption that features 1((-1) of different samples have been pushed a little bit towards a specific common direction. We can obtain that there exists at least one learning iteration in the first phase, in which AF((-1) and fi((-1) of most samples have similar directions, and Av and vi()) have similar direc- tions (see Appendix]ffo for more discussions). Note that the assumed initial common direction is quite vague, and it is far from the TFC phenomenon. However, if we take the vague common direction as the starting point, we can further prove the significant self-ennhnnemmen of the common direction, which is responsible for the TFC phenomenon.

---

Under review as a conference paper at ICLR 2023
Cusegoey
Layer
Layer
Layer-4 4
Layer 5
Lager
Caregory
Layer
Layer
Layer
Layer
Layer
Simburity
-0.6
4099
00.6
0.5
0.99
0.9
Flng polk
Simlarity
0.9
0.6
1,0
0.8
1.0
-10
-1.
Trok
Sinlartty
0.6
a
(6)
Book
Simlunity
1.0
a
6.
...9
40.9
0.8
08
900
200
$00
800
500
900
1.0
90
2500 5000
2000 3000
2000 3000
2000 3000
2000 3000
Seration
iteration
iteratton
ieeration
Itertionn
Iteration
Itaration
Inoration
Iteration
Itaration
Figure 5: The average cosine similarity between the feature ((11 and the vector Av over dif- ferent samples in the first phase. We conducted experiments on 9-layer MLPs trained on the (a) CIFAR-10 dataset, and the (b) Tiny ImageNet dataset. The shade in each subfigure represents the standard deviation of the cosine similarity over different samples.
Cuegory
Tayar
Layor
Layer 4
Layer
Layer
Categrry
Tayer
Tayer 3
Tayer
Tayer 5
Lager
Car
0.
0.6
0.6
0.8
0.6
Flns pole
06
0.6
066
0.2
0.4
0.5
0.2
102
0.4
0.4
0.4
(9)
(9)
Truck
04
0.4
0.4
0.6
0.5
0.6
0.5
Bottle
0.6
0.4
0.6
0.4
0.6
0.2
0.4
9010
X00
900
2800
7000
2500
3000
2000 3000
2000 2000 Heraaiet
2000 3000
Homsioo
Horration
Denution
Serali
Figure 6: The change of (4) the first phase. We trained 9-layer MLPs on the (a) CIFAR-IO and the (b) Tiny ImageNet. Each layer of the MLP had 512 neurons. The Appendix D provides results on the MNIST. The shade represents the standard deviation over different samples.
Thus, Theorem 2 explains how the significance of the common direction is enhanced by all training sampley in the category c, i.e,, F and V.". become increasingly similar. We can consider cos(acV() ((() AF 2 0 in Theorem 2 means that features of training samples in the same cat- egory с are all pushed towards a common direction V.VV and make Ar. highly similar to a.vv i.e., making sample features E((L-)) la in the category e become increasingly similar to each other: On the other hand, cos(a..Vv AV."), (2( 2 0 in Theorem 2 means that training sam- ples in the category all push v:O) towards u.Ezex., [r!" la], and make Av roughly parallel to a.Euex. [+:""" l.], i.e., pushing weights of different neurons v.00) towards the average feature. This phenomenon is verified in Figure 5 where cos(a, E((-D) is always positive over dif- ferent samples of the same category. The above analysis also well explains the dynamics behind (AV()) f((-1)) cos(VV"), AF! AF((-i) in Lemma
(1))
input sample x E X and common di- rection C after the t-th iteration, if the noise term el is small enough satisfy |AV(O) F(i p( VI AVA p!l > IAV."" p("-Dv!"" st AE an p(i-"),, we can obtain cos( AVV(" ccosV)) AF( 2 0, where AV AW!l oin
Lemma 3. (Proof in Appendix K Given
o(t) cay and v.O w(t) ci
cis cay'
made by the training sample x after the 1-th iteration. To this end, approximately consider the change of features Ar(!-1) af-
AE denotes the change of features AE( r(t-") -E
1
ter the t-th iteration negatively parallel to feature gradients ((1-1), although strictly speaking, the change of features is not exactly equal to the feature gradients.
Theorem 2. (Proof in Appendix D Under the aforementioned background assumption, for any F:00 [2)
training samples x, x' E Xo in the category c, ( 12 ((") [2) 0 (i.e., E!!) and F Va have kinds of similarity in very early iterations), then cos(a. |2, (2) 2 0, and
1 is a constant shared by all samples in category c. cos (a (. A( 2 0, where at E(-,++11
We conduct experiments to verify the relationship between the feature r((l11) and the vector v:() To this end, we measure the change of the value ,0 =cos(
Figure ree- ports the average od value over different samples at each iteration. For each sample x. ( is always positive and usually increases over iterations, which verifies Lemma 3 Besides, the assumption for
(Av (AV."
a tiny (1) in Lemma 3 is verified by experimental results in Appendix K
In sum, Step 1 focuses the significance of the common direction enhanced by training samples in a certain category. In order to further analyze the learning effect of training samples from different categories, we propose Assumption T according to extensive experimental observations.
7

---

Under review as a conference paper at ICLR 2023
Accuracy (%)
CIFA--1
MNIST
Figure 7: The training accuracy of MLPs on different categories at the end of the first phase. The MLP only learned features of a single or two categories in the first phase.
40-
randem grawiing
20
1 2.3 4 8
0 012345678
calegury index
category indes
Assumption 1. We assume that the MLP encodes features of very few (a single or two) categories the first phase, instead of simultanoously learning all or most categories in this phase.
Assumption T indicates that MLPs first Iearn single or two categories in the first phase. Figure
verifies that only a single or two categories exhibit much higher accuracies than the random guessing at the end of the first phase. This means that the learning of the MLP is dominated by training samples of a single or two categories in very early iterations. Please see Appendix O for more results on different DNNs.
Extending the enhancement of the significance of the common direction to a more general case that considers all training samples from different categories, i.e., and-"aand oz v become increasingly similar. The overall learning dynamics in the first phase can be roughly de- scribed, by combining Theoremāaand Assumption as follows. Assumption [ indicaates that MLPs encode features of very few (a single or two) categories in early epochs. In other words, the over- all learning effects of all training samples are dominated by very few categories è, Based on this, Theorem 2 indicates two effects. First, features F(I-1) of different samples are all pushed towards the vector are vi(), where ag is determined by the dominating category/categories é. Second, v. is pushed towards Ex. (1r((-1/,) Therefore, features rf(-")of different samples and og v:( en- hance each other, just like a "self-enhancee" system." The "self-enhanced system" starts from from the assumed state that ( and of most samples have similar directions, and Av;"0 and v(()) have similar directions. In other words, the component along the common direction CO v in A wW"l_ooAv Ael"' will be further enhanced.
Step 2:
Step 3: Explaining the increasing feature similarity and the increasing gradient similarity. i.., explaining the TFC phenomenon. As aforementioned, features F!(11)) of different samples are consistently pushed towards the same vector o. v((") It increases the similarity between features of different samples ex (x._'" (cos(P!"-")) ing similarity between feature gradients can be also explained from two views. (1) The increasing feature similarity over different samples makes different training samples generate similar gating states D in each ReLU layer. The increasing similarity of ReLU layers' gating states between different samples also increases the similarity of feature gradients between different samples in the same category Be, xx.=o/x p(t-"( 1,, (())) (2) Another view is that the component along the common direction C(VV in w(O) is enhanced in the first phase. Because CO) denotes the principle weight direction of the i-th column ue() of w(O,, each weight vector w/? is optimized towards the common direction cC. Eq. (3) shows that the increasing cosine similarity between (!) and cC for all weight vectors will boost the similarity between feature gradients of different samples.
F((-"(=>]] in the first phase. On the other hand, the increas-
Vanishing gradients on correctly classified samples destroy the "self-enhanced system." " All our explanation focuses on the early epochs of training, when only few training samples of one two dominating categories can be confidently classified. However, when the optimization of a single or two dominating categories in the first phase soon saturates the end of the first phase, gradients on the correctly classified samples of the dominating categories vanish. Then, gradients from training samples of other categories weaken the dominating role of a single or two categories in the learning of the MLP. Thus, the "self-enhancee system" is destroyed, and the learning of the MLP enters the second phase.
3.3 THEORETICALLY ALLEVIATING THE TFC PHENOMENON
In previous sections, we have discovered and explained a fundamental yet counter-intuitive TFC phenomenon with the MLP. This is the distinctive contribution of this study, which has not been theoretically explained for long time. Besides, we find that we use the above findings to explain that four typical operations can usually alleviate or strengthen the TFC phenomenon, i.e., normalization, momentum, initialization, and L2 regularization. Although these operations have
8

---

Under review as a conference paper at ICLR 2023
CIFAR-10
MNNST
Tiny ImageNot
CIFAR-10
MNIST
Tiny ImageNot
Cosime Similarity of features
04
(x)
(b)
1000
200
2300
3000
1000
3000
1030
2000
2500
5000
iteration
iteration
iteration
iteration
iteration
iteration LF
L-9, BN
L-9 L-7, BN
L-7
L-
11055---99-1 L-
y-
Figure 8: Effects of (a) normalization and (b) initialization. We trained L-layer MLPs, where each layer had 512 neurons. A shorter first phase indicates that the TFC phenomenon is more alleviated. Effects of momentum and L2 regularization are shown in Appendix M..2
been widely used, previous studies failed to theoretically explain their effectiveness. To this end, our analysis can explain a high likelihood for such operations to affect the TFC phenomenon, although is not proof of strict sufficient condition or a necessary condition for the TFC phenomenon.
Centering operations for normalization. Based on theoretical analysis, we explain that the cen- tering operation normalization operations (e.g., that in batch normalization (BN)) can alleviate the TFC phenomenon in the first phase. Specifically, according to Theorem 2 the "self-enhanced system" of decreasing feature diversity requires features r7!" of any two training samples and x' in the same category to be similar to each other. However, the centering operation prevents features 12(1) of different samples from being similar to each other, because subtracts the mean feature F(D) E.ex (P(" [x] from features of all samples, i.e., F (112 F. l--("") Therefore, the dissimi- larity between features of different samples breaks the "self-enhanced system." Please see Appendix M. for more discussions.
We conducted experiments to verify the above analysis. We compared MLPs trained with and without BN layers. Specifically, we added BN layer after each linear layer construct MLPs. Figure 8 a) shows that the feature similarity in MLPs with BN layers kept decreasing. This verified that BN layers alleviated the TFC phenomenon.
Momentum. Our theorems explain that momentum in gradient descent can alleviate the TFC phe- nomenon. Based on Lemma 3 the "self-enhanced systemi of the decreasing of feature diversity requires weights along other directions e!!) be small enough. However, because the momentum operation strengthens influences of the initialized noisy weights w:l it strengthens singular values of (0, to some extent, thereby alleviating the TFC phenomenon. Specifically, larger momentum coefficient usually more alleviates the TFC phenomenon. To this end, we trained MLPs with differ- ent momentum coefficients, and experimental results Appendix M.2 verified the above analysis.
Initialization. We explain that the initialization of MLPs affects the TFC phenomenon. According to Lemma 3 the "self-enhanced system" requires very small weights along noise directions e(") However, increasing the variance of the initialized weights ((()
t= can boost singular values of e
which alleviates the TFC phenomenon. Please see Appendix M.3 for more discussions.
To verify the above claim, we conducted experiments by comparing MLPs trained using different initializations with different variances. We used 7 to control the variance the initialization, i.e. t=0 ~ (O,, yoārd), where is a constant computed following (Glorot & Bengio 2010). Figure (b) verifies that the initialization with large variance alleviated the TFC phenomenon.
Le regularization (ridge loss). We also explain that the L2 regularization (the ridge loss) can strengthen the TFC phenomenon. The total loss is given as L(W,) = _ (W,) + W,W li. where (W.) represents the cross entropy loss, and >/|Wa]] denotes the ridge loss. As aforementioned, the TFC phenomenon requires singular values of to be small enough. However, because the loss of |[W.] penalizes singular values of e() it strengthens the TFC phenomenon. The experimental verification is provided in Appendix(M.4
CONCLUSION
In this paper, we find that in the early stage of the training process, the MLP exhibits a fundamental yet counter-intuiiive TFC phenomenon, i.e. the feature diversity keeps decreasing in the first phase. We explain this phenomenon by analyzing the learning dynamics of the MLP. Furthermore, we explain the reason why four typical operations can alleviate the TFC phenomenon.

---

Under review as a conference paper at ICLR 2023
ETHIC STATEMENT
we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs). We discover and explain the reason for the feature collapse phenomenon in the first phase, i.e, the diversity of features over different samples keeps decreasing in the first phase, until samples of different categories share almost the same feature, which hurts the optimization of MLPs. There no ethic issues with this paper.
REPRODUCIBILITY STATEMENT
We have provided the proof for all theoretical results in Appendix E Appendix K and Appendix L We have also provided experimental details in Appendix B The code has been attached with the submission.
Appendix F Appendix H.I
REFERENCES
Alessandro Achille and Stefano Soatto. Information dropout: Learning optimal representations through noisy computation. IEEE transactions on pultern analysis and machine intelligence, 40 (12):2897-2905, 2018.
Ali Arab, Betry Chinda, George Medvedev, William Siu, Hui Guo, Tao Gu, Sylvain Moreno, Ghas- san Hamarnh,, Martin Ester, and Xiaowei Song. A fast and fully- aullmaaet deep-learning ap- proach for accurate hemorrhage segmentation and volume quantification in non-contrast whole- head ct. Scientific Reports, 10(1):1-12, 2020.
Sanjeev Arora, Nadav Cohen, and Elad Hazan. On the optimization of deep networks: Implicit acceleration by overparameterization. In International Conference on Machine Learning. pp- 244-253. PMLR, 2018.
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxin- der S Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et closer look at memorization in deep networks. In International Conference on Machine Learning, pp. 242. PMLR, 2017.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Pierre Baldi and Kurt Hornik. Neural networks and principal component analysis: Learning from examples without local minima. Neural networks, 2(1))53-58, 1989.
Hao Cheng, Dongze Lian, Shenghua Gao, and Yanlin Geng. Evaluating capability of deep neural networks for image classification via information plane. In European Conference on Computer Vision, pp. 185- Springer, 2018.
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Artificial intelligence and statistics, pp. 192-204. PMLR, 2015.
Amit Daniely, Roy Frostig, and Yoram Singer. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity. Advances In Neural Information Processing Systems, 29:2253-2266 2016.
Gianna M Del Corso, Antonio Gulli, and Francesco Romani. Ranking a stream of news. In Pro- ceedings of the 14th international conference on World Wide Web, pp. 97-106, 2005.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x scale, 2020.
words: Transformers for image recognition at
Simon S Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. Gradient descent provably optimizes wee-pparamteriize neural networks. In International Conference on Learning Representattinss 2018.
10

---

Under review as a conference paper at ICLR 2023
Stanislav Fort, Pawet Krzysztof Nowak, Stanislaw Jastrzebski, and Srini Narayanan. Stiffness: A new perspective on generalization in neural networks. arXiv preprint Saee:1901.0999... 2019.
Angus Galloway, Anna Golubeva, Thomas Tanay, Medhat Moussa, and Graham W Taylor. Batch normalization a cause of adversarial vulnerability. arXiv preprint arXiv: 2019.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256. JMLR Workshop and Conference Proceedings. 2010.
XY Han, Vardan Papyan, and David Donoho. Neural collapse under mse loss: Proximity to and dynamics the central path. arXiv preprint arXiv:2106.02073 2021.
Moritz Hardt and Tengyu Ma. Identity matters in deep learning. arXiv preprint arXiv:1611.04231, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition, 2015.
Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer, generalize better: closing the general- ization gap in large batch training of neural networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1729-1739, 2017.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. PMLR, 2015,
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kemel: Convergence and gen- eralization in neural networks. arXiv preprint arXiv:1806.07572 2018.
Stanistaw Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Three factors influencing minima in sgd. arXiv preprint arXiv:1711.04623 2017.
Jennifer Jepkoech, David Muchangi Mugo, Benson K Kenduiywo, and Edna Chebet Too. The effect of adaptive learning rate on the accuracy neural networks. International Journal of Advanced Computer Science and Applications, 12(8), 2021.
Kenji Kawaguchi. Deep learning without poor local minima. Advances in Neural Information Processing Systems, 29:586-594, 2016.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7:7, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86( 86(11):2278-2324 1998.
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes. In International Conference on Learning Representations, 2018
Xiang Li, Shuo Chen, Xiaolin Hu, and Jian Yang. Understanding the disharmony between dropout and batch normalization by variance shift. In Proceedings of the IEEF/CVF conference com- puter vision and pattern recognition, pp. 2682-2690, 2019.
Andrew L Maas. Awni Y Hannun, Andrew Y Ng, et al. Rectifier nonlinearities improve neural network acoustic models. In International Conference on Machine Learning, volume 30, pp. 3. Citeseer, 2013.
Mor Shpigel Nacson, Jason Lee, Suriya Gunasekar, Pedro Henrique Pamplona Savarese, Nathan Srebro. and Daniel Soudry. Convergence of gradient descent on separable data. The 22nd International Conference Artificial Intelligence and Statistics, pp. 3420-3428. PMLR, 2019.
11

---

Under review as a conference paper at ICLR 2023
Quynh Nguyen. Mahesh Chandra Mukkamala, and Matthias Hein. On the loss landscape of class of deep neural networks with no bad local valleys. arXiv preprint 1rXiv:1809. 2018.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. Sensitivity and generalization in neural networks: an empirical study. In International Conference on Learning Representations, 2018.
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40): 24652-24663, 2020.
Itay Safran and Ohad Shamir. Spurious local minima common in two-layer relu neural networks. In International Conference on Machine Learning, pp.
PMLR, 2018
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynam- ics of learning in deep linear neural networks. arXiv preprint arXiv:1312.611 2013.
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. arXiv preprint 1rXhv:1703.00810 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1506, 2014.
Umut Simsekli, Levent Sagun, and Mert Gurbuzbalaban. A tail-index analysis of stochastic gradient noise in deep neural networks. In International Conference on Machine Learning, pp. 5827-5837. PMLR, 2019.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng. and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language pro- cessing, pp. 1631-1642. 2013.
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The im- plicit bias of gradient descent on separable data. The Journal of Machine Learning Research, 19 (1):2822-287) 2018.
Eli Stevens, Luca Antiga, and Thomas Vichmann. Deep learning with PyTorch. Manning Publica- tions, 2020.
Richard Karl Vogl. Deep Learning Methods for Drum Transcription and Drum Patlern Genera- fion/submitted by Richard Vogl. PhD thesis, Universität Linz, 2018.
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641, 2019.
Tsui- Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness neural networks: An extreme value theory approach. In International Conference on Learning Representations, 2018.
Wolchover. New theory cracks open the black box of deep learning. Quanta Magazine, 3, 2017.
Aolin Xu and Maxim Raginsky. Information -thooretic analysis of generalization capability of learn ing algorithms. Advances in Neural Information Processing Systems, 2017:2525-2534, 2017.
Tengyu Xu, Yi Zhou, Kaiyi Ji, and Yingbin Liang. Convergence of sgd in learning relu models with separable data, arXiv preprint arXiv:1806 04339, 2018.
Zhiqin John Xu. Understanding training and generalization in deep learning by fourier analysis. arXiv preprint 14849918 2018
12

---

Under review as a conference paper at ICLR 2023
A RELATED WORK
Understanding the optimization and the representation capacity of DNNs is an important direction to explain DNNs. The information bottleneck theory (Wolchover 2017 Shwartz-Ziv & Tishby 2017) quantitatively explained the information encoded by features in intermediate layers of DNNs. Xu & Raginsky (2017). Achille & Soatto (2018) and Cheng et al. (2018) used the information bottleneck theory to evaluate and improve the DNN's representation capacity. Arpit et al. (2017) analyzed the representation capacity of DNNs with real training data and noises. In addition, several metries were proposed to measure the generalization capacity or robustness of DNNs, including the stiffness (Fort al. 2019). the sensitivity metrics (Novak al., 2018), the Fourier analysis (Xu 2018), and the CLEVER score (Weng et al., 2018). In comparison, explain the MLP from the perspective of the learning dynamics, i.e., we explain the TFC phenomenon in early iterations the MLP.
Analyzing the learning dynamics is another perspective to understand DNNs. Many studies analyzed the local minima in the optimization landscape of linear networks (Baldi & Hornik 1989 Saxe et al,
2013 Hardt & Ma 20166] Daniely et al., (2016) and nonlinear networks (Choromanska et al. 2013
Kawaguchi| 2016] Safran & Shamir (2018). Some studies discussed the convergence rate of gradient descent separable data (Soudry al. 2018 Xu al., 2018 Nacson et al. 2019). Hoffer et al.
(217)) and Jastrzebski et al. (2017) have investigated the effects of the batch size and the learning rate on SGD dynamics. In addition, some studies analyzed the dynamics of gradient descent in the
overparameterization regime (Arora et al., 2018) Jacot et al. 2018 Lee et al., 2018 Du et al.) 2018)) Besides, (Papyan al.) 2020; Han et al.1 explored the neural collapse phenomenon, which was observed at the end of the training stage. Unlike previous studies, we analyze the learning dynamics of features and weights of the MLP, order to explain the TFC phenomenon in the early training process of the MIP.
13

---

Under review as a conference paper at ICLR 2023
B COMMON PHENOMENON SHARED BY DIFFERENT DNNS FOR DIFFERENT TASKS.
In this section, we aim to demonstrate an interesting phenomenon of the decrease of the feature diversity when we train an MLP in early iterations. Specifically, the training process of the MLP can usually be divided into the following two phases according to the training loss. In the first phase, the training loss does not decrease significantly, and the training loss suddenly begins to decrease the second phase.
The two-phase phenomenon of the training loss well-known, because many previous studies (Sim- sekli et al. 22019; Saxe et al. 2013 Vogl 2018 Nguyen et al. 2018 Arab et al. 2020 Jepkoech et al. 2021] Stevens et al. 2020) have shown this phenomenon during the training process in their papers. However, previous studies did not theoretically explain the emergence of such a phenomenon. In- stead, they usually understood this phenomenon in an intuitive manner, i.e., initialized DNNs failed to find clear optimization direction, and thus these DNNs usually spent long time searching for a reliable optimization direction. In this way, the training loss did not decrease significantly in very early epochs of training.
More crucially, the feature diversity decreases in the first phase. This phenomenon is widely shared by different DNNs with different architectures for different tasks. As Figure Figure 2 and Figure show, the feature diversity keeps decreasing (i.e., the cosine similarity between features of differ- ent samples keeps increasing) until samples of different categories share almost the same feature in the first phase. We can consider this as the temporary feature collapse (TFC). This TFC happens in various DNNs, including multi-layer perceptrons (MLPs), convolutional neural networks, and re- current neural networks. DNNs trained with different loss functions and different learning rates may all exhibit TFC phenomenon. Specifically, we calculated the feature cosine similarity between fifty samples from ten categories the CIFAR-10 dataset, the MNIST dataset, and the Tiny ImageNet dataset. The abseissa and ordinate of each heatmap represent the sample index. For each grid, color indicates the cosine similarity of that sample pair. Note that all the features are extracted after the ReLU layer. Thus, the cosine similarity is always greater than zero
Besides, Figure 1 in the main paper shows, samples from different categories share diverse fea- tures in the beginning of the training, but share almost the same feature at the end of the training. Specifically, we used t-SNE for visualization (initialized by PCA).
Let us take the 9-layer MLP trained on the CIFAR-10 dataset for an example, where each layer of the MLP had 512 neurons. As Figure e shows, before the 1300-th iteration (the first phase), both the feature diversity and the gradient diversity kept decreasing, i.e., both the cosine similarity between features over different samples and the cosine similarity between gradients kept increasing. After the 1300-th iteration (the second phase). the feature diversity and the gradient diversity sud- denly began to increase, i.e. their similarities began to decrease. Therefore, the MLP had the lowest feature diversity and the lowest gradient diversity at around the 1300-th iteration. Specifically, the training loss was evaluated on the whole training set.
Itcration 0
Iteration 900
Iteration 1000
Iteration 3000
Isoration 10000
1.0
0.8
0.6
0.4
33
Figure 1: Cosine similarity between features of different samples on the CIFAR-IO dataset. We trained a 9-layer MLP, where each layer had 512 neurons. The cosine similarity between features of different samples kept increasing until samples of different categories share almost the same feature in the first phase. The features were used in the fourth linear layer of the MLP. The TFC phenemonon happens in the 1000-th iteration. The abscissa and ordinate of each heatmap represent the sample index. For each grid, color indicates the cosine similarity of that sample pair.
14

---

Under review as a conference paper at ICLR 2023
lteration 0
Iteration 500
Iteration 700
Iteration 1000
Ieeation 10000
1.0
0.8
33 25
33 25
0.4
0.2
25 33
Figure 2: Cosine similarity between features of different samples on the MNIST dataset. We trained 9-layee MLP, where each layer had 512 neurons. The cosine similarity between features different samples kept increasing until samples of different categories share almost the same feature in the first phase. The features were used the fourth linear layer of the MLP. The TFC phenemonon happens in the 700-th iteration. The abscissa and ordinate of each heatmap represent the sample index. For each grid, color indicates the cosine similarity of that sample pair.
Itoration
Itenation 2900
Fteration 3000
Ireration 4000
Iteration 10000
53
33
41
25
Figure 3: Cosine similarity between features of different samples on the Tiny ImageNet dataset. We trained a 9-layer MLP, where each layer had 512 neurons. The cosine similarity between features of different samples kept increasing until samples of different categories share almost same feature in the first phase. The features were used in the fourth linear layer of the MLP. The TFC phenemonon happens in the 3000-th iteration. The abscissa and ordinate each heatmap represent the sample index. For each grid, color indicates the cosine similarity of that sample pair.
B. ON THE CIFAR-10 DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by different MLPs on the CIFAR-10 10 dataset (Krizhevsky et al. 2009p. For different MI.Ps, we adopted the learning rate ? = 0.1, the batch size bs 100, the SGD optimizer, and the ReI.U activation function. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained on the CIFAR-10 dataset
are shown Figure
Trining Ascuracy(%) (%)
Training Lows
2.4
Testing Loss
20
5-layer MLP with neurons cach layer
10
(8)
10
10
10
102
100
10
(0)
10
7-layer MLP with neurons in each layer
Testing Accuracy(%)
Cosine simiarity of Fastures
o
Cosine similarity
of gradicats
7-layer MLP with 512 neurons in layer
9-layer MLP with acurons in each layer
0.2
161
102
10a
500
1000
1500
300
1000
1500
iteration
iltration
itcration
(e)
Figure 4: (a) The training loss of four MLPs trained on the CIFAR-10 dataset. (b) The testing loss of four MLPs. (c) Training accuracies of four MLPs. (d) Testing accuracies of four MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the third linear layer of Ms.ss
15

---

Under review as a conference paper at ICLR 2023
B.2 ON THE MNIST DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by different MLPs on the MNIST dataset (LeCun et al. 1998). For different MLPs, we adopted the learning rate 0.01, the batch size bs = 100, the SGD optimizer, and the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained on the MNIST are shown in Figure
Thaining Lows
2.4
Testing Loss
2.4
of gradiennts Training Accuracy
90
1.6
0.8
0.8
S-layer MI. with 128 meurons in each layer
(3)
107
104
10"
(b)
107
10
10
(6)
10
5-layer MLP with 256 neurons each layer
Testin Accunicy"l)
90
Cosine similarity
similarity
features
7-layer MLP with 256 neurons in cach layer
0.9
0.6
30
7-layer MLP with $12 each layer
0.4
0 1000
2000
1000
1000
2000
3006
iteration
iteration
iteration (1)
(d)
(e)
Figure 5: (a) The training loss of four MLPs tranined on the MNIST dataset. (b) The testing loss of four MLPs. (c) Training accuracies of four MLPs. (d) Testing accuracies of four MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the third linear layer of MLPs.
B.3 ON THE TINY IMAGENET DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by different MLPs on the Tiny ImageNet dataset (Le & Yang 2015). Specifically, we randomly selected the follow- ing 50 categories, orangutan, parking meter, snorkel, American alligator, oboe, basketball, rocking chair, hopper, neck brace, candy store, broom, seashore, sewing machine, sunglasses, panda, pret- zel. pig, volleyball, puma, alp, barbershop, ox, flagpole, lifeboat, teapot, walking stick, brain coral, slug, abacus, comic book, CD player, school bus, banister, bathtub, German shepherd, black stork, computer keyboard, tarantula, sock, Arabian camel, bee, cockroach, cannon, tractor, cardigan, sus- pension bridge, beer bottle, viaduct, guacamole, and iPod for training. For different MLPs, adopted the learning rate 0.1, the batch size bs = 100, the SGD optimizer, and the ReLU acti- vation function. Besides, used data augmentation methods, including random cropping and random horizontal flipping. Note that we took random cropping with 32x32 sizes. training loss, the testing loss, the training accuracy, the testing accuracy. the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained on the Tiny ImageNet are shown in Figure
B.4 ON THE CENSUS DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by different MLPs on the UCI census income tabular dataset (Census) (Asuncion & Newman 2007). For different MLPs, we adopted the learning rate = 0.1, the batch size bs 1000, the SGD optimizer, and the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, cosine similarity features, and the cosine similarity of feature gradients of MLPs trained on the census shown in Figure
B.5 ON THE COMMERCIAL DATASET
In this subsection, demonstrated that the two-phase phenomenon was shared by different MLPs on the UCI TV news channel commercial detection dataset (Commercial) (Asuncion & Newman 2007). For different MI.Ps, adopted the learning rate n 0.1, the batch size bs 1000, the
16

---

Under review as a conference paper at ICLR 2023
Training Loss
4.0
Testing Lows
4.0
Twining Acourcyy
60
3.5
20
5-Inyer MLP
each layer 7-layer MLP with neurons in each layer
10
10
10
10
10P
10
10F
102
107
10*
10
(8)
(b)
(6)
Testing Accurscy
30
Coeine similarity
0.8
Cosine similarity of gradictts
9-layer MLP with 512 neurons in layer
of l'features
20
0.
0.4
0.2
102
100
10*
10
2500
5000
2500
5000
(d)
Figure 6: (a) The training loss of three MLPs tranined on the Tiny ImageNet dataset. (b) The testing loss of three MLPs. (c) Training accuracies three MLPs. (d) Testing accuracies of three MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The features and the feature gradient were used in second linear layer of MLPs.
0.0
Testing Loss
Training Accuracys%)
100
Training
60
8-layer MLP with 128 neurons each layer
0.4
200
600
800
1000
200
600
800
1800
(b)
9-layer MT.P with
Tesing Accuracy
1.00
Cosize similatity
1.0
OorT similasty
0.91
neurons each layer 10-layer MLP with 128 neurons each layer
0.8
of feanures
0.7
0.6
80
0.7:
9.3
0.6
1000
0.3
600
200
400
000
500
iteration
Eteration (e)
iteration
(8)
Figure 7: (a) The training loss of three MIPs trained on the Census dataset. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing accuracies of three MLPs. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the fifth linear layer of MLPs.
SGD optimizer, and the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients MLPs trained on the census are shown in Figure
Cosine similarity Tmining Accuncy(%)
Training Loss
Teating Loes
80
0.6
0.6
7-layer MLP with 128 neurous in each layer
2000
400
600
800
1000
0
700
400
600
800
1000
200
400
6GG RIO 1800
(5)
(b)
8-layer MLP with 128 neurons layer
Testin Accuracy
100
Cosine similrity
0.9
0
gradients
9- neurons in each layer
0.7
60
0.5
800 1000 ileration
400 iteratioon
400 800
ileratiion
(d)
(0)
Figure 8: (a) The training loss of three MLPs trained the Commercial dataset. (b) The testing loss of three MLPs. (c) Training accuracies of three MIPs. (d) Testing accuracies of three MIPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the fifth linear layer of MLPs.
17

---

Under review as a conference paper at ICLR 2023
B.6 ON THE COLA DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by the revised LSTMs on the CoLA dataset (Warstadt al., 2019). We used two-layer unidirectional LSTMs concatenated with MLPs. Specifically, we trained two LSTMs with 5-layer MLPs, where each layer of the MLP had 256 and 512 neurons. We adopted the learning rate 7) 0.1, the batch size bs = 1000, the SGD optimizer, and the ReLU activation function. The training loss, the testing loss, the training accuracy,
testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of LSTMs trained on the CoLA are shown in Figure Since training samples in the CoLA dataset were imbalanced, we constructed a new training set by randomly sampling 2000 training samples from two categories, respectively, DNNs were trained on this new training set.
Training Accurscy%%
Training Loss
Texting Loos
0
56
0.68
6.67
2-layer LSM,, S-layer MI.P with 128 each
1375
2750
4125
3500
1375
2750
4125
3500
(6)
Testing Accuracy50)
64
68
Cosine similurit
of faatures
Cosime similanity
of gradients
0.
2-layer LSTM, S-layer MLP 256 each layer
0.92
8.85
56
0.80
iteration
iteration
iteration
(d)
(c)
(1
Figure 9: (a) The training loss of two LSTMs trained on the CoLA dataset. (b) The testing loss of two LSTMs. (c) Training accuracies of two LSTMs. (d) Testing accuracies of LSTMs. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the third linear layer of MLPs.
B.7 ON THE SST-2 DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by the revised LSTMs on the SST-2 dataset (Socher et al. 2013). We used unidirectional LSTMs concatenated with MLPs. Specifically, we trained three LSTMs with 4-layer MLPs, 4-layer MLPs, and 5- MLPs, respectively, where each layer of the MLP had 32, 64, 128 neurons. We adopted the learning rate 1 = 0.1, the batch size by 500, the SGD optimizer, and the ReLU activation function. Since the training of LSTMs on the SST-2 with the SGD optimizer is unstable, we randomly selected 15000 training samples from the training set. We trained LSTMs on these 15000 training samples. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of LSTMs trained on the SST-2 are shown in Figure 10
B.8 ON THE AGNEWS DATASET
In this subsection, we demonstrated that the two-phase phenomenon was shared by the revised LSTMs on the AGNEWS dataset. We used two-layer unidirectional LSTMs concatenated with MLPs. Specifically, we trained three LSTMs with 4-layer MLPs, 4-layer MLPs, 5-layer MLPs, respectively, where each layer of the MLP had 32, 64, and 128 neurons, respectively. We adopted the learning rate 77 = 0.1, the batch size bs 500, the SGD optimizer, and the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of LSTMs trained on the AGNEWS are shown in Figure
B.9 DIFFERENT TRAINING BATCH SIZES
In this subsection, demonstrated that the two-phase phenomenon shared by MLPs trained on the CIFAR-10 10 dataset with different training batch sizes. For different MLPs, we adopted the learning 7 0.1, the SGD optimizer, and the ReL.U activation function. Besides, used
18

---

Under review as a conference paper at ICLR 2023
Training Loss
12
Testing Loss
1.2
Cooine similarity Training Accurscy
100
0.9
0.9
0.6
40
1-Aayer LSTM, 4-aayer
2000
4000
5000
2000
2000
4000
00W
2000
20
2000
4000 2000 (c)
8000
with 52 ncurons in cach layer
(8)
(6)
2-layer LSTM, 4-layer MLP with 64 nourons each layer
Testing Accuracy(55)
100-
Cosine simileriyy
1.0
of Reatures
0.0
gradients
3-layer LSTM, S-layer MLP neurons layer
20
0.4
2000
2000
fteration
iteration
sterarion
(d)
(c)
(
Figure 10: (a) The training loss of three LSTMs trained on the SST-2 dataset. (b) The testing loss of three LSTMs. (c) Training accuracies three LSTMs. (d) Testing accuracies three LSTMs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of
different samples in category. The feature and the feature gradient were used the second linear layer of MLPs.
Training Accuracy (%)
Training Loss
1.
100
80
Testing
0.9
60
2-layer LSTM, 4-layer MLP with 32 neurons each layer
2000
4000
8000
8000
2002
6000
2000
4005
6000
8000
2-1ayer LSTM, 4-Jayer MLP
Testing AccunacyS0
100-
Cosine similarity of Eeatures
Casine similarty
1.0
layer
0.8
uradieens
2-layer LSTM, 5-layer MI neurons each layer
6.
20
0.4)
ierraton
(c)
(0)
Figure 11: (a) The training loss of three LSTMs trained the AGNEWS dataset. (b) The testing loss of three LSTMs. (c) Training accuracies of three LSTMs. (d) Testing accuracies of three LSTMs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the second linear layer of MLPs.
two data augmentation methods, including random cropping and random horizontal flipping. We trained three 7-layer MLPs with 256 neurons in each layer, with bs 100, 500, 1000 respectively. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained with different batch sizes are shown in Figure T2
Training Accuracy(%5
1.8
1.8
Training
40
1.2
0.6
102
10
10
10'
10
10
0
10a
10'
batch size 100
butch size 500
Testing Accuracy*))
80
Cosine simmlurit
Cosine similarity
batch size 1000
60
of fealures
of gratieens
0.6
0.6
20
0.5
0.2
10/
10a
10
0
250
500
750
1000
250
500
750
1000
iteration
iteration
iteration (
(d)
(e)
Figure 12: (a) The training loss of three MLPs trained with different batch sizes. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing accuracies of three MLPs. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the second linear layer of MI.Ps.
19

---

Under review as a conference paper at ICLR 2023
B.10 DIFFERENT LEARNING RATES
In this subsection, we demonstrated that the two-phase phenomenon was shared by MLPs trained on the CIFAR-10 dataset with different learning rates. For different MLPs, adopted the batch size bs = 100, the SGD optimizer, and the ReLU activation function. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. We trained two 7-layer MLPs with 256 neurons in each layer, with learning rates = 0.1, 0.01 respectively. The training loss, the testing loss, the training accuracy, the testing accuracy. the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained with different learning rates are shown Figure
Training Accuracy (%%
Trainin Loss
1.0
1.8
Testing
40
1.2
1.2
10'
104
102
10
104
0
10
10a
101
(8)
(b)
(c)
learning
Testing Accuracy!5)
60
Cosine simil aiiianty of features
Cosine simuurrty of gradients
0.0
learning tale 0.01
0.6
40
0.4
10a
2000 iteration
iteration
iteration
(e)
Figure 13: (a) The training loss of two MLPs trained with different learning rates. (b) The testing loss of two MLPs. (c) The training accuracies of two MIPs. (d) The testing accuracies of two MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the second linear layer of MLPs.
B.11 DIFFERENT ACTIVATION FUNCTIONS
In this subsection, we demonstrated that the two-phase phenomenon was shared by MLPs with different activation functions. For different MLPs, adopted the learning rate 7 0.1, the batch size bs = 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. We trained three 9-layer ML.ss with 512 neurons in each layer with the ReLU activation function, the Leaky ReLUU (slope=0.1) activation function, and the Leaky ReLU (slope=0.01) activation function, respectively. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained with different activation functions are shown in Figure 14
Thaining Loss
2.4
Tosking Lons
2.4
Training Accuracy
2.1
2.1
MLP with ReLU
(4)
(b)
2000 4000
8050 (6)
MLP with LeakyReLU (slope-0.01)
Testing Accuracy(5))
60
similarity
of faaures
Coliti similarily
gradieets
MLP with LeskyReLU (slope-0.1)
29
2000
50D0
8000
2000
300
1030
2000
iteration
Stemiion
iteration
(8)
(e)
Figure 14: (a) The training loss of three MLPs with different activation functions. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing accuracies of three MLPs. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the second linear layer of MIPs.
20

---

Under review as a conference paper at ICLR 2023
Trining Loss
2.2
Loss
2
Training Arcuracy
1.8
30
7-layer MLP with 512 nourous in each layer,
20:
500
1000
1500
2000
500
1000
1500
2000
10
500
1000
1500
2000
(8)
(b)
(0)
7-layer MLP with 512 neurons in each layer, momentum 0.5 7-layer
Texting Accuracy
50
Cosine similucit
0.6
Cosine similarity
of graaiienss
30
0.0
0.4
neurons in each layer, momentum 0.9
20
3..3
A (e)
22
10
iteration
steration
iteration
(8)
Figure 15: (a) The training loss of three MLPs with different momentums trained the CIFAR-I0 dataset. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing
accuracies of three MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient
were used the second linear layer of MLPs.
Training Accuncyy%)
Training Loss
Texting Loss
90
1.8
7-layer with each layer,
0.0
500
1000
1500
2000
0.0
500
1000
1500
2000
500
1000 (c)
1500
2000
momentum 0
(8)
(b)
7-layer MI with 512 each layer,
Testing Accuacy(%)
Cosine sinilacity
Cosine similarity
of gradientts
0.5
of features
0.5
0.6
7-lnyer MI. with 512 neurons each layer, momentum 0.9
30
1000
2000
500
iteration
iteration (A
(d)
Figure 16: (a) The training loss of three MLPs with different momentums trained on the MNIST datasets. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing
accuracies of three MLPs. (e) Cosine similarity between features different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used the second linear layer of MLPs.
B.12 DIFFERENT MOMENTUMS
In this subsection, we demonstrated that the two-phase phenomenon was shared by ML.Ps trained on the CIFAR-10, MNIST and Tiny ImageNet dataset with different momentums. For different MLPs, we adopted the learning rate = 0.1, the batch size bs = 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. We trained 7-layer MLPs and 9-layer MLPs with 512 neurons in each layer with the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained with different momentum shown in Figure 15 respectively.
Figure T6 Figure 17, Figure 18 Figure T9 and Figure 20)
1. DIFFERENT WEIGHT DECAYS
In this subsection, we demonstrated that the two-phase phenomenon was shared by MLPs trained on the CIFAR- 10, MNIST and Tiny ImageNet dataset with different weight decays. For different MLPs, we adopted the learning rate 0.1. the batch size bs 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. We trained 7-layer MLPs and l-yer MLPs with 512 neurons in each layer with the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of ML.Ps trained with different
21

---

Under review as a conference paper at ICLR 2023
Training Accuracy(5)
Training
3.6-
Testing
3.6
3.4
7-layer MLP with 512
3.20
1250
2500
3750
5000
3.2
1250
2500
3750
5000
1250
2500 (c)
3750
5000
neurons in each layer.
momenium 0
(a)
(b)
7-layer MLP with 512 neurons
Testing Avcuracy(
layer,
15
Cosine similarity
of featurng Rature
0.6-
Cosine similarity
of graadients
0.6.
momentum 0.5 7-layer MLP with 512 neurons in each layer, momentum 0.9
10
0.4
0.4
5
0.2
5000
2500
5000
2500
5000
iteration
iteration
iteration
(d)
(c)
(1)
Figure 17: (a) The training loss of three MLPs with different momentums trained on the Tiny Im-
ageNet dataset. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d)
Tiesting accuracies of three MLPs. (e) Cosine similarity between features of different categories. (f)
Cosine similarity between gradients of different samples in a category. The feature and the feature
gradient were used in the fourth linear layer of MLPs.
of gradionts Training (isuumyyy
Training Loss
Tesing Loss
2.2
40
30
1.8
20
9-layer MLP with 512 neurons in cach layer,
10
1000
1500
2000
500
1000
1500
2000
momentum 9-layer MLP with in each layer, mementum 0.5 9-layer MLP with 512 neurons in each layer, momentum
(8)
(b)
(6)
Testing Accuracy(5)
40
Com y
Cosine similarity
Cosine similarity
30
features
0.4:
20
0.2
101
602
2000
itcration
iteration
iteration
(d)
(c)
(f)
Figure 18: (a) The training loss of three MLPs with different momentums trained on the CIFAR- 10 dataset. (b) The testing loss of three MLPs. (c) Training accuracies of three MLPs. (d) Testing
accuracies of three MLPs. (e) Cosine similarity between features different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used the second linear layer of MLPs.
weight decays are shown in Figure respectively.
Figure
Figure
23
Figure
24
Figure 25 and Figure 26
22

---

Under review as a conference paper at ICLR 2023
24
Training (ccurncy"%)
90
1.8
1.8
Training
1.2
Testngg
60
9-layer each layer, momentum 0
0.6
30
300
1000
1500
2000
500
1000
1500
2000
0
500
1000
1500
7000
Testing Accuracy(%)
%
Cosine similarity
Cusine similuity
0.8
9-layer MLP with 512 neurons each layer,
8
of features
0.6
momentum 0.5
0.5
500 1000
1500
2000
1000
2000
0.2 1000 2000
iteration
ileration
iteration (1)
(d)
(e)
Figure 19: (a) The training loss of two MLPs with different momentums trained on the MNIST
dataset. (b) The testing loss of two MI.Ps. (c) Training accuracies of two MIPs. (d) Testing
accuracies of two MLPs. (e) Cosine similarity between features of different categories. (f) Cosine
similarity between gradients of different samples category. The feature and the feature gradient were used in the second linear layer of MLPs.
Training Ascurnayyos
Tminnng Loss
Testing Loss
9-layer MTP with 512 neurons in each layer, momentum 9-layer neurons each layer, momentum 0.5 9-layer MLP with 512 neurons in each layer, 0.9
1256 2500 3750
5000
3.4
1250 2500 3750
5000
1250
2500 3750 5000
(8)
(b)
(c)
Testing Accuracy())
Cosine similarity of features о o
Cosine simmluriy of I'grndionts
10
2500 3750 s000
2500
5000
iteration
(i)
Figure 20: (a) The training loss of three MLPs with different momentums trained on the Tiny Im- ageNet dataset. (b) The testing loss of three MIPs. (c) Training accuracies of three MIPs. (d)
Testing accuracies of three MLPs. (c) Cosine similarity between features of different categories. (f)
Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the fourth linear layer of MIPs.
Training Loss
22
Testng Loss
2.2
Cosime similurity Triniin Acouracy
5o
2.0
1.8
7-layer MLP with 512 neurons in each layer, weight decay 0.001
500
1000
1500
2000
0
500
1000
1500
2000
0
530
1000
1500
7000
(b)
Texting (ocuracy(%)
50
Casine cimilarity
0.6
of Deatures
of gradients
7-layer neurons each layer, weight decay 0.0001
30
0.5
0.4
20
0.4
10
500
1000
1500
2000
1000 iteration
2000
0.2
1000 iteration
2000
iteration
(d)
(c)
(f)
Figure 21: (a) The training loss of two MLPs with different weight decays trained on the CIFAR- 10 dataset. (b) The testing loss of two MLPs. (c) Training accuracies of two MLPs. (d) Testing
accuracies of two MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used the second linear layer of MI.ss.
23

---

Under review as a conference paper at ICLR 2023
Training Loss
Tosting Loss
Cosine similarity Tmining Accuracy(50
90
1.2
1.2
7-layer MLP with 512 layer, weight decay 0.001
0.0
500
1000
1500
2000
0.0
500
1000
1500
2000
1000
2000
Testing Accuracy1%)
40
similuuiy
0.7-
of gradients
0.8-
7-layer MLP with 512 neurons in cach layer. weight decay 0.0001
fo
30:
0.4.
500
1000
1500
2000
1000
2000
1000
2000
iteration
iteration
iteration (9)
(d)
(e)
Figure 22: (a) The training loss of two MLPs with different weight decays trained on the MNIST
dataset. (b) The testing loss of two MLPs. (c) Training accuracies of two MLPs. (d) Testing
accuracies of two MIPs. (e) Cosine similarity between features of different categories. (f) Cosine
similarity between gradients of different samples category. The feature and the feature gradient were used in the second linear layer of MLPs.
Training Loss
Testing Loss
Training Avquacyy%)
15'
10
3.6
7-layer MLP with neurons each layer. weight decay 0.0001
1230
2500
3750
5000
0
1250
2300
3750
5000
1250
3750 (6)
5000
(4)
(b)
Testing Accuracy
Cosine similantty
Cosine similurty
of features 80.7
of gradletts
0.e
7-layer MI.P with 512 neurons each layer, weight decay 0,00001
20
8.66
0.4
1250 2500 3750
5000
7400
4500
7440
4800
iteration
iteration (c)
iteration (f)
(d)
Figure 23: (a) The training loss of two MLPs with different weight decays trained the Tiny ImageNet dataset. (b) The testing loss of two MIPs. (c) Training accuracies of two MI.ss. (d)
Testing accuracies two MLPs. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the third linear layer of MLPs.
Training Loss
Tosting Loss
2.
Training Accuracy
2.0
266
1.6
9-layer neurons each layer, weight docay 0.001
2500
(8)
(c)
Testing Accumec (7)
0.7
0.6
40
Cosime similuiity
of fffatturess
A Rah
Cosine similarity
of gadients
0.6
la MLP with 512 neurons in each layer. weight decay 0.0001
0.4
20
0.2
1250
2500
3750
5000
2000
10000
itcration
itcration
itcration (f)
(d)
(c)
Figure 24: (a) The training loss of two MLPs with different weight decays trained on the CIFAR- 10 dataset. (b) The testing loss of two MLPs. (c) Training accuracies of two MLPs. (d) Testing
accuracies of two MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used the second linear layer of MI.Ps.
24

---

Under review as a conference paper at ICLR 2023
Ttaining Loss
Texting Loss
2.4
Training Accuracy
1.8
18
60
1.2
0.6
9-layer MLP with neurons in each layer, weight docay = 0.001
0
500
1000
1500
2000
500
1000
1500
2000
0
500
1000
1500
2000
Tosting Accuracyt5)
90
Cosine similarity
Cosine similarity
of fogtures
7.7
gadisag
9-layer MLP with 512 layer. weight decay 0.0001
80
3.
500
1006
1500
2000
1000 iteration
2000
1000 iteration
2000
itoration
(d)
(c)
Figure 25: (a) The training loss of two MLPs with different weight decays trained on the MNIST
dataset. (b) The testing loss of MLPs. (c) Training accuracies of two MLPs. (d) Testing
accuracies of two MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients different samples category. The feature and the feature gradient were used in the second linear layer of MLPs.
Thining Accuncy (36)
Triining Loss
3.9
Testing
3.8
3.7
9-layer MI. MI.Po with 512 neuross each layer, weight docay 0.0001
1250
2500
5000
Texting Accurayy56)
0.8
15
Cosine simlarity
0.68
Cosinc similarity
of features
AffoC
grddents
0.6
9-layer MLP 512 neurons each layer, weight decay 0.00001
0.64
0.62
www.
0.2
1250
2500
5750
3000
2000
4000
itcration
itcration
itcration
(d)
(c)
(9)
Figure 26: (a) The training loss of two MLPs with different weight decays trained on the Tiny
ImageNet dataset. (b) The testing loss of two MLPs. (c) Training accuracies of two MLPs. (d)
Testing accuracies of two MLPs. (c) Cosine similarity between features of different categories. (f)
Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the third linear layer of MLPs.
25

---

Under review as a conference paper at ICLR 2023
B.14 THE TFC PHENOMENON WITH THE FOCAL LOSS
In this subsection, we demonstrated that the two-phase phenomenon was shared by MLPs learned the CIFAR-10 dataset with the focal loss. Specifically, for different MLPs, adopted the learning rate 1/ 0.1, the batch size bs = 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizomtal flipping. We trained 9-layer MLPs and 7-layer MLPs with 512 neurons in each layer with the ReLU activation function. The training loss, the testing loss, the training accuracy, the testing aceuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained with different focusing parameters are shown in Figure 27 and Figure 28 Figure 27 and Figure 28 show that TFC phenomenon was still observed by different MLPs with the focal loss on the CIFAR-10 dataset.
2.25
2.25
Training Accurucy (%)
50
Training Loss
Testing Loss
L:75
40
1.25
20
0.75
0.75
10
0.0
10
10
10'
(8)
(b)
(c)
3.0
Testing Accuracy (%)
0.7
Cosine similarity of Teatures
Cosine similurity
0.7
0.6
of gradients
= 5.0
0.5
0.5
10
0.4
0.3
10
10
3000
3000
iteration
iteration
iteration
(d)
(c)
(f)
Figure 27: (a) The training loss of three 7-1ayer MLPs with different focusing parameters trained on the CIFAR-10 dataset.
(b) The testing loss of three MLPs. (c) Training accuracies three MLPs. (d) Testing accuracies of three MLPs. (e) Cosine similarity between features of different
categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the third linear layer of MLPs.
2.25
2.25
Training Loss
1.75
Training Accuracy
30
1.25
Testing
10
= 0.0
10
10
10'
10
10"
10"
10'
10
(a)
(b)
(c)
3,0
Testing Accuracy (%)
40
Cosine similarity of features
0.7
Cosine similarity
0.89
= 5.0
of gradients
0.6
20
0.5
0.4
10'
0.4
0.2
ileration
ileration
ileration
(d)
(+)
Figure 28: (a) The training loss of three 9-layer MLPs with different focusing parameters trained on the CIFAR-10 dataset.
(b) The testing loss of three MLPs. (c) Training accuracies of three
MLPs. (d) Testing accuracies of three MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the third linear layer of MLPs.
26

---

Under review as a conference paper at ICLR 2023
B.15 DIFFERENT TRAIN/TEST SPLIT FOR DATASETS
In this subsection, we demonstrated that the two-phase phenomenon was shared by MLPs trained the CIFAR-I0 dataset with different train/test splits. There are 50000 samples in the training and 10000 samples in the testing set on the CIFAR-10 dataset. We combined the training set and the testing set into one dataset and split with the train/test split ratios of 5:1, 4:2, and 3:3, respectively. Note that the ratio of 5:1 was the official ratio for the CIFAR-10 dataset. For different MLPs, we adopted the learning rate r) 0.1, the batch size bs 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. We trained 9-layer MLPs with 512 neurons each layer with the ReLU activation function on these three different datasets. The training loss, the testing loss, the training accuracy, the testing accuracy, the cosine similarity of features, and the cosine similarity of feature gradients of MLPs trained on different train/test split ratios are shown in Figure 29 Figure 29 shows that the TFC phenomenon was still observed by different train/test split ratios.
Training Accuracy (%)
2.2
Training Loss
2.0
Testing Loss
40
1.8
20
1.6
1.6
1.4
14
10'
10
(a)
(b)
(c)
4:2
Testing Accuracy (%)
Cosine similarity of features
0.7
Cosine similarity of gradients
0.8
40-
30
0.6
20
10
10
1500 iteration
3000
1.000 itcration (f)
3000
iteration (d)
(c)
Figure 29: (a) The training loss of three MLPs with different train/test dataset split ratios trained on
the CIFAR- 10 dataset. (b) The testing loss of three MIPs. (c) Training accuracies of three ML.Ps. (d)
Testing accuracies of three MLPs. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in a category. The feature and the feature gradient were used in the third linear layer of MLPs.
27

---

Under review as a conference paper at ICLR 2023
C DISCUSSION OF THE PRACTICAL VALUES: THE LEARNING-STICKING PROBLEM
In this section, we aim to discuss the learning-sticking problem in the learning of MLPs. In fact, this problem appears in various DNNs, including MLPs, CNNs, and RNNs, when the task is difficult enough. Explaining and solving the occasional sticking of the training of DNNs values on different tasks. We consider the carning-sticking problem as the first phase with an infinite length. Moreover, we theoretically explain mechanisms of several heuristic solutions the learning-sticking problem.
of significant
To this end, the learning-sticking problem can be solved based on our study, as shown in Figure 30 Figure 31 Figure 32, Figure 33 Figure [4 and Figure 35 Specifically, we trained a 9- layer MLP on the CIFAR-10 dataset, where each layer of the MLP had 512 neurons and its initial weights were sample from NO = mo T). ow was computed following (Glorot & Bengio, 2010) and "1 = 0.1. We trained a VGG-11 model on the CIFAR-10 dataset and its initial weights of fully connected layers were sample from (V T (+ = 0.1). We trained VGG-13 model on the CIFAR-10 10 dataset and its initial weights of fully connected layers were sample from E /) (n 0.1). We trained two ResNet-18 18 models (without BN layers) on the CIFAR- 10 dataset and the Tiny ImageNet dataset, respectively, and initial weights of fully connected layers were sample from (o ( (yi 0.1).
We observed that these DNNs all suffered from the Searinngsttckingg problem (i.e., the loss mini- mization of these DNNs get stuck), when their initial weights were sampled from (0 E = m (orange curves). According to our study, the technique of increasing the variance of initial weights can shorten the first phase, thereby solving the learning-sticking problem. To this end, we trained compared versions of these DNNs, and the only difference from previous DNNs is that the variance of initial weights was increased to yo (72 1). Figure 30 Figure 31 Figure 32 Figure 33
Figure 34 and Figure 335 verify that we could solve the Bearning-ttccking problem by increasing the variance of initialization.
Actually, far beyond solving the dearning-stickiinn problem, the two-phase phenomenon of MLPs is generally considered a counter-intuitive phenomenon. In this paper, our distinctive contribution to explain the counter-intuitive two-phase phenomenon of MLPs theoretically.
Thaining Loxs
2.4
loss minimization
Training Accuracyin p
0.6
Testing Loss
2.4
Testing Accuracy
0.6
Y1
gets stuck
0.4
2.0
0.4
72 1.0
the first phuse
0.2
1.6
0.2
9-layer MLPs
Trained
1.2
10000
0.0
10000
10000
0.0
10000
CIFAR-10
iteration (6)
iteration (b)
iteration (C)
iteration (d)
Figure 30: (a) The training loss of two MLPs trained on the CIFAR-10 dataset. When the loss minimization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the "learning-sticking" problem can be solved by techniques of shortening the first phase, such as the technique of increasing the variance of initial weights, which is a theoretically certificated solution in our study (blue curve).
(b) The training accuracy of two MLPs. (c) The testing loss of two MLPs. (d) The testing accuracy of two MLPs.
Trokenng Aucunay (16]
Aanaoo Acmsssyy 1961
24
100
ges stuce
Loos
-
rist phaso
Triiiing
ls sooten
30
Tostan
9- MLPs traindd on MNIST
2535 iuuriton
3000
1500 iteration
3099
0.0
2500 iuunioon (c)
iturssion
Figure 31: (a) The training loss of two MLPs trained on the MNIST dataset. When the loss min- imization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the Wleaning-stiikingg problem can be solved by techniques of shortening the first phase, such as the technique of increasing the variance of initial weights, which is a theoretically certifi- cated solution in our study (blue curve). (b) The training accuracy of two MLPs. (c) The testing loss of two MLPs. (d) The testing accuracy of two MLPs.
28

---

Under review as a conference paper at ICLR 2023
Tuukkkr Acoasey (%6)
Teeting Acnuncy [1)
mannitaii get suck
Testing Looe
Training
1.2
plac
1.2
30
VGG-11 CIBAR
0.1
2500 Decation
500g
2506 Decation (b)
5000
1500 Recation
5993
2596 lteestion
5000
(4)
Figure 32: (a) The training loss of two VGG-11 models trained on the CIFAR- 10 dataset. When the loss minimization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the "learning-sticking problem can be solved by techniques of shortening the first phase, such as the technique of increasing the variance of initial weights, which a theoretically certificated solution in our study (blue curve). (b) The training accuracy of two VGG-11 1 models. (c) The testing loss of two VGG-11 models. (d) The testing accuracy of two VGG-11 models.
24
2.4
Accrry (80
100
Thining Loos
loss micimization gets
Tosting Less
1.0
y 0.1
phose Shortee
VGG-13
Imined on
CIFAB-11
2500
2300 Dtecalion (b)
5000
Deraation (x)
Decaaionn (c)
Iteraion
(d)
Figure 33: (a) The training loss of VGG-13 models trained on the CIFAR-10 10 dataset. When the loss minimization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the "learning-sticking" problem first phase, such as the technique of increasing the variance of initial weights, which is a theoretically certificated solution in our study (blue curve). (b) The training accuracy of two VGG-13 models. (c)
be solved by techniques of shortening the
The testing loss of two VGG-13 13 models. (d) The testing accuracy of two VGG-13 13 models.
Dunnin austunn 199)
Testing Accunay 151
Teemnng Loss
2.0
loss minimizalion
1.0
gets stack
the flrst phase
Tooloy
y
shorten
5 36
ResNet 18 Iniized
CIFA
5000
5000 ilrraion (b)
10000
iluration
ilertiion
ileralion
(c)
(4)
Figure 34: (a) The training loss of two ResNet-18 models trained on the CIFAR- 10 dataset. When the loss minimization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the "learning-sticke" first phase, such as the technique of increasing the variance of initial weights, which a theoretically certificated solution in study (blue curve). (b) The training accuracy of two ResNet- 18 models. (c) The testing loss of two ResNet-18 models. (d) The testing accuracy of two ResNet- 18 models.
problem be solved by techniques of shortening the
Aann Kemanayy [S]
Joes mintmttzabon gets pluse
15 shorton
= 0.1
15
ResNet trined on Tiny ImageNet
1933
3500
1000
1985e
19550
itembtion
iteration (b)
iteration (C)
itematon (6)
Figure 35: (a) The training loss of two ResNet-18 models trained on the Tiny ImageNet dataset. When the loss minimization gets stuck (orange curve), we can consider it as the first phase with an infinite length. Therefore, the searming-stickingg problem can be solved by techniques of shortening the first phase, such the technique of increasing the variance of initial weights, which is a theo- retically certificated solution our study (blue curve). (b) The training accuracy of two ResNet- models. (c) The testing loss of two ResNet-18 18 models. (d) The testing accuracy of two ResNet-18 models.
29

---

Under review as a conference paper at ICLR 2023
D MORE RESULTS ON OTHER DATASETS
In this section, we provide more results on the MNIST dataset and the Tiny ImageNet dataset. Figure and Table empirically verify the strength of the primary common direction, which are supplementary to Figure 4 and Table I in the main paper, respectively. Figure 37 illustrates the change of 0 = cos ( Fl ((=-))))))))) cos(V))) (F in the first phase, which is supplementary to Figure in the main paper.
Canagoey
Layer
Layer
Layer4
Layer
Layer
Calegory
Layer
Loyer
Layer 4
Layer
Layer
lo-3
le3
lec
le-
les
Steength
Flag pole
19
Eight
@a
0.0
2,3
2.0
2.7
3.
le.3
0.0
1c3
2.2
0.0
le:3
2.88
0.0
0.0
3.7
Stren gth
Boilie
0.0
12345 0.0001131
12345 dirocttin
6013345 60345 10343
0.0
237300
12345
0.01
0.0
12315
direction
direction
direction
direction
direction
direction
direction
direction
direction
Figure 36: The strength of top-ranked common directions the (a) MNIST dataset and the (b) Tiny ImageNet dataset. We trained a 9-layer MIP, where each layer of the MLP had 512 neurons. We computed the strength of common directions on the two categories with the highest training accu- racies. 8 = |IC,AV;
: ||r measures the strength of weight changes along the i-th common direction, where AV; Te ..)- It can be observed that the strength of the primary direction was much greater than the strength of other directions.
Table 1: Strength of components of weight changes along the primary common direction and other directions. We trained 9-layer MIP on the MNIST dataset. Fach layer of the MLP had 512 neurons. It can be observed that the strength of the primary common direction was much greater than those of other directions.
Category Sicia
Eight
Zoro
Layer
Layer
Layer
Layer
Layer
Layer
Layer 3
Layer 4
Layer
Layer 6
MINST
Spienc
367.1
364.5gr2a
381.9
4444
506.0
41.7
448.2
120.0
198.
300m
5/
14.9
15.9
15.5aa
15.621
135g20
24.6aa
30.0448
1844za
17.2gz
156611
82
13.1
16.4gas
18.1a8
11.7110
23.9
17.0
12.21
15.1
16.3g1
12.5
15.1g14
15.0g1
29.4gsa
21.
15.5
21.2a1
14.7
Category
Layer 2
Layer 3
Layer
Layer
Layer 6
0.7
0.7
0.7
0.7
Eight
0.6
0.5
0.6
0.6
0.6
0.6
0.4
0.5
0.5
0.5
0.6
0.7
0.7
0.7
0.7
Zero
0.6
0.6
0.5
0.5
0.6
800
900
800
900
800
900
800
900
800
900
iteration
iteration
iteration
iteration
iteration
Figure 37: The change of ( 9-layer MLP on the MNIST dataset. Each layer of the MLP had 512 neurons. The shade represents standard deviation over different samples.
(AV r(("))
cos( cos(v (
in the first phase. We trained
30

---

Under review as a conference paper at ICLR 2023
E PROOF FOR THE LEMMA
In this section, we present the detailed proof for Lemma 1.
Lemma For the decomposition AW; AV.C Aec, given weight changes over different sampley AW, can compute the common direction C by minimizing the fitting error Dec when we use Aer,CC to approximate Awe, over different samples across different iterations. Le., minc,avle (BnglTaan:Tal Beex ||Ase|alle)) AV AW, and = AW. AW, CC CTC
Deelz AW, la AVil=C.. Thus, oblain
C S.t. Ae,C = 0. Such settings minimize ||
proof. Let de, [ denote the j-th column of the matrix Acr E Raxd Given a sample æ, we can represent A [j] by the vector C and residual term A [i] as follows:
E [ = AC + A D
(1)
where C*
CT De, D1' = O, and is a scalar.
Then,
Ae bill2 = |ac + Aet 1i'l
(AC de; b1') TAC + AcTLI)
=xCT (As, (b)') A
=x2CT | A 11"
Obviously, | Lill the smallest when A 0. other words, As [s] does not contain the
component along the direction C and CT Ae [ = Therefore, - [ reaches minimum if and only if Ae,C= = O.
When
AeT bill reaches its minimum, | l| becomes the smallest. Thus, we have:
AW, =CAV, + + CT ATWWW CAV, + +CT
(3)
= =C +0
Then, AV, can be represented as follows.
AV. CT CTC
(4)
Substituting Eq. into AW, CAVT + AAT, we have
A = AWT CCT
(5)
CTC
F PROOF FOR THE LEMMA
In this section, we present the detailed proof for Lemma
Lemma 2. (We can also decompose the weight w(()) into the component along the common direction and the component Et in other directions.) Given the weight W. and the common
direction C, the decomposition W,T V,CT + E, can be conducted as V. TTa
W.C.
and Ei=
W. W.T. CCT T s.t. &CC 0. Such settings minimize ||ee||r.
31

---

Under review as a conference paper at ICLR 2023
proof. Let Ef [j] denote the j-th column of the matrix ef R We represent e/ [s] by the
vector and residual term ef [s] as follows:
T - XC +
(6)
where CT (T = and À is scalar.
Then,
|| bill = |A0
=>>C+ (AC +
(7)
= (E
=x²CTC+ sT 2
Obviously, || [i]1 becomes the smallest when À = 0. other words, e, [ does not contain the
component along the direction C and CT E [i] = 0. Therefore, ||e_ [ill reaches its minimum and only if e, O.
When ||_ [i]]|2 reaches its minimum, || becomes the smallest. Thus, we have:
W. -V, +ep C. - CV. + C -C +0
(8)
Then, V can be written follows.
V. CTw, CTC
(9)
Substituting Eq. intto W, CV. + 7 we have
= W, - WT CCT CTC
(10)
G DECOMPOSITION OF COMMON DIRECTIONS
Actually, the estimation of the common direction C' is similar to the singular value decomposition (SVD), although there are slight differences.
We compute the average weight change AW, = Ezex A Wela, where AWalz denotes the weight change made by the sample z. Then, we decompose AW, into components along five common directions as AW. = CIAVI, 1, + C2AV2, + + CsAV + AE,,, where Ci=C is termed the primary common direction. Ci, Cz, Ca. Ca. and C. are orthogonal to each other. Cz, Ca, Ca and C. represent the second, third, forth, and fifth common directions, respectively. C, represents i-th common direction. AV,, denotes the average weight change along the i-th common direction decomposed from AW..
Specifically, we first decompose the average weight change AW, after the t-th iteration as AW, CAV. AET_ We remove all components along the common direction C' from AW, c, and obtain AW new.t AW, CAV. AεT. Then, we further decompose AW = C2AV. + A In this way, we can consider C2 as the secondary common direction, while C₁ C is termed as the primary common direction. Thus, we conduct this process recursively and obtain common directions (C,, C.VV.. + A
Accordingly, AW, is decomposed into AW, = C, CV CAVET +
32

---

Under review as a conference paper at ICLR 2023
H
DECOMPOSITION OF THE WEIGHT CHANGE MADE BY A SAMPLE T
H.1 PROOF FOR THEOREM 1.
In this subsection, we present the detailed proof for Theorem 1.
Theorem 1. The weight change made by a sample can be decomposed into (h + 1) terms after the 1-th iteration as follows.
AW
AW primary,+ e Σ
AWpl.a) noise,
rewrillen
p 1( +N0 (0)
(11)
where AW!l) =
v c(+1)" (U+1)AV(41)) F10 p(1-1)" /|1 12 denotes the component
along the primary common direction, and aw D D ((+1 De( (411) F(") p(1(t=1)" /||F!") l de- mise,
notes the component along the k-th common direction in the noise term. Et c(t11,)= EnsleV where the SVD of el+1)) "ghhh' is given as et (1+1) -EVV (h S h²), and Eus denotes the e((+1) = Ex eittkkk) Us and Vo denote the k-th column of the matrix u and V, respectively. Besides, have vk E (L,2,...,,h) .h, uy gU41)
k-th singular value
0. Con-
sequently. we have r/0 = (( CU+ r(") /| 1 E R², and (0) (++1) ( Aelltt)
(1+1)" p!l) p(( 11 IBB Raxa
(0) In addition, according to the proof. We can represent weight matrix as w(O) cWD),!)) + Et back propagation and chain rule, we have AW!") = -nD(() (1) 1:00 p( p((-1)" where 1:(0) 7 denotes the learning rate.
@Loss and SF
According to Lemma I and Lemma 2, we have As; (1+1) g(++1) = 0 and e, (++1) g(++1) o. After the 1-th iteration, the weight change made by a training sample æ can computed as follows.
AW.") = -DD 1,(0) p((--))
-nll D 1(+ (((+1) 1o
-DD AW ((0) ((--1) 10
-D.") [Tutt]] + tet (itp)
(((+t))(((t)) + Ae((12) (+1)
F:(0 p((-1)) (.(0)
= -D!" ((t)
((+1)) (a41) AM(+ ((41)) C(41) ((++) AE (e((+1)
(1+1) C(LH)) AV + +
(++)) De det
(+1)
|F
f!"--))
F((1)
D [ [ratt)C(++ AU4H) (1()))) +Et (1+1) AE
(++)
En 1 pul 11) / |
/|000>>||1000 |,,00
C(4)) CULH)AV( p() n,:00
|
(0 (111) de U+D" F(O)(-11) 1!11?
(12)
((+1,k) Enelde V.T where the singular value decomposition of g (++1) is given as ((+2) 4VV et
and Σn, denotes the k-th singular value. 14, and V denote the k-th column of the matrix 24 and v, respectively. We can derive the following equations.
33

---

Under review as a conference paper at ICLR 2023
Aw - - (( c(l+t)) A F(0) p(11!11)"
(1(+1)
+ D ((t Det (1+1) F/ Ft
E(t-1)"
|
= -D C(+ cU+ cUt F(t) pll p(l-1))
p2,0
(13)
D DEt (1+1)* Fl p((-1) / F
h at
Aw
AW i.noie
primary.t
In addition, if we set D v g( g(4 AV; 1.9 a+1) el (e(tt)) pll p (1. Then we can re-write the Eq.
p()) /
12. and
(0)
13) as follows.
AW rewritten 10 (1-11) + Ki
(1)
(14)
Sh0 H.2 THE EXPLANATION FOR THE PHENOMENON THAT S AND SO DECREASE MONOTONICALLY.
DO NOT
In this subsection, we explain the phenomenon that 5 S and (!)) does not decrease mono- tonically in Table
T in Appendix and Table the main paper (Page 6). In fact, we first decom-
(l+1)
-E (1+1,k) according to the SVD. Then AW is computed as (1.b) noiee,t
pose E2
noise.t
D (1-1,)) As( ((41)) F(D /
- 1 Accordingly, the strength of weight changes along
the primary direction is computed as s(1) primary Tem Buex || (Aw
primary:elelll] The
strength of weight changes along the k-th noise direction is computed ||
s
clelle In this way, S(D, s(() and s(()) do not decrease mono- (I+1)
Eee[Tam.Tamll Ten) Ezex Exxx
tonically, although ||= (e((+1.1)] [[,, ||= (a((+1.2) Ir. and [ee(++1,)) || are directly decomposed from e
based on the SVD and decrease monotonically.
ANALYSIS BASED ON EQ. (3) IN THE MAIN PAPER AND EXPLANATION FOR THE PARALLELISM.
According the Eq. (3) in the main paper, we have
(((-1) = (C(1) p(1)) B + E ED. p())
(15)
Thus, if COT D.2)p(0 is large enough (i.e., keeping optimizing w.!'/ along the common direction CO for a long time), then the feature gradients [((1-) of different samples will be roughly parallel
to the same vector B. This is because C(()T D F(0) is scalar and the term cD(0) ((()) is small. In
other words, the diversity between feature gradients
F!!l =ll-1) of different samples decreases. Here,
3 [8,, B2, ... [3]] and [c1, [21,6,," ". "., cc)
J DISCUSSION ON THE BACKGROUND ASSUMPTION.
In the above section, we demonstrate that on the ideal state, i.e., w!(!)" has been optimized towards the common direction ( for a long time, we can consider that the feature gradients f((-)) of different samples will be roughly parallel to the same vector B3. In this way, can explain that the diversity between feature gradients p(l-1) of different samples decreases.
34

---

Under review as a conference paper at ICLR 2023
In comparison, in the current section, we mainly discuss the trustworthiness of the background assumption in Section 4.2 in the main paper. We aim to discuss that on the assumption that features r(l-1) of different samples have been pushed a little bit towards specific common direction, can find at least one learning iteration in the first phase where A AF(l-1) and F(1-1) of most samples have similar directions, and vi and AV have similar directions. The assumption that features p(i-1) of different samples have been pushed a little bit towards specific common direction is an intermediate state between the chaotic initial state of the MLP and the ideal state introduced in the above section. In this way, we can assume that C() D F( is large.
According to Eq. (2) in the main paper and Lemma 2, we have F(1-1) = W!O) D F(!) and w = v c(0 + Thus, we have
1(-11)= = -w("² 10
(V, + (1) )D( p!D) D(D) vV D 1:() + +
(16)
(9) D
If the scalar CO0 D p(0) is large, we can roughly consider
D +()) p(1 v y( = -v: (CW) D ( // V
(17)
It means that the feature gradient 1(1-1) is roughly parallel to the vector V!"). Furthermore, the feature gradient f-(111) and the change of feature AF((-") can be considered negatively parallel to each other, we have
DET / F(1-1) V
(18)
Similarly, we have AF // V+1 Therefore, can roughly consider that V,O) k, kAr((-)) and (0)
())
'+1 Re ke Ar 11-1) where kr, k4+1 E IR are two scalars. Then, we can derive that
(1- AV 4+1 v!l) ket keti AF 1+1 kAF((-))
(19)
If features 1-(111) of different samples have been pushed a little bit towards specific common direction, then it is easy to find at least one learning iteration that AF((-1) and F Fe((-1) of most samples have similar directions, i.e. AF((-1) // r((l-), Meanwhile, we can find least one learning iteration in the first phase where the change of feature in t-th iteration A and (t (t+11 1)- th iteration A (1-1) roughly the same. In other words, AF((-1) AF(-") Thus, have
A
(Re+1 k))A((-") // AF!"") // v.
(20)
In this way, we can obtain that V and AV(!) have similar directions.
K PROOF FOR LEMMA 3
In this section, we present the detailed proof for Lemma
Lemma 3. Given an input sample E X and common direction (i0) after the t-th iteration, if the noise term (9) is small enough to satisfy |AV!") y y y0)00) AV) F (d--1)
>
A F(-1) (1) Ae 1((i1)),, we obtain cos(AV.") (.(A)..EE cos(VV() Ar((-1) 0,
where AV!()) Aw!")" c())
AE((-1)) denotes the change of features AF(!-1) 1 F. - made by the training sample x after the t-th iteration. this end,
et pas and vi()) w(()" ow)
cúb cor
we approximately consider the change features AF((-1) after the t-th iteration negatively parallel to feature gradients fe(--1), although strictly speaking, the change of features is not exactly equal to the feature gradients.
35

---

Under review as a conference paper at ICLR 2023
proof. Given sample x, we can prove that cos (AV. F(u-1)) cos(Ve") Ar(( 2 0.
According to chain rule, we have
AW -n (111) FQ-1)²
(21)
According to Lemma and Lemma 2, have C() _e(") (1) and ol (() 0. Then, we have
A Av u p
v,0' A0--
coa( (AV( D( cos(V)"), 1( =
(22)
||V || ||
[1 1- |1
Therefore, we have
sign(cos((4 A (1)
F(( (V, (V,"),((())))
sign(
F(1 (AVV) (1)
[V1) 1/ (221I,,,"2l1FFF"""""""" (2l1 12 12 ( (2))
sign( (
F(
1 I.ve
10) woO' D."h 1 122
2 E
- 12))
sign( (1)
P(1-1)1 IV O" cr + + ) (1) F.
112 |12l, " ((
(|2))
Dign([AV,") (1)
1. IV UT (v
cw. + ( (AW! !))
F. (1 1)
- /(-n (-n |
/(IAV, 11a (1 11 112llV1 ||lll (12)
(1)
=iig (1)
1. [(V,"' '1) V v."C"'" + +V."' (') F
/(- p
| |all A f1 12ll, IallE"" (12)
=sign( AAV (1)
F! 1( '0' vI" + +v'"' (C) + )
/(-n PL ((111)
|AAV 1ell F (1 IeII, IallE!""" (l2)
=sign((AV,!) ( F(U-1) IVV.'' CI AV + v.00' (1) AE
Ef
+ +,U'" V CO. TTT + +1² EL AV. 1/(n 1 1 1)
LAv 1ell
|ell "Ilalll:""lll11 [1a1le(("" (12)))
2
(1) F. -VV,"'(('''' CI" AV + +v,0'' (1)
& Ae )((--1)
/(-n P. (1111)
|AAV (1) 1ell Iell "I "I IeIV Iell (12)
sign([AV (1)
F! MV O0' " CWAV F(-1)) v.'h" c (1) F(
1(-n (
||AV IellE H- " " VallEt" ll2)
(A /(-n
=sign(
(1)
V v.O COAVO'' F(--1) AV!!! p(l-")v:00" (00) Ae
(1)
p((-1))
|AV bell 1 12ll (l))
(23)
According to our assumption, the noise term Et is small enough AV p( yuu""""" (C "V'V""" p(
to satisfy
A F. py (1) Ast ( p(1-1)|
This assumption is verified in Figure 38 Then can ignore the last term and obtain
sign sign((AV())) Fl ail-1)(())
CUT CO E( + AV. pe
10 el Ae 1h) p((-1))
/(-n) pp 11 2
|AV 12ll!
" ll2))
x sign(( ( D.C CO oVAVV"" (((-1))
(-1)
ا || | ||
F. - |2 [( (l2)) S
(24)
36

---

Under review as a conference paper at ICLR 2023
Frobenius Norm
0.020
lVV! F. "1," ce ou cv "
F." "I
0.015
0.010
|ay! pl ad pf-
0.005
0.000
800
810
820
830 840 850 960 870 8E0 890 iteration
Figure 38:
Visualization of the Frobenius norm of the two components D C AV F(1-1) and AV p(-"D(() Ae F
AV F(1-1) (1)
We trained 9-layer MLP on the MNIST dataset, where each layer had 512 neurons. Iterations
chosen the end of the first phase.
Thus,
(AV, FF!! _((l-))) cos(V, "Fl ((1-1))) S
(25)
In this paper, approximately consider AF and F!!-1) negatively parallel to each other. Thus, we have sign( (gn(coo(AVV") E((T-))- cos(V,(), AF((-1)) = sign(cos(AV," _((-1)
(- (-cos(V,)), Ft 1)(-1)))) 0.
L PROOF FOR THEOREM 2
In this section, we aim to prove that training samples of the same category have the same effect in the first phase.
Theorem 2. Under the aforementioned background assumption, for any training samples x, x' E X in the category c, if [CO) similarity in very early iterations), then cos(a,AV:!) A la, F! F(l-1))) 1,)20, and cos(a, vill A (1-11) 2 0, where ar E{-1,+1}
D 1 b(00
[e)) > (i.e., F(") and F!1)) have kinds of
constant shared by all samples category
proof. Given a sample x and sample x' from the same category, we can prove that (os(AV(() aa Ft (1111) i.) cos( Avll) |o,, p((-1) lar) O.
sign(cos(( A Ie. FL b)) 12) Cos(AV
at, Fl ((1-)) bc))
((VV (1)
1 (11 [o] (AV!")
we (((l))))) lwr))
sign( CO0 AW!") x p( ((1D) [z] Cwn AW
pl (1 p(1 (2])
C (c)
CO C0
sign([C("' AW(") ((((11)) [a] (C(0) AW!!)) lx ((1))) le]])
sign([C(") (-nD(") (2F 1e
[al (dl(((")) (-nD! lo 1. (T)
F(
lrr))
sign([C()) (a(!)) -0w! (((()))))) (2)))
(26)
According to the assumption that (C0) """ D (00)) wel
F lx and P(") have kinds of similarity, we can consider
O. In this way, for the category c, there exists constant
which satisfies sign(cos(ce (a.AV) Av Xx belongs to the category c.
a. Fl (1-1) (.) 2 O, where a E {-1, +1} and training sampl
(x) (o AF. (x) 2 Thus, we have sign(cos( (o..oo.VV (V(()) In, F( p((-))] lx) cos(ae ((()) AF(1-1) (x)) o. In addition, the above proof indicates
According Lemma we have cos(Av,)) lx, ((()
that (con(co((e))) A vil) F.ll11 ((11)) [[) Therefore, have (con( (((0) (F((=")=) 0
37

---

Under review as a conference paper at ICLR 2023
CIFAR-10
MNIST
Tiny ImageNet
Oxsine Similarity of features
9-layer, norm1
9-layer
7-layer, nomm
0.4
7-layer
1000
2000
1060
2000
0.3
2500
5000
lteration
iteration
iteration
Figure 39: Cosine similarity of features between samples in different categories. We trained 7-layer MLPs and 9-layer MLPs on the CIFAR-10, the MNIST, and the Tiny ImageNct dataset.
M DISCUSSION FOR FOUR TYPICAL OPERATIONS
M.1 CENTERING OPERATIONS FOR NORMALIZATION
The output feature the 1-th linear layer W.Z.f. the input sample x can be described as [fi, f2................. ................................ fa] w(D)pef p((t-1) RR,, where fi denotes the i-th dimension of the feature. In this way, the batch normalization operation can be formulated as BN(fi) = Tseate p4)/a,) Bahin where Ticaae and Satin denote the scaling and the shifting parameters, respectively. In this way, the batch normalization operation subtracts the mean feature samples. Therefore, features of different samples in same category are longer similar to each other.
F(()) = E [F(2) le] from features of all
We also propose simplified normalization operation (i.e. centering operations for normalization) to alleviate the TFC phenemonon in the first phase. The centering operations for normalization is (f: 14)/a, where μ4, and σ; denote the mean value and the standard
given as norm;(fi) deviation of Sa over different samples, respectively. This operation is similar to the batch normal- ization (Toffe & Szegedy 2015), but do not compute the scaling and shifting parameters in the batch normalization.
In order verify the centering operations for normalization can alleviate the TFC phenemonon dur- ing the training process of the MLP, we trained 7-layer MLPs and 9-layer MLPs with and without the centering operations. Specifically, for the centering normalization operation norm1, we added the centering operations after each linear layer, except the last linear layer. Each linear layer in the MLP had 512 neurons. Figure 39 shows that the feature similarity in MLPs with centering operations kept decreasing, while the feature similarity of the MLP without centering operations kept increas- ing. This indicated that centering operations for normalization alleviate the TFC phenomenon.
M.2 MOMENTUM
We explain that momentum in gradient descent can alleviate this phenomenon. Based Lemma 3, the "self-enhanced system" of the TFC phenemonon requires singular values of weights along other directions e to be small enough. However, because the momentum operation strengthens influences of the initialized noisy weights w-(1) it strengthens singular values of 5 to some extent, thereby alleviating the TFC phenemonon.
(!)
Specifically, considering the momentum with the coefficient m, the dynamics of weights W4+1 can be described as,
OLoss
ALoss
W411 W. aw, OW,-1'
(27)
where 7) denotes the learning rate. Because we only focus on weights in single layer, without causing ambiguity, we omit the superscript (I) to simplify the notation in this subsection. In this way, we can write the gradient descent as
T
1 - m T+1 +aLoss
Wr+1 Wo +
Σ
(28)
m aW.
Since 0 m 1, the coefficient 1-MTI1 decreases when the variable increases. Thus, large 1-m
represents that influences of Wo on Wr+1 are significant. Because 87+1 is decomposed from
38

---

Under review as a conference paper at ICLR 2023
Cosine Samilarty
CIFAR-10
MNIST
Tiny ImageNet
CIFAR-10
MNIST
Tiny ImugeNNet
of features
82
200 6o
022
0A
Uu
1000
2000
2000
3003
2500
5000
itcration
iteration
iteration
itcration
iteratoon
iteration
L-9, m-0
L-9, 9, m-0.5
L-9,
m=0.9
1-1o-4
1-
L-7. L-7,A-1e-5
A-1-44
m-0
L-7, m-0.5
L-7, m-0,9
A-1e3
2-1e5
Figure 40: Effects of (a) momentum and (b) Le regularization. We trained L-layer MLPs, where each layer had 512 neurons. A shorter first phase indicates that the TFC phenomenon more alleviated.
Wr+1 and singular values of er+1 are mainly determined by the noisy Wo. Accordingly, singular values of Cr-1 are relatively large, which disturb the "self-enhanced system" and alleviate the TFC phenemonon.
To verify the above analysis, we trained MLPs with 0, 0.5, 0.9, respectively. Figure 40 verifies that a larger value of m usually more alleviates the TFC phenomenon.
M.3 INITIALIZATION
We explain that the initialization of MLPs also affects the TFC phenemonon. According Lemma 3, such "self-enhanced system" requires singular values of weights along other directions e(") to be small enough. However, because increasing the variance the initialized weights W will increase singular values of Et tialize weights with Xavier normal distribution (Glorot Bengio 2010), i.e. Wo ~ (o, (o
(!) based on Lemma 2, alleviating the TFC phenemonon. Specifically, we ini-
Tana fantn and fan denote the input dimension and the output dimension of the linear layer, respectively. In this way, large yields large singular values of initial weights (!) w wr!") cogC))
where var
2
CO Ca Large singular values of initial weights Wo lead large singular values of e("). Therefore, large variance of initialized weights disturbs the "self-enhanced system" and alleviates the TFC phenemonon.
Wo. Based Lemma 2, also have E
M.4 L2 REGULARIZATION (RIDGE LOSS)
L regularization is equivalent to the weight decay in the case of gradient descent. The total loss given as C(W,) _ (W.) + >|[Wi where CCE(W,) represents the cross entropy loss, and W||W denotes the ridge loss. In this way, we have the following iterates by using gradient descent
W+11 = W - VL (W₂)
= W; - CE (W,) - 2mAW,
(1-21)) 2nA),, V
CCE (Wt),
(29)
According to Lemma 3, such "self-enhanced system" requires singular values of weights along other
g00g directions E to be small enough. Based on Lemma 2, we also have Et = wr - -w(") C cw
In this way, larger À yields smaller singular values of c(") which disturbs the "self-enhanced system" and strengthens the TFC phenemonon. Figure bb Figure 9(d) in the main paper verify that larger coefficient À more strengthened the TFC phenemonon.
39

---

Under review as a conference paper at ICLR 2023
N EXPLANATIONS FOR MORE DNNS.
The theoretical analysis of this study can explain which kinds of DNNs are more likely to exhibit the TFC phenomenon in early epochs. In fact, discovered the two-phase phenomenon and the TFC phenomenon in various DNNs, including MLPs and modern CNNs, e.g., VGG-11 models and VGG-13 models. Specifically, we trained VGG-11 models and VGG-13 models on the CIFAR- 10 dataset and the Tiny ImageNet dataset. We adopted the learning rate 7) 0.01, the batch size bs 100, and the SGD optimizer. The training loss, the testing loss, the training accuracy. and the testing accuracy are shown in Figure 41 and Figure VGGs exhibited TFC phenomenoa practice.
Figure 41 e) and Figure 42 e) show that
Testing Loss
Trining A ocuurryy
Thining
1.
VGG-11
10
10
(b)
(c)
VGG-13
TesingA (9)
Cosiia simillaity of Restunes
of ggndieents
0.85
0.96
0.33
1.01
0.99
1000 iteraaiin
200
iteration
iteration
(e)
Figure 41: (a) The training loss of VGG-11 model and a VGG-13 model trained on the CIFAR-10 dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d) Testing
accuracies of two models. (e) Cosine similarity between features of different categories. (f) Cosine
similarity between gradients of different samples a category. The feature and the feature gradient were used in the third linear layer of the MLP in models.
Triminn Aonuaay (%))
Teeting Loss
Thining
3.6
1.2
VGG-11
10
10
16
18
(8)
(b)
(c)
VGG-13
Tesing Acmurac (86)
Cosine similarity
0.55
Costne similliity
gradrenss
00225
0,559
0.903
0.75
8.86
0.75
1091
iteration
iteration
iteration
(e)
Figure 42: (a) The training loss of a VGG-11 model and a VGG-13 model trained on the Tiny ImageNet dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d)
Testing accuracies of two models. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the third linear layer of the MLP in models.
Furthermore, we found that our theoretical analysis can be generalized to modern CNNs and transformers. We conducted experiments on ResNet-18, ResNet-34 (He et al.) 2015), and Vi- sion Transformers (ViTs) (Dosovitskiy et al. 2020). Because both ResNets and ViT's were the two most classical network architectures that had been examined for years. it showed that ResNet- 18, ResNet-34, and ViT did not exhibit the TFC phenomenon (or the TFC phenomenon only existed in very few iterations within the first epoch), owing the of normalization operations in these
40

---

Under review as a conference paper at ICLR 2023
DNNs. However, according to theoretical analysis, if the batch normalization (BN) operations in ResNet- 18/34 and the layer normalization (LN) operations in ViTs were removed, then the TFC phenomenon significantly strengthened.
First, we trained ViTs, ResNet-18, and ResNet-34 models on the CIFAR-10 dataset. The classifi- cation heads in both ViTs and ResNet-18/33 were implemented by 4-layer MLP. Specifically, trained two different ViTs with the patch size P 4, the heads = 18, the dropout rate = 0.1, the embedding dropout rate 0.1, the learning rate 7) 0.1, the batch size bs 100, and the SGD optimizer. For ResNet-18 18 and ResNet-34 models, we adopted the learning rate // - 0.01, the batch size bs 100, and the SGD optimizer. Besides, we used two data augmentation methods, including random cropping and random horizontal flipping. The training loss, the testing loss, the training accuracy, and the testing accuracy are shown blue curves in Figure 43 Figure (4] Figure 44] and Figure 46 These figures verify that ResNets and ViT'' first phases were very short, and the TFC phenomenon only existed few iterations, which could be ignored.
Second, in comparison, we further constructed four baseline networks by removing the BN layer from ResNet-118/3 and removing LN layers from ViTs. Orange curves in Figure 43 Figure 44 Figure
45 and Figure 46 verify that such new ResNet-18/34 and new ViTs exhibited significant TFC phenomenon.
2.2
Thiining Arcumayy
Thainkogg Lose
Tosting 1oos 1.o8
60
1001
1900
1.5
40
0.81
0.81
ResNet-18
2000
4000 (8)
6000
8000
10000
2000
4000)
(6)
6000
2000
10000
0
2000
4000,
6000
5000
10000
ResNe-18
-00
(without BN)
60
Costra Costrendirzzian ditraiinity
dimilbttyy gradierts
-50
of feeturess
0.90
40
30
20
10:
2000
4000
6000
10000
10QU
2000
2000
Itaration
iterntion
(5)
Figure 43: (a) The training loss of ResNet-18 and ResNet-18 18 (without BN) trained on the CIFAR- 10 dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d) Testing
accuracies of two models. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples a category. The feature and the feature gradient were used in the third linear layer of the MLP in models.
60
Toating Loos
Toedin Lose
2.0
1.5
1.5
20
1.0
ResNet- 34
2000
4000
6000
8000
4000
8000
ResNet-34
Testing/ Acrunay (8)
(without BN)
60
Costrn dimiiaiity
Costue sirriiattyy
0.9
40
0.8
20
00.7
0.8
0.6
0.7
2008
4000
6080
R00n
550
3000
500
3000
Figure 44: (a) The training loss of'a ResNet-34 and ResNet-34 (without BN) trained on the CIFAR- 10 dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d) Testing
accuracies of two models. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples a category. The feature and the feature gradient were used in the third linear layer of the MLP in models.
41

---

Under review as a conference paper at ICLR 2023
Toinung
Comeesimlen 1
Hidden 0--612
Lagarse,, MLP ston-256
2000
4000
6008
1000
2000
4000
6000
8000
D-512,
1.0
Lagassed,
90
of Datures
0.35
0:05
Ditthou
25
0.80
50.35
0.8
0.8
0.55
1033 Iteraoo
13eration
(4)
Figure 45: (a) The training loss of VIT and a VIT (without LN) trained on the CIFAR-10 dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d) Testing accuracies of two models. (e) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient used in the third linear layer of the MLP in models.
2.1
Testing 1750
2.0
2.0
1.99
Hidden size Dw612
2000
4000
60000
8000
10000
2000
4010
(000)
3000
10000
2000
4000 5000 8000
(c)
10000
Hiddhen o-512.
Laparse7, MI sixem56
40
Codteddimitatt amtiotty
3.0
Creine smmbeety
Lapesse7, stze=200
20.50
(without..))
0.9
20
0.
2000
19B2 Itmrttio
3810
storition
(n]
Figure 46: (a) The training loss VIT and a VIT (without LN) trained on the CIFAR-10 dataset. (b) The testing loss of two models. (c) Training accuracies of two models. (d) Testing accuracies two models. (c) Cosine similarity between features of different categories. (f) Cosine similarity between gradients of different samples in category. The feature and the feature gradient were used in the third linear layer the MLP in models.
42

---

Under review as a conference paper at ICLR 2023
O MORE EXPERIMENTAL RESULTS OF ASSUMPTION 1.
Assumption 1. We assume that the MLP encodes features of very few (a single or two) categories in the first phase, instead of simultaneously learning all or most categories in this phase.
In this section, we aim verify that Assumption is common fact in various DNNs, includ- ing MLPs, VGGs, and ResNets. To this end, we have conducted new experiments to show that DNNs encoded features of very few (a single two) categories in early epochs. Specifically, trained a 9-layer MLP on the CIFAR-10 10, the MNIST dataset, and the Tiny ImageNet dataset, respec- tively. Each layer of the MLP had 512 neurons. Besides, We trained VGG-11 model, VGG-13 model, and a R 8 on the CIFAR-10 dataset. We evaluated the training accuracy at the end of the first phase. For the Tiny ImageNet dataset, randomly selected the following 50 categories, orangutan, parking meter, snorkel, American alligator, oboe, basketball, rocking chair, hopper, neck brace, candy store, broom, seashore, sewing machine, sunglasses, panda, pretzel, pig, volleyball, puma, alp, barbershop, ox, flagpole, lifeboat, teapot, walking stick, brain coral, slug, abacus, comic book, CD player, school bus, banister, bathtub, German shepherd, black stork, computer keyboard, tarantula, sock, Arubian camel, bee, cockrouch, cannon, tructor, cardigan, suspension bridge, beer bottle, viaduct, guacamole, and iPod for training. Figure 47 Figure (88) Figure 49 and Figure (50 show that various DNNs encoded features of very few (a single or two) categories in early epochs.
9-aayer MLP traned the CIFAR-10 dataset
9-1ayer MLP triind on the MNIS datasst
100
1007
Aucinyy (N)
Accorag (%)
80
catzpory
cotegrey
(6)
9 Blaayer MLP trined the Tiny ImageNett Dutaset
The training accuracies MLPs on the CIFAR-10 dataset, MNIST dataset, and Tiny ImageNet dataset. The accuracies were evaluated at the end of the first phase. MLPs encode features of very few (a single or two) categories in the first phase, instead of simultancously learning all most categories in this phase. (a) The training accuracy of 9-layer MLP trained the CIFAR- 10 dataset. (b) The training accuracy 9-layer MLP trained on the MNIST dataset. (c) The training accuracy of 9-layer MLP trained the Tiny ImageNet dataset.
Figure 47:
VGD0-1 CIPAR-1D
Iragelcc
Figure 48: The training accuracies VGG-11 models on the CIFAR- dataset and the Tiny Ima- geNet dataset. The accuracies were evaluated the end of the first phase. VGG-11 models encode features of very few (a single or two) categories in the first phase, instead of simultaneously learn- ing all most categories in this phase. (a) The training accuracy of VGG-11 models trained the CIFAR-10 10 dataset. (b) The training accuracy of VGG-11 models trained on the Tiny ImageNet dataset.
43

---

Under review as a conference paper at ICLR 2023
VGD 15 suaad CEAB 30 daseet
VOS 13 Enttned the Tay Trayshee Dobaet
Figure 49: The training accuracies VGG- 13 models on the CIFAR-10 dataset and the Tiny ImageNet dataset. The accuracies were evaluated the end of the first phase. VGG-13 models encode features of very few (a single or two) categories in the first phase, instead of simultaneously learning all most categories in this phase. (a) The training accuracy of VGG-13 13 models trained on the CIFAR-10 dataset. (b) The training accuracy of VGG-13 models trained on the Tiny ImageNet dataset.
Redhoet trated CTBAR:8 diowt
Rodittil Try Tubret
28.04
eiiggr tnde
Figure 50: The training accuracies of ResNet- 18 models on the CIFAR-10 dataset and the Tiny ImageNet dataset. The accuracies were evaluated at the end of the first phase. ResNet-18 18 models encode features of very few single or two) categories the first phase, instead of simultaneously learning all or most categories in this phase. (a) The training accuracy of ResNet-18 8 models trained on the CIFAR-IU 10 dataset. (b) The training accuracy of ResNet-18 models trained on the Tiny ImageNet dataset.
44

---

Under review as a conference paper at ICLR 2023
P PROPOSE AN IMPROVED TRAINING METHOD
In this section, we use our theory to develop a new normalization method. The new normalization operation was designed considering the following two findings.
Our theoretical analysis told us that the centering operation in BN could alleviate the TFC phe- nomenon.
Previous studies found some shortcomings of the BN operation, i.e. the BN operation usually caused unstable features. Thus. the BN operation was found incompatible with the dropout (Li et al., 2019), hurt the classification accuracy in adversarial training (Galloway et al. 2019), and decreased the quality of images generated by generative models (Salimans et al. 2016))
Therefore, according to our analysis, we only need to update the dynamic normalization parameters (i.e., (i and o; in the following equation) in the first phase to avoid the learning-sticking problem, instead of applying the dynamic normalization parameters in the entire training process. In this way, we can simultaneously solve the learning- -ttikinngg problem and avoid unstable features.
Specifically, we are given the output feature F = [fi, f., fa] E Rh of the I-th linear layer w.zl the input sample x, where fi denotes the i-th dimension of the feature. The new normalization operation is given
nfrm(f.) = (fa p.)/o..
(30)
where de and o; denote the mean value and the standard deviation of St over different samples, respectively. We only update the mean value 14 and the standard deviation o, the first phase, as follows.
Engtaan [fi].
{{ [Vaa [fi),
# = { p1,--1, ST
of of=
a SS
(31)
where we keep updating a, = 0.99a,-1 0.01 Ebusch [buaclcass( Fl through all the t previous batches to represent the current cosine similarity between features of different samples. If a, is greater than threshold r 0.3, then we consider the learning process to be in the first phase and normalize the feature. Otherwise, a; S T, then we consider it has already jumped to the second phase, stop updating μ and of, and use constants μr and of to generate stable features. We set m 0.1 and compute μs,, and of,, in the t-th batch as follows.
Fi, (1 m))5 + mlEreraach [[t] ae >T oi = {( m) + m Varzetius [[]], > 14 ST
-
(32)
To this end, we conducted experiments on two types of MLPs (i.e., 9-layer MLPs and 11-layer MLPs) to compare the proposed method with BN. For each type of MLP, we trained three versions MLPs on the CIFAR-10 10 dataset. The vanilla MLP had 512 neurons in each layer. We added the proposed norm operation after the first, the third, the fifth, and the seventh linear layers, and con- structed the network MLP-norm. For a fair comparison, we constructed a bascline MLP, namely MLP--N, by adding the BN operation in the same positions as in MLP-norm. In addition, scaling and shifting parameters in the BN operation were closed. Figure [1 shows that both the MLP-norm and MLP-BN alleviated the learning-sticking sticking problem. However, MLP-norm was optimized much faster than MLP-BN, because our theoretical analysis told us that it was not necessary to continue updating μ4 and of, if the learning process did not have risk of feature collapse, thereby alleviating the optimization problems found in (Li et al.
2019 Galloway et al. 2019).
11-lape MI
Alayer
MLP MLP BN
R-som
spach
350
200
epoch
100
(2)
(2)
Figure 51: (a) The training loss of three 11-layer MLPs trained on the CIFAR-IO dataset, where each layer had 512 neurons. (b) The training accuracies of three 1-llayer MLPs. (c) The training loss of three 9-layer MLPs trained on the CIFAR-10 dataset, where each layer had 512 neurons. (d) The training accuracy of three 9-layer MLPs. Note that the vibration of the blue curve could be explained as the failure of jumping out of the first phase, due to the strong power of the "self-enhancement system."
45


Reviews:
[{"review_id": "PPlMMMdzp1", "reviewer": "Reviewer_4cUL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors examine in depth the learning-sticking problem in early training of DNNs from the perspective of learning dynamics. \nThey offer a novel explanation for what often happens to network weights and inner activations during the problem - which they call \"the temporary feature collapse\" (TFC) phenomenon\nThey provide thorough theoretical (and empirical) analysis and justification of their idea.\n", "summary_of_the_review": "A solid contribution. It presents a new phenomenon in early DNNs training and shows its possible explanation. The claims are supported by thorough theoretical and empirical analyses.\nHowever, practical impact of the findings seems to be limited. ", "strength_and_weaknesses": "PLus\n- There are not many studies that look in depth on the early phase of DNNs training and that examine what exactly happens to the network \ninternal structure before the training error begins to decrease (and why this period is sometimes unexpectedly long).\nThe paper tries to fill in this gap.       \n- The paper shows a counter-intuitive (but possibly common) mechanism that causes a NN-model to get stuck in the early phase of training. \nThe idea presented is new and surprising to me. \n- The provided theoretical (and empirical) analysis in broad and thorough, with convincing results.\n\n\nMinus\n- Although the temporary feature collapse phenomenon is interesting, the impact of the findings on NN applications and future research may be rather limited.\n  The authors show that TFC can be easily avoided by commonly used techniques (clever weight initialization, momentum, l2 regularization, batch normalization) \n  which implies that TFC phenomenon rarely occurs in current applications.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nAfter rebuttal:\n\nI have read the authors' comments, the new version of the paper and other reviews. I agree with most of the criticism regarding the original submission (e.g.,the need to test more NN-architectures, the need of real-world applications/impact, strong assumptions, and other theoretical analysis issues). On the other hand, I appreciate how the authors revised the paper to clarify these concerns. Most of the flaws are resolved in the final submission.\n\nI think the work is novel, interesting and solid enough to be accepted despite its remaining flaws (mainly the relatively limited practical impact). However, since my score seems too optimistic compared to other reviewers with similar opinions on the latest version, I lower my score to 6 for consensus (although it is hard for me to decide between 6 and 8).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KM6-BRLi7cF", "reviewer": "Reviewer_7KPL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work finds that the MLP exhibits a fundamental yet counter-intuitive TFC phenomenon in the early stage of the training process. After that, the authors explain this phenomenon from the perspective of learning dynamics and explain why four types of operations can alleviate the TFC phenomenon. Extensive theories and experiments are conducted to support the arguments of this work. ", "summary_of_the_review": "This work finds that in the early stage of the training process, the MLP exhibits a fundamental yet counter-intuitive TFC phenomenon and explains the reason why four typical operations can alleviate the TFC phenomenon. However, from my field, I cannot realize the value it can bring to the real world, and it seems that the theory analyses only focus on DNNs, which might not be suitable for CNNs or transformers. Could the authors give a further explanation because I am only partially in your field? Currently, I tend to vote marginally below the acceptance threshold but I am not an expert in the related field. I will hear more from other reviewers and the authors’ responses.\n\n\n=== After rebuttal ===\n\nI have read the authors' responses and other reviewers' comments. The responses solve my concerns well. I am willing to raise my score.", "strength_and_weaknesses": "Strength:\n\n(1)  The authors discover the common TFC phenomenon in early learning of the MLP.\n\n(2)  The authors explain this phenomenon from the perspective of learning dynamics. \n\n(3)  The authors explain why four types of operations, including normalization, momentum, initialization, and regularization, can alleviate the TFC phenomenon.\n\nWeaknesses:\n\nThis work explains the TFC phenomenon from the perspective of learning dynamics. I have some questions as follows. \n\n(1)  The authors argue that there exist two stages in the training process of DNNs, however, how to determine the percentage of two stages? If the first stage is short, what does this explanation tell us? And can you explain the second stage of the training process? \n\n(2)  It seems that all the explanations are based on DNNs, so is it suitable for CNNs or the recent transformer models? Furthermore, it is also more valuable if the authors point out what it will bring to real-world applications.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IJG3Uxtx5Fn", "reviewer": "Reviewer_nGWd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the two-phase phenomena in training deep neural networks both empirically and theoretically. The work discovers and explains the reason for the feature collapse phenomenon in the first phase, i.e., the diversity of features over different samples keeps decreasing in the first phase, until samples of different categories share almost the same feature, which hurts the optimization of MLPs. It explains such a phenomenon in terms of the learning dynamics of MLPs. Additionally, the work theoretically analyzes the reason why four typical operations can alleviate the feature collapse.", "summary_of_the_review": "Overall, this work is well-written and well-presented.\nIt studies an interesting TFC phenomenon in deep learning training, with quite extensive experiments and theoretical justifications.\n\nIt is a bit unclear on the universality and practical guidance of TFC.\n", "strength_and_weaknesses": "#Strength: the work is well presented and well written. It discovers an interesting phenomenon empirically with extensive experiments, and investigates it theoretically: the phenomenon that different neurons in a layer are optimized towards a common direction in the first phase\n\nIt provides explanations of why four typical operations can alleviate the feature collapse: \n\n#Weakness: The experiments are mostly done on MLP and VGG network architectures. More experiments should be done on more popular/modern architectures such as ResNet or even large Vision transformer networks.\n\nThe analysis is based upon gradient flow and quite strong assumptions.\nAlthough it explains the benefits of four typical operations from the new perspective, it is unclear for providing practical guidance for improving optimization design. The paper can be significantly improved by introducing new methods for improved training.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1E9v87GqIZI", "reviewer": "Reviewer_4Wyv", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on learning multi-layer perceptrons (MLPs), particularly the feature collapse phenomenon in the first phase, i.e., the diversity of features over different samples keeps decreasing in the first phase of training until samples of different categories share almost the same feature. The authors explain such a phenomenon in terms of the learning dynamics of MLPs.", "summary_of_the_review": "Based on the above comments, this paper studies an interesting phenomenon, the feature collapse phenomenon in the first stage of training. While this result is interesting and could provide insight into the training dynamics of MLP, this result is only observed under strict settings, the analysis seems to provide limited guidance on why this happens as it is based on strong assumptions and only considers the samples from the same classes, and it is not clear how this phenomenon affects the performance (since one can either alleviate or strengthen this phenomenon by some common techniques like batch normalization and weight decay). ", "strength_and_weaknesses": "## Strength\n-  The feature collapse phenomenon in the first phase is very interesting. It could provide insights into understanding the learning dynamics of MLPs. This phenomenon complements the neural collapse phenomenon [A] observed at the end of the training stage. \n-  Partial analysis based on assumptions is provided to understand the feature collapse phenomenon.\n## Weakness\n- The feature collapse phenomenon happens only in certain cases. For example, this phenomenon disappears when batch normalization is used. This point was not clear until section 4.3. Given that batch normalization is a common practice in training deep neural networks, it should be clearly specified the training settings for observing the feature collapse phenomenon at the beginning (e.g., in both abstracts and introduction). \n- Section 4.1 provides two perspectives to analyze the feature collapse phenomenon. But there is no description of why these two perspectives are needed and how they could provide an explanation. For example,  perspective 1 focuses on the change of the weight vectors and observes the existence of the common optimization direction shared by different weight vectors. But recall that the feature collapse phenomenon refers to similar features across different inputs. It is not clear how perspective 1 can provide an explanation for the feature collapse phenomenon. \n- The analysis in section 4.2 is based on very strong assumptions. For example, Theorem 2 assumes that the features of different samples from the same category have been pushed in the same direction. This seems already assume the collapse phenomenon. \n- It is not clear how the analysis in section 4.2 deals with samples from different classes. Assumption 1 comes out of the blue, and there is no description for it. \n- Overall, I found it is not easy to follow section 4.2. The presentation of the analysis could be improved. \n- Section 4.3 shows that some techniques (like batch normalization) can alleviate the feature collapse phenomenon, while others (like L2 regularization) can strengthen this phenomenon. So should we alleviate or strengthen this phenomenon? How does this phenomenon affect performance? \n\n[A] Papyan, Han, Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 2020.\n\n____________________ After rebuttal_______________\n\nI appreciate the authors' great efforts in addressing my comments. I have increased my rating, but I still have concerns about the analysis. Per the comment \"The key problem of explaining the TFC phenomenon is to prove that the significance of such a common direction is likely to be further enhanced, just like a \"self-enhanced system.\" Without the proof of the \"self-enhanced system,\" the assumed initial common direction does not significantly decrease the diversity of features, and cannot exhibit the TFC phenomenon.\", I am still not clear why the common gradient direction is not enough. Also, if it behaves like a \"self-enhanced system\", then the features will only become more collapsed, but why it disappears in the second phase? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject