Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper studies representations of neural networks trained with cross entropy, when trained beyond zero training error, utilizing the unconstrained layer peeled model (ULPM). ULPM is motivated by the fact that many neural networks are highly overparametrized and assumes that the representations, on which the last linear layer acts, can be optimized freely, i.e.  they are unconstrained.
In this model, the authors derive several theoretical results, related to the recently observed neural collapse [28]. In particular, they derive a relation to max-margin analysis, show that neural networks converge to Karush–Kuhn–Tucker (KKT) points of a certain minimum-norm separation problem, and show that the loss landscape is benign, as it has no spurious minima.
main_review: In its current form, the paper is not ready for publication. This is mostly due to clarity issues. Aside from linguistic errors and typos, there are whole paragraphs which are very difficult to follow. As a consequence, also the theoretical contribution suffers, as its meaning and significance cannot be assessed adequately. I believe however, that this paper would make a decent contribution, once the major issues are resolved.

### Strengths:

- The paper belongs to a group of recent works which study the so called neural collapse in the unconstrained layer peeled model. However, in contrast to previous work, the authors do not only study the *global* minimum, but also *local* optimization dynamics and curvature.
- The paper does a good job positioning itself within the related work in section 5.2
- The presented example 3.1 was very welcome. It proves the existence of problematic points in the loss landscape, which is not obvious, and motivates the subsequent theory. I would have liked it even more, if it was paired with a figure.

### Weaknesses:

- As already mentioned, the paper is hard to follow. 
  - Paragraphs, which are quite difficult to understand include remark 3.1, the discussion of reference [43] between lines 300 and 309, and the first paragraph of section 3.2, which contradicts itself.
  - line 32 to 34: The terms "within-class cross-sample features" and "interpolating the in-sample training data" need more background.
  - line 84: I don't under stand what is meant with "balanced superpositions of symmetric copies of the ground truth according to the hidden symmetries in the objective function".
  - In theorem 3.1, it is not clear, what is precisely meant with "along the direction of an Karush-Kuhn-Tucker (KKT) point". Does this mean, that the referred to limit point and KKT point are positive multiples of each other?
  - It seems, that a combination of theorem 3.1 and 3.2 would be the main point of the 3.1.2. Convergence Results section. However, this combination is not discussed at all.
  - If Lemmas 3.2 and 3.3 should be kept in the main text, then these need to be discussed in more detail.
  - Definition 3.2 defines the tangent space at a point as the point’s orthogonal complement. This indicates that this point lies on a sphere. However, this does not seem to be the case.
  - What is meant with "collapsed margin minimize the neural collapse margin" (line 157). Similarly, it is unintuitive, why the quantity in definition 3.1 is called the neural collapse margin, as, a priori, it is not connected to neural collapse. As a reader, I expected the neural collapse margin to be defined via a minimum over all $W$ and $H$.
  - It is unclear, whether there is a difference between the Layer-Peeled Model (1 Introduction) and the Unconstrained Layer-Peeled Model (2.2 Problem Setup) 
  - I will list some minor linguistic errors at the end of the review, so that the authors can correct them. 
  
- There are several claims, which need to be discussed in more detail, or appear to be wrong
  - In line 57, it is claimed that "The analysis provides insights on how gradient descent separates data during the training of neural networks with neural collapse and the benefit of training after interpolation on generalization and robustness". However, robustness and generalization aspects appear not to be covered in this work.
  - Caption of table 1: "We provide strongest theoretical results with minimum modification on the training objective function" (compared to the related work). As already mentioned, I was unable to understand the authors argument, why their work provides stronger results than [43]. Still, this is a bold claim.
  - line 62: "Our result doesn’t introduce any extra feature norm constraint or feature norm regularization, which are not commonly used in the realistic deep learning."\
  While this is true, one might argue that the omission of weight decay, as done in the analysis, is also "not commonly used in the realistic deep learning". Similarly, line 116 ("Similar simplification is common used in previous theoretical works [24,9,39,43], but ours don’t have any constraint or regularization on features and stands closer to realistic neural network models.")  needs more justification.
-  line 92: "For simplicity, we assume the dataset is balanced". It seems that neural collapse only happens in the balanced situation. Therefore this assumption is not made due to simplicity.
  
- The experiments are rather limited. The authors present two experiments. The first one is a simulation in the UPLM regime, which confirms the theoretical findings. However, considering, that these have already been proven in this work, the results of this experiment are unsurprising. I would advise the authors to move this to the supp. mat. and use the freed space for studying the realistic case in more detail.\
The second experiment, i.e., the "realistic setting", investigates neural collapse on a VGG-13 net. Considering that such experiments were already performed by [28] (however, with weight decay), I do not see much benefit in this experiment in its current form. 
Furthermore, it seems like the experiments consist of only a single run, so the statistical robustness of the findings cannot be assessed.

- Throughout the paper, it is assumed, that training neural networks without weight decay is a realistic setting. This needs justification, either by a reference or through experiments. It would also be interesting to see, whether the speed of collapse differs for models trained with and without weight decay.


**Typos etc:** 

There are many typos and similar errors in the paper. In the following, I will report ones I found only on page 7.

- line 215: exists -> exist
- line 216: direction -> directions
- line 221: start a new sentence between "it" and "thus".
- line 222: form -> from
- line 228: conditions- > condition
- line 231: turns -> turn
- line 235: justify -> justifies, exist -> exists
- line 236: move -> moving, leads -> lead
- line 240: there is a dot at the start of the line. Further more -> Furthermore
- line 245: algorithm -> algorithms
...

**References** \
as in the paper.
limitations_and_societal_impact: The authors addressed some limitations. These are however not the limitations I see in this work, which are rooted in the unrealistic setting that models are trained without weight decay.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies the optimization dynamics of the layered peeled model - a simplified model for the study of neural collapse in overparametrized deep learning, wherein the last layer features are regarded as free optimization variables. This paper studies this problem in an unconstrained setting; that is, without using weight decay or norm constraints. To this end, the authors tie their problem to that of a maximum margin (min norm) alternative, and show that the former converges (in direction) to the latter, even if their magnitudes diverge. By studying the max margin problem, the authors then show that no strict saddles exist, and thus noisy-gradient descent type algorithms can converge to a global minimum of the problem. Finally, the authors demonstrate this behavior numerically in a toy problem as well as on the Fashion MNIST dataset.
main_review: This paper is interesting, and it contributes to demystifying the neural collapse phenomenon. The sequence of ideas and proof technique is mostly clear. I have the following observations:

Main comments:
* When comparing to similar recent contributions, the authors stress that they study the unconstrained version of the layered peeled model and argue that this is more realistic and reflective of the current practice on training deep learning algorithms. This reviewer is somewhat confused by this, as the experimental results in [29] (which first demonstrated neural collapse) do use weight decay. Moreover, the results here regard convergence in direction, which is typical of the analysis of gradient descent for logistic loss. Naturally, this involves the weights norms diverging. This also separates from what is observed in practical settings, where the weights do not diverge. As a result, I feel like the comment on their setting being "closer to current practice" is a bit overstated. The analysis and proof technique, and the assumptions and settings, is simply different.

* The authors present results for convergence of gradient flow - that is, assuming continuous dynamics. Can the authors comment on the extensions of these ideas to discrete time (stochastic) gradient descent?

* Respectfully, this paper is hard to read at times simply because of the huge number of typos and grammatical mistakes. A non-exhaustive list follows below.

* On their experimental results section, the authors first demonstrate the training dynamics on their ULPM, and then on the Fashion MNIST dataset. In the latter case, the variation of the classifiers norm increases rather decreases, which basically contradicts the analysis presented here. Naturally, there are differences between the practical setting and the one analyzed theoretically, but can the authors expand on the reasons for this behavior?

* On theorem 3.4, the authors show that there exist directions in the tangent space that lead to lower value of the objective function. I'm confused by the inequality on Eq. (13), which is not a strict inequality. As a result, this does not imply necessary a descent direction, as I think is needed for the argument - though I might be missing or misunderstanding something. Could the authors clarify this point?

Non-exhaustive minor comments:
- L53, "use it provide" -> use it to provide
- The sentence on line 77 "To understand how gradient or its variants descent helps deep learning to find solutions with good generalization performance on the test set." has no subject, and no meaning.
- L85: "that critical points other than the good ones always lies" -> critical point lie.
- L98 "within in the k-th class" -> within the k-th class
- L103: "to be simplex equiangular" -> to be a simplex..
- NC2: closing parenthesis missing
- L107: "satisfies neural collapse conditions or is neural collapse solution" -> satisfies the neural collapse conditions or is a neural collapse solution.
- L114: "Similar simplification is common used [..] but ours don’t" -> but ours doesn't.
- Gradient flow is never defined. Instead, the authors just define the gradients, on line 127. I.e, the definition on Eq (14) in the appendix should be brought to the main text.
- Line 157 "collapsed margin minimize" -> minimizes
- L202: "gradient flow move" -> moves
-L 212 "simple gradient descent get stuck" -> gets stuck
- L231 "Now we turns" -> Now we turn
- Extra parenthesis on Eq 12
- "Our next result justify" -> justifies
- "for every suboptimal points, there exist a direction in the tangent space such that move along this direction will leads to a lower objective
value." -> for every suboptimal point, there exists a direction in the tangent space such that moving along this direction will lead to a lower objective value
- Many punctuation marks missing or incorrect (Eq 13, 14, 12, 10, 9, 8)
- "For a large family of stochastic optimization algorithm" -> algorithms
- "we observe that logarithm" -> that the logarithm
- "We doesn’t introduce" -> we don't introduce. 
Again, this list is not exhaustive.
limitations_and_societal_impact: Adequately addressed.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The work studied the inductive bias of the last-layer features and classifiers from neural networks with cross-entropy loss, based on the unconstrained layer-peeled model. The work (i) builds a relationship between the max-margin analysis with the neural collapse and provides the inductive bias analysis to the unconstrained feature, and (ii) shows that the loss function without weight decay enjoys a benign landscape and characterize the neural collapse property of the global minimizer.
main_review: The work provides an interesting study on the theoretical investigation of the neural collapse phenomenon of the last-layer features and classifiers of deep networks based on the layer-peeled model, where the neural collapse has been recently discovered empirically [28]. The presentation of the work is clear and easy to follow. However, the reviewer finds that the two results on max-margin analysis and landscape analysis are a bit disconnected. Below are some more detailed comments: 
 
* Section 3.1: In Theorem 3.1, can the assumption L_CE(t_0) < log 2 be achieved using some initializations? How stringent is this assumption for the result to hold? Some discussion on explaining this could be helpful here. 
 
Also, for the results in Section 3.1, it would be great if one can show that the global solution of (2.3) in the tangent space (or limiting point of the flow of (2.3)) is the neural collapse and KKT solution for 3.1, it could make the story more coherent to Section 3.2. Currently, it seems that the two sections are a bit disconnected.
 
* Section 3.2, the reviewer finds that the claim about loss could be a bit misleading. Although the loss does not have any weight decay, nor constraints at first glance, the analysis is conducted over the tangent space of a manifold ||W||_F^2 +||H||_F^2 = C, which can be reviewed as a constraint on the loss function. 
 
Also, for Theorem. 3.4, it would be better to show a second-order directional descent instead of properties on loss function (zero-order). In the current form of (13), the direction might not strictly decrease the function value.

limitations_and_societal_impact: The result is based on a simple layer-peeled model. It could be interesting how to extend the analysis to study deep networks under more practical settings.

The current study only focuses on the training. It would be interesting how to study generalization and robustness based upon the current analysis.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: Recent empirical work has identified an interesting phenomenon called Neural Collapse (NC), which says features from the trained neural network *converge* to a structural simplex, which can be viewed as the *maximal margin* solution in multi-class setting. This paper studies this phenomenon with cross-entropy loss under the assumption that the features are unconstrained and make the following contributions: 
(1.) Associate the late-time gradient descent dynamics to a constraint optimization problem and relates the limit points of the parameters to KKT points of the optimization problem (thm 3.1).
(2.) The authors provides partial description about the landscape of the loss, in particular, an example showing gradient flow could fail to converge to max margin solutions (NC solutions) . 
main_review: As the multi-class analogue of the binary max margin solution, NC  has received some amount of interest recently. The goal of this paper is to study this phenomenon under cross-entropy loss with $H$, the features unconstrained. This model is a substantial simplication of the original neural network models, whose features are constrained in the sense they all rely on common parameters of the networks. Such models has been previously studied by various authors: (1) When the loss is MSE [3]  (2) cross-entropy with compact support [1] [2]. This work consider the cross-entropy setting *without* compactness assumption, in which the global minima are at infinity. Overall, this paper makes some interesting observations (e.g. Example 3.1), but the originality and significancy is not high enough to merit for publication as explained below. 

(1) the unconstraint setting has been widely studies recently, including mse loss, cross-entropy + compact. Removing the comactness or normalization assumption (e.g. weight decay) does not add much extra deep insight. 
(2) The techniques and framework of the paper is similar to that of [4]. 
(3) In my opinion, the setup "Unconstrained + Late time dynamics" is overly simple and is not of the most interesting case given existing works [1-3]. Early time dynamics and loss landscape are much more interesting and could provide more useful insights under this simple setting. 

In addition, the presentation and clarity of the paper need improvement as detailed below. 

[1] Neural Collapse with Cross-Entropy Loss
[2] On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
[3] Neural collapse with unconstrained features.
[4] Gradient descent maximizes the margin of homogeneous neural networks

Some major points: 

In the abstract, the authors claims that "all critical points are strict saddle points except the only global minimizers", I didn't see a theorem about this in the main text. Do you mean Thm 3.4 imply this claim? If so, please add explanation. Even this hopes, you still need the assumption $q_{min} >0$ (see below) and can not say anything in the early training phase where $q_{min} < 0$. In addition, the "$\leq $" (rather than "$<$") in eq (13)/(14) make the theorem less interesting, is it possible to strengthen it to "$<$"? 

About the terminology "unconstrained layer peeled model": how is this model different than the *unconstrained model* used in e.g. [26]? Why adding the description *layer peeled* which seems unnecessary if they are the same. 


The claim in the introduction "how gradient descent separates data with neural collapse" seem a bit overstate. The statement "how gradient descent separates data" is more accurate to describe the dynamics from $q_{min} <0$ to  $q_{min}  >0$, i.e. from initialization to zero classification error, rather than the terminal phase of training, which is all the NC story about.

Other comments:  
 
Be a bit more verbal, explain the reason for $log 2$ upper bound in Thm 3.1.  Is this use to assume the classification error is zero? 

"Inductive Bias of Gradient Descent" this seems non-conventional. More often than not,  people use *implicit* bias of gradient descent or *inductive* bias of architectures 

Right hand side of equation (4) is missing *min*; in addition, one should use *inf* since the domain is non-compact and the minimizers are at infinity. 

equation after 127, missing negative signs; 

157: "collapsed margin minimize the neural collapse margin": minimize --> maximize. 

170: what is $\alpha(W, H)$ do you mean $(\alpha W, \alpha H)$ ?

171: the exponent in $q_{min}$ should be 1 rather than 1/2; because 1/2 does not make the term scaling invariance; 

Lemma 3.2. : the "~" above $W/H$ is not in the optimal location.   "of the
following minimum-norm separation problem": following --> above or eq (5); 

In Thm C1 in appendix (same as Thm 3.4?) requires $q_{min}>0$, i don't see it in Thm 3.4 ; pleas clarify; 

188: "is the angle between" --> add "cos". Suggestion: group $W$ and $H$ into one variable, say $\Theta = (W, H)$, define an inner product on this space; this should help clean up the notation a bit and avoid repeating using Trace(WW^T) + trace(HH^T) etc; 

I find it helpful to bring up the (\epsilon, \delta) KKT definition to the main text; 


222. "but there also exist some better points close to it thus stochastic gradient  method can easily escape form them": add reference here; form -> from

Thm 3.3 or closely related seems to already appear in previous works under the unconstraint setting; see [1-3] above; 


Section 4 seems to just justify neural collapse empirical and does not provide additional information beyond what was illustrated in the original NC paper; 


————————

Update: 

NC is an interesting direction and the authors have obtained some very promising results. If this paper wasn’t accepted this time, I encourage the authors to polish the presentation, strengthen the results and resubmit the paper.  
limitations_and_societal_impact: Yes; the authors point out some limitation of their work;
review_title: null

Meta-review: This paper studies an asymptotic implicit bias phenomenon termed "neural collapse".  It constitutes a very interesting line of work, however reviewers overall had many concerns, and especially during internal discussion it was highlighted that the paper is very hard to follow and feels unpolished in many places.  Therefore I urge the authors to continue their work, but to be attentive to reviewer concerns, especially on presentation clarity.
Decision: Reject