Average gradient outer product as a mechanism for deep neural collapse
Authors: Daniel Beaglehole
Peter Súkeník
Marco Mondelli
Mikhail Belkin
Venue: NeurIPS 2024 poster (2024)
Keywords: Theory of deep learning
neural collapse
average gradient outer product
kernel methods
feature learning
Primary area: learning_theory
TL;DR: We demonstrate that feature learning through the average gradient outer product is a setting for deep neural collapse.

Abstract: Deep Neural Collapse (DNC) refers to the surprisingly rigid structure of the data representations in the final layers of Deep Neural Networks (DNNs). Though the phenomenon has been measured in a variety of settings, its emergence is typically explained via data-agnostic approaches, such as the unconstrained features model. In this work, we introduce a data-dependent setting where DNC forms due to feature learning through the average gradient outer product (AGOP). The AGOP is defined with respect to a learned predictor and is equal to the uncentered covariance matrix of its input-output gradients averaged over the training dataset. Deep Recursive Feature Machines are a method that constructs a neural network by iteratively mapping the data with the AGOP and applying an untrained random feature map. We demonstrate theoretically and empirically that DNC occurs in Deep Recursive Feature Machines as a consequence of the projection with the AGOP matrix computed at each layer. We then provide evidence that this mechanism holds for neural networks more generally. We show that the right singular vectors and values of the weights can be responsible for the majority of within-class variability collapse for DNNs trained in the feature learning regime. As observed in recent work, this singular structure is highly correlated with that of the AGOP.

Average gradient outer product as a mechanism for deep neural collapse
Daniel Beaglehole*.
Peter Súkeník** *.2
Marco Mondelli²
Mikhail Belkin
'CC San Diego
Instittte of Science and Technology Austria
Abstract
Deep Neural Collapse (DNC) refers to the surprisingly rigid structure the data representations in the final layers of Deep Neural Networks (DNNs). Though the phenomenon has been measured a variety of settings, its emergence is typically explained via data-agnostic approaches, such as the unconstrained features model. In this work, we introduce feature learning through the average gradient outer product (AGOP). The AGOP is defined with respect to a learned predictor and is equal to the uncentered covariance matrix of its input-output, gradients averaged over the training dataset. The Deep Recursive Feature Machine (Deep RFM) is a method that constructs a neural network by iteratively mapping the data with the AGOP and applying untrained random feature map. We demonstrate empirically that DNC occurs in Deep RFM across standard settings computed at each layer. Further, we theoretically explain DNC in Deep RFM in an asymptotic setting and as result kernel learning. We then provide evidence that this mechanism holds for neural networks more generally. In particular, we show that the right singular vectors and values of the weights can be responsible for the majority of within-class variability collapse for DNNs trained in the feature learning regime. As observed in recent work, this singular structure highly correlated with that of the AGOP
detendeennddnt setting where DNC forms due to
consequence of the projection with the AGOP matrix
Introduction
How Deep Neural Networks (DNNs) learn a transformation the data form prediction and what are the properties of this transformation constitute fundamental questions in the theory of deep learning. promising avenue to understand the hidden mechanisms of DNNs is Neural Collapse (NC) [Papyan et al. 2020]. NC is a widely observed structural property of overparametrized DNNs. occurring in the terminal phase of gradient descent training on classification tasks. This property is linked with the performance of DNNs, such as generalization and robustness (Papyan et al. 22020 Su jet al. 2023). NC defined by four properties that describe the geometry of feature representations of the training data in the last layer. Of these, in this paper we focus on the following two, because they are relevant in the context of deeper layers. The within-class variability collapse (NC1) states that feature vectors of the training samples within single class collapse to the common class-mean. The orthogonality or simplex equiangular frame property (NC2) states that these class-means form an orthogonal or simplex equiangular tight frame. While initially observed in just the final hidden layer, these properties were measured for the intermediate layers of DNNs as well [Rangamani et al. 2023 He and Su 2022,, indicating that NC progresses depth- wise. This has lead to the study of Deep Neural Collapse (DNC), a direct wise extension standard NC (Súkeník et al.; 2023).
'Equal contribution. Correspondence to: Daniel Beaglehole (dbeaglehole@bucsd. edu). Peter Súkeník (peter www.eikeiitaaaaaaaaccc...
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The theoretical explanations of the formation of (D)NC have mostly relied on a data aagna-ti model, known as unconstrained features model (UFM) MMxxn et al., 2020 Lu and Steinerberger, [022]. This assumes that DNNs are infinitely expressive and, thus, optimizes for the feature vectors in the last layer directly. The deep UFM (DUFM) was then introduced to account for more layers (Súkeník et al. 2023] Tirer and Bruna 2022]. The (D)UFM has been identified as a useful analogy for neural network collapse, as (D)NC emerges naturally in this setting for a variety of loss functions and training regimes (see Section 2. However, (D)UFM discards the training data and most of the network completely, thus ignoring also the role of learning in DNC formation. This is a serious gap to fully understand the formation of DNC in the context of the full DNN pipeline.
In this paper, we introduce a setting where DNC forms through a learning algorithm that is highly dependent on the training data and predictor iterated linear mapping onto the average gradient outer product (AGOP). The AGOP is the uncentered covariance matrix of the input-output gradients of a predictor averaged over its training data. Recent work has utilized this object to understand various surprising phenomena in neural networks including grokking, lottery tickets, simplicity bias, and adversarial examples (Radhakrishnan al. 2024a]. Additional work incorporated layer-wise linear transformation with the AGOP into kernel machines, to model the deep feature learning mechanism of neural networks [Beaglehole et al. 2023]. The output of their backpropagation-fiiee method, Deep Recursive Feature Machines (Deep RFM). is a standard neural network at i.i.d. initialization, where each random weight matrix is additionally on the input to that layer. Their method was shown to improve performance of convolutional kernels on vision datasets and recreate the edge detection ability of convolutional neural networks.
muthillllippied by the AGOP of kernel machine trained
Strikingly, we show that the neural network generated by this very same method, Deep RFM, consistently exhibits standard DNC with little modification (Section F. We establish in this setting that projection onto the AGOP is responsible for DNC in Deep RFM both empirically and theoretically, We verify these claims with an extensive experimental evaluation, which demonstrates a consistent formation of DNC in Deep RFM across several vision datasets. We then provide theoretical analyses that explain the mechanism for Deep RFM in the asymptotic high-dimensional regime and derive NC formation as a consequence of kernel learning (Section 5). Our first analysis is primarily based on the approximate linearity of kernel matrices when the dimension and the number of data points are large and proportional. Our second analysis demonstrates that DNC is an implicit bias of optimizing over the choice of kernel and its regression coefficients.
We then give substantial evidence that projection onto the AGOP is closely related to DNC formation in standard DNNs. In particular, we show that within-class variability for DNNs trained using SGD with small initialization is primarily reduced by the application of the right singular vectors of the weight matrix and the subsequent multiplication with its singular values (Section 6 These singular structures of weight matrix W are fully deducible from the Gram matrix of the weights at each layer, W W. Radhakrishhan et al. [2024a). Beaglehole et al. [2023) have identified that, in many settings and across all layers of the network, W W is highly correlated with the average gradient outer product (AGOP) with respect to the inputs to that layer, in statement termed the Neural Feature Ansatz (NFA). Thus, the NEA suggests neural networks extract features from the data representations at every layer by projection onto the AGOP with respect to that layer.
Our results demonstrate that (i) AGOP is a mechanism for DNC in Deep RFM, and (ii) when the NFA holds, i.e. with small initialization, the right singular structure of W. and therefore the AGOP, induces the majority of within-class variability collapse in DNNs. We thus establish projection onto the AGOP as a setting for DNC formation that incorporates the data through feature learning.
2 Related work
Neural collapse (NC). Since the seminal paper by Papyan et al. 2020 the phenomenon of neural collapse has attracted significant attention. Galanti et al. [2022] use NC to improve generalization
bounds for transfer learning, while Wang et al. 2023 discuss transferability capabilities of pre-trained
models based on their NC on the target distribution. Haas et al. [0022] argue that NC can lead to
improved OOD detection. Connections between robustness and NC are discussed in Su et al. 2023
For survey, we refer the interested reader to Kothapalli] (2033]
The main theoretical framework to study the NC is the Unconstrained Features Model (UFM) Mixon et al. 2020 Lu and Steinerberger; 2022]. Under this framework, many works have shown the emergence of the NC, either via optimality and/or loss landscape analysis [Weinan and Wojtowytsch Lu and Steinerberger, 2022 Zhou et al. 2022 via the study of gradient-based optimization
2022

---

2022 aan et al. 2022 Wang et al. 2022]. Some papers also focus on the generalized class- imwa lacce setting, where the NC does not emerge its original formulation (Thrampoulidis
[Ji et al.
et al.
2022 Fang et al. [02 Hong and Ling, 2023 Deviating from the strong assumptions of the model, line of work focuses on gradient descent in homogeneous networks [Poggio and Liao Xu et al.
UFM
2020
2023 Kunin et al. 2022]. while others introduce perturbations (Tirer et al. [022].
AGOP feature learning. The NEA was shown to capture many aspects of feature learning in neural networks including improved performance and interesting structural properties. The initial work on the NFA connects to the emergence of spurious features and simplicity bias, why pruning DNNs may increase performance, the "lottery ticket hypothesis", and mechanism for grokking in vision settings [Radhakrishnan et al. 2024a]. Zhu et al. [2023] connect large learning rates and catapults in neural network trainigg better alignment of the AGOP with the true features. Raadhakrishaan et al. [2024b] demonstrate that the AGOP recovers the implicit bias of deep linear networks toward low- rank solutions in matrix completion. [Beaglehole et al. [0224 identify that the NEA is characterized by alignment of the weights to the pre-activation tangent kernel.
Background and definitions
3.1 Notation
For simplicity of description, we assume a class-balanced setting, where = Kn is the total number of training samples, with K being the number of classes and the number of samples per class. Note however that our experimental results and asymptotic theory hold in the general case that the classes of unequal size. We will in general order the training samples such that the samples of the same class are grouped into blocks. With this ordering, the labels y E IR X can be written as T @ 1,, where J denotes K x K identity matrix, X denotes the Kronecker product and 1, a row vector of all-ones of length n.
For matrix A E IRd xda and column vector Rd x1 the operation A@ v, divides all elements of each row of A by the corresponding element c. We define the norm of e gdxn ||A| e gax1 as the column-wise fy-norm and not the matrix norm.
Both a DNN and Deep RFM of depth L can be written as:
f(x) = m ( (m))))))
where my is an affine map and is a non-lineat activation function. For neural networks, the linear map of my is the application of a single weight matrix Wi, while for Deep RFM the linear transformation is product of weight matrix and a feature matrix M written W,M,!' 1/2 The training data X E RdixN X is stacked into columns and we let X; be the feature representations the training data after / layers of DNN or Deep RFM before the linear layer for I > 1, with X1 X.
3.2 Average gradient outer product
The AGOP operator acts with respect to dataset X e Ro and any model, that accepts inputs from the data domain f dd x1 - RK*x where K is the number of outputs. Writing the (transposed)
Jacobian of the model outputs with respect to its inputs 0f(x) Rdox the AGOP is defined as:
8z
K N L dx δx
af(xa) of(Ea)
AGOP(F,X)
(I)
Note while the AGOP stated such that the derivative is with respect to the immediate model inputs will also consider the AGOP where derivatives taken with respect to intermediate representations of the model. This object has important implications for learning because the AGOP of a learned predictor will (with surprisingly few samples) resemble the expected gradient outer product (EGOP) of the target function (Yuan et al. 2023). While the AGOP is specific to a model and training data, the EGOP is determined by the population data distribution and the function to be estimated, and contains specific useful information such as low-rank structure, that improves prediction (Trieedi al. 2014).
Remarkably, it was identified in Radhakrishnan et al. [2024a] Beaglehole et al. 2023 that neural networks will automatically contain AGOP structure in the Gram matrix of the weights in all layers
3

---

of the neural network, where the AGOP at each layer acts on the sub-network and feature vectors at that layer. Stated formally, the authors observe and pose the Neural Feature Ansatz (NFA):
Ansatz 3.1 (Neural Feature Ansatz [Radhakrishnan al. 2024a]). ). Let I be a depth- neural network trained on data X using stochastic gradient descent. Then, for all layers [T]]
K (x(s)) Of(rr)) ) Or
W Wi,
1.
N c=1==1
Ox
The second argument is the AGOP of the neural network where the gradients are taken with respect to the activations at layer and not the initial inputs. The correlation function p accepts two matrix inputs of shape (p. q) and returns the cosine similarity of the matrices flattened into vectors of length pq. whose value is in [-1,1]. Note Radhakrishnan et al. [2024a] formulates this similarity as proportionality, which is equivalent to correlation exactly equal to 1.
Crucially, the AGOP can be defined for any differentiable estimator, not necessarily neural network. In Radhakrishhan et al. [2024a], the authors introduce kernel learning algorithm, the Recursive Feature Machine (RFM), that performs kernel regression and estimates the AGOP of the kernel machine in an alternating fashion, enabling recursive feature learning and refinement through AGOP.
3.3 Deep RFM
Subsequent work introduced the Deep Recursive Feature Machine (Deep RFM) to model deep feature learning neural networks [Beaglehole et al. 2023]. Deep RFM iteratively generates representations by mapping the input to that layer with the AGOP of the model w.r.t. this input, and then applying
an untrained random feature map. To define the Deep RFM, let (A2)/=1 7.+1 Id d; - R be set of kernels. For the case of notation, we will write ky (X/. X1) for matrix of kernel evaluations between columns of X and X₁. We then describe Deep RFM in Algorithm
Algorithm 1 Deep Recursive Feature Machine (Deep RFM)
input X1, Y, [ 1++ T., ( (kernels: (ki),. depth: L., feature maps: {f1) ridge: 7)
output a1--1: {M]
for 1 E 1,.., do
Normalize the data, X₁ - X1 |||
Learn coefficients, aq (KI, + + Construct predictor, fi(.) = (Xt,
0file'e) os(le!)) Compute AGOP: Mi _c,i=1 2z
Ois
Transform the data X 2/ (M XI). end for
Normalize the data, X - X 2 N1+1ll Learn coefficients.
a++1 = (XLL XL+1) +
Note that the Deep RFM as defined here considers only one AGOP estimation step in the inner- optimization of RFM, while multiple iterations are used in [Baglehhle et al.) (2023). The high- dimensional feature maps oi(-) are usually realized as aWW bi), where Wi is a matrix with standard Gaussian or uniform entries, b, is an optional bias term initialized uniformly random, and is the ReLU or cosine activation function. Thus, Φ typically serves as random features generator.
The single loop in the Deep RFM represents reduced RFM learning process. The RFM itself is based on kernel ridge regression, therefore we introduce the ridge parameter 7,
3.4 Deep Neural Collapse
We define the feature vectors 2 of the i-th sample of the e-th class as the input to my For neural networks, define E the feature vectors produced from mi-1 before the application of where I ? 2. For Deep RFM, we define E the feature vectors produced by the application of AGOP, i.c.

---

Input data
Layer
Layer
Layer 13
Layer 19
0.8
0.6
0.1
0.2
0.0
-0.2
1.0
0.1
8.1
0.0
Figure I: Neural collapse with Deep RFM on (A) CIFAR-IO and (B) MNIST. The matrix of inner products of all pairs of points in X₁ extracted from layers / E {1,3, 7, 13, 19) of Deep RFM. The columns show the Gram matrices of feature vectors transformed by the AGOP from Deep RFM, Hall- The data are ordered so that points of the same class are adjacent to one another, arranged from classes 10. Deep RFM uses non-linearity a(-) cos(-) in (A) and a--) = RcLU(-) in (B).
(XI - A + |
z/ = M, 1/2 Je again where > 2. In both model types, we let To. = x Let p := Σ d/s Fr = 1 - #/4 be the corresponding class means, and μ², j RRdi the matrices of class means.
NC1 defined in terms of the within-class and between-clat variability at layer
K
Σ
Σ
1 K
Σ
( - P(OTT
Definition 3.2. the context of this work, NC is achieved at layer / the following two properties satisfied:
The within-class variability at layer is O. This property can be stated for the features either after before the application of the activation function In the former case, the condition requires via = 2y for all i, (I.....n} [ni] (or, in matrix notation, X = n² X 1, ); in the latter, 2/1 = E for all i.j [n]] (or, in matrix notation, X; = X 1.))
DNC1:
The class-means at the 1-th layer form either an orthogonal basis or a simplex equiangular tight frame (ETF). As for DNC1, this property can be stated for features after or before a. We write the ETF class-mean covariance ΣETF & (1 + RL)44K
DNC2:
1 kc- In the former case, the condition requires either p' or (n²)T p'x the latter (īt') T I or (7²) EETE In this work, we will measure this condition on the centered and normalized class means, n²'. We let μd (p' m(a) + (n' 1'e) or p (iL Ma) 2 (Tz - -a)) depending on whether measure collapse on X or X, respectively.
4 Average gradient outer product induces DNC in Deep RFM
We demonstrate that AGOP a mechanism for DNC in the Deep RFM model. In this section, we provide extensive empirical demonstration that Deep RFM exhibits -NC-NCI and NC2 and that the induction of neural collapse due to the application of AGOP.
We visualize the formation of DNC in the input vectors of each layer of Deep RFM. For | > 2, let X,
M1 be the feature vectors at layer 1 projected onto the square root of the AGOP, M, 1/2 at that layer. Otherwise, define X X1 as the untransformed representations. In Figure we show the Gram matrix, X, X₁, of centered and normalized feature vectors X, Xx
+ X
A extracted from several layers Deep RFM, as presented in Algorithm

---

trained on CIFAR-10 10 and MNIST (see Appendix for similar results on SVHN). Here the global mean My is subtracted from each column of X. After 18 layers of DeepRFM. the final Gram matrix is approximately equal to that of collapsed data, in which all points are at exactly their class means, and the centered class means form ETF. Specifically, all centered points of the same class have inner product 1, while pairs of different class have inner product - -K 1)- Note that, like standard neural networks, Deep RFM exhibits DNC even when the classes are highly imbalanced, such for the SVHN dataset (Figure 5 in Appendix E).
We show collapse occurring as a consequence of projection onto the AGOP, across all datasets and across choices of feature map (Figures B and in Appendix E) We observe that the improvement in
NCI is entirely due to M, 1r2 and the random feature map actually worsens the NC1 value. This result intuitive as Deep RFM deviates from a simple deep random feature model just by additionally projecting onto the AGOP with respect to the input at each layer, and we do not expect random feature maps to induce neural collapse on their own. In fact, the random feature map will provably separate nearby datapoints, as this map is equivalent to applying rapidly decaying non-linear function to the inner products between pairs of points (described formally in Appendix A).
5 Theoretical explanations for DNC in Deep RFM
We have established empirically in range of settings that Deep RFM induces DNC through AGOP. We now prove that DNC in Deep RFM (1) occurs in an asymptotic setting, and (2) can be viewed as an implicit bias of RFM kernel learning algorithm.
5.1 Asymptotic analysis
Many works have derived that, under mild conditions on the data and kernel, non-linear kernel matri- ces of high dimensional data are well approximated by linear kernels with a non-linear perturbation [Karoui 2010 Adlam et al. 2019 Hu and Luj 2002). In this section, we provide proof that Deep RFM will induce DNC when the predictor kernels 1 and random feature maps {Φt} satisfy this property. In particular, in the high-dimensional setting in these works, non-linear kernel matrices, written k(X)=
for simplicity, are well approximated by
k(X) R 711 + X XI,
where X > 0 is the perturbation parameter. Following Adlam and Pennington 2020 for additional simplicity we consider centered kernels, where y = O. We associate with Deep RFM two separate (unrestricted) centered kernels h and kmap. with perturbation parameters Ā and Amap, corresponding to the predictor kernels and random feature maps respectively.
We will show that Deep RFM exhibits exponential convergence to DNC, and the convergence rate e
depends on the ratio a/Amap- These two parameters modulate the distance of non- non-linear ridge regression with each kernel to linear regression, and therefore the extent of DNC with Deep RFM. Namely, as we will show, if 1e is close to the linear kernel, i.c. if À is small, then the predictor in each layer resembles interpolating linear regression, an ideal setting for collapse through the AGOP. On the other hand, if kmap is close to the linear kernel, i.e., if Xmaa is small, then the data is not easily linearly separable. In that case, the predictor will be significantly non- con-linear in order to interpolate the labels, deviating from the ideal setting. We proceed by explaining specifically where & and Kmap appear in Deep RFM and why linear interpolation induces collapse. We then conclude with the statement of our main theorem and an explanation of its proof.
We describe the Deep RFM iteration we analyze here, which has a slight modification. Instead of normalizing the data at each iteration, we scale the data by value =- at each iteration. This modification is sufficient to control the norms of the data, and is easier to analyze. The recursive procedure begins with a dataset X, at layer in Deep RFM, and constructs X, from the AGOP M, of the kernel ridgeless regression solution with kernel k; = R. After transforming with the AGOP and scaling, the data Gram matrix layer is transformed to,
X Xis1 = K -xx, M,Xi,
(3)

---

for K == 1 21(1+ Ammp). Then, we will apply non-linear feature map Pmap corresponding to kernel Kmap and write the corresponding Gram matrix at depth ++
X X++1 = Pmap (maap(((1) P
= kmap (X1+1) X X1+1 + Amaa_.
(4)
Intuitively, the ratio \/ A/Amap determines how close linear k: is when applied the dataset, and therefore the rate of NC formation.
We now explain why Deep RFM with a linear kernel is ideal for NC, inducing NC in just one AGOP application. To see this, note that ridgeless regression with a linear kernel is exactly least-squares regression one-hot encoded labels. In the nigh-dimensional setting consider, should find a linear solution f()) = 3 æ that interpolates the labels, the AGOP of f is B3 Since we interpolate the data, 3 x y for all (z, y) input/label pairs, applying the AGOP collapses the data to 3,3 x - By. Therefore, NC will occur in a single layer of Deep RFM when Amap >> X, in which case the data Gram matrix has large minimum eigenvalue relative to the identity perturbation to k, so the kernel regression is effectively linear regression. Note that this theory offers an explanation why non-linear activation is needed for DNC in neural networks: Amap O when & mapp is linear, preventing this ideal setting.
We prove that DNC occurs in the following full-rank, high-dimensional setting. Assumption 5.1 (Data is high dimensional). We assume that the data has dimension d i/ n. Assumption 5.2 (Data full rank). We assume that the initial data Gram matrix X X, has minimum eigenvalue at least X 0.
The assumptions that the Gram matrix of the data is full-rank and high-dimensional needed only one requires neural collapse in every layer of Deep RFM, starting from the very first one. In contrast, we consider collapse starting at any given later layer of Deep RFM, then we only need that the smallest eigenvalue of the corresponding feature map is bounded away from O. This in turn requires that the number of features at that layer is greater than the number of data points, a condition which routinely satisfied by the overparameterized neural networks used in practice.
We present our main theorem.
Theorem 5.3 (Deep RFM exhibits neural collapse). Suppose we apply Deep RFM any dataset X with labels Y E IR N*K choosing all {bi) and {ka]i as the feature map 4 mapp and kernel above, with no ridge parameter (7 = 0). Then, there exists universal constant C > 0, such that for any CAmmp (1 e), and for all layers LE (2...., L} of Deep RFM,
0 < E & 1, provided X < Am1p)m
X YTyl & (1,,) YTV - + 0(52> xmp)
This theorem immediately implies exponential convergence to NC1 and NC2 up to error O (T.x2 m a small value determined by the parameters of the problem. As a validation of our theory, we see that this exponential improvement in the DNC metric occurs in all layers (see Figures 3 and 4 Note the exponential rate predicted by Theorem 5.3 and observed empirically for Deep RFM is consistent with the exponential rate observed in deep neural networks [He and Su 2022
The proof for this theorem is roughly as follows. Recall that, by our argument earlier in this section, linear kernel will cause collapse within just one layer of Deep RFM. However, in the more general case we consider, a small non-linear deviation from linear kernel, X, is introduced. Beginning with the first layer, partial collapse occurs if the ratio X/ A/Ambu is sufficiently small. Following the partial collapse, in subsequent layers, the data Gram matrix will sufficiently resemble the collapsed matrix, so that the non-linear kernel solution on the collapsed data will behave like the linear solution, leading to further convergence to the collapsed data Gram matrix, fixed point of the Deep RFM iteration (Lemma
5.2 Connection parametrized kernel ridge regression
Next, we demonstrate that DNC emerges in parameterized kernel ridge regression (KRR). In fact, DNC arises as a natural consequence of minimizing the norm of the predictor jointly over the choice of kernel function and the regression coefficients. This result proves that DNC is implicit bias of
7

---

kernel learning. We connect this result to Deep RFM by providing intuition that the kernel learned through RFM effectively minimizes the parametrized KRR objective and, as a consequence, RFM learns kernel matrix that is biased towards the collapsed Gram matrix Y Y
We define the parametrized KRR problem. Consider positive semi-definite kernels (p.s.d.) has- R x - R of form ky (2, 2) = d[| =| M); where || = = v(2 =) M 2), is strictly decreasing, strictly positive univariate function s.t. (00) = 1, and M is a p.s.d. matrix. This class of kernels covers wide range of kernels including the Gaussian and Laplace we in our experiments. Let Har be the Reproducing Kernel Hilbert Space (RKHS) corresponding to our chosen kernel KM where functions f E HM map to single output class. The parametrized kernel ridge regression for a single output class (K = 1) with ridge parameter corresponds to the following optimization problem over such f and M on dataset X and labels Y e RRXN;
inf (((()- IM2
(S)
When predict outputs over K >> classes, we jointly optimize over K independent ecalar-vluued S. one for each output class, and, again, a single choice of matrix M. The objective function is then sum the individual objective values over all classes. Our modification for multiple outputs corresponds to how Deep RFM and kernel ridge regression, more generally, are solved in practice a single kernel matrix is used across all classes while kernel coefficients are computed for each class independently. Note this modification for multiple classes is also equivalent to optimizing over single vector-valued function where the function space HM a particular vector-valued RKHS (see Appendix B for more details).
Parameterized KRR differs from standard KRR by additionally optimizing over the choice of the kernel through the matrix M. For all M. including the optimal one for Problem (5). an analog of the representer theorem for vector-valued RKHS can be shown (Micchelli and Pontil 2004
and the optimal solution to (5) can be written as f(=) Ee,==1, aeslias (Fes, 2), where Oea is K- dimensional vector. Let us therefore denote A the matrix of stacked columns Oei. We can re-formulate Problem (5) as the following naite-dimensioall optimization problem:
inf tr ((Y- AYM)(Y - AkM)7) + (ut((kMAAA)) A.M
(6)
where, abusing notation, km := (M,, X) is the matrix of pair-wiss wise kernel evaluations on the data.
We now relax the optimization over matrices M by optimizing over all p.s.d., entry-wise non-negative kernel matrices & e Ra*K x Ra* with ones on diagonal and compute the optimal value of the following relaxed parametrized KRR minimization:
inf ir ((Y A AK)²) + pur(KA A). A.K
Optimizing over all p.s.d., entry-wise non-negative & with ones diagonal is not always equivalent to optimizing over all M. Thus, in general, this provides only a relaxation. However, if the data X is full column rank (as in our asymptotic analysis), then the optimizations (6) and 7 have the same solution and, thus, the relaxation is without loss of generality. We establish this equivalence formally
in Appendix B
We are now ready to state our optimality theorem. The proof is deferred to Appendix C
Theorem 5.4. The unique optimal solution to the (relaxed) parametrized kernel ridge regression objective (Problem 7) is the kernel matrix k* exhibiting neural collapse, k* IK & (1,1,)== Y
Finally, we informally connect this result to RFM. We argue that RFM naturally minimizes the parametrized kernel ridge regression objective (Problem 5 through learning the matrix M. This claim is natural since the RFM is a parametrized kernel regression model, where M is set to be the AGOP. This choice of Mahalanobis matrix should minimize (5) as AGOP captures task speciffc low-rank structure, reducing unnecessary variations of the predictor in irrelevant directions Chen et al. [0023.. Then, given any setting of M, RFM is conditionally optimal w.r.t. the parameter o, as this method simply solves the original kernel regression problem for each fixed M. Further, as argued in Section[5.] choosing & to dot product kernel and M to be the AGOP of an interpolating linear classifier implics that has k. the optimal solution outlined in Theorem 5.4
8

---

Layer
Layer
Layer
Layer 4
Layer
A
tr [ww
tx (
0.03
5.011
200
200
200
107
B
30.0
tr( Dw
tr
0.03
120
240
63
128
240
202
48 122
240 adc
60
120 260
ado
120 160 203
Epachs
Epochs
Epochs
Epochs
Epocits
ReLU(USV ((X))
SV 1 O(X)
(X)
Figure 2: Feature variability collapse from different singular value decomposition components in (A) an MLP on MNIST, and (B) a ResNet on CIFAR-10. We measure the reduction in the NCI metric throughout training at each of five fully-connected layers. Each layer is decomposed into its input, ((X), the projection onto the right singular space of W, SV SVVTSV and then U, the left singular vectors of W, and the application of the non-linearity.
Within-class variability collapse through AGOP in neural networks
Thus far, we have demonstrated that Deep RFM exhibits DNC empirically, and have given theoretical explanations for this phenomenon. Here, we provide evidence that the DNC mechanism of Deep RFM, i.e., the projection onto the AGOP, responsible for DNC formation typical neural networks, such as MLPs, VGG, and ResNet trained by SGD with small initialization. We do so by demonstrating that within-class variability collapse occurs predominantly through the multiplication by the right singular structure of the weights. As the NFM, which is determined by the right singular structure of each weight matrix, is highly correlated with the AGOP, our results imply the AGOP is responsible for NC1 progression.
fully-connected layer of neural network consists two components: multiplication by weight matrix W,, followed by the application of an element-wise non-linear activation function (termed the non-linearity). Both components of the layer crucially contribute the inference and training processes of neural network. The question we address in this section is whether the non-linearity or the weight matrix is primarily responsible for the improvement DNC1 metrics.
We udditionally decompose the weight matrix into its singular value decomposition W; = U.S.V,T, viewing a fully-connected layer as first applying S.V,T, then applying the composition with U, ((U,.). This decomposition allows us to directly consider the effect of the NFM, W,T W = ViSP, and therefore the AGOP, DNC formation.
The grouping of layer into the right singular structure and the non-linearity with the left singular vectors is natural, DNC1 metrics computed on this decomposition are identi- cal to the metrics computed using the whole W; and then the non-linearity. We see this fact by considering the standard DNC1 metric tr(Ew) tr(Eg) [Tirer et al. 2023 Ranga mani et al.) 2023). A simple computation using the cyclic property of the trace gives tr(Ew) (r(BB)) ur(UE UT) tr(UEgU
tr(Ew) tr(EB) -1, where Σ matrices refer to the within-class variability after the application of SVT, while the E matrices correspond to the output of the full weight matrix.
While both the non-linyarity and S.V.T. and only these two components, can influence the DNC1 metric, we demonstrate that S.V. is responsible for directly inducing the majority within-class variability collapse in neural networks trained by SGD with small initialization. We verify this claim

---

by plotting the DNC1 metries of all layers of an MLP and ResNet network trained on MNIST and CIFAR-10. respectively (Figure where each layer is decomposed into its different parts - (1) the input to that layer, (2) the embedding after multiplication with Si V and (3) the embedding after multiplication with the left singular vectors U, and application of the non-linearity &-
We see that the ratio of within-class to between-class variability decreases mostly between steps (I) and (2), due to the application of the right singular structure. Similar results are in Appendix E for all combinations of datasets (MNIST, CIFAR-I0, SVHN) and architectures (MLP, VGG, and ResNet). This is a profound insight into the underlying mechanisms for DNC that of independent interest, especially given we train with a standard algorithm across many combinations of datasets and models.
We note that, while in this setting the ReLU does not directly reduce NC1, the ReL.U is still crucial for ensuring the expressivity of the feature vectors. Without non-linearity, the neural network cannot interpolate the data or perform proper feature learning, necessary conditions for DNC to occur at all.
These results are in regime where the NFA holds with high correlation, with ResNet and MLP having NEA correlations at the end of training (averaged over all layers and seeds) of 0.74 + 0.17 and 0.74+0.113 on CIFAR-I0 and MNIST, respectively (see Appendix L for values across all architectures and datasets). Therefore, as W, W = VT S.V,T should project into a similar space as the AGOP. We note that the matrix quantities involved are gigh-dimensional, and the trivial correlation between two i.i.d. uniform eigenvectors of dimension d concentrates within + +Od-11/2) For the width 512 weight matrices considered here, this correlation would be approximately (at most) 0.04. Therefore, we conclude that the AGOP structure can decrease within-class variability in DNNs.
Unlike DNC1, we have observed a strong DNC2 progression only in the very last layer. This is due to the fact that DNC2 was observed in the related work Rangamani et al. [2023), Parker et al. [2023] in a regime of large initialization. In contrast, the NFA holds most consistently with small initialization [Radhakrishnan et al. 2024a Beaglehole et al. 2024), and in this feature learning regime, the low-rank bias prevents DNC2 L. et al. 2020 Sūkeník et al. 2024).
7 Conclusion
This work establishes that deep neural collapse can occur through feature learning with the AGOP. We bridge the unsatisfactory gap between DNC and the data with previous work mostly only focusing on the very end of the network and ignoring the training data.
We demonstrate that projection onto the AGOP induces deep neural collapse in Deep RFM. We validate that AGOP induces NC in Deep RFM both empirically and theoretically through asymptotic and kernel learning analyses.
We then provide evidence that the AGOP mechanism of Deep RFM induces DNC in general neural networks. We experimentally show that the DNC1 metric progression through the layers can be mostly due to the linear denoising via the application of the center-right singular structure. Through the NFA, the application of this structure is approximately equivalent to projection onto the AGOP, suggesting that the AGOP directly induces within-class variability collapse in DNNs.
Acknowledgements
We acknowledge support from the National Science Foundation (NSF) and the Simons Foundation for the Collaboration on the Theoretical Foundations of Deep Learning through awards DMS-2031883
and #81 #814639 as well as the TILOS institute (NSF CCF-21 12665). This work used the programs (1) XSEDE (Extreme science and engineering discovery environment) which is supported by NSF grant numbers ACI- 154-154, and (2) ACCESS (Advanced cyberinfrastructure coordination ecosystem: services & support) which is supported by NSF grants numbers #2138259, #2138286, #2138307, #2137603, and #2138296 Specifically, we used the resources from SDSC Expanse GPU compute nodes, and NCSA Delta system, via allocations TG-CIS220009. Marco Mondelli is supported by the 2019 Lopez-Loreta prize. We also acknowledge useful feedback from anonymous reviewers.
10

---

References
Ben Adlam and Jeffrey Pennington. The neural tangent kernel in high dimensions: Triple descent and multi-scale theory of generalization. In International Conference Machine Learning, pages 74-84. PMLR, 2020.
Ben Adlam, Jake Levinson, and Jeffrey Pennington. A random matrix perspective mixtures of nonlinearities for deep learning. arXiv preprint arXiv:1912.00827, 2019.
Daniel Beaglchole, Adityanarayanan Radhakrishnan, Parthe Pandit, and Mikhail Belkin. Mechanism of feature learning in convolutional neural networks. arXiv preprint arXiv:2309.0083 00330000077,,, 2023.
Daniel Beaglehole, loannis Mitliagkas, and Atish Agarwala. Gradient descent induces align- ment between weights and the empirical ntk for deep non-linear networks. arXiv preprint arXiv:2402.05227, 2024.
Yunlu Chen, Yang Li, Keli Liu, and Feng Ruan. Kernel learning in ridge regression" automatically" yields exact low rank solution. arXiv preprint arXiv:23 173/6 2023.
Youngmin Cho and Lawrence Saul. Kernel methods for deep learning, Advances in neural information processing systems, 22, 2009.
Carlo Ciliherto, Youssef Mroueh, Tomaso Poggio, and Lorenzo Rosasco. Convex learning of multiple tasks and their structure. In International Conference on Machine Learning, pages 1548-1557, PMLR, 2015.
Cong Fang, Hangfeng He, Qi Long, and Weijie J Su. Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training. In Proceedings of the National Academy of Sciences (PNAS), volume 18, 2021.
Tomer Galanti, András György, and Marcus Hutter. Improved generalization bounds for transfer learning via neural collapse. In First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward ICML, 2022.
Jarrod Haas, William Yolland, and Bernhard T Rabus. Linking neural collapse and normalization with improved out-of-distribution detection in deep neural networks. Transactions Machine Learning Research (TMLR), 2022.
X. Y. Han, Vardan Papyan, and David L Donoho. Neural collapse under mse loss: Proximity to and dynamics on the central path. International Conference on Learning Representations (ICLR), 2022.
Hangfeng He and Weijie J Su. law of data separation in deep learning. arXiv preprint 1rXi::2210.17020, 2022.
Wanli Hong and Shuyang Ling. Neural collapse for unconstrained feature model under cross-entropy loss with imbalanced data. arXiv preprint arXiv:2309.09725, 2023.
Hong Hu and Yue M Lu. Universality laws for high-dimensional learning with random features. IEEE Transactions on Information Theory, 69(3):1932-19644 2022.
Wenlong Ji, Yiping Lu, Yiliang Zhang, Zhun Deng, and Weijie Su. An unconstrained layer-peeled perspective on neural collapse. In International Conference on Learning Representations (ICLR), 2022.
Noureddine El Karoui, The spectrum of kernel random matrices. The Annals of Statistics, pages 1-50, 2010.
Vignesh Kothapalli. Neural collapse: A review on modelling principles and generalization. In Transactions Machine Learning Research (TMLR), 2023.
Daniel Kunin, Atsushi Yamamura, Chao Ma, and Surya Ganguli. The asymmetric maximum margin 2022.
bias of quasi-homogeneous neural networks. arXiv preprint
11

---

Zhiyuan Li, Yuping Luo, and Kaifeng Lyu. Towards resolving the implicit bias of gradient descent for matrix factorization: Greedy low-rank learning, arXiv preprint arXiv:2012.09839, 2020.
Jianfeng Lu and Stefan Steinerberger. Neural collapse under cross-entropy loss. In Applied and Computational Harmonic Analysis, volume 59, 2022.
Charles Micchelli and Massimiliano Pontil. Kernels for multi task learning. Advances neural information processing systems, 17, 2004.
Dustin G Mixon, Hans Parshall, and Jianzong Pi. Neural collapse with unconstrained features. arXiv preprint ar arXiv:2011.11111 2020.
Vardan Papyan, X. Y. Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. In Proceedings of the National Academy of Sciences (PNAS), volume 117, 2020.
Liam Parker, Emre Onal, Anton Stengel, and Jake Intrater. Neural collapse in the intermediate hidden layers classification neural networks. arXiv preprint arXiv:2308.02760, 2023.
Tomaso Poggio and Qianli Liao. Explicit regularization and implicit bias in deep network classifiers trained with the square loss. arXiv preprint arXiv:2101.00072, 2020.
Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit, and Mikhail Belkin. Mechanism for feature learning in neural networks and backpropagation free machine learning models. Science, 143366690):46111444 1467, 2024a.
Adityanarayanan Radhakrishnan, Mikhail Belkin, and Dmitriy Drusvyatskiy. Linear recursive feature machines provably recover low-rank matrices. arXiv preprint arXiv:2401.04553, 2024b.
Akshay Rangamani, Marius Lindegaard, Tomer Galanti, and Tomaso Poggio. Feature learning in deep classifiers through intermediate neural collapse. Technical Report, 2023.
Jingtong Su, Ya Shi Zhang. Nikolaos Tsilivis, and Julia Kempe. On the robustness of neural collapse and the neural collapse of robustness. arXiv preprint arXiv:2311.07444 2023.
Peter Súkeník, Marco Mondelli, and Christoph Lampert. Deep neural collapse is provably optimal for the deep unconstrained features model. arXiv preprint arXiv:2305.13115, 2023.
Peter Súkeník, Marco Mondelli, and Christoph Lampert. Neural collapse versus low-rank bias: Is deep neural collapse really optimal? arXiv preprint 4rXiv:2405.14468. 2024.
Christos Thrampoulidis, Ganesh Ramachandra Kini, Vala Vakilian, and Tina Behnia. Imbalance trouble: Revisiting neural-collapse geometry. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
Tom Tirer and Joan Bruna. Extended unconstrained features model for exploring deep neural collapse. In International Conference on Machine Learning (ICML), 2022.
Tom Tirer, Haoxiang Huang, and Jonathan Niles-Weed, Perturbation analysis of neural collapse. arXiv preprint 10. 16658, 2022.
Tom Tirer, Haoxiang Huang, and Jonathan Niles-Weed Perturbation analysis of neural collapse. In International Conference on Machine Learning, pages 34301-34329, PMLR, 2023.
Shubhendu Trivedi, Jialei Wang, Samory Kpotufe, and Gregory Shakhnarovich. A consistent estimator of the expected gradient outerproduct. In UAI, pages 819-828, 2014.
Peng Wang, Huikang Liu, Can Yaras, Laura Balzano, and Qing Qu. Linear convergence analysis of neural collapse with unconstrained features. In OPT 2022: Optimization for Machine Learning (NeurIPS 2022 Workshop), 2022.
Zijian Wang. Yadan Luo, Liang Zheng, Zi Huang, and Mahsa Baktashmotlagh. How far pre-trained models are from neural collapse on the target dataset informs their transferability. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 5549-5558, 2023.
12

---

E Weinan and Stephan Wojtowytsch. On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers. Mathematical and Scientific Machine Learning. 2022.
Max A Woodbury. Inverling modified matrices. Department of Statistics, Princeton University, 1950.
Mengjia Xu, Akshay Rangamani, Qianli Liao, Tomer Galanti, and Tomaso Poggio. Dynamics in deep classifiers trained with the square loss: Normalization, low rank, neural collapse, and generalization bounds. In Research, volume 6, 2023.
Gan Yuan, Mingyue Xu, Samory Kpotufe, and Daniel Hsu. Efficient estimation of the central mean subspace via smoothed gradient outer products. arXiv preprint urXiv:2312.15465 2023,
Jinxin Zhou, Xiao Li, Tianyu Ding, Chong You, Qing Qu, and Zhihui Zhu. On the optimization landscape of neural collapse under mse loss: Global optimality with unconstrained features. In International Conference on Machine Learning (ICMI.), 2022.
Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishhan, and Mikhail Belkin. Catapults in sgd: spikes in the training loss and their impact on generalization through feature learning. arXiv preprint arXiv:2306. 200.:0013600045,, 2023.
13

---

A Random features cannot significantly reduce within-class variability
We state the following proposition (first appearing in Cho and Saul (0099] that random features mapping with ReLU will separate distinct data points, especially nearby ones. Therefore, we expect theoretically that NC1 metrics will not improve due to this random feature map.
Proposition A.1. (Cho and Saul [2009[] Let I. Rd be two fixed vectors of unit length. Let W Dxd be a weight matrix whose entries are initialized i..dd from Assum Then, for (-) (eLU(.).
Assume I y = r.
(lo(Wa)"o(Ww)) B(a(Wx)) (( = (sin 0 (TT - 0) cos 0)
(8)
where 0 cos-Ar. Moreover, the variance of this dot product scales with 1/D, and the product is sub-exponential. making it well-concentrated.
This proposition gives a relationship between the dot product x: y of two vectors and the dot product of the outputs of their corresponding random feature maps (Wr) o y). This is relevant because for unit vectors, the dot product is direct indication of the distance between the two vectors.
This means that the distances between data points generally increase, but not drastically. They increase irrespective of the angle between data points, however they tend to expand relatively more, the angle 0 is close to O. Therefore, the DNC1 metric should on average marginally increase or stay constant. This is in agreement with Deep RFM measurements on all datasets (Appendix E). for both ReLU and cosine activations. Importantly, this also supports the connection between neural networks and Deep RFM, since in DNNs the DNCI metric did not significantly change after applying the ReLU in our setting (e.g. Figure 2
consider
B Additional results on parametrized kernel ridge regression
Vector-valued Reproducing Kernel Hilbert Spaces Since we consider multi-class classification, the function we learn is multi-output and therefore we have to use vector-valued kernel ridge regression in the most general case. Similar kernel ridge regression with a single output, its multi-dimensional generalization also has interpretation through minimization in an RKHS function space, just with functions of vector-valued RKHS. The vector-valued RKHS a Hilbert space H of vector-valued functions on a domain X' such that there exists mositive-definite matrix- valued function :: x X' - R xxX such that for any x E X' and any probe vector c E IR K xx1 the function I'(x, )) in H and reproducing property similar to scalar valued RKHS holds: f(x) T (f, ((x For more on vector-valued RKHS see e.g. Ciliberto et al. [2015].
In our setting, only consider subclass of vector-valued RKHS in which the matrix valued function T can be decomposed as [' 2) = k(xx, =) TK for some scalar-valued kernel k : X' x X' - IR. This particular type of RKHS is referred to separable and is the parametrization used for Deep RFM and kernel ridge regression in practice.
Now we formally connect the Problems 6 and under the assumption that the data X has full column rank.
Proposition B.1. When the data Gram matrix X has minimum eigenvalue X > 0 (otherwise assuming the same setup Section 5.2) the relaxation in Problem 7 has the same solution as Problem (
Proof of Proposition B. We first show any realizable (as & applied to the Euclidean distance on some dataset I?) can be constructed by applying kas to our data X under Mahalanobis distance with appropriately chosen M matrix. Let be a realizable matrix i.e. any desired positive semi- definite kernel matrix for which there exists a dataset R'' that satisfies k o(d(R', RR)), where d: Rexd n Ra*= denotes the matrix of Euclidean distances between all pairs of points in the first and second argument. Our construction first takes the entry-wise inverse r Since & is realizable, this must be matrix of Euclidean distances between columns of a data matrix R of the same dimensions as X. Assuming the gram matrix of X invertible, we simply solve the system R = NX for matrix N and set M NT N which yields k Φ(r) @(d( R,, R)) d(dar (X, X)) where (: denotes the operation that produces a Mahalanobis distance matrix for the Mahalanobis matrix M.
'(k).
14

---

We now give construction demonstrating that the solution h* to Problem (7) is realizable up to arbitrarily small error using our parametrized kernel on dataset R under Euclidean distance. We can realize the ideal kernel matrix that solves (6). Y Y up to arbitrarily small error by choosing R according to a parameter > O, such that @ R. R))
particular, for feature vectors 32; 2 of the same label in columns i of X, we set the feature vectors R4, Rj for columns i, R to have |R; - R2 || = Then, for 24, 2jj in X of different class, we set R2, Rj as columns of R such that || R, R > e=! Then k R,, R,) is identically for R2, R2 from the same class and converges 0 for points from different classes, as 0, giving that h Φ(r) YTY.
Y as e
For any choice of c > 0, we can apply the procedure descrihed two paragraphs above to construct M that realizes the same ic as applying kM dataset under the Mahalanobis distance. Therefore, the solution to 7 can be constructed as the infimum over feasible solutions to ( completing the proof.
C
Additional proofs
C.1
Asymptotic results
Lemma C.1 (Woodbury Inverse Formula (Woodbury 1950 )
(P+UVT) P UI+V VTp!
Lemma C.2 (Fixed point of collapse). Let A = = + AmppI, the collapsed data gram matrix following an application of the non-linear random feature map 2 map. Then,
(A+)-1
>maa' Xmmup (Amap + nn) Y
y nUUT where U E Raxk is the matrix of normalized eigenvectors of Y By Lemma C.1 and that U =
Proof of Lemma C.2 We write
(A+)-1
(Amap I nUUT)
Xma
U (I + nn UT UT
X2 maa
Xmmip J
UUT
Xmaap
1 1 ndmap
Xmap'
UUT
Amap Amaa + n)
Amap y Y.
Amapp
Amap
+
Proof of Proposition 5.3 We have solved the kernel ridgeless regression problem to get coefficients
X + 21) y. Then, the predictor at layer evaluated on the training data is,
f(G) (X) X Mna.
As in Deep RFM, let M be the AGOP. Then,
M = (V((")))(())))) Xiaa Xi
We drop the subscript / for simplicity. Therefore,
x XM = x x (xTx+ "YTY (xTxxI+ x
15

---

Let A = XTX. Applying Lemma C.1
(A+ - -A-1 (T+
(T+ SA (1+) A-1
Therefore,
X I (
(I (T+ A-1 riy (I I- ( (1-3 "11)
xr).
Applying that, by assumption, Admip is a small value Cii. and as maap is the minimum eigenvalue of
A, we have that \|A & AAmip elm- Therefore, (1+ 2A-1) -1 λA-1 ÅA - (AA-1)² 2
-
ÂA-1 + O(Gim)) ) where O((in)) refers to matrix of spectral norm most C for some constant C 0. Then,
MX-- 7A-1 O(ciua)) yTy (I (((i)) = - +((mm))
+
Let Ā =
1-1* AA-A*1
and 6Λ = 1|A1 - 1 Then, applying Lemma C.1
A-1
(1+ -aa) (1+)) (I+AA(A4)) A(A")-
Let V (I+CA
Assuming partial collapse has already occurred, i.e., c 2 <
1/2,
1 < (1 (aXmau)) 22
In this case, using that Y (A+)-1 = (At) = (1+ \map)) Y Y (Lemma C.2
Y =
- +((mm))
= =YTY - - Y +
+ (Ciaa)
(Am) V (A*) -1 - (A*) AU (A*)
=YTY - 2d(1 + 2 Y +
- -EEA(1 + )mma)) (A*)² VĀY (1 + Amip T A (A*)
Therefore, where A 1- 25(1 + X
> 1/2 by choice of π map (1 + Amap )n 6,
|X X/+1 - | = | MX - YTI
0 + 2-
2x -1 X -
< <((mm) EE 7- 22 + X
< O(emm) + +cal1 c)
= Olemm) + xx) - -
remains to show that partial collapse, CaA map < 1/2, happens in the first iteration. To ensure this condition, recall,
MMXX- - VTY Âr-YTY- AYTY + O(2)))
Therefore, because |A-' 11 & Xmap where Ti is some error matrix with norm 1,
XTMX=YTy- + Ammb nV + 0 xm)))
Therefore,
I TT < 1/2,
provided Admip C1/2, for some universal constant C₁.
16

---

C.2 Non-asymptotic results
Theorem. The unique optimal solution to the (relaxed) parametrized kernel ridge regression objective (Problem 7) is the kernel matrix k" exhibiting neural collapse, kt = II @ [1,1,1)= -1,1,))=Y11 Y
Denote [(A, = tr (A(k² + { -2 tr (AAY)) Note that this prob- lem is separable the rows of A. we only look at one row at a time, we can solve this problem explicitly for that row if we assume that k is fixed and positive definite, simply by solving solving for the zero gradient. Doing this for every row of A and summing up, we get that Problem 6 can be reduced to the following problem:
Proof of Theorem 5.4
sup Σ Y:: (k + nY,
where Y, RN the vector of labels for class c. After writing the eigenvalue decomposition of as VSV and using (k + pT) -k V(S + μT) -1 "VVVVVT this supremum is equivalent to:
sup Σ Σ
Kn
A
A 14
where A₁ are the i-th eigenvalue and eigenvector, respectively. By continuity of the Problem i both variables, we can without loss of generality plug k* into this despite being low-rank. This can be seen by contradiction - the L(A*, ,**) would not be equal to Σx= Y. (k* + - Y. we can find converging sequence k; of symmetric PD kernel matrices that converges k* for which the two objectives are equal and by continuity they converge to the objectives evaluated at A 1" thus they must be equal as well. After simple computation get:
C(A" k*)
Kn M
With slight abuse of notation let us now re-scale y, so that each row is unit norm and divide the loss by K. Then we get (abusing the notation):
C(A" k*)
n
n + μ
Now, Inn (y"":) 0;)² since (vi)
Kn forms an orthogonal basis and therefore tight frame with constant 1. Therefore the expression
1 K
Kn
A +
sup
can be viewed as weighted average of the Actp terms. To simplify the expression, denote := 1 K R - (Y 22 Using Cauchy-Schwar inequality on each individual summand, we see that
(Y v) & Σ -1 "iy and the equality is only achieved if all the entries of e; corresponding to entries of Y. equal to are the same. Thus, get > (Y v²) & ||call2 = 1 with the equality if and only if, for each c, all the entries qi corresponding to the entries of Y, equal to are the same. This gives that Wi & R
Now, take any feasible k. Then.
Kn
K
X A *
D X k +
C(k,a*)
Σ
Wi
-
n n
The first inequality is due to the monotonicity of the function g(x) x and w. & -: the second strict inequality is due to Jensen after noting that g is strictly convex and using the Perron-Frobenius theorem which says that \ has multiplicity 1; and the last inequality is due to Kn X; Kn. which follows from the well-known trace inequality and the fact that & must have all diagonal elements equal 1. This concludes the proof.
17

---

D Experimental details
In both Deep RFM and neural networks, we one hot encode labels and use +1 for within/outside of each class.
For the neural network experiments, use 5 hidden layer MLP networks with no biases and ReLU activation function. All experiments use SGD with batch size 128. We use default initialization for the linear readout layer. The layers all use width 512 in all experiments. All models trained with MSE loss. We measure the AGOP and NC metrics every 10 epochs. VGG was trained for 600 epochs with 0.9 momentum, learning rate 0.01, and initialization 0.3 in the intermediate layers (0.2 for MNIST), ResNet was trained for 300 epochs, no momentum, 0.2 init. scale in the intermediate layers, and learning rate 0.05. MLPs were trained for 500 epochs, no momentum, 0.3 init. scale (0.2
for MNIST), and learning rate 0.05.
We use the standard ResNet18 architecture (from the Pytorch torchvision library), where replace the classifier (a fully-connecte network) at the end with 5 MLP layers (each layer containing linear layer followed by ReLU, without any biases). We truncate the VGG-11, as defined in torchvision models, to the ReLU just before the final two pooling layers, so that the pooling sizes matches MNIST, which contains 28x28 images, then attach at the end 5-layer MLP as the classifier.
For the Deep RFM experiments, we generally use the Laplace kernel k, which evaluates two datapoints I, E Rd as k(I, 2) = exp (-12-212/44 where is a bandwidth parameter. For the experiments with ReLU on MNIST,
use the Gaussian kernel with bandwidth L instead. In these experiments, we set = 2.0. We perform ridgeless regression to interpolate the data, i.e. f = 0. We use width 1024 for the random feature map in Deep RFM with ReLU activation function. For experiments with the cosine activation, we used Random Fourier Features corresponding to the Ei-Laplacian kernel with bandwidth 0.05 and 4096 total features.
Unless otherwise specified, we perform experiments using the first 50,000 points from each of MNIST, SVHN, and CIFAR-10 loaded with torchvision datasets.
For neural network experiments, we averaged over seeds and report standard deviation error bars, truncating the lower interval to 0.01x the mean value. We report Pearson correlation for the NFA values (where each matrix subtracted by its mean value). As noted in the main text, we compute correlation of the NFM with the square root of the AGOP as in Radhakrishnan et al.) [2024a].
Each experiment was performed under 1.5 GPU hours.
single NVIDIA A100 GPU. Each experiment completed in
All code is available at this link: https://githin ght.bb....u....... com/ See in particular nc_ nc_nnn py and deep_rfm. py for NN and Deep RFM code.
rfm
The code for Random Fourier Features (used in Deep RFM with cosine activation function) was adapted from bestps//ggihhe coom/hichamjaaatti anati/ git
E
Additional plots
In this section, give the full set of results for all combinations of datasets and neural network architectures for Deep RFM and standard DNNs (VGG, ResNet, and MLPs). In AnguresBaadd we plot the NC1 and NC2 metrics for Deep RFM as function of the layer for ReLU and cos activation functions. Figures 4 and we visualize the formation of DNC in Deep RFM as in the main text. In Figures 7 8 and 9 demonstrate that the right singular structure can reduce the majority of NC1 reduction in MLPs, ResNet, and VGG, respectively. In Figures 10 and 12 we verify the NEA, DNC, and plot the training loss.
18

---

(r
(r(55))
1 p ΣErFl
10a
0.30
0.25
M
10-1
0.05
0.00
0(M-XX)
10"1
1
0.04
10:5
0.03
10
0.02
0.01
107
0.10
+(M-XX)
0.08
10a
10
0.04
10-2
0.02
Layer of RFM
Layer of RFM
Figure 3: Neural collapse with Deep RFM on additional datasets with a(.) = ReLU(-). We show
Ew tr EB, our NC1 metric on the left, and | ΣgTF |. our NC2 metric, on the right. The first row is CIFAR-10, second is MNIST, third is SVHN. We plot these metrics as a function of depth of Deep RFM for the original data (green), the data after applying the square root of the AGOP M, 1/2 (orange), and the data after the AGOP and non-linearity (blue).
19

---

tr(
1 ERTPl
10
0.25
M
10a1
0.20
0.15
10-3
10
11
13
24
15
16
18
19
3
15
101
0.05
(MM
0.04
10-
0.01
0.00
101
(MM))X)
10a
M
10
0.06
10
0.02
10-3
0.00
10
15 16
Layer of Deep RFM
Layer of Deep RFM
Figure 4: Neural collapse with Deep RFM on additional datasets with o(.) cos(-). show Ew tr EB. our NC1 metric on the left, and jăpTT ΣETF/|. our NC2 metric, on the right. he first row is CIFAR-10 10, second is MNIST, third is SVHN. We plot these metrics as a function of depth of Deep RFM for the original data (green), the data after applying the square root of the AGOP M 1/2 (orange), and the data after the AGOP and non-linearity (blue).
20

---

A
Input data
Layer
Layer 7
Layer
Layer
088
0.6
04
02
0.0
-02
6.8
0.6
0.4
02
0.0
02
C
1.0
088
0.6
02
0.0
0.4
Figure 5:
Visualization of neural collapse for Deep RFM on additional datasets with a(.) cos(-).
As in the main text, we plot the Gram matrix of the centered and normalized feature vectors X' We see the data form the ETF in the final column. (A) corresponds to CIFAR-10, (B) SVHN, and (C)
MNIST.
Input data
Layer
Layer 7
Layer 13
Layer 19
1.0
08
08
02
0.
06
c
06
04
02
Figure 6: Visualization of neural collapse for Deep RFM on additional datasets with af-) ReLU(.). As in the main text, we plot the Gram matrix of the centered and normalized feature vectors X,. The first column displays the Gram matrix of the untransformed (but normalized and centered) data. We see the data form the ETF the final column. (A) corresponds CIFAR-10 10, (B) SVHN, and (C)
MNIST
21

---

A
Layer
Layer
Layer
Layer
Layer
10.0
tr (Ew)
tr tr(E
0.2
0001
200
402
200
400
250
400
500
302
560
100
300
520
B
10.0
tr (Ew)
(rE
0.03
302.0
10:0
tr ur(Ew)
tr E 8
2.0
6.1
100
200
309
400
500
100
200
300
400
500
100
250
390
400
500
100
200
300
400
500
300
200 300 400 500
Epochs
Rpachs
Epochs
Epochs
Epochs
ReLUKUSV +(X))
SV (
Figure 7: Feature variability collapse (NC1) from different singular value decomposition components on an MLP. The first row (A) is MNIST, second (B) is CIFAR-10, third (C) SVHN. As in the main text, we plot the NCI metrics, for the original feature vectors ((X) (blue), the data after applying right singular structure (green), and the data after the full layer application (red).
22

---

A
Layer
Layer
Layer
Layer 4
Layer 5
tr(Ew)
tr(E
0.01
0.001
120
240
120
240
300
120 182
300
160
320
180 300
10.0
tr [w
1.0
tr tr(2n)
30
127
100
240
350
60
120
120
240
300
50
120
100
240
200
60
120
150
240
300
60
120
120
240
300
100,0
tr(Ew)
1.0
tr(2.) B
0.3
120
120
180
100
180
Epochs
Epochs
Epochs
Epechs
Epochs
ReLU(USV o(X))
SV O(X)
+
Figure 8: Feature variability collapse (NC1) from different singular value decomposition components on ResNet. The first (A) is MNIST, second (B) is CIFAR-10. 10, third (C) SVHN. As in the main text, we plot the NCI metrics, for the original feature vectors & (X) (blue), the data after applying the right singular structure (green), and the data after the full layer application (red).
23

---

A
Layer
Layer
Layer 3
Layer
Layer 5
tr 1r
0.1
tx tr(_.
0.21
620
120
B
10.0
tr (r(2)w)
tn }0
0.01
120
245
360
400
620
120 200
300 450 620
120 262
260
430
600
320 200
360
450
600
322 260
350
400
600
C
10.0
tr(Ew)
1.0
tr (50
0:01
Epochs
Epochs
Epocis
Epochs
Epochs
ReLU(USV +(X))
SV (
(
Figure 9: Feature variability collapse (NC1) from different singular value decomposition components on VGG. The first row (A) is MNIST, second (B) CIFAR-I0 10, third (C) is SVHN. As in the main text, we plot the NCI metrics, for the original feature vectors (XX) (blue), the data after applying the right singular structure (green), and the data after the full layer application (red).
24

---

A
10.9
0.9
08
1.0
Correlation
Value
Los5
0.3
0.3
0.01
Epochs
Epochs
Epochs
0.9
0.9
1.0
Corre 3300
Value
L,00
0.2
0.3
0.2
500
1.0
Corrlatoon
Value
Lons
0.5
0.4
0.3
300
400
502
200
300
600
550
Epochs
Epochs
Epochs
(EE
Layer
Layer
Layer
Layer2
Layor
Figure 10: Train loss and NFA correlations for MLPs. The first row (A) is MNIST, second (B) CIFAR-10, third (C) is SVHN. In the first column, plot the evolution of tr Ew/ Σy and tr EB/ tr Er, where Ex Σw + Eg. In the second column we plot the development of the NFA, where correlation is measured between W, Wt and the square root the AGOP with respect the inputs at layer 1, X₁. The third column is the train loss of the neural network.
25

---

A
18.0
0.4
011
Correlation
Value
0.01
0.3
0.03
B
1.0
1.0
0.0
Complaton
0.1
Volue
0.4
C
Epochs
Epochs
10.0
1.
1.0
0.3
Corneaaton
0.1
Volun
0.3
0.1
0.4
0:41
0.03
Epochs
Epochs
Epochs
tr(E5)/tr(25) (r(
Layer
Layer
Layer
Layer
Figure 11: Train loss and NFA correlations for ResNet. The first row (A) is MNIST, second (B) is CIFAR-10, third (C) is SVHN. In the first column, we plot the evolution of tr Ew/ tr Σr and ir En/ Er, where Σx Σw ΣB In the second column we plot the development of the NFA, where correlation is measured between W, Wi and the square inputs at layer I, Xt. The third column is the train loss of the neural network.
the AGOP with respect the
26

---

A
1,0
16.0
10
1.0
0.7
0.6
0.5
Loss
2.01
0.1
0.4
0.03
0.3
120
240
380
480
0.0
120
240
100
200
B
Epochs
Epochs
Epochs
1.0
100
0.0
3.0
0.6-
0.5
0.01
0.4
0.3
0.1
0.03
120
240
480
0.0
122
240
100
200
C
Epochs
Epochs
Epochs
10:
0.8
0.7
0.5
0.2
0.1
120
240
120
240
150
Epochs
tr(E5)/tr(25)
Layer Layor
(((2)))
Figure 12: Train loss and NEA correlations for VGG. The first row (A) is MNIST, second (B) CIFAR-10, third (C) is SVHN. In the first column, we plot the evolution of tr Ew / tr En and tr EB/ tr Σr, where VT + Ew Eg. In the second column where correlation is measured between W Wi and the square root of the AGOP with respect to the inputs at layer Xt. The third column is the train loss of the neural network.
plot the development of the NFA,
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We justify the claims in our abstract in the paper's content.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We carefully discuss the assumptions of our theorems and our implications of experiments in the content of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
28

---

Justification:
We provide all assumptions and proofs in the main paper and supplemental
sections.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All relevant information is included in the main paper and supplemental sections.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
29

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
An anonymous link to code is provided in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ "cipps.c/puullc//guu /ttoe//uuuddeeoooouuuubmiiiiia ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: These details are available in the main paper and supplemental sections.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We include error bars showing one standard deviation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
30

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This information is listed in the supplemental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: httt::::////uripie http:pp.......cccccccccccceeeea.
Answer: [Yes]
Justification: Our paper satisfies this code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper theoretical nature, and therefore has no immediate societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
31

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our results are theoretical in nature and do not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
32

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "laxGrox7Fm", "reviewer": "Reviewer_AtBe", "summary": "The authors study two effects associated with neural collapse: the within class variability going to zero and the orthogonality/tight-frame of the class means. They study the deep recursive feature machine model, and show that neural collapse forms in that setting as well, due to the projection of the data onto the average-gradient outer product (AGOP). They show both empirical and theoretical results on this phenomenon, leveraging high-dimensional gaussian equivalence of nonlinear random feature models. Further, they show that the right singular vectors of the weight matrices are responsible for most the within-class variability collapse, projecting onto the subspace spanned by the gradients.", "questions": "One can show that for a $\\ell_2$ regularized deep network that the weights pick up rank one spikes proportional to $W^{\\ell}_{ij} \\propto  \\frac{\\partial f}{\\partial x^\\ell_i} \\phi(x^{\\ell}_j)$ where $\\phi$ is a nonlinearity and $x^\\ell$ is the preactivation.. This usually means that the *left* singular values of the weight matrices should pick up terms aligned with $\\nabla f$. See for example the update equation for $W$ in 2.2.4 of LeCun:\n\nhttp://yann.lecun.com/exdb/publis/pdf/lecun-88.pdf\n\nIs there any easy way to square this with the results on RFMs, that the *right* singular values align with $\\nabla f$ terms?\n\nIt would be good to be explicit and put a subscript below the $\\nabla$s on the $\\nabla f_\\ell(x^\\ell_{c i})$ in algorithm 1 to be clear what you're differentiating with respect to. \n\nI don't understand why 4.3 is called non-asymptotic analysis. I don't think you're proving non-asymptotic bounds compared to 4.2. If anything 4.3 seems completely unrelated. Can you please give it a title where someone can understand what you're trying to do? Once again the rest of the paper is quite readable but this subsection is a mess.", "limitations": "This work elucidates an important phenomenon in deep learning theory. Developing a principled understanding of feature learning is likely to have implications for the interpretability and reliability of AI systems.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The writing of the paper is for the most part quite readable. The literature review is thorough and puts the results of this paper in a good context. The empirics are extensive and compelling. Moreover, the theoretical ideas leveraging the equivalence of nonlinear random features to a linear kernel with additional identity term make for a compelling argument about the mechanism for neural collapse in RFMs. Given the good mix of theory and experiment, I recommend this paper for acceptance.", "weaknesses": "Section 4 is doing many things at the same time. It may be better to split it into an empirical evidence section, and then do a section on the theoretical results. In particular, it would be good to give an idea of where the theoretical ideas are going at the start of 4.2 before setting out to prove the deep neural collapse results. This would substantially improve the readability of this section. \n\nThis goes double for section 4.3. The opening paragraph of that section is unreadable:\n\n*\"Next, we show that the formation of the neural collapse is not only implicitly given by the specific\noptimization procedure of the Deep RFM, but is also implicitly regularized for in the parametrized\nkernel ridge regression, a model class that includes RFM (i.e., a single layer of Deep RFM)\"*\n\nI don't really understand what this is saying, or even what you're trying to accomplish in the entire subsection. I tried many times to read it. The whole subsection should be rewritten. There are many sentences there that make no sense to me. Here is another one:\n\n*\"Since we do not explicitly regularize M, we will drop the dependence on it, treat k as\n a free optimization variable and compute the optimal value of the following relaxed problem:\"*\n\nThis is certainly not something you can do generally. For example if I had a matrix parameterized as $A = M_1 M_2$ and optimized just the $M_i$ with no explicit regularization, there are many cases where this isn't the same as optimizing $A$. Maybe you mean to say something else but once again I can't understand what you're trying to say. The prior subsection was compelling enough that I am discounting this rather poor form of writing. Please rewrite this section. \n\nMore generally, there are many sentences throughout the paper that are not well-worded and seem to run on. Improving the writing would benefit the quality, precision, and reach of this otherwise strong paper. If in your rebuttal you can provide examples of improved presentation, I may raise my score higher.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed review. In our response here, we will pay significant attention to improving the writing and organization of our work, especially Section 4. Please also read our global response where we discuss the changes in presentation in detail to all reviewers. We proceed by addressing each weakness and question individually.\n\n**Section 4 is doing [...]**\n\nThank you for this suggestion, please see our global response for the changes we will make in our revision. \n\n**The opening paragraph of Section 4.3 is unreadable.**\n\nWe will present our result differently. As an example of our improved presentation, we would rewrite the opening paragraph of this section as follows:\n\n“We have shown so far that mapping with the AGOP induces DNC in Deep RFM empirically and theoretically in an asymptotic setting. In this section, we demonstrate that DNC emerges in parameterized kernel ridge regression: in fact, DNC arises as a natural consequence of minimizing the norm of the predictor jointly over the choice of kernel function and the regression coefficients. This result proves that DNC is an implicit bias of kernel learning. We connect this result to Deep RFM by providing intuition that the kernel learned through RFM effectively minimizes the parametrized kernel ridge regression objective and, as a consequence, RFM learns a kernel matrix that is biased towards the collapsed gram matrix $yy^T$.”\n\n**Dropping the dependence on $M$ is not always equivalent to the original problem.**\n\nYou are right that optimizing over all $k$ is not always equivalent to optimizing over all $M$. Thus, in general, this provides only a relaxation. However, if the gram matrix of the data is invertible (this is a realistic assumption if we consider layers $\\ell>1$ of Deep RFM, where we are allowed to pick the feature dimension to be large enough so that this is satisfied), then the two optimizations have the same solution (where the min is taken to be an infimum) and, thus, the relaxation is without loss of generality. \n\nWe first show any $k$ realizable by our input-level kernel (under Euclidean distance on some dataset $R$) can be constructed by applying the input-level kernel to our data $X$ under Mahalanobis distance with appropriately chosen $M$ matrix. Let $k$ be a realizable matrix - i.e. any desired positive semi-definite kernel matrix for which there exists a dataset $R'$ on which our input-level kernel satisfies $k = \\phi(d(R', R')))$, where $d(\\cdot, \\cdot)$ denotes the matrix of Euclidean distances between all pairs of points in the first and second argument. Our construction first takes the entry-wise inverse $r=\\phi^{-1}(k).$ Since $k$ is realizable, this $r$ must be a matrix of Euclidean distances between columns of a data matrix $R$ of the same dimensions as $X$. Assuming the gram matrix of $X$ is invertible, we simply solve the system $R=NX$ for a matrix $N,$  and set $M=N^TN,$ which yields $k=\\phi(r)=\\phi(d(R, R))=\\phi(d_M(X, X)),$ where $d_M(\\cdot, \\cdot)$ denotes the operation that produces a Mahalanobis distance matrix for the Mahalanobis matrix $M$.\n\nWe now give a construction demonstrating that the solution $k^*$ to problem $(2)$ is realizable up to arbitrarily small error using our input-level kernel on a dataset $R$ under Euclidean distance. We can realize the ideal kernel matrix that solves problem $(3)$, $yy^T$, up to arbitrarily small error by choosing $R$ according to a parameter $\\epsilon > 0$, such that $\\phi(d(R, R)) \\rightarrow yy^T$ as $\\epsilon \\rightarrow 0$. In particular, for feature vectors $x_i,x_j$ of the same label in columns $i,j$ of $X$, we set the feature vectors $R_i, R_j$ for columns $i,j$ in $R$ to have $||R_i-R_j||=0$. Then, for $x_i,x_j$ in $X$ of different class, we set $R_i, R_j$ as columns of $R$ such that $||R_i-R_j|| > \\epsilon^{-1}$. Then $k(R_i, R_j)$ is identically 1 for $R_i, R_j$ from the same class and converges to 0 for $R_i, R_j$ from different classes, as $\\epsilon \\rightarrow 0$, giving that $k = \\phi(r) \\rightarrow yy^T$. \n\nFor any choice of $\\epsilon>0$, we can apply the procedure described two paragraphs above to construct $M$ that realizes the same $k^*$ using our input-level kernel applied to our dataset $X$ under the Mahalanobis distance. Therefore, the solution to problem $(2)$ can be constructed as the infimum over feasible solutions to problem $(3)$, completing the proof.\n\n**Is there any easy way to square this with the results on RFMs, that the right singular values align with $\\nabla f$ terms?**\n\nThis is a keen observation. You are correct that, although the NFA describes structure in $W^T W$, the correlation between NFM and AGOP is enabled by the alignment between the left singular vectors of $W$ and the gradients with respect to the pre-activations, as argued in [1]. This can be seen through decomposing the AGOP as $W^T K W$, where $K$ is the covariance of $\\frac{df}{dx}$ and $x$ are the pre-activations. Then, the NFA holds provided that the right singular structure of $W$ is not perturbed through the inner multiplication by $K$, an alignment that naturally occurs through training by gradient descent. See [1] for a more detailed description of this phenomenon.\n\n[1] Beaglehole, Mitliagkas, Agarwala. \"Feature learning as alignment: a structural property of gradient descent in non-linear neural networks.\" arXiv pre-print, 2024.\n\n**It would be good to be explicit and put a subscript below the $\\nabla$'s on the $\\nabla f_\\ell(x_{ci}^\\ell)$**\n\nThank you for this comment. We will fix this in the revision.\n\n**Why is Section 4.3 called “non-asymptotic analysis”?**\n\nWe will rename this subsection “Connection to parametrized kernel ridge regression”.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CeyUZ3Di9o", "reviewer": "Reviewer_1F1v", "summary": "The submission introduces a mechanism for Deep Neural Collapse (DNC) using the average gradient outer product (AGOP). The authors also propose the Deep Recursive Feature Machines (Deep RFM) model, which employs AGOP in its architecture to empirically and theoretically demonstrate DNC. The main contribution is that AGOP-based explanation is a data-based approach while prior work focused on data-agnostic explanations.", "questions": "* Can the authors clarify if other metrics, such as the Neural Tangent Kernel (not the limit), would effectively predict this behavior? Or AGOP is unique in this aspect?\n* What are the practical implications of this work?", "limitations": "See weaknesses.", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* Using a data-based approach based on AGOP to explain DNC is novel to the best of my knowledge\n* The paper offers both theoretical analysis and empirical evidence supporting the role of AGOP in inducing DNC\n* The experiments are performed on different architectures and datasets", "weaknesses": "*  I found the paper challenging to read\n* I am unsure about the practical implications of this work", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We address your concerns below. \n\n**I found the paper challenging to read.**\n\nWe will make a number of changes to our presentation. Please see our global response to all reviewers for a summarized description of these changes.\n\n**Can the authors clarify if other metrics, such as the Neural Tangent Kernel (not the limit), would effectively predict this behavior? Or AGOP is unique in this aspect?**\n\nPrior work has shown that neural collapse can occur if the empirical Neural Tangent Kernel has a particular block structure dependent on the inputs and labels [1]. In Deep RFM, we consider a different setting, where we fix a kernel function to be independent of the data, i.e. the NTK in the infinite-width limit, and then understand how a linear transformation prior to the kernel evaluation can induce DNC. \n\n[1] Seleznova, Weitzner, Giryes, Kutyniok, Chou. “Neural (Tangent Kernel) Collapse.” NeurIPS, 2023.\n\n**What are the practical implications of this work?**\n\nWhile the scope of this work is theoretical, there are a number of works demonstrating the practical applications of (deep) neural collapse and Recursive Feature Machines. More specifically, neural collapse has been used as a tool in several important contexts, including generalization bounds, transfer learning, OOD detection, network compression and robustness, see lines 85-90 and the references therein. Furthermore, RFM itself has been shown to be a practical tool for tabular data (Radhakrishnan et al., Science 2024) and scientific applications [1,2].\n\nOur work also has broad implications for our understanding of deep learning. In particular, this work is part of a large body of research demonstrating that Deep RFM is a useful proxy to explain interesting phenomena in deep networks. In addition to neural collapse considered in this work, RFM has been shown to exhibit grokking in modular arithmetic [3], recover convolutional edge detection, and learn the same features as fully-connected networks on vision tasks (Radhakrishnan et al., Science 2024). These findings point toward Deep RFM and AGOP as powerful tools to improve our understanding of neural networks, which would likely lead to significant practical implications.\n\n[1] Aristoff, Johnson, Simpson, Webber. “The fast committor machine: Interpretable prediction with kernels.” arXiv pre-print, 2024\n\n[2] Cai, Radhakrishnan, Uhler, “Synthetic Lethality Screening with Recursive Feature Machines.” arXiv pre-print, 2023.\n\n[3] Mallinar, Beaglehole, Zhu, Radhakrishnan, Pandit, Belkin, “Emergence in non-neural models: grokking modular arithmetic via average gradient outer product.” arXiv pre-print, 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tFBJ65mNcI", "reviewer": "Reviewer_uXDR", "summary": "This paper studies deep neural collapse (DNC) in deep neural networks (DNN) through the prism of the neural feature ansatz (NFA) and deep recursive feature machines (RFM). It is comprised of several results:\n- empirical evidence that DNC occurs in deep RFMs,\n- a theoretical analysis of DNC in a high-dimensional RFM setting,\n- a theoretical analysis of DNC in a kernel learning setting,\n- empirical evidence that the mechanisms which lead to DNC in RFMs and traditional DNNs are the same.", "questions": "- Are the feature maps $\\Phi_l$ and kernels $k_l$ unrestricted in Algorithm 1, or do they have to match in the sense that $k_l(x,x') = \\langle \\Phi_l(x), \\Phi_l(x')\\rangle$?\n- What is the motivation behind considering _normalized_ features _before_ the non-linearity in section 4.1? Could the authors clarify the role of these non-standard conventions?\n- Why are the setting different between sections 4.1 and 4.2? (relative to normalizations). Is neural collapse still empirically observed with the choices of section 4.2? It raises the suspicion that it could not the case, in which case Theorem 4.3 would not really explain the results of section 4.1 (e.g., because the asymptotic regime is not reached in practice).", "limitations": "See weaknesses.\n\nIn its current state, I think that the paper is slightly below the acceptance bar and would require minor, if not major, changes before it can be fully appreciated by the NeurIPS community. I would be happy to raise my score if the authors address the points raised above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "This paper shows that deep neural collapse occurs in a similar way in deep networks and deep recursive feature machines. It thus provides a simplified setting in which to investigate deep neural collapse, which is an important research direction to further our understanding of deep learning. Specifically, it shows that neural collapse can be obtained just by iterating linear regression problems, without backpropagating through a deep network.", "weaknesses": "My main issue with the paper is its writing, which makes it quite difficult to read.\n- The notations could be improved in several places throughout the paper (see minor points below).\n- I could not follow most of section 4.2, despite being rather familiar with kernel methods and their behavior in high dimensions. \nOn a high level, I don't understand how a linear kernel could be the best setting for neural collapse. The text contradicts itself, as it simultaneously state that \"if [$\\lambda_k = 0$] [...], collapse will occur in just one layer] , but also that \"this theory offers an explanation for why non-linear activation is needed\". A linear layer can collapse within-class variability but also typically collapses class means together, and thus cannot induce neural collapse (see paragraph below). \nOn a technical level, $k_\\Phi$ and $\\lambda_\\Phi$ are referred to before being defined, and I do not understand the roles played by $k$/$\\lambda_k$ vs $k_\\Phi$/$\\lambda_\\Phi$. Assumption 4.2 is also referred to before being stated.\n- Section 4.3 is also slightly difficult to read. \nI took me several tries to guess that $k_M(x,x') = \\tilde k_M(x,x')\\mathrm{Id}$, which should appear in the paper. The terms \"input-level\" and \"output-dimension level\" kernels should be introduced for non-specialists in multi-task kernel learning.\nI also do not understand the point of introducing $M$ if it is dropped afterwards. Theorem 4.4 could simply be stated as \"the optimal feature map for ridge regression is the one which already predicts the label: $\\Phi(x) = y$\". This result is not very surprising, and is not very integrated in the paper. I suppose that it is some kernel-level equivalent of the unstructured feature model, and suggests that weight decay might be instrumental in bringing about neural collapse? The normalization of $k$ should be restated in the definition of Problem 3 (otherwise the optimal loss is obtained when $k \\to 0$).\n- The message of section 5 could be presented more clearly. What I understood was that it argues that RFMs and DNNs achieve neural collapse through the same means. I suggest making this point before introducing RFMs (in particular, stating the NFA correlations). I also did not understand why this mechanism is referred to as \"denoising\". \n\nMy second issue is that I was not convinced by the claim that it is the right singular vectors and singular values which lead to neural collapse. By the same logic as lines 309-315, the right singular vectors do not change the DNC1 metric (with a \"full\" SVD where $U$ and $V$ are invertible). Similarly, if I were to divide operations in the network as $V^T\\sigma$ and $US$ as opposed to $\\sigma U$ and $SV^T$, I should see that it is now $US$ which is responsible for neural collapse (again with a full SVD). This conclusion also depends on the chosen metric for evaluating collapse. Why do the authors consider the ratios of traces of between- and within-class covariances, rather than the trace of their ratio (the Fisher linear discriminant)? It seems that it would reverse one of the conclusions of the analysis, since the trace of the Fisher discriminant ratio $\\mathrm{tr}(\\Sigma_W^{-1} \\Sigma_B)$ is invariant to invertible linear transformations, and decreases under non-invertible linear transformations, so can only be improved through the non-linearity. If the conclusion of which part of the network is responsible for DNC depends strongly on the chosen metric, can we really ascribe meaning to the question? It seems to me that it is really the sequence of weights and non-linearity which _together_ induce DNC, and trying to separate their effects is not really possible.\n\nFinally, Proposition A.1 was first published by Cho and Saul in _Kernel Methods for Deep Learning_, NIPS 2009. Besides, the expression of the kernel in eq. (5) can be simplified with algebra and trigonometry (compare with their eq. (6)).\n\nMinor notes and suggestions:\n- I suggest using a so-called \"diverging\" colormap (such as \"bwr\") in Figure 1 to clearly separate positive from negative correlations, and use the same range for both datasets.\n- I suggest replacing \"Gram matrix\" with \"(uncentered) covariance\" to refer to $W^TW$, as weight matrices $W$ are generally decomposed in rows which correspond to individual neurons.\n- The notation $||X||$ to refer to the vector in $\\mathbb R^N$ of column norms of a matrix $X \\in \\mathbb R^{d\\times N}$ is never introduced (and clashes with the usual convention that this is a matrix norm).\n- Why is the last layer denoted $W_{L+1}$ instead of $m_{L+1}$?\n- The choice of layer-indexing is confusing and seems inconsistent throughout the paper. Contrarily to what is stated in section 3.1, isn't $X_l$ the features after $l-1$ network layers? I suggest to denote the input as $X_0$ instead of $X_1$ to simplify the notations. Also, it seems that $M_l^{1/2} X_l$ should be referred to as $\\tilde X_{l+1}$ rather than $\\tilde X_l$ given the chosen conventions.\n- Typo: missing a norm in the definition of $\\bar H_l$ line 128.\n-In section 4.2, I suggest defining activations before the kernels, e.g., $\\tilde X_{l+1} = \\kappa^{-1/2} M_l^{1/2} X_l$ and $X_{l+1} = \\Phi_{\\rm lin}(\\tilde X_{l+1})$. I also suggest choosing a different notation for $k_{\\rm lin}$ and $\\Phi_{\\rm lin}$ which are confusing as they imply linearity, and to avoid the awkward \"non-linear feature map $\\Phi_{\\rm lin}$\".\n- Typo line 260: the output space of $k_M$ should be $\\mathcal R^{C\\times C}$.\n- I suppose that $\\lambda = \\mu$ in section 4.3.\n- In the caption of Figure 2, I suppose that \"fully-connected\" should be removed in the case of ResNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We note that we will significantly improve the presentation of our paper, and specifically Sections 4.2 and 4.3. Please see the global response for a summary of our changes. We now proceed to address individual comments. \n\n**I could not follow most of section 4.2 [...]**\nWe will clarify this. The two kernels $k_{\\mathrm{lin}}$ and $k_{\\Phi}$ have opposite effects. In particular, the closer $k_{\\mathrm{lin}}$ is to a linear kernel, the more accelerated collapse will be for Deep RFM. In fact, a linear kernel induces NC in just one AGOP application. To see this, note that ridgeless regression with a linear kernel is exactly least-squares regression on one-hot encoded labels. In the high-dimensional setting we consider, we find a linear solution $f(x) = \\beta^T x$ that interpolates the labels, and the AGOP of $f$ is $\\beta \\beta^T$. Since we interpolate the data, $\\beta^T x = y$ for all $(x,y)$ input/label pairs and, hence, the data collapses to $\\beta\\beta^T x = \\beta y$.\n\nIn contrast, having $k_{\\Phi}$ far from a linear kernel accelerates collapse, and the non-linear activation function is needed so that $k_{\\Phi}$ is significantly non-linear. In fact, if $k_{\\Phi}$ were exactly linear and the original data were not linearly separable, then collapse cannot occur at any depth of Deep RFM. In that case, the predictor would need to contain a non-linear component in order to fit the labels exactly, deviating from the ideal case described above.\n\n**Section 4.3 is also slightly difficult to read [...]**\nThank you for your suggestions. We address your points one-by-one:\n\n- We introduce $M$ because it is the key part of the parametrized kernel ridge regression which can be understood as the objective function of the RFM iteration. We emphasize that when the data (or the features) are full rank, dropping $M$ does come without loss of generality. Due to the character limit, we kindly refer you to our response to reviewer AtBe for details.\n- We appreciate your interpretation of this model as an unconstrained kernel model. We agree with this interpretation and will call it in this way in the revision. \n- Theorem 4.4 is well integrated in our paper as the kernel ridge regression loss can be understood as the quantity the RFM learning algorithm is minimizing. Thus, the result describes DNC as an implicit bias of the RFM iteration.\n- For additional structural changes to Section 4.3, see our global response.\n\n**The message of section 5 [...]**\n“Linear denoising” highlights the nature of the mechanism: by only applying linear transformations, we decrease the within-class variability of the data and improve DNC1 by only discarding class-irrelevant information (i.e., noise) via the non-trivial null-space of the weight matrix. See our global response for additional changes.\n\n**Why splitting into $\\sigma U$ and $SV^T$?**\nIn the compact SVD (which is an equivalent re-writing of the full SVD), it is the matrix $V^T$ which has non-trivial null-space, while the matrix $U$ always has a trivial null-space. Note that the non-triviality of the null-space allows for the DNC1 metric to decrease. Hence, this is the correct split of the layer.\n\nMore generally, the main message of this section is not which part of the SVD decreases the DNC1 metric, but rather that the linear transformation of $W$ has a stronger effect on NC1 than the non-linearity $\\sigma$. In fact, the most natural grouping of the layer - into the non-linearity alone and the entire weight matrix alone - gives equivalent NC1 values to our grouping, as the two standard metrics for NC1, $\\mathrm{tr}(\\Sigma_W \\Sigma_B^\\dagger)$ and $\\mathrm{tr}(\\Sigma_W)/\\mathrm{tr}(\\Sigma_B)$, are invariant to the rotation by the left singular vectors of $W$. \n\n**This conclusion also depends on the chosen metric...**\nThis is an interesting consideration, however we kindly disagree that Fisher linear discriminant cannot decrease due to a linear mapping. Consider the example in which the feature vectors of $X$ are not collapsed, but the differences between points of the same class lie in the null-space of $W.$ Then, $WX$ is perfectly collapsed and any NC1 metric would be identically zero. \n\nWhile the two metrics are similar, we choose our metric because it directly quantifies how close the feature vectors are to their class means - a common and intuitive interpretation of NC1. Our metric is also frequent in the literature, see e.g. Rangamani et al., 2023 and Tirer et al., 2023.\n\n**Are the feature maps $\\Phi_l$ and kernels $k_l$ unrestricted?**\nThe feature maps and kernels do not have to match. We will clarify this when we introduce the two objects. \n\n**What about the normalized features in 4.1?**\nWe normalize the features as neither the AGOP projection nor the random feature map have a native ability to rescale the data. Therefore, while we still see collapse in terms of the angles between data points, the dynamics of the data norms are more difficult to control in this setting. \n\n**Why are the setting different between sections 4.1 and 4.2? [...]**\nNeural collapse would still be observed without normalizing the data, in the sense that all points of the same class will be parallel and all points of different classes will be perpendicular (or at angle $-\\frac{1}{K-1}$). However, we are not guaranteed convergence of the data Gram matrix to $yy^T$ in general without the re-normalization, as the diagonal entries of $X^T X$ will not be identically $1$. In our analysis, it is sufficient to scale the Gram matrix at each layer with $\\kappa^{-1}$. The main advantage of this scaling is that it is analytically much simpler than explicitly projecting the data to be norm $1$.\n\n**Minor notes.**\nWe will address all these points in the revision. As an example, we will rename $k_{\\mathrm{lin}}$ and $\\Phi_{\\mathrm{lin}}$ to $\\widehat{k}$, indicating the predictor kernel, and $\\Phi_{\\mathrm{map}}$, indicating the feature map applied to the data.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gRCr5hwstP", "reviewer": "Reviewer_Ljpy", "summary": "Given the complexity of the process of neural network training, any understanding of robust phenomena that can be identified in the training process has potential value that can guide the design of models and algorithms. Neural Collapse (and its deep counterpart) is one such phenomenon that has been identified and reproduced across multiple model classes and datasets. This work shows that Neural Collapse also occurs for a recursive kernel-based model known as Deep RMF, when trained using an algorithm that is based on projection onto a matrix constructed from an outer products of gradients computed locally at each layer.\nAdditionally, the authors present experimental results that document neural collapse in these models when trained on standard datasets. They also show that in standard neural networks, the projection of features onto the gradient outer product leads to neural collapse, rather than the effect of the nonlinearity.", "questions": "Are the models in appendix D trained with SGD? If so, they indicate the importance of the AGOP projection in causing neural collapse in standard models and I believe this result should be highlighted. That being said, this result may be of independent interest regardless of the analysis of Deep RMF.", "limitations": "Limitations and societal impacts have been addressed", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is clearly written, and presents both theoretical results and some empirical results that complement them, since they apply to datasets that violate the assumptions under which the results hold. They prove that deep Neural Collapse can indeed occur in models beyond standard neural networks trained with gradient descent.\nThe experimental results (specifically in Appendix D) demonstrate that the projection onto the gradient outer product matrix (AGOP) leads to neural collapse in standard models, motivating the further study of this object.", "weaknesses": "Given that the main results apply both to a non-standard kernel method and a non-standard training algorithm, it is unclear what the implications of the results are for more well-known models and algorithms. If the authors believe that these results have implications of this form, they should be presented more clearly. Algorithms that are not based on backpropagation are interesting both as possible means of explaining learning in biological systems where backpropagation is unrealistic, and in distributed settings where backpropagation may incur a prohibitive communication overhead. However, the motivation of the algorithm used appears to be that it is a simple model that demonstrates certain phenomena that arise in the training of deep networks. \n\nThe authors assume that the gram matrix of the data is full-rank. This requires assuming that the number of datapoints is smaller than or equal to the input dimension (which subsumes assumption 4.1). Standard datasets violate this assumption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We address all your concerns below:\n\n**Given that the main results apply both to a non-standard kernel method and a non-standard training algorithm [...]** \n\nOur first motivation for studying DNC with Deep RFM is that, unlike the standard analytical approaches for neural collapse, the Deep RFM procedure is highly dependent on both the input samples and their labels. Specifically, the collapse process of Deep RFM depends on mapping with the AGOP of a predictor trained to fit these input/label pairs. Moreover, as we demonstrate in Theorem 4.3, the rate of this collapse throughout the layers is highly dependent on the specific structure of the data through its relationship to the parameters $\\lambda_\\Phi$ and $\\lambda_{\\mathrm{lin}}$. We also provide significant evidence that this AGOP mechanism is present in deep neural networks in Section 5 of our original manuscript, indicating that our results apply to more practical architectures.\n\nSecondly, the ability of Deep RFM to recover interesting behaviors of deep learning has broad implications for our understanding of neural networks. In fact, the same Deep RFM model we study has recovered a number of other interesting deep learning phenomena including, e.g., grokking modular arithmetic [1], convolutional feature learning [2], and even low-rank matrix completion [3]. As you mention, Deep RFM is a simpler model than neural networks, in the sense that we understand its individual components: (1) a kernel regression step, (2) mapping with the AGOP, and (3) random feature projection. Therefore, our work additionally justifies Deep RFM as a simplified proxy to understand neural networks.\n\n[1] Mallinar, Beaglehole, Zhu, Radhakrishnan, Pandit, Belkin, “Emergence in non-neural models: grokking modular arithmetic via average gradient outer product.” arXiv pre-print, 2024.\n\n[2] Radhakrishnan, Beaglehole, Pandit, Belkin, \"Mechanism for feature learning in neural networks and backpropagation-free machine learning models.\" Science, 2024.\n\n[3] Radhakrishnan, Belkin, Drusvyatskiy, \"Linear Recursive Feature Machines provably recover low-rank matrices.\" arXiv pre-print, 2024.\n\n**The authors assume that the gram matrix of the data is full-rank. This requires assuming that the number of datapoints is smaller than or equal to the input dimension (which subsumes assumption 4.1). Standard datasets violate this assumption.**\n\nThe strong assumption that the gram matrix of the data is full-rank is needed only if one requires neural collapse in every layer of the network, starting from the very first one. In contrast, if we consider collapse starting at any given later layer of Deep RFM, then we only need that the smallest eigenvalue of the corresponding feature map is bounded away from 0. This in turn requires that the number of features at that layer is greater than the number of data points, a condition which is routinely satisfied by the overparameterized neural networks used in practice. \n\n**Are the models trained with SGD?**\n\nThe neural network models are trained with standard SGD. We agree that this result is of independent interest, and we will emphasize in the main section that our training procedure is standard. Let us also clarify that Deep RFM is not trained by any gradient-based method, and is instead optimized through an iterated three-stage process: (1) solve kernel ridgeless regression, (2) compute the AGOP of the predictor from step 1, and (3) apply a high-dimensional feature map.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their thorough feedback on our manuscript. We will make a number of clarifying changes to the organization and presentation of our results. We list major changes here.

First, we will split Section 4 into two new sections. The first section will contain the empirical results in Subsection 4.1 together with the first row of Figure 3 in Appendix D of our original manuscript, which shows quantitative improvements in our DNC metrics for Deep RFM. We will then make Sections 4.2 and 4.3 into their own section, which will be renamed “Theoretical explanations of DNC in Deep RFM”.

Second, we will make the following improvements to the presentation of Section 4.2:
- We will first define $k_\Phi$ and $\lambda_\Phi$ at the end of the first paragraph.

- We will then add a paragraph following the first of Section 4.2 in our original manuscript to outline the argument and results of this subsection:
“We show that Deep RFM exhibits exponential convergence to DNC, and the convergence rate depends on the ratio $\lambda_{k} / \lambda_\Phi$. These  two parameters modulate the extent of DNC with Deep RFM. Namely, as we will show, if $k_{\mathrm{lin}}$ is close to the linear kernel, i.e., if $\lambda_{k}$ is small, then the predictor in each layer will resemble interpolating linear regression, an ideal setting for collapse through the AGOP. On the other hand, if $k_{\Phi}$ is close to the linear kernel, i.e., if $\lambda_\Phi$ is small, then the data will not be easily linearly separable. In that case, the predictor will be significantly non-linear in order to interpolate the labels, deviating from the ideal setting. We proceed by explaining specifically where $k_\Phi$ and $k_{\mathrm{lin}}$ appear in Deep RFM and why linear interpolation induces collapse, and then follow with the statement of our main theorem."

Third, we will do the following improvements to the presentation of Section 4.3:
- We will rename the subsection “Connection to parametrized kernel ridge regression" to clarify its role.

- We will rewrite the introduction of the subsection to clearly state its message: DNC is a natural consequence of minimizing the norm of the predictor over the choices of both the kernel function and the kernel coefficients in parametrized kernel ridge regression. We also clearly state why this result is relevant in our context – the RFM learning algorithm can be understood as an optimization procedure for the parametrized kernel ridge regression objective. 

- We will discuss matrix-valued kernels in more detail and write the explicit formula for the kernel we are describing, as the reviewer uXDR suggests. 

- We will discuss in more detail both why it is important to introduce $M$ and why we are able to consider the relaxed optimization over kernel matrices $k$. In particular, introducing $M$ is necessary as that is the key added parameter in the parametrized kernel ridge regression and it corresponds to the AGOP matrix used in RFM. Dropping the $M$ and optimizing over $k$ does not always lead to an equivalent optimization, as pointed out by reviewer AtBe, however their solutions are equivalent in the setting of our asymptotic analysis - where the data gram matrix is invertible. We explain this point in detail in our response to reviewer AtBe. Note that the invertibility assumption is guaranteed to hold when we consider collapse beyond the first layer of Deep RFM, at which we have inflated the data dimension through a high-dimensional feature map $\Phi_{\mathrm{lin}}$. For more detail on this point, see our response to reviewer Ljpy.

- We will call the problem $(3)$ “unconstrained kernel model.”

Fourth, we will do the following changes to Section 5:

- We will rename the section “AGOP as an effect of linear denoising in neural networks”, and emphasize the role of AGOP as responsible for improvements in NC1 in neural networks. In particular, we will rewrite the first paragraph as follows:
"We provide evidence that the DNC mechanism of Deep RFM, i.e., the projection onto the AGOP, is responsible for DNC formation in typical neural networks, such as MLPs, VGG, and ResNet trained by SGD with small initialization. We do so by demonstrating that DNC occurs by this iterated linear mechanism through the right singular structure of the weights. As the NFM, which is determined by the right singular structure of $W$, is highly correlated with the AGOP, our results imply the AGOP is responsible for NC1 progression."

Meta-review: The submission introduces a interesting mechanism for Deep Neural Collapse (DNC) using the average gradient outer product (AGOP).  Given that the main results apply both to a non-standard kernel method and a non-standard training algorithm, it is unclear what the implications of the results are for more well-known models and algorithms.  Personally, how one can use DNC to improve the non-deep learning based feature learning algorithm can be more interesting than the story in the paper now. (e.g. Only from this manuscript, I can't know the performance of DeepRFM ). Although reviewers raised concern of studying non-standard mechanism, from my personal perspective, this paper can be an interesting add to the machine learning community. Thus I vote for acceptance.
Decision: Accept (poster)