Neural Collapse To Multiple Centers For Imbalanced Data

Authors: hongren  yan
yuhua  qian
furong  peng
jiachen  luo
zheqing  zhu
feijiang  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 08:03:21

Abstract: Neural Collapse (NC) was a recently discovered phenomenon that the output features and the classifier weights of the neural network converge to optimal geometric structures at the Terminal Phase of Training (TPT) under various losses. However, the relationship between these optimal structures at TPT and the classification performance remains elusive, especially in imbalanced learning. Even though it is noticed that fixing the classifier to an optimal structure can mitigate the minority collapse problem, the performance is still not comparable to the classical imbalanced learning methods with a learnable classifier. In this work, we find that the optimal structure can be designed to represent a better classification rule, and thus achieve better performance. In particular, we justify that, to achieve better classification, the features from the minor classes should align with more directions.  This justification then yields a decision rule called the Generalized Classification Rule (GCR) and we also term these directions as the centers of the classes. Then we study the NC under an MSE-type loss via the Unconstrained Features Model (UFM) framework where (1) the features from a class tend to collapse to the mean of the corresponding centers of that class (named Neural Collapse to Multiple Centers (NCMC)) at the global optimum, and (2) the original classifier approximates a surrogate to GCR when NCMC occurs. Based on the analysis, we develop a strategy for determining the number of centers and propose a Cosine Loss function for the fixed classifier that induces NCMC. Our experiments have shown that the Cosine Loss can induce NCMC and has performance on long-tail classification comparable to the classical imbalanced learning methods.

Main paper: 
 introduction: 
 Deep neural networks are popular choices classification tasks [1, 2, 3, 4, 5, 6, 7] . Researchers try to demystify the deep representations learned from data [8, 9] . A recent paper [10] observed "Neural Collapse" (NC) phenomenon: all the backbone network output features from each class converge into their corresponding vertices of an equiangular tight frame (ETF) and the within-class variability collapses.The layer-peeled model (LPM) [11] and unconstrained feature model (UFM) [12] are the simplified model to study NC, in which the backbone output feature and the classifier weights are assumed free variables to optimize. NC phenomena occur with different loss functions. The optimality of UFM satisfies NC under the CE loss with constraints [13, 11, 14] , regularization [15] , or even no explicit constraint [16] . MSE objectives also induce NC at global optimality [12, 17, 18, 19] . There is another line of works that extend UFM or LPM on deeper linear layers [19, 20, 21, 22] . Data imbalance has been recently considered in NC literature [11, 23, 20] . In particular, Fang et al. [11] originally observe the "minority collapse" phenomenon that the classifier weights of the minority classes will approach each other when the imbalance level goes high. Thrampoulidis et al. [23] theoretically study the Unconstrained-features SVM problem, whose global minima take the form of Simplex-Encoded-Labels Interpolation (SELI), a more general structure compared to the ETF. Dang et al. [20] inspects the ReLU-activated output features of deep linear network collapse to a general orthogonal frame for imbalanced data, where the orthonormal vectors of the frame are rescaled.There is a line of works that connect the NC to the DNN performance [10, 24, 25] . Some researchers treat NC as a tool to alleviate minority collapse problem in imbalanced learning [26, 27, 28] . We reproduce these methods with backbone network ResNet50 [29] and datasets cifar10/cifar100, Table 3 shows NC-inspired methods ETF and ARB only outperform the plain model (ResNet50 with CE loss) slightly (or even worse in a few settings), which indicates that minority collapse is one but not the only problem that harms the generalization of learning model. One possibly important issue is that NC on training sample does not necessarily imply the NC on the distribution, as pointed out by Hui et al. [30] . This inconsistency can lead to severe performance degeneration when the sample is too scarce to represent certain classes.Neural Collapse implies "maximal separateness" between classes, which inspires some works to consider fixed classifiers in the training [31, 32, 33, 34, 35] . However, these methods do not display advantages over the classical imbalanced learning methods equipped with learnable classifiers.In this paper, we study the connection between the optimal structure induced by neural collapse and its corresponding classification rule, and propose a MSE-type loss function that improves the imbalanced learning with fixed classifier. Specifically, 1. Through the analysis of hard-to-predict feature distribution (the features that are distributed randomly around the mean of the classifiers), we find that the classification accuracy is improved if the features from the minor classes align with more directions and those from major classes with less directions, which corresponds to a decision rule called the Generalized Classification Rule (GCR) discriminated from the Regular 1-Nearest Neighbor Classification Rule (RCR) induced by general NC in literature, and we also term these directions as the centers of the classes.2. We design an MSE-type objective that describes the average distance between the centers and a given feature. We show in the theoretical framework of Unconstrained Feature Model (UFM) that, for balanced or imbalanced data and fixed or learnable classifiers, the output features collapse but skew from the classifiers at terminal phase of training (TPT), which is different from the original Neural Collapse phenomenon and is termed "Neural Collapse to Multiple Centers" (NCMC) (Theorem 3.3 and 3.4); moreover, we find that RCR (with respect to classifiers) becomes a surrogate of GCR at NCMC (Remark 3.9 and Proposition C.1).3. We design a practical loss function for fixed classifiers and a class-aware strategy for determining the number of centers for each class. The loss induces the NCMC which is justified in theory and experiments and achieves comparable performance on several datasets with varying imbalance ratios to the classical imbalanced learning methods such as LDAM [36] , KCL [37] , ARBLoss [27] , which indicates that NCMC can improve generalization in imbalanced learning. EQUATIONwhere h θ (•) : R D → R d is the feature mapping (d ≥ K), W 0 = [w 1 , . . . , w K ] ⊤ ∈ R K×d with w k ∈ R d the weight vector of class k, and b ∈ R K the bias of classifier, respectively. Θ = {W 0 , b, θ} is the set of the trainable network parameters, which includes the parameters θ of a nonlinear compositional feature mapping (e.g., h θ (x) = σ (W L (. . . σ (W 2 σ (W 1 x)) . . .) where σ(•) is an element-wise nonlinear function). Let [A] be the set {1, 2, . . . , A} for positive integer A.The network parameters are optimized by minimizing an empirical riskEQUATIONwhere L(•, •) is a loss function (e.g., cross-entropy or MSE) and R(•) is a regularization term (e.g., squared L 2 -norm). Let us denote the feature vector of the i k -th training sample of the k-th class by h k,i k (i.e., h k,i k = h θ (x k,i k ) ), with i k ∈ [n k ]. Denote the centralized mean of feature from class k by h k := h k -µ G where h k := 1n k n k i k =1 h k,i k and µ G := 1 K K k=1 h k ; let H := [h 1 , h 2 , . . . , h K ].Recently noticed NC phenomenon [10] shows the weight vectors align with the class-mean featuresEQUATIONfor all k ∈ [K] at the terminal phase of training. where we use I K to denote the K × K identity matrix, 1 K to denote the all-ones vector of size K × 1. The alignment may have alternative shapes for other problem settings such as h is ReLU-activated feature or the data class is imbalanced. 
nc for unconstrained features model with regularized mse loss: 
 To understand the emergence of symmetric structures, recent papers study the "unconstrained features model" (UFM), where the features {h)} and W 0 are treated as free variables. The rationality behind this simplification is the powerful expressivity of a trained neural network. Some use UFM to study the NC phenomenon under MSE loss. EQUATIONEQUATIONgives the NC to ETF, where λ W0 , λ H , and λ b are positive regularization hyper-parameters and ∥ • ∥ F denotes the Frobenius norm. A closely related model is the Bias-Free modelsEQUATIONwhich proves to converge to an Orthogonal Frame [19] . In the next section, we study a bias-free variant of (6). 
main results: 
 In this section we show why the regular classification rule is not optimal, and propose the generalized classification rule and its surrogate losses, then offer a UFM analysis on the NC phenomenon under these losses. Since our paper focuses on bias-free model, we will simply call W 0 the classifier.Let X ⊂ R d0 (d 0 is the input dimension) be the underlying data population, and Z := h(X ) ∈ R d represent the feature population of the trained backbone network h; Assume the trained classifier w 1 , w 2 , . . . , w K align with training samples drawn from X perfectly and thus achieve zero training error.Additionally, assume the classifiers are orthonormal and equally normed. Meanwhile, let the classes of training data C 1 , C 2 , . . . C K be arranged such that the class sample sizes follow a descending order, i.e. n 1 > n 2 > . . . > n K .According to NC, when the neural network arrives at the terminal phase of training, the regular classification rule (RCR) with respect to W 0 isEQUATIONi.e., the 1-nearest neighbor decision rule.Assume Z ∼ α 1 P 1 + α 2 P 2 , the mixture of the subpopulation P 1 which is correctly classified by the decision rule and the subpopulation P 2 that is classified at random, where α 1 and α 2 are the positive weights with α 1 + α 2 = 1.In the analysis, let P 2 have the form z = µ + p, with p ∼ N (0, sI d ) and µ := w where s is a small positive number and w = 1K K k=1w k , which is termed a Hard-To-Predict feature distribution near the global mean of the classifier. For convenience we denote eventsE k := {z ∈ C k |z ∼ P 2 } for all k ∈ [K], and note thatK k=1 P {E k } = 1By virtue of the expressivity of the deep neural network, the population of the major classes will be more concentrated to the classifier than that of the minor classes, so that populations trained by major classes have less probability falling in P 2 , i.e,EQUATIONThen by the gaussianity of p and the orthogonality of the classifiers w 1 , w 2 , . . . , ..., w K , the probabilities of correctly classifying data from P 2 are the same, that is,EQUATION⊤ k z > w ⊤ k ′ z ⇐⇒ w ⊤ k (µ + p) > w ⊤ k ′ (µ + p) ⇐⇒ w ⊤ k p > w ⊤ k ′ p given w ⊤ k µ = w ⊤ k ′ µ.Therefore, by the total probabilityEQUATIONThis calculation inspires us to come up with a different classification rule that outperforms RCR on P 2 . Indeed, we can assign the label y = k to the set { w(k) j } f k j=1 of vectors, for all k ∈ [K]; denote F = K k=1 f k and w ext = 1 F K k=1 f k j=1 w (k)j ; also similar to the RCR, assume all these F vectors are equally normed and mutually orthogonal. These vectors correspond to a Generalized Classification Rule (GCR)EQUATIONor equivalently, f k -nearest neighbor classification rule, where number of nearest neighbors, f k , of an example depends on which class it belongs to. Again by the properties of normal distribution and total probability formula, we obtainEQUATION)Recall the ascending sequence of P {E 1 } . . . P {E K }, so that any choice of f 1 ≤ f 2 ≤ . . . ≤ f K gives higher correctly-classified probability for z ∼ P 2 . If we in addition consider pairwise comparison instead of the one-vs-all fashion in (9), we have for any pair kEQUATIONfor RCR, andEQUATIONfor GCR. In this case, we requiref k < f k ′ for k < k ′ for all pairs of k, k ′ ∈ [K], or equivalantly, f 1 < f 2 < • • • < f K .The analysis as a whole has yielded the following proposition.Proposition 3.1. assume (1) all the classifiers, {w k } in RCR or { w(k) j } in GCR are orthonormal frames; (2) Z ∼ α 1 P 1 +α 2 P 2 , the mixture of the subpopulation P 1 which is correctly classified by the decision rule and the subpopulation P 2 where α 1 and α 2 are the positive weights with α 1 + α 2 = 1;(3) P 2 has the form z = µ + p or z = µ ext + p depending on which classification rules used, with p ∼ N (0, sI d ) where s is a small positive number; (4) P {E 1 } ≤ P {E 2 } ≤ . . . P {E K }. ThenP GCR ≥ P RCR for f 1 ≤ f 2 • • • ≤ f K .The Hard-To-Predict features in the proposition 3.1 are considered to be drawn randomly around the mean of the classifiers. Thus, we are motivated to use more orthogonal directions to classify the minor class. However, the GCR in the analysis does not apply to the practical training of the neural network easily. Indeed, the loss vanishes quickly when training directly with this rule. For this reason, we consider finding a surrogate loss that a) induces neural collapse to an orthogonal frame, and b) the classification rule at the neural collapse approximate GCR.We first define the "centers" that resembleEQUATION)and the data featuresEQUATIONEQUATION1 , . . . , v(K) f K ⊤ is a matrix consisting of f k rows of d-dimensional vectors v (k) ⊤ j's and satisfies equalityEQUATIONwhere[•|•] is the column augmentation of the matrix, and diag(•, . . . , •) is the diagonalization of the block matrices. Then a center of class k is defined asEQUATIONA multi-center frame is the matrix consists of f K rows of wEQUATIONf1 , w1 , . . . , wEQUATIONLet C denote the constraint of the tuple (V, W 0 ) such that [V ⊤ |W 0 ⊤ ] satisfies (16) and∥w k ∥ 2 > 0 are positive for all k ∈ [K].By the definition 3.2, w(k) 1 = w (k) 2 = . . . = w (k) f k for all k ∈ [K],verbally, the centers of each class are equally-normed, and v(k) ⊤ j w ′ k = 0 for all tuple (k, k ′ , j) ∈ [K] × [K] × [f k ] with k ′ ̸ = k. Note d ≥ (f + 1)K is a necessary condition for the existence of (f + 1)K mutually orthogonal d-dim vectors in equation (16) . Figure 3 illustrates the centers of Class 1 and Class 2.Let w (k) := 1 f k f k j=1 w (k)j be the mean of the centers of class k. Since all w (k) j , j ∈ [f k ] are equally normed and equi-angular for each k, we can denoteα * k = ∠(w (k) , w (k) j ) and ρ * k = ∠(w (k) , w k ) with no ambiguity. It is easy to check that cos α * k := cos 2 θ+f k sin 2 θ f k and cos ρ * k := f k sin θ √ f k cos 2 θ+f 2 k sin 2 θ.Then we define the bias-free regression loss for the UFM w.r.t the feature h k,i k and the multi-center frame by1 2S ∥Wh k,i k -y k,i k ∥ 2 2 := 1 2S f j=1 (w (k) ⊤ j h k,i k -1) 2align with the centers of the class+ 1 2S f j ′ =1 K k ′ ̸ =k (w (k ′ ) ⊤ j ′ h k,i k ) 2seperate with the centers of other classes (20) to measure the average extent to which a feature h k,i k collapses to its class centers w (k) j 's while stays away from centers of other classes, where y k,i k is the "f k -hot coding":y k,1 = y k,2 . . . = y k,f k = [ 0, . . . , 0, k-1 m=1 fm 0's 1, . . . , 1, f k 1's 0, . . . , 0] ⊤ .Let Y := [y 1,1 , y 1,2 , ..., y 1,n1 , y 2,1 , . . . , y K,n K ] ∈ R F ×N , the minimization of the regression loss over all features subject to C turns into our prototype optimization problem in this paperEQUATION)The following theorem (proved in appendix D) characterizes the global solutions of the optimization when the data is balanced.EQUATION)EQUATIONEQUATIONEQUATION)or otherwise, the objective P is minimized by (V, W * 0 , H * ) = (0, 0, 0).For imbalanced data and non-identical expansion factors f k , the following theorem shows the relationship between the optimal conditions and the parameters f k , θ, and n k (proved in appendix D).EQUATIONwEQUATION)EQUATION)or otherwise the objective P is minimized by (V, W * 0 , H * ) = (0, 0, 0).Compared to the Theorem 3.3, Theorem 3.4 indicates that ∥h k ∥, ∥w k ∥, and the ratio between them depends on all the expansion factors f j 's and the size of the class n j . Both theorems show the features of class k converge in the direction of w (k) , the mean of the centers of class k. We term this type of collapse "Neural Collapse to Multiple Centers (NCMC)". Remark 3.5. The optimality of V is controlled by the centers. Remark 3.6. NCMC differs from UFM analyses in existing literature since h * k and w * k are not aligned at optimum, and the norm of the optimal classifier depends on the expansion factors of the classes f k . Corollary 3.7 (Corollary of Theorem 3.4). The optimality condition of P as V and W 0 are both fixed satisfiesEQUATIONEQUATIONEQUATION)EQUATIONRemark 3.8. According to the proof of the corollary, the fixed classifier case does not require the condition w.r.t the lower bound of cos α * . Moreover, it is also clear that although h * k aligns with w (k) , the length of it relies on the value of f k nonlinearly. Indeed, from the Corollary,∥h * k ∥ 2 = f k cos α * f k cos 2 α * + λ H S = 1 cos α * k + λ H S f k cos α * k has the global maximum cos α * k = λ H S f k if λ H S < f k . Remark 3.9.NCMC induces an approximate rule to GCR in the following two aspects:(1) the centers are "almost orthogonal" to each other: two centers from different classes are orthogonal to each other. The angle between two enters from the same class is arccos(⟨w (k) j ,w (k) j ′ ⟩ ∥w k ∥ 2 ) = arccos sin 2 θ for j ̸ = j ′ . as θ is small, the angle is close to π 2 .(2) Under NCMC of our problem setting, the RCR w.r.t. the W 0 (see (7) ) can be considered a surrogate of GCR to some extent: if a hard-to-predict feature can be classified by RCR with a margin correctly, then it can be classified correctly by GCR with probability larger than 1 2 . We will discuss this more formally in the proposition C.1.In the Theorem 3.3 and Theorem 3.4, we present the NC conditions for P. However, solving P requires optimization of a scaled orthonormal frame on a non-euclidean manifold, which is computationally expensive for overparameterized models. We hope the classifier can be fixed while not losing its performance severely. We first analyze the fixed classifier via UFM (proof in the appendix D), and later in the next section we propose a practical loss function for the fixed classifier.The proposition3.1 indicates the extra dimensions help improve the classification of "hard-to-predict" samples in the distribution, and basically the expansion factors should satisfyf 1 < f 2 < . . . < f K when the class size decreases, i.e., n 1 ≥ n 2 ≥ . . . ≥ n K . The principle of generating these f k 's is twofold: (1): f 1 ≥ 1; (2): the ratio of ascending {f k }) shall approximates the ratio of descending of {n k }, i.e., f k f k+1 ≈ n K-k n K-k-1 for all k ∈ [K]. Concretely, we use the Class-Aware Strategy to generate the expansion factors f k when f ≥ 2:Step 1: Given the descending list[n 1 , n 2 , . . . , n K ] and scale [n 1 , n 2 , . . . , n K ] to n1 N , n2 N , . . . , n K N ; Step 2: Calculate [a 1 , a 2 , . . . , a K ] where a k = ⌊ (f -1)Kn k N ⌋ + 1, to ensure each element is positive;Step 3: Reverse the order of the list to [a K , a K-1 , . . . , a 1 ] then add 1 from the left until the sum of the elements in the list equals f K.For example, when (n 1 , n 2 , n 3 ) = (1, 3, 3) and f = 3 then Step 1 outputs ( 1 7 , 3 7 , 3 7 ); Step 2 outputs (1, 3, 3); Step 3 outputs (f 1 , f 2 , f 3 ) = (4, 4, 1). 
experiments: 
 In this section, we (1) propose a cosine loss function for fixed classifier; (2) verify NCMC induced by the cosine loss through experiments; (3) show how f and θ influence the learning performance; (4) Compare long-tail classification performance to SETF method with fixed classifier and other classical methods with learnable classifier.We set long-tailed classification tasks on five datasets, CIFAR-10 [38] , CIFAR-100 [38] , SVHN [39] , STL-10 [40] , and large dataset ImageNet [41] with two architectures ResNet-50 and densnet-150 (details in appendix F). Let τ := nmax nmin represent the imbalance ratio of the long-tailed sampling from the dataset, where n max and n min are the size of the largest class and the smallest class, resp. The accuracy results are the average of three repeated experiments with different seeds. The best and second-best results are boldfaced and underlined.The Cosine Regression Loss. Motivated by the toy example 3.1 and the theoretical justification of NCMC, we propose the regularized loss for the fixed unit-norm multi-center frame that satisfies definition 3.2:EQUATION)where Cos(w, h) = ∥⟨w, h ∥h∥ ⟩ -1∥ 2 2 is termed Cosine Loss, λ and β are regularization coefficients (we relegate the selection of the coefficients to appendix F ). This is called a Cosine loss discard all w(k ′ ) j terms of loss (20) for features from class k, where k ′ ̸ = k, since in practice h k aligns with w (k) only if ⟨w (k ′ ) j , h k ⟩ = 0.The derivative of the loss with respect to some feature h then is given byEQUATION(the derivation is postponed to the Appendix E). It shows the gradient changes both the magnitude and the direction of the features with a scale ∥h∥ -2 of the feature, compared to the dot-regression loss in [26] . The regularizer guarantees the gradient does not vanish for the features with large norm and explode for the features with small norm since Cos(w, h) is scaled by ∥h∥ -2 .Note that, the loss uses the Multi-Center Frame W in the training while, due to Remark 3.9, we still use W 0 as the classifier. In the context, we term the objective (34) as f 1 = f 2 = . . . = f K the "Average Loss" (AL), and that with f k selected by class-aware strategy the "Class-Aware Loss" (CAL).We design experiments to verify loss P, AL and CAL induce NCMC. The collapse is measured byN C := h k,i k / ∥h k,i k ∥ -w (k) / w (k) 2 2. For simplicity, we calculate the mean and standard deviations of the vector h k,i k / ∥h k,i k ∥ -w (k) / w (k) when the mean and standard deviation tends to zero, we can show that NC occurs. We fix f = 20 and θ = 0.2; we pick the tuple [f 1 , f 2 , . . . , f K ] by the Class-Aware Strategy. Figure 1 shows that both AL and CAL induce NC. The change of mean is rapid and the standard deviation first increases and then converges to zero slower. Variability is defined as the average of the sum of σ = 1 -h k,i k ∥hk,i k ∥ w k∥w k ∥ over all k and i k , and measures the alignment of H and W, and we observe the in all three objectives the variabilities stay away from zero. Figure 4 plots the NCMC of loss P with or without regularization on the norm of features. The regularization results in heavier NC.We also provide plots of NCMC on VGG and LeNet for CAL that demonstrate the universality of the phenomenon (refer to Table 5 ). We also draw heatmaps of the neural collapse for better visualization. Fig. 6 is the heatmap of H⊤ H where H = [ h1 , h2 , . . . hK ] and hk := h k ∥h k ∥ is the normalized class-mean features. Fig. 7 is the heatmap for the normalized features in class 9 of CIFAR-10; it is noted that all features in this class stay close to each other from the initialization to the end of training. We conduct an ablation study with ResNet50 on CIFAR-100. Table 1 presents the performance of ResNet50 on both balanced CIFAR-100 and its imbalanced sample with or without Mixup training under CAL at imbalance ratio τ = {0.005, 0.01, 0.02, 1(balanced case)}. We do not use objective P since it is inferior to the Cosine Regression Loss (see Table F .1). P is incompatible with mixup training and it may be degenerated by the myriad of distraction terms in the objective.As shown in Table 1 , the performances get improved for almost all imbalanced ratios when the CE is replaced with our designs, especially CAL. And our method is compatible with Mixup training. However, the SETF and our method have lower accuracy than the CE in the balanced case. The analysis in the Proposition 3.1 tells us that in the balanced case where f k 's are identical, our theory will not have a significant positive effect on the classification. Our method has a similar form to SETF who fixes the ETF classifier and emphasizes the gradient norm balance among the classes. On the contrary, CAL concentrates on the fitting to the multiple centers. We compare our method CAL to CE and SETF on four small datasets CIFAR-10, CIFAR-100, SVHN, STL-10 (Table 2 , and Table 5 ). It shows the stability of our Class-Aware Strategy and displays a significant improvement to the original networks.The hyper-parameters have influences on the performance: if θ is small, the centers of a class are very close to each other, else if θ gets larger, the margin of the classifier will be smaller; parameter f encodes partial imbalance information of the classes, when f is small all f k are close to each other and cannot offer useful supervision for the "hart-to-predict" samples. Refer to appendix F for more information about hyperparameter selection.We summarize in Table 6 the performance of CAL as f and θ vary. The result empirically demonstrates: (1) fixing f , the accuracy roughly peaks at some θ bounded away from 0 and π/2.(2) at θ = π/2, The frame collapses to the classifier, resulting in an accuracy rate similar to SETF [26] where the norm of the gradient is weighted by the class imbalance ratio; (3) Frame that is orthogonal classical methods including NC-inspired methods ARB loss and SETF, margin-based LDAM-DRW, contrastive learning method KCL and original CE. The Table 3 shows the methods in comparison. We observe that our method is comparable to or even better than the others. The comparison indicates: (1) Our method has some advantages for heavily imbalanced cases. One of the underlying mechanisms is when the minor classes are underestimated due to the lack of sample, they are likely to display the gaussianity such that the proposition 3.1 and class-aware strategy work fine; as the imbalance ratio τ rise, the minor classes lose their randomness during training, where our method fails.(2) Our method does not compete with the KCL and ARBLoss on ImageNet, reflecting the limitations of our method in the flexibility of the direction and magnitude of the classifier weights when learning large datasets. We also compare our method to a recent work RBL [42] in the Appendix H that demonstrates the potential limitation of the fixed classifier and the advantage of CAL.(3) two classical methods LDAM-DRW and KCL do not compete with other NC-inspired methods trained with Mixup training, demonstrating the effectiveness Mixup training strategy. (4) Our method with the parameter chosen in the experiment outperforms CE negligibly or performs worse than CE when the imbalance ratio approaches 1. However, we find picking f = 10, θ = 0.5 gives an accuracy rate 92.5 ± 0.2 on Cifar-10 with τ = 0.1, which shows the significance of hyperparameter selection for our method. 
conclusion: 
 In this paper, we rethink the regular 1-Nearest Neighbor Classification Rule (RCR) in imbalanced learning; an analysis of the Hard-To-Predict feature indicates under certain circumstances the generalized classification rule (GCR) is superior to RCR, which implies that minor classes should compare to more "neighbors" in the classification. Then we introduce neural collapse to multiple centers (NCMC) under an MSE-type loss, where the centers play a role similar to the neighbors in GCR. According to the framework of the Unconstrained Features Model, the features of each class collapse to the class mean of the centers in balanced or imbalanced settings for learnable or fixed classifiers. We notice that at NCMC, RCR resembles GCR in terms of the hard-to-predict feature distribution. We then propose the cosine loss, a surrogate regression objective of the MSE-type loss called Cosine Loss, that applies to the fixed classifier; and develop the class-aware strategy for determining the number of centers of each class, inspired by the analysis of the Hard-To-Predict Feature. The cosine loss practically induces the NCMC at the terminal phase of training; the combo of the class-aware strategy and the loss with the fixed classifier demonstrates its effectiveness in long-tailed classification. Our work shows the possibility of obtaining a task-specific classification rule by designing the optimal structure at neural collapse under customized losses; it provides a connection among the optimal structure of the feature-classifier alignment, the classification rule, and the generalization in the learning problem. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: the paper poses no such risks Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We use the datasets and refer the papers, see section 4.1. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? 
 
 Appendix: Since the original paper [10] , neural collapse has been intensively studied, with [12, 11] introducing the now widely accepted unconstrained features model (UFM), and layer peeled model. The literature of NC can be loosely categorized into: (a) the study of the geometry of NC [43, 44, 45, 11, 46, 47, 12, 17, 33, 48, 49, 50] , (b) finding connection between NC phenomena and NN performance [27, 25, 51, 52, 26, 53, 54, 35, 55] and (c) empirical observations concerning NC [56, 57] .Long-tail learning refers to the training from data with long tailed-class distributions [58] . We list several categories of methods that are related to our work. Re-sampling. Early works [59, 60, 61, 62] re-sampling is a sampling strategy that balances the sample used for learning by over-sampling the samples from minor classes or under-sampling the samples from major classes. Over-sampling may make the model overfit these classes and harm the generality. Under-sampling removes some samples of major classes, which might remove the key data for representation learning and bring the performance drop.Re-weighting. Another idea is to assign different weights for different classes, even instances. [63, 64, 65, 66] re-weight the loss according to the sample size of the class. [67] introduce the prior probabilities from a Bayesian view and balance the exponential logits via the class sample size.Focal loss [68, 69] re-weight each instance according to their hard-to-learn level, i.e., making the model take more care of the wrong-recognized samples. Wang et al. [70] provide a data-dependent generalization bound that explains the success of re-weighting and logit adjustment strategy.Two-stage Learning. The representations learned from instance-balancing sampling are believed to be general. Many researchers attempt to separate learning process to two stages. [36] trains the model in a normal approach in stage 1 and then uses deferred resampling strategy to finetune with class-balanced resampling or uses deferred re-weighting to re-weight different classes in stage 2. [71] disentangles the basic feature learning and re-balancing learning via two separated branches. [72, 73] decouple the representation learning and classification. They claim that instancebalanced sampling gives more general representation and originally use instance-balanced sampling to learn the representations at first stage, then fix the feature and retrained the classifier (cRT) by techniques such as label-aware smoothing (LAS) and learnable weight scaling (LWS) in the second.Contrastive Learning. Contrastive learning is a learning paradigm that learns representation that maximizes the similarity between positive and negative samples [74] . Khosla et al. [75] initiated Supervised Contrastive Learning (SCL) paradigm by leveraging class labels. For imbalanced classification tasks, Contrastive learning usually faces imbalanced positive/negative pairs of samples.To balance the feature space, KCL [37] balances feature space by using the same number of positive pairs for all the classes; class complement methods [76, 77] are proposed to construct positive and negative pairs for the rebalance of the class. Recent advances in multi-modal foundation models such as CLIP [78] and VLLTR [79] have displayed remarkable performance on the downstream long-tail classification tasks.Learning to Mitigate Minority Collapse. Yang et al. [26] propose the method (we name it "SETF") that learns the representation from imbalanced data with last-layer classifiers fixed as a simplex ETF during training and prove the optimal last-layer features converge to ETF structure; Gao et al. [42] propose the method that optimize the ETF structure under rotation and the uses post-hoc logit-adjustment for prediction. Liu et al. [28] use NC Regularization to minimize the within-class variability and maximize the between-class separateness of the output features; Inspired by neural collapse phenomenon in balanced case, Xie et al. [27] design class-balanced CE loss (termed "ARB loss") that aims to balance the gradient among classes. Zhong et al. [54] discover the minority collapse phenomenon in semantic segmentation and use neural collapse as a regularization to improve the discriminate ability of the network. Learning Test Agonistic Label distribution. In [80] , the authors consider an imbalanced learning task where the training data is long-tailed while the distribution shift between training data and test data is unknown. They deal with the problem by applying diverse expert networks in training to handle different class distributions and aggregate the experts in the test time. Yang et al. [81] propose another expert-mixing strategy that tackles the mild changes in the label distributions.Learning with fixed classifiers. Apart from SETF, other symmetric structures, such as regular polytopes [31, 32, 33] , Hadamard matrix [34] , and hierarchy-aware frame [35] have the classifier fixed in the network training process. Additionally, fixing the classifier is also a computationally friendly strategy since the classifier does not need backpropagation. ′ = 1 K K k=1 w (k) . Denote γ := max j∈[f k ] ⟨v (k) j , z⟩ - max j ′ ∈[f k ′ ] ⟨v (k ′ ) j ′ , z⟩.The difference here to proposition 3.1 is the mean µ ′ induced by NCMC, instead of original NC without the centers under the regular MSE loss. Then we prove that in one-vs-one setting, the original RCREQUATIONwith a small margin implies correct classification with probability over 1 2 by GCR. Proposition C.1. Let W be defined as in definition 3.2 with unit-norm centers and classifiers; assumeEQUATION(2) for anyϵ ′ > ϵ 0 , ⟨w k , z⟩ -⟨w k ′ , z⟩ < cos 2 θ K sin θ ( 1 f k ′ -1 f k ) -ϵ ′ sin θ implies P { max j∈[f k ] ⟨w (k) j , z⟩ < max j ′ ∈[f k ′ ] ⟨w (k ′ ) j ′ , z⟩} > 1 2 .Proof. Due to the orthogonality of V and W 0 , ⟨vj , z⟩ or ⟨v(k ′ )j ′ , z⟩ are still independent isotropic Gaussian, conditioning on the value ofEQUATIONsinceEQUATIONMoreover, the maximum of i.i.d gaussians has continuous (differentiable) density, so ℓ(x) := P {γ > x} is a continuous decreasing function, where ℓ(-∞) = 1, ℓ(∞) = 0, andℓ( cos 2 θ f k K - cos 2 θ f k ′ K ) = P { max j∈[f k ] ⟨v (k) j , p⟩ > max j ′ ∈[f k ′ ] ⟨v (k ′ ) j ′ , p⟩} = f k f k + f k ′ ,Therefore there exists only oneEQUATIONEQUATIONThen with probability over 1 2 , (1) the last few conditions impliesEQUATION= maxEQUATIONEQUATIONfor ϵ < ϵ 0 .(2) Similar to (1), using formula ( 40) and ( 38), we deduce that with probability over 1 2 :EQUATIONEQUATION)for ϵ ′ > ϵ 0 .Proof of Theorem 3.3. In this proof we consider the balanced case where n 1 = n 2 = . . . = n K , andS = f N , Y = I K ⊗ 1 f ⊗ 1 ⊤ n .The proof is to lower bound thef (W, H) := 1 2f N ∥WH -Y∥ 2 F + λ W0 2 ∥W 0 ∥ 2 F + λ H 2 ∥H∥ 2F by a column of inequalities. First, observe thatEQUATIONEQUATIONEQUATIONThe inequality (a) follows from settingw (k ′ ) ⊤ h k,i = 0 (54) for all k ′ ̸ = k and i ∈ [n].In (b) we used Jensen's inequality, which (due to the strict convexity of (• -1) 2 and ∥ • ∥ 2 ) holds with equality iffEQUATIONfor all k ∈ [K].k = h k,i k for all i k ∈ [n k ].Continuing from the last inequality, we haveEQUATIONWe get (c) by Jensen's inequality, which holds with equality iffEQUATION)EQUATION)EQUATION)then continuing from the RHS of the last inequalityEQUATIONIn (d) we use Jensen inequality for the first term, the equality holds whenEQUATIONand young's inequality a 2 + b 2 ≥ √ ab for second and third term, with a = λ W01 K K k=1 ∥w k ∥ 2 2 and b = nλ H 1 K K k=1 ∥h k ∥ 2 2. It holds with equality iffEQUATION)Note the sequel of equality conditions are satisfied by null solution (V, W * 0 , H * ) = (0, 0, 0), so it remains to show when the solution is not trivial.According to the equality conditions and the symmetry w.r.t. k ∈ [k] and j ∈ [f ], the RHS of the last inequality turns into the expressionEQUATIONEQUATIONEQUATION)which obtains minimum at∥w k ∥ 2 = 0 if Kλ W0 - λ W 0 nλ H cos α ≥ 0 and ∥w k ∥ 2 = -Kλ W 0 + λ W 0 nλ H cos α λ W 0 nλ H cos 2 α if Kλ W0 - λ W 0nλ H cos α < 0, where α = ∠(w (k) j , h k ), and for simplicity we denoteEQUATIONEQUATION)In the first case, the minimum is 1 2 ; in the second case, we have to find the smallest value of f (α), which is equivalent to find the lower bound (or upper bound) of α (or cos α) since f (α) is increasingin α if Kλ W0 - λ W 0 nλ H cos α < 0.We observe that α is non-zero, or otherwise by the equality conditions from above, h k has to align with all w (k) j which is impossible. since w (k) j are equiangular and equally normed, for any norm-fixed vector q ∈ R d satisfying⟨w (k) 1 , v⟩ = . . . = ⟨w (k) f , v⟩, we have min α ⇔ max α cos α ⇔ min α f j=1 w (k) j -q 2 (68)which has its optimum when q aligns with w (k) according to the convexity of the last minimization problem, and thus cos α * = cos 2 θ+f sin 2 θ f , the optimal direction also form an angleρ = arccos f sin θ √ cos 2 θ+f sin 2 θ with w k . We have bound α ∈ [K nλ H λ W0 , cos 2 θ+f sin 2 θ f]. the minimum of the objective is achieved at the boundaries. It is easy to see that the value of the objective function is lower than 1 2 , which is the value at the right boundary. Therefore, at the optimum, cos α * = cos 2 θ+f sin 2 θ f andcos ρ = f sin θ √ cos 2 θ+f sin 2 θFor the orthogonality of h k 's and w k 's, since leth k = γ 1 w (k) and h k ′ = γ 2 w (k ′ ) , since w (k ′ ) ⊤ h k = 0, ⟨h k , h k ′ ⟩ = γ 2 1 f f j=1 ⟨w (k ′ ) ⊤ , h k ⟩ = 0. Then we use definition of w (k) j to extend the 0 = ⟨h k , h k ′ ⟩ = 1 f 2 ⟨w k , w k ′ ⟩ sin 2 θ which implies ⟨w k , w k ′ ⟩ = 0.We also have learnt thatEQUATION)The imbalanced case is proved with a slightly different strategy.Proof of Theorem 3.4. In the imbalanced caseEQUATIONEQUATIONEQUATIONEQUATIONEQUATIONEQUATIONEQUATIONwhere (a), (b), and (c) are the same as the last proof. The equality of (e) holds only whenEQUATIONYoung's Inequality. We decompose the objective in (e) different from that in the proof of theorem 3.3 because the existence of non-identical f k fails optimal condition (61). Now we minimizef k w (k) ⊤ h k -1 2 + 2S λ H λ W0 n k ∥h k ∥ ∥w k ∥for each k, where the method in the last proof is applicable, which result in∥w k ∥ 2 = -Sλ W 0 f k n k + λ W 0 n k λ H cos α * k λ W 0 n k λ H cos 2 α * k (77) when cos α * k > S f k λ W 0 λ H n kThe reader can check that the result of the imbalanced setting contains the balanced one as a special case.When W and W 0 are fixed, the optimal conditions become simpler.Proof of Corollary 3.7. Similar to the proof of Theorem 3.4,EQUATIONEQUATION(79)EQUATIONEQUATIONEQUATIONwherew (k) ⊤ h k = w (k) ⊤ j h k = ∥h k ∥ 2 cos α k for all j ∈ [f k ] and α k is the angle between h k and w (k) j for every j ∈ [f k ]. from (a) to (c), the equality holds iffEQUATIONSince each summand is the minimum of a convex function of ∥h k ∥ 2 , P attains its minimum when all of the summands are minimized separately in (d), that is, by the analogous analysis to Theorem 3.3,∥h * k ∥ 2 = f k cos α * k f k cos 2 α * k +λ H S .Since we only use the information of θ at the end of the proof, this generalized NC can hold also true given θ k 's have different values for each k ∈ [K] in the construction of Multi-Center Frame, demonstrating the generality of our method. However, in this paper we are limited to the case of identical θ for all classes.In this section we calculate the derivative of the cosine loss Cos(w, h).For Cos(w, h) = ∥⟨w, h ∥h∥ ⟩ -1∥ 2 , d Cos(w, h) dh = -2 • (1 -a)Jw,where J is the Jacobian of h ∥h∥ w.r.t. h and a = ⟨w,h⟩ ∥h∥ It is straightforward to calculate J = 1 ∥h∥ 3 (∥h∥ Ihh ⊤ ). ∥h∥ Ihh ⊤ has eigenvalue ∥h∥ -∥h∥ 2 on the direction h ∥h∥ and ∥h∥ on all other directions orthogonal to h ∥h∥ , so thatEQUATIONEQUATIONF Implementation Details and ResultsWe run experiments with backbone ResNet50 and DenseNet150 on the four datasets (CIFAR-10, CIFAR-100, SVHN and STL-10) by 1 A100 GPU, and run ResNet50 on ImageNet by 2 A100 GPU with an extra linear layer that expand the dimension of the backbone feature to be larger than f K. We use DenseNet150 with a reduction 0.5; the growth rate is set 12 on CIFAR-10 and CIFAR-100 and 32 on SVHN and STL-10. The reason we adjust growth rate to make the dimension of output feature higher than f K) which is necessary for our method. To display the neural collapse phenomenon, an extra linear layer is added between the backbone and the classifier.We use the code released by [73] to produce the imbalanced datasets. We train the model on the four dataset for 200 epochs, with a step learning rate initialized to 0.1 decaying to 0.01 and 0.001 at epoch 160 and epoch 180, batch size of 128, a momentum of 0.9, and a weight decay of 2e -4. We train on ImageNet for 200 epochs, with a CosineAnnealing learning rate initialized to 0.1, batch size of 512, a momentum of 0.9, and a weight decay of 5e -4.Given imbalance ratio τ = nmin nmax , where n min and n max are the minimal and maximal numbers of training samples in all classes, the numbers of training samples are decayed exponentially from n max to n max among classes. We take the canonical data normalization and augmentation for the five datasets. We use both the regular method and Mixup method [82] to demonstrate the effectiveness of our proposed structure. The hyper-parameter a = 1 by default controls the shape of the symmetric β distribution when Mixup is used. We fix the classifiers as orthonormal vectors for every setting.We train the network by weighted cosine loss with a norm regularization which has the formEQUATIONwhere h k,i k is an example in class k, and λ = 6e -4.The classification rule of our method, as suggested by remark 3.9 is c = argmax k∈[K] BN (h) ∥BN (h)∥ , w k where BN (h) is the batch normalization of h; we use BN since the data may have non-zero mean; For SVHN and STL-10 datasets, we first normalize the backbone outputs h, then train and predict by the same loss and decision rule. For Densenet, we use the BN ( h ∥h∥ ) as the backbone output, apply the original loss to train, and predict by the same decision rule for ResNet50. The code is modified from [26] , and can be found in the supplementary material. Then the upper bound of the gradient norm of a class k becomesEQUATIONCompared with the upper bound of the gradient norm of the unweighted cosine lossEQUATIONOur weighted loss gives the linearity between the upper bound f k and the gradient norm.Figure 4 presents the NC phenomenon under objective P with or without regularization on the feature norm.Figure 5 shows NCMC for different architectures, including DenseNet, ResNet, VGG, and LeNet. 2 .Table 6 shows how the accuracy of imbalanced learning is changed by the parameters f , and θ on CIFAR-100.When picking f k according to the class-aware strategy, we hope f m n m ≈ f l n l for l, m ∈ [K]. This condition, combined with the equality f K = k k=1 f k n k refuses small f when there is a heavy class imbalance. For example, when the K classes are balanced, the strategy requires f m ≈ f l for l, m ∈ [K], and thus f k ≈ f by the equality; on the other hand, for extreme imbalance, say if the ratio τ ≤ 1 bK with b a large constant, we have f K > f K ≥ bK, thus f > b. As f increases from 10 to 20, imbalanced ratio τ = 0.005 (the heaviest ratio in the imbalanced settings), and θ = 0.2, we achieve 80.6 ± 0.5 on cifar10, and 46.6 ± 0.4, both better than the reported in Table 2 .Moreover, we suggest small θ ∈ (0, π/4) in practice (e.g., cos θ ≥ √ 2 Figure 6 shows the inner product of normalized class-mean features during training. We observe that (a) records the contracted mean features at initialization; then the training separates the class means gradually to be orthogonal.We also capture the feature collapse of the most minor class (class size=25) for CIFAR-10 dataset in Fig 7 . It is interesting to note that the collapse occurs almost at the beginning.Impact of our work. 1. Our topic is Imbalanced Classification which is a general concern in machine learing.2. We consider a classification rule that work better than regular classification rule under certain theoretical assumptions in the imbalanced setting. The analysis of "hard-to-predict feature" is novel and can be further developed in general machine learning theory.3. We have studied imbalanced learning through the lens of Neural Collapse, which provides insight into the connection among the optimal feature-classifier alignment, the classification rule, and the performance of DNN.4. We proposed a loss and the strategy for fixed classifier that has comparable performance to methods with learnable classifier; the loss and strategy can apply to general machine learning models.Limitations.1. The theoretical analysis of our motivation only considers Gaussian case;2. Due to the computationally expensive optimization on the Stiefel manifold for large dimension f K, the UFM analysis on learnable classifier in Theorem 3.3 and 3.4 are not justified in practical networks;3. Although the proposed loss is applicable to general classification models, we only conduct experiments on two architectures, ResNet and DenseNet;4. The proposed method for imbalanced learning does not have impressive performance on highdimensional datasets (large number of classes, large number of features), one of the possible reasons is the use of global expansion factor to expand the backbone feature dimension imposes negative effect on the representation of an deep architecture; 5. Although the class-aware strategy is a novel idea, the implementation is far from optimal. The example given in section 3.5 shows f 1 n 1 = 4, f 2 n 2 = 12, and f 3 n 3 = 3 which will result in an emphasis on the middle class instead of the minor class. How to create a strategy that is more "class-aware" will be our future work.6. We compare our work with RBL [42] in Table 7 . It is remarkable that the post-hoc logit adjustment significantly improves RBL. Our method outperforms RBL in two settings for cifar100. We conjecture that in a setting of a small imbalanced ratio and a large number of classes, the hardto-predict distribution may dominate the performance. The reason that our method CAL has lower accuracy compared to RBL in other settings, is three-fold: 1. Imbalanced learning with MSE loss is less effective than CE loss in general; 2. Our experiment setting is chosen as close as possible to that where the theoretical analysis (proposition B.1 and C.1) is conducted, for example, to match the isotropy of the Gaussian, we normalize/batch normalize the feature to ensure it is unit norm and centered before the classification, and the weight are unit vectors through training. This setting possibly harms the learnability and flexibility of our model. 3. Under the class-aware MSE loss, we use the original classifier to be the surrogate classification rule of our general classification rule. However, the rule is designed especially for "hard-to-predict" unseen data and thus is not necessarily optimal for the classification of other unseen data. When the hard-to-predict unseen data takes a very small portion of the population, our design may lose its effectiveness. The success of RBL and PLA inspires us to find an optimal classifier for the loss P and CAL. • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We discuss impact in appendix H Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).), since large θ generates concentrated centers of the classes, which is more likely to overweight and overfit the minor classes (consider the case of cos θ = 1 where all the centers collapse to one direction, then the gradient directions will be severely skewed). 


Decision: This paper studies the Neural Collapse (NC) phenomenon in imbalanced learning and finds that minor classes should align with more directions for improved classification results.

Initially, this paper received mixed scores, but the authors effectively addressed these concerns in their rebuttal. Consequently, I recommend accepting the paper. Nonetheless, the authors are required to follow the reviewers' suggestions and integrate their rebuttal content into the original paper.

Reviews: review: paper_summary: paper_summary: This paper studies the Neural Collapse phenomenon under the imbalanced training data. The authors extend the optimal structure of neural collapse classification to a multiple center setting to enhance the model performance. Specifically, the authors propose to leverage the Generalized Classification Rule to make the minor classes align with more directions. Moreover, a practical MSE-type objective function has been proposed to train a "neural collapse to multiple centers" model. The proposed method achieves comparable performance with existing baselines.
strength_weakness: strength_weakness: - The performance improvement of the proposed CAL in Table 4 is marginal in comparison to other baselines. 

- The writing of the paper can be further improved. 
  - The challenge and motivation of this work hasn't been well addressed in the introduction. Content in line 43-63 is more likely to appear in related work rather than introduction. 
  - It is hard for readers to understand the meaning of a sentence when too many abbreviated words come together, e.g., at line 73-77.
questions: questions: Please refer to the weakness section.
limitations: limitations: The authors have discussed the limitation of the proposed method, including the assumption made in the theoretical analysis, the computational cost and the not impressive performance on high-dimensional datasets.
paper_summary: paper_summary: This paper studies the Neural Collapse (NC) phenomenon in imbalanced learning. Specifically, the authors find that the minor classes should align with more directions to achieve better classification results. Such finding yields the Generalized Classification Rule (GCR). The authors study NC under UFM. They find that the features of a class $k$ tend to collapse to the mean of centers of class $k$ which is termed Neural Collapse to Multiple Centers (NCMC), and RCR (the original classifier) approximates GCR at NCMC. Based on the above studies, the authors propose Cosine Loss and show from experiments that Cosine Loss can induce NCMC and has comparable performance to classical long-tail learning methods.
strength_weakness: strength_weakness: This paper is generally well-written without much weaknesses. Here are a few possible points.

1. The introduction of $\widetilde{\boldsymbol{w}}_j^{(k)}$ is a bit abrupt at line 121-124. I would suggest more explanations including in the corresponding appendix section. Also, the tilde symbol is missing in Eq.(8).
2. Some minor typos. Line 95 “denote the (mean?) of features of class $k$…”. Line 96 “$\boldsymbol{h}_k:==$” double =’s. Line 104 “$\{\boldsymbol{h})\}$” redundant ‘)’. End of line 138 “satisfies” to “satisfy”. Line 197 “approximates” to “approximate”.
3. Citation recommendations. I believe your work would benefit from referencing some additional literature to provide a more comprehensive context for your study. Specifically, i recommend citing the following articles:
  -  A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning (NeurIPS 23)
  - Understanding imbalanced semantic segmentation through neural collapse (CVPR 23)
  - Deep long-tailed learning: A survey (TPAMI 23)
  - Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition (ICML 24)
questions: questions: One question: why study under UFM? Is this generalizable?
limitations: limitations: See *Weaknesses*.
paper_summary: paper_summary: This paper addresses the issue of minority collapse in imbalanced learning, finding an optimal structure to represent a better classification rule. The authors induce a new definition called NCMC and design an MSE-type loss to alleviate the minority collapse phenomenon.
strength_weakness: strength_weakness: The experimental results did not show significant improvement; the accuracies of these loss functions are quite similar. For example, in Table 4's CIFAR-100 experiments, the results might all be within the margin of error, suggesting that the newly proposed loss function may not be very effective.
questions: questions: - Could the authors provide more visualizations of multiple centers?

- In the experiments, what is $\tau$ and $\theta$ refer to? How is the degree of imbalance discussed?

- Is the conclusion only applicable to MSE loss, or could it also apply to CE loss or other loss functions designed to handle imbalanced data?

- Is the NCMC phenomenon, like NC, model-agnostic? Would different backbones affect the conclusion?
limitations: limitations: N/A
paper_summary: paper_summary: This paper explores Neural Collapse (NC) in the context of imbalanced data, proposing the concept of Neural Collapse to Multiple Centers (NCMC). It establishes that aligning features from minor classes with more directions improves classification accuracy, introducing the Generalized Classification Rule (GCR). The authors design an MSE-type objective and a practical loss function that induces NCMC, achieving performance comparable to classical imbalanced learning methods. Their findings are validated both theoretically and experimentally, offering new insights into the application of NC in imbalanced learning scenarios.
strength_weakness: strength_weakness: + Figure 1 should be larger for better visualization for readers.
+ Theorems 3.3 and 3.4 prove that the optimization problem $\mathbf{P}$ can induce the "Neural Collapse to Multiple Centers" solution, but the loss function in $\mathbf{P}$ does not appear in the experiments. It might be better to include the loss function $\mathbf{P}$ in the comparison experiments and compare, analyze, and explain the results. 
+ If multiple centers are better than a single center for each class in hard-to-predict feature distribution, why don't the authors directly use the loss function in $\mathbf{P}$ instead of designing a Cosine Regression Loss? The authors should explain this intent.
+ [1], as a study about long-tailed learning and NC, should be compared within this study. 
+ I personally think that Proposition 3.1 deserves more space and discussion in the main text, as it serves as a key motivational component for the paper. Feel free to adopt this suggestion; this point does not affect my rating. 

[1] Feature Directions Matter: Long-Tailed Learning via Rotated Balanced Representation. Gao Peifeng, Qianqian Xu, Peisong Wen, Zhiyong Yang, Huiyang Shao, Qingming Huang Proceedings of the 40th International Conference on Machine Learning, PMLR 202:27542-27563, 2023.
questions: questions: see Weaknesses.
limitations: limitations: see Weaknesses.

Comments: title: Response
We appreciate your higher rating.
comment: We are grateful for the reviewer's reconsideration.
Thanks for the detailed response. I will raise my score to 6.
Thank you for your re-evaluation of our work. We will revise the paper carefully according to the rebuttal.
Thanks for the authors' response, which has addressed most of my concerns. I will raise my score to borderline accept. I hope the authors can further revise the paper as they promised in the rebuttal.
Thank you for your feedback.
Thank you for your response. I would keep my scores.
We appreciate your higher rating.