Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment
Authors: Naoya Hasegawa
Issei Sato
Venue: ICLR 2025 Poster (2025)
Keywords: long-tailed recognition
imbalanced learning
neural collapse
logit adjustment
multiplicative logit adjustment
machine learning
learning theory
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We establish a theoretical basis for understanding the effectiveness of multiplicative logit adjustment in long-tailed recognition.

Abstract: Real-world data distributions are often highly skewed. This has spurred a growing body of research on long-tailed recognition, aimed at addressing the imbalance in training classification models. Among the methods studied, multiplicative logit adjustment (MLA) stands out as a simple and effective method. What theoretical foundation explains the effectiveness of this heuristic method?
We provide a justification for the effectiveness of MLA with the following two-step process. First, we develop a theory that adjusts optimal decision boundaries by estimating feature spread on the basis of neural collapse. Second, we demonstrate that MLA approximates this optimal method. Additionally, through experiments on long-tailed datasets, we illustrate the practical usefulness of MLA under more realistic conditions. We also offer experimental insights to guide the tuning of MLA hyperparameters.

Published as a conference paper at ICLR 2025
MULTIPLICATIVE LOGIT ADJUSTMENT APPROXIMATES NEURAL-COLLAAPEEAAARR DECISION BOUNDARY ADJUSTMENT
Naoya Hasegawa & Issei Sato
The University of Tokyo
Slasegawa-naoya411 sato) ecc. wwt.c..-tkyo..oo ac. jp
ABSTRACT
woal-world data distributions are often highly skewed. This has spurred a growing body of research long-tailed recognition, aimed at addressing the imbalance in training classification models. Among the methods studied, multiplicative logit adjustment (MLA) stands out simple and effective method. What theoret- ical foundation explains the effectiveness of this heuristic method? We provide justification for the effectiveness of MIA with the following two-step process. First, we develop a theory that adjusts optimal decision boundaries by estimat- ing feature spread on the basis of neural collapse. Second, we demonstrate that MLA approximates this optimal method. Additionally, through experiments on long-taile tailed datasets, we illustrate the practical usefulness of MLA under more re- alistic conditions. We also offer experimental insights to guide the tuning of MLA hyperparameters.
INTRODUCTION
Publicly available benchmark datasets commonly used to evaluate classification models, such as MNIST (Lecnnetal all (198) and CIFAR100 (Krizevsky,, 2009), usually have balanced number of samples per class. In contrast to benchmark datasets, empirical evidence indicates that real-world data often follow an exponential distribution (Reed, (XTI). This also holds for classification prob- lems involving real-world data (Span & Perona, 2007: Li et all 2017 2022a). Such distributions are commonly referred to as long-tailed data. long-tailed data, few classes (head classes) have a large number of samples, while most classes (tail classes) have only limited number of samples. This imbalance poses a significant challenge in classification tasks, known as long-tailed recognition (LTR). LTR focuses on improving the accuracy of models trained on long-tailed data when evalu- ated on uniformly distributed data. LTR, there are many classes, and model predictions are often biased toward head classes. Since tail classes make up the majority, this bias significantly reduces the overall average accuracy across all classes (Zhang et all 2021: Yang et all 2022a).
Various methods have been proposed for LTR. State-of-the-art methods (Ma all 2021; Long all 2022: Tian et all 2022) are generally complex but often consist combinations of simpler techniques, such resampling (Drummond & Holte, 2IXE and two-stage training (Kang et al.
2020) One straightforward method is logit adjustment (LA) (Menon el all 2020; Kim & Kim.
2021). Among these techniques, post-hoc LA is technique that adjusts the linear classifier on top of the feature map without additional training after the standard training process, offering both effectiveness and simplicity. prominent example of post-hoc LA is additive LA (ALA) (Menon' et all 2020), which is based on Fisher-consistent loss. Another type of post-hoe LA is multiplica- tive LA (MLA) (Kim Kimi, 2020), which aims to adjust the decision boundaries on the basis of the differences in the feature spread across classes. Since MLA has also demonstrated significant empirical success (Hasegawa & Sato, MI2K), raises the following question.
Is there theoretical foundation behind the effectiveness of MLA?
Contribution We provide a theoretical guarantee that MLA achieves near-optimal decision boundary adjustments. Our contributions are summarized follows.

---

Published as a conference paper at ICLR 2025
We derive theoretical framework for adjusting decision boundaries optimally by using feature spread estimates from neural collapse (NC) (Papyan et all (020) (Section 1
We demonstrate that MLA effective for LTR by proving it has similar decision boundaries to the method based on the aforementioned theory (Section 9.33). This clarifies under what conditions this approximation holds and how adjustments should be made.
We experimentally validate that the approximation of MLA holds under realistic, non-ideal conditions where NC is not fully realized (Section ED
We provide empirical guidelines for hyperparammette-tuumii of MLA (Section 5.4).
2 RELATED WORK
Long-Tailed Recognition LTR methods fall into three types: "Information Augmentation", "Module Improvement", and "Class Re-balancing" (Zhang all, 202I). "Information Augmen- tation" aims to mitigate accuracy degradation by supplementing the limited information available
for tail classes eLw 2020: Chu 2022; Wang et all 202T: Lretal 20226; [Wan et all. 202A5). "Module Improvement" focuses on enhancing individual components of the network to boost overall performance. For example, Kang et al. (2020) developed a two-stage training method that separates the training of feature maps and classifiers. Yang all (2022h) and Li all (2023)
proposed methods to promote NC in feature maps. Recent approaches also use contrastive learning (Hadsell all 2006) (Ma et all 2021; Tian et all, 2022: ell 2022e Wang et all, [02;) Kang
2t at. 2023) and vision-language models such as CLIP (Radford et all (021) eMletall 2024: I: Tian
2112 Long all (022). Despite these advancements, managing these methods can be chal- lenging due to slow convergence eL al, ))2 or complex models (Hasegawa & Sato, 2(2%)
"Class Re-balancing" strategies adjust class imbalance prevent accuracy degradation. Techniques in this category include resampling (Drummond & Holre, 20N3: Li all 2022h) and loss reweight- ing (Cu et all 2019; Ma et all (022).. However, reweighting known be ineffective for training overparameterized models (Byrd & Lipton, 2019; Zhai et all 2022). LA is a simple yet effective method that falls under this category.
Logit Adjustment LA modifies logit values by using the number of samples in each class, to bridge the distribution gap between the training and test data. LA methods fall into two categories: loss-function adjustments (Cao et all, 2019; Menon all, 2020: [Tang et all, 2020; Kiniet all, 2021; Wang et al, (023H) and post-hoc LA methods that do not require additional training (Menon et all, 2020; Kum & Kim
2020.. The latter particularly advantageous as eliminates the need for retraining during hyperparameter tuning, thereby reducing both effort and time. Consequently, this paper delves deeper into post-hoc LA. Among post-hoc methods, there are two types depending on how the logits are adjusted: ALA (Menon et all 2021) and MLA (Kim & Kim. 2020). ALA is grounded a Fisher-consistent loss (Menon el all, 2020), providing some theoretical justification for its method. However, ALA assumes infinite number of training samples, raising questions about its effectiveness with finite sample sizes. In contrast, MLA intuitively adjusts decision boundaries
by using the size of each class cluster (Kım & Kim 2020). MLA has empirically outperformed ALA in certain cases (Hasegawa & Sato, 2023).
Neural Collapse NC is term introduced by Papyan et all (2122 to describe four phenomena observed in the terminal phase of training classification models. These phenomena are as follows.
NC1: Training feature vectors within each class converge to their respective class means.
NC2: The class means training feature vectors converge to form a simplex Equiangular Tight Frame (ETF) (Strohmer & Heath 200E).
NC3: The class means of the training feature vectors align with the corresponding weight vectors of the linear classifier.
NC4: In prediction, the model outputs the class whose mean training feature vector is closest to the input feature vector in Euclidean distance.
NC is known to improve generalization accuracy and robustness (Papyan et al, 2020). Addition- ally, Galanti all (202T) theoretically demonstrated that NC can occur even with test samples or
2

---

Published as a conference paper at ICLR 2025
samples from unseen classes. Various studies have explored the conditions under which NC arises (Rangamani & Banburski-Pahge 2022: all 2122). and it is known that models trained using cross-entropy loss can also exhibit NC (Lu & Stemerberger, 202T ettall all, (022). However, it has been observed that training with imbalanced data may hinder NC (Fang et all 2021: Thhrmpouliddi all 2022, Dang et all 2024).. To address this problem, some methods have been proposed to promote NC even with imbalanced data (Yang et al, 20228; L all 2023). Yang et all (212226 introduced the ETF classifier, which fixes the linear classifier weights to form an ETF, thereby en- couraging NC.
3 PRELIMINARIES
We outline the notations used throughout this paper. For a comprehensive list, refer to the table in Appendix A For any integer i. [i] represents the set (1.....)). We use X' C RR to denote the instance space and y = [K] to denote the label space. In a K-class classification problem, each sample (æ, v) E X x Y' is drawn from a distribution P. The training dataset, S = { U1) x; E X. E >)}= consists of independent and identically distributed (i.i.d)) samples drawn from P. Partitioning by class, we define S₁ {xx (a.,, 34) E S,y k} as the set of samples drawn from the class-conditee conditionl distribution Px. Let N = |S] be the total number of samples, and n& = [Sy] be the number of samples in class k. Without loss of generality,
assume the classes are sorted in descending order of sample size, i.e., 7 2 n4+1 for all E [K- 1]. The imbalance factor is defined by p RKK ming rig" which measures the degree of imbalance in the training dataset, where p >> holds in LTR scenarios. In contrast the imbalanced training dataset, the test dataset used for evaluation has a uniform label distribution. Specifically, we denote Sy PA by the test dataset for class k with ňg samples. Then, it holds that ň. = ng for all k, K E.
masg
Let f R' -> Rd be feature map from the set F C {f R Rd). We focus the input to the final layer of this feature map. Suppose the features are given by f(x)- = h(()., where W1 Raxdi is linear layer, and h E H : IR - Rd represents the output of the second-to- last layer of the feature map. For simplicity, we assume that h is bounded, meaning there exists a constant > such that sup [[())) s B, where || denotes the Euclidean norm for vectors.
Let the expected value and mean of the features be denoted as (y((ss) Ex- [=x)] and (S)(S))= 1 Exes, far), respectively. When f is clear from the context, we abbreviate these to nA
(px)) and (Sx)).
These features are fed into linear classifier W E RaxK K and we obtain the logits g(x) = W f(x), where .T denotes the transpose of a matrix. The linear classifier weights can be ex- pressed using column vectors Wy E IR, where W = [wa...., wx). Thus, the logit for class k can be written as (x) = w. f(x). We assume that the weights of the linear classifier form an ETF. This assumption holds under NC and when we use ETF classifier (Yang ct all 20226). This leads to the following equation:
k otherwise.
We wa
(1)
Logit Adjustment MLA is method that adjusts logits by scaling the norm of the linear classifier weights for each class (Kim & Kim, 2020). This is equivalent in classification outcome to adjusting the logit for class k, 9u (x), by scaling it as ny -"x (x)... Here, 7x > 0 is hyperparameter. Kim Kiml (2020) used projected gradient descent to normalize the norm of the linear layer, [ww]] to
1. This normalization step is unnecessary in our setting because we use an ETF classifier and the weights remain fixed. In contrast, ALA adjusts logits additively. This is equivalen"" to modifying the logit for class k gx(x), by subtracting 7/ log Tlk, where 7+ > 0 is hyperparameter.
Kum & Kmm (020)) adjusted logits to n1
7x
gx(2).. while Menon et all (2121) adjusted them to
Dh (x) Y+ log Σ Note that both methods differ from the adjustment used in this paper only by
constant multiple or constant term. However, since this does not affect the ranking of logits across classes or the classification outcomes, we adopt the adjustments described in the main text of this paper.
3

---

Published as a conference paper at ICLR 2025
4
MAIN ANALYSIS
MLA is a post-hoc adjustment method that increases the probability of classifying samples into tail classes by changing the angles of the decision boundaries (Kum & Kim, 2020). To derive the appropriate angles, it is necessary to estimate the sizes of the feature spread for each class. Galanti (202)) demonstrated that NC also occurs in test samples and provided quantitative measure of feature spread. Following their approach, we verify the effectiveness of MLA theoretically through the lens of NC. In the following sections, encourage NC. This assumption can also be derived from the assumption that NC2 and NC3 hold. Using an ETF classifier also ensures that the weight vectors are normalized, as shown be effective by Kim & Kim (2020)..
employ the ETF classifier (Yang et all 2(2))) to
First, in Section we outline the assumptions and additional notations necessary present our theory. Next, in Section 2 we demonstrate how to optimally adjust the decision boundaries from the perspective of NC. Section P. we indicate that MLA approximates this adjustment. Finally,
Section 4 compares ALA and MLA in view of our theory.
4.1 PRELIMINARIES: SVD AND RADEMACHER COMPLEXITY
We consider the singular value decomposition of W When W has rank r, we can express as W -i= sannes where for each [r].
Rd, U1 E Rd and 8I R>o. Moreover, for all L E [r]. have || = ||//|| 1. and for 1 * I' [r]. vi Uy uj up O. Without loss of generality, we assume that Al E [r [r-1]: 1].
. 8t 2 8141- In many deep neural networks, weight matrices be replaced with low-rank representations while maintaining performance (Frankle & Carbin, 201& [delbayev Carreira-Perpiin
2020, Kajitsuka Sato 2023). Thus, it is reasonable consider mnindd di) in practical situations.
We define the set Hi as:
Hi = ( h) I JFF W',, vr is the 1-th right singular vector of W
(2)
To evaluate the generalization performance of Hi, we use the Rademacher complexity. Rademacher complexity and empirical Rademacher complexity of Hi are defined by:
The
Rag (H4) E Ppt
sup (o),))EHi ES Σ ofe7 h(xz))
(3)
Rs (H4) = Eg
Σ o h(22,)) sup (w,,))HH Th 2,E8,
(4)
where [0 lo......, n. represents the i.i.d. Rademacher random variables. In many cases, C(H,,X) where C(Hi, X') is some complexity
the Rademacher complexity scales as Rn, (Hi) <
measure independent of nı (Cap all 2019; Galanti et all 2021). For instance, this holds for rectified linear unit (ReLU) neural networks; see Appendix C for further details. We make this
assumption in our settings. We define the mean of the complexity C(F, = Ei= ((H,,
4.2 OPTIMAL DECISION BOUNDARY ADJUSTMENT FROM THE PERSPECTIVE OF NC
We indicate that the following result holds for general neural networks. For the proofs of the propo- sitions presented in this section and for the case of ReLU neural networks, see Appendix D
In NC, the training features converge their respective class means (Papyan et all 2020). It has also been demonstrated that test features are within certain range from the class mean of training features, with this range depending on the sample size (Galant et all 2021). Following a sim- ilar approach, can quantitatively measure the angular deviation between test features and the corresponding training class mean, as a function of the sample size. In LTR, sample sizes vary sig- nificantly across classes, resulting in considerable imbalance in the angular deviation. Thus, aim to adjust the decision boundary accordingly.
To achieve this, first define the angular bound probability IT(0; k) follows.
4

---

Published as a conference paper at ICLR 2025
Prop.
Prop.
[] S Pr
in
[[ k) S Pr
in
[(
1((S))
II(O K3) x Pr
-i
[( b) + ()
Wi
f(m)
Ok.
0
0
Pr
0g R
8 p
0
O
Figure 1: Overview of Propositions M and A The angular bound probability (0; k) represents the lower bound of the probability that the feature vector of æ sampled from Ps lies within the shaded region. Proposition D indicates that [I(0; k) decision boundary by maximizing [[ k) + II(O ar k') with respect to fg.K' and O
Ô (1//Tk), Proposition Q offers the optimal
Definition (Angular bound probability). The angular bound probability II(0; k) for class k is the lower bound of the probability that the angle between I fax and p(Sa), for æ Pe. is less than O.
That is, we have that I1(0; k) Præ~Ps
(x)) Pi s(())) 0], where 2(--)) is the angle between the two vectors. The angular bound probability can also be seen as concentration of the class k features. The following proposition provides quantitative measure of II(#; k):
Proposition 1. Suppose n& > 2. For f E F, assume that for all x E St, f(x) = ((Sx) holds.
Consider any that satisfies the following condition:
W 2 ThN (I()
Vnglle (Oi|)p((sa)11
AC(F,X) + 4B + +B+
B1/
2 2og
<0< 2'
(5)
where || ||2 denotes the spectral norm. For such 0, the following holds:
II(0: k) 1- π
1 ((
(AC(F, + 4B 4 1 B
20 Vnn [((sa)]
2 log
(6)
20
The overview of this proposition is illustrated the left side of Figure When NC occurs, note that f ((()) holds for all æ E Sy by NCI and w ((.) that the left-hand side of Eq. (5) and
(T holds by NC3. Additionally, observe [I k) are ((1/ where Ô denotes asymptotic notation ignoring logarithmic terms. From this proposition, we can quantitatively infer that as the sample size of a class increases, the features become more concentrated within a narrower region, with higher probability. This allows us consider the optimal angle for the decision boundary.
Consider the optimal angle for the decision boundary between two classes, k and k'. Since we use an ETF classifier, the linear classifier weights forms an ETF structure, meaning the angle between Wy and wy is 1 arccos
R Now, consider decision boundary that forms angle 0k. with uc. The angle between this boundary and we is Ogr, & = 1 -2.,e. Here, 0 < 0R,22 Ogr, < db.
In this setup, the probability that sample
Pa lies on the wi side of the boundary is least T( k). Similarly, the probability that a sample x ~ Per lies on the wy side is at least I[ As K"). Therefore, the accuracy of classification between the two classes is at least (II(OR,er) k) I[00 (())),,;)))) which depends on the angle 0R of the decision boundary. Thus, we seek the value of 0R.1' that maximizes this lower bound. The following proposition provides the solution to this problem.
Proposition 2. Suppose Toe >> 2. For f E F, assume that for both & (ki, k'., f(x) = (((S)) holds for all x E Sz. Consider the following maximization problem:
max II(Ok,Mr;)) k) + 11(0 ki k") s.t. 0g. 0x. > 0, 0K + 0 = =b.
(7)
9 0,,0 M
5

---

Published as a conference paper at ICLR 2025
The unique solution fi & within the range where 0k,k' and Ow satisfy Eq. (E) is given by: (| y
[1((S))) 31 Vnw
O v/ (p((s))l + [| )llm/// 05 1 Il (p(S))ll/m + [ )l
(8)
The overview of this proposition is illustrated in the right side of Figure II. When ni and nigs sufficiently large, Eq. (5) holds over wide range of 0 2- Moreover, K + 0c,
& arccos
- _ and x 0 < holds, making Eq. (S) meet the condition of Eq. (5) when K' is sufficiently large. This is reasonable assumption in LTR (Yang et all, 2022a). In practice, for CIFARIOO-LT, ab- ~ 0.01, meaning that 0%,'', G < 2 is sufficiently valid.
This result can easily be extended to multi-class cases (refer to Proposition E), In multi-class cases, it becomes an optimization problem of the angles 0g.K' for every pair of classes k. k' + k in V defined as = {8x. |k + K' E e.).
Proposition 3. Suppose n& > for all l E Y. For f E assume that for all k E y. f(x) =
(((s)) holds for all x E Sk- Consider the following maximization problem:
max Σ
[(0k,wrk)) +I((0wask) s.I. Vk, k' E V,O, Ow k > and Og + Ogg = vb
(9)
The unique solution x E 0* within the range where all 0%. O satisfies Eq. (B) is given by: (1 /nk O = t/ [| Vnn + ((Ss))llvm)
(10)
These theories suggest replacing the linear layer with 1-vs-1 multi-class classifier with decision boundaries of O+. To generalize, we introduce hyperparameter 7iv1, and define of (Tiv))
( ][nng 71v1
with 6 i =\t ((vv) | + K E y). Note that O% O
(1.) [ng Tiv1 + "Im
holds when yvvv 2. We refer to the 1-vs-1 multi-elass classifier that classifies based on this O =
as Ivs Iadjusstrr. See Appendix D for the specific algorithm of 1vsl adjuster.
4.3 MLA SOLVES LTR PROBLEMS WITH DECISION BOUNDARIES AKIN TO IVS ADJUSTER
We demonstrate that MLA addresses the LTR problem. Since Section ! proves Ivsladjuster has the optimal decision boundaries for LTR, we verify MLA operates similarly to I MLA can be viewed as technique that multiplicatively adjusts the cosine similarity between features and the linear layer weight vectors. This increases the probability of classifying samples into tail classes. Although our theory in Section aims find the optimal decision boundaries between classes, can be connected to MLA by considering the magnitude of logits under simple assumptions.
For the following discussion, we assume that K is sufficiently large. Under this condition, it holds that & arccos
_.. allowing us to assume that cos(O (Tiv1)), cos(0% uu (Tiv1)) > 0.
R-T
Let the MLA factor, multiplicatively applied to the logits of class k. be denoted by Kg. In other
words, we consider the case in which the logit (x) for class & is adjusted to (x). We present that when adjusting the decision boundaries that the ETF classifier ensures equal norms of the linear layer weight vectors across all classes. To adjust the decision boundaries be equivalent such that the following holds for all k + k' E y:
the basis of I vssladie adjusterr K& x n, 7iva holds. Note
Ivsladjuster by using MLA, we should adjust Kk
KA cos (07, (7tw1) Ki cos(Oi:.((Tiv))
(11)
A problem arises here because there are only K MLA factors, while they must satisfy K(K- 1)/2 equations. Consequently, when K > 2, there is typically no solution. However, by considering
the following approximation, it is possible to obtain an approximate solution. First, note that as
K - 00, it holds that d + 5. Letting = 2 v. and considering the continuity of cos, we can approximate and solve Eq. (IT) follows:
Ki tan (Tiv)) cos(e) - sin(e)
(12)
KAy
~ (
(13)

---

Published as a conference paper at ICLR 2025
(lll2) ( I then we have:
Now, let Te.,e((ivi)
(1(wx) I|
1
0j (1a n + [1
The (Tivi)
2 Thi (7iv1) +
(14)
Next, for 2 < 1, we approximate tan (30) using the rational function ((0)
1-8- This
approximation satisfies the following properties: Lemma 1.
(00) ta) O,
(15)
1
tan
(16)
2
lim (0) lim tan
(20) +00.
T
(17)
6-1-0
8-110
Lemma 2. Let 0 [0, 1). Then, the following holds:
0 05 1
(0) S - (0)
(18)
2 (0) - (0
1
<1
(19)
2
The proof of Lemma is trivial and thus omitted. The proof of Lemma A can be found in Appendix
8 o These lemmas suggest that the approximation ((0) = i-8 for tan (2)) f) is particularly close
when 0 or (71va) 1, but it provides global approximation. This is especially suitable for LTR settings, where Tk, (Tiv1) tends to deviate significantly from
Using these results, further approximate Eq. (I ( as:
2 f f (1ivi)
Kk Kg
tan
(f (7ww1) ~ 1 0 o (11v1)
k.
(20)
Therefore, if we set K&
Ii for any a Rxo- we can approximately satisfy Propo- sition B This corresponds to performing MLA with 7x Tivl, under the assumption of feature normalization.
4.4 DISCUSSION: COMPARISON OF MLA WITH ALA
In Section B
demonstrate that MILA is grounded in the theoretical framework of decision boundary adjustment developed in Section 2 We now turn our attention to ALA, presenting the issues that arise when applying the theory from Section A to ALA. This highlights the differences between MLA and ALA, and indicates the relative advantages of MLA.
ALA inconsistent with the concept of optimal decision boundary adjustment described in Proposition B This is because the decision boundaries adjusted by ALA changes depending on the norm of each feature, [x)())) For instance, let's consider an instance æ whose logit of class k is represented depends on the magnitude of [()) cos(<(f(a)) wxx) - 7+ log nig for each In other words, the classification result depends both the angle [xf(w,, wx) and the norm f
gt(xx) = [x) (oo)))oos(((( w.)). The output of the classification label for
(x)) ||. This implies that the optimal decision boundaries during inference are affected by the norm of individual fea- tures. This dependency conflicts with Proposition B, in which the optimal decision boundaries independent the norm of each feature.
On the other hand, suppose assume that [=) is equal for all E X, ensuring that the angles of the decision boundaries remain constant for each sample. Following the same reasoning in Section A.3 can also approximate Ivsladjuster by ALA. However, compared to MIL,,
7

---

Published as a conference paper at ICLR 2025
this approximation cannot hold globally when Tk,k (Twi) varies significantly. This inconsistency with LTR settings, in which Tha (Tiv1) can deviate considerably, makes ALA incompatible. See Appendix (EI for details.
These points are corroborated by the subsequent experimental results, which demonstrate the sub- stantial differences between the properties of MLA and ALA.
5 EXPERIMENTS
We experimentally demonstrate that MLA is an appropriate approximation of Ivs Ivsladjuster from two perspectives. First, we outline the experimental setup in Section 1.111 Then, we demonstrate that MLA and Ivsladjuster share similar decision boundaries in Section 5.2 and yield equivalent classi- fication accuracy Section 5.3 Finally, we present insights into MLA hyperparameters Section
5.4
Code is available at https: /ttp://tttus com /N4 MLA Apwroxipproxiiaaomameeeeeeee NCCD NCDBA,
5.1 SETTINGS
Our experimental setup primarily follows Hasegawa Sato (2023). We used CIFARIO, CIFARI00 (Knzhevsky, 201E), and ImageNet (Deng all 2009. as datasets. Following Cu al (2019) and
Lulctal (2011), we created long- tailed versions of these datasets, namely CIFAR CIFAR 100- LT, and ImtgeNet-LT, with imbalance ratio of p 100. To demonstrate that this method applies to general modal data, not just images, we also used the tabular dataset Helena (Guyon et all, 2019). We can also show the effectiveness of the method on real data using Helena because this is an inherently imbalanced dataset with p ~ 40.
For the network architecture, we used ResNeXt50 eLal 2010 for ImageNet-LT and ResNet34 eHe all 2016) for the other image datasets unless otherwise specified. For Helena, we used multi-layer perceptron (MLP), following Kadra at (2022T). We adopted the ETF classifier as the linear classifier to promote NC and fix the linear classifier weights. Thus, only the parameters of the feature map were trained. We used cross-entropy loss as the loss function and applied weight decay (Hanson & Prati, (189) and feature regularization as regularization techniques. Feature regulariza- tion was employed to promote NC (Hasegawa Sato, 2023) and to prevent significant diffcrences in the norms of the training features across classes, fulfilling the condition for MLA to approximate Iss adjuster. We tuned the hyperparameters fiv1. 7+, and "xx using validation datasets. For further detailed experimental settings, refer Appendix [
We examined the behavior of MLA, ALA, and Ivs ladjuusst when applied post-hoc to models trained on the training datasets. Specifically, we aimed to assess whether MLA provides effective ap- proximation of Ivs in terms of decision boundary adjustment and classification accuracy. ALA, another post-hoe LA method, was included benchmark for comparison.
5.2 DECISION BOUNDARY ANGLES
We investigate how the decision boundary angles between classes are adjusted. Similar to the anal- ysis Sections B and 4. MLA and ALA can be interpreted as methods for adjusting the angles of the decision boundaries between classes. Let 0% and o represent the angles of the decision boundaries between classes A: and k derived from MLA and ALA, respectively.
For MLA, the following holds from Eq. (I2):
[N nx7x 7x
+ sin sin - 16) cos
0% arctan
(21)
For ALA, we assume that fx)) || is equal across all instances and sufficiently large, denoted as [] Then, from Eqs. (55) and (5G), the following holds:
7+ log
aresin
(22)
2
2 sin
8

---

Published as a conference paper at ICLR 2025
CIFARIOO-LT (ResNet34)
CIFARIO-LT (ResNet34)
AIA
MIA
ALA
ML.A
64
2.
20
Clers Indok
60
0
50
800
40 Class Irdes
4g
Class Index R
Figure 2: Heatmaps showing difference in angles of decision boundaries between each method and Ivs Iajjuset On the left is the result for CIFAR100-LT, and the right is the result for CIFARIO- LT. The left side of each figure displays 02 E while the right side displays 0 OK Values that became NaN are shown in gray. In the case of CIFARI00-LT, the angle differences between MIA and vsl adjuster generally small.
CIFAR10O-LT (ResNet34)
CIFAR100-LT (ResNeXt50)
CIFARIO-LT (ResNet34)
ImageNet-LT (ResNeXt50)
53
I
I
Accuracy (%)
54.5
I
Derrgg Accurasy (98)
52.8
Average Accuracy
80.0
52
53.5
52.6
53.0
52.5
51
T
52.5
79.0
52.4
52.0
78.5
MLA
ALA
MLA
MLA
Figure 3: Average accuracy of each model trained on each dataset and adjusted by different methods. The error bars represent the mean and standard deviation across five trials with different seed values. 1vl is short for ladjuuuster. MILA and Iss adjuster consistently achieve comparable accuracy.
We investigate the differences between these angles and Er used in Ivsladjuster and demonstrate that MLA serves sufficient approximation of Ivsl adjuster. Figure D displays heatmaps com- paring 0F - of and 0% 0F for CIFAR 100-LT and CIFARIO-LT. Detailed experimental settings and results for other datasets are provided in Appendices B and S
respectively. The heatmaps representing the difference between MLA and vssl Idjuuser are generally faint datasets with a sufficient number of classes, such as OFFRROO-LT.. This means that 0% 0%. glob- ally small across all k. k' E Y. This is consistent with our theoretical predictions from Section D indicating that MLL is a sufficient approximation of Ivsladjuster in large-class settings. The same phenomenon is observed for ImageNeet-LT and Helena. Note that in these latter two datasets, this phenomenon occurs even though training accuracy is not sufficiently high and NC is not fully realized (see Table M). This suggests that our theory holds under more relaxed conditions.
CIFARIO- IO-LT, which has fewer classes, decision boundaries of both MLA and ALA differ significantly from those of Ivsladjuste This may be because the number of classes is small that approximations such as ~ z/ are not valid. However, even in this case, MLA can still be regarded approximate method in that it achieves comparable test accuracy to I Ivsladjuster. See Section
5.3 TEST ACCURACY
We compare the test accuracy when each method is applied post-hoc to trained models. Figure dis- plays the average test accuracy with error bars for each model trained on each dataset and adjusted by different methods. The results on Helena and the detailed accuracy table can be found in Ap- pendix EEL In all experiments, MLA can achieve high average accuracy comparable to Iys Iysladjuster. Remarkably, this approximation holds in cases where the assumptions mentioned in Section ! do not hold. such as when the number of classes is small (CIFAR1O-LY) 1O-LTO or when training accuracy low and NC has not sufficiently occurred (ImageNet-LTT and Helena). In contrast, ALA achieves

---

Published as a conference paper at ICLR 2025
Table 1: Optimal hyperparameters and training accuracy for each model and dataset. For models that achieve 100% training accuracy, 7& and 1iv are slightly higher than 0.5.
Dataset
Model
72
1iv1
Training Accuracy (%)
CIFAR100-LT
ResNer34
0.710 + 0.037
0.770 + 0.060
100.0
CIFARTOO-L3
ResNeXt50
0.770 + 0.087
1.010 0.058
100.0
CIFAR IO-LT
ResNet34
0.780 + 0.518
0.410 + 0.120
100.0
ImageNet-LT
ResNeXi50
0.200 + 0.000
0.250 + 0.000
86.6
Helena
MLP
0.480 + 0.093
0.580 + 0.060
42.1
lower accuracy than Ivs lasiuuuuster in some cases. ALA is based on Fisher consistency (Menon et all 2020), assuming that the number of training samples is sufficiently large. While theory also assumes a certain level of training sample size, it has been shown to work well even with more prac- tical sample sizes. In addition, MLA and 1vs Ivsladjuster achieve comparable accuracy ALA under unfavorable conditions, such as experiments on CIFAR 10-LT and ImageNet-LTT, where the assumptions of our theory do not strictly hold. These results demonstrate that MLA is robust and practical method for improving test accuracy, even in real-world scenarios where ideal theoretical conditions are not met.
5.4 HYPERPARAMETERS
Table M summarizes the optimal values of MLA and Ivs ladjuuser hyperparameters, >& and Vivi. and the training accuracy for models trained on each training dataset.
On CIFARIO-LT, which has small number of classes, the values of > and Viv1 are significantly different. This is probably because the approximation of Ivsladjuster to MLA does not hold well when the number of classes is small.
From here on, we focus on cases in which the number of classes sufficiently large. On CIFARI00- LT, where training accuracy has reached 100%, the optimal value is slightly higher than the theoret- ically derived value of 0.5. This suggests that when training reaches terminal phase and NC occurs, the feature spread can be bounded to a smaller order than Ó (1/ /Ta).. For instance, recent research has aimed bound generalization error to smaller order than (L/ /TE) (Wer & Ma 2019), and this may be related.
On Helena and ImageNet-LT, where training accuracy has not reached 100%, the optimal values can smaller than 0.5. This is likely because the feature spread between classes does not differ much compared to when NC sufficiently occurs. Even in such cases, MIA and 1vs lajjustrr can achieve high test accuracy by adjusting hyperparameters (see Section 5 and Appendix EK). These results are expected to be key insights when searching for the optimal hyperparameters for MLA.
CONCLUSION
We provide a solid theoretical foundation for MLA, a simple heuristic method LTR, and demon- strate its utility. First, on the basis of NC, we estimate the feature spread for each class and present theory that derives the optimal method for adjusting the decision boundaries. Furthermore, we indicate that MLA is an approximation of this method and theoretically guarantees improvements in test accuracy. This provides clear insights into the conditions under which MLA is effective and how logits should be adjusted quantitatively. Additionally, through experiments, we demonstrate that the approximation holds under more relaxed conditions and that MILA exhibits practical usefulness by achieving accuracy comparable to better than ALA. This research forms crucial foundation for future ad advancements in LTR and imbalanced learning methods.
One limitation of this study is the lack of more in-depth theoretical analysis of ALA. While the current theory does not align with ALA, there may be approaches or assumptions that could guaran- tee its performance with practical number of samples. Future research could explore this direction. Another potential avenue is the combination of MLA with other methods to develop more sophisti- cated methods and achieve state-of-the-ar improvements in accuracy.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENIS
We extend our heartfelt gratitude to everyone who supported us in conducting this research.
We would like to thank the students of our lab and our fellow graduate friends for their insightful discussions, which greatly contributed to shaping the direction and resolving the challenges of this research. Their enthusiastic debates were crucial to the progress of this work.
We also express our deep gratitude to the reviewers this paper. Their detailed and construc- tive comments significantly improved the structure and logical development of the paper, greatly enhancing the quality of this research.
This work was supported by JSPS KAKENHI Grant Number JP24H00709.
REFERENCES
Konstantinos
Panagiotis Alexandridis, Shan Luo, Anh Nguyen, Jiankang Deng, and Stefanos Zafeiriou. Inverse Image Frequency for Long-tailed Image Recognition, October 2023.
Jonathon Byrd and Zachary C. Lipton. What is the effect of importance weighting in deep learn- ing? In 36th International Conference on Machine Learning, ICML 2019, volume 2019-June, pp. 1419, December 2019. ISBN 978-1-5108
doi: 10448500/axiv.8812.03333
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning Imbalanced Datasets with Diseel-isttibtionn-wweeea Margin Loss. In Advances in Neural Information Pro- cessing Systems, volume 32. Curran Associates, Inc., 2019.
Peng Chu, Xiao Bian, Shaopeng Liu, and Haibin Ling. Feature Space Augmentation for Long- Tailed Data. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.). Computer Vision ECCV 2020, Lecture Notes in Computer Science, pp. 694-710, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58526-6 doi: 10.1007/978-3-030-55555
Yin Cui, Menglin Jia, Tsung Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June:9260-9266 January 2019. ISSN 10636919. doi: 10.1109/CVPR 20099.0999,,
Hien Dang, Tho Tran Huu, Tan Minh Nguyen, and Nhat Ho. Neural Collapse for Cross-entropy Class-Imbalancee Learning with Unconstrained ReLU Features Model. In Forty-First Interna- tional Conference on Machine Learning, June 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. ImageNet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255. IEEE, June 2009. ISBN 778-1-4244-3992-8 doi: 10.1109/CVPR.2000 55006448.
Chris Drummond and Robert C Holte. C4.5, Class Imbalance, and Cost Sensitivity: Why Under- Sampling beats Over-Sampling. In Workshop Learning from Imbalanced Datasets II, vol- ume 1, 2003.
Cong Fang, Hangfeng He, Qi Long, and Weijie J. Su. Exploring deep neural networks via layer- peeled model: Minority collapse imbalanced training. Proceedings of the National Academy of Sciences of the United States of America, 118(43), January 2021. ISSN 10916490, doi: 10.1073/ pnas.2103091
Jonathan Frankle and Michael Carbin. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In International Conference on Learning Representations, September 2018.
Tomer Galanti, András György, and Marcus Hutter. On the Role of Neural Collapse in Transfer Learning. In International Conference on Learning Representations, October 2021.
Noah Golowich, Alexander Rakhlin, and Ohad Shamir. Size-Independent Sample Complexity of Neural Networks. In Proceedings of the 31st Conference On Learning Theory. pp. 297-299. PMLR, July 2018
11

---

Published as a conference paper at ICLR 2025
Isabelle Guyon, Lisheng Sun- Sun-Hosoya, Mare Boullé, Hugo Jair Escalante, Sergio Escalera, Zhengy- ing Liu, Damir Jajetic. Bisakha Ray, Mehreen Saeed, Michèle Sebag, Alexander Statnikov, Wei-Wei Tu, and Evelyne Viegas. Analysis of the AutoML Challenge Series 2015-2018. In Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren (eds.), Automated Machine Learn- ing: Methods, Systems, Challenges, The Springer Series Challenges in Machine Learning. pp. 177-219. Springer International Publishing, Cham, 2019, ISBN 978-3-030-05318-5 doi: 111000/997-33003-5555
R. Hadsell, S. Chopra, and Y. LeCun. Dimensionality Reduction by Learning Invariant Map- ping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR volume 2. pp. 1735-1742, June 2006. doi: 10.11090VVP.220011000
X. Y. Han, Vardan Papyan, and David L. Donoho. Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path. In International Conference on Learning Representations, January 2022.
Stephen José Hanson and Lorien Pratt. Comparing Biases for Minimal Network Construction with BackkPropagations Advances in neural information processing systems 1, 1:177-185, 1989.
Naoya Hasegawa and Issei Sato. Exploring Weight Balancing on Long-Tailed Tailed Recognition Problem. In The Twelfth International Conference on Learning Representations, October 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2016 6- 2016-Decem, pp. 770-778, December 2016. ISBN 978-1-4673-8850-4, doi: 10.1 20.1109/CCPR.2211..900
Yerlan Idelbayev and Miguel Á. Perrein. Low-Rank Compression of Neural Nets: Learn- ing the Rank of Each Layer. In 2020 IFFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8046-8056, June 2020. doi: 10. 10.10099C2242200020000000
Wenlong Ji, Yiping Lu, Yiliang Zhang, Zhun Deng, and Weijie J. Su. An Unconstrained Layer- Peeled Perspective on Neural Collapse. International Conference on Learning Representations, January 2022.
Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka. Well-tuned Simple Nets Excel on Tabular Datasets. In Advances in Neural Information Processing Systems, November 2021.
Tokio Kajitsuka and Issei Sato. Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators? In The Twelfth International Conference on Learning Representations, October 2023.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling Representation and Classifier for -ailed Recognition. In Interna- tional Conference on Learning Representations, 2020.
Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng. Exploring Balanced Feature Spaces for Representation Learning. In International Conference on Learning Representations, March 2023.
Byungju Kim and Junmo Kim. Adjusting decision boundary for class imbalanced learning, IEEE Access, 8:81674-81685, December 2020. ISSN 21693536. doi: 10.1109/ACCESSS.2000 2991231.
Ganesh Ramachandra Kini, Orestis Paraskevas, Samet Oymak, and Christos Thrampoulidis. Label- Imbalanced and ral Information Processing Systems, volume 23, pp. 71384-539-3, doi: (0.445550aamiiv2100.11555
Classification under Overparameterrizatin.. In Advances in Neu-
March 2021. ISBN 978-1-
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Science Department, University of Toronto, Tech., pp. 1-60, 2009. ISSN 1098-6596. doi: 10.1.1.222.9220.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basee learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, January 1998. ISSN 1558-2256. doi: 10.1109/5.726791.
12

---

Published as a conference paper at ICLR 2025
Michel Ledoux and Michel Talagrand. Probability in Banach Spaces. Springer, Berlin, Heidelberg, 1991. ISBN 978-3-642-20211- 978-3-642-20212--1 doi: 10.1007/978-3 3-4---2222222-244
Kaican Li, Kai Chen, Haoyu Wang. Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Yan Yeung. Xiaodan Liang, Zhenguo Li, and Hang Xu. CODA: A Real- World Road Corner Case Dataset for Object Detection in Autonomous Driving. Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022, volume 1 13698, pp. 406-423. Springer Nature Switzerland, Cham, 2022a. ISBN 978-3-031-19838. 978-3-031-1983944 doi: 10. 101100/978---001-19999999 19839-7.24.
Mengke Li, Yiu-ming Cheung, and Yang Lu. Long-Tailed Visual Recognition via Gaussian Clouded Logit Adjustment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6929-6938, 2022b.
Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio S. Feris, Piotr Indyk, and Dina Katabi. Targeted Supervised Contrastive Learning for Long-Tailed Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6918- 6928, 2022c.
Wen Li. Limin Wang, Wei Li, Eirikur Agustsson, and Lue Van Gool. WebVision Database: Visual Learning and Understanding from Web Data, August 2017.
Jialun Liu, Yifan Sun, Chuchu Han, Zhaopeng Dou, and Wenhui Li. Deep representation learning on long-tailed data: A learnable embedding augmentation perspective. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Puttern Recognition, pp. 2967- 2976, February 2020. doi: 10. 10.11099CVP442200222000 202000000000000000000000
Xuantong Liu, Jianfeng Zhang, Tianyang Hu, He Cao, Yuan Yao, and Lujia Pan. Inducing Neural Collapse in Deep Long-tailed Learning. International Conference on Artificial Intelligence and Statistics, pp. 11 11534-11 1544. PMLR, 2023.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X. Yu. Large- Scale Long-Tailed Recognition in an Open World. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), April 2019. doi: 1904 05160.
Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg, Alan Blair, Chunhua Shen, and Anton van den Hengel. Retrieval Augmented Classification for Long- Tail Visual Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6959-6969 2022.
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017 Conference Track Proceed- ings, August 2017. doi: 10448500/axxi.116888.o
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In International Con- ference on Learning Representations, September 2018.
Jianfeng Lu and Stefan Steinerberger. Neural Collapse with Cross-Entropy Loss, January 2021.
Teli Ma, Shijie Geng, Mengmeng Wang, Jing Shao, Jiasen Lu, Hongsheng Li, Peng Gao, and Yu Qiao. A Simple Long- -ailed Recognition Baseline via Vision-Language Model. https:/arriv.og//a//2111...
November 2021.
Yanbiao Ma, Licheng Jiao, Fang Liu, Yuxin Li, Shuyuan Yang, and Xu Liu. Delving into Seman- tic Scale Imbalance. In The Eleventh International Conference on Learning Representations, September 2022.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. International Conference on Learning Representations, July 2020. doi: 20350//arxvv.000.7733....
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. The MIT Press, 2012. ISBN 0-262-01825-X.
13

---

Published as a conference paper at ICLR 2025
Vardan Papyan, X. Y. Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences of the United States of America, (11440):2665224466) August 2020. ISSN 10916490. doi: 10. 10.1003/pnas. 2015509117.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Pro- ceedings of the 38th International Conference on Machine Learning, pp. 8748-8763. PMLR, 2021.
Akshay Rangamani and Andrzej Banburski-Fatee Faheey.. Neural Collapse in Deep Homogeneous Classi- fiers and The Role of Weight Decay. In ICASSP. IEEE International Conference Acoustics, Speech and Signal Processing Proceedings, volume 2022-May, pp. 4243-4247, 2022. ISBN
978-1-66540-540-9. doi: 10. 10.11990CCAASP449222200977777777
William Reed. The Pareto, Zipf and other power laws. Economics Letters, 74(1):15-19, December 2001. ISSN 01651765, doi: 10. 00.0016/50166--77501000000553330
Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5):513-523, January 1988. ISSN 0306-4573, doi:
Merrielle Spain and Pietro Perona. Measuring and Predicting Importance of Objects in Our Visual www.aacccccccccccccccccccoooo...
World.
November 2007.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learn- ing Research, (5(56))1929-1999) 2014. ISSN 1533-7928.
Thomas Strohmer and Robert W Heath. Grassmannian frames with applications to coding and communication. Applied and Computational Harmonic Analysis, 14(3):257-275, 2003. ISSN 10635203. doi: (0001//$1063-503((0)3)))))
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. Long-tailed classification by keeping the good and removing the bad momentum causal effect. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, pp. 1513-1524, Red Hook, NY, USA, December 2020. Curran Associates Inc. ISBN 978- 778-1-71382-954--
Christos Thrampoulidis, Ganesh R. Kini, Vala Vakilian, and Tina Behnia. Imbalance Trouble: Re- Geometry. Advances in Neural Information P Processing Systems, August visiting 088550/axxiv.220..555.5
2022. doi:
Changyao Tian, Wenhai Wang, Xizhou Zhu, Jifeng Dai, and Yu Qiao. VL-LTR: Learning Class-wise Visual-Linguistie Representation for Long-Tailed Visual Recognition. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022, Lecture Notes in Computer Science, pp. 73-91, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-19806 9. doi: 10.10/79978-3 3-031- 1981--990669999 9.5.
Binwu Wang, Pengkun Wang. Wei Xu, Xu Wang, Yudong Zhang, Kun Wang, and Yang Wang. Kill Two Birds with One Stone: Rethinking Data Augmentation for Deep Long-tailed Learning. In The Twelfth International Conference on Learning Representations, October 2023a.
Haotao Wang, Aston Zhang, Yi Zhu, Shuai Zheng, Mu Li, Alex J. Smola, and Zhangyang Wang. Detection in Tailed
Partial and Asymmetric Contrastive Learning for Recognition. In Proceedings of the 39th International Conference on Machine Learning, pp- 33446-23458 PMLR, June 2022.
Jianfegg Wang, Thomas Lukasiewicz, Xiaolin Hu, Jianfei Cai, and Zhenghua Xu. RSG: A Simple but Effective Module for Learning Imbalanced Datasets. In 2021 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 3783-3792, June 2021. doi: 10. 10.1109/CVR44447.220 20222 .00378.
14

---

Published as a conference paper at ICLR 2025
Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. A Unified Generalization Analysis of Re- geightiig and seeit-Adjsttmnnt for Imbalanced Learning. In Thirty-Sevntth Conference Neural Information Processing Systems, November 2023b.
Colin Wei and Tengyu Ma. Improved Sample Complexities for Deep Neural Networks and Robust Classification via an All-Layer Margin. In International Conference on Learning Representations, September 2019.
Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated Residual Transformations for Deep Neural Networks. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 1492-1500, 2017.
Lu Yang, He Jiang, Qing Song, and Jun Guo. A Survey on Long- Tailed Visual Recognition. In- ternational Journal of Computer Vision, (0(7))1837-1872) July 2022a. ISSN 1573-1405. doi:
10.1007/61266-022--112228
Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie. Zhouchen Lin, and Dacheng Tao. Inducing Neural Collapse in Imbalanced Learning: Do We Really Need Learnable Classifier the End of Deep Neural Network? 2022b.
In Advances in Neural Information Processing Systems, October
Han-Jia Ye, Hong-You You Chen, De-Chuan Zhan, and Wei-Lun Chao. Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning. July 2022.
Runtian Zhai, Chen Dan, Zico Kolter, and Pradeep Kumar Ravikumar. Understanding Why Gen- cralized Reweighting Does Not Improve Over ERM. In The Eleventh International Conference on Learning Representations, September 2022.
Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep Long-Taike Learn- ing: A Survey. October 2021. doi: 10.48550/arxiv..1 10.04596.
15

---

Published as a conference paper at ICLR 2025
A ACRONYM AND NOTATION TABLE
Table 2 lists the acronyms referenced throughout this paper, and Tables B, AL and & provide sum- mary of the notations used in this paper.
Table 2: Acronym table
Abbreviation
Definition
Iv1
Ivsl adjuster
ALA
Additive logit adjustment (Menon all 2020
ETF
Equiangular tight frame
iid
Independent and identically distributed
LA
Logit adjustment
LTR
Long-tailed recognition
MLA
Multiplicative logit adjustment (Kim & Kim 2020)
MLP
Multi-layer perceptron
NC
Neural collapse (Papyan et all 2020)
ReLU
Rectified linear unit
Table 3: Notation table for operators and set
7
The angle between the two vectors The Euclidean
for vectors
12, |1 1|e
The spectral norm and Frobenius norm for matrices
The Landau notation, ignoring logarithmic terms
The approximation function for tan, i.e., s(0) 1-0
Avg Sm, mf ES, h(xxx')
The average over pairs of different elements The from 1 i..., {1.....i}
[2]
Table 4: Notation table for data
x.
An instance space and a label space
v. y
An instance and a label
K
The number of classes, i.c., [)]
The instance dimension
P. Pk
A sample distribution and class-conditional distribution
S. Sk
A training dataset and a class class-specife specific training dataset
S
A class-specific test dataset The size of S and Sk
N, TM
iuk
The size of Sy
An imbalance factor, =
mAx
G.
Rademacher random variables
16

---

Published as a conference paper at ICLR 2025
Table 5: Notation table for models
f. IIfll F, F,F,FM
feature map and a ReLU neural feature map
The value of [=))) assuming that [=] equal for all æ E X The set of feature maps, ReLU neural feature maps, and ReLU neu- ral feature maps with constraints on M
h.
The output of the second toooddtt--sstt layer of the feature map, and that of the ReLU neural feature map logit and the logit for class k
9. 9k
W. wk
The weights of a linear classifier and its vector for class k The s-th linear layer from the end of the feature map The rank of W
W
st, uA, V1!
The I-th singular value, left singular vector, and right singular vector of W1 from its singular value decomposition
H
The set of h
Hi. Hi
The set of (vr, h) for f E F and set of (v), h) for f f FF The Rademacher complexity and the empirical Rademacher com- plexity for Hi (see Section II
Ra, (Hi), Rs (H11)
x(),
The complexity of Mi and The mean of C(H,, X) for I E [r]
C(F,X)
9
The depth of the feature map The dimension of fax) and the dimension h(x)
di
B
The upper bound of h. i.e., SUP=ex,eet |(xx)] - B
M
The upper bound of 1 [|W||
HP(P), (PP)
The expectation of f for
(y(S).
The mean of f(x) for x E S
((P)
Same as M7(P) and (7((S)
11(0;)) I1(0;k) k)
The angular bound probability for f and for f (see Section 222). Hyperparameters for Ivsladjuster, MLA, and ALA
711 Xx. 7+ , Ge 0 (71v1)
The angle of the decision boundary, the optimal angle, and the opti- mal angle adjusted by Tiv1
K otx
The angles of the decision boundaries derived from MIA and ALA
O 0" O
The set of OR o and (f
The angle between the weight vectors of an ETF classifier, i.e., vb arccos
K-1
Tk.k"
The ratio [[ M
mykkk
The normal vector of the decision boundary between ie and lv'
B RELATED WORK
B. MULTIPLICATIVE LOGIT ADJUSTMENT
MLA has been validated for its effectiveness across various studies. Hasegawa & Sato (2025) demonstrated that MLA performs on par with or better than ALA under realistic conditions. Addi- tionally, Ye et all (2022) highlighted significant differences in the feature space occupied by each class in LTR, confirming the effectiveness of class-dependent temperatures, which equivalent to MLA. Beyond image classification, various studies suggest the utility of MLA-like adjustments in other fields. For instance, in object detection, Alexandridıs et all (2025) proposed an MLA variant based object occurrence frequency, called inverse image frequency. Similarly, in text retrieval tasks, the inverse document frequency (Salton & Buckley, (988), which adjusts the score function by multiplying the inverse frequency of words in document, is widely used. Thus, MLA is not only applicable to LTR in classification problems but also serves as fundamental technique with potential applications across broad range of domains.
17

---

Published as a conference paper at ICLR 2025
C
PROOF
Here, we present the detailed proofs and supplementary explanations for the theories proposed in this paper. Appendix eal contains the proofs for the propositions discussed in Section B12 Appendix
provides specific examples of applying our theory to ReLU neural networks. Appendix C includes the proof of the lemma presented Section 4
C.I.
MAIN ANALYSIS
For all k where n& > 1, the operation of averaging a pair of elements æ, E Sş. is defined as follows:
1 Avg ES (x,x²)= nx(ng - 1)
Σ
( Tz2).
(23)
2y ESt #1 ES, (
First, we propose two lemmas required for the proof of the propositions in Section 17 Then, we prove the propositions using the lemmas.
Lemma 3. Suppose n& > 2. For any f E F,8 > O, the following holds with a probability of at least 1 - 6:
p/ (1) f(xx')) 1| Σ A W....'sS. (h((r) h(=2)))
[4C(F,2)+
+B B 2 log r
(24)
Tik
Proof. We derive this lemma following the proof of Theorem 3.3 in Mohr et (2012).
By the
triangle inequality, the following holds:
Σ laiiss (h(x) - h(x'))
lf(x) -
st |/|1/||/1 (h(x)) n(x1))l
W 11, Σ |o (h(x) h(x')))
(25)
We use this inequality to bound Ex. [[]((x) - f (x')l]. Then, for each E [r]. we calculate the
probabilistic upper bound of Ex u (h(x) h(x)))l Avg. es V (h(x) h(x'))
and apply the union bound to complete the proof.
Note that Ex [|, (h(z) - h(x'))] ES [Avg /0/ o0 (h(x) - h(x'))]]- For
St. St ph define (St) as follows:
(i(S))=
sup (n [al] (h(x)) hxx'))(] A Avga Avga 10 (h(x) h(a'))l). (26) (w..)EH
18

---

Published as a conference paper at ICLR 2025
Let Sk. Sk p be datasets that differ only in the j-th sample æJ. and define (5(S)) similarly
(.)). The difference between (r(()) and (1(5)) can be upper-bounded as follows:
((()) - (r(())) =
sup ( (Avg ESt 07 (h(x) - h(a1))
(m.,))HHi
AvKo,x'SS 17 (h(x)) h(a2))))
(h(a))) -((aa)) |0] (h(aa) - h(a:))))
(nk
sup
1
(l1) (h(x)) h(x3))))
sup i (m.))HH
11h (1k
2
sup 1ik wEX.(v,.NEEM)
||(x)]
(27)
2B
(28)
Therefore, using McDiarmid's inequality, we can prove that for any 5 > 0 the following holds with probability at least 1 - ō:
2 log
(Sx) S Es, (Sx) + BV
Note that for any f E F, the following inequality holds:
E Px [l (hx(r) h(a')) I] N'est m7 (h(x)) h(xx))) s ((x)))
Using Eq. (25), we can derive the following:
Left hand side of Eq. 24
||
11 Σ (Sx)
(30)
Thus, aim to take union bound of Eq. (1 over all [ to derive upper bound.
Next, we bound Es, (Sx)):
Es, (Sx)] = Es, (w.,)))
sup
(Eu,ar~ [1o) (h(x)) h(x'))[
Avgn, 10 (h(x)) h(x'))))
S Es
sup (mi,))HH
(Avgz.r'ss 1/ (h(æ)) h(x'))
Avga ES 1 ToT((n (h(2) h(az))1)
Es
sup
Σ
10 (h(2), ((
(vih)EH, ne (n² 1)
ESE
10 (h((zz) h(
EyESt myeSSiaaa)
19

---

Published as a conference paper at ICLR 2025
E S
sup (o),))EH,
1
Σ
2 ( n(tr))1
Thk (1i - 1)
2, ES
Σ
ay1oi (h(zz)) h(
ayESk (
$ 2E Sx
sup
Σ
0yluy (h(x,) - n(2,))))
(m) hiEH nh (n - 1)
m, ES
2 ES.. Māa}
21 o.
sup (nH,,) ny
Σ o,l0ī h(20))))
1)
æ E& my ES
<
Esp
Σ E
sup (ui
Σ ESt.
ozloe (h(ax) - m(
2 ESc
E.S
Σ E
sup (e), ES
Σ (1((2;)))) - 12(2k,)))
(31)
20 ESc
2n1k
Ea
sup (w),))EH æt ESt
Σ )
1
2
Es²
sup (b.,))EH,
1o (((())))1
Eo
Σ -0,
11k (ni - 1)
E
Nr:Rn, (Hi) BEd
Σ
(32)
Tik 1
The transformation in Eq. (31I is similar to that of Talagrand's contraction lemma (Ledoux & Tala- grand, 1991)..
In addition, the following inequality holds:
mg
T
E
Tj
E
Σ
Jj
Eg
J
==1
Eg
Σ
03
+ 2 Σ
Ojos'
j=l
Vnk-
(33)
20

---

Published as a conference paper at ICLR 2025
From Eqs. 29
322
and (3:3), it is proven that following holds with probability at least 5,
2nk
B
(Sx) S
Ri (Hi) +
+
2 log
11
VTLR
V
5
2m
B
(C(H),, + B)
2 log
(ng-
1lynk
VTu
1
1
4C(Hi, 2²) + 4B + B
2 log x
(34)
Vnk
Substituting Eq. (344) into Eq. (GO) and taking the union bound for all / proves this lemma.
Lemma 4. Suppose ny 2. For assume that for all æ Sk. f(x) (((s)) holds. For any 0 < 6, < 1, define 0 as follows:
r| 4C(FF 4B B 2 8)
0
(35)
28
TOR (
Assume that 5, 8' satisfy < I For x ~ Pk, the probability that the angle between f(x) and p(Sa) is less than 0 is at least 1 (8 + 8').
Proof. We bound E S [1]((x) ((Sx)]l] and apply Markov's inequality. this,
Expanding
E PeNPe (1l [=()))]]]
1
B P
nxf(x) Σ f(a')
nik
1
<
a -
Pr. P
llf(x) f(x')1)
T
L L.
E Py ((r) f(x')l].
(36)
Using Markov's inequality, for any 8' > 0, with probability at least 1 - 8'.
fl((x) - p(S))) S
E Pa Su Su [1lf(()) (s
Em (x) -f f(
(37)
Before applying Lemma this case, we demonstrate that the second term on the left-hand side of
Eq. (22)) is zero in this situation. When f(æ) = =S((S;) satisfies for all æ E Se. f(æ) = f(x') holds for all x, x' E St. Then, we have Σi= sannis (h(x) h(x")) O. Since u, are the left singular vectors of W1, they are linearly independent for different Therefore, for all I E [r] and for all x, x' E Sk.
ia (h(x:) h(xx)) = 0.
(38)
Then, we combine the equation Eqs. (57) and (XX) with the result Lemma B by union bound. For
any ó, 8 > 0, with probability at least (5 + 8'). the following holds:
Ilf(x) ((a))l
1 W ll2
(C(FF 4B B 2 log r
8' Th
2 | (0
(39)
π
Since 1-111-16100 (6++8) is lower bound for the probability, even restricting even the usefulness of the theorem.
8 < does not compromise
21

---

Published as a conference paper at ICLR 2025
The angle between f(z) and ((Sp) can be expressed as aresin [f({) pe 1l
when [|( I 1. Since arcsin(t) is strictly monotonically increasing for 0 t 1, when we have 0 I g.
Sp
(f(()) (((m)-((ae u(Sill
2. the upper bound the angle between f()) and p(Sx) can be written as -(m))
< aresin (20) v This means that the upper bound the angle be- tween the features and the average training features of the class less than 0 with probability at least 1 (6 8*).
aresin
[
Proposition Suppose ny > For f F, assume that for all a E Sv. f(x) = p((ak) holds.
Consider any 0 that satisfies the following condition:
rl (I(E))
4B + B 2 2og (0(
Vnnllp ((())
<0<
(5)
[
where || ll2 denotes the spectral norm. For such 0, the following holds:
W
20 nn [((s))]
[T(0; k) = 28 74
[4C(F, 4B 4 B 2 log
(6)
W
1|2
Proof. Consider following 8' > 0:
r W l/2
8
(40)
28 M (S
r| W
20 /np (Sx)l
8'
4C(F, 4 By log
(41)
20 /nx (S+)
x x|
Given B 2 and the condition on 0,
W Thk (SxI)
4C(F, + +B+ B log ymk |((S)]l
8 8'
(42)
|W1ll2
Defining as above, we get = Lemma M the probability that the angle between f f(x) and (Sx)) is less than is at least
(T (AC(F, X) + 4B + B 2 log t) Therefore, by
rl
28
1 1 (6 - 8) =1 π 12
20 /0glll [
AC(F,x)+4BBB B 2 2log
(43)
20 TE (S
W
Proposition 2. Suppose nk. ny > 2. For f E F, assume that for both E (k, K'), f(x) = ((S))
holds for all x E St. Consider the following maximization problem:
max 11(0k.r;k) + (((0w,ii) 0 D
s.t. O Ow > O, OR. + Ow.k = sb
(7)
The unique solution Ni 02 within the range where 0g.k and Ow satisfy Eq. (E) is given by:
Of (1x (1w)))) /nw
(1x) ynk
0 = (I TTu + 11( )l V
(8)
+ (1p((S)))) (I I rg
Proof. Noting that O = - - 02. we have:
(TT(O;k)
(TT():))
80 (1(00 + (1(0m..A
(44)
20
00
1==8..
22

---

Published as a conference paper at ICLR 2025
Since 0i & 0 and 0g + f% v, with (l (1))
1 (P.all(((s))ll the
following holds:
k))
aTT(O;k))
(45)
00
00
10
Thus, we have 8
38,, prt k) + [1(0 k)) O when O2,5 Og and Ow O
Next, we prove that this is the only solution that gives the maximum value within the aforementioned range. We have:
0²
O'II(OR v))
API((Ey'))
(II(ON,)))) + (I(0gr1/))))
(46)
002 K.k
00
002
02.6"
Here, we define Ork =
r 1l2
4C(F, x) +B + 11, and (02,0.))
log
10
Note
that (cx(O 20,0
2
Then, the following holds:
02,
8LTT(OK.KiG
82
Or
(B+ BB BY/G(0mw)))
06% k,k'
002 K.K'
BE.K'
a&k
( (On((2,x))
Bak Oh (k(Ok,
20n,
0
Ohk
(3 (C+((000))) BCA(OkaN))
00k.k'
8 k
20%
B + BC& (Ou,w.) BCN(OO,N))
B
B
+
On. Ca (
Ch (k
02
On.
ak
2,3 + 2BC (Ok,.) (6 ) BCa(((xx)))
(47)
k,
From the condition, there exists 8 1 such that 02.K' 2g implying 2 log 2 > 2, we have
AnM as
1. Since (Ox,)))) >
(PTT(Ok,Nr;k))
ak (28 (B (Bx,)(
S
062 k,k'
2ak
(8 (8+ (C -+(((((x.x)) -2))
O.
(48)
Similarly, we can prove that 8'II(0:k") 802
0.
0=0-02-
Therefore, Eq. ( is the only solution that gives the maximum value within the range where OR and 0g satisfy Eq. (E))
Proposition 3. Suppose ny > for all k y. For f E F. assume that for all l V, f(x) (SA)) holds for all x E Sy. Consider the following maximization problem:
max
11(0x.r;)) [I(0 S.I. Vk, E V. V.Ox, Ov.k x and 0x + 0g = vb.
(9)
The unique solution x E 0* within the range where all 07 O satisfies Eq. ( is given by: (1((5)) )ll T
0 = [|
(10)
1h ( V
23

---

Published as a conference paper at ICLR 2025
Proof. The following holds, and the result follows trivially from Proposition 0
max Σ (((0R + 11(OR kikk) S E
ΣΣ max 11(0 + 11(0 wik)) (49)
kEY ROPK ONt
C.2 CASE OF RELU NEURAL NETWORKS
In this section, we specifically demonstrate the values of C(H,, X) and B when the feature map restricted to ReLU neural feature maps. This allows us validate the assumption R,, (H,) I C(Hi.))) posed in this paper and to illustrate the practicality of the proposed theory. Vih
Let F represent the set of ReLU neural feature map, and define ReLU neural feature map as f(x) = W max(0, W max(0, W F. Here, denotes the depth of the network, and the max operation is applied element-wise. We define the J=M as the set of ReLU neural feature map where the product of the Frobenius norms from the first layer to the second-to-lat layer bounded by M. That is,
FM
F W max(0, w ... Wzx)
F
I[ I1 s M 3 (50)
We focus on functions that belong F When discussing ReLU neural networks, we add an overline to distinguish them from the general case. For instance, h refers function that returns the output of the second-to-last layer of a ReLU neural feature map. Similarly, (T(A)) represents the average of the features f(x) for the dataset Sy.
For such networks, we similarly define H their Rademacher complexity, and empirical Rademacher complexity as follows:
H/ ((1) h) FE
. f-
W is the 1-th right singular vector of W },
(51)
Rn, Hi
E B
sup (n), n a,
1
Σ aool h(w;)
(52)
As.
M
E.
sup (v..))))) M Th
Σ o,ol o, h(oz)) my
(53)
According to Golowich et all (2018), the empirical Rademacher complexity for this family of func- tions bounded by:
As,
11411
E,
sup (w..T))e7H
ΣO,U, h(x,,)
T&k
wFESs
1
V log + 1) M sup |
v Vng
#Ex
1 Vnk
+ 1)M sup ||=||- (1.5
(54)
xEX
Taking the expectation over St, we can replace C(F (1.5 + 1)M sup ||||| Since C(F, X') does not depend on nk. this confirms the assumption. We can also replace = M supaex || because supnex,neR [|((x)] M sup ||]]|-
In this case, Lemmas and ! can rewritten as follows. Since these are simple substitutions, we omit the proofs.
24

---

Published as a conference paper at ICLR 2025
Lemma 5. Suppose ni For any I E J and any > the following holds with a probability at least - 6:
P' [2) f(x')l) -||W 112
A Avga, of ES |o (h(x)) hi(x'))l
1|2 sup
|ا|||
(6 88 8 v (2 ;)
(55)
VIk
Lemma 6. Suppose ny > For F.
aassume that for all a E Sk, f(z) TT(Sk) holds. For
any 0 < 6, < 1, define 0 as follows:
W
M12 suP ||=||
(oV7+8+
0 28'
2 33)
(56)
(
Assume that 8, 8' satisfy 0 < z. For Pk. the probability that the angle between f(x) and (T(Sk) less than is at least 1 (8 + 8').
We define the angular bound probability IT(;; k) for a ReLU neural network in Section 2.2 Then, Proposition M can be rewritten follows. The proof is omitted here well
Lemma 7. Suppose ny > 2. For J E FzM. assume that for all æ E Sk. F(x) = Ti(S)) holds. Consider any 9 satisfyyng the following conditions:
tr W 112M SuPmEx |||||
6
2
[T(SSAI]]
<
<
T
(57)
n (Sx)
|
2
For such 6, the following holds:
Tr W 12
20 20/mk m
(xx)
II(O:) =
1
M sup #EX
|>|||
6 + 8
V
2
20 T (x)ll
W
(58)
Propositions 0 and 3 also hold similarly, ensuring the validity of lvs Ivs adjjsster.
C.3 MLA APPROXIMATES ADJUSTER
In this section, we provide the proof of Lemma R
Lemma 2. Let E [0, 1). Then, the following holds:
(()) < tan
(
(0)
0 0 0 1 2
(18)
2
2 (0)
1
tan
(0)
2 < <1
(19)
π
2
Proof. When 0 0, the result is trivial since <(0) tan(0)
0. For the remainder, we assume
0 E (0, 1) unless otherwise stated. Define h(() tan( (00)
tan (an())) Since h(()) x x
d(6)
proving the following will conclude the proof:
S h()) < π
(D<05
(59)
2
First, we demonstrate that h(0) is monotonically non-increasing.
z/2 cos?
tan (1 0) tan taattann)))) (30) 8)
20)
h h((0)=
g2
0(1-9)
-tan (an))
2 cos2 50)
(60)
02
25

---

Published as a conference paper at ICLR 2025
Now, define 1(0) 811-0) tan (30) for [0,1); (g0)
(0))) = ạ (-20 + 1) cos² (50) +0(1 0) sin (0) cos (50)
2
co6 4 (50)
cos² (50)
π 20 cos (500) + T 8) sin 2 cos (0)
cos (
(61)
Since 1(0) = O, we have 1(0) < for 0 E [0, 1). Therefore, from Eq. (6L)), (0 < 0 for 0 E (0, 1), indicating that h0)) monotonically non-increasing.
Since h()) 1. to prove Eq. (59), we must indicate that limg h(()) rule, this can be verified.
2 By applying l'Hôpital's
tan (1
lim h(0) 8+40
lim 8-+0
0
π (1 0)
lim 8-+0
tan (0) +
2 cos 2 [00
π
(62)
2
Thus, Eq. (99) holds, completing the proof.
D ALGORITHM OF Ivsl ADJUSTER
Algorithm II outlines the detailed steps of Ivs lasjujuster, 1-vs-1 multi-class classifier that performs classification based on the decision boundaries proposed Proposition B. The derivation of normal vector ma Rd for the decision boundary between & and k is follows. Since ma.k lies in the same plane as wy and wk' and the angles with each are specified, it can be expressed using o, 3 R as follows. Note that assume | here.
my = anw + F.w
m W0 = cos
2
o 07 (71v1)
(63)
mg Wv cos
+ 0z (71v1)
2
Solving this system of equations for and B, we obtain the following:
sin 0% (71w) + sin 0% , (7iv1) cos
cos? r/ sin Of (71v1) sin 0% (71v1) cos +p
(64)
cos2 2
Thus, the decision boundary be defined a plane passing through the origin with ma.e as the normal vector. Note that Algorithm M1 simplifies this by using the normal vector multiplied by I cos2 v. Additionally, when K is sufficiently large, my, can be approximated as ma. sin 0f (71v))w; sin 07 & (Tivi)
E DISCUSSION: APPROXIMATION OF ALA
Similar to the approximation of MLA in Section we explore approximating ALA to Ivsladjuster. ALA modifies the logits by adding correction term, - log ngg where 7 0.
First, we assume that for any x E X, the feature norm is constant, i.c. f(x) = Ifl. Similar to the MLA case, denote the coefficient of the ALA by mk- To adjust the decision boundaries in
26

---

Published as a conference paper at ICLR 2025
Algorithm Ivs lajiuste
1:
function ONE_ VS SANE_S_ONERE (N) W, )
2: counter[1... [I] be zero-initialized array
3:
for all k < 2 do for all k < do
4:
5:
if k = k' then
6:
continue
7:
end if
8:
let a = sin o ((ivi) + sin 0 (7ivl) cos ip
9:
let B = sin ( + sin 0 (71v1) cos vb
10:
let m.r = 000 + BrD
11:
x. f(x) ) > 0 then
12:
counder [k] < counter(k] +
end if end for
14:
end for
16:
return arg max(counter)
17:
end function
ALA such that it satisfies Proposition B. t) must be set so that the following condition holds for all + k e.:
||||| cos(( The || cos(O (((vi)) TB8*
[[f]l(cos(O)).. (7iv1) cos(02 x (Tiw)))) 1/h 7/60--
(65)
For the left-hand side:
cos cos(0 (TivI)) - cos cos(O0x (0 (71v1))
8 (1vv1) + 0 (71x1)
0 1 (1iv)) - - (f1vv))
=issin
sin
2
vb
2 sin
sin
((ivl))
+bb sin
1/ (7iva)
2
sin
sin
sin
Ol (TivI)
(66)
2
1/7
2 sin
sin
2
The The (7iv) +
Let us denote the right-hand side ((Tv.)((1vv)). We set > > that can be approximated near Thi (71v1) = 1 by 244 log(Tk,N (7iv1) )). Specifically, we set 7 to satisfy:
s(1) ) = 0 = 22 log(1),
(67)
g((1) = v sin
272
(68)
2
The solution 74 2/2 sin( )) Thus, when The (TIvI) 1:
Tk Th: ~ 2 log Fu. (71v1)
= > (log()) (sx)) (( log(I)u(S2)) (11221111)))
7 (rivi log n4y - 7iv1 log no + 2(log ((x)] log (1
7 Mivi (log ni logn=>) m.)
(69)
From this, conclude that when Th. Tk.ar(1ivl (7iva) 1 and ([ = (مe for all k k' y, the ALA results in the decision boundaries as Ivs ladjuuuse... However, under LTR settings, TR.er (7vv)) vary significantly depending on io and N', making this approximation less realistic in practice.
27

---

Published as a conference paper at ICLR 2025
F EXPERIMENTS
In this section, we summarize the details of the experimental settings and results that are not fully covered in the main text. Appendix ET outlines the detailed experimental settings. Appendix E2 provides supplementary results for the experiments in Section 5.2 while Appendix D presents additional results for the experiments Section 5.3 under different conditions.
F.1 SETTINGS
We describe the detailed training settings in this section. We mainly followed the hyperparameters used in Hasegawa & Sato (2023). First, provide the details for experiments on image datasets, and then describe the experiments on tabular data, highlighting the differences.
F.1.1 DATASETS
Following Cun et al (2119) and etll (2011), we created long-tailed versions of image datasets. For tuning the hyperparameters Tiv1, Y+ 7x. we used validation datasets. Since CIFARIO and CIFAR100 do not have validation datasets,
created validation datasets using portion of the training datasets. Following Liu all (2019), we constructed the validation datasets by extracting only 20 samples per class from the training datasets and using the remaining samples the training datasets. For CIFAR 100, we set n1 to 480, and for CIFARIO 10, we set to 4980. The imbalance factor o was set 100. The classes within each dataset were divided into three groups Many, Medium, and Few-on on the basis of the number of training samples 7i. For CIFAR 100LT and ImageNet-LT, we categorized classes & as Many if they had more than 1000 training samples (ng > 1000), Medium if they had between 200 and 1000 training samples (200- < R& S 1000), and Few otherwise. For CIFARIO-LT, we classified classes as Many they had more than 100 samples, Medium if they had between 20 and 100 samples, and Few otherwise.
F.1.2 EVALUATION METRICS
Unless otherwise specified, we used the following hyperparameters with ResNet. We chose stochas- tic gradient descent with momentum = 0.9 as the optimizer and applied a cosine learning rate scheduler (Loshchilov Hutter 2017) to gradually decrease the learning rate from 0.01 0. The batch size was set to 64, and the number of training epochs was 320. The loss function used was cross-entropy loss, and regularization included a weight decay of 0.005 (Hanson Prati, (989) and feature regularization of 0.01 (Hasegawa & Sato, 2023). Although we used an ETF classifier for the linear layer, we did not use Dot-Regression Loss (Yang et all, 20226), following Hasseaaw Satd (2023). The optimal E (TivI, 7x } for LA and Issaajjster were determined using cross-validation on validation datasets, exploring values from (0.00, 0.05...... 2.00).
Next, we describe the experimental setup for ImageNet-LT. The learning rate was gradually de- creased from 0.05 to O, with the number of training epochs set to 200. Regularization involved a weight decay of 0.00024 and a feature regularization of 0.00003. All other settings were the same as those used for 1FAAR00-LTT.
For the experiments in Section 52.2 used the optimal values of f1v1, T+ Yx Yx determined by the validation data. For ALA, the value [[ was calculated by averaging the norm the class mean features for each class.
The accuracy reported in Section 5 represents the mean and standard deviation across five inde- pendent experiments, each using different random seeds. The values of 7/vl, 7x. and the training accuracy mentioned in Section 5.4 correspond to these experiments. All experiments were con- ducted on single NVIDIA A100.
F.1 TABULAR DATA
Next, we describe the experiments conducted on tabular data. Since tabular data have characteristics that differ significantly from image data, conducted these experiments to demonstrate that our results can generalize to other modalities beyond images. Apart from the settings described below, the procedure was identical to that used for image data. Following Hasegawa & Sato (2023)) we used
28

---

Published as a conference paper at ICLR 2025
CIFAR100-LT (ResNeXt50)
ALA
MLA
0.4
20
20
Class Index k'
0.2
40
40
0.0
60
60
0.2
80
80
- 0..
100
100
0
20
40
60
80
100
20
40
60
80
100
Class Index k
Figure 4: Heatmaps showing the difference in angles between the decision boundaries adjusted by each method and those adjusted by Ivs Ivsladjuster of ResNeXt50 trained CIFAR100-LA The angle differences between MLA and Ivs ladjuster are generally small compared to the difference between ALA and Ivsladjuster.
the Helena dataset with 100 classes. Since this dataset is not pre-split into validation and test sets, randomly sampled 20 non-overlapping samples per class for validation and test sets respectively. The distribution of this dataset resembles a long-tailed distribution with p = 40. For each class, those with more than 500 training samples categorized Many, those with 200 n& 500 as Medium, and the rest as Few.
In line with Kadra al (20221), we trained MLP with sufficient regularization. We used 9-layer MLP with 512-dimensional hidden layers as the feature map. The model was trained for 400 epochs using AdamW (Loshchrloy & Hutten, (0IX). Regularization methods included 0.15 dropout rate (Snvastavaa et all 2014), 0.15 weight decay, and 0.001 feature regularization.
F.2 DECISION BOUNDARY ANGLES
We present the results of the experiments from Section 5 conducted with different models and datasets. Figure H illustrates the differences in the angles of the decision boundaries of ResNeXt50 trained on CIFAR100-LT, while Figures 5 and f display the results for ImageNet-LTT and Helena, respectively. In all cases, the decision boundaries adjusted by MLA and Ivsl Ivsladjuster tend to be more similar each other than those adjusted by ALA.
F.3 TEST ACCURACY
In this subsection, we first present the error bar plot for the average test accuracy on Helena, which is not shown in Section B.3 followed by detailed table of test accuracy of models adjusted by each method.
Error Bar Figure shows average accuracy of models trained on Helena, adjusted by different methods. Notably, models trained on Helena, similar to those on ImageNet-LT, did not achieve training accuracy of 100%, indicating that NC has not fully occurred (see Table M... Despite this, the results show that MLA and Ivsladjuster achieve comparable accuracy, consistent with the experi- ments on other image datasets.
Table
We compare the test accuracy when each method is applied post-hoc to trained models. We refer to the models only trained the training data, without the application of any post-hoc
29

---

Published as a conference paper at ICLR 2025
ImageNet-LT (ResNeXt50)
ALA
MLA
0
0
0.4
200
200
Class Index k'
0.2
400
400
0.0
600
600
-0.2
800
800
-0.4
1000
1000.
200
400
600
800
1000
200
400
600
600
1000
Class Index k
Figure 5: Heatmaps showing the difference in angles between the decision boundaries adjusted by each method and those adjusted by Ivs ldjjuster of ResNeX150 trained on ImageNet-LT, The angle differences between MLA and lvs ladjusser are generally small compared the difference between ALA and I Ivsladjuster.
Helena (MLP)
ALA
MLA
0
0.4
20
20
Class Index k'
0.2
40
40
0.0
60
60
- 0.2
80
80
0.4
100
100
100
100
20
40
60
80
100
20
40
60
80
100
Class Index k
Figure 6: Heatmaps showing the difference in angles between the decision boundaries adjusted by each method and those adjusted by Ivsladjuster MLP trained on Helena. The angle differences between MLA and Ivs Ivsladjuster are generally small compared the difference between ALA and Ivs I adjuster.
30

---

Published as a conference paper at ICLR 2025
Helena (MLP)
29.6
Average Accuracy (%)
29.4
29.2
29.0
28.8
28.6
ALA
1v1
MLA
Figure 7: Average accuracy of MLP trained on Helena and adjusted by different methods. MLA and Ivs lajuutjr achieve comparable accuracy as with the experiments on other datasets in Figure E
Table 6: Accuracy of ResNet34 adjusted with each method on CIFARIO0-LT. MLA and Ivs Iajjuuusstr achieve comparable average accuracy. MLA outperforms ALA in average accuracy on 1IFAR100-LT.
Method
Many
Medium
Few
Average
Baseline
77.9 77.940.3
46.8+1.0
15.3+0.3
47.6 47.640.5
ALA (7+ 1.0)
72.3+0.2
47.9-1.1
28.3
50.140.4
ALA (best)
75.640.9
49.2+1.0
25.2
50.740.4
Iv1 (7iv1 = 0.5)
76.440.3
51.7=0.9
24.7+0.7
51.74002
Iv1 (best)
74.2
53.0
29.6
52.9
MLA (7x = 0.5)
75.610.3
52.3
27.1
52.410.2
MLA (best)
73.2
52.8 1.9
30.9111
53.0.o.3
adjustment methods, as Baseline. We present two cases for each adjustment method: when the hy- perparameter values are set according to the values derived from each theory (0.5 or 1.0), and when the parameters are tuned using validation data (best). addition to the overall average accuracy (Average), we also report the average accuracy for the Many, Mediam, and Few categories. Refer to Appendix EIll for details on these categorizations. The results for CIFAR100-LT and CIFAR 10- LT are presented in Tables B and M respectively. Table E reports the accuracy of ResNeXt50 CIFARIOO-LT, while Tables and 100 display the results of ImageNet-LT and Helena, respectively.
In all results, the Baseline shows higher accuracy for the Many category, naive training long- tailed data results outputs biased toward head classes. The Ivsladjuster and post-hoc LA methods successfully adjust for this bias by increasing the accuracy of Few classes, thereby improving the overall average accuracy. As shown in Section 5.3, MLA achieves nearly the same average accuracy the Ivslajjuster across all datasets. Additionally, MLA outperforms ALA in terms of average accuracy for CIFARIOO-LT.
F.4 COMBINATION WITH OTHER METHODS
Since MLA only adjusts the logits during inference, it can be seamlessly integrated with other LTR techniques. Notably, methods classified under "Information Augmentation" and "Module Improve- ment" in the taxonomy by Zhang et all (2 are particularly compatible. Here, we present ex- perimental results combining MLA with the approach proposed by |Wang all (2023a), which falls under "Information Augmentation." " We compare the average accuracies a model trained with cross entropy loss against with MI.A. The dataset used is CIFAR 100-LT of our implementation,
31

---

Published as a conference paper at ICLR 2025
Table 7: Accuracy of ResNet34 adjusted with each method on CIFARIO-LT. Although the num- ber of classes is insufficient and the assumptions of our theory do not hold completely, MLA and Ivs Ivsl lajjuster achieve comparable average accuracy.
Method
Many
Medium
Few
Average
Baseline
89.2 1..
76.3
60.8
76.8+0.8
ALA (7+ 1.0)
87 22..
78.5
71.7
79.941.0
ALA (best)
87.7+1.2
78. 78.2+3.0
70.5+2.7
79.7+1.1
Ivl (7iv1 0.5)
88.0+2.2
78.6
68.944.2
79.5+0.8
Ivl (best)
87.941.8
79.324.0
69.0+2.7
79.6+1.1
MLA (7x = 0.5)
88.1
78.7
68.414.3
79.4.0.8
MLA (best)
88.0
79.2
68.812.9
79.611.1
Table 8: Accuracy of ResNeXt50 adjusted with each method Cifar100-LT. MLA and Ivsladjuster achieve comparable average accuracy. MLA outperforms ALA in average accuracy on CIFARI00- LT.
Method
Many
Medium
Few
Average
Baseline
77.4 10.3
49.8
18.2
49.4
ALA (7+ = 1.0)
73.310.6
50.910.5
31.010.4
52.410.3
ALA (best)
76.340.3
51.920.4
26.2+0.4
52.2+0.2
Ivl (71v1 0.5)
75.640.3
53.520.5
27.640.7
53.0+0.2
Ivl (best)
70.5
54.0.1.2
36.610.9
54.240.3
MLA (Yx = 0.5)
74.540.3
53.7
30.3
53.5 53.540.2
MLA (best)
72.9
53.8
33.440.6
54.0
Table Accuracy of ResNeXt50 adjusted with each method on ImageNet-LT. Although insufficient NC has occurred (see Table MI) and the assumptions of our theory do not hold completely, MLA and Ivs addjuster achieve comparable average accuracy.
Method
Many
Medium
Few
Average
Baseline
68.1+0.2
43.0+0.2
17.0+0.5
49.1+0.2
ALA (7+ = 1.0)
65.3+0.2
48.1401
31.2+0.7
52.4+0.1
ALA (best)
63. 10.4
48.7.0.2
36.710.9
52.6
Ivl (71v1 0.5)
48.710.5
48.011..4
48.4
48.3
IvI (best)
62.8+0.4
49.3
34.7+0.7
52.5+0.2
MLA (7x 0.5)
38.940.7
44.04..3
52.310.6
43.240.3
MLA (best)
62.4
49.3
36.0
52.640.2
Table 10: Accuracy of MLP adjusted with each method on Helena. Although insufficient NC has occurred (see Table M and the assumptions of theory do not hold completely, MLA and Ivs adjuster achieve comparable average accuracy.
Method
Many
Medium
Few
Average
Baseline
36.1
21.610.6
17.4410
25. 25.240.3
ALA (74 1.0)
34.7
26.510.8
23.810.3
28.440.a
ALA (best)
31.2+1.2
28.8
27.410.
29 29.2+0.4
Ivl (7ivi
0.5)
32.9+10
29.0+11
24.7+0.
28.940.4
IvI (best)
31.5
29.8
25.6
29.0
MLAA (ix
0.5)
31.1
30.211.0
26.0
29.1.0.5
MLA (best)
31.341.7
29.8
25.8+1.0
29.040.3
32

---

Published as a conference paper at ICLR 2025
Table 11: Accuracy of ResNet32 trained with Wang al, (20255)) and adjusted with MIA
CIFARI MLA can be used effeetively in combination with other LTR methods to improve accuracy.
Method
Many
Medium
Few
Average
Wang all (2)23
73.5
45.1
13.5
44.9
Wang et al (2023a
MLA
63.8
51.7
30.2
49.1
Table 12: Accuracy of ResNet34 adjusted with each method on CIFARI00-LT with 200. MLA and Ivs adjjsster achieve comparable average accuracy, MLA outperforms ALA average accuracy on CIFAR100-LI,
Method
Many
Medium
Few
Average
Baseline
78. 78.4
48.4
12.5
43.1
ALA (7++ 1.0)
69.940.8
46.3
21.4+1.0
43.440.7
ALA (best)
76.340.3
50.320.4
19.3+1.0
45.740.4
Iv1 (7iv1 = 0.5)
76.5+0.4
53.320.4
20.2+0.6
47.040.2
Ivl (best)
74.640.8
53.620.5
23.441.2
47.840.4
MLA (7x = 0.5)
75.4
53.1 $3.1
22.510.6
47.610.3
MLL (best)
73.5
52.9
24.410.4
47.7
while other code and experimental settings adhere to the official implementation by Wang et all ((UZKE)) Note that these results are not directly comparable with our other experiments due dif- ferences in setup. The results are summarized in Table III As shown, combining MLA with other LTR methods can further improve accuracy.
F.5 HIGHER IMBALANCE FACTOR
To evaluate the effectiveness of MLA more imbalanced datasets, conducted experiments on the CIFAR-LT dataset with = 200. All other experimental settings were identical to those described in Section 5.1 The results are shown in Tables ID] and IB. As in the case with p 100, the results demonstrate that MLA closely approximates Iss adjuster in all scenarios. MLA achieves higher mean accuracy than ALA CIFAR100-D in this case well.
Table 13: Accuracy of ResNet34 adjusted with each method on CIFARIO-LT with p = 200. MLA and Ivsladjuster achieve comparable average accuracy.
Method
Many
Medium
Few
Average
Baseline
91 01.641.6
69.7
54.0+3.3
70.040.6
ALA (7+ = 1.0)
81.7
69.3
69.3
73.041.8
ALA (best)
89. 89.2.11.9
70.512.0
64.513.0
73.710.8
IvI (Tivl = 0.5)
88 88.443.2
71.442.0
64.944.
73.940.6
IvI (best)
87.543.9
71.7
66.1154
74.240.7
MIA (7x = 0.5)
88.813.0
71.412.0
64.1
73.7
MLA (best)
88.043.4
71.7
65.445
74.1+0.8
33


Reviews:
[{"review_id": "MiW2i4PoF5", "reviewer": "Reviewer_Hejv", "summary": "This paper provides a theoretical foundation for the Multiplicative Logit Adjustment (MLA) method used in long-tailed recognition tasks. First, the authors develop a theory for optimally adjusting decision boundaries by leveraging feature spread estimates derived from neural collapse (NC). They then demonstrate that MLA effectively approximates this optimal adjustment method. Experiments conducted on various long-tailed datasets validate the practical applicability of this approximation, showing that MLA performs well under realistic conditions.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper presents a theory for optimally adjusting decision boundaries based on NC and links it to the MLA method, providing a strong theoretical explanation for MLA and adding depth to its empirical success.\n- The authors conduct a series of experiments on long-tailed datasets to validate the theory and demonstrate the practical utility of MLA.\n- The authors offer both theoretical and empirical comparisons between MLA and ALA, clarifying their differences and highlighting the advantages of MLA.", "weaknesses": "- The iNaturalist dataset is a widely used large-scale long-tailed dataset. Conducting experiments on this dataset would be beneficial to examine the behavior of MLA, ALA, and the 1vs1 adjuster in a larger-scale real-world context. Additionally, testing on CIFAR-LT with more extreme imbalance ratios would further evaluate the effectiveness of the proposed method.\n\n- The theoretical framework relies on some strict assumptions that may not always hold in practice, such as the assumption of conditions under which NC occurs, which may not be valid in highly imbalanced real-world datasets. Similarly, the assumption that $\\psi \\rightarrow \\pi/2$ does not hold when $K$ is small.\n- There are some typos, such as on line 373, where \"the optimal decision boundaries during inference is effected by\" should be corrected to \"the optimal decision boundaries during inference are affected by\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ub8a3CRMZt", "reviewer": "Reviewer_EiUb", "summary": "The paper aims to provide a theoretical justification for Multiplicative Logit Adjustment (MLA) in long-tailed recognition by linking it to Neural Collapse theory. The authors develop a theoretical framework for optimal decision boundary adjustment based on feature spread estimates from Neural Collapse and demonstrate that MLA approximates this optimal method.", "questions": "The paper mentions that \"MLA has demonstrated significant empirical success,\" but it does not provide sufficient evidence to support this claim. Could more literature or experimental results be added to demonstrate the current advantages of MLA in long-tailed recognition?\n\nThe theoretical comparison between MLA and ALA is addressed in some aspects, but why is there no comparison with other advanced baselines in long-tailed recognition (such as the latest contrastive learning methods or distribution-balanced loss)?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The framework that connects MLA with Neural Collapse is novel.  The assumptions and conditions are clearly stated, and the notation is consistent and well-defined.", "weaknesses": "1. **Motivation**:\n   The paper fails to establish a compelling motivation for the study. The claim that \"MLA has demonstrated significant empirical success\" is not sufficiently substantiated. There is no systematic analysis of the current strengths and challenges of MLA in long-tailed recognition.\n\n2. **Theory-Practice Gap**:\n   The connection between optimal decision boundary adjustment and actual performance improvements in long-tailed scenarios is weak. While the paper seems to demonstrate the effectiveness of MLA as a close approximation to the 1vs1adjuster from both theoretical and experimental perspectives, and contrasts it with Additive Logit Adjustment (ALA), it does not thoroughly analyze how MLA addresses the specific challenges of long-tailed learning from a theoretical standpoint. Additionally, there is no comparison with state-of-the-art long-tailed baselines or discussion on how MLA could be integrated with them. The experimental section focuses more on hyperparameter tuning for MLA rather than examining its impact on long-tailed learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5xlVRHSi6L", "reviewer": "Reviewer_K7Rv", "summary": "The paper addresses the problem of class imbalance in classification models and analyzes the effectiveness of the Multiplicative Logit Adjustment (MLA) method. The authors propose a theory on optimal decision boundaries based on neural collapse, demonstrating that MLA approximates these optimal boundaries. The paper is supported by experiments across various modalities, providing insights into MLA’s performance under different conditions.", "questions": "1. Why did you primarily focus on ResNet models for the image analysis? Were there specific reasons for this choice over other architectures?\n2. Why did you create imbalanced versions of balanced datasets instead of using inherently imbalanced datasets? For example, datasets that exhibit natural class imbalances could offer more realistic evaluations.\n3. Although the paper focuses on post-hoc LA methods, did you compare the results with loss-function-based approaches? How does MLA's performance compare to other logit adjustment methods?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The problem of class imbalance is clearly motivated, and the investigation into MLA is a valuable contribution to this area of research.\n* The mathematical framework is well-developed, offering a solid theoretical foundation for understanding the method.\n* The analysis extends beyond image datasets to include other modalities, such as text, enhancing the paper's applicability.\n* Extensive experiments are conducted, which reinforce the theoretical claims and provide a comprehensive evaluation of MLA’s performance.", "weaknesses": "* While the theoretical concepts are accurately communicated, the writing could be improved by breaking up overly long sentences to enhance readability (e.g., lines 098-104).\n* The results in Tables 6-10 are intriguing, but more explanation is needed to clarify the implications of \"Many,\" \"Medium,\" and \"Few\" categories. Additionally, the reasons for the Baseline outperforming others in the \"Many\" category should be discussed in more depth.\n* The descriptions for tables and figures are too brief, making it difficult for readers to grasp the full context of the results. Adding more detailed captions would help in understanding the findings.\n* The paper contains numerous formal definitions and formulas, which can disrupt the reading flow, especially for readers who are not deeply familiar with this specific research area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)