Long-tailed Recognition with Model Rebalancing
Authors: Jiaan Luo
Feng Hong
Qiang Hu
Xiaofeng Cao
Feng Liu
Jiangchao Yao
Venue: NeurIPS 2025 poster ()
Keywords: machine learning
imbalanced learning
Primary area: general_machine_learning

Abstract: Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization to the tail classes. Despite the promise of previous methods from the perspectives of data augmentation, loss rebalancing and decoupled training etc., consistent improvement in the broad scenarios like multi-label long-tailed recognition is difficult. In this study, we dive into the essential model capacity impact under long-tailed context, and propose a novel framework, Model Rebalancing (MORE), which mitigates imbalance by directly rebalancing the model's parameter space. Specifically, MORE introduces a low-rank parameter component to mediate the parameter space allocation guided by a tailored loss and sinusoidal reweighting schedule, but without increasing the overall model complexity or inference costs. Extensive experiments on diverse long-tailed benchmarks, spanning multi-class and multi-label tasks, demonstrate that MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods. These results highlight MORE's potential as a robust plug-and-play module in long-tailed settings.

Long-tailed Recognition with Model Rebalancing
Jiaan Luo! 1.4+ Feng Hong! Qiang Hu' Xiaofeng Cao² Feng Liu
Jiangchao Yao
CCooperative Medianet Innovation Center, Shanghai Jiao Tong University
School of Computer Science and Technology, Tongji University
Schooo of Computing and Information Systems, The University of Melbourne
Shanghai Artificial Intelligence Laboratory (luojiaan, Teng.hong, qiang. hu, hu, Sunarker] edu. xiaof laaffenccaottongg edu. cn memb.......... edu.
feng liul Comimelb
Abstract
Long-tailed recognition is ubiquitous and challenging in deep learning and even the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization the tail classes. Despite the promise of previous methods from the perspectives of data augmentation, loss rebalancing and decoupled training etc., consistent improvement in the broad scenarios like multi-label long-tailed tailed recognition is difficult. In this study, we dive into the essential model capacity impact under long-tailed context, and propose a novel framework, MOdel REhalancing (MORE), which mitigates imbalance by directly rebalancing the model's parameter space. Specifically, MORE introduces a low- rank parameter component to mediate the parameter space allocation guided by a tailored loss and sinusoidal reweighting schedule, but without increasing the overall model complexity or inference costs. Extensive experiments on diverse long-tailed benchmarks, spanning multi-class and multi-label tasks, demonstrate that MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods. These results highlight MORE's potential as robust plug-an plug-and-1 and-play module in long-tailed settings. The code is available here.
Introduction
Deep learning has revolutionized numerous domains, from computer vision large language models, with unprecedented performance largely fueled by large-scale well-curated datasets [Russakovsky al., 2015]. However, many real-world uncurated data in diverse scenarios like medical diagnosis follows long-tailed distributions, where small subset of dominant classes comprises the majority of samples, while numerous minority classes remain severely underrepresented [Krizhevsky et al., 2009]. Such ubiquitous imbalance presents a fundamental challenge for modern deep learning models that easily overfit high-frequency classes while exhibiting degraded performance w.r.t. low-frequency classes [Zhang et al., 2023]. It thus has been critical to explore robust methods against long-tailed challenges under multi-label, multi-class and even finetuning scenarios [Chen et al., 2025].
There are range of methods developed to address the long-tailed recognition challenge from the perspectives of data augmentation, decoupled training [Kang et al., 2020b], loss rebalancing [Ma et al., 2023], and contrastive learning [Zhu et al., 2022, Du et al., 2024]. Despite promise in specific contexts, they struggle to deliver consistent improvements in broader and more complex scenarios.
"The first two authors contribute equally.
The corresponding author is Jiangehao Yao (Sunarkerdsjtu.c edu, en).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

For example, while logit adjustment [Menon et al., 2021] that builds the class frequency corrections, and probabilistic contrastive learning [Du al., 2024] that rebalances feature representations are effective, they usually fail to take effect in the label coupling scenarios like multi-label long-tailed learning. For data augmentation ways [Shi et al., 2023], although proper re-sampling balances the class bias, the incurring cost is non-negligible, especially for the finetuning of large foundation models.
Different from previous perspectives, we explore a novel but more essential direction, which focuses on manipulating the model space of majorities and minorities and can be easily generalized to different scenarios efficiently. We start from an intuition that preserving proper model space for minority class in the manner of low-rank decomposition can help combat the imbalance challenge at the model level. Then, with a principled analysis of Rademacher complexity under space decomposition, we show that such a construction can actually support tightening the generalization bound of long-tailed learning [Menon et al., 2013, Wang et al., 2023], which guarantees the rationale of this new direction.
Based on the above analysis, we propose novel approach called MOdel REbalancing (MORE), which partitions the parameter space to reserve dedicated capacity for minority classes while preventing dominance from majority classes (Eq. (I)). To guide this parameter space reallocation, we introduce a tailored discrepancy-basse -based loss that measures the contribution of the low-rank component to the model's 's predictions (Eq. (6)). with class-wise weighting that encourages the low-rank component to focus on tail classes (Eq. (3)). This process is further optimized through a sinusoidal reweighting schedule that dynamically adjusts the influence of our reallocation loss throughout training- baining-starting low to establish generalizable features (Eq. (5)). At inference time, low-rank components are fused with no additional computational overhead. The contributions are summarized as follows:
We provide theoretical insights into the manner of model space manipulation under class imbalance (Theorem 1), demonstrating that by properly partitioning the model space for majority and minority classes, the generalization bounds of long-tailed learning can be further tightened, which enlightens a new direction to combat the long-tailed challenges at the model space level.
We propose novel method, MOdel REbalancing (MORE), for long- tailed recognition without increasing the overall model complexity or inference costs, which builds on low-rank parameter decomposition and designs a tailored discrepancy- beaerpaacy-bbeed based loss with sinusoidal scheduling that guides the proper space for minority classes and simultaneously safeguards the training of majority classes.
We conduct extensive experiments across a diverse set of datasets, and the results show that MORE consistently improves long-tailed recognition in both single-label and multi-label settings, including those integrated with CLIP-based finetuning. The in-depth analysis discloses that MORE reduces the tendency to converge to saddle points, and proves the rationale of the module design.
2 Related Work
2.1 Single-Label Long-tailed Learning
For single-label long-tailed learning, there are substantial explorations in the recent years [Zhang et al., 2023]. At the data level, the researchers considered over-sampling and data mixing [Chawla et al., 2002, Zhong et al., 2021, Shi et al., 2023], while other transfer learning approaches [Yin al., 2019, Wang al., 2021a, Jin et al., 2023, Li et al., 2024a] aimed to enhance minority class feature space. However, limitations in synthetic data quality mainly restricted their effectiveness. At the model level, methods like decoupled frameworks [Kang al., 2020a, Desai al., 2021] separate feature representation learning from classifier optimization to reduce imbalance effects. Re-weighting techniques [Ma et al., 2023, Jiang et al., 2023, Hong et al., 2024, Luo et al.. 2024] adjust the class importance during optimization, encouraging the model to pay more attention to underrepresented classes. Decision boundary adjustment [Cao et al., 2019, Menon al., 2021, Li et al., 2022, Hong et al., 2023, Wang et al., 2025] makes effective by imposing class-specine margins, narrowing the performance gap between majority and minority classes. Recently, contrastive learning approaches [Zhu et al., 2022. Zhou et al., 2023b, Cui et al., 2024, Du et al., 2024, Zhou et al., 2024] show promise for long-tailed recognition by encouraging uniformly discriminative features in all classes. Fine-tuning foundation models [Shi et al., 2024, Li et al., 2024b] has also gained traction as a new paradigm, where lightweight approaches have demonstrated notable efficacy in long-tailed learning.

---

2.2 Multi-Label Long-tailed Learning
Due to the label coupling effect in multi-label long-tailed learning [Ridnik et al., 2021], the meth- ods for single-label long-tailed learning usually cannot be directly applied [TTrekegn al., 2021]. To address this challenge, various modeling methods have been explored. Recurrent neural net- works [Wang et al., 2016, Yan al., 2018] and graph convolutional networks [Chen et al.. 2019] have been introduced to learn joint image-label embeddings that better capture label dependencies. Despite architectural advances, binary cross entropy (BCE) remains foundational due its decompo- sition of multi-label tasks into class wise binary objectives. To tackle label imbalance, distribution balanced loss [Wu et al., 2020] incorporates re-weighting based on label co-occurrence statisties, while asymmetric loss [Ridnik et al., 2021] introduces asymmetric focusing factors to treat positive and negative labels differently. Recent studies have explored AUC-based methods [Yang et al., 2021, Wang et al., 2022), offering deeper insights into domain adaptation for long-tailes problems in multi-class settings. Additionally, vision-languag models like CLIP [Radford et al., 2021] have been adapted for multi-label recognition [Sun et al., 2022, Xia et al., 2023], leveraging label semantics to enhance generalization and mitigate imbalance through cross- cross-modal supervision.
3 Method
3.1 Problem Setup
Consider a standard classification task under imbalanced data settings. Let the training dataset be denoted as S = U (a, (.),. where |S| = N is the total number of training samples, æ; X an input sample, and y E 3 S {I..... C} its corresponding label from total of C classes. We denote the number of samples in each class as N2,,....... Nc), and assume, without loss of generality, that N₂ < Nj for any < j. In practice, the disparity in sample counts can be substantial, with N1 & Nc. capturing the essence of long-tailed distributions commonly found in real-world datasets. The relative class proportions are represented by { TT...... . Teh. where each T, = NN/N reflects the empirical prior of class i. For multi-label classification, for each sample a; X. y/ E {0, 1 represents its corresponding one-hot label vector, indicating the set of labels assigned to æt. Let N; denote the total number of samples in which label i appears, and let the total number of label occurrences across all samples be N' Σ N. The empirical prior for label i is then defined as x. = N representing the proportion of samples containing label i.
3.2 Space Decomposition
Prior research [Wang et al., 2023] has established wise fine-grained generalization bounds for the balanced risk, in which critical factor is shrinking Rademacher complexity of the model. Intuitively, in long-tailed learning, minority classes usually suffer from the limited representational capacity due to their scarce examples, resulting in substantially higher Rademacher complexity for these classes. This raises an interesting hypothesis: whether can we manipulate the model space for majority classes and minority classes to pursue a better generalization?
We start our intuition by decomposing the parameter space with a low-rank decomposition technique, which is, though, similar to the well-known Low-Rank Adaptation (LoRA) [Hu et al., 2022] in the form, but fundamentally different in the optimization process. Formally, for a neural network with parameters 6. comprising weight matrices (W.) across various layers, our core design lies in systematically decomposing each weight matrix into specialized components:
W. = W + W W + B/ Vi (1,2. ... M).
.L,,
(1)
where W E Rm. captures generalizable knowledge primarily benefiting majority classes, W E m specializes in representing minority-speccin patterns, B, Rm and A, E R with rank r < min(m;, kx) guarantee the low-rank property of W At the network level, this decomposition yields two complementary parameter subsets 09 = (W?, W9 W +W; M ). In the following, we provide a theoretical proof that if we properly preserve the space 0¹ for minority classes, we will have some potential to achieve better generalization bound for long-tailed learning.
W W and
O² = (W/. W .., MM J. and composes
whole by 00 (W/
3

---

Class Imbalance
Space Reallocation
Performance
Sinusoidal Reweighting
Head
MORE
x
Simpls
89 d
Lease
Mid.
Tel
Figure I: An overview of the proposed method's framework. The left figure illustrates how our model rebalancing designed. The right figure presents the performance on the NUS WIDE-SCENE dataset across the Many/Medium/Few splits. Our method demonstrates a significant improvement in the performance of minority classes, while maintaining or enhancing the performance of other
classes.
3.3 Theoretical Understanding
Basics. To begin with, we provide some necessary notations and basics. For baseline model Fo and our proposed model F, where Fu = {f(=; 0) 0 6),0 S R and F {f 8 8*) 09 E gg 0 E O,), Og S TR + S RRdd, de < d. In long-tailed learning, standard generalization bounds fail to adequately capture performance across the class spectrum. Prior works [Ren et al., 2020, Wang et al., 2023] established class-wise generalization bounds that highlight how empirical Rademacher complexity significantly limits the generalization capabilities for minority classes. This insight motivates our further analysis in the following theorem when manipulate the model space for majority and minority classes. For the detailed proof, please refer to Appendix B.
Theorem 1. Given a function set F, loss function L, and training set following class- conditional distribution D, the balanced risk for any function is defined (f) & Eynn (ff_,, y)]- For the baseline model Fo and our proposed model as defined above, with class proportions Ty = Ny/N for each class y, the proposed model F enjoys a tighter generalization bound compared to the baseline Fo. That is, for any f E F and fo E Fo. it holds that Rfa(s) $ (Jo)
Remark 1. By decomposing model functionality into general and minority-specific components, we separately examine their contributions to the overall complexity. Our analysis of class-specific Rademacher complexities reveals that the proposed approach redistributes modeling capacity toward minority classes while maintaining the overall complexity bound. The tighter generalization bound proves that our intuition has formal guarantees for more equitable performance across all classes.
3.4 Model Rebalancing
Motivated by the theoretical analysis in Section 3.3 on decomposing model parameters, we now detail the practical instantiation of parameter rebalancing through low-rank adaptation and a tailored discrepancy- loss, which adaptively emphasizes tail-specific learning to enhance performance in long-tailed distributions.
3.4.1 Parameter Space Reallocation
Given the parameter decomposition 0 8 0', our objective during training is to allocate 0f for minority expertise while reserving 8 for majority classes and general knowledge, thereby enhancing minority classes' representation through space reallocation. Remember that f(æ; 0) denotes the output logits produced by parameters for input æ. Under optimal training conditions, the complete model output f(: 8) demonstrates robust performance across the entire class distribution. Concurrently, fx;; 89) exhibits strong performance exclusively on majority classes while performing poorly on minority classes, confirming that 09 successfully avoids encoding minority knowledge.
To this end, we propose a tailored loss function that encourages such space reallocation by leveraging logit-level contrastive supervision. Concretely, we discrepancy- method to measure the

---

influence of l on the model's 's output logits. We compute the &g distance between the output logits of the model with 8 H' and the model with 8":
M( 0) [=; 00 @ 0¹) f(x;; 89)
This discrepancy term M quantifies the contribution of 6 to the model's s prediction. Intuitively, this term captures the class-specific knowledge introduced by 6 that is not present in 69. We propose model rebalancing loss, which encourages 0e to learn minority expertise and @ to capture generalized knowledge, with class-wise weight based on the empirical class distribution:
LMORE (0) =
Σ (x.
Tg.(a;;;)
This reweighting approach is intentionally designed to assign larger values to the majority. As result, the loss will enforce stronger penalties for discrepancies on majority-class samples, thereby driving of to minimize its influence on majority predictions. In contrast, for samples in minority classes, which have smaller weights, the loss imposes weaker penalties, allowing 0 to retain and amplify its distinct representational contribution. This shifts the learning focus of O' toward minority classes, promoting effective reallocation of the model's internal capacity. In multi-label recognition, the label y E (0, e] is one-hot label vector. The multi-label version of LMORE is defined as follows,
Am MORE ( = S
Σ ΣΣΣ (x.y) j=1
y
Ty M(x; 00,0")
1i=1 yi
where the summation is taken over all active labels (i.e., yj = 1) to ensure that capacity reallocation remains effective and balanced across all labels, even in complex label distributions.
3.4.2 Sinusoidal Reweighting
To further facilitate learning through the model rebalancing loss, we introduce a dynamic weighting scheme (T) based on a sinusoidal schedule on training time step which is characterized as follows,
a(T) AA sisin
T T
(5)
where the peak amplitude controlling the maximum influence of the rebalancing loss, and T is the total number of training iterations. The explanation behind this design is to gradually adjust the strength of the MORR to balance learning priorities across different phases of optimization. At the early stage of training, the model should prioritize learning paase-grained, easily separable representations, which are predominantly governed by majority-class samples. Therefore, we assign small weight to the reallocation loss LMORE, reducing its regularization effect and allowing 09 to establish strong generalizable features. As training progresses into the middle phase, the weight assigned to LMORE increases, enabling the low-rank parameters Of to effectively learn from minority classes that are often overlooked. In the later stages, the weight is reduced again to prevent overfitting to the reallocation signal and to maintain unbiased convergence behavior.
3.4.3 Optimization and Inference
Our overall training framework integrates standard objective optimization with specialized space reallocation through dynamic weighting mechanism. Given our decomposed parameter structure 0 = 0 + 0 the training objective is defined as follows.
min C(O, = Cbase (0) + (aT) (MORE(())
(6)
where Chase denotes the primary loss function of different baseline methods, and LMORE functions as a specialized regularizer that systematically reallocates model space to protect minority classes. This dual-objective approach guides parameter optimization toward more balanced allocation of model parameter space across majority and minority classes. Fig. 1 illustrates this training process. The pseudo-code of our training process is shown in Appendix A.
During inference, we seamlessly merge the decomposed parameters, yielding two erucial benefits: 1) Identical inference computational complexity to standard models. 2) No increase in model storage requirements. These efficiency characteristics are particularly valuable in production environments, where inference speed constitutes primary bottleneck [Aminabadi et al., 2022].
5

---

4 Experiments
4.1 Experimental Setup
Datasets and evaluation metrics. We evaluate the proposed method on a suite of widely used long-tailed benchmarks, covering both single-label and multi-label image recognition settings. We conduct experiments under varying imbalance factors (IF), defined as the ratio of sample counts in the most frequent class (Nmmax) to the least frequent class (Nimin). For single-label recognition, we adopt CIFAR- 100-T [Krizhevsky et al., 2009] and Places-LT [Liu et al., 2019]. CIFAR- is a long-tailed variant of the standard CIFAR- 1-00 dataset, consisting of 100 classes with an imbalance factor of 10 and 100, where the number of samples per class follows an exponential decay. LT contains 62.5k training images from 365 scene classes, with the number of samples per class varying from 5 to 4,980, and an imbalance factor of 996. For multi-label recognition, we conduct experiments on four diverse datasets: MIMI. [Zhou and Zhang, 2006], Pascal- VOC [Everingham al., 2010), NUS-W WIDE-- [Chua et al., 2009], and MS-COCO [Lin et al., 2014]. These datasets represent a spectrum of complexity, with the number of classes ranging from 5 (MIML) to 80 (MS-COCO) The average number of labels per image varies from 1.24 (MIML) to 3.5 (MS-COCO), while the imbalance factors span from relatively balanced (1.53 for MIML) to severely imbalanced (352.92 for MS-COCO). For more information about multi-label datasets, please refer to Appendix C.I. This comprehensive evaluation suite enables us to assess our method's 's robustness across different multi-label recognition scenarios with varying degrees of class imbalance. We follow standard protocols in long-tailed classification by treating all classes equally during testing and reporting results across three splits: Many, Medium, and Few, based on the number of training samples per class. For single-label and multi-label datasets, we report top-1 accuracy and mean Average Precision (mAP) respectively the evaluation metrics.
Baselines. We compare our method with a range of strong baselines commonly used in long-tailed classification. For single-label tasks, we include models trained with standard cross-entropy loss (CE), class-balanced loss (CB) [Cui et al., 2019], logit adjustment (LA) [Menon et al., 2021], 1. balanced contrastive learning (BCL) [Zhu et al., 2022], and probabilistic contrastive learning (ProCo) [Du et al., 2024]. For label classification, we evaluate against binary cross entropy (BCE), focal loss (Focal) [Lin et al., 2017], and asymmetric loss (ASL) [Ridnik et al., 2021]. which are widely adopted for handling label imbalance. Additionally, since recent advances have shown the effectiveness of vision-language models in multi-label settings, we also perform experiments based the CLIP framework, enabling a broader evaluation across modalities.
Implementation details. Our code is implemented with Pytorch 1.12.1. Experiments based on CIFAR-I00-LT and MIML are carried out on NVIDIA GeForce RTX 3090 GPUs, while other experiments are carried out on NVIDIA A 100 GPUs For fair comparison, we use ResNet32 on DEFAR-10-LLY ResNet34 on MIML, Pascal VOC and NUS- WIDEESCENE, ResNet50 on ImageNet- LT and pre-trained ResNet- 152 on Places-LT. We train each model with batch size of 64 (for Pascal-VOCH 128 (for ImageNet-LT) / 256 (for CIFAR- 100-LL, MIML and SESS-IDE-SCENEE / 512 (for Places-LTT) / 1024 (for MS-COCO), SGD optimizer with momentum of 0.9, weight decay of 0.0002. For multi-label tasks, the initial learning rate is set to 3e-4, with cosine learning- rrate scheduling along training. For tasks based on CLIP model, we use CLIP's Transformer-bant trained text encoder to extract label features. During training, only vision encoder is tne-tuned, using a pre-trained ResNet34 model. Other settings are aligned with those of non-CLIP-based models.
pre-
4.2 Comparison Results
The efficacy of our proposed method, MORE, is assessed through comparative experiments on widely used single-label and multi-label long-tailed classification benchmarks, including finetuning pre-trained CLIP model scenarios. Detailed results are presented in Table and Table 2.
Single-label recognition. We first evaluate MORE on CIFAR- employing two distinct imbalance factors (IF=10 and IF=100) to test its adaptability. As evidenced in Table 1, MORE consistently elevates performance across all class frequency splits (Many, Medium, Few) when integrated with strong baselines like LA and ProCo. This consistent improvement, irrespective of the imbalance severity on CIFAR-100-LTT underscores the robustness of MORE and its general applicability. The performance advantages become even more critical on the large-scale Places-LT dataset, which presents far more severe imbalance (IF 996) and substantially larger number of

---

Table 1: Top-1 1 accuracy (%) (1) results for Many, Medium, Few and overall classes on CIFAR-100-BE and Places- LT datasets. For CIFAR-100-1 results are categorized by imbalance factors (IF).
CIFAR- 100-LT IF=10
IF=100
Places-LT
Method
Many
Medium
Few
All
Many
Medium
Few
All
Many
Medium
Few
All
75.3
62.1
44.5
61.4
73.1
45.1
9.2
44.1
46.0
22.3
5.2
27.5
CB
66.1
63.5
55.8
62.1
72.8
44.8
11.9
44.7
46.0
23.6
10.4
29.1
BCL
70.7
62.7
58.5
64.2
66.8
52.8
31.9
51.4
42.4
41.6
30.4
39.7
69.9
62.8
57.4
63.7
65.3
51.7
31.9
50.5
42.0
40.3
27.4
38.4
+MORE
70.9
64.4
58.2
64.8
65.3
52.3
33.6
51.2
39.5
42.0
30.6
39.5
ProCo +MORE
71.3
64.0
58.7
65.0
67.4
52.2
33.4
51.9
43.0
41.5
31.6
40.1
72.2
64.4
60.2
65.9
68.4
53.5
34.0
52.9
43.3
42.2
33.1
40.8
classes (365). MORE continues to deliver substantial gains, particularly for the under-represented (Medium and Few) classes, which are the primary bottleneck for such severe class imbalance. Specifically, when synergistically combined with LA on Places-LT, MORE improves accuracy for Medium and Few classes by noteworthy 1.7% and impactful 3.2%, respectively. Collectively, these results affirm not only MORE's effectiveness in directly mitigating class imbalance and its compatibility with established rebalancing methods. For more comparison results, please refer to Appendix C.2.
Multi-label recognition. Our proposed method, MORE, demonstrates notable effectiveness in enhancing multi-label long-tailed classification, consistently improving performance when integrated with established baseline methods across diverse benchmarks, as detailed in Table 2. For results in Table with standard deviation (Std), please refer to Table 10. The versatility of MORE is initially showcased on the MIML dataset, where its application with BCE, Focal, and ASL loss functions yields significant overall mAP gains of 3.8%, 4.0%, and 3.2%, respectively. As the dataset complexity increases, such in PASCAL-VOC and NUS- WIDE-SCENEE-bol of which exhibit larger label spaces (20 and 33 classes) and more severe class imbalance (with imbalance factors of 20.92 and 159.933) the benefits of MORE become more pronounced for mndr--rppreeented classes, On VOC,, MORE brings substantial improvement of 3% to the Few mAP when combined with ASL. A similar trend is observed on JUS-WIDE-SCENEE where Few mAP increases by 1.7%, 1.8%, and 1.9% when MORE is applied to BCE, Focal, and ASL, respectively, without sacrificing performance on the Many classes. is worth noting that in some cases, Medium classes may perform worse than Few classes. Similar observations have been made in Zhou et al. [2023b], Xia et al. [2023]. This could be attributed to the varying levels of intrinsic difficulty between classes
Finetuning the pretrained CLIP model. We further evaluate the effectiveness of MORE by finetuning a pretrained CLIP on the above multi-label datasets. With the powerful pre-trained model. we observe that MORE continues to provide consistent improvements across datasets, with more pronounced gains on Medium and Few classes. As detailed in Table 2, MORE's application to the MIML dataset enhances BCE, Focal, and ASL baselines, increasing overall mAp by 1.7%, 1.8%, and 1.8%, respectively. On the more challenging PASCAL-VOC dataset, MORE combined with ASL achieves a notable 8.3% improvement for the Few classes, accompanied by 3.2% gain in overall mAP. Similar trends are observed on NUS-WIDE WIDE-SCENE, where MORE enhances the performance of Few classes by up to 3.5%, along with a 1.5% increase in overall mAP. On the MS-COCO dataset, MORE also yields substantial gains, improving Few classes performance by 3.2% and overall mAP by 2.9%. These results indicate the compatibility of MORE with finetuning foundation models.
Training overhead analysis. We conducted additional experiments on the CIFAR-100-LT with the imbalance factor of 10 on single NVIDIA 3090 GPU over 200 epochs. The results are shown in Table which indicates that the overhead of MORE is relatively tolerable. In our implementation. we apply low-rank decomposition to all convolutional layers of the ResNet backbones, with other layers remaining unchanged. For CLIP-based models, we freeze the text encoder and fine-tune only the vision encoder and decompose all convolutional layers within it. We commonly use rank r = 0.1. For parameter comparisons during training: on a ResNer-34 backbone, the learnable parameters are approximately 25.4M with MORE and 21.3M without. Similarly, for ResNet-50, the counts are about 29.4M with MORE and 25.6M without. This modest training overhead stems from the low-rank parameters and is relatively contained.
7

---

Table 2: mAP (%) performance (1) for Many, Medium, Few, and overall classes. Experimental evaluations conducted across four benchmarks for multi-labe image recognition. Results presented for two training paradigms: training from scratch and finetuning pretrained CLIP, combining with different baseline loss functions.
From Scratch
Finetuning Pretrained CLIP
Dataset
Split
Focal
ASL
BCE
Focal
ASL
MORE
MORE
MORE
MORE
MORE
MORE
Many
85.1
88.8
84.1
84.9
88.4
96.3
96.4
95.8
96.6
96.4
Medium
77.9
81.2
78.1
82.7
79.5
82.5
90.8
93.0
91.4
93.2
91.9
93.7
MIML
Few
85.3
88.2
86.2
88.3
85.8
89.2
93.2
95.0
92.8
95.6
92.5
95.9
All
80.8
84.6
80.9
81.8
85.0
92.4
92.6
94.4
92.9
Many
68.6
69.7
68.1
69.9
69.9
69.1
86.9
87.1
86.9
87.4
87.3
88.9
PASCAL-
Medium
57.6
59.6
57.7
60.2
59.0
60.0
84.0
84.2
84.5
84.8
85.1
87.1
VOC
Few
$2.6
55.0
53.6
54.1
52.9
55.8
81.4
84.5
83.6
89.1
82.2
90.5
All
58.8
60.7
59.0
60.9
59.9
61.0
84.1
84.9
84.8
86.5
84.9
88.1
Many
77.8
78.2
77.6
78.1
78.0
78.3
75.1
73.6
74.3
74.9
75.6
74.8
NUS-
Medium
47.6
48.8
47.4
48.7
48.5
49.3
44.7
44.0
45.0
45.0
44.4
46.0
WIDE-
Few
39.5
41.2
40.2
42.0
40.7
42.6
32.5
36.5
33.0
39.4
35.2
38.7
SCENE
All
54.3
55.4
$4.4
55.6
55.1
56.1
50.2
50.7
50.2
52.3
51.0
52.5
Many
64.2
65.1
64.5
65.2
64.9
65.2
52.9
50.4
52.1
50.4
49.0
51.7
MS.
Medium
60.5
61.8
61.3
62.0
61.5
62.3
53.8
55.4
54.2
56.5
54.9
57.9
COCO
Few
26.9
27.9
27.3
27.7
27.2
28.0
23.7
26.8
23.7
27.4
25.5
28.7
All
57.9
59.1
58.6
59.3
58.8
89.6
51.1
52.5
51.3
53.4
51.9
54.8
Table 3: Training overhead analysis. Experiments are conducted on CIFAR 100-LT with the imbal- ance factor of 10 on a single NVIDIA 3090 GPU.
Table 4: Analysis of (a; 09 @ 0*) f 09) across samples from Many, Medium, Few classes in the final trained models various datasets.
Method
Training Time (Minutes)
Dataset
Head
Medium
Few
LA
14
MIML
0.016
0.018
0.020
LA w/ MORE BCL
21
MIML w/ CLIP
0.064
0.083
0.091
49
VOC
0.062
0.067
0.071
ProCo
64
VOC w/ CLIP
0.026
0.028
0.036
Differential impact of tail-specific parameters logits. To gain a deeper understanding of
mechanism, we analyze the absolute logit difference, [yy( 6 (00))='- fa (a:; (*)], gauging the impact of the tail-speeific parameters 0¹, Table shows consistent trend across all datasets: the difference is smallest for Head classes and largest for Few classes. This indicates that 0 responds more strongly tail-class samples, amplifying their representations. The key insight this relative ordering across groups, which reflects a differential effect on the logits. This pattern confirms our method effectively boosts logits for underrepresented classes.
4.3 Ablation Study
We perform additional ablation studies on crucial aspects of our proposed MORE: its key compo- nents and hyperparameter choices, including the reweighting schedule (Eq. (5)), peak amplitude A (Fig. 2(c)), rank (Fig. 2(d)), the discrepancy metric (Eq. (2), Fig. 3(b), and Fig. 3(a)), and LmOre as whole (Eq. (6), Fig. 3(c), and Fig. 3(d)). We validate MORE's robustness across different image resolutions (Fig. 2(a) and Fig. 2(b)). Moreover, we analyze the loss landscape (Table 6) to provide further evidence for the model rebalancing achieved.
Ablation on different reweighting schedules (Eq. (5)). To assess the impact of different weighting schedules on the rebalancing loss, we conduct ablation experiments using three distinct methods for the coefficient (<(T): constant setting, cosine-based decay schedule (<(T) = A cos(TT/277) that gradually reduces the influence of the rebalancing loss, and sinusoidal schedule as defined in Eq. (5). These methods reflect different assumptions regarding the optimal timing and intensity of
8

---

Table 5: Top-1 accuracy (%) (f) results on single-label dataset and mAP (%) (1) results on multi-label datasets with different weighting schedules on a(T). Experiments conducted on single-label dataset (CIFAR-100-LT) and multi-label datasets, comparing different method combinations. Const. denotes constant weighting, cos denotes cosine-based weighting, and sin denotes sinusoidal weighting.
Multi-Class Dataset
Multi-Label Dataset
Method
a(r)
IF=10
IF=100
Method
a())
MIML
vOC
NUS
COCO
const.
63.9
50.7
const.
82.3
59.3
54.9
58.2
LA+MORE
64.0
50.9
BCE+MORE
84.0
59.6
55.2
58.4
sin
64.8
51.2
sin
84.6
60.7
55.4
59.1
const.
65.2
52.1
const.
83.1
60.4
55.5
59.1
ProCo+MORE
cos
65.4
52.4
ASLAMORE
COs
83.8
60.8
55.7
59.3
sin
65.9
52.9
sin
85.0
61.0
56.1
59.6
IAA (N
810
(a) MIMI.
(b) NUS.
(c) A.
(d) Rank.
Figure 2: (a, b) MAP (%) (1) at 112x 112 and 224 resolutions, on MIML and
SCENE, respectively. (c) The impact of the peak amplitude A in MORE. (d) The impact of the rank r in MORE. In (c) and (d), experiments are conducted on MIML combined with BCE.
supervision from the rebalancing loss. We apply this ablation across both single-label and multi-label recognition settings. In Table 5, the sinusoidal method consistently improves performance over the constant and cosine-based schedules in both single-label and multi-label settings, showing benefits across all datasets. These results validate that modulating the strength of rebalancing loss over time via sinusoidal scheduling allows the model to better balance generalization.
peak amplitude A and rank r. We analyze the influence of the peak amplitude A and rank the r in MORE, as illustrated in Fig. 2(c) and Fig. 2(d), respectively. Due to the weight normal- ization in Eq. (3), A exhibits sensitivity to C, thus report the normalized amplitude A' A/C for clarity. Experimental results demonstrate that MORE consistently yields performance improve- ments across broad spectrum of A' values, with optimal performance observed approximately 2.0. Similarly, MORE maintains robust improvement across various rank values r, achieving peak performance approximately 0.1. Notably, the baseline mAP remains below 83%.
Ablation
Ablation on the discrepancy metric (Eq. (2)). We conduct experiments to evaluate different methods for measuring distributional divergence. In our approach, we use ε distance measure the discrepancy between the distributions of f(0 and (@9). For this purpose, KL divergence is also common metric for this purpose. We compare the performance of both KL divergence and l distance, as shown in Fig. 3(b) and Fig. 3(a). The results demonstrate that while KL divergence yields some improvement over the baseline, & distance leads to significantly better results. This indicates that & distance is a more effective measure of distributional differences in our method.
Ablation on LMORE as a whole (in Eq. (6)). In Fig. 3(c) and Fig. 3(d), we compare our proposed MORE with the baseline method BCE, and MORE w/o LMORE on VOC and COCO. Evidently, MORE w/o LMORE performs comparably to the bascline. However, MORE (incorporating LMOre) markedly improves overall performance, with notable gains on Few classes. This indicates that the LoRA-like parameter decomposition (Eq. (I)) alone does not effeectively alleviate class imbalance; effective mitigation is only achieved when this decomposition is combined with LMORE to realize model space rebalancing.
Robustness across input resolutions. We evaluate MORE under resolutions of 112x 112 and 224x 224 on MIML and SUS-WIDE-SSEEE datasets. As shown in Fig. 2(a) and Fig. 2(b), MORE consistently improves performance across both resolutions and with both BCE and ASI. losses, demonstrating robustness to different resolutions.

---

w/ CLIP
(b) w/o CLIP.
(c) VOC.
(d) COCO.
Figure 3: (a.b) mAP (%) (t) using KL divergence and & distance (MORE) as the discrepancy measure, with and without CLIP, respectively. (c,d) mAP (%) (1) of the baseline model, MORE w/o MOREE compared to MORE, on Pascal-VOC and MS-COCO, respectively.
Table 6: Loss landscape metrics across different methods on MIML. We report four class-wise imbalance indicators: Imb.Amia (4), Imb.Amax (4), Imb. Tr (L), and Imb.y (4), computed the ratio between the largest and smallest absolute values of each Hessian-basee metric across classes. Lower values indicate more balanced curvature across classes. X (1) and To (J) represent Xmin and y for the class with the fewest samples (class 0), where higher Amin and lower y suggest a flatter landscape and reduced tendency to converge to saddle points.
Method
CLIP
Imb.Amar Imb.Amin
Imb. Tr
Imb.y
(0)
"Nu
BCE
6,837
53.48
182.2
24.67
-2181
0.1896
BCE w/ MORE
V
5.193
8.303
11.33
1.612
-59..3
0.0137
BCE
/
13.30
21.90
23.51
2.832
-3168
0.1257
BCE w/ MORE
2.175
1.671
1.436
1.461
-1.932
0.0200
Flat minima of loss landscape. In imbalanced learning, minority classes tend converge saddle points the loss landscape, which often leads to poor generalization [Dauphin et al., 2014, Zhou al., 2023a]. Following [Rangwani al., 2022]. we focus on four metries derived from the Hessian matrix: the minimum eigenvalue Ami. the maximum eigenvalue Amax. the eigenvalue ratio 7, and the trace of the Hessian Tr. These four metries could indicate the sharpest directions of curvature and the overall sharpness of the landscape for each class. A low value of Xunin and a large value of indicate non-conved region and empirically suggest convergence to saddle point, where optimization is less stable and generalization is typically weaker. To measure the degree of imbalance in the curvature across different classes, we use four class-wise imbalance indicators: Imb.Amin- Imb.Amaz: Imb. Tr, and Imb.7, where each is computed as the ratio between the largest and smallest absolute values of respective metric across all classes. Table demonstrates that our method substantially mitigates curvature imbalances across metrics. When combined with the MORE method, all four imbalance factors show significant reductions. Additionally, for the class with the fewest samples (class 0), the combination with MORE leads to noticeable optimizations in both Amin and y. These results indicate that our method not only smoothens the loss landscapes but also effectively balances the per-class curvature, particularly for underrepresented classes, thereby enhancing generalization performance. For more ablation studies, please refer to Appendix C.2.
5 Conclusion
In this work, we have introduced MOdel REbalancing (MORE), a novel method for addressing class imbalance by rebalancing the model space. By decomposing model parameters into main and low-rank components, MORE explicitly enhances the representation of underrepresented tail classes through tailored loss formulation and sinusoidal reweighting approach. This approach ensures efficient and balanced learning dynamics without increasing model complexity or inference overhead. Extensive experiments across diverse long-tailed benchmarks, including both multi-class and lulti-label tasks, demonstrate that MORE consistently improves performance, particularly for tail classes, and effectively integrates with existing imbalance mitigation techniques. Future work will extend MORE to other imbalanced learning scenarios, such as few-shot learning and domain adaptation, to further enhance its applicability and robustness. Further refinements can focus on mitigating representational trade-offs for semantically disparate tail classes via semantic grouping. Moreover, a stricter decoupling of general and tail-specific parameters, such as through orthogonality constraints, also offers promising direction to enhance model modularity and reduce redundancy.
10

---

Acknowledgement
Jiaan Luo, Feng Hong, Qiang Hu and Jiangchao Yao are supported by the National Key R&D Program of China (No. 60702,, National Natural Science Foundation of China (No. (2306178) and STCSM (No. 22DDZ2229005), 111 plan (No. BP0719010), Xiaofeng Cao is supported by National Natural Science Foundation of China (No. 62476109, No. 62206 (08). Feng Liu is supported by the ARC with grant number DE240101089, LP240100101 DP230101540 and the NSF&CSIRO Responsible Al program with grant number 2303037.
References
Emanuel Sanchez Aimar, Arvi Jonnarth, Michael Felsberg, and Marco Kuhlmann. Balanced product of calibrated experts for long-tailed recognition. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 19967- 19977. IEEE, 2023. doi: N0.11000CPPS577222220000000 httptt htt.ddo.oor/...1.///// CVPR52729 2023.01912.
Reza Yazdani Aminabadi, Samyam Rajbhandari,, Ammar Ahmad Awan, Cheng Li, Du Li. Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, and Yuxiong He. Deepspeed- inference: Enabling efficient inference of transformer models unprecedented scale. In Felix Wolf, Sameer Shende, Candace Culhane, Sadaf R. Alam, and Heike Jagode, editors, SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13-18, 2022, pages 46:1-46:15 IEEE, 2022. doi: 10.1109/SC41400 404,24400042222000555 URL https://doi. org/ 10. 1109/SC41404.1... 2.0.22000550
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Aréchiga, and Tengyu Ma. Learning imbalanced datasets with labelldistribution--ave margin loss. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Allhé-BBuc Emily B. Fox, and Roman Garnett, editors, Ad- vances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 1565-1576, 2019. URL https: petttss:/ppooett http://ppoeeeenn...ee//// www.enng.....uppee www.neuuriii..m weeuriss.ccppaprr////.. wwcpappee/009//aaaeeeeeeeeeeeeee 02446111880aafffarr0044442222220 AbC html.
Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. SMOTE: synthetic minority over-sampling technique. Artif. Intell. Res., 16:321-357, 2002. doi: 10. 10.1613//AIR.953,
URL https: htt:///ti.oor/////////
1613/jair .933.
Jiahao Chen, Bin Qin, Jiangmeng Li, Hao Chen, and Bing Su. Rethinking the bias of foundation model under long-tailed distribution. CoRR, abs/2501. 1..55011.11995. 5955, 2025. doi: 10.48550/ARXIV.250.111 org/10. 18550/arXivy 2501 15955.
URL https:
Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recog- nition with graph convolutional networks. In IEEE Conference Computer Vision and Pattern Recognition, CVPR 2019, Long Beach. CA, USA, June 16-20. 2019, pages 5177-5186. Computer Vision Foundation / IEEE, 2019. doi: 10.1109/CVPR.20199000 URL http:// thecvf. com/ With With_Graphhhhhttttt Caraph_CCovolutiin L_____ettoesse_ cvcpee____________ Cwwpe_/11____peeeeeeee html.
Tat-Seng Chua, Jinhui Tang, Richang Hong. Haojie Li, Zhiping Luo, and Yantao Zheng. NUD::IDE: real-world web image database from national university of singapore. In Stéphane Marchand- Maillet and Yiannis Kompatsiaris, editors, Proceedings of the 8th ACM International Conference on Image and Video Retrieval, CIVR 2009, Santorini Island, Greece, July 8-10, 2009. ACM, 2009. org/10 10. 1111//1444300
doi: 10. 0..1145/166633 1646452. URL https: : htt:://dii.rg//////
1646452.
Jiequan Cui, Zhisheng Zhong, Zhuotao Tian, Shu Liu, Bei Yu, and Jiaya Jia. Generalized parametric contrastive learning. IEEE Trans. Pattern Anal. Mach. Intell., 46( (6(12):7463-7477 2024. doi: 10. (000///AAM.22223327866660
URL https: http://oi..ooo wtw.:...rgg/.... 11t..////....../P.//...... 2023 3278694.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 9279,2277) 2019.

---

Tianjie Dai, Ruipeng Zhang, Feng Hong, Jiangchao Yao. Ya Zhang, and Yanfeng Wang. Unichest: -onquer-and-divie pre-training for multi-source chest x-ray classification. IEEE Trans. Medical Imaging, 43(8):2901-2911 2024.
Yann N. Dauphin, Razvan Pascanu, Çaglar Gülçehre, KyungHyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger, editors, Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2933-2941, 2014. httphtttps///proeeeeeee 2014/hash/1n htli//nnsh/e22250ddde233332 wttaaa//000000000000000000003330 Absttabbbbbtrrcccc httbbsttrat..tl........
neurips cc/ paper/
Alakh Desai, Tz- Ying Wu, Subarna Tripathi, and Nuno Vasconcelos. Learning of visual relations: The devil is in the tails. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada. October 10-17, 2021, pages 15384-15393, IEEE, 2021. doi: 10. 1100ICCV4892.2002.011222
https::: /toi//oiio/// org/ 10 www.//9IIICVVV992. 2021 01512.
Chaoqun Du, Yulin Wang, Shiji Song, and Gao Huang. Probabilistic contrastive learning for long- tailed visual recognition. IEFE Truns. Pattern Anal. Mach. Intell., 46(9):5890-59900 2024. doi: 10.1
102. URL https: http:::/ddooooor/// org/ 1109/TPAMI 20244 33691022.
Mark Everingham, Lue Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis., 88(2):033 2010. doi: 10.1007/S 12111-106----00 0133-009-027--4 URL https: / htt:://oi..oo org/ 10. 1007 ht////112231-11 -27--275-4
Gregory Holste, Yiliang Zhou, Song Wang, Ajay Jaiswal, Mingquan Lin, Sherry Zhuge, Yuzhe Yang, Dongkyun Kim, Trong-Hieu Nguyen Mau, Minh-Triet Tran, Jaehyup Jeong. Wongi Park, Jongbin Ryu, Feng Hong, Arsh Verma, Yosuke Yamagishi, Changhyun Kim, Hyeryeong Seo, Myungjoo Kang, Leo Anthony Celi, Zhiyong Lu, Ronald M. Summers, George Shih, Zhangyang Wang, and Yifan Peng. Towards long-tailed, multi-label discase elassification from chest x-ray: Overview of the CXR-LT challenge. Medical Image Anal., 97:103224, 2024.
Feng Hong, Jiangchao Yao, Zhihan Zhou, Ya Zhang, and Yanfeng Wang. Long-tailed partial label learning via dynamic rebalancing. In The Eleventh International Conference on Learning Representationss ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL ittps://openreviiee
Feng Hong. Jiangchao Yao, Yueming Lyu, Zhihan Zhou, Ivor W. Tsang, Ya Zhang, and Yanfeng Wang. On harmonizing implicit subpopulations. In ICLR. 2024.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Interna- tional Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL
Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zhang, and Bo Han. Detecting didto--diitribution data through in-distribution class prior. In ICML, 2023.
Yan Jin, Mengke Li, Yang Lu, Yiu-ming Cheung, and Hanzi Wang. Long-tailed visual recognition via self-heterogeneous integration with knowledge excavation. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 23695-23704, 2023.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020a. URL https: /ttttp://oopnreviieee
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo. Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In 8th International Conference on Learning Representattinss ICLR 2020, Addis Abaha, Ethiopia, April 26-30, 2020. OpenReview.net, 2020b. URL https:// wwt httt/forummid-iee uede/forrrmmd--.g
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
12

---

Mengke Li, Yiu-ming Cheung, and Yang Lu. Long- taigettaied visual recognition via gaussian clouded logit adjustment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6939,, 2022.
Mengke Li, Zhikai Hu, Yang Lu, Weichao Lan, Yiu-ming Cheung, and Hui Huang. Feature fusion from head to tail for long-tailed visual recognition. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Eighth AAAI Conference Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium Educational Advances in Artificial Intelligence, FAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 13581-1 255811 AAAI Press, 2024a. doi: 00.1609/AAALV38122200 URL https:: pttps://doi.. org/ 10. 1609/aaai...../a/..a..aa..a....t v38i12. 29262.
Mengke Li, Ye Liu, Yang Lu, Yiqun Zhang. Yiu-Ming Cheung, and Hui Huang. Improving visual prompt tuning by gaussian neighborhood minimization for long-tailed visual recognition. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024b. URL http: http::::/ppppp.eeeeee nips. hccpaper_files. Abett Conf weeeeeeeeeeeaaeee
Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision - ECCV 2014 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V, volume 8693 of Lecture Notes in Computer Science, pages 740-755. Springer, 2014. doi: 10. 978-3-319- 10602- 48. URL https::///o/ org/ 1007 9978/9778333 319- 10602- 1_48.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pages 2980-2988, 2017.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large- long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2537-2546, 2019.
Jiaan Luo, Feng Hong, Jiangchao Yao, Bo Han, Ya Zhang, and Yanfeng Wang. Revive re- weighting in imbalanced learning by density ratio estimation. In Amir Globersons. Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neu- ral Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, Decem- ber 10 15. 2024, 2024. URL http: http:://aapppeeeeeee nips. cc/ hacpppppr_ffte faaeerrfflls//eeeeeeeeeeeeeeeele htteee//pppee/2//////e// 92440ec643f4eSt (tte://9111070999776616165551 -tt.////oee.66----aaaaaaaoooo-o- Abstract-
.ttll.
Yanbiao Ma, Licheng Jiao, Fang Liu, Yuxin Li, Shuyuan Yang, and Xu Liu. Delving into semantic scale imbalance. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https: sttpss//oppenrviiee net/ forumiid=07tc52KRI.O
Aditya Krishna Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla. On the statistical consistency of algorithms for binary classification under class imbalance. In Proceedings of the 30th International Conference on Machine Learning, ICMI. 2013, Atlania, GA, USA, 16-21 June 2013, volume 28 of JMLR Workshop and Conference Proceedings, pages 603-611. JMLR.org, 2013. URL http: / http::///occeeeneeeee mlr. www.ppressvveeee /tt/////////nnnnneee www////nnnnnnnneemmmmem/o.aa.aem html.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In 9th International Conference on Learning Representattinns ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Marina Mcila and Tong
13

---

Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021. Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748-8888888 8763.876..78 PMLR, 2021. URt.ttpp:/ wtt.hhtt::/ppcceeeiiiiiee html.
mlr.
Harsh Rangwani, Sumukh K. Aithal, Mayank Mishra, and Venkatesh Babu R. Escaping sad- dle points for effective generalization class-imbalances data. In Sanmi Koyejo. S. Mo- hamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems 35: Anmual Conference Neural Information Pro- cessing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 Decem- wwtppccccppppeppeeeeeeeeeee paapprrapleeeeeeeeeeppppppppeeee saaper_fllespppperreee
ber 9, 2022, 2022. URL http://papers: Abstract-
wtteee/pppeee///////aaeeee html.
Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Balanced meta-softmax for long-tailed visual recognition. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL https: pproceeedgo/ccengss.. neuriip.. 130i12222211222222233323233
cc/paper/2022
html.
Tal Ridnik, Emanuel Ben Baruch, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. Asymmetric loss for multi-label classification. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 82-91. IEEE, 2021. doi: 10.1 11111//C4449222222.000000 ICCV48922. 2021 00015.
1/tp//..11000 URL https://
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. Ins. J. Comput. Vis., 115(3):21 2015. doi: 10.1007/S1 263-015-0816-Y Y. URL httph:tt///// //tp:.///oooooooo org/ 10. 1007 s 111-226630118-----111
Jiing-Xin Shi, Tong Wei, Yuke Xiang, and Yufeng Li. How re-sampling helps for long-tail learn- ing? In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Anmual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, Decem- nips. cc/ sece/pppr_fffeee farrrffiels//eeee www.f//pppeeee/22//... aaper/2023//ass
ber 10 16, 2023, 2023, URL - Conf erence @effaT0bcbbd43f6bdd00 www.eeeedddeeeedeeeeeee3e33 Abstract-
.ttml.
Jiang-Xin Shi, Tong Wei, Zhi Zhou, Jie-Jing Shao, Xin-Yan Han, and Yufeng Li. Long-tail learning with foundation model: Heavy fine-tuning hurts. In Forty-first International Conference on Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL wwt.:nee//ooooe www.///ww.eetoouumiieeeeee http://rrmmiddccssttaa22
itttss://openrevier:
Ximeng Sun, Ping Hu, and Kate Saenko. Dualcoop: Fast adaptation multi-label recognition with limited annotations. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and
A. Oh, editors, Advances Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA. USA, November 28 December 2022, 2022. URL http: http:// http//aaappppeeeeeeeeeeeeeeeee. mips. wtcppappr_fiiiie www.ffflllsspplee patees/pppr//222/ html.
Adane Nega Tarekegn, Mario Giacobini, and Krzysztof Michalak. A review of methods for imbal- anced lubti-label classification. Pattern Recognit., 118:107965, 2021. doi: 10.1016/J.PATCOO 2021.107965. URL https: //woi. org/ 1016/j patcog. 2021 107965.
Eugene Vorontsov, Chiheb Trabelsi, Samuel Kadoury, and Chris Pal. On orthogonality and learning recurrent networks with long term dependencies. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW Australia, 6-11 August 2017, volume 70 of Proceedings Machine Learning Research, pages 3570- 3578. PMLR, 2017. URL /ttt:://tp//eeeee http:///ooceeeeneeeeeee mlr. press//70/vorontsor7 http//oooooooooeeeee html.
Jianfeng Wang, Thomas Lukasiewicz, Xiaolin Hu, Jianfei Cai, and Zhenghua Xu. RSG: simple but effective module for learning imbalanced datasets. In IEEE Conference on Computer Vision
14

---

and Pattern Recognition, CVPR 2021, virtual, Jume 19-25, 2021, pages 3784-3793. Computer
Vision Foundation IEEE, 2021a. doi: 10.1109/CVV444443.2210..003 // /pppaaccess. thecvf. com/ content/CVPR2022 /ttmlll222//mmll httmm//aam////./.m
URL https:
_aanng_s______ RSG_A. L_SSimlle____________eee but_
2PPPP__2221____ 2021__aperrrr html.
Jiang Wang, Yi Yang. Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. CNN-RNN: A
unified framework for multi-label image classification. In 2016 IEEE Conference on Computer Vision and Pattem Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2285- 2294. IEEE Computer Society, 2016. doi: 10. 10.1109/CVPR.20162255. URL https: //doi: org/ 10. 1111/19//CC CVPR. 2016.022222 251
Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X. Yu. Long-tailed recognition by routing diverse distribution-wware experts. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021b. URL https://openreevier net
Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. Openauc: Towards auc-oriented open-set recognition. Advances in Neural Information Processing Systems, 2022.
Zitai Wang, Qianqian Xu, Zhiyong Yang. Yuan He, Xiaochun Cao, and Qingming Huang, A uni- fied generalization analysis of re-weighting and logit-adjustment for imbalanced learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference Neu- ral Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, Decem- ber 10 16, 2023, 2023. URL http://paperss mips cc http/ppper__fff files/ www.fffelleleeaeeeeeeeee httppppppe/////a//////// html.
973a0f50443cf991 htt.rr99111cccaaeeeeeeeceeeee
Zitai Wang, Qianqian Xu, Zhiyong Yang. Zhikang Xu, Linchao Zhang, Xiaochun Cao, and Qingming Huang. A unified perspective for loss- imbalanced learning via localization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. doi: 10. 1091 3609440.
Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, and Dahua Lin. Distribution--alanne loss for multi-label classification in long-tailed datasets. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IV, volume 12349 of Lecture Notes in Computer Science, pages 162-178. Springer, 2020. doi: 10. 10.1007/97333-000-555------ https: /doi: http://ooo.ooo///////// 10. 1007 1-97788-030-855-44 8_10.
URL
Peng Xia, Di Xu, Lie Ju, Ming Hu, Jun Chen, and Zongyuan Ge. LMPT: prompt tuning with class- specific embedding loss for long-tailed multi-label visual recognition. CoRR, (bs/2305500436, 2023. 2305. 04536.
doi: 10.44850/AAXXI.23055. 044566............ https:/s:://do http:/::///o...///////////////// 00ee/10.485550 A88500/arXiv
Yan Yan, Ying Wang, Wenchao Gao, Bo-Wen Zhang, Chun Yang, and Xu-Cheng Yin. Lstm² Multi-label ranking for document classification. Neural Process. Lett., 47(1):117-138, 2018. doi: 10.1007/511063-01--96666
URL https://doi. http://i....rr////// 1007 100.1100000111110000001011
Zhiyong Yang, Qianqian Xu, Shilong Bao, Xiaochun Cao. and Qingming Huang. Learning with multiclass auc: Theory and algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. Feature transfer learning for face recognition with under-represented data. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beuch, CA, USA, June 16-20, 2019, pages 5704-5713, Computer Vision Foundation IEEE, 2019. doi: 10. 10.1/99///PR 2019.000.2099008... 9,05855 URL http:// nttt://openacceess thecvf com/ content wwwoonttttcvr______eeeee http/22999hhmmmeee /tttttmmm///iiiiieeeei Seriiineeuuuree.e Transsfrrrrrr_ Learning forrning_foo__i Faceerfoooeraera__eeeeeeeeeeeeee Faceeaacceeaaeeeeeeeeeeee Recognitcctiiiioon html.
Represented_Dataa aatpaa___a___aa___ pppr
Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep long-tailed learning: survey. IEEE Trans. Pattern Anal. Mach. Intell., 10755-1679- 2023. doi: 1091 TPAML.20333268115 18. URL https: org//0//////// 1111/99/AAMTT 2022 3268118.
15

---

Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long- tailed recognition. In IEEE Conference on Computer Vision and Pattern Recog- nition, CVPR 2021, virtual, June 19-25, sion Foundation / IEFF, 2021. doi:
2021, pages 16489-16498. Computer Vi- URL.
10. 10.1 1.110//CVPR-444722222...4 101622.
https://openaaccess thecvf Colibration_for_Longg htwpeeeeeeeeggnttooooooooo.oe..
http://hoonnnnpriininnngo Improving- html.
Yixuan Zhou, Yi Qu, Xing Xu, and Hengtao Shen. Imbsam: A closer look at sharpness-awarr mini- mization in Blass-ibbllance recognition. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 11311-11321. IEEE, 2023a. doi: 10. 2023 01042.
(109//CCV5117000223..1111
Utthttpss//doi org/
Zhi-Hua Zhou and Min-Ling Zhang. Multi-instance multi-label learning with application to scene classification. In Bernhard Schölkopf. John C. Platt, and Thomas Hofmann, editors, Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pages 1609-1616. MIT Press, 2006. /ttLtttts//pprceeeeeee httpaa//ees..eeeeeeeeeeeeeeeee.. -ww..eeeeeebbbaaattttt.oaa html.
Zhihan Zhou, Jiangchao Yao, Feng Hong, Ya Zhang, Bo Han, and Yanfeng Wang. Com- bating representation learning disparity with geometric harmonization. In Alice Oh, Tris- tan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b. URL http: /ttp::////ppeaaaee wwwppppprs.pp.....ppoopppppspssm nips cc/ waarr/fleeeeappee/ee2 -trtraaa-aaat27222--aat--------a Conference html.
Zi-Hao Zhou, Siyuan Fang, Zi-Jing Zhou, Tong Wei, Yuanyu Wan, and Min-Ling Zhang. Con- tinuous contrastive learning for long-tailed semi-supervised recognition. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Anmual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, Decem- ber 10 15, 2024, 2024. URL http:: mippperrrr.iis...omee.e. cc/ hacc//ppe__feeeeeeeeeeee filerrffille/eeeeeeeeeeeeee wwweeeespppre/2//////// Sc1170c249cd8e hat.oo220ccceeebeeeeeaaeaee.ee. Awtraaa--tttttttttataaaae Conferenccoo .ttml.
Jianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen, and Yu-Gang Jiang. Balanced contrastive learning for long-tailed visual recognition. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 6898-6908 IEEE, 2022. doi: 10.1109/CVPR52688 202.000000.222077757 URL https: //t:::///do/// org/10. 10. 1109/CVPR52688. 2002.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All statements in the abstract and introduction are aligned with the main contribution of the paper. All claims are supported either by rigorous theoretical results or extensive experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiaatinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [Yes]
Justification:
See Section 3.3 and Section B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 4.1 and Section 4.2.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The datasets used in this work are publicly accessible. The code of this paper will be released after anonymized review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/ Squidess/CoeSubbrgg
olicyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ ctc/public/gutii Seeiiides/oooSddii issiisiinnn wwwrbbmmsssiooooososooooo onnoliccy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section 4.1 and Section 4.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: See Table 10.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: Yes]
Justification: See Section 4.1 and Section 4.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te:r//s....eeeeeeeeeeeeee cc
Answer: [Yes]
Justification: This paper is conducted with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section E.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All code, data, and models used are properly cited. The license, copyright, and terms of use carefully followed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Thorough documentation, including instructions, detailed comments, and examples will be provied when the code is public.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
22

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpr///nnuu../eeeeee tteuurip..cc/ooffeeeeeeee.. htt//onfeeeenee222222222 /LLM) for what should or should be described.
23

---

A Algorithm
We summarize the pseudo-code of MORE to demonstrate the procedure of implementing our method in detail, as shown in Algorithm 1.
Algorithm 1 Algorithm of MORE
Initialize: 8 (WT. W W W ) and 0 (W], 2 Whd
for
to do Sample mini batch B <
Calculate Lhase via (a;; 09 + 0*) and v), E B
Calculate LMORE via Eq. (3)
Calculate a()) via Eq. (5)
Take gradient descent Vgs,e ,r (Crase + a(T) Optional: anneal the learning rate with T
end for
B Theoretical Supplement
In this section, we present a formal proof for Theorem 1. imbalanced learning, the effectiveness of model is typically evaluated based on its ability to minimize the balanced risk, which averages the risk across all classes mitigate the impact of class imbalances. For any f E F, the balanced risk defined as:
Real (S) - Σ E BB E,,,,,(((())),))
where C' denotes the number of classes, Dy represents the data distribution for class y, loss function. Let Go (Lo o fo : fo Fo} denote the hypothesis space of bascline model, and g= fLo f denote the hypothesis space of our proposed model.
Lemma 1. Following Wang et al. [.023]. any f E F, the balanced risk can be bounded by the following inequality:
C ( (
1 3 C
Rs, (9) +
(8)
where Rf) denotes the empirical risk, R.s, (9) denotes class-specific Rademacher complexity, and e is a confidence term. The class-specific Rademacher complexity is defined as:
(s) = Eo
sup BEG
1 N inn-y
oggat))
where o, E (+1,-1) are independent random variables.
To prove Theorem 1, we introduce the following necessary assumptions that are formally required in the proof:
Assumption The hypothesis space G sufficient to fit the data distribution on dataset S. Formally, there exist 9opt = {91. I
.....9 }.9: a subspace Gsub S sufficient to fit This common assumption, ensuring that has adequate expressive power.
E G and gs that are optimal to fit D, so that there exists
Assumption 2. Through parameter decomposition, the hypothesis space is partitioned as G Gmaj U Grew where Gres is constrained by a low-rank parameter т, as shown in Fig. 4(a). For an appropriate choice of r. Gaub C 9muj C G is sufficient to fit the majority class distribution majj-
Remark 2. The decomposition in Assumption 2 implies that the empirical risk approximately preserved, i.e., (e) (fo),, where f F and fo E Fo, which is generally satisfied by a proper The empirical verification presented in Fig, 4(b) supports this point inherent in Assumption 2.

---

100
0.80
maa
1,08
6.92
Gres
0.40
0.2
6,00
Epoch
(a)
(b)
Figure 4: (a) Decomposition of hypothesis space G. (b) Empirical risk comparison between baseline and MORE. Experiments are conducted on MIML using BCE base loss.
Proof.
The class-specifie Rademacher complexity Eq. (9) quantifies the capacity of g to fit random noise in the samples of each class. In our approach, we decompose Ç based on class roles; for majority classes (y E maj)), the effective hypothesis space Gmaj S G; for minority classes (y E mmi)), the effective hypothesis space is Gmin G. The risk of decomposed Rademacher complexity terms is defined as:
R1 Σ (9mu)) yE
R
Σ VE Y
Rs,(Gmin))
Σ Rs, Ri + R2.
(10)
We now demonstrate that the decomposed hypothesis space effectively reduces the overall risk of Rademacher complexity. For minority classes, Gmin = G. which maintains identical to the baseline hypothesis space Go in expressive power. Thus for any y E Ymins we have:
R.,, (Amin) = Rs, (9) Rs, (Go).
(11)
Summing over all minority classes, we obtain the following:
R₂ = Σ
Rs, (9min) = Σ R.s (9o).
(12)
yEYui
y
For majority classes, Gmuj S G. For any Ymaj- since the supremum is taken over a smaller set, have:
R.S. (Gma)) Ear
sup gESmni
Σ oig(zs)
Enr
sup go Eo
Σ
osgo(sz))
Rs,(Go) (Go).
Ny
N, says y
23 my
(13)
Summing over all majority classes, obtain the following:
R
Σ
Rs, (Gmaj) S Σ Rs, (G)).
(14)
ymy
YEY
By combining Eq. (10), Eq. (12) and Eq. (14), we obtain the reduced risk of Rademacher complexity:
1
1 Σ Rs, (9)
C Σ (G))
(15)
Finally, by combining Eq. (15) and Eq. (8), we derive the tighter bound for the balanced risk under Assumption and Assumption 2, for any f E F and fo E Fu:
R (ff) < RC (fo).
(16)
25

---

Experimental Supplement
C.1 Statistics of Datasets
To assess the effectiveness the proposed approach, we perform an extensive set of experiments across four benchmark multi-label datasets, each with distinct characteristics, outlined in Table 7. These datasets span broad range of complexities, including variations in scale, domain, and class distribution, providing a comprehensive evaluation of the robustness and generalizability of our method. This allows us to examine the performance of our approach in multi-label recognition tasks, particularly in the presence of varying degrees of class imbalance challenge commonly encountered in real-world applications.
Table 7: Statistics of multi-lahel datasets used in our experiments. The datasets exhibit varying levels of complexity in terms of class count, training sample size, and class imbalance. The imbalance factor is calculated as the ratio between the maximum and minimum number of samples per class, with higher values indicating more severe class imbalance.
Dataset
Classes
Samples
Min Sam-
Max Sam- ples/Class
Avg. La- bels/Sample
Imbalance Factor
ples/Class
MIML
3,000
289
441
1.24
1.53
Pascal-VOC 2007
20
5,000
96
2,008
2.5
20.92
NUS-WIDE-SCENE
33
17,500
75
11,995
3.4
159.93
MS-COCO
80
82,800
128
45.174
3.5
352.92
C.2 More Comparison Results
Single-label recognition. Our approach may be conceptually analogous to Mixture-of-Experts (MoE) methods like RIDE [Wang al., 2021b] and BalPoE [Aimar al., 2023], as both paradigms augment model capacity for Few classes, albeit through fundamentally different optimization schemes. We performed a comparative analysis as shown Table 8. The results show that integrating MORE with a strong baseline (ProCo) consistently outperforms MoE methods. Crucially, while MoE methods increase model size and computational cost with expert branches, MORE achieves these improvements with no additional inference-tine parameters, highlighting its efficiency. To further verify the scalability of MORE, we evaluated our method on the large-scale ImageNet-LT dataset with ResNet-50 backbone. As shown in Table 9, MORE provides stable improvements across all partitions, with the most significant gains on Few classes while maintaining performance on many and medium classes. These findings are consistent with our results other datasets (e.g., CIFAR-100-LE Places-LT), confirming that our method scales effectively to larger and more diverse data. Notably, overall gains in certain settings may appear relatively modest. This may be attributed to performance saturation on well well-represented classes, which restricts large improvements in the overall metric and concentrates method's substantial gains on the more challenging Few classes.
Table 8:
Top-1 accuracy (%) (f) for comparison results rized by imbalance factors (IF).
Results catego-
Method
Backbone
Params
IF=10
IF=100
CE
Resnet32
0.57 M
61.4
44.1
CB
Resnet32
0.57 M
62.1
44.7
RIDE (4 experts)
Resnet32
1.04 M
62.5
51.4
BalPoE
Resnet32
1.37 M
65.2
51.9
ProCo
Resnet32
0.57 M
65.0
51.9
+MORE
Resnet32
0.57 M
65.9
52.9
Multi-label recognition. Due to space limitations, the standard deviations for the results presented in Table 2 included in the main text. Results with standard deviations provided in Table 10.
More ablation studies. further analyze our method's contributions, we ablate against two low-rank baselines on the VOC dataset: 1) BCE (09 0²), which pairs our decomposition with
26

---

Table Top-1 accuracy (%) (1) results for Many, Medium, Few and overall classes on ImageNet-LT datasets.
Method
Many
Medium
Few
All
CE
69.6
42.2
14.5
49.0
CB
69.7
42.7
16.7
49.6
LA
63.7
51.9
34.7
54.1
+MORE
65.0
52.6
36.1
55.1
ProCo
66.2
53.9
37.3
56.3
+MORE
66.9
54.8
38.0
57.2
Table 10: mAP (%) performance metrics (f) for overall classes. Experimental evaluations conducted across MIML, PASCAL-VOR AUS-WIDE-SCENE and MS-COCO benchmarks for multi-labed image recognition.
BCE
Focal
ASL
Dataset
MORE
MORE
MORE
MIML
80.8
84.6
80.9
85.010.4
81.810.4
85.0
PASCAL-VE VOC
58.8
60.7.0.3
59.010.4
60.9
59.910.6
61.0
NUS-WIDE-SCENT
54.310.2
55.4
54.410.2
55.610.1
55.1
56.1 10.2
MS-COCO
57.940.7
59.1
58.640.3
59.320.3
58.840.4
59.6
BCE loss, and 2) BCE (LoRA), which fine-tunes with LoRA under BCE, As shown in Table 11, our method substantially outperforms both. This demonstrates that the gains stem from our overall design rather than the low-rank formulation itself. While LoRA only alters update dynamics in fixed-capacity model, our approach rebalances model capacity to specifically counter class imbalance.
Table 11: mAP (%) performance (f) comparison with low-rank variants.
Method
BCE
BCE (09 + 0%)
BCE (LoRA)
BCE (MORE)
Performance (%)
58.8
59.0
58.4
60.7
D
Further Discussions
Beyond unified tail space. Our framework represents all Few classes within a unified low-rank space, which may introduce representational trade-offs when certain classes are semantically disparate. While our parameter decomposition already isolates tail-specific features from the influence of Head classes, future work could further mitigate this intra-tail competition. One promising direction is to partition Few classes into semantically coherent groups (e.g., via clustering) and learn dedicated set of low-rank parameters for each group. A simpler alternative to increase the rank the shared tail space to enhance its expressive capacity.
Decoupling general and tall-specific parameters. The interplay between the general parameters 89, and the tail-specifi parameters 8', presents another promising direction for future investigation. Although guided by different objectives, their joint optimization may still result representational overlap. Enforcing stricter decoupling, for instance via an orthogonality constraint between 09 and 0', could yield more modular model by minimizing this redundancy. Although certain constraints can introduce optimization challenges like impeded convergence [Vorontsov et al., 2017], exploring appropriate regularization approaches remains a valuable direction that could further improve performance Few classes.
27

---

E
Social Impact
Our research for long-tailed recognition has significant socictal implications beyond its technical contributions. By addressing the fundamental challenge of class imbalance in machine learning, our work contributes to more equitable Al systems that can better serve diverse populations and use cases. Long-tailed distributions are ubiquitous in real-world scenarios, particularly in critical domains such as medical diagnosis [Dai et al., 2024, Holste et al., 2024), where rare conditions often receive inadequate representation in training data. By improving model performance minority classes without sacrificing accuracy on majority classes, our approach helps create more reliable and fair AI systems that can recognize and respond appropriately to less common but equally important cases. The parameter space manipulation technique enables more effective learning from imbalanced datasets without requiring additional computational resources during inference. This efficiency particularly valuable for resource-constraine performance across all classes is essential for ethical deployment. By advancing the theoretical understanding of model space allocation in imbalanced learning scenarios and providing practical, efficient implementation, that can better serve the full spectrum of human needs, including those of underrepresented groups whose data may naturally fall into the "long tail" of many real-world distributions.
environments and applications where equitable
work contributes the development of more inclusive Al technologies
F
Limitations, Discussions, and Future Work
Our work introduces MORE as novel approach to long-tailen recognition through model space manipulation, demonstrating strong empirical results across diverse datasets. While effective, we ac- knowledge several limitations and future directions. First, our current static parameter decomposition applies uniformly across layers, whereas dynamic decomposition strategy could adaptively allocate capacity based on layer importance for minority classes. Second, given limited training resources, we have primarily validated MORE on visual recognition tasks; extending our approach to other modalities (text, audio, video) could reveal broader applications of our parameter space manipulation principles. As foundation models continue to grow importance, adapting MORE for extremely large-scale pre-trained models while maintaining parameter efficiency remains an exciting avenue for future exploration.
28


Reviews:
[{"review_id": "85UI1LeGWB", "reviewer": "Reviewer_SoG6", "summary": "This paper proposes MOdel REbalancing (MORE), an approach to long-tailed recognition that rebalances the model’s parameter space instead of data or loss. By decomposing model weights into general and low-rank components, MORE allocates capacity specifically for tail classes. A discrepancy-based loss and sinusoidal reweighting guide this process during training. The method improves generalization for underrepresented classes without increasing inference cost, and achieves strong results across both single- and multi-label long-tailed benchmarks, including CLIP finetuning.", "questions": "1. Can the authors clarify how their method is conceptually and empirically different from existing low-rank adaptation techniques? What evidence shows that this difference is critical in the long-tailed setting?\n\n2. Can the authors evaluate the method on larger and more standard long-tailed benchmarks to better demonstrate its scalability and generalization?\n\n3. Can the authors include comparisons or ablations against alternative capacity-based baselines to isolate the source of performance improvements?\n\n4. How consistent are the improvements across different datasets, and can the authors explain the relatively modest gains observed in some settings?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThe method does not introduce additional inference cost, making it appealing for real-world deployment. Besides, MORE is plug-and-play and can be combined with existing reweighting or contrastive learning methods.\n\n**Weaknesses**\n\n1. Lack of conceptual differentiation from LoRA. Although the authors claim their low-rank decomposition is “fundamentally different” from LoRA, the distinction is not clearly articulated in terms of optimization behavior or empirical ablation. Figure 3 alone is **insufficient** to convincingly demonstrate that LoRA is ineffective in the long-tail setting. Without a sharper contrast, the contribution risks being viewed as an application of existing adaptation techniques to long-tailed learning.\n\n 2. Evaluation breadth. Although the experiments cover a diverse range of tasks, they are limited to relatively small- or medium-scale datasets. Common benchmarks in long-tail learning—such as ImageNet-LT and iNaturalist—are notably absent. This limits the ability to draw conclusions about the method’s scalability and robustness in more realistic or challenging scenarios.\n\n3. The paper does not explore strong baselines that manipulate model capacity, such as mixture-of-experts methods [1, 2] or self-supervised learning approaches [3], which offer alternative solutions to tail-class underfitting. Moreover, using extra parameters is a common and effective strategy—parameter-efficient finetuning approaches like [4] have also shown strong performance in such scenarios. Including comparisons or ablations against these baselines would help clarify whether the performance gains come from the specific design of MORE or from the general benefit of added parameter capacity.\n\n4. Modest overall gains in some settings. On certain benchmarks (e.g., NUS-WIDE, CIFAR100-LT and Places-LT), the overall gains are relatively small , raising questions about trade-offs and consistency across datasets.\n\n[1] Long-tailed Recognition by Routing Diverse Distribution-Aware Experts\n\n[2] Balanced Product of Calibrated Experts for Long-Tailed Recognition\n\n[3] Generalized Parametric Contrastive Learning\n\n[4] Parameter-Efficient Long-Tailed Recognition", "was_revised": true, "final_justification": "Part of my concerns about the theoretical and experimental aspects has been addressed.", "rebuttal": "> Lack of conceptual differentiation from LoRA. Although the authors claim their low-rank decomposition is “fundamentally different” from LoRA, the distinction is not clearly articulated in terms of optimization behavior or empirical ablation. Figure 3 alone is insufficient to convincingly demonstrate that LoRA is ineffective in the long-tail setting. Without a sharper contrast, the contribution risks being viewed as an application of existing adaptation techniques to long-tailed learning.\n> Can the authors clarify how their method is conceptually and empirically different from existing low-rank adaptation techniques? What evidence shows that this difference is critical in the long-tailed setting?\n\nThank you for your comment. To address your concerns, we elaborate on the key differences below:\n\n* **Purpose and Design Motivation**: LoRA targets efficient fine-tuning by adding low-rank updates to frozen weights, aiming to reduce computational cost. Our method, in contrast, is designed specifically to address class imbalance by explicitly decomposing parameters into general and tail-specific components, without relying on a frozen backbone.\n\n* **Optimization Behavior**: LoRA applies uniform optimization across all classes using standard losses (e.g., cross-entropy), which may favor head classes. In contrast, our method incorporates a discrepancy-based loss that prioritizes tail classes, guiding tail-specific parameters to learn tail-class representations and enabling imbalance-aware learning. This positions our contribution as a novel extension of low-rank ideas to imbalance mitigation, rather than a mere application of existing techniques.\n\nTo further support these conceptual differences, we conducted additional ablations on the VOC dataset with BCE, comparing variants grounded in low-rank structures: (1) BCE ($\\theta = \\theta^g + \\theta^t$), which adopts our decomposition strategy but retains the standard BCE loss; and (2) BCE (LoRA), which applies LoRA for incremental fine-tuning of the pre-trained model with BCE. Results show MORE method surpasses these baselines with superior performance. This is because low-rank parameters like LoRA merely change update dynamics within a fixed-size network, without increase model capacity. This indicates that the performance gains of MORE stem not from the form of low-rank decomposition, but rather from its overall design, which enables the rebalancing of model capacity to effectively counter class imbalances.\n\n| Method                               | Pretrained Backbone | Frozen Backbone | Performance (%) |\n| ------------------------------------ | :-----------------: | :-------------: | :-------------: |\n| BCE                                  |          ✘          |        ✘        |      58.8       |\n| BCE (MORE)                           |          ✘          |        ✘        |      60.7       |\n| BCE ($\\theta = \\theta^g + \\theta^t$) |          ✘          |        ✘        |      59.0       |\n| BCE (LoRA)                           |          ✔          |        ✔        |      58.4       |\n\n\n> Evaluation breadth. Although the experiments cover a diverse range of tasks, they are limited to relatively small- or medium-scale datasets. Common benchmarks in long-tail learning—such as ImageNet-LT and iNaturalist—are notably absent. This limits the ability to draw conclusions about the method’s scalability and robustness in more realistic or challenging scenarios.\n> Can the authors evaluate the method on larger and more standard long-tailed benchmarks to better demonstrate its scalability and generalization?\n\nThanks for your thoughtful recommendation. To address this, We conducted supplementary experiments on ImageNet-LT, using a ResNet-50 backbone. The accuracy (%) results are as follows:\n\n| Method        | Many | Medium | Few  | All  |\n| ------------- | :--: | :----: | :--: | :--: |\n| CE            | 69.6 |  42.2  | 14.5 | 49.0 |\n| CB            | 69.7 |  42.7  | 16.7 | 49.6 |\n| LA            | 63.7 |  51.9  | 34.7 | 54.1 |\n| LA w/ MORE    | 65.0 |  52.6  | 36.1 | 55.1 |\n| ProCo         | 66.2 |  53.9  | 37.3 | 56.3 |\n| ProCo w/ MORE | 66.9 |  54.8  | 38.0 | 57.2 |\n\nThese results show that MORE provides consistent and stable improvements across all partitions, with particularly notable gains on Tail classes while maintaining performance on Many and Medium classes. This aligns with our findings on CIFAR-100-LT, Places-LT and other large-scale datasets in multi-label settings, confirming the method's effectiveness in scaling to larger, more diverse datasets. Due to computational resource and time limits, we are currently conducting experiments on the iNaturalist dataset and will present the results as soon as they are completed.\n\n> The paper does not explore strong baselines that manipulate model capacity, such as mixture-of-experts methods [1, 2] or self-supervised learning approaches [3], which offer alternative solutions to tail-class underfitting. Moreover, using extra parameters is a common and effective strategy—parameter-efficient finetuning approaches like [4] have also shown strong performance in such scenarios. Including comparisons or ablations against these baselines would help clarify whether the performance gains come from the specific design of MORE or from the general benefit of added parameter capacity.\n> Can the authors include comparisons or ablations against alternative capacity-based baselines to isolate the source of performance improvements?\n\nThank you for this valuable suggestion. Regarding mixture-of-experts (MoE) methods [1,2], we note that our approach shares conceptual similarities in addressing tail-class underfitting through capacity adjustment. To address your concern, we have included additional experiments comparing MORE against RIDE [1] (using 4 experts) and BalPoE [2] on CIFAR-100-LT, with accuracy (%) results shown below. The results demonstrate that integrating MORE with a strong baseline like ProCo yields consistent improvements. In addition to accuracy, we also report the number of inference-time parameters (third column) to highlight that, unlike MoE-based baselines which expand model capacity via expert branches, MORE requires no additional inference parameters, achieving improvements without increasing model size.\n\n| Method           | Backbone | Params | Epoch | IF=10 | IF=100 |\n| ---------------- | :------: | :----: | :---: | :---: | :----: |\n| CE               | ResNet32 | 0.57M  | 200   | 61.4  | 44.1   |\n| CB               | ResNet32 | 0.57M  | 200   | 62.1  | 44.7   |\n| RIDE (4 experts) | ResNet32 | 1.05M  | 200   | 61.8  | 49.1   |\n| BalPoE           | ResNet32 | 1.37M  | 200   | 65.1  | 52.0   |\n| ProCo            | ResNet32 | 0.57M  | 200   | 65.0  | 51.9   |\n| ProCo w/ MORE    | ResNet32 | 0.57M  | 200   | 65.9  | 52.9   |\n\nOur method does not inherently involve self-supervised learning. Similarly, parameter-efficient finetuning like [4] primarily deals with freezing pretrained parameters for adaptation in single label settings, which differs from our core contribution.  To further enhance comprehensiveness, we will include additional discussions on methods [3] and [4] in the revised version of the paper.\n\n[1] Long-tailed Recognition by Routing Diverse Distribution-Aware Experts\n\n[2] Balanced Product of Calibrated Experts for Long-Tailed Recognition\n\n[3] Generalized Parametric Contrastive Learning\n\n[4] Parameter-Efficient Long-Tailed Recognition\n\n> Modest overall gains in some settings. On certain benchmarks (e.g., NUS-WIDE, CIFAR100-LT and Places-LT), the overall gains are relatively small, raising questions about trade-offs and consistency across datasets.\n> How consistent are the improvements across different datasets, and can the authors explain the relatively modest gains observed in some settings?\n\nThanks for your feedback. To address your concerns, we provide the following elaboration:\n\n* **Consistent Improvements Across Settings**: We would like to emphasize that our method actually shows reliable enhancements across diverse datasets and settings, including both single-label and multi-label long-tailed scenarios. Previous methods almost cannot achieve such generalizability and consistency. \n\n* **Focus on Underrepresented-Class Performance**: Our gains are particularly evident in tail classes, where our mechanism helps effectively mitigate underfitting due to data imbalance. For instance, on CIFAR-100-LT, MORE improves tail-class accuracy by 1.5% (IF=10) and 1.6% (IF=100). Similarly, it boosts tail-class performance by 2.5% on Places-LT and 1.9% on NUS-WIDE-SCENE, underscoring its targeted benefits for underrepresented classes.\n\n* **Performance Saturation in Certain Settings**: Regarding the relatively modest overall gains in some settings, we are inclined to attribute to the fact that, with constrained data availability in tail classes and specific model architectures, the trained models often approach a natural performance upper bound. As this ceiling is neared, the learned features become increasingly saturated, making incremental improvements more challenging to achieve. This phenomenon is common in the settings where head-class dominance already pushes baseline accuracies relatively high, leaving limited room for global gains despite substantial tail improvements. We will expand on these insights in the discussion section of the revised paper to provide further clarity."}, {"review_id": "EhyYo9PLUz", "reviewer": "Reviewer_XTwi", "summary": "This paper introduces ***MORE*** (MOdel REbalancing). The core idea is to rebalance the model's parameter space through a low-rank parameter decomposition and incorporates a difference-based loss function and a sinusoidal re-weighting schedule to achieve this. Experimental results show that ***MORE*** significantly improves the model's generalization on tail classes across several multi-class and multi-label long-tailed benchmarks.", "questions": "See Weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The design of the $L_{more}$ loss is particularly insightful. By penalizing the activation of the tail-specific parameters ($θ^t$) on head-class samples, the framework effectively incentivizes these parameters to specialize in learning features pertinent to the tail classes. This is a clever and direct mechanism to encourage parameter specialization.\n\n2. The sinusoidal re-weighting schedule is a simple yet elegant mechanism. It effectively creates a dynamic curriculum, allowing the model to alternate its focus between learning generalizable features and specialized features (for tail classes) at different training stages. This simple periodic function achieves an effect similar to more complex, hand-crafted curriculum learning strategies.\n\n3. The use of low-rank parameterization for the tail-specific parameters is well-justified. This design choice significantly reduces the number of parameters dedicated to the tail classes, which inherently acts as a strong regularizer.\n\nWeaknesses:\n\n1. While the paper correctly claims no increase in inference cost, the impact on training cost is not addressed. The formulation, particularly as implied by Equation (2), suggests that the training process requires two forward passes per iteration. The lack of empirical data (GPU hours) on this training overhead is a significant omission. This information is crucial for assessing the practical feasibility of the method, especially for training on large-scale datasets.\n\n2. I have some reservations about the sensitivity of the method to its key hyperparameters (the rank r and the balancing coefficient α). A critical trade-off exists for the rank r: if it is too small, the representational capacity of the tail-specific parameters $θ^t$ may be insufficient; conversely, if it is too large, the benefits of the low-rank assumption are diminished. It is unclear how these hyperparameters should be set for new datasets or different backbone architectures without extensive tuning. Furthermore, the assumption that a single shared low-rank space $θ^t$ can effectively serve all tail classes might be a limitation. If certain tail classes are semantically distinct and require features that are largely \"orthogonal\" to those of other tail classes, this shared space may fail to adequately capture their unique characteristics.\n\n3. The paper could further explore the relationship between the general ($θ^g$) and tail-specific ($θ^t$) parameters. While the loss function applies them to different objectives, they are still optimized jointly. Have the authors considered enforcing a stricter decoupling, for instance, through an orthogonality constraint between $θ^g$ and $θ^t$? This could prevent representational overlap and lead to a more modular model.", "was_revised": true, "final_justification": "The authors have satisfactorily addressed my initial concerns regarding training overhead and hyper-parameter sensitivity in their rebuttal, providing a detailed analysis of the issues I raised. Thus, I will maintain my original score.", "rebuttal": "> While the paper correctly claims no increase in inference cost, the impact on training cost is not addressed. The formulation, particularly as implied by Equation (2), suggests that the training process requires two forward passes per iteration. The lack of empirical data (GPU hours) on this training overhead is a significant omission. This information is crucial for assessing the practical feasibility of the method, especially for training on large-scale datasets.\n\nThanks for your comment. To address your concern, we conducted additional experiments on the CIFAR-100-LT with the imbalance ratio of 10 on a single NVIDIA 3090 GPU over 200 epochs. The results are shown in the table below, which indicates that the overhead is relatively tolerable. We will include these details in the final paper for enhanced transparency.\n\n| Method      | Training time (Minutes) |\n| ------      | :---------------------: |\n| LA          | 14                      |\n| LA w/ MORE  | 21                      |\n| BCL         | 49                      |\n| ProCo       | 64                      | \n\n\n> I have some reservations about the sensitivity of the method to its key hyperparameters (the rank r and the balancing coefficient α). A critical trade-off exists for the rank r: if it is too small, the representational capacity of the tail-specific parameters $\\theta^t$ may be insufficient; conversely, if it is too large, the benefits of the low-rank assumption are diminished. It is unclear how these hyperparameters should be set for new datasets or different backbone architectures without extensive tuning. Furthermore, the assumption that a single shared low-rank space $\\theta^t$ can effectively serve all tail classes might be a limitation. If certain tail classes are semantically distinct and require features that are largely \"orthogonal\" to those of other tail classes, this shared space may fail to adequately capture their unique characteristics.\n\nThanks for this insightful feedback. Regarding hyperparameter sensitivity, our empirical evaluations show that our method is relatively robust, with similar settings generalizing well across diverse datasets and backbones, without the need for extensive per-task tuning. Furthermore, leveraging prior characteristics of the dataset-such as class imbalance ratios or feature distributions-to guide the selection of hyperparameters like the rank $r$ remains a valuable direction for future exploration.\n\nYour point about the potential limitation merits deeper investigation. Indeed, without our parameter decomposition, semantically distinct features in tail classes would compete for representation space with other head and tail classes. By introducing the decomposition parameters, our method already mitigates competition from head classes, allowing $\\theta^t$ to focus more effectively on tail characteristics. To futher address this concern, one approach is to moderately increase the rank of tail-specific parameters for greater capacity; alternatively, we can partition tail classes into multiple groups based on semantic similarity (e.g., via clustering on pre-trained embeddings) and assign group-specific low-rank parameters, thereby alleviating potential orthogonality issues among tail features. \n\nWe appreciate these suggestions and will include further discussion in the revised paper to highlight potential avenues for enhancement.\n\n> The paper could further explore the relationship between the general ($\\theta^g$) and tail-specific ($\\theta^t$) parameters. While the loss function applies them to different objectives, they are still optimized jointly. Have the authors considered enforcing a stricter decoupling, for instance, through an orthogonality constraint between ($\\theta^g$) and ($\\theta^t$)? This could prevent representational overlap and lead to a more modular model.\n\nThank you for this constructive suggestion. While certain orthogonality constraints can pose optimization difficulties—such as the ones discussed in [1], where some forms of orthogonal regularization negatively impact the convergence rate—we still believe that appropriate orthogonalization strategies could further minimize representational overlap and potentially improve performance on tail classes. This remains a valuable direction and is worth further exploration. We will include a discussion in the conclusion section of the revised paper to reflect the merit of this suggestion.\n\n[1] On Orthogonality and Learning Recurrent Networks with Long Term Dependencies"}, {"review_id": "A2ZqPMQjoO", "reviewer": "Reviewer_jeuY", "summary": "This paper proposes MORE, a method designed to rebalance the model parameter space in imbalanced learning scenarios. The authors introduce a tailored loss function that incorporates logit-level contrastive supervision for space reallocation, along with a dynamic weighting scheme for model rebalancing. Experiments and ablation studies across different datasets are conducted to demonstrate the effectiveness of the proposed approach.", "questions": "- The connection between the theoretical analysis in Section 3.3 and the methodology in Section 3.4 is unclear. This relationship should be clarified in the rebuttal. It would also be helpful if Section 3.4 included a brief remark explicitly referencing how the theory informs or motivates the method.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros\n\n- The paper is generally well written, and the motivation is clear and well justified. The authors build on meaningful observations to guide their design.\n\n- The proposed method is simple yet effective. Ablation studies are thorough, and the approach shows solid performance across various long-tailed learning scenarios.\n\n\nCons\n\n\n- The baselines are relatively weak. The paper lacks comprehensive comparisons against recent state-of-the-art methods in long-tailed single-label classification. Including stronger baselines would help solidify the claimed improvements.", "was_revised": true, "final_justification": "I have read the response and other reviewers' comments. Most of my concerns have been addressed, and I maintain my positive rating for this paper.", "rebuttal": "> The baselines are relatively weak. The paper lacks comprehensive comparisons against recent state-of-the-art methods in long-tailed single-label classification. Including stronger baselines would help solidify the claimed improvements.\n\nThank you for this valuable suggestion. Regarding mixture-of-experts (MoE) methods [1,2], we note that our approach shares conceptual similarities in addressing tail-class underfitting through capacity adjustment. To address your concern, we have included additional experiments comparing MORE against RIDE [1] (using 4 experts) and BalPoE [2] on CIFAR-100-LT, with accuracy (%) results shown below. The results demonstrate that integrating MORE with already strong baseline like ProCo yields consistent improvements. In addition to accuracy, we also report the number of inference-time parameters (third column) to highlight that, unlike MoE-based baselines which expand model capacity via expert branches, MORE requires no additional inference parameters, achieving improvements without increasing model size.\n\n| Method           | Backbone | Params | Epoch | IF=10 | IF=100 |\n| ---------------- | :------: | :----: | :---: | :---: | :----: |\n| CE               | ResNet32 | 0.57M  | 200   | 61.4  | 44.1   |\n| CB               | ResNet32 | 0.57M  | 200   | 62.1  | 44.7   |\n| RIDE (4 experts) | ResNet32 | 1.05M  | 200   | 61.8  | 49.1   |\n| BalPoE           | ResNet32 | 1.37M  | 200   | 65.1  | 52.0   |\n| ProCo            | ResNet32 | 0.57M  | 200   | 65.0  | 51.9   |\n| ProCo w/ MORE    | ResNet32 | 0.57M  | 200   | 65.9  | 52.9   |\n\n[1] Long-tailed Recognition by Routing Diverse Distribution-Aware Experts\n\n[2] Balanced Product of Calibrated Experts for Long-Tailed Recognition\n\n> The connection between the theoretical analysis in Section 3.3 and the methodology in Section 3.4 is unclear. This relationship should be clarified in the rebuttal. It would also be helpful if Section 3.4 included a brief remark explicitly referencing how the theory informs or motivates the method.\n\nThanks for highlighting this point. To address your concern, we elaborate as follows:\n\n* **Theoretical motivation**: the theoretical foundation in Section 3.3 motivates and justifies the core components of our MORE method. In particular, Theorem 1 demonstrates that by decomposing the model parameters based on class roles, we partition the hypothesis space. This decomposition reduces the overall Rademacher complexity of the model, which in turn tightens the generalization bound on the balanced risk, particularly benefiting tail classes in long-tailed distributions. \n\n* **Methodological realization**: Section 3.4 operationalizes the above theoretical insights into a concrete implementation. Specifically, (1) The low-rank parameter decomposition reflects the model-space partition discussed in Section 3.3. (2) The discrepancy-based loss guides the optimization such that the low-rank components learn tail-specific representations, while the shared components capture general patterns. (3) The sinusoidal reweighting schedule further balances the trade-off between generalization and specialization, aligning with the theoretical goal of capacity reallocation over training time.\n \nTo make this connection clearer, we will add a brief remark at the beginning of Section 3.4 explicitly referencing Section 3.3 as followes: Motivated by the theoretical analysis in Section 3.3 on decomposing model parameters, we now detail the practical instantiation of parameter rebalancing through low-rank adaptation and a tailored discrepancy-based loss, which adaptively emphasizes tail-specific learning to enhance performance in long-tailed distributions."}, {"review_id": "5bfppU2W7C", "reviewer": "Reviewer_XVA7", "summary": "This paper introduces MORE (MOdel REbalancing), a novel method for tackling long-tailed visual recognition by rebalancing the model’s parameter space. MORE decomposes the weights of each model layer into general and minority-specific components using a low-rank parameterization, guided by a tailored discrepancy-based loss and sinusoidal reweighting schedule. This design allows the model to allocate dedicated capacity to underrepresented classes without increasing inference complexity. Theoretical analysis demonstrates that this decomposition tightens the generalization bounds. Extensive experiments across multi-class and multi-label benchmarks validate the effectiveness of the proposed method.", "questions": "Please see the \"Strengths And Weaknesses\" part.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n- The paper is generally well-written and easy to follow.\n- The proposed method is novel, addressing the long-tailed learning challenge from a new perspective by rebalancing the model's parameter space.\n- The authors provide a theoretical analysis demonstrating that the proposed decomposition leads to a tighter generalization bound, which enhances the soundness and credibility of the approach.\n- The method is easy to implement and can be seamlessly integrated with other long-tailed learning techniques.\n\n\n\n**Weaknesses:**\n\n- For the multi-class image recognition experiments, the authors primarily evaluate on CIFAR100-LT and Places-LT. However, ImageNet-LT and iNaturalist are widely used large-scale long-tailed benchmarks. Including results on these datasets would better validate the generalizability and effectiveness of the proposed method.\n\n- The paper should be better to clarify which modules of ResNet and CLIP are decomposed using LoRA, respectively. Additionally, it would be helpful to specify the chosen rank $r$ for LoRA and compare the number of learnable parameters with and without LoRA during training.\n- It would be valuable to report the values of $M(x; \\theta)$ across samples from different classes in the final trained model. Furthermore, presenting the difference $f_y(x; \\theta^g \\oplus \\theta^t) - f_y(x; \\theta^g)$ across class samples would help support the effectiveness of the proposed loss $L_{\\text{MORE}}$, as increasing the ground-truth logit is key to improving model performance.\n- There are also some typos, such as in line 40, where \"Rachemacher complexity\" should be \"Rademacher complexity\".", "was_revised": true, "final_justification": "I appreciate the authors’ clarifications on the large-scale benchmark evaluation, the decomposition strategy with LoRA, and the analysis of class-wise logit differences. The responses have generally addressed my concerns, and I will maintain my score of 4.", "rebuttal": "> For the multi-class image recognition experiments, the authors primarily evaluate on CIFAR100-LT and Places-LT. However, ImageNet-LT and iNaturalist are widely used large-scale long-tailed benchmarks. Including results on these datasets would better validate the generalizability and effectiveness of the proposed method.\n\nThanks for your thoughtful recommendation. To address this, We conducted supplementary experiments on ImageNet-LT, using a ResNet-50 backbone. The accuracy (%) results are as follows:\n\n| Method        | Many  | Medium | Few   | All   |\n|---------------|:-----:|:------:|:-----:|:-----:|\n| CE            | 69.6  | 42.2   | 14.5  | 49.0  |\n| CB            | 69.7  | 42.7   | 16.7  | 49.6  |\n| LA            | 63.7  | 51.9   | 34.7  | 54.1  |\n| LA w/ MORE    | 65.0  | 52.6   | 36.1  | 55.1  |\n| ProCo         | 66.2  | 53.9   | 37.3  | 56.3  |\n| ProCo w/ MORE | 66.9  | 54.8   | 38.0  | 57.2  |\n\nThese results show that MORE provides consistent and stable improvements across all partitions, with particularly notable gains on Tail classes while maintaining performance on Many and Medium classes. This aligns with our findings on CIFAR-100-LT, Places-LT and other large-scale datasets in multi-label settings, confirming the method's effectiveness in scaling to larger, more diverse datasets. Due to computational and time limits, we are currently conducting experiments on the iNaturalist dataset and will present the results as soon as they are completed.\n\n> The paper should be better to clarify which modules of ResNet and CLIP are decomposed using LoRA, respectively. Additionally, it would be helpful to specify the chosen rank $r$ for LoRA and compare the number of learnable parameters with and without LoRA during training.\n\nThanks for this valuable suggestion. To clarify the decomposition: in our implementation, we apply low-rank decomposition to all convolutional layers of the ResNet backbones, with other layers remaining unchanged. For CLIP-based models, we freeze the text encoder and fine-tune only the vision encoder and decompose all convolutional layers within it.\n\nRegarding the rank $r$, we commonly use $r = 0.1$. For parameter comparisons during training: on a ResNet-34 backbone, the learnable parameters are approximately  25.4M with MORE and 21.3M without. Similarly, for ResNet-50, the counts are about 29.4M with MORE and 25.6M without. This modest training overhead stems from the low-rank parameters and is relatively contained. We will incorporate these details into the revised paper in Section 4.1 or appendix. \n\n> It would be valuable to report the values of $M(x;\\theta)$ across samples from different classes in the final trained model. Furthermore, presenting the difference $f_y(x;\\theta^g \\oplus \\theta^t) - f_y(x;\\theta^g)$ across class samples would help support the effectiveness of the proposed loss $L_{\\mathrm{MORE}$, as increasing the ground-truth logit is key to improving model performance.\n\nThanks for this insightful suggestion. We conducted an additional analysis of $|f_y(x;\\theta^g \\oplus \\theta^t) - f_y(x;\\theta^g)|$ across samples from head, mid, and tail classes in the final trained models on various datasets. The averaged values are as follows:\n\n| Dataset      | Head   | Medium | Few    |\n| ------------ | :---:  | :---:  | :---:  |\n| MIML         | 0.016  | 0.018  | 0.020  |\n| MIML w/ CLIP | 0.064  | 0.083  | 0.091  |\n| VOC          | 0.062  | 0.067  | 0.071  |\n| VOC w/ CLIP  | 0.026  | 0.028  | 0.036  |\n\nThese results suggest that $|f_y(x;\\theta^g \\oplus \\theta^t) - f_y(x;\\theta^g)|$ is smaller for head classes and larger for tail classes across all settings. This indicates that the tail-specific parameters exhibit a stronger response to tail-class samples and effectively amplify their representations. Note that the order across groups is more important than the absolute values, as it reflects their differential effect on the final logits. This pattern supports the effectiveness of our method, as it aligns with the goal of increasing ground-truth logits for underrepresented classes. We will include these analyses and visualizations in the revised paper to further substantiate the method's mechanisms.\n\n> There are also some typos, such as in line 40, where \"Rachemacher complexity\" should be \"Rademacher complexity\".\n\nThanks for pointing this out. We have carefully proofread the entire manuscript and corrected the typos. These changes will be included in the revised version."}]

Author rebuttal:

Meta-review: This paper proposes a new method for long-tail learning by rebalancing the model’s parameter space, rather than relying on data or loss functions. By decomposing model weights into general and low-rank components, the proposed method allocates capacity specifically for tail classes. The method improves generalization for underrepresented classes without increasing inference cost, and achieves strong results across both single- and multi-label long-tailed benchmarks, including CLIP finetuning.

After rebuttal, the paper has received four positive evaluations. All reviewers have updated their final ratings and noted that the majority of their concerns in both theoretical and experimental aspects have been resolved through the author-reviewer discussion. Additionally, every reviewer has recognized the paper’s contributions and expressed satisfaction with its technical strengths.

The AC concurs with the reviewers' assessments and recommends acceptance.
Decision: Accept (poster)