Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts
Authors: Zhe Zhao
HaiBin Wen
Zikang Wang
Pengkun Wang
Fanfu Wang
Song Lai
Qingfu Zhang
Yang Wang
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Long tail learning
Multi-objective optimization
Primary area: deep_learning_architectures

Abstract: Traditional long-tailed learning methods often perform poorly when dealing with inconsistencies between training and test data distributions, and they cannot flexibly adapt to different user preferences for trade-offs between head and tail classes. To address this issue, we propose a novel long-tailed learning paradigm that aims to tackle distribution shift in real-world scenarios and accommodate different user preferences for the trade-off between head and tail classes. We generate a set of diverse expert models via hypernetworks to cover all possible distribution scenarios, and optimize the model ensemble to adapt to any test distribution. Crucially, in any distribution scenario, we can flexibly output a dedicated model solution that matches the user's preference. Extensive experiments demonstrate that our method not only achieves higher performance ceilings but also effectively overcomes distribution shift while allowing controllable adjustments according to user preferences. We provide new insights and a paradigm for the long-tailed learning problem, greatly expanding its applicability in practical scenarios. The code can be found here: https://github.com/DataLab-atom/PRL.

Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generatte Diverse Experts
Zhe Zhao?.3, Haibin Wen', Zikang Wang', Pengkun Wangl.?; Fanfu Wang,,
Song Lai Qingfu Zhang". Yang Wang 1.2x
University of Science and Technology of China (USTC), Hefei, China
Suzhou Institute for Advanced Research, USTC, Suzhou, China
3 City University of Hong Kong, Hong Kong, China
*Harbin Institute of Technology. Harbin, China
'MorongAl, Suzhou, China
Lanzhou University, Lanzhou, China
(zz4543 @ mail.ustc. eeu..eedu.en haibin65535 @ gmail.com. 2021211940@su.hin wangff21 llzu.edu.co en, songlai2-c my.cityy.edu.bb. qingfu.zhang @cityu.edu.kk, pengkun wang @ uuttcedu.cc. angyan @ ustc.edu.cn)
Abstract
Traditional long-tailed learning methods often perform poorly when dealing with inconsistencies between training and test data distributions, and they cannot flexibly adapt to different user preferences for trade-offs between head and tail classes. To address this issue, we propose a novel long-tailed learning paradigm that aims tackle distribution shift in real-world scenarios and accommodate different user preferences for the trade-off between head and tail classes. We generate a set of diverse expert models via hypernetworks to cover all possible distribution scenarios, and optimize the model ensemble to adapt to any test distribution. Crucially, in any distribution scenario, we can flexibly output dedicated model solution that matches the user's preference. Extensive experiments demonstrate that our method not only achieves higher performance ceilings but also effectively overcomes distribution shift while allowing controllable adjustments according to user preferences. We provide new insights and a paradigm for the long-tailes learning problem, greatly expanding its applicability in practical scenarios. The code can found here: hatps://github.comm/// wtt./ataaabb-atmomm//oooooo
Introduction
In many real-world tasks such as object detection and image classification, we face the challenge of long-tailed distributions. Since the samples of the head classes account for the vast majority in the datasets while the tail class samples are extremely scarce [21, 16, 6, 17], this extreme imbalance in the data makes the model prone to overfitting towards the head classes during training, resulting in poor performance on the tail classes [34, 8, 30, 20, 39].
To address the long-tailed distribution problem, existing research has proposed series of methods such as ce-samlling [25, 5, 24, 10] and modifying the loss function [17, 6], with the common idea of focusing improving the performance of the tail classes. However, these methods typically assume that the distributions of the training and test data remain invariant, and thus cannot well handle the
Pengkun Wang and Yang Wang are corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

common situations of distribution shift between training and testing in real-world scenarios. Some more recent works such as RIDE [32] and LADE [12] propose using multiple expert models to obtain stronger distribution adaptability. Building on this, SADE [38] further adaptively combines the outputs of these experts during testing to adapt to the current test distribution. These approaches alleviate the problem of distribution mismatch between training and testing to some extent [23].
However, in addressing distribution shifts across different test scenarios, the goal of these multi-expert model-based methods is still to maximize the overall performance, i.e., pursuing the optimal overall performance metric across all classes, and obtaining a fixed trade-off for this purpose[12 13, 40]. But in different application scenarios, users may have different preferences and needs for the relative trade-off between head and tail classes. Simply pursuing the overall optimal solution may not meer this flexibility requirement[17. 35, 43]. For example, in classifying lung CT images, when screening for difficult cases, we care more about whether all possible disease types (i.e., tail classes) can be covered to avoid missed diagnoses, compared to routine physical examinations. For some serious diseases such as lung cancer, we may also be willing to moderately increase the false positive rate in exchange for higher coverage of the tail classes, to ensure that no patients are missed. Another example is wildlife detection. Within nature reserves, we want the model to accurately detect common species (i.e., head classes) to understand their population sizes. But when looking for rare species (i.e., tail classes), we care more about covering all species, even at the cost of some false detections.
As can be seen, in different application scenarios, there are significant differences in user preferences for the weighting of head and tail categories, which current long-tailes learning methods often fail to fully satisfy. Therefore, developing an interpretable and controllable method for handling long-tail distributions that adapts to specific user preferences for head and tail categories becomes new research direction in the field of long-tailed learning.
In light of this, we propose an interpretable and controllable long-tail learning method (PRL). This method aims not only to overcome potential distribution shifts from a single training distribution to any testing distribution but more importantly, flexibly adjust the weights of head and tail categories according to actual user demands. To address these challenges, we introduce a new long-tailed learning paradigm based on a diverse set of experts and hypernetworks, which can adapt to a wide range of distribution scenarios and meet personalized user preferences.
To tackle the aforementioned challenges, we propose new long-tailed learning paradigm based on diverse experts and hypernetworks, as illustrated in Figure. For the first challenge, existing multi-expert model-based methods train fixed expert models for specific distributions, requiring strong distribution assumptions and struggling to handle more complex and variable distributions. Therefore, instead of maximizing the performance of each expert individually, we pursue modeling and optimizing the hypervolume over the entire Pareto front curve, learning a set of solutions that cover all possible distribution scenarios. This requires us to sample with the goal of covering the entire Pareto front during optimization. For the second challenge, unlike LADE and SADE which output fixed trade-off solution under distribution shift, we can flexibly output a dedicated model solution that matches the user's ' preference in any test distribution scenario. In this way, our method can not only adapt to changes in the test distribution, but also allow controllable adjustment of the head-tail off according to the user's 's actual needs.
Our contributions can be summarized as follows:
New scenario and insight: we make the first attempt at a controllable trade-off based on user preferences in the context of long-tailed learning and test distribution shift scenarios, greatly expanding the applicability in real-world scenarios.
New learning paradigm: we propose a new interpretable and controllable long-tailed learning method that can acquire the ability to overcome test distribution shift from a single distribution dataset and satisfy user preferences in any shifted distribution scenario.
Compelling empirical resulis: extensive experiments demonstrate that our method achieves higher performance ceilings, effectively overcomes test distribution shift, and can be controlled by user preferences.
2

---

Trained model
Trained
model
Training set
Test set
Training set
Test set
(a) Traditional long-tailie learning method
(b) Multi-expert long-toiled leorning method
P1
Dirichlet distribution
Trained
Hypernetwork
model
Training set
Test
(c) Our proposed method
Figure 1: Illustration of our method: (a) Existing methods train for specific long-tailed distribution
but may fail on arbitrarily skewed test distributions. (b) Multi-expert learns different experts for different distributions from one training set but lacks flexibility for arbitrary distributions/preferrnn (c) Our method samples preference vectors during training to simulate distributions, and can flexibly adjust the preference vector during testing for flexible long-tailed classification.
Related Work
Long-tailed distributions are prevalent in real-world data, leading to imbalanced datasets that pose challenges for machine learning models [30, 20J. To address this issuc, researchers have proposed various methods, including re-sampling, loss function modification, and multi-expert models.
Re-sampling methods balance class distributions by oversampling tail classes [24, 3] or undersampling head classes [7]. Loss function modification approaches assign higher weights to tail class losses [27, 26] or use meta- veta-learning to alleviate undersampling issues [14, 32]. Multi-expert models train multiple experts on different class distributions and combine their outputs, adapting to various test distributions [37, 38, 31J. Most existing methods assume specific distributions during training or testing, limiting real-world applicability with distribution shifts, and cannot accommodate varying user needs for head and tail class trade-offs. We propose an approach to overcoming distribution assumptions and achieve interpretable, controllable trade-offs in long-tailed learning.
3 Theory
In this section, we analyze the distribution shift problem from theoretical perspective and provide the definition and properties of the environment's total variation distance, laying the theoretical foundation for the methods section.
Definition (Distribution Discrepancy across Environments). Given M training environments E..... EMd, with class prior probability vectors π²' Tr...., respectively, where πt" (TT" ...., TT) T denotes the probability of the kithh class appearing environment En.. and ΣK th = If there exist i,j, k. such that Ii + E. holds, then these environments are said have distribution discrepancy.
Traditional empirical risk minimization (ERM) methods on single training distribution struggle to handle distribution diserepancy, which can affect generalization. This limitation can be characterized by the following theorem:

---

Theorem (Limitation of ERM). Let f(;; )) be a classifier learned via ERM on Em, then its risk on the test environment Eress is:
K (Ttt m. Pen-- ()
Riest = (f) +
(1)
i=1
where Rm and (f are the risks of on &m and Evsa, respectively, and Teleat is the class prior of the test environment.
To measure the distribution discrepancy across environments, we introduce the environment's total variation distance (ETVD):
Definition 2 (Environment Total Variation Distance). The total variation distance between environ- EE
ments & and E is defined us: ö(E,, Eg)
m zi. and the ETVD of M environments is
defined as: A A(E1,..., = max i.)
E))
Using ETVD, we can further bound the risk of the ERM-learned classifier on the test environment: Corollary 1. Under the assumptions of Theorem 1, let M max (x: 0), i), then
Rf) S Rm (f) + 2M (8(Ems Evet) A
(2)
This corollary shows that test risk of the ERM-learned classifier is affected not only by the distribution discrepancy between the training environment and the test environment but also by the distribution discrepancy among training environments (i.e., ETVD). To overcome the diversity shift, we propose minimizing the empirical risks across multiple training environments to capture the distributional characteristics of different environments, thereby learning a set of diversity experts.
Next, we provide a theoretical analysis of the domain adaptation algorithm based on diversity experts proposed in this paper. characterize the generalization performance of this algorithm, first introduce the following notations:
Let (fi...., respectively, and f be the final classifier obtained by ensembling these N experts. Define the empirical risk of the ensemble classifier on environment En as:
fN} } be the N experts learned via ERM the N training environments {E,,...,EN, En }.
Rm Σ (fi
We can obtain the following theorem regarding the generalization performance of the ensemble classifier:
Theorem 2. Under the above notations and definitions, the risk of the ensemble classifier f on the test environment Eren satisfies:
N
N
N- A(E,,.... Ex)
Reese()) S
Σ
Rma(Sm) + 2M
Σ m=1
8(Em, Erest) +
(4)
N
N
N
m==1
where M = max (()x)))))))))
Theorem 2 shows that the test risk of the ensemble classifier consists of three parts: the average empirical risk of all experts, the average total variation distance between the training environments and the test environment, and the weighted average of ETVD among the training environments. Compared single-environment ERM, the diversity experts method learns set of experts to capture the distributional characteristics of different environments, which can reduce the distribution discrepancy between the training environments and the test environment, thereby achieving better generalization performance.
Methodology
4.1 Problem Formulation
Consider K-class classification problem with a training set D ((mn,::)) where each class has N₂ samples. Let Pursin denote the empirical distribution over D. The goal is to lear classifier

---

f: RK that generalizes well across various test distributions Piest- Traditional empirical risk minimization (ERM) methods optimize the loss under P irains but may fail adapt to changes in Piea. especially in long-tailed scenarios.
To improve the robustness of f. optimize the losses under multiple importance-weighted distribu- tions, Define an M-dimensional simplex:
}
AM
E
0
11
Each a E AM corresponds to an importance-weightee distribution Pa:
Pa (ar v) = Σ
O Pa(x v) Pa(v)
(6)
k=1
where Pa( v) and Pv( = N₂ N are the conditional distribution and prior for class k. respectively.
The objective is to learn set of classifiers F : ( that achieve low risk simultaneously across all Pa. forming the Pareto optimal solution:
min Rpa, (F), .... (Pam(F)
where (F) == B(a.o)~Pa
1 Σ L( (()())))
(z). w) Pursuing an approximate Pareto solution
M
across all distributions leads to models with stronger generalization capabilities.
4.2 Diverse Experts
Let X' and Y' denote the input and output spaces, respectively. We introduce T classifiers (f diverse experts. These experts share a feature extractor de X Rd but use different classifier heads {gw, 1
(x(x) = Iw. (x)(x)),
= 1 T
(8)
To generate diverse experts, we introduce a hypernetwork hp that takes random noise z E I& as input and outputs the classifier head parameters w, E Rdxc.
W;- = (z;). 2; ~ Dir(a),
1=
where Dir(a) is the Dirichlet distribution with parameter a E R The hypernetwork hy consists of three linear layers with ReLU activations.
During training, sample {z from Dir(a) and use he to generate [wi). The loss function for a training batch is:
T
C ECCA()))
(10)
where L; is the classification loss for the i-th expert fú defined the same as SADE: L₁ is the standard cross-entropy loss; L2 is the balanced softmax loss, where the logits adjusted by adding the log of the prior probabilities of each class; La the inverse softmax loss, where the logits are adjusted by adding the log of the prior probabilities and subtracting the scaled log of the inverse prior probabilities.
4.3 Stochastic Convex Ensemble
Let L, (+, D) denote the loss function of the i-th expert fi on dataset D, where O (0, v represents all trainable parameters. The objective is to jointly optimize the losses of all 7 experts:
min
L,)),
(1))
5

---

we compute the test-time preference To promote diversity among experts, we introduce the Stochastic Convex Ensemble (SCE) strategy. which aims to minimize the worst-case loss of the convex combination of experts:
min max PEAT
Σ prLi(O,D)
(12)
where (pI,-.. (P) E Ar is the weight vector, and ÅT = (p E R44 Ei=1 = 13 is the T-dimensional simplex.
Inspired by the max-min incquality, we relax the SCE objective to:
T
(), + A- log Σ exp (tc10.D))
min
Σ
(13)
where À 0 is hyperparameter. As A - 0, the relaxed objective approaches the original SCE objective. The term A log Ei= exp (O, D)) promotes diversity among experts.
4.4 Preference-Contrrolee Trade-off
During testing, we can control the trade-off between head and tail classes using preference vector a (ai, 02, az) E A3, where A is the 3-dimensional simplex.
Given a trained preference vector r (2)) A vector A E A as:
r a f
(14)
r T'*
where denotes the Hadamard product. The test-time preference vector F is then input to the hypernetwork by to generate the classifier head parameters for each expert:
W, = hy hy
1,,....,
(15)
where Wi RdxC is the weight matrix for the i-th expert's classifier head. For test sample with feature vector x E IRd, the output of the i-th expert is:
W
if normalized
ú1
(16)
W
+ 6.T
otherwise
where ||e is the Frobenius and b, RC is the bias vector for the i-th expert. By adjusting a', we can control the model's focus on head or tail classes, enabling flexible trade-off' to suit different application needs.
An observation on our method. better under- stand this part, use Figure 2 to demonstrate the effectiveness of preference control overcoming distribution shifts, well as the flexibility of our method. For preferences, the coordinate system dicensiimnnion orthogonal coordinate; for ac- curacy, the coordinate system represents the perfor- mance on the farward50, uni., and backward50 splits of the CIFAR 1EFAR100-LTY 100-L dataset. The dark plane repre- sents the plane formed by different preference vectors, and the outer surface represents the corresponding performance on the three distributions for these pref- erence vectors. The yellow dots are the results of run- ning SADE, whose preferences the results of each run are random dots, lying below our purple plane, indicating that their performance lower than our method (i.e., being dominated in the Pareto optimal set). This figure illustrates that our method can cover unknown distributions without additional training, and unlike previous methods, it can trade off performance by adjusting the preference vector. We will analyze this more depth in the experimental section.
D0%
SADE
0.5
0
0.5
x
0.5
uncontrollable, so
1
Figure 2: Mapping from preference to model properties.

---

Thp-1 ecouracy on CBNR1C2-7181003
Top-1 acouragy (988100-71/1100)
Top-1 occuaayy on CIAR100 L71R200)
3.0
0.7
top3
T0p-
0.7 W.n. 8
0.6
0372 W.o 0,0
0.5
0.
0= 2:0
0
5.
Oars 8
2.1.
0.4
Dus 8
U00 8
-. 2.6
-.0.0
0w..8
0.0
2,0
30
3.0
0.0
2.0
1.0
30
5.0
alsseelasses lobel fonwarsto
classss lost buskward75
huperclame label on boonwerib
(a]
(b)
(C)
Figure 3: Analysis of the preference control for the trade-off between head and tail class performance. We present the results on three distributions. The vertical axis represents accuracy. The horizontal axis shows the results after clustering by frequency, from head classes tail classes (left to right).
Experiments
In this section, we first evaluate the superiority of PRL in terms of both standard and test-agnostic long-tailed recognition to demonstrate that our method has higher performance ceiling under the traditional setup. Then, we analyze the effectiveness our method in changing the trade-off for long-tailed classes through input preferences. Furthermore, conduct necessary ablation studies.
5.1 Experimental Setups
Datasets. We evaluate method on four benchmark datasets: ImageNet-LT [20], CIFARI00-LT [4], Places-LT [20], and iNaturalist 2018 [29]. These datasets have varying imbalance ratios, ranging from 10 to 256. CIFAR 100-LT has three versions with different imbalance ratios. Detailed statistics are in Appendix D.
Baselines. We compare PRL with various state-of-the-art long-tailed recognition methods, including two-stage methods (MiSLAS [41]), logit-adjusted training (Balanced Softmax [15], LADE [12]), ensemble learning (RIDE [32], SADE [38]), causal inference (Causal [28]), representation learning (LSC [33]), and balanced posterior averaging (BalPoE [1]). These methods address the long-tail problem from different perspectives. Further details are provided in the appendixA.
Evaluation protocols and implementation details. We evaluate the models on multiple test datasets with different class distributions using micro accuracy. We report the accuracy of many-shot, medium- shot, and few-shot classes. We use the same setup for all methods, including ResNeXt-50 for ImageNet-LT, ResNet-32 for CIFAR100-LT, R52 for Places-LT, and Re for iNaturalist 2018 as backbones. We employ hypernets (MLPs) to output trainable parameters of experts and adopt the cosine classifier for prediction. Unless specified, we use 1.2 for the Dirichlet distribution, a == 0.3 for stochastic annealing. SGD with momentum 0.9, train for 200 epochs, and set the initial learning rate to 0. with linear decay. During test-time training, we train aggregation weights for 5 epochs with a batch size of 128, using the same optimizer and learning rate as in training. Other details please refer to Appendix G.
5.2 Comparative Evaluation on Standard and Test-Agnostic Long-Tailes Recognition
We conduct extensive experiments four widely-used long-tailed datasets, including CIFARIOO-LT, Places-LT, iNaturalist 2018, and ImageNet-LT, to evaluate the performance of our proposed PRL method in comparison with state-of-the-an approaches.
Results on standard long-tailed recognition. Table 1 demonstrates the effectiveness of our pro- posed method, PRL, on four benchmark datasets under the standard cong-tailed recognition setting, where the test class distribution is uniform. PRL consistently achieves the highest top-1 accuracy across all datasets, outperforming the previous state-of-the-as methods, LSC [33] and BalPoE [1]. On CIFAR 100-IT,, PRL improves accuracy by 0.6% 0.8% compared to LSC and BalPoE, showcasing its robustness to different imbalance ratios (IR=10, 50, and 100). Similarly, on Places-
7

---

Table 1: Top-1 accuracy on CIFARIOO-LT, Places-LT, iNaturalist 2018, and ImageNet-LT, where the
test class distribution uniform.
CIFARI00-LT
Method
Places-LT
iNaturalist 2018
ImageNet-LT
IR=10
IR=50
IR=100
Softmax
59.1
45.6
41.4
31.4
64.7
48.0
Causal [28]
59.4
48.8
45.0
32.2
64.4
50.3
Balanced Softmax [15]
61.0
50.9
46.1
39,4
70.6
52.3
MiSLAS [41]
62.5
51.5
46.8
38.3
70.7
51.4
LADE [12]
61.6
50.1
45.6
39.2
69.3
52.3
RIDE [32]
61.8
51.7
48.0
40.3
71.8
56.3
SADE [38]
63.6
53.8
48.8
40.9
72.7
58.8
LSC [33]
65.0
56.5
51.8
41.3
73.9
60.2
BalPoE [1]
64.8
56.3
52.0
40.8
75.0
59.3
PRL(ours)
65.6
$7.3
52.8
41.6
75.1
60.8
Table 2: Top-1 accuracy on CIFAR100-LT (IR100) with various unknown test class distributions.
Forward-L3
Uni.
Backward-Li -T
Method
Prior
25
10
5
10
25
50
Softmax
62.0
56.2
52.5
46.4
41.4
36.5
30.5
25.8
21.7
17.5
54.2
52.0
46.1
43.6
40.8
38.4
36.3
33.7
MiSLAS
58.8
$7.2
55.2
53.0
46.8
43.6
40.1
37.7
32.1
LADE
56.0
55.5
52.8
51.0
48.0
45.6
43.2
40.0
38.3
35.5
34,0
LADE
62.6
60.2
55.6
52.7
48.2
45.6
43.8
41.1
41.5
40.7
41.6
RIDE
63.0
59.9
57.0
53.6
49.4
48.0
42.5
38.1
35.4
31.6
29.2
SADE
65.2
62.5
58.8
55.4
51.2
48.8
43.0
43.9
42.4
42.2
42.0
LSC
67.8
64.2
60.2
58.1
53.2
$1.6
44.7
45.7
44.2
44.7
48.0
BalPoE
69.0
65.2
61.2
59.0
54.2
51.7
45.7
46.6
45.2
45.2
45.8
PRL (ours)
69.5
65.7
61.7
59.5
54.7
$2.2
46.2
47.1
45.7
45.7
48.5
LT, iNaturalist 2018, and ImageNet-LT, PRL obtains the best performance, surpassing the existing methods by clear margin. The superior performance of PRL in the standard long-tailed recogni- tion setting validates the efficacy of our approach in mitigating the bias towards head classes and improving the recognition accuracy of tail classes.
Results on distribution-shift long-tailed recognition. We evaluate the performance of PRL and other methods in the distribution-shift long-tailed recognition setting, where the test class distribution is unknown and different from the training distribution. Tables and 3 show the Top-1 accuracy results various test class distributions (including forward LT, uniform, and backward LT) for CIFARIO0-LT (IR=100) and ImageNet-LT, respectively.
On different test distributions both datasets, PRL consis- tently outperforms all compared methods. On CIFAR100 LT (IR100), PRL achieves the highest accuracy across all settings, surpassing LSC [33], BalPoE [1], and SADE [38]. Even under the most challenging backward LT distribu- tion, PRI can maintain its outstanding performance. On ImageNet-LT, PRL obtains the best results across all test distributions, significantly outperforming LSC, BalPoE, and SADE.
Many
Top-1 d Coauaog Cccrrcc
The consistent improvements achieved by PRL highlight its higher performance ceiling, indicating the effectiveness of our method design in overcoming distribution shifts. We experiments on Places- LT and iNaturalist 2018, where PRL also achieves impres- sive results. Please refer to the AppendixF for detailed results.
further conduct distribution-shi
Figure 4: A more comprehensive exam- ple of how preference influences perfor- mance.

---

Table 3: Top-1 accuracy on ImageNet-LT with various unknown test class distributions.
Forward-LT
Uni.
Backward-LT
Method
Prior
25
50
Softmax
66.
63.8
60.3
56.6
52.0
48.0
43.9
38.6
34,9
30.9
27.6
61.9
59.5
57.2
54.4
52.3
50.0
47.0
45.0
42.3
40.8
MISLAS
58.0
49.2
46.1
44,0
39.5
LADE
63.4
62.1
59,9
57.4
54.6
52.3
49.9
46.8
44,9
42.7
40.7
LADE
65.8
63.8
60.6
57.5
54.5
$2.3
50.4
48.8
48.6
49.0
49.2
RIDE
67.6
66.3
64.0
61.7
58.9
56.3
54.0
51.0
48.7
46.2
44.0
SADE
69.7
67.5
65.4
62.3
60.3
58.3
56.7
54.9
54.3
53.1
52.6
LSC
72.0
69.7
67.5
65.3
62.7
60.2
59.2
58.5
57.9
57.5
57.0
BalPoE
72.2
69.7
67.2
64.3
62.2
59.5
58.5
57.7
56.9
56.7
56.6
PRI. (ours)
72.7
70.2
68.0
65.8
63.2
60.7
59.7
59.0
58.4
58.0
$7.5
Table 4: Control of trade-off preference for long-tailed classes with different preferences, bold text, underlined text, and dashed underline respectively indicate the highest performance of the head, middle, and tail classes in this line.
2.7)
R= 2.55
R=(1.9, 1.1)
Dist.
Many
Middle
Few
Many
Middle
Few
Many
Middle
Few
Forward
50
61.4
50.4
36.5
61.0
52,6
31.5
61.1
48.9
40.3
61.6
48.3
28.4
60.6
49.6
31.5
59.7
49.4
33.1
Uni
61.6
51.4
33.2
61.6
51.5
33.2
61.6
51.4
33.2
Backward
25
63.8
49.4
31.1
60.2
48,2
32.1
63.2
48.2
32.2
66.6
47.1
30.6
66.1
48.9
30.9
64.6
47.8
31.7
User preference control. We evaluated the model's performance many-shot, medium-shot, and few-shot classes on CIFAR100-LT under different preference settings (R=(1.0, 2.7), R=(0.5, 2.5), R=(1.9, 1.1)). Table 4 shows that by adjusting the preference value R. trade-off between many-shot and few-shot classes. When R=(10,, 2.7), the model performs best on many-shot classes; when R=(1.9, 1.1), the model performs better few-shot classes at the cost of a slight drop in performance on many-shot classes. R=(0.5, 2.5) achieves the best performance on medium-shot classes, indicating that our method can balance performance with an appropriate setting. As shown in Figure 3, we analyze the performance trade-off's between head and tail classes across three different distributions, demonstrating how our preference control mechanism allows flexible adjustment of model behavior. The results clearly show how adjusting preferences affects accuracy across different class frequency groups.
effectively control the
different classes
Figure provides a more comprehensive visu- alization of the performance the head classes under the forward50 distribution. The plane represents the performance of the head classes without inputting any preference, while the red dots indicate the preference positions in polar coordinates that can improve the performance of the head classes, and the green dots repre- sent the preference positions that may degrade the performance. These experimental results demonstrate the effectiveness of our method in controlling the trade-off off for long-tailed classes based on user preferences. By adjusting the pref- erence without the need for retraining the model, we can flexibly adapt to different application scenarios and requirements, achieving a desired trade-off long-tailed recognition tasks that aligns with practical needs.
acoursor UTNRLogy
tett cluss
S1.0
636
w.d.
300
P.50
P-2
P-10
P5
F22 UnEom
82
95
8:10
8:25
850
Figure 5: Ablation analysis, including the ablation of the hypernetork and Chebyshev polynomials.

---

Ablation study. We conduct ablation studies on CIFARIO0-LT to evaluate the impact of removing the hypernetwork (w.o. hnet) and removing the Chebyshev polynomial (w.o. stch) on the model's perfor- mance under different unknown test class distributions (as shown in Figure 5). The complete model (ours) performs best across all distributions. Removing either the hypernetwork or the Chebyshev polynomial leads to performance degradation, highlighting their importance in dynamically adjusting the model behavior to adapt to distribution shifts and learning preference-awarr representations. This ablation study verifies the effectiveness of different components in our method, which work together to better handle unknown test distributions and data imbalance issues long-tailed recognition.
Conclusion
This study introduces novel long-tailed learning paradigm to address distribution shifts between training and testing datasets. Our hypernetwork-basee approach generates adaptable classifiers, achieving Pareto optimality for real-time adaptation. During inference, the model adjusts based user-defined trade-offss between head and tail classes, enhancing flexibility. Empirical results show improved accuracy and adaptability to class imbalances and distribution shifts. Our work establishes an interpretable, generalizable, and controllable framework for long-tailed learning, meeting diverse user needs.
Acknowledgements
This work was supported by the Natural Science Foundation of China Youth Project (No. 62402472), the Natural Science Foundation of Jiangsu Province of China Youth Project (No. BK20240461), the Research Grants Council of the Hong Kong Special Administrative Region, China (GRF Project No. CityU 11215723), National Natural Science Foundation of China (No.62072427. No. 1227702.,,, the Project of Stable Support for Youth Team in Basic Research Field, CAS (No. YSBR-00S) and Academic Leaders Cultivation Program, USTC.
10

---

References
[1] Emanuel Sanchez Aimar, Arvi Jonnarth, Michael Felsberg, and Marco Kuhlmann. Balanced product of calibrated experts for long-tailed recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19967-19977, 2023.
[2] Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106:249-259, 2018.
[3] Jonathon Byrd and Zachary Lipton. What the effect of importance weighting in deep learning? In International conference on machine learning, pages 872-881. PMLR, 2019.
[4] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbal- anced datasets with peneel-istrbbuion-awwee margin loss. In Advances in Neural Information Processing Systems, 2019.
[5] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321- 357, 2002.
[6] Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pages 9268-9277, 2019.
[7] Chris Drummond, Robert Holte, et al. C4. 5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling. In Workshop learning from imbalanced datasets II, volume 11, 2003.
[8] Kai Gan and Tong Wei. Erasing the bias: Fine-tuning foundation models for semi-supervised learning. In Proceedings of the 41st International Conference on Machine Learning, pages 14453-14470, 2024.
[9] Hui Han, Wen- Yuan Wang, and Bing-Huan Mao. Borderline-smote: over-sampling method in imbalanced data sets learning. In International conference on intelligent computing, pages 578-887, Springer, 2005.
[10] Haibo He and Edwardo A Garcia. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, (119)1263-1222 2009.
[11] Claus Hillermeier. Nonlinear multiobjective optimization: generalized homotopy approach, volume 135. Springer Science Business Media, 2001.
[12] Youngkyu Hong. Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, and Buru Chang. Disentangling label distribution for long-tailed tailed visual recognition. In Computer Vision and Pattern Recognition, 2021.
[13] Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Learning deep representation for imbalanced classification. In Computer Vision and Pattern Recognition, 2016.
[14] Muhammad Abdullah Jamal, Matthew Brown, Ming Hsuan Yang, Liqiang Wang, and Boqing methods for long-tailed visual recognition from a domain
Gong. Rethinking adaptation perspective. In Proceedings of the IEFÉ/CVF conference on computer vision and pattern recognition, pages 7610-7619, 2020.
[15] Ren Jiawei, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, al. Balanced meta-softmax for long-tailed visual recognition. In Advances in Newral Information Processing Systems, 2020.
[16] Bingyi Kang, Saining Xie, Mareus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis, Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09211 2019.
[17] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.

---

[18] Xi Lin, Zhiyuan Yang, Qingfu Zhang, and Sam Kwong. Controllable pareto multi-task learning. arXiv preprint arXiv:2010.06313, 2020.
[19] Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. In International Conference on Machine Learning. 2024.
[20] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large- scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2537-2546, 2019.
[21] Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. arXiv preprint arXiv:2007.07314 2020.
[22] Kaisa Miettinen. Nonlinear multiobjective optimization, volume 12. Springer Science & Business Media, 1999.
[23] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenen- baum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint
2018.
[24] Li Shen, Zhouchen Lin, and Qingming Huang. Relay backpropagation for effective learning of deep convolutional neural networks. In Computer Fision-ECCT 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VII 14, Springer, 2016.
pages
[25] Jiang- Xin Shi, Tong Wei, Yuke Xiang, and Yu-Feng Li. How re-sampling helps for long-tail learning? Advances in Neural Information Processing Systems, 36:75669-75687, 2023.
[26] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta- weight-net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, 32, 2019.
[27] Jingru Tan, Changbao Wang, Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjie Yan. Equalization loss for long-tailed object recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11662-11671, 2020.
[28] Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in neural information processing systems, 33:1513-1524, 2020.
[29] Grant Van Horn, Oisinand Mac Aodha, et al. The inaturalist species classification and detection dataset. In Computer Vision and Pattern Recognition, 2018.
[30] Grant Van Horn and Pietro Perona. The devil is in the tails: Fine-grained classification in the wild. arXiv preprint (tfi::17000015500 2017.
[31] Kun Wang, Hao Wu, Guibin Zhang, Junfeng Fang, Yuxuan Liang, Yuankai Wu, Roger Zim- mermann, and Yang Wang. Modeling spatio-temporal dynamical systems with neural discrete learning and levels-of-experts. IEEE Transactions on Knowledge and Data Engineering, 2024.
[32] Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X Yu. Long-tailed recogni- tion by routing diverse distributtion-wwree experts. In International Conference on Learning Representations, 2021.
[33] Tong Wei, Zhen Mao, Zi-Hao Zhou, Yuanyu Wan, and Min-Ling Zhang. Learning label shift correction for test-agnostic tailed recognition. 2023.
[34] Wei Xu, Pengkun Wang, Zhe Zhao, Binwu Wang, Xu Wang, and Yang Wang. When imbalance meets imbalance: Structure-driven learning for imbalanced graph classification. In Proceedings of the ACM on Web Conference 2024, pages 905-913, 2024.
[35] Yuzhe Yang and Zhi Xu. Rethinking the value of labels for improving class-imbalanced learning. In Advances in Neural Information Processing Systems, 2020.
12

---

[36] Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. Feature transfer learning for face recognition with under-represented data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5704-5713, 2019.
[37] Songyang Zhang, Zeming Li, Shipeng Yan, Xuming He, and Jian Sun. Distribution alignment: A unified framework for long- tail visual recogmition. Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 2361-2370, 2021.
[38] Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. Self-supervised aggregation of diverse experts for test-agnostic long-tailed recognition. Advances in Neural Information Processing Systems, 35:34077-34090, 2022.
[39] Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[40] Zhe Zhao, Pengkun Wang, HaiBin Wen, Wei Xu, Song Lai, Qingfu Zhang, and Yang Wang. Two fists, one heart: Multi-objective optimization based strategy fusion for long-tailed learning.
[41] Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long-tailed recognition. In Computer Vision and Pattern Recognition, 2021.
[42] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 4066):145211464, 2017.
[43] Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. Bbn: Bilateral- branchh network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9719-9928, 2020.
13

---

Appendix
Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Seternttwor--Generaeeee Diverse Experts
A
Baselines Details
In this section, provide comprehensive overview of some state-of-the-art methods for long-tailed recognition, which will serve as baselines for comparison with our proposed PRI. approach.
Two-stage methods decouple representation learning and classifier training to mitigate the bias towards head classes. MiSLAS [41] introduces a mixup-based strategy in the second stage to enhance the learning of tail classes. By separating the learning process, these methods can alleviate the negative impact of imbalanced data on feature extraction.
Logit-adjusted training methods focus on modifying the logits during training to address class imbalance. Balanced Softmax [15] introduces class-balanced term to the softmax function, which adaptively adjusts the logits based on the sample frequencies. LADE [12] disentangles the learning of feature representations and classifier by adding learnable logit adjustment term. These methods effectively prevent the model from being biased towards head classes.
Ensemble learning methods leverage multiple classifiers or experts to capture the diversity of the data. RIDE [32] trains multiple experts with different resampling strategies and dynamically combines their outputs based on the sample distributions. SADE [38] further improves upon RIDE by introducing self-adaptive distillation mechanism to transfer knowledge among experts. By exploiting the diversity of experts, these methods can better handle imbalanced data.
Causal inference methods aim to address the long-tail problem by designing causal clas- sifiers. Causal [28] proposes causal inference framework that identifies the causal effect of each class on the predictions, thus reducing the bias introduced by the imbalanced data distribution.
Representation learning methods tackle long-tailed recognition learning more balanced and discriminative features. LSC [33] introduces contrastive learning framework that balances the instance-level and group-level distributions simultaneously, leading more effective representations for tail classes.
Balanced posterior averaging methods focus on combining the predictions of multiple experts based on their posterior probabilities. BalPoE [1] proposes balanced posterior averaging strategy that assigns higher weights to experts with better performance on tail classes, thus achieving better trade-off between head and tail classes.
While these methods have made significant progress in addressing the long-tail problem, they often rely on specific assumptions about the data distributions during training or testing, limiting their applicability in real-world scenarios. Moreover, most of these methods do not provide mechanism for users to control the trade-off between head and tail classes based on their specific needs. In contrast, proposed PRL approach overcomes these limitations by learning diverse set of experts that can adapt to various test distributions without any prior assumptions, while also enabling interpretable and controllable trade-offs through Pareto optimization.
B Supplementary materials for related work
B.1 Long-Tailed Learning (LTLL)
Long-tailed distributions, where a few classes (heads) have abundant samples while many classes (tails) have few samples, are widespread in real-world data [30, 20]. This imbalance poses significant challenges for machine learning models, which tend to perform poorly on tail classes. To address this issue, various Long-tailed learning (LTL) methods have been proposed.
14

---

B.1.1 Resampling Methods
Oversampling [5, 9]: Generates synthetic examples for minority classes.
Alleviates imbalance by increasing tail class samples.
Can lead to overfitting and high computational cost.
Undersampling [7, 2]: Removes examples from majority classes.
Simple and efficient approach to balance classes.
Discards potentially valuable head class information.
B.1.2
Loss Adjustment Methods
Focal Loss [17] and variants [6]]
Imposes larger penalties well-classified examples, encouraging focus hard sam- ples.
Requires careful tuning of focusing parameter.
Class-Balanced Loss [6]:
Re-weights loss based on effective number of samples per class.
Assumes equal importance of classes, which may not hold in practice.
LDAM [4]]
Explicitly models each example's contribution the gradient direction.
- Requires additional hyperparameters and complex optimization.
B.1.3
Module Improvement Methods
Decoupled Learning [16, 43]:
Separates representation and classifier learning for better feature extraction.
Requires architectural changes, may not generalize well.
Few-Shot Experts [32]:
Employs additional experts to handle few-shot classes.
Increased model complexity and training difficulty.
Sell-Supervised Pretraining [14]:
Leverages self-supervision to improve feature representations.
Requires additional pretraining, benefits may be task-specific.
B.1.4 Transfer Learning Methods
Data-Based Transfer [20, 14]:
Knowledge distillation and feature transformation transfer head knowledge tails.
Assumes head and tail distributions are related, may suffer negative transfer.
Model-Based Transfer [36]:
Utilizes models pretrained on heads to facilitate tail class learning.
Again assumes related head and tail distributions.
Despite progress, existing LTL methods face limitations in addressing the inherent head-tail trade-off, handling distribution shifts, and accommodating varying user preferences. overcome these issues, we formulate L.TI. as a multi-objective optimization problem and propose a hypernetwork-baasee diverse expert learning paradigm, achieving interpretable and controllable solutions tailored to user needs under test distribution shifts.
15

---

B.2 Related Work for Multi-Objective Optimization and Hypernetworks
B.2.1
Multi-Objective Optimization
Let X S IX" be the decision space and consider m objective functions f R, I,..., m to be minimized simultaneously. The multi-objective optimization problem (MOP) can be stated as:
(17)
xEX
general, there does not exist a single solution x* E X' that minimizes objectives simultaneously due to the conflicting nature of the objectives. Instead, the solution concept is that of Pareto optimality [22].
Definition (Pareto Optimality). solution x E X Pareto optimal if there does not exist another X such that fx( f((x*) ) all 1....,m and fx(x) < fz(xx) for least one j.
The set of all Pareto optimal solutions is called the Pareto set, and its image in the objective space is the Pareto front. The goal MOPs is to approximate the Pareto front as well as possible.
B.2.2 Chebyshev Scalarization
A common approach to approximate the Pareto front is through scalarization methods that transform the MOP into a scalar optimization problem [22, 19]. The weighted Chebyshev scalarization is defined
min max wi (x)(((x)) -2,) sssm
(18)
where w = (wi...., wm) E RT a weight vector with E w; = 1, and z" = (2f 2m ) is
utopian reference point [11]. By varying w, different Pareto optimal solutions can be obtained.
B.2.3 Hypernetworks for Multi-Objective Optimization
Hypernetworks( [8] provide promising approach for multi-objective optimization of neural networks. hypernetwork h& : Z + è is a neural network that takes a low-dimensional input z E and outputs the parameters 0 e Ô of target neural network fe X + V. By sampling different Z, the hypernetwork generates an ensemble {fo, Js where 0; = ha (2.). This ensemble can approximate the Pareto front of the multi-objective optimization problem:
min Tn (fo), -.. I'm (fo) 85e
119)
where L; θ + R are loss functions corresponding to the m objectives. parameters @ can be optimized via scalarizations like the Chebyshev method:
The hypernetwork
max wi (Li(fnsta) - Isism
z3)
min Ewwp(w)
(20)
where p(w) is a distribution over weight vectors w. This enables learning a diverse set of target networks approximating the Pareto front flexible and controllable manner.
C Proof of Propositions
C.1 Proof of Theorem
Proof. The risk of classifier f on the test environment Evess is defined as:
Riest (f) ((m.,)))Prmatr.)) (x; 0), v)]
(21)
Using the law of total expectation, decompose the risk as:
Riest (f) Σ
EggPaate v= (2;
(22)
1=1
16

---

Similarly, the risk of classifier f on the training environment &m can be expressed as:
(m)
Σ T.
P..
(23)
8=
Since the classifier f is learned via ERM on Em. we have Pm (xly = Puea(a i) for all i {I,...,k, Kj. Therefore.
K Σ
1et "i Eg
(24)
1=1
Rgg Panl (o
K
+
n )! ) Ban
1=1
Rm (f) +
( ( mi -e (orl ( [((f(2; 0)
This completes the proof.
C.2 Proof of Corollary 1
Proof. From Theorem 1, we have:
K
Rieat (f) = 1m(f) +
Σ test ) (ef=:
mT.
(25)
1=1
Using the definition of M, we can bound the expectation term:
0), s M
(26)
Therefore,
K
Ress(() S (mx +
Σ
lest nT;
M
i=1
K
Rm + M
lest
Ry, (f) + 2M -8(Em. Eres)
where the last equality follows from the definition of the total variation distance 5(Em, Ewsa) D |.."" -!|0
Next, use the triangle inequality to bound 5(Em, Evea):
ò (Em, Evst) < 8(Em; E,) + +6E,, Evea)
max
6(E,,E)) 5(Eg, Eves)
M)
AE,,,... ... EM) 8(E,, Eles)
for any j E (f,...,,M
My. By taking the average over all j. we obtain:
1 Σ 8(E,, Erest)
6(Em, Evea) S A(E,, EM) + M
(27)
Combining this with the previous bound on Riest (f), we have:
(AlE..... AAEE....,EM) + M j=l
1 M
Risa (f) S Rma(f) + 2M
8(E,, Evsa)
(28)
which completes the proof.
17

---

C.3 Proof of Theorem 2
Proof. By definition, the risk of the ensemble classifier on the test environment Eiett is:
Rist (f) = l8 (i ( ((((()),))
(m.y)) (el))()))
N
Riea(f))
Applying Corollary 1, we have:
N
Rieat (f) S
(mi + 2M (8(Em()) Evea) +
N
==1
2M
Σ
(fi)) +
Σ 6(Em()):Eus) + 2. 2M.
N
where m(i) denotes the index of the training environment used learn expert fs. Now, we bound the second term:
8(Em)()) Erea)
Σ
8(Em Eves))
N
imm(t)=m
Nu 5(Em Evest)
N
N 8(Em, Erea)
N
8(Em, Eres)
where Nm is the number of experts learned from environment Em, and used the fact that Em No N.
Finally, we have:
N Rm (fm) 2M
N 8(Em, + AEE
R
S
N
N
m=1
m=1
N -
Rm 2M
8(Em, Eres) +
A(EI,...,EN))
N
N
where the last equality follows from the definition of ETVD.
Next, will explain the connection between the theoretical part of the paper and the proposed method, to aid better understanding
Theorem states that for classifier of learned via ERM on a single environment Em, its risk on the test environment Eve is influenced not only by the distribution discrepancy between the training and test environments, but also by the distribution discrepancy among the training environments (i.e., ETVD). This reveals the limitation of the single-environment ERM method.
2. Corollary further quantifies an upper bound on the risk of the ERM-learned classifier in the test environment. This upper bound consists of the training risk Rm,(f), the TVD
18

---

(Em, Etsa) between the training and test environments, and the ETVD A ",,....,EM!! among the training environments. To overcome distribution shift, we need learn set of diverse expert models that can capture the distribution characteristics of different environments.
3. Theorem provides a tighter upper bound on the risk of the diversity aware ensemble classifier f in the test environment. Compared to the single-envirooment ERM, the average empirical risk term 인 Em= (fm) in the upper bound indicates that the ensemble classifier can reduce empirical risk, while the presence of N Lm=1 8(Em &ee) shows that the diversity-aware expert method, by learning a set of experts to capture the distribution characteristics of different environments, narrow the distribution gap between the training and test environments, thereby achieving better generalization performance.
The above theoretical analysis demonstrates that, in the long-tailed learning domain, introducing multiple training environments and minimizing the empirical risks on these environments learn set of diverse experts can effectively address the problem of distribution shift between the training and test environments, leading better generalization performance. These theoretical insights provide important guidance for further improving our algorithm.
D Datasets
To evaluate the effectiveness of our proposed method, we conduct experiments on four long-tailed datasets: CIFAR100-LT, ImageNet-LT, iNaturalist 2018, and Places365-LT. These datasets diverse range domains and exhibit varying degrees of class imbalance, providing a comprehensive testhed for long-tailed learning algorithms.
Table 5: Statistics of the loigetailed datasets.
Dataset
# Classes
# Train
Test
Imbalance Ratio
CIFAR100-LT
100
50,000
10,000
(10, 50, 100)
ImageNet-LT
1,000
115,846
50,000
256
iNaturalist 2018
8.142
437,513
24,426
500
Places365-LT
365
1,803,460
36,500
~50
CIFAR100-LT [4] is a long-tailed version of the CIFAR 100 dataset, comprising 60,000 color im- ages of size 32 32 pixels across 100 classes. The long-tailed distribution is induced by exponentially decreasing the number of samples per class, resulting in an imbalance ratio of up to 100.
ImageNet-LT [20] is long-tailed subset of the ImageNet dataset, containing over 115,000 images spanning 1,000 classes. The class cardinalities follow a Pareto distribution with a = 6, leading to maximum imbalance ratio of 256.
iNaturalist 2018 [29] is real-world dataset with a natural long-tailed distribution, comprising approximately 450,000 images across 1.122 species. The number of images per species varies drastically, with an imbalance ratio of up to 500, posing a significant challenge due to the extreme class imbalance and high intra-class variation.
Places365-LT is loigetailed version of the Places365 dataset [42], which consists of over 1.8 million images spanning 365 scene categories. We induce a loig-tailed distribution by randomly subsampling the images for each class, resulting in an imbalance ratio of approximately 50. This dataset is particularly challenging due to the large number of classes and the inherent visual ambiguity present in scene recognition tasks.
E
Pseudo Code
Here pscudo codes explaining the core aspects of the method:
19

---

Algorithm Diverse Expert Learning with Hypernetworks
1: Input: Training data with long-tailes tailed distribution Drrain
2: Output: Ensemble of expert models & = (E1, Ez...., Ex
3: Initialize shared feature extractor F
4: Initialize hypernetorr Hg with weights 0
5: Initialize expert loss function (e.g.. oiverse
6: for each epoch do
7: for each batch B C D train do
8:
f= F()) { Shared feature extraction|
9:
for k 1 to do
10:
Φ = Ho (zx) {Generate expert weights ΦR from hypernetwork]
Eg (f( (Obtain expert predictions using Φ }
12:
L L(E,, y, extra_infee [Compute expert losses]
end for
14:
Lain DiversityLoss(E) [Encourage expert diversity)
Liotal Σx La ALdiv
16:
0= V VoCooll
end for
18:
end for=0
Table 6: List of Key Symbols in Pseudo Code
Symbol
Description
D
Training data with long-tailed distribution
EE, Eg...., Ex}
Ensemble of expert models
F
Shared feature extractor Hypemetwork with weights Expert loss function (c.g., DiverseExpertLoss)
He
C
2k
Input to hypernetwork for generating weights of expert Fj. Weights of expert F& generated by hypernetwork Loss of expert E&
LE
Laiv
Diversity loss encourage expert diversity
Ltotal
Total loss for updating hypernetwork weights
F Results on ImageNet-LTT and iNaturalist 2018 Datasets
On the representative Places-LT dataset. PRL method achieves the best Top-1 accuracy under various unknown test class distributions. Specifically, in the Forward-LT setting, as the proportion of unknown classes decreases from 50% to 2%, the Top-1 accuracy of PRL drops from 47.9% to 42.8%, but still significantly outperforms other baseline methods. Under the Uniform distribution, PRL reaches the highest accuracy of 41.9%. In the Backward-LT setting, PRL's accuracy gradually increases from 41.7% % to 44.1%. 1%, again surpassing all counterpart methods. These results thoroughly validate the outstanding performance and robustness of our method in handling various unknown class distributions.
On the iNaturalist 2018 dataset, PRL also exhibits excellent performance. In the Forward-LT setting, when the proportion of unknown classes decreases from to 2, PRL's Top-1 accuracy slightly increases from 73.7% to 73.8%, and reaches the best performance of 74.3% under the Uniform distribution. In the Backward-LT setting, although PRL's accuracy slightly decreases from 74.0% to 73.9%, it still outperforms all comparison methods. These results further confirm the broad effectiveness of our method across different datasets and scenarios.
Overall, by successfully tackling the challenges of long-tailed distributions and unknown class distributions, the PRL method demonstrates superior performance on two representative long-tailed datasets, thereby validating the superiority of our method.
20

---

Table 7: Top-1 accuracy on Places- with various unknown test class distributions.
Forward-LT
Uni.
Backward-LT
Method
Prior
25
10
5
10
25
50
Softmax
42.7
40.2
38.0
28.4
25.4
23.4
42.7
41.7
41.3
41.0
40.0
39.4
38.5
37.8
37.1
35.6
MISLAS
40.9
39.7
39.5
39.6
38.8
38.3
37.3
36.7
35.8
34.7
34,4
LADE
42.8
41.5
41.2
40.8
39.8
39.2
38.1
37.6
36.9
36,0
35.7
LADE
46.3
44.2
42.2
41.2
39.7
39.4
39.2
39.9
40.9
42.4
43.0
RIDE
43.1
41.8
41.6
42.0
41.0
40.3
39.6
38.7
38.2
37.0
36.9
SADE
46.2
44.8
42.8
42.7
41.1
40.4
40.2
40.9
41.2
41.4
41.6
47.5
46.1
44.5
43.7
42.1
41.4
41.2
41.5
42.1
43.2
43.4
BalPoR
PRL (ours)
47.9
47.0
45.3
44.4
42.8
41.9
41.7
42.1
42.6
43.7
44.1
Table Top-1 accuracy on iNaturalist 2018 with various unknown test class distributions.
Forwand-LT
Uni.
Backward-LT
Method
Prior
2
Softmax
X
65.4
65.5
64.7
64.0
63.4
BS
X
70.3
70.5
70.6
70.6
70.8
MiSLAS
70.8
70.8
70.7
70.7
70.2
LADE
68.4
69.6
LADE
69.1
69.3
70.2
RIDE
X
71.5
71.9
71.8
71.9
71.8
SADE
x
72.3
72.6
72.7
73.0
73.2
LSC
BalPoE
73.1
73.5
73.8
73.6
73.5
PRL (uurs)
73.7
73.8
74.3
74.0
73.9
G
Complexity Analysis
Although the introduction of hypernetworks increases the total number of parameters the model, as shown in the table below, in most cases, this does not lead to significant increase in computational complexity.
Model
Hypernetwork
Params (MB)
GFLOPs
ResNet-32
x
0.8
13.1
2.6
13.1
ResNeXt-50
X
38.2
391.8
V
632.7
393.5
ResNet-50
X
39.1
2982.0
159.51
2982.0
Table 9: Model size and computational cost with and without hypernetworks.
The hypernetwork is responsible for outputting the trainable parameters D for each expert classifier head. In this case, the number of parameters in the hypernetwork becomes E x D K, where E is the number of input channels to the output layer of the hypernetwork, and is the number of experts. Thus, the time complexity of the hypernetwork is O(D).
the total number parameters in the model without the hypernetwork is O(N), and the computa- tional complexity of the model's main operations is O(M x N), where M > is complexity factor, then when N > D, the overall time complexity becomes O(D) + O(M x N) = O(M x N).
Therefore, while the hypernetwork increases the total number of parameters, its impact on the overall computational complexity is relatively small, especially when the number of parameters D generated by the hypernetwork is much smaller than the total number of parameters N in the main model. As can be seen from the table, the introduction of hypernetworks results in almost no increase GFLOPs (floating-poii operations) across different models.
21

---

H
Limitations
While the proposed novel approach of using hypernetwork generate multiple diverse expert models shows great potential in enabling controllable adjustment of head and tail class weights for long-tailed datasets, as well as improving robustness to distribution shifts, the introduction of the hypernetwork also brings new challenges to model training and convergence. As an additional neural network module, the hypernetwork needs to generate the weight parameters for the classifier heads of each expert, thereby significantly increasing the total number of trainable parameters in the model, which may affect training stability. We analyze this issue in Section G, nevertheless, further research into more efficient training and stable controllability is still necessary.
I Broader Impacts
The proposed novel approach of generating multiple expert models via hypernetworks enables dynamic adjustment of head and tail class weights for long-tailed datasets, and improves model robustness to distribution shifts. This flexibility and robustness are of significant value in many practical applications. The present work provides new viable solution the important challenges of long-tailed distributions and distribution shifts, holding promise to enhance the generalization capabilities and practical applicability of existing models, thereby contributing to the technological advancement in relevant fields.
22

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We describe the contributions of this paper in detail.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of this paper in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
23

---

Justification:
We provide a complete demonstration process and data support.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide implementation details and code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
24

---

Answer: [Yes]
Justification: We provide the complete code as well as the details of the publicly available datasets used.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https:: potttps///nis..o cc/
public/ guides Codeeubomi ss aasioooPlcyy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: n ps cc/ /pppplll c/gut des CodeSubm's ss onPol ccy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide the relevant details and analysis of the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We reported the average results of multiple tests in the experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
25

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided the code for easy reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htt /ttt://///urppa///e http:////ccc/ccccccccceoeeeeoee
Answer: [Yes]
Justification:
We are fully qualified.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the implications in the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
26

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We give references to the datasets and code used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers' naw thcode com/ dataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
27

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "SUuhE5jbZT", "reviewer": "Reviewer_LGeA", "summary": "The paper addresses the problem of learning long-tailed distributions, with the imbalance of head and tail classes. The paper introduces a long-tail learning paradigm based on diverse set of experts and hypernetworks. The proposed method can meet personalized user preferences and can adapt to wide range of distribution scenarios. The paper proposes this problem as multi-objective optimization, with the goal to learn whole pareto front. The authors also propose theoretical aspects considering distribution shifts and show that diversity experts methods learns a set of experts to capture the distributional characteristics of different environments, hence reducing the distribution discrepancy between the training and test environment.", "questions": "1. The paper mentions the proposed method to be interpretable and controllable long-tail learning method whereas I don’t think a model with an ability of adapting to  preference vectors can be considered interpretable. More explanation of this could be helpful to understand why authors believe the interpretability of the model. \n\n2. While the main problem is centered around preference based long-tail learning method, theoretical proofs just talking about distribution shift is not fitting very well. While the proofs look correct (to the best of my knowledge) and I agree that multiple experts can reduce the distribution discrepancy between training and test environment, why will this be useful for long-tailed distribution is not clear to me. Is there any explanation on why all the experts won’t still focus on head classes ?\n\n3. Additional information about how unknown test class distribution is created would be helpful.\n\n4. How preference vector for testing has been decided ? \n\n5. Hypervolume calculation usually needs information on reference vector, how is that taken into consideration here ?\n\n6. While the proposed method learns entire front in a single model, are the competing methods trained multiple models to cover the pareto front ? \n\n7. What are your comments on scalability of the proposed method for large target networks ?", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses an important problem of learning long-tailed distribution.\n2. The idea introduced based on diverse set of experts using hypernetwork, which can adapt to meet personalized user preferences sounds reasonable. \n3. The paper writing seems clear and well written. \n4. The extensive experiments demonstrate the proposed method performs better than the baseline. \n5. The paper covers theoretical aspects of distribution shifts as well.", "weaknesses": "1. The paper mentions the proposed method to be interpretable and controllable long-tail learning method whereas I don’t think a model with an ability of adapting to  preference vectors can be considered interpretable. More explanation of this could be helpful to understand why authors believe the interpretability of the model. \n\n2. While the main problem is centered around preference based long-tail learning method, theoretical proofs just talking about distribution shift is not fitting very well. While the proofs look correct (to the best of my knowledge) and I agree that multiple experts can reduce the distribution discrepancy between training and test environment, why will this be useful for long-tailed distribution is not clear to me. Is there any explanation on why all the experts won’t still focus on head classes ?\n\n3. Chebyshev polynomial has suddenly been introduced in ablation study, a brief mention about it would be beneficial for the readers. \n\n4. There are works which use hypernetworks for MOO [1], I think inclusion of such works would be useful. \n\n5. Inclusion of proofs considering long-tailed distribution would have been more beneficial for this problem setup. \n\n6. Given that this work considers optimizing for the whole pareto front, mention of hypervolume values would be useful for future works in this direction. \n\nTypos \nLine 36: requirement. [16, 29, 36] -> requirement [16, 29, 36] \n\n[1] Navon, A., Shamsian, A., Fetaya, E. and Chechik, G., Learning the Pareto Front with Hypernetworks. In International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1 and Q1:** Thank you for your question. The four perspectives in the **public response section** are intended to answer this question. Please refer to the four perspectives in the public response due to space limitations.\n\n**W2 and Q2:** Thank you for your question. The explanation of this part in the original text is indeed brief. It is related to the loss functions that guide different experts. The specific answers have been mentioned in perspectives 2 and 3 of the public response. Here we summarize and will promptly add them to our paper:\n- This paper selects three experts with different loss functions: forward expert, uniform expert, and reverse expert. Their different objectives enable different experts to adapt to different class distributions. During training, by sampling weight vectors from the Dirichlet distribution and combining the outputs of the three experts with weighted averaging, the Dirichlet distribution parameters are adjusted to simulate different class distributions. By optimizing the hypernetwork to minimize the expected loss, a set of expert parameters that perform well under different weights is obtained, which is equivalent to minimizing the expected loss of the ensemble model on the test distribution. This adaptive ability enhances the robustness of the model, enabling it to cope with distribution shifts in real-world scenarios.\n\n**W3:** Thank you for your suggestion! In this paper, the Chebyshev polynomial we refer to for implementation uses the log-sum-exp function to replace the max function, which plays a smoothing role (i.e., Section 4.3 of the method). The idea mainly originates from STCH. In the context of long-tailed learning in this paper, its role is to dynamically adjust the importance of experts, thereby better handling the long-tailed distribution problem. We will add necessary explanations and citations to our paper.\n\n**W4:** Thank you for your comment. We will make proper citations in subsequent versions to facilitate a better understanding of this paper.\n\n**W5:** Your question is excellent. Although the focus of this paper is on solving more problems of real-world distribution shifts based on long-tailed methods, some proofs based on long-tailed settings are indeed still necessary. In addition to the response to W1 and W2, we will supplement the necessary explanations in our paper.\n\n**W6:** To facilitate subsequent work in this direction, we will briefly introduce this concept and include it in subsequent versions. Additionally, we have supplemented some content regarding hypervolume in our answer to your Q6. Please refer to it.\n\n**W7:** We will correct the errors in subsequent versions.\n\n**Q3:** To evaluate the robustness of the algorithm under unknown test class distributions, we refer to LADE and SADE and construct three types of test sets: uniform distribution, positive long-tailed distribution, and negative long-tailed distribution. The positive and negative long-tailed distributions contain multiple different imbalance ratios ρ. For ImageNet-LT, CIFAR100-LT, and Places-LT, the imbalance ratio is set to ρ ∈ {2, 5, 10, 25, 50}. For iNaturalist 2018, since each class has only 3 test samples, the imbalance ratio is adjusted to ρ ∈ {2, 3}. We will supplement the explanation in the revised version.\n\n**Q4:** The decision process for the preference vector during testing is as follows:\n- By adjusting the user-specified preference vector, the attention of the model between head and tail classes can be controlled, achieving flexible trade-offs to adapt to different application requirements. The preference vector during testing is calculated based on the pre-trained preference vector and the user-input preference vector. (*Due to space limitations, please refer to the answer to W4 for reviewer VYLm.*)\n\n**Q5:** In this method, the computation of hypervolume is mainly reflected in the following two aspects:\n\nThe method guides the computation and utilization of hypervolume by introducing reference vector information in both the training and testing stages. \n- During training, by sampling preference vectors from the Dirichlet distribution and using a hypernetwork to generate expert model parameters, sampling and modeling of the Pareto front are achieved. \n- During testing, by combining the pre-trained reference preference vector and the user-specified preference vector, the preference vector for testing is obtained, reflecting the localization and utilization of the hypervolume. \n\nThis setting of reference vectors provides flexibility, enabling the model to adapt to different task requirements.\n\n**Q6:** Regarding your question, I guess the competing methods may refer to some long-tailed learning baselines. \nAlthough some existing competing methods such as LADE and SADE also adopt the idea of multi-expert models, they have considered the trade-offs in long-tailed distributions and used multi-expert structures to some extent, allowing them to cover different regions of the Pareto front to a certain degree. However, the completeness and continuity of the coverage are difficult to guarantee, and controllability cannot be achieved. In contrast, this paper generates a continuous expert space through a hypernetwork, which can more comprehensively approximate and cover the Pareto front.\n\n**Q7:** To evaluate the robustness of the algorithm under unknown test class distributions, we refer to LADE and SADE and construct three types of test sets: uniform distribution, positive long-tailed distribution, and negative long-tailed distribution. The positive and negative long-tailed distributions contain multiple different imbalance ratios ρ. For ImageNet-LT, CIFAR100-LT, and Places-LT, the imbalance ratio is set to ρ ∈ {2, 5, 10, 25, 50}. For iNaturalist 2018, since each class has only 3 test samples, the imbalance ratio is adjusted to ρ ∈ {2, 3}. We will supplement the explanation in the revised version to improve the completeness and readability of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vTw1HMN3cY", "reviewer": "Reviewer_qXeH", "summary": "This paper addresses the crucial and challenging problem of long-tailed learning under distribution shifts between training and testing data, which is highly relevant to real-world applications. The authors propose a novel and insightful learning paradigm that aims to obtain a set of diverse expert classifiers to adapt to arbitrary test distributions while allowing flexible control of the trade-off between head and tail classes based on user preferences. This new perspective greatly expands the applicability and practicality of long-tailed learning methods, making it a significant contribution to the field.", "questions": "Please answer the questions in the Weaknesses, which helps me better understand the theoretical contribution of this paper.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1.\tThe theoretical contribution of this paper is to analyze the limitations of the traditional empirical risk minimization (ERM) method in dealing with distribution shifts, and to quantify the distribution differences using the concept of environmental total variation gap (ETVD). These theoretical analyses provide sufficient theoretical basis and motivation for the new diversity expert learning method proposed in this paper.\n2.\tThe proposed framework elegantly combines hypernetworks and Dirichlet distribution sampling to train multiple diverse experts. This innovative design empowers the model to learn adaptability to a wide range of distributions from a single training set.\n3.\tThe comprehensive experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed approach.\n4.\tThe work opens up new possibilities for applying long-tailed learning to a wider range of real-world scenarios where the test distribution is unknown or subject to change. The controllable trade-off based on user preferences enhances the interpretability and usability of the model, making it more adaptable to different application requirements.", "weaknesses": "1.\tI agree with the novelty of this paper. The method proposed by the authors brings a new research perspective to the field. My doubts lie in part of the theoretical interpretation. Regarding the role of the Dirichlet distribution in generating diverse experts, can the authors analyze the effect of the hyperparameters of the Dirichlet distribution on the performance of the algorithm?\n2.\tAs for the above question, I hope the author carries out some experimental analysis to support his theory.\n3.\tI think the explanation of Figure 2 seems a bit obscure, and the author would be better off giving a concise explanation.\n4.\tSome symbol errors need to be corrected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1:** You raised a very insightful question. The probability density function of the Dirichlet distribution is:\n\n$$f(x_1, \\ldots, x_K; \\alpha_1, \\ldots, \\alpha_K) = \\frac{1}{B(\\alpha)} \\prod_{i=1}^K x_i^{\\alpha_i - 1}$$\n\nwhere $\\alpha=(\\alpha_1,\\ldots,\\alpha_K)$ are the hyperparameters of the distribution, and $B(\\alpha)$ is the normalization constant.\n\nIntuitively, the hyperparameters $\\alpha$ of the Dirichlet distribution control the characteristics of the generated weight vector $x=(x_1,\\ldots,x_K)$:\n\nWhen $\\alpha_i>1$, the generated weight vector tends to take larger values in the $i$-th component; when $\\alpha_i<1$, the generated weight vector tends to take smaller values in the $i$-th component; when $\\alpha_i=1$, the $i$-th component of the generated weight vector follows a uniform distribution.\n\nTherefore, the value of the hyperparameter $\\alpha$ affects the diversity of the combination weight vectors generated by the controller network: when the values of all $\\alpha_i$ are large, the generated weight vectors tend to concentrate in the central region, reducing the diversity of expert combinations; when the values of all $\\alpha_i$ are small, the generated weight vectors tend to be dispersed in each corner, increasing the diversity of expert combinations; when the values of different $\\alpha_i$ differ greatly, the generated weight vectors will have obvious preferences in certain components, leading to an imbalance in expert combinations.\n\nIn the long-tailed context, this diversity and imbalance will further affect the model's performance:\n- Moderate diversity helps the model adapt to different test environments, but excessive diversity may lead to some extreme weight combinations, affecting the model's stability.\n- Reasonable imbalance helps the model focus on certain key experts, but excessive imbalance may cause some experts to be ignored, affecting the model's generalization ability.\n\nTherefore, choosing appropriate Dirichlet distribution hyperparameters is crucial for balancing the diversity and stability of expert combinations. We also report experimental results of other weight combinations in the paper. Thank you again for your question.\n\n**W2:** We have supplemented additional experiments in the Appendix. Please refer to the Appendix.\n\n**W3:** Figure 2 intuitively demonstrates the superiority of the method proposed in this paper in dealing with distribution shifts and flexibly controlling preferences. Figure 2 aims to illustrate two advantages of the proposed method: the ability to overcome distribution shifts and the flexibility of preference control.\n\nIn Figure 2, we use two three-dimensional coordinate systems. The first coordinate system represents the value space of the preference vector, with each dimension corresponding to a preference (such as preference for head classes, tail classes, or balance). The second coordinate system represents the model's performance on three distributions (forward50, uniform, and backward50) of the CIFAR100-LT dataset.\n\nThe dark plane in the figure represents the value plane corresponding to different preference vectors, while the outer surface represents the corresponding performance on the three distributions. The yellow points are the results of the SADE method. Since its preference is uncontrollable, the results of each run are random points, and they are all located below the purple plane of the method proposed in this paper, indicating that its performance is inferior to the method proposed in this paper (i.e., it is dominated in the Pareto optimal set).\n\nThis figure shows that the method proposed in this paper can cover unknown distributions without requiring additional training, and unlike previous methods, it can balance performance on different distributions by adjusting the preference vector. These two advantages will be further analyzed in the experimental section.\n\n**W4:** We will revise the errors mentioned by the reviewer in the formal version. Thank you for your reminder.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WjNlzLOTFQ", "reviewer": "Reviewer_VYLm", "summary": "This paper addresses long-tailed learning with a focus on tackling distribution shift and accommodating different user preferences for the trade-off between head and tail classes. The authors propose a method called PRL, which generates a set of diverse expert models via hypernetworks to cover all possible distribution scenarios, optimizing the model ensemble to adapt to any test distribution. Experiments across various long-tailed datasets validate the method's effectiveness.", "questions": "Please see above.", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This work is well-motivated. Addressing distribution shift and accommodating different user preferences for the trade-off between head and tail classes is highly practical.\n- The authors provide a range of experiments to demonstrate the method's effectiveness.", "weaknesses": "- The paper uses $\\theta$ and $\\phi$ in Figures 3 and 4, but they are not defined beforehand.\n- In Section 5.2, the paper investigates the model's performance under different preference values $R$ but does not explain their meaning.\n- In Section 4.4, it is unclear how the trained preference vector is obtained.\n- I am confused about how to ensemble the experts after training is complete.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1:** Thank you for your correction. $R = (\\theta, \\phi)$ represents our preference vector, where $\\phi$ is the radian representation mentioned in Equation (14). Figures 3 and 4 depict the control of different preference vectors on model performance.\n\n**W2:** $R = (\\theta, \\phi)$ represents our preference vector, where $\\phi$ is the radian representation mentioned in Equation (14). Thank you for your correction. We will add revisions in subsequent versions to improve readability.\n\n**W3:** Thank you for your question. During the training process, the preference vector is obtained through training as a learnable parameter. During the testing process, the preference vector is jointly determined by the learned preference vector from the training stage and the input preference vector according to Equation (14). For the specific process, please refer to the answer to your W4.\n\n**W4:** In the training stage, we optimized the joint loss function of all experts using the Stochastic Convex Ensemble (SCE) strategy, obtaining a set of well-trained expert models and a hypernetwork. After training is completed, we can dynamically integrate these expert models based on the user-specified test preference vector to adapt to different testing scenario requirements. The specific steps are as follows:\n1. Calculate the preference vector for testing:\n$$\\hat{\\mathbf{r}} = (\\mathbf{r} \\odot \\mathbf{\\alpha^*}) \\oslash (\\mathbf{r} \\cdot \\mathbf{\\alpha^*})$$\nwhere r is the training preference vector, ⊙ represents the Hadamard product, ⊘ represents element-wise division, and · represents the dot product.\n2. Input $\\hat{\\mathbf{r}}$ into the hypernetwork to generate the test-time classifier head parameters for each expert:\n$$\\hat{\\mathbf{W}}_ i = h_\\psi(\\hat{\\mathbf{r}}), \\forall i \\in \\{1, \\dots, T\\}$$\n3. For a test sample x, the output of the $i_{th}$ expert can be calculated using the following formula:\n$$\\hat{\\mathbf{y}}_i = \\hat{\\mathbf{W}}_i^\\top \\mathbf{x} + \\hat{\\mathbf{b}}_i \\quad \\text{or} \\quad \\hat{\\mathbf{y}}_i = (\\hat{\\mathbf{W}}_i/\\|\\hat{\\mathbf{W}}_i\\|_F )^\\top (\\mathbf{x}/\\|\\mathbf{x}\\|_2)$$\n4. The final ensemble output can be obtained by a weighted combination of all expert outputs. The weights can be uniform or set according to the performance of experts on the validation set.\nBy adjusting the test preference vector, we can dynamically change the ensemble weights of the expert models, achieving flexible trade-offs between head and tail classes to meet different application requirements.\n\nWe appreciate the reviewer for pointing out this shortcoming. If there are any further questions or suggestions, we welcome discussion at any time. Thank you again for the valuable comments from the reviewer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ou65Qb6vYb", "reviewer": "Reviewer_Vosp", "summary": "This paper aims to overcome potential distribution shifts from a single training distribution to any testing distribution and adapt to different user preferences for trade-offs between head and tail classes. The proposed method leverages hypernetworks to generate a diverse set of expert models, enabling the system to adapt to any test distribution and cater to user-specific preferences. Extensive experiments demonstrate the method's superiority in performance and adaptability, providing new insights and expanding the applicability of long-tailed learning in practical scenarios.", "questions": "Please refer to the questions in the weakness section.", "limitations": "No.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Addressing the issue of test distribution invariance and incorporating diverse user preferences for trade-offs between head and tail classes is both meaningful and practical.\n2. The paper offers theoretical foundations and practical results.\n3. Experiments validate the effectiveness of the proposed method.", "weaknesses": "1. Some important concepts require further explanation. For example, what is the meaning of \"environment\" in Sec.3?  It is unclear why to minimize empirical risks across multiple training environments since the training set is typically fixed. Besides, I wonder how to ensemble the experts when the test distribution varies. It seems that the authors did not explain this key issue. \n2. In Section 5.1, the paper states \"use $\\alpha = 1.2$ for the Dirichlet distribution\". However, according to Eq.(5), $\\alpha$ should be a vector. It is recommended to provide a more detailed explanation.\n3. Some more competitive baselines such as PaCO$^1$, DDC$^2$, and DirMixE$^3$ should be considered for comparison.\n4. There are some typos, such as in line 36, where \"Simply pursuing the overall optimal solution may not meet this flexibility requirement. [16, 29, 36]\" should be \"Simply pursuing the overall optimal solution may not meet this flexibility requirement [16, 29, 36].\" In line 95, $\\pi_m^k$ should be $\\pi_k^m$.\n5. Some figures, such as Figure 1 and Figure 3, are not referenced in the text.\n\n------\n\n$^1$ Parametric contrastive learning, ICCV 2021 \n\n$^2$ A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning, NeurIPS 2023\n\n$^3$ Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition, ICML 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1:**\nThank you for raising the question. Let me try to further clarify some key concepts based on the content of the paper:\n- **Concept of environment:**\nIn Section 3, \"environment\" refers to a dataset with different class prior probability distributions. Each environment Em has its own class prior probability vector πm. Traditional empirical risk minimization (ERM) methods only train on a single training distribution, making it difficult to handle distribution differences between environments. Therefore, we propose to minimize the empirical risk in multiple training environments to learn a diverse set of experts that capture the distributional characteristics of different environments.\n- **Motivation for minimizing empirical risk in multiple training environments:**\nAlthough the training set is usually fixed, the authors believe that by constructing multiple training environments with different class distributions, the distribution shifts during testing can be better addressed. By learning a set of experts on these environments, the model can capture the characteristics of data under different long-tailed distributions, thereby gaining stronger generalization ability and distribution adaptability. This approach can reduce the distribution difference between the training environments and the testing environment.\n- **How to integrate experts during testing:**\nDuring testing, we introduce a preference vector α to control the trade-off between head and tail classes. Given the well-trained preference vector r, the authors calculate the preference vector r for testing and input it into the hypernetwork to generate the classifier head parameters for each expert. By adjusting α, the model's attention to head or tail classes can be controlled, achieving flexible trade-offs. You can refer to the answer to W4 for reviewer VYLm.\n\n**W2:** Thank you very much for the reviewer's question. Our expression in the paper may not be clear enough, leading to some misunderstandings. Please allow me to provide a detailed explanation here.\n\nIn our method, we made a special setting for the hyperparameters α of the Dirichlet distribution. Specifically, we set α to a vector of length 3, where each component is equal to 1.2, i.e., α=[1.2, 1.2, 1.2]. This means that we are actually sampling independently from three Dirichlet distributions with the same parameters, i.e., [Dirichlet(α), Dirichlet(α), Dirichlet(α)].\n\nThis setting is quite common in the literature related to hypernetworks and multi-objective optimization. For example, in the paper [reference number] that we refer to, the authors also adopt a similar approach, setting the hyperparameters of the Dirichlet distribution to a vector with the same elements. Additionally, we explored using different hyperparameter settings and reported them in the main text. To avoid similar confusion, we will provide a clearer explanation of this point in the revised version. \n\nThank you again for the reviewer's careful review and valuable comments, which help us further improve the quality and readability of the paper.\n\n**W3:** We have supplemented the necessary baseline comparisons in the appendix and will include them in the official version. Thank you for your suggestion.\n\n**W4:** Thank you for your comment. We will correct these errors in the official version.\n\n**W5:** Thank you for your attention to detail! We will add these references in the official version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate all the reviewers for their valuable comments. Your feedback has helped us improve the quality of the paper and strengthen the arguments. We are pleased that most reviewers have a positive attitude towards our work :). 

We are very grateful to the reviewers for acknowledging **our efforts in addressing important and challenging problems in long-tailed learning (Reviewers LGeA, qXeH, Vosp)**. We are also glad that they recognized **our innovations in theory and methods (Reviewers qXeH, Vosp), our comprehensive experiments (Reviewers LGeA, qXeH, VYLm, Vosp), and clear writing (reviewer LGeA)**. We have made our best efforts to respond to each suggestion, and we are very happy to further communicate if the reviewers have any questions. In addition, to deepen the reviewers' understanding of this paper, we will explain the interpretability and controllability of our method from the following perspectives:

**Perspective 1: Multi-expert structure enables different experts to focus on different classes**

This paper employs the same three experts with different loss functions as SADE: the forward expert tends to adapt to the long-tailed distribution similar to the training set. The uniform expert tends to adapt to the distribution with balanced classes. The reverse expert tends to adapt to the reverse long-tailed distribution with fewer head classes and more tail classes.

Through theoretical analysis, we can obtain the optimal solution for each expert, which corresponds to different class distributions:

- Forward expert:
$v_{1}^{*} (x) = \arg \max_{v_{1}} \mathbb{E}_ {(x,y) \sim p_{train}(x,y)} [ \log p(y\|x; v_{1}) ]$

- Uniform expert:
$v_2^*(x) = \arg\max_{v_2} \mathbb{E}_ {(x,y) \sim p_{uniform}(y)p_{train}(x|y)} [\log p(y|x; v_2)]$

- Reverse expert:
$v_3^*(x) = \arg\max_{v_3} \mathbb{E}_ {(x,y) \sim p_{inv}(y)p_{train}(x|y)} [\log p(y|x; v_3)]$

These results demonstrate that the multi-expert structure indeed enables different experts to focus on different classes, providing a foundation for the model to adapt to different distributions.

**Perspective 2: Dirichlet sampling and hypernetwork enable the model to adapt to different class distributions**

During the training process, we sample the weight vector $\boldsymbol{\alpha}=(\alpha_1, \alpha_2, \alpha_3)$ from the Dirichlet distribution $p(\boldsymbol{\alpha}; \boldsymbol{\beta})$, and weight the outputs of the three experts:

$$v(x) = \sum_{i=1}^3 \alpha_i v_i(x)$$

By adjusting the parameters of the Dirichlet distribution $\boldsymbol{\beta}$, we can control the distribution of the sampled weight vector to simulate different class distributions:
- When $\beta_1 > \beta_2 = \beta_3$, simulate the long-tailed distribution.
- When $\beta_1 = \beta_2 = \beta_3$, simulate the uniform distribution.
- When $\beta_1 < \beta_2 = \beta_3$, simulate the reverse long-tailed distribution.

Moreover, we introduce a hypernetwork $h_{\boldsymbol{\phi}}(\boldsymbol{\alpha})$ to map the sampled weight vector to the parameters of the experts:

$$\boldsymbol{\theta}_ {i} = h_{\boldsymbol{\phi}}(\boldsymbol{\alpha})_i, \quad i = 1, 2, 3$$

By optimizing the hypernetwork to minimize the expected loss:

$$\min_{\boldsymbol{\phi}} \mathbb{E}_ {\boldsymbol{\alpha} \sim p(\boldsymbol{\alpha}; \boldsymbol{\beta})} \left[ \frac{1}{n_s} \sum_{(x, y) \in D_s} \ell(y, v(x; h_{\boldsymbol{\phi}}(\boldsymbol{\alpha}))) \right]$$

We can obtain a set of expert parameters that perform well under different weight vectors. From an optimization perspective, this is equivalent to minimizing the expected loss of the ensemble model on the test distribution:

$$\min_{\boldsymbol{\phi}} \mathbb{E}_ {\boldsymbol{\alpha} \sim p(\boldsymbol{\alpha}; \boldsymbol{\beta})} \left[ \mathbb{E}_ {(x, y) \sim p_{test}(x, y)} \left[ \ell(y, v(x; h_{\boldsymbol{\phi}}(\boldsymbol{\alpha}))) \right] \right]$$

This shows that through Dirichlet sampling and hypernetwork optimization, we can obtain an ensemble model that adapts to various class distributions and performs well under different distributions. This adaptive ability enhances the robustness of the model, enabling it to cope with distribution changes in real-world scenarios.

**Perspective 3: Learning the entire frontier through Pareto optimization**

When optimizing the hypernetwork, we are actually learning the entire Pareto frontier. This is because we optimize the expected loss of the ensemble model under the weight vector sampled from the Dirichlet distribution. Different weight vectors correspond to different points on the frontier, representing different class distributions. By minimizing the expected loss, we are essentially balancing all these distributions and learning the entire Pareto frontier.

Theoretically, if a solution $\boldsymbol{\theta}(\boldsymbol{\alpha})$ is a local Pareto optimal solution to the expected loss optimization problem, then in a neighborhood of $\boldsymbol{\alpha}$, we can find a smooth mapping $\boldsymbol{\theta}(\boldsymbol{\alpha})$ that is Pareto optimal in the entire neighborhood. This result supports that our method can learn a set of continuous Pareto optimal solutions covering the entire Pareto frontier, providing flexibility to adapt to different practical needs.

**Perspective 4: Advantages over previous methods**

Compared with previous long-tailed learning methods, our method has several advantages in interpretability and controllability:
- Through the multi-expert structure and hypernetwork, our method can adapt to different class distributions.
- By learning the entire Pareto frontier, our method can control the behavior of the model by adjusting the weight vector to achieve dynamic adaptation to different class distributions. This controllability is not available in previous methods.
- We provide some relatively rigorous and coherent theoretical analyses to support the effectiveness of the method and enhance the interpretability of the model.

Meta-review: This paper proposes a novel long-tailed learning method. It can generate a set of models to cover all distributions, and flexibly output a model that matches the user's preference. Empirical results validate the effectiveness of the proposed method.

The studied problem is an important and practical problem. The paper provides meaningful theoretical results and motivation for the new diversity expert learning method. The proposed method is novel enough. The paper provides an extensive set of experiments to support the claims.

In addition, the authors can further improve the paper according to the detailed reviews.
Decision: Accept (spotlight)