GUIDE: Gated Uncertainty-Informed Disentangled Experts for Long-tailed Recognition
Authors: Yuan Dong
Zhe Zhao
Liheng Yu
Di Wu
Pengkun Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Long-Tailed Recognition
Multi-Expert Learning
Hierarchical Disentanglement
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: GUIDE disentangles representation, policy, and optimization in multi-expert LTR, achieving stable training and SOTA performance.

Abstract: Long-Tailed Recognition (LTR) remains a significant challenge in deep learning. While multi-expert architectures are a prominent paradigm, we argue that their efficacy is fundamentally limited by a series of deeply entangled problems at the levels of representation, policy, and optimization. These entanglements induce homogeneity collapse among experts, suboptimal dynamic adjustments, and unstable meta-learning. In this paper, we introduce GUIDE, a novel framework conceived from the philosophy of Hierarchical Disentanglement. We systematically address these issues at three distinct levels. First, we disentangle expert representations and decisions through competitive specialization objectives to foster genuine diversity. Second, we disentangle policy-making from ambiguous signals by using online uncertainty decomposition to guide a dynamic expert refinement module, enabling a differentiated response to model ignorance versus data ambiguity. Third, we disentangle the optimization of the main task and the meta-policy via a two-timescale update mechanism, ensuring stable convergence. Extensive experiments on five challenging LTR benchmarks, including  ImageNet-LT, iNaturalist 2018, CIFAR-100-LT, CIFAR-10-LT and Places-LT, demonstrate that GUIDE establishes a new state of the art, validating the efficacy of our disentanglement approach. Code is available at Supplement.

Published as a conference paper at ICLR 2026
GUIDE: GATED UNCERTAINTY-INFORMER DISEN- TANGLED EXPERTS FOR LONG-TAILE RECOGNITION
Yuan Dong', Zhao Liheng Yu', Di Wu', Pengkun Wangl.2 University of Science and Technology of China (USTC), Hefei 230026, China
Suzhou Institute for Advanced Research, USTC, Suzhou 215123, China
CCity University of Hong Kong
dongyuan 22122220 gma com, {zz4543, yuliheng, wdcxy)email ustc. edu. cn pengkun@usto edu.cn cn
ABSTRACT
Long-Tailed Recognition (LTR) remains a significant challenge in deep learning. While multi-expert architectures are prominent paradigm, we argue that their efficacy is fundamentally limited by series of deeply entangled problems at the levels of representation, policy, and optimization. These entanglements induce homogeneity collapse among experts, suboptimal dynamic adjustments, and un- stable meta-learning, In this paper, we introduce GUIDE, a novel framework con- ceived from the philosophy of Hierarchical Disentanglement. We systematically address these issues at three distinct levels. First, we disentangle expert repre- sentations and decisions through competitive specialization objectives to foster genuine diversity. Second, we disentangle policy-making from ambiguous sig- nals by using online uncertainty decomposition to guide dynamic expert re- finement module, enabling a differentiated response to model ignorance versus data anbiguity. Third, we disentangle the optimization of the main task and the meta-policy via two-timescale update mechanism, ensuring stable convergence. Extensive experiments on five challenging LTR benchmarks, including ImageNet- LT, iNaturalist 2018, CIFAR-100-LJ) CIFAR-100 10-LT and Places-LIT, demonstrate that GUIDE establishes a new state the art, validating the efficacy of our disen- tanglement approach.
INTRODUCTION
Multi-expert architectures are the state-of-the-art paradigm in Long-Tailed Recognition (LTR), con- sistently advancing performance real-world datasets with power-law class distributions (Wang et al. 2021 Zhang et al. 2022; Sanchez Aimar et al. 2023 Liu et al. 2019). The core intuition is
that committee of specialized experts collectively provide more robust coverage of the entire
data distribution, from head to tail (Shazeer et al. 2017 Jacobs et al. 1991].
However, despite empirical success, this paradigm is approaching fundamental ceiling. We gue that progress is saturating due to a deep, unaddressed issue: a dependency chain of entangled learning processes that creates systemic performance bottlenecks. This chain begins with founda- tional failure in representation learning that propagates to cripple the system's adaptive policy and destabilize its optimization.
The foundational problem is representaaioon-deciiioon entanglement. Existing methods often induce diversity through indirect means at the decision level, for instance using varied logit adjustments (Meon et al. 2021 Cao et al.] (2019). Crucially, they do not explicitly decouple the representation learning process itself from the dominating influence of head classes (Kang et al., 2020). This allows powerful gradients from these classes to foster homogeneity collapse, where all experts converge towards a similar, head-centric feature space (Kornblihh et al.) 2019 Morcos et al. 2018 Shen al.) 2015). This collapse renders specialization ineffective and forms the root of subsequent failures.
Work done during a research internship at USTC-DILab.
"orresponning authors.

---

Published as a conference paper at ICLR 2026
(a) Entangled Multi-Expert Model
(b)
GUIDE Framework
Representation
Shared Model
Shared Model
Representation
Head
Tail
Head
Tail
Classes
Classes
Classes
Classes
Entanglement
Gradient Dominance
Dynomic adjustment
Broadeast
Experts
Broudcast
Experts
Decision
Feature
Decision
Seporation
Separation
Decision
Homagenized Experts
High loss
Average
Sample
Expert Losses
High loss
Average Expert Losses
Epistemic
Aleatoric
Sample
Error learning
Uncertainty
strategy
Figure 1: (a) Conventional entangled multi-expert systems suffer from homogeneity collapse due to shared representations and simplistic error-driven adaptation, leading to flawed, cause-agnostic handling of difficult samples. (b) Our GUIDE framework breaks this entanglement by enforcing explicit feature and decision separation. This enables principled, diagnostic approach where an epistemic / aleatoric uncertainty analysis guides the targeted allocation of learning resources.
This representational collapse directly compromises the system's adaptive capabilities, leading to cause-symptom entanglement. A committee of functionally similar experts lacks the diverse per- spectives needed for reliable diagnosis of difficult samples. Advanced LTR methods thus rely on ambiguous signals like high training loss, conflating the symptom with its cause. This cause can be model ignorance (epistemic uncertainly) or inherent data ambiguity (aleatoric uncertainty) (Kendall & Gal 2017 Lakshminarayanan et al.) 2017 Shen et al. (2018). Without genuine expert diversity, distinguishing these becomes unreliable, leading to a chronic misallocation of model resources.
Finally, the instability the representation and policy levels creates a volatile optimization land- scape, culminating in learning meta-learning entanglement. The slow, deliberate optimization of an adaptive strategy is fundamentally odds with the fast, high-variance optimization of the pri- mary recognition task. Since this primary task already operates on collapsed representations and guided by a flawed adaptive policy, its high-variance gradients overwhelm the subtle updates quired to refine the meta-policy. This pervasive instability prevents the system from converging to robust and effective self-organizing strategy (Heusel al. 2017).
To resolve these interconnected entanglements in unified manner, we propose GUIDE, a Gated Uncertainty-Inforrmed Disentangled Experts framework built upon the principle of hierarchical dis- entanglement. As illustrated in Figure I GUIDE re-architects the learning process to systematically decouple these coupled mechanisms, addressing each level the dependency chain in order.
First, to establish stable foundation at the representation ation level, we introduce competi- tive specialization objectives. This active pressure forces experts into distinct functional niches by decorrelating features and maximizing predictive divergence (Krogh & Vedelsby 1994] Liu & Yao, 1999]. The approach is theoretically grounded tightening the ensemble's loss bound and effec- tively prevents homogeneity collapse (Wang et al. 2017 2025). Building this foundation of diverse experts, GUIDE can perform reliable diagnosis at the policy level. We shift from reactive adaptation to principled, diagnostic intervention. Our framework learns differentiated policy by decomposing predictive uncertainty into its epistemic and aleatoric components. dynamic expert refinement module then allocates refinement capacity based robust assessment of model igno-
2

---

Published as a conference paper at ICLR 2026
rance, ensuring targeted learning (Gao et al. 2017). Finally, the stability from the disentangled representation and policy levels creates protected optimization landscape. At the opt imi zat ion level, a two-timescale update rule decouples the fast primary learning from the slow meta- meta-learning. This ensures the stable convergence of strategic adjustments (Heusel et al. 2017).
We summarize contributions follows:
New problem and insight: we identify a dependency chain of fundamental entanglement problems long-tailed recognition and propose GUIDE, a novel hierarchical disentanglement framework to systematically resolve it.
New learning framework: faanniiwkraworrr
we design suite of three interconnected mechanisms: a competitive specialization framework to build diverse representational foundation, an uncertainty-guided adaptation policy enabled by this diversity, and a two-timescale optimization process made stable by the preceding levels.
Compelling empirical results: extensive experiments five challenging long-tailed bench- marks, ImageNet-LT, iNaturalist 2018, CIFAR- CIFAR-IO-LT and Places-LT, establish new state of the art for GUIDE, validating our systemic disentanglement approach.
RELATED WORK
Multi-Expert Long-Tailed Learning and Representation Collapse. Multi-expert LTR methods
Wang et al., 2021 Zhang et al.) [2022] Sanchez Aimar et al. 2023) encourage diversity indirectly at the decision level, leaving representations vulnerable to homogeneity collapse. This representation- decision entanglement, driven by head-class gradients, forces experts into correlated feature spaces and limits their collective capacity. GUIDE directly resolves this by enforcing competitive special- ization both feature and decision levels, simultaneously decorrelating features and maximizing predictive divergence to learn complementary experts. Relatedly, MDCS (Zhao et al. (023) also promotes expert diversity using consistency self-distillation, though our hierarchical approach ad- dresses broader cascade of entanglements, from representation optimization.
Uncertainty-Guidee Adaptation. Adaptive LTR methods often suffer from cause cuuse-symptom en- tanglement, reacting to ambiguous signals like high training loss with suboptimal, one-size-fits-all adjustments. While prior work uses uncertainty for auxiliary tasks (Chen & Su 2023 Yang et al. 2024), GUIDE is the first to our knowledge to leverage decomposed epistemic (model) and aleatoric (data) uncertainty for a differentiated, structural adaptation policy. This enables targeted refinement for model ignorance (high epistemic) while promoting robustness against data noise (high aleatoric).
Meta-Optimization in LTR. Meta-learning for LTR (Ren et al. 2018 Shu et al. 2019) faces optimization instability due interference between the fast primary task and the slow meta-policy optimization. This learning and meta-learning entanglement hinders convergence. GUIDE resolves this using Two-Timescall Stochastic Approximation (TTSA) scheme (Borkar 1997). using differ- ential learning rates to decouple the two optimization loops and ensure stable policy convergence.
3 GUIDE: AN EFFECTIVE HIERARCHICAL DISENTANGLEMENT METHOD
The GUIDE framework is designed systematically resolve the entanglement problems in long- tailed learning through Hierarchical Disentanglement approach. We decompose the learning pro- cess into three distinct levels, each targeting a specific entanglement respects their underlying dependencies. The overall architecture is illustrated in Figure 2 Below, we detail the mechanisms at each level, motivating their design and explaining their synergistic roles. Our framework is built upon principled design, theoretically motivated by established concepts in ensemble learning and optimization theory. Rather than proposing new foundational theorems, our contribution lies in demonstrating how synergistic application of these principles can resolve the core entanglements in LTR, with our design choices being validated through extensive empirical analysis.
sequential manner that
3

---

Published as a conference paper at ICLR 2026
Level 1
Input 4 Backbone
Level
Epistemic Uncertainty
Feature Seporation
v Experts
Output Decomposer
Uncertainty Decomposer
Decision Separation
O
Aleatoric Uncertainty
8
DREM
Competitive
Specialization Loss
Shared Base Pathway
Expert Refine Pathway
Gate Controller
Diff
Level
Avg Expert Loss
Gate Modulation
Slow Meta-Policy yp
Fast Task Optimizer
Refined
Outer Loop Update y
Inner Loop Update 0
Feature
Figure 2: The Hierarchical Disentanglement architecture of GUIDE. An input is processed via shared backbone and a committee of experts. Level enforces diversity by penalizing feature and decision overlap. This enables Level @, where expert disagreement is decomposed into epistemic (model) and aleatoric (data) uncertainty. These signals drive a gate controller to modulate the Dy- namic Expert Refinement Module (DERM). Finally, Level @ decouples optimization into fast inner loop for task parameters (0) and a slow outer loop for the meta-policy (Φ), which closes the meta-learning loop by updating the gate controller.
3.1 LEVEL 1: REPRESENTATION DISENTANGLEMEN VIA COMPETITIVE SPECIALIZATION
The foundational level of our framework confronts the representation- iinieettattoo-deciiioo decision entanglement caus- ing homogeneity collapse. In standard multi-expert systems, head-class gradients dominate, forcing all experts into a functionally redundant, head-centric feature space. Our core insight is that gen- uine diversity cannot be a byproduct; must be an explicit optimization objective. We introduce paradigm of competitive specialization, establishing dynamic equilibrium where experts collabo- rate while competing for distinct functional niches. This design is guided by Theorem
Theorem (Diversity-driven Bound Tightening). For ensemble of F. experts, (a) the ensemble's negative logelikel (NLL) is upper-boundee by the average of individual expert NLLs. (b) This performance gap is directly related to the predictive diversity among experts, which can be measured and optimized by maximizing the Jensen- -ansnn-Shanno Divergence (JSD).
A formal statement and proof provided in Appendix A.2 Theorem 1 connects the ensemble's
performance bound to its predictive diversity. While this bound is not an equality, it provides a JSD, standard measure of distribution diversity, an effective
strong theoretical motivation proxy objective for encouraging expert specialization. Motivated by this principle, our Level O objective integrates collaborative loss with two competitive regularizers. The main collaborative task is driven by a standard cross- entsspy loss, Lmain- on aggregated, logit-adjusted predictions. For a given input æ with true label y, the main loss over an ensemble of E experts is:
exp EEE
(z(2)
Lmuin
log
Σe exp EE 2e (x))
Here, the adjusted logit for class from expert is (z) ze (z) Te log (C).n(). where (x) the raw logit vector, Tirain() is the training prior, and т, is an expert-specific hyperparameter. Although this loss promotes collaboration, it is insufficient prevent homogeneity collapse,

---

Published as a conference paper at ICLR 2026
introduce two synergistic regularizers to actively enforce diversity. For representation diversity, minimize the cosine similarity between the feature vectors extracted by different experts:
Leccouple
2
Σ
f(() fx())
(2)
E(E-1
ISijIS [[(())l2 12)(0))2+
where fx(x) and (a) are feature vectors from experts i and j, and & a small constant for numeri- cal stability. For prediction diversity, we explicitly maximize the JSD of the predictive distributions from the experts. The final Level O objective orchestrates these collaborative and competitive forces:
Ganal
Cmain + Adcooooo A
(3)
The terms Adec and Aaiv are scalar weights that balance the three objectives, and pe.r(-)a::) is the temperature-scaled predictive distribution from expert e. By orchestrating these forces, Level O resolves the representation-dcciisoo entanglement. Its immediate effect is the cultivation of truly diverse ensemble. This diversity is the critical precondition for Level @, as it transforms the dis- agreement among experts from noise into reliable, high-fidelity diagnostic signal.
3.2 LEVEL @: POLICY DISENTANGLEMENT VIA ADAPTIVE META-LEARNING
Capitalizing on the functionally diverse ensemble forged Level we can now effectively address the cause-symplom entanglement. This problem arises from reacting to ambiguous signals like high loss, which conflates model ignorance (epistemic uncertainty) with data ambiguity (aleatoric uncer- tainty). Our solution is principled, two-stage process: first diagnose the true cause of difficulty, then execute a targeted intervention, with behavior guaranteed by Theorem 2
Theorem (Policy Monotonicity). The refinement strength of the Dynamic Expert Refinement Mod- ule (DERM) for given class is monotonically increasing function with respect model's epis- temic uncertainty and a monotonically decreasing function with respect to its aleatoric uncertainty.
Our implementation first performs online diagnosis. To achieve this, leverage the fighfiielity signal from our diverse experts and decompose the pre- dictive uncertainty by adopting a widely-used formulation from Bayesian deep learning (Kendall & Gal 2ak7:
A proof is provided in Appendix A.2
et al. (017). This approach provides clear and empirically effective distinction between model ignorance (epistemic) and data ambiguity (aleatoric), which is critical for our targeted intervention and validated in Section 4.3
1 F Aleatoric Uncertainty (Ale): Aler(x) = E H(per((11z)
Σ
(4)
Epistemic Uncertainty (Epi): Epir(z) = H((p((1))) Aler(x).
(5)
Here, H-)) denotes the Shannon entropy and (-1x) is the average predictive distribution. Subse- quently, the DERM executes the targeted policy. consists of foundatt pathway, Fround and expert-specific refinement pathways, Prefinees which are neural network modules designed to process the features extracted by the backbone. For a given class e, these are combined via an adaptive residual mixture:
(x; c) Frouna(x) + ge,e WeFefineeeeFtonnde - Fr))nd()))
(6)
The refinement strength gate ge is a learnable function of stable, class-level Exponential Moving Averages of the diagnosed uncertainties, denoted Epi (t) and Ale(!) for class at step t:
Je.c a. Epii.e Be AIe V
(7)
Je,e (āe.c) (gmax Imin) Imiu-
(8)
5

---

Published as a conference paper at ICLR 2026
In this formulation, fae, Tel are learnable parameters, a(-) is the sigmoid function, and Gmin: 9max are hyperparameters defining the gate's range. This mechanism resolves the cause- symptom entanglement. Its effect is to replace chaotic, error-driven reactions with principled allocation of model capacity. This targeted learning process inherently stabilizes the training dy- namics, creating a more tractable optimization landscape, which is the essential prerequisite for Level @ to successfully learn robust meta-policy.
3.3 LEVEL C OPTIMIZATION DISENTANGLEMENT VIA
UPDATES
Operating within the stabilized learning environment established by the preceding levels, we can finally confront the teerminn--etalllammin entanglement. This conflict occurs because the slow, deliberate optimization of the adaptive meta-policy is easily overwhelmed by the fast, high- Sigh-variance optimization of the primary task. Our solution explicitly decouple these processes by assigning them different timescales, a strategy grounded in Two-Timesen Timescale Stochastic Approximation (TTSA), as formalized in Proposition
Proposition 1 (Satisfaction TTSA Conditions). By partitioning the model parameters into fast variables 0 and slow variables é, and employing an update scheme where their respective learning rates satisfy ns & qo, our optimization process constitutes a valid TTSA algorithm.
A discussion provided in Appendix A.2 We realize this principle by partitioning the model's learnable parameters based on their function and required update speed. This results in two distinct sets of variables:
Fast Variables 9: Main network parameters for feature extraction and recognition, including the shared backbone and expert-specific DERM pathways (Fround. Freane,e).
Slow Variables d: The parameters {a.: Pes Yehe governing the Strategy Controllers, which constitute the adaptive meta-policy.
These sets are updated on their respective timescales. The inner loop (fast timescale) updates the main network parameters each training step & with a learning rate rjo:
0a-1 = 0g - no & (Tk,, yas Oz. d.).
Conversely, the outer loop (slow timescale) updates the meta-policy controllers e less frequently (every epoch t) with a smaller learning rate 77. optimizing performance on validation set V:
Φ = Φ.
(e (fmmmi (No, Yos Oz, dx))-
(10)
This principled decoupling is the capstone of our hierarchical approach. Having first established diverse ensemble and then a stable, diagnostic policy, the optimization landscape is now sufficiently well-behaved for meta-learning algorithm to operate effectively. The effect of Level Ø is to create a protected optimization channel, allowing the meta-policy tanglements in a specific, hierarchical order, GUIDE guides the entire framework towards a robust, self-organizing equilibrium, fully unlocking the potential of the multi-expert paradigm.
safely converge. By solving the en-
4
EXPERIMENTS
4.1
EXPERIMENTAL SETUPS
Datasets. We validate our method on a suite of five widely-used benchmarks. Our primary analysis is conducted on four challenging datasets: CIFAR-100-LT (Cao et al., 2019). ImageNet-LT [Lu et al. 2019), Places-LT (Liu et al. 2019), and iNaturalist 2018 (Van Horn et al. 2018). which cover wide range of scales and imbalance ratios. To further demonstrate general applicability, full set of results on a fifth benchmark, 1IFAR-IO-LT (Cao et al. 2019), is provided in Appendix B.3 Detailed statistics for all five datasets are in Appendix
We compare GUIDE against comprehensive set of state-of-the-art methods. Our comparison strategy is principled and transparent. We primarily report results from original pub- lications to ensure fidelity. For methods lacking results relevant benchmark, we reproduced
Baselines.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of Top-1 accuracy (%) with state-of-the-art methods. Dashed underline indi- cates results reproduced by using the official codebases of the respective authors. Best results are in bold, second best are underlined.
CIFAR-I00-LT
Method
ImageNet-LT
iNaturalist 2018
Places-LT
IR=10
IR=50
IR=100
Softmax
59.1
45.6
41.4
48.0
64.7
31.4
LDAM-DRW Cao al.
2019
58.7
46.6
42.0
48.8
64.6
36.8
Balanced Softmax [Ken
2020
61.0
50.9
46.1
52.3
70.6
39.4
LADE (Hone al. 2021
61.6
50.1
45.6
52.3
69.3
39.2
MiSTAS Zhong al. 2021
62.5
51.5
46.8
51.4
70.7
38.3
RIDE (3 experis) Wang
2021
61.8
51.7
48.0
56.3
71.8
40.3
SADE Zhang el al. 6022
63.6
53.8
48.8
58.8
72.7
40,9
BalPoF (Sanchez Aimar of al.
2023
64.8
56.3
52.0
59.3
75.0
40.8
MDCS (Zhan et al. 2023
57.2
53.2
$9.3
72.5
42,4
LSC (Wor al 2024
65.0
56.5
51.8
60.2
73.9
41.3
BCL
Zhu ct al
2022
64.9
56.6
51.9
56.0
71.8
PRI.
7Zha et al.
2024
65.6
57.3
52.8
60.8
75.1
41.6
ProCo (D
2024
65.5
57.1
57.3
73.5
ConCutmix Pan al.
2024
64.5
57.4
58.5
72.1
(Sun 2025
$4.4
70.8
FeatRecon Yi
2025
65.3
57.0
52.5
56.8
72.9
41.3
GUIDE
69.2
60.3
56.4
62.5
76.1
42.2
Longer training
PaCo (Cui et al. 2021
64.2
57.0
73.2
SADE Zhang et al. 2022
65.3
74.0
39.5
NCL (Li el al.. 2022
$9.5
76.9
BCL. Zhu al. 2022
66.9
58.6
53.9
59.0
74.8
BalPol: (Sanchoz
2023
68.1
60.1
55.9
59.7
76.9
MDCS (Zhao
60.1
56.1
60.7
75.6
ProCo (Du al. 2024)
67.8
58.9
54.2
58.5
75.6
ConCutmux (Pn
2024
68.3
59.5
55.1
58.9
75.7
FeatRecon ei
67.2
58.5
54.0
3593
73.9
GUIDE
70.2
62.3
63.4
77.8
Table 2: Detailed performance breakdown (%) on CIFAR-I00-LT with ResNet-32. Our method shows substantial gains on Few-shot classes, addressing the core challenge of L.R..
CIFAR-100-LT (IR=100)
CIFAR-I00-LT (IR=50)
Method
Many
Medium
Few
Overall
Many
Medium
Few
Overall
LDAM-DRW (Cao et al. 2019
61.2
39.5
14,4
42.0
64.9
45.1
28.7
46.6
RIDE (3 experts) Wang et al. 2021
64.7
46.5
22.3
48.0
67.5
50.8
27.9
51.7
ZZang et al. 2012
63.4
49.2
25.1
49.8
67.2
53.5
31.8
53,9
BalPoE (Sanchez Aimar el al. 2023
65.3
51.1
28.0
$2.0
69.2
55.9
34.5
56.3
(Zhao al | 2024
68.7
55.3
31.2
52.8
67.5
53.5
39.0
56.6
GUIDE
71.3
59.1
36.0
56.4
70.2
56.1
47.4
60.3
GUIDE (longer)
74.4
60.0
36.1
$7.7
72.4
57.5
50.1
62.3
their performance using official codebases. These reproduced results are clearly marked in main comparison table. detailed justification for each reproduced result are provided in Appendix B.2
Implementation and Evaluation Details. To ensure a fair comparison, we employ standard backbones for all experiments: ResNet-32 for LIFAR-100-13 and CIFAR-IO-LT, ResNet-50 for ImageNet-LT and iNaturalist 2018, and ResNet- 152 for Places-LT. We report Top- 1 accuracy on the official balanced test sets, supplemented by fine-graine grained analysis Many, Medium, and Few-shot splits. All models are trained from seratch using an SGD optimizer under both standard and longer training schedules. Full implementation details provided in Appendix B.

---

Published as a conference paper at ICLR 2026
Table 3: Top-1 accuracy (%) on CIFAR-1O0-LT (IR=100) with various unknown test class distri- butions. 'Prior' indicates whether the method uses the test class prior for post-hoc adjustment. All models are from the standard training schedule.
Forward-LT
Uni.
Backward-LL
Method
Prior
25
10
2
10
25
50
Softmax
X
63.3
62.0
56.2
52.5
46.4
41.4
36.5
30.5
25.8
21.7
17.5
MiSLAS Zhhong et al
2021
X
58.8
57.2
55.2
53.0
49.6
46.8
43.6
40.1
37.7
33.9
32.1
LADE
Hong et
2021
X
56.0
55.5
52.8
51.0
48.0
45.6
43.2
40.0
38.3
35.5
34.0
LADE
Hong
62.6
60.2
55.6
52.7
48.2
45.6
43.8
41.1
41.5
40.7
416
RIDE
Wane et al.
2021
X
63.0
59.9
57.0
53.6
49.4
48.0
42.5
38.1
35,4
31.6
29.2
SADE Zhang ct 2022
X
65.2
62.5
58.8
55.4
$1.2
48.8
43.0
43.9
42.4
42.2
42.0
BalPolt Sanche/ Atmar
61.2
59.0
45.7
45.2
LSC
(Wei 2024
67.8
64.2
60.2
58.1
53.2
51.6
44.7
45.7
44.2
44.7
48.0
PRI. (Zhao of al 2024
X
69.5
65.7
61.7
59.5
54.7
52.2
46.2
47.1
45.7
45.7
48.5
GUIDE
X
68.5
66.7
64.4
62.6
59.2
56.4
53.3
49.7
46.7
43.2
40.7
GUIDE (longer)
x
71.9
70.0
67.3
64.7
60.8
$7.7
54.4
50.6
47.7
43.6
41.5
4.2 COMPARISON WITH STATE-OF-THE-ARE METHODS
Performance on Standard Benchmarks. As shown in Table GUIDE establishes new state of the on nearly all standard long-tail tailed benchmarks, including ImageNet-LL,, iNaturalist 2018, and Places-LT,, under both standard and longer training schedules. The performance gains are partic- ularly pronounced on large-scale datasets, highlighting the scalability of our hierarchical disentan- glement approach. The detailed breakdown CIFAR- 100-LT (Table 2 reveals the source of this advantage: GUIDE delivers significant performance gains in the challenging medium- and few-shot regimes, making notable progress the core problem of LTR.
Robustness to Distribution Shift. A critical goal of LTR is to learn representations that are not merely adapted to the training distribution but are fundamentally robust to real-world distribution shifts. We go beyond standard evaluation and assess this capability on a suite of unseen, skewed test distributions, following the protocol of Zhang al. (2022) and Zhao al. (2024). This setup, particularly the challenging Backward-LT distributions which completely invert the training class frequencies, serves as a strong proxy for out-of -isttr-distiitttion (OOD) robustness.
As detailed in Table 3 GUIDE demonstrates superior generalization under these shifts. While com- petitive distributions that still favor head classes (Forward-LT),, GUIDE's true advantage emerges in the most difficult scenarios. It surpasses all prior methods on the standard uniform test set and, more strikingly, establishes a significant performance margin on the OOD-like Backward-LT dis- tributions. This strong performance under severe distribution shifts validates that our hierarchical disentanglement fosters more fundamental and less biased understanding of tail classes, leading to enhanced robustness beyond the training prior.
4.3 ABLATION STUDIES, ANALYSIS, AND DISCUSSION
To rigorously and empirically validate our central thesis of Hierarchical Disentanglement, conduct a series of in-depth ablation studies. Our ablation baseline, denoted the Entangled Baseline, is a strong multi-expert model with three experts shar- ing a backbone, trained via standard logit-adjusted cross-entropy loss and including no explicit disen- tanglement mechanisms. Collectively, our studies confirm the critical necessity of each proposed com- ponent and the framework's 's overall robustness. Be- yond validating our framework's components, we also analyze the practical trade-offs of its inference strategies, demonstrating its flexibility for diverse real-world applications. See Appendix B.4 for more detailed analysis.
Table 4: Ablation study on the contribution of each disentanglement level. Performance consistently improves as each level of disen- tanglement is introduced, culminating in our final reported performance.
Many
Medium
Few
Overall
Baseline
62.1
45.0
21.3
45.8
64.9
51.1
26.0
50.4
/
65.7
51.5
27.1
51.3
64.2
49.8
25.5
49.9
/
68.5
55.4
30.1
52.8
67.3
53.8
28.9
$2.1
/
67.9
54.2
29.3
52.5
GUIDE
71.3
59.1
36.0
56.4
8

---

Published as a conference paper at ICLR 2026
Table 5: Analysis of individual mechanisms. (Left) The synergistic effect of diversity losses in Level
(Right) Comparison of different gating policies for the DERM in Level
Level O: Diversity Losses
Level @: Gating Policy
Configuration
Accuracy (%)
Configuration
Accuracy (%)
Entangled Baseline
45.8
No Adaptation (gate=0.5)
52.1
Leccople only
47.7
Static (Inverse Freq.)
53,6
Lav only
47.5
Agnostic (Total Uncert.)
54,9
+ Both (Level O Full)
50.4
GUIDE Policy (Ours)
56.4
Table Trade-off analysis for inference strategies. The two-step method consistently improves few- shot accuracy at predictable latency cost across datasets of different scales. Speed is measured on an NVIDIA RTX 4090 GPU.
Efficiency
Top-1 Accuracy (%)
Dataset
Inference Strategy
Latency (ms/img)
Throughput (img/s)
Overall
Few-shot
Two-Step (Default)
9.2
1054
56.4
36.0
CIFAR-100-LT
Single-Pass
5.1
1923
55.1
33.9
Two-Step (Default)
25.6
391
62.5
47.6
ImageNet-LT
Single-Pass
14.1
709
60.7
45.2
The Necessity of Each Disentanglement Level.
Our comprchensive ablation in Table 4 analyzes all component combinations, revealing their indi- vidual and synergistic contributions. First, each level demonstrates significant standalone efficacy, with Level ! and Level @ yielding the largest gains of +4.6% and +5.5% respectively. Second, all two-component combinations outperform any single component, showing their complementary nature. Notably, the combination of Level and @ is the strongest pair (52.8%), underscoring critical insight: principled policy (Level @) operates most effectively on diverse representational foundation (Level O). Finally, the full GUIDE model, integrating all three levels, attains peak performance of 56.4%. This result significantly surpasses all partial configurations, demonstrating powerful synergy that validates our central thesis: the framework's 's full potential is unlocked only by sequentially resolving all three entanglente lements.
Deconstructing the Disentanglement Mechanisms. We further dissect the mechanisms in Table 5 The analysis of Level reveals a powerful synergy between our two diversity losses. While applying only representation decorrelation (Edecouple) or prediction divergence (Laiv) yields modest gains, their combination yields substantial +4.6% gain over the baseline, confirming that enforeing diversity both feature and decision levels is critical. For Level @, we validate the efficacy of our uncertainty-guidee policy. The results in Table 5 (Right) show the 'GUIDE Pol- icy' (56.4%) significantly outperforming the 'Agnostic (Total Uncert.)' alternative (54.9%). This provides strong empirical validation that our decomposition yields meaningful signal for adapta- tion. This quantitative result, combined with the qualitative analyses in Appendix B..7 (Fig. 5) and Appendix B.3 (Table 13). confirms that our Level @ design both principled and effective.
Sensitivity Analysis. Figure 3 illustrates GUIDE's robustness to key design choices. The frame- work's performance is largely insensitive to the precise values of diversity loss weights, Adccouple and Adiv. shown Figure 3a.). Similarly, the model is robust to the choice of temperature T for uncertainty estimation, as detailed in our analysis in Appendix [3. Furthermore, Figure (lb) demonstrates the critical role of expert diversity. Increasing the number of experts from one to three results in substantial 10.9% absolute gain for few-shot classes. Our choice of E = 3 is justi- fied as it provides the peak overall performance, with marginal gains for additional experts, thus representing optimal trade-off between performance and efficiency.
Inference Strategy Trade-offs. GUIDE offers a flexible trade-off between inference accuracy and efficiency, quantified in Table Our default two-step refinement strategy maximizes perfor- mance, yielding critical gains on few-shot classes for both CIFAR-100-LT, (+2.1%) and the large- scale ImageNet-L.T (+2.4%). For real-time applications, single-pass soft gating variant nearly

---

Published as a conference paper at ICLR 2026
(a) Sensitivity to Loss Weights
(b) Impact of Number of Experts
56.8
80
Overall Top-1 Accuracy (%)
Overall Top-1 Accuracy (%)
56.6
Max span across Adv 0.35%
60
54.2
56.4
55.6
56.4
56.2
40
$6.0
Aut value
Many
20
Medium
55.8
0.2
1.0
Across Aas range
Ovanall Top-1
0.0
0.1
0.2
0.3
0.4
0.5
2
3
4
Representation Diversity Weight (eecouple)
Number of Experts (E)
Figure 3: Sensitivity analysis of GUIDE. (a) Overall Top-1 accuracy is stable across broad range of diversity loss weights. (b) Increasing experts from 1 to dramatically boosts performance Few-shot classes. The overall accuracy for each configuration is annotated, peaking at 3 (our final configuration, 56.4%) and showing diminishing returns for E' = 4.
doubles the throughput with only a marginal drop in accuracy. This flexibility allows practition- tailor GUIDE's deployment to specific latency and performance requirements. While these measurements were conducted on a high-end GPU, the relative throughput trade-off (~ 1.8x) is consistent across datasets of varying scales and is expected to hold on other server-grade hardware, confirming the engineering feasibility of our approach.
CONCLUSION
This paper identifies fundamental entanglement problems as the key bottleneck in multi-expert Long-Tailed Recognition. We introduce GUIDE, a framework that resolves these issues via Hi- erarchical Disentanglement. GUIDE enforces expert diversity through competitive specialization and enables principled, stable adaptation using an incertainty-guided two-timescale optimization scheme. This approach achieves new state-of-the-art results on five major benchmarks, with pro- nounced improvements on few-shot classes. Our work demonstrates that disentangling coupled learning dynamies powerful principle for designing gent-gennration models for imbalanced data.
ACKNOWLEDGEMENTS
The authors gratefully acknowledge the support from the National Natural Science Foundation of China (NSFC) under Grant Nos. 62402472, and 12227901. This work was also supported by the Natural Science Foundation of Jiangsu Province (No. BK20240461), the Project of Stable Support for Youth Team in Basic Research Field, CAS (No. YSBR-005), and the Academic Leaders Cultiva- tion Program at USTC. The AI-driven experiments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Sciences.
REFERENCES
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, NY, USA, 2006. ISBN 9780387310732. doi: 00.1117/1.21191199
Vivek S. Borkar. Stochastic approximation with two time scales. Systems & Control Letters, 29(5): 291-294, 1997.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with abbel-distibuttin-aue margin loss. In Proceedings the Advances in Neural Information Processing Systems (NeurIPS). 2019.
10

---

Published as a conference paper at ICLR 2026
Jiahao Chen and Bing Su. Transfer knowledge from head to tail: Uncertainty calibration under long-tailed distribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Intersciencee, Hobo- ken, NJ, USA, 2nd edition, 2006, ISBN 9780471241959, doi: 10. 10.1002/047174882XX
Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 715-
724, 2021.
Chaoqun Du, Yulin Wang, Shiji Song, and Gao Huang. Probabilistic contrastive learning for long- tailed visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Lianli Gao, Zhao Guo, Hanwang Zhang, Xing Xu, and Heng Tao Shen. Video captioning with attention-based Istm and semantic consistency. IEEE Transactions on Multimedia, 19(9):2045- 2055, 2017. doi: 10.1 00.1099/MM.2017.2229911
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). 2017.
Youngkyu Hong, Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, and Buru Chang. Dis- entangling label distribution for long-tailed visual recognition. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6626-6636, 2021.
Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79-87, 1991.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2017.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural net- work representations revisited. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Depart- ment of Computer Science, University of Toronto, 2009. Technical Report.
Anders Krogh and Jesper Vedelsby. Neural network ensembles. cross validation, and active learning. In Proceedings of the Advances Neural Information Processing Systems (NeurIPS), 1994.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2017.
Jun Li, Zichang Tan, Jun Wan, Zhen Lei, and Guodong Guo. Nested collaborative learning for long- tailed visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6949-6958, 2022.
J. Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145-151, 1991. doi: 10.1 10.1109/18.611 155.
Yong Liu and Xin Yao. Ensemble learning via negative correlation. Neural Networks, 12(10):
1399-1404, 1999. doi:
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X. Yu. Large- scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition (CVPR), 2019.
11

---

Published as a conference paper at ICLR 2026
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tale -aig-taal learning via logit adjustment. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Ari S. Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neu- ral networks with canonical correlation. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2018.
Bálint Muesányi, Michael Kirchhof, and Seong Joon Oh. Benchmarking uncertainty disentangle- ment: Specialized uncertainties for specialized tasks. arXiv preprint arXiv:2402. 2024.
Haolin Pan, Yong Guo, Mianjie Yu, and Jian Chen. Enhanced long-tailed recognition with con- trastive cutmix augmentation. IEEE Transactions Image Processing, 2024.
Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Balanced meta- softmax for long-tailed visual recognition. In Proceedings of the Advances Neural Information Processing Systems (NeurIPS), 2020.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In Proceedings of the 35th International Conference on Machine Learning (ICML). PMLR, 2018.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):21 1-252, 2015, doi: 10.1007/s1 -21---1--111...
Emanuel Sanchez Aimar, Arvi Jonnarth, Michael Felsherg, and Marco Kuhlmann. Balanced product of calibrated experts for long-tailed recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19967-19977, 2023.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experrs layer. In Proceedings of the International Conference on Learning Representations (ICLR), 2017. arXiv:1701.06533
Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 37-45, 2015. doi: 10.11999CVPR.2215579999999
Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. Unsupervised deep hashing with fimilarity-ddapiiv. and discrete optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(1 (0(12):3034-0044 2018. doi: 10.
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Deyu Meng, and Zongben Xu. Meta- weight-net: Learning an explicit mapping for sample weighting. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2019.
Siyu Sun, Han Lu, Jiangtong Li, Yichen Xic, Tianjiao Li, Xiaokang Yang, Liqing Zhang, and Junchi Yan. Rethinking classifier re-training in coigetailed recognition: Label over-smooth can balance. In International Conference on Learning Representations, 2025. Poster Track.
Grant Van Horn, Oisin Mae Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Adversarial cross- modal retrieval. In Proceedings of the 25th ACM International Conference Multimedia. MM '17, pp. 154-162, New York. NY, USA, 2017. Association for Computing Machinery. ISBN 9781450349062. doi: 10. 10.1145//1232663122533
12

---

Published as a conference paper at ICLR 2026
Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella Yu. Long-tailed recognition by routing diverse distribution-wwarr experts. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Zheng Wang, Zhenwei Gao, Yang Yang, Guoqing Wang, Chengbo Jiao, and Heng Tao Shen. Geo- metric matching for cross-modal retrieval. IEEE Transactions on Neural Networks and Learning Systems, 36(3):5509-5555 5521, 2025. doi: 10. L109/TNNLS
Tong Wei, Zhen Mao, Zi-Hao Zhou, Yuanyu Wan, and Min-Ling Zhang. Learning label shift cor- rection for test-agnostic long-tailed recognition. In Proceedings of the International Conference on Machine Learning (ICML), 2024.
Kuo Yang, Duo Li, Menghan Hu, Guangtao Zhai, Xiaokang Yang, and Xiao-Ping Zhang. Uncertainty- sampling for long-tailed semi-supervised learning. arXiv preprint arXiv:2401.0444 2024.
Lingjie Yi. Michael Yao, Weimin Lyu, Haibin Ling, Raphael Douady, and Chao Chen. Geom- etry of long-tailed representation learning: Rebalancing features for skewed distributions. In International Conference on Learning Representations, 2025.
Yifan Zhang, Bryan Hooi, Lanqing Hong. and Jiashi Feng. Self-supervised aggregation of di- verse experts for test-agnostic long-tailed recognition. In Proceedings of the Advances Neural Information Processing Systems (NeurIPS), 2022.
Qihao Zhao, Chen Jiang, Wei Hu, Fan Zhang, and Jun Liu. Mdes: More diverse experts with consis- tency self-distillation for long-tailed tailed recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11597-11608, October 2023.
The Zhao, Haibin Wen, Zikang Wang, Pengkun Wang, Fanfu Wang, Song Lai, Qingfu Zhang, and Yang Wang. Breaking long-tailed learning bottlenecks: A controllable paradigm with seeemettwrrk-eneerttee diverse experts. In Proceedings of the Advances in Neural Information Processing Systems (NeurlPS), 2024.
Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long-tailed recogni- tion. In Proceedings of the IEER/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jianggang Zhu. Zheng Wang, Jingjing Chen. Yi-Ping Phoebe Chen, and Yu-Gang Jiang, Bal- anced contrastive learning for long-tailed visual recognition. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6908-6917, 2022.
13

---

Published as a conference paper at ICLR 2026
Appendix
GUIDE: Gated Uncertainty-Informel Disentangled Experts for Long-tailed Recognition
The content of the Appendix is summarized as follows:
Section A Methodological Complements. This section provides in-depth details for our pro-
posed framework.
Section A.1
Full mathematical formulations of key objectives.
Section
Rigorous proofs for all theorems and propositions.
Section
A.3
Detailed pseudocode for the GUIDE training procedure.
Section
Further discussion on design rationale and the novelty of our work.
E Experimental Complements. This section offers additional details and results to ensure the reproducibility and robustness of our experiments.
Section B
Section B.1
Detailed experimental setups, including dataset statistics and implementation
specifies.
Section B.2 2
Clarification on the comparison protocol for baseline methods.
Section B3
Additional results on the CIFAR- ---LR benchmark.
Section B.4
Extended evaluation diverse and pathological scenarios (MEET-LT &
ZineFluor).
Section B.5
Quantitative analysis of the two-step inference strategy and its robustness.
Section B.6
Quantitative analysis of the expert diversity.
Section B.7
Qualitative visualizations for representation disentanglement and the learned
adaptive policy.
Section B.8
Case study of the DERM gating policy.
Section B.9
Analysis of computational cost, convergence, and performance trade-offs.
Section B.10
Qualitative analysis of challenging fine-grained examples and failure modes.
Section B.
Sensitivity analysis of key hyperparameters (loss weights and meta-policy
learning rate).
Section B.12
Comprehensive ablation study on the DERM architecture components. Sensitivity analysis with respect to the temperature scaling hyperparameter.
Section B.13
Section C Limitations and Future Work. A discussion on the potential limitations of our current work and promising directions for future research.
A METHODOLOGICAL COMPLEMENTS
This section provides comprehensive details that supplement Section the main paper. begin by presenting the full formulations of key objectives, followed by the formal proofs of our guiding theorems, and conclude with a discussion of the core design rationale.
A.1 FULL FORMULATIONS OF KEY OBJECTIVES
For completeness, we provide the full mathematical definitions of concepts that were presented conceptually in the main paper.
Logit Adjustment. The logit-adjusted scores (x) used in the main loss (Eq. equation I computed for each expert e and class as:
Σe (x) = (x) c + Te log Tirain (c):
(11)
where 2x(2)) is the raw logit output from expert Tirain (c) the prior probability of class in the training and T. hyperparameter controlling the strength of the adjustment for each expert.
14

---

Published as a conference paper at ICLR 2026
Jensen-Shannon Divergence (JSD). The JSD used in the prediction diversity loss (Eq. equa- tion 3 is defined as the difference between the entropy of the average prediction and the aver- age of the individual expert entropies. Given set of temperature-scaled predictive distributions (Pe,r(-(z)))):
E
(1)() H
=Eeer(lo) - Σ
1
H(per(+))))
(12)
E
where H(-) is the Shannon entropy. A full derivation is provided in the proof of Theorem (Ap- pendix A.2
A.2
THEORETICAL FOUNDATIONS AND PROOFS
Here, we present the rigorous proofs for the theorems and propositions that guide our framework's design.
Proof of Theorem (Diversity-driven Bound Tightening). For clarity and completeness, we first restate Theorem below.
Theorem 1 (Diversity-driven Bound Tightening). For an ensemble of E experts with predictive distributions (pe()=)))==1 and their average p(-1x) E. Pe(tar):
(a) The ensemble's Negative Log-Likelihood (NLL) is upper-bounded by the average of the individual expert NLLs.
(b) The gap in this bound is directly related to the predictive diversity, which is formally cap- tured by the JSD. Maximizing JSD serves to tighten this performance bound.
Proof. The proof is presented in two parts, corresponding to the two claims of the theorem.
Part (a): Proof of the NLL Upper Bound. We begin by recalling Jensen's 's inequality (Bishop) 2006), a fundamental property of convex functions. For a convex function and a set of points (Z1 _E} with uniform weights:
F
E
Σ f(ze).
(13)
6=1
e=1
The function f(=) log(2) is strictly convex over its domain 0, its second derivative is strictly positive:
de
("(z)
d=2 (-
log(z))
22 0.
(14)
Let us define the points as the probability assigned by expert to the true class
Ze Pe. (y)r()
(15)
The average of these points ensemble's probability for the true class y:
E
1
Σ Ze
Σ =
(16)
E
Applying Jensen's inequality with our chosen function f and points z. directly yields the upper bound on the ensemble's NLI.:
E
log(Pr(ula))
<
log(pe.r
(17)
E
This proves part (a) of the theorem.
15

---

Published as a conference paper at ICLR 2026
Part (b): Relating the Bound Gap to JSD. The Jensen-Shannen Shannon Divergence for set of E prob- ability distributions {pe ) with uniform weights is defined as the average Kullback-eeine (KL) divergence from each distribution to their average distribution p.
JSD((Pe)) E
Σ DRL(Pellp)
(18)
We derive the well-known identity for JSD. Substituting the definition of KL divergence, pic) we get (omitting conditioning on x for brevity):
Pxc(ulla) = Ee p(c) log q(e)
E
c
Pc(c) p(c)
JSD((pe)) =
(c(c) log
E
==1
1
Pe(c)(log log ((c)
(19)
E
c
Σ
Σ (c logb
1
ΣE
_
pe(c) log p(c)
E
E
c=1
The first term is the average negative Shannon entropy:
Σ
Σ E
F
(o(p)oo - Σ(HHp) - = Σ
E
H(p)).
(20)
E
The second term, by swapping the order of summation, becomes the entropy of the average distri- bution:
E
1
E
(c(c) logg(c)
= - log p(c)
Σ
palc)
F.
eml
(21)
Σ p(c) log p(c) H()).
Combining these results yields the identity that directly connects JSD predictive uncertainty,
well-established property of the Jensen-Shannon Divergence (Lin 1991 Cover & Thomas 2006):
JSD((pc))= = H
E Σ H(p.).
(22)
This term quantifies the disagreement among experts. A higher JSD signifies greater predictive diversity. While the gap in the Jensen bound is not strictly equal to JSD, maximizing JSD empirically encourages the individual posteriors Pe to diverge, which reduces the correlation of errors and thus effectively "tightens" the bound in practice, leading to improved ensemble performance. This proves part (b).
Proof of Theorem 2 (Policy Monotonicity). For clarity and completeness, first restate Theorem 2 below.
Theorem 2 (Policy Monotonicity). Assuming the learnable controller parameters de. and B. > O, the refinement strength gate ge.e of the DERM is a monotonically increasing function with
(t) respect the class-wise epistemic uncertainty Epic. and a monotonically decreasing function with respect to the class-wise aleatoric uncertainty Ale
Proof. The proof relies on a straightforward application of the chain rule the gate's definition.
The refinement strength gate Ze,e is given by the function:
Je.c (.eec)) (gmax Fmin) Dwin:
(23)
16

---

Published as a conference paper at ICLR 2026
where the pre-aue activatiion input Ie,e defined as:
Te- The derivative of the sigmoid function o(2) = (1 + -- is strictly positive for all finite 2: o' d'(=) = (=) - (2))
Je,c = ae Epiy.e - B -AleT. +
(f)
(24)
E IR.
(55)
Part 1: Monotonicity with respect to Epistemic Uncertainty. We compute the partial derivative of Je,e with respect to Epi7. using the chain rule:
dge.c dge.c Oge.c dEpi
(26)
0ge.c DEpiT,c
(t)
The two components of the derivative are:
Úge.c = o' (āe,c) (Imax (quin);
(27)
dge,c
Öge.c =ce-
@Epi
(1)
Op.
(28)
By assumption, (9max - 9min] > 0 and a₂ > 0. Since o'()) () also positive, their product strictly positive:
dge.c o' (' - Imin)
ae
(29)
OEpi (4)
>0
A strictly positive derivative implies that Ie,e is a monotonically increasing function of Epi
Part 2: Monotonicity with respect to Aleatoric Uncertainty. Similarly, the partial derivative with respect to Ale is:
(t)
0ge.c (t)
Dge.c Oge,c (t)
(30)
dAleT.c
0ge.c aAleT,e
The second component of this derivative is:
Dge.c. Be-
(31)
dAleT.c
(t)
By assumption, Be > 0. Therefore, the overall partial derivative is strictly negative:
Oge.c
= o'(,eee) Imin) (-Be)
Be) < 0.
(32)
dAle (2)
A strictly negative derivative implies that Ie,e is monotonically decreasing function of Ale This completes the proof.
(8)
Discussion on Propasition 1 (Satisfaction of TTSA Conditions).
Proposition 2. By partitioning the model parameters into fast variables 0 and slow variables é. and employing an update scheme where their respective learning rates satisfy ni & Tjo, our optimization process constitutes valid two-Timescale Stochastic Approximation (TTSA) algorithm.
This proposition is not theorem but statement about the alignment of our proposed algorithm with existing, powerful convergence theory. Here, we provide detailed justifi- cation for this claim.
Discussion.
17

---

Published as a conference paper at ICLR 2026
1. The Core Principle of TTSA:
Two- Tio-Tiieseale Stochastic Approximation theory (Borkar 1997) analyzes the behavior of systems with two coupled stochastic update rules operating at different rates. The fundamental insight is that for the slow-moving variables, the fast- moving variables appear to have already converged to their equilibrium. The slow updates then track this slowly changing equilibrium point. This separation of timescales prevents the high-variance updates of the fast process from destabilizing the slow, deliberate optimization of the slow process.
Mapping GUIDE to the TTSA Framework: Our framework maps perfectly onto this struc- ture:
Fast Variables (0): The parameters of the backbone and the DERM pathways
(Fround, r)) Their objective is to minimize the GUIDE loss (Eq. equation 3 for a fixed policy.
Slow Variables (o): The parameters of the Strategy Controllers (fa.. Ne Te).). Their objective is to find the optimal policy by minimizing the validation loss.
3. Verification of Timescale Separation Condition: The key condition for TTSA to hold is that the learning rates for the two processes are asymptotically separated. Formally, if 1oo and No.k are the learning rates at step k; they must satisfy:
00
Jo.k = 00,
Σ
Mo. <
00, and
lim k-+00 178.k
aa.k O.
(33)
In our practical implementation, we ensure this condition is met by using large, scheduled learning rate for 0 (e.g.. 17 starts at 0.1 and decays) and a very small, often fixed learning rate for (c.g.. To le-4). This large ratio (1/ < 176) throughout training serves strong finite-time approximation of the asymptotic condition.
4. Implication for Stable Convergence: By designing our optimization procedure to align with TTSA principles, we provide strong theoretical justification for the stability of our system. The fast updates (Eq. equation (9) learn effective representations for a given adaptive policy, while the slow updates (Eq. equation I0) safely and gradually steer the entire system towards better, more robust policy without causing catastrophic interference. This principled decou- pling is the cornerstone of our Level disentanglement.
A.3 ALGORITHM PSEUDOCODE
Algorithm [ provides a detailed step-by-step procedure for training the GUIDE framework. It il- lustrates the interplay between the fast-timescale updates for the main network parameters (0) and the slow-timescale updates for the meta meta-policy controllers (ø), integrating all three levels of our hierarchical disentanglement approach.
A.4 DESIGN RATIONALE AND FURTHER DISCUSSIONS
On the Novelty of the GUIDE Framework. While components used in GUIDE, such as multi-expert architectures and diversity losses, are established concepts, their prior applications in LTR have been hampered by a failure to address their conflicting dynamics. The core nov- elty of our work therefore the invention of these individual components, but the overarch- ing principle of Hierarchical Disentanglement and the principled framework that instantiates it. We are the first to diagnose the limitations of current SOTA methods as cascade of entangle- ments cettts--eppesentttioneeeeee herent, multi-level solution that explicitly decouples these processes. The resulting synergy, guided by our disentanglement principle, is what leads to state-of-the-art art performance. A visual confirma- tion of the effective representation disentanglement Level is provided in Figure 4
cause/symptom, and seaeming/metalllarning Beannnngaand and to propose co-
Rationale for the DERM Architecture. The Adaptive Residual Mixture in Eq. equation is a deliberate design choice for the DERM. In preliminary studies, we compared it against simpler alter- natives, such as a direct multiplicative gating mechanism, e.g., f, (x:; c) Ja.s Freiie,e [ Found (I)). The residual formulation offers two decisive advantages:
18

---

Published as a conference paper at ICLR 2026
Algorithm GUIDE Training Procedure
Require: Training set D, validation set V. epochs N; learning rates 11. Moi temperature EMA rate Ogma's gates Imin Itaxx weights Xiec Adivi meta update interval K
Ensure: Trained parameters 8,6
1: Init main net (for Fiound, (Freline, classifiers)
2: Init controllers & {are3Bes Jette.
3: Init class-wise EMAs: Epir,e < 0, AleT, + 0 for all (I,...,C)
4: for epoch to N do
5: for each mini-batch {[z B:)e C D do
6:
Level @ policy execution (use current )) for each expert do
7:
compute Pe.zn f of a. Epir.... - BeAler, + Te (Imax - Imin) + 9min for all end for
8:
9:
10:
fround t- (2; (2;
11:
fe < fiiund Je.y ( (Freinee (Joondd; 0) [tound] for each
12:
Ze - Classifier (ee;;
b unadjusted logits per expert
Level O: objectives
14:
compute Lmain using adjusted ensemble logits (Eq. equation compute Liecouupe from (Ja)e (Eq. equation
16:
compute JSD) from (softmax(ze/T)) (Eq. equation 12
D unadjusted logits with
temp
17:
LGUIDE Lmain + Adoe Coccoupp Adi
18:
Level @ fast update
19:
+ 0 To V CGUUDE
20:
end for
21:
if epoch mod K 0 then
22:
// Diagnose on (class-wise uncertainties) initialize accumulators U I], + for all for each (z.. ye) do compute {xv) (unadjusted), get (piy (xv)
23:
25:
26:
append U +- Epir(#e), end for
Uzle (1,)(
27:
28:
for c 1 to Co
29:
Epir,e & (1 Oema Epir, + Qema mean (UP))
30:
Aler.c (1 a'ema) )leTT acma mean (?lle
31:
end for
32:
1/ Meta-optimize controllers
33:
Lval + (rx. M.lev [Cmuin (Iv, Yo: 0, [1)] d no V Lval
34:
35:
end if
36:
end for
1. Stable Optimization:
The residual connection provides a direct information pathway from the foundation block, which mitigates vanishing gradients. This structure allows the model to learn the refinement as a delta or correction, a task which is often more tractable optimization problem.
2. Flexible Adaptation: The formulation enables the module smoothly interpolate between the foundation representation (when ge.c R 0) and the fully refined representation (when ge,e 1). This provides
stable and flexible mechanism for allocating refinement capacity. The effectiveness this learned policy is visualized in Figure
This design inspired by the success residual adapters and gated units in other domains, which have proven effective for modular and conditional computation.
Justification for the Uncertainty Decomposition Formulation. Section 3.2 we define
Aleatoric and Epistemic uncertainty based on the ensemble's outputs. This formulation standard
19

---

Published as a conference paper at ICLR 2026
and aidely-accepted definition in the Bayesian deep learning community [Lakshminarayaaan al. 2017). which we adopt for its clear interpretability:
Aleatoric Uncertainty, estimated as the average entropy across experts, captures inherent data ambiguity. A high value indicates that all experts consistently agree sample is ambiguous, property irreducible with more data.
Epistemic Uncertainty, equivalent the JSD, captures the model's own ignorance via the disagreement among experts. A high value indicates that the model has not reached a consensus, a deficiency that is reducible with more data.
Discussion Disentanglement: We acknowledge recent work by Mucsányi al., (2024) pointing out that this classical decomposition is not theoretically perfect and that aleatoric and epistemic un- certainties can be entangled. However, we argue that in the specific context of Long- Tailed Recog- nition, this formulation provides a highly effective diagnostic signal. The primary challenge in LTR distinguishing "model ignorance" (stemming from data scarcity in tail classes) from "data ambiguity" (stemming from inherent noise or intra-class variance in head classes).
Empirical Validity: Our experiments provide strong evidence that, despite theoretical limitations, this decomposition successfully captures these distinct sources of difficulty for our task. As shown in (Right) our policy guided by these decomposed signals (56.4%) significantly outperforms the agnostic policy using total, non-decomposed uncertainty (54.9%). This confirms that the decom- posed signals contain crucial, actionable information that our framework successfully leverages to guide targeted refinement.
On the Use of a Validation Set. Our framework utilizes small, held-out validation set during training for two distinct purposes: computing stable, class-wise uncertainty statistics and updating the meta-policy parameters (1)). This is a standard and principled practice in the meta-learning and bilevel optimization literature, including its application to long-tailed recognition (Ren al. 2018). For all experiments, this validation set was created by splitting small portion (e.g., 1-2 samples per class, or a fixed 5% split) from the original training set, as is common practice. This ensures the test set remains completely unseen and is used exclusively for final evaluation, thereby avoiding any data leakage and ensuring a fair assessment of generalization performance.
B EXPERIMENTAL COMPLEMENTS
B. EXPERIMENTAL SETUP DETAILS
Datasets. We validate our GUIDE framework on five widely-used benchmarks for long-tailed recognition. Detailed statistics are summarized in Table
CIFAR-100-LT (Cao et al.
2019): : A long-tailed version of CIFAR-100
Krizhevsky 2009
with imbalance ratios (IR)
TO, 50, and 100.
CIFAR-1O-LT (Cao al.
2019) A long-tailed version of CIFAR-10 (Krizhevsky 2009 with
imbalance ratios (IR) of 0, 50, and 100.
ImageNet-LT (Liu et al., 2019): A long-tailed subset ImageNet-2012 (Russakovsky et al. 2015) with 1,000 classes and an IR of 256.
iNaturalist 2018 (Van Horn et al. 2018): A real-world, large-scall scale dataset with natural long- tailed distribution, 8,142 classes, and an IR of up to 500.
Places-LT (Liu al. 2019): long-tailed version of the Places365 scene benchmark, with 365 classes and maximum IR of 996.
Implementation Details. Our implementation is based the PyTorch framework and all ex- periments were conducted on NVIDIA RTX 4090 GPUs. Across all datasets, we follow standard protocols ensure fair comparison with prior work.
For 1IFAR-10-LT and CIFAR-100-LT, we use a ResNet-32 backbone. All models are trained using an SGD optimizer with momentum of 0.9, Nesterov momentum enabled, and weight decay of
20

---

Published as a conference paper at ICLR 2026
Table 7: Statisties of the benchmark datasets used in our experiments.
Dataset
# Classes
# Train Samples
# Test Samples
Imbalance Ratio
10FAR-100-LT 100
10,000
10,000
10, 50, 100
CIFAR-1O-LI
10
10,000
10,000
10, 50, 100
ImageNet-LT
1,000
115,846
50,000
256
Places-LT 8,142
iNaturalist 2018
437,513
24,426
500
365
62,500
36,500
996
5e-4. The batch size is set 128. For the standard training schedule, we train for 200 epochs with an initial learning rate of 0.1. The learning rate is decayed by a factor of 0.1 at epochs 160 and 180, following 5-epoch linear warmup. For the longer schedule, train for 400 epochs, with decays at epochs 320 and 360, following a 10-epoch warmup.
For ImageNet-LT and Places-LT, LT, employ large-scale ResNet backbones. On ImageNet-LT, use ResNet-50, training for 180 epochs (standard) or 400 epochs (longer) with a batch size of 64 per GPU. The initial learning rate 0.025, following cosine annealing schedule. On Places-LTT, to ensure a strong comparison with prior work, we use deeper ResNet- 152 backbone. It is trained for 90 epochs (standard) 180 epochs (longer) with batch size of 128 per GPU (total 512). adopting the same learning rate and cosine schedule as ImageNet-LT.
For iNaturalist 2018, we also use ResNet-50 backbone. We train for 100 epochs (standard) or 200 epochs (longer) with total batch size of 256. The initial learning rate is 0.1 with weight decay of 2e-4, decayed via cosine schedule.
Across all datasets, our framework is configured with E 3 experts. For logit adjustment, we assign each expert a different, fixed strength parameter T. to encourage diverse range of decision boundaries. The diversity loss weights, Adecoupe and Adiv- are consistently set to 0.1, and these losses are linearly warmed up over the first 10-40 epochs depending on the total training length. The temperature for diversity calculation is set to T = 1.0. The optimization disentanglement realized by updating the strategy controllers (ø) with a small, fixed learning rate of ne 1e-4, while the main network (0) is updated with the much larger, scheduled learning rate.
GUIDE Framework Details:
B.2 BASELINE COMPARISON DETAILS
To maintain fair and transparent comparison in Table we primarily cite results from the original papers. However, for some methods, results on certain benchmarks were not available. In these instances, we executed the official, publicly available code provided by the authors, adhering strictly to their specified configurations and hyperparameters. We only report results for which the original implementation was clearly applicable to the target dataset.
The specific results reproduced by us are marked with a dashed underline in Table [ For example, the original BCL paper focused CIFAR and ImageNet-LT, its performance iNaturalist 2018 was reproduced by us. Similarly, other missing entries for relevant baselines were filled by running their code. This approach ensures the broadest possible comparison while maintaining methodological rigor.
B.3 ADDITIONAL RESULTS ON CIFAR-1O-LT
To further validate the general applicability and robustness of our GUIDE framework, we con- ducted additional experiments the CIFAR-IO-LT benchmark. This dataset, while simpler than 100-LT a standard testbed for evaluating long-tailed recognition algorithms. We followed the same experimental protocol for CIFAR- using ResNet-32 backbone and evaluating across imbalance ratios of 10, 50, and 100 under both standard and longer training schedules.
As shown in Table 8 GUIDE consistently outperforms strong state-of-the-art baselines across all settings. The performance gains remain significant, particularly at higher imbalance ratios (c.g., +1. 1..% over the strongest baseline IR=100 for standard training). These results demonstrate that
21

---

Published as a conference paper at ICLR 2026
the principles of hierarchical disentanglement not limited to complex, large-scale datasets but are fundamentally effective for long-tailed problems in general.
Table 8: Top-1 aceuracy (%) on CIFAR-IO-LT with ResNet-32. GUIDE demonstrates superior
performance across all imbalance ratios, confirming its general applicability. bold, second best are underlined.
Best results are in
Standard Training
Longer Training
Method
IR=10
IR=50
IR=100
IR=10
IR=50
IR=100
DDAM-DRW (Co et al
2019)
88.5
79.3
77.0
89.1
80.1
77.8
MiSLAS (Zhong cl al 202T)
90.1
85.7
82.1
90.6
86.4
82.9
RIDE Wang al. 2021
89.2
87.8
87.2
89.8
88.4
87.8
SADE Zhang
2022
88.6
87.3
86.9
89.2
87.9
87.5
BalPoET (aanchez Aimar et al
2023
90.8
87.7
87.3
91.3
88.1
NCL (Li ct all 2022
91.1
89.5
87.3
91.6
90.0
87.9
FeatRecon YY al
2025
91.6
87.8
85.2
92.0
88.4
85.8
ConCutmix Pan ct
2024)
91.4
88.0
86.1
91.9
88.6
86.7
ProCo Du 2024
91.9
88.2
87.5
92.3
88.8
88.2
GUIDE
92.5
90.1
87.6
93.0
90.7
88.3
B.4 EXTENDED EVALUATION ON DIVERSE AND PATHOLOGICAA LONG-TAILED SCENARIOS
To further demonstrate the broad applicability and robustness the GUIDE framework, we ex- tended our evaluation to challenging scenarios outside the domain of standard computer vision. These principled experiments on newly proposed long-tailed geospatial benchmark (MEET-LT) and pathological long-tailed chemical dataset (ZineFluor validate that GUIDE's hierarchical disentanglement principles are effective across diverse data modalities and under extreme imbalance conditions.
A New Benchmark: Long- Joig-Tailed Fine-Grained Geospatial Recognition (MEET-LT), In- spired by reviewer feedback, we propose a new, challenging long-tailed benchmark, MEET-LT, to directly test our framework's 's core capabilities in novel domain. We constructed this benchmark from the Multi-Enviromment Entity Typing (MEET) dataset, which contains 80 fine-grained classes of remote sensing imagery.
Benchmark Construction: Following the standard protocol used for CIFAR-LT, we created a long- tailed training set from the original MEET training data with severe imbalance ratio of IR=100 using an exponential decay function. The original balanced test set was preserved for fair evaluation. This process creates challenging scenario that is both dovain-novel and directly relevant to paper's 's core problem.
Experimental Results: We conducted comprehensive evaluation MEET-LT using ResNet- 50 backbone. As shown in Table 9 GUIDE achieves a of overall accuracy of 75.5%. Crucially, the source of this significant +4.0 point gain over the strong BCL baseline (71.5%) lies in its transformative impact on the most data-scarce classes. GUIDE boosts the Few-shot accuracy to 57.5%, remarkable +7.5 point absolute improvement. This directly validates that GUIDE's core mechanisms, particularly the uncertainty-guider refinement (Level @), are exceptionally effective learning from sparse data in new, specialized domains.
Performance on the Pathological Long-Tailed ZincFluor Dataset. We further tested GUIDE on the ZincFluor dataset, real-world scientific discovery benchmark with a pathological long-tail distribution (I 137.54). This task involves classifying molecular fluorescence levels from graph- structured data, where accurate identification of rare tail classes critical. For this evaluation, we adopted the GCN-based experimental protocol from prior work this dataset.
The results in Table 10 confirm GUIDE's exceptional performance in this extreme scenario. Our method achieves Tail Top-2 accuracy of 67.7%, substantially outperforming all other baselines, in- cluding the strong contrastive learning method BCL (59.6%). This superiority stems from GUIDE's ability to beyond representation learning and employ dynamic, uncertainty-aware policy to
22

---

Published as a conference paper at ICLR 2026
Table 9: Performance (%) on the proposed MEET-LT benchmark (IR=100) using ResNet-50 back- bone, GUIDE shows dominant advantage, particularly on the challenging few-shot classes. Best results are bold, second best are underlined.
Method
Many-shot
Medium-shot
Few-shot
Overall
CE
82.5
65.1
35.8
67.3
CE-DRW
84.1
68.2
40.5
69.8
LDAM-DRW
83.0
66.5
38.2
68.1
Balanced Softmax
85.2
70.3
44.1
71.2
BCL
86.1
72.0
50.0
71.5
GUIDE
86.5
76.2
57.5
75.5
focus on challenging tail-class molecules. This provides strong evidence that GUIDE is highly effective framework for high-stakes scientific discovery tasks where data is non-Euclidean and tail- class performance is paramount.
Table 10: Top-1 accuracy (%) the pathologically long-tailed ZincFluor dataset (T 137.54) The shaded section indicates the primary observation indicator. Bold indicates the best performance while underline indicates the second best.
Fluor Level Accuracy (%)
Tail Top Accuracy (%)
Method
4
7
Top-6
Top-4
Top-2
CE
85.19
70.49
19.71
25.62
0.000
0.00
73,40
0..0)
19.78
18.35
36.70
BS
82.73
30.66
43.21
28.51
0.00
25.00
72.34
0.00
28.17
24.33
36.17
BCL
86.45
51.17
51.82
22.31
17.43
40.38
69.15
50.00
41.84
44.24
59.57
CE-DRW
94.52
45.62
27.59
26.86
12.84
42.31
67.02
33.33
34.99
38.87
50.17
LDAM-DRW
91.93
47.27
28.91
20.66
22.94
28.85
69.15
33.33
33.97
38.56
51.24
KPS
91.10
45.70
51.09
23.97
1.83
19.23
71.28
0.00
27.90
35.64
LORT
72.23
25.81
1.75
33.88
0.00
26.92
75.53
0..00
23.01
25.61
37.76
GUIDE
90.20
41.50
57.50
22.
15.20
38.40
68.10
69.50
44.70
45,40
67.70
B.5 ANALYSIS OF INFERENCE STRATEGY
Our default inference process for unlabeled data relies on two-step refinement strategy: an initial pass generates prediction ÿ, which serves proxy select the class-wise refined pass. To quantitatively validate this approach, we conducted an analysis on the validation set of CIFAR-1004 100-LT (IR=100), where ground-truth labels are available.
gate Je.j for second,
Methodology. For each sample in the validation set, we first perform an initial forward pass using the foundation pathway Fiound to get prediction ÿ. We record its accuracy as AcC Then, we use this ý to select the learned gates and perform the full DERM forward pass to get the final prediction, with accuracy AcCinul-
The results, summarized in Table 11 confirm the efficacy of the refinement step. The final accuracy shows marked improvement over the initial pass, demonstrating that the DERM, when guided by imperfect proxy label, effectively refines the representations and improves classification.
Results.
Robustness to Mis-guided Refinement. potential risk of our two-step inference that an incorrect initial prediction (ū * y) could lead to "mis-guided"" refinement, where the model uses a policy from a wrong class. To quantify our framework's robustness against this risk, we ana- lyzed the subset of initially misclassified samples on the CIFAR-100-11 validation set. Our findings demonstrate a strong error-correction capability: 21.7% these samples were correctly classified after the refinement step, when guided by an incorrect proxy label. This strongly suggests that the learned class-wise policies are robust and semantically meaningful; guidance from a related (though incorrect) class often provides beneficial signal for refinement than adaptation
23

---

Published as a conference paper at ICLR 2026
Baseline(Homogeeeity Collapse)
GUIDE[Competiivv Specialization)
Tail Class1
Tail Class2
Tail Class3
Expert1
Expert2
Expert3
Figure 4: t-SNE visualization of feature representations from three experts for selected tail classes on CIFAR- 100-LT. (Left) A baseline multi-expert model exhibits significant feature overlap. indi- cating homogeneity collapse. (Right) Our GUIDE framework, with its competitive specialization objective, successfully learns decorrelated feature subspaces for each expert, confirming effective representation disentanglement.
Table 11: Quantitative analysis of the two-step inference strategy on the CIFAR-100-LT (IR=100) validation set. The refinement step significantly improves accuracy.
Metric
Accuracy (%)
Initial Prediction Accuracy (AcCimnaa)
53.2%
Final Refined Accuracy (Accunal
56.5%
Improvement
+3.3
all. This analysis provides strong evidence that the risk of negative feedback from mis- dris-guidance is well-mitigated by the robustness of the learned policies.
B.6 QUANTITATIVE ANALYSIS OF EXPERT DIVERSITY
core premise our work that GUIDE's hierarchical disentanglement fosters more function- ally diverse ensemble than other contemporary multi-expert methods. To provide rigorous quan- titative evidence for this claim, we compare the expert diversity of GUIDE against a family of state-of-the-at multi-expert baselines: RIDE (Wang al. 2021). SADE (Zhang al. 2022). and BalPoE (Sanchez Aimar et al. 2023). These methods represent various powerful but "entangled" approaches, they do not employ the same explicit, multi-level mechanisms enforce representa- tion and decision separation. The comparison is conducted on the CIFAR-100-LE (IR=100) test set, with all models configured to use three experts for fair comparison.
Metrics. We employ two standard metrics to quantify diversity:
Representation Diversity: We use Centered Kernel Alignment (CKA) with a linear kernel to measure the similarity between the pre-classifier feature representations each pair of experts. A lower CKA value indicates higher representational diversity.
Prediction Diversity: We use the Q-statistic to assess the pairwise disagreement in experts'
predictions (correct vs. incorrect). A Q-statistic value closer to indicates greater independence (higher diversity) in prediction outcomes.
Results and Analysis. As shown in Table 12, while all multi-expert methods exhibit some level of diversity, GUIDE achieves demonstrably superior degree of disentanglement, The other SOTA
24

---

Published as a conference paper at ICLR 2026
methods, despite their different architectural innovations, consistently show high CKA similarity (> 0.9), suggesting their experts still converge to highly correlated feature spaces a clear sign of unresolved representation entanglement. While some methods like RIDE are effective at reducing prediction similarity (Q-statistic), they struggle to decouple the underlying representations. In stark contrast, GUIDE's competitive specialization mechanism (Level ①) drastically reduces similarity on both metrics. This quantitative analysis provides strong evidence that by explicitly and simulta- neously targeting diversity both the feature and decision levels, GUIDE transforms the ensemble into a committee of far more complementary specialists, which is the foundational prerequisite for our framework's 's superior performance.
Table 12: Quantitative analysis of expert diversity on CIFAR-100-LT (IR=100). Lower values indi- cate higher diversity. Compared a family of strong multi-expert SOTA methods. GUIDE's explicit disentanglement mechanisms produce a significantly more diverse committee of experts.
Model
Avg. CKA Similarity (1)
Avg. Q-Statistic (L)
BalPoß Sanchez Aimar al. 2023
0.95
0.88
RIDE Wang el al. 2021)
0,94
0.83
SADE Zhang ct al. 2022
0,92
0.85
GUIDE
0.75
0.68
B.7 QUALITATIVE VISUALIZATIONS
To provide intuitive understanding of how GUIDE's disentanglement mechanisms operate in practice, we present two sets of visualizations.
Visualizing Representation Disentanglement (Level Figure4shove t-SNE visualizations of feature representations from the three experts for selected tail classes on SEFAR-100-LT. Com- pared to the entangled baseline where expert features largely overlap, GUIDE's s competitive spe- cialization objectives successfully push the expert representations into more distinet, decorrelated subspaces. This visually confirms that our Level 1 disentanglement prevents homogeneity collapse and fosters genuine expert diversity.
Visualizing the Adaptive Policy (Level @). Figure 5 illustrates the learned adaptive policy of plot their sample count against their
the DERM. For selection of classes from ImageNet-LT,, measured epistemic/aleatoric uncertainty and the final gate value Je learned by the controller. The plot clearly shows that classes with few samples (tail classes) tend have high epistemic uncertainty, which in turn leads to a high refinement gate value. Conversely, head classes with high aleatoric uncertainty (e.g., classes with large intra-class variation) receive lower gate value, preventing the model from over-allocaal over-allocating resources to irreducible data ambiguity. This confirms our Level e policy operates as intended.
B.8 CASE STUDY OF THE DERM GATING POLICY
To provide a more intuitive understanding of how the Dynamic Expert Refinement Module (DERM) allocates resources, we conduct case study on specific classes from the iNaturalist 2018 dataset. We analyze the final learned gate values (ge) in relation to the diagnosed class-level uncertainties (Epir,e and Aler,)). The results, presented Table 13 validate that the gating policy operates in a principled and interpretable manner.
Analysis. The case study reveals three distinct operational modes of the DERM, corresponding to different sources of learning difficulty:
Case A (High Epistemic Uncertainty): For Salamandra atra, a rare tail-class species with only 8 training samples, the model exhibits high epistemic uncertainty (0.85), indicating lack of knowledge. As predicted by our framework, GUIDE assigns a very high gate value (0.92), directing maximum refinement capacity to learn better features for this class.
25

---

Published as a conference paper at ICLR 2026
Refinement Strength (g.)
Low
Epistemic Uncertainty (Model Ignorance)
0.5
Medium
Aleatorio Uncertainty (Data Ambiguttyy
High
0.7
0.4
Noisy Head Class: High Ambiguity suppresses Refinement
0.5
0.3
0.4
0.2
0.2
0.1
10
10"
10"
Sample Count per Class on ImageNet-LT (Log Scale)
Figure 5: Visualization of the learned gating policy on ImageNet-LT, where each point represents class. The policy correctly allocates high refinement strength (large gate value go) to data-scarce classes, which exhibit high epistemic uncertainty (model ignorance). Conversely, classes with high aleatoric uncertainty (data ambiguity) receive lower refinement, preventing the model from overfit- ting noise. This confirms our policy operates intended.
Case B (High Aleatoric Uncertainty): For Amanita muscaria, common but visually diverse fungus, the model shows low epistemic uncertainty due to ample data but high alcatoric uncer- tainty (0.78), reflecting the inherent ambiguity of the class. Consequently, GUIDE assigns low gate value (0.31), correctly preventing the model from over-investing resources on samples with irreducible noise.
Case C (Low Uncertainty): For Danaus plexippus, a common and visually distinct butterfly, the model is confident its predictions, resulting in uncertainty of both types. As expected, GUIDE assigns a low gate value (0.25), conserving refinement capacity it not needed.
This case study provides concrete evidence that Level @ policy successfully disentangles the causes of difficulty and executes targeted, resource-efficient adaptation strategy.
Table 13: Case study of learned gate values (g.) for representative classes from iNaturalist 2018. The policy correctly allocates high refinement (high 9.) to classes with high model ignorance (Epir,c) and low refinement to those with high data ambiguity (AIer.c).
Case
Class Example (iNat 2018)
# Train
Epir,e (f)
Aler,c (1)
(1)
A
Salamandra atra (Black Salamander)
8
0.85
0.21
0.92
B
Amanita muscaria (Fly Agaric)
450
0.25
0.78
0.31
C
Danaus plexippus (Monarch Butterfly)
500
0.12
0.15
0.25
B.9 ANALYSIS OF COMPUTATIONAL COST AND PERFORMANCE TRADE-OFTS
To provide comprehensive understanding of the practical implications of our framework, present a holistic analysis of its bont-benefit trade-offs. We conducted detailed comparison against leading multi-expert methods, including RIDE, SADE, and BalPoE, the most challeng- ing CIFAR-100-LT setting (IR=100). Our analysis grounded in data sourced from the original precise measurements.
publications (Wang al. 2021 Zhang al. 2022 and
26

---

Published as a conference paper at ICLR 2026
Cost-Benefit Analysis. As summarized in Table [4. multi-expert LTR methods exist spec- trum of computational complexity. Models like RIDE and SADE are designed for high effi- ciency, employing lighter architectures with significantly lower computational costs. In contrast, our GUIDE framework intentionally invests more computational resources to instantiate its compre- hensive hierarchical disentanglement mechanisms, including dynamic expert refinement pathways. This strategic design choice results in higher training FLOPs and memory usage.
However, this investment yields transformative return in performance, particularly where matters most. While GUIDE's inference latency is only moderately higher than the baselines, its perfor- mance on tail classes is in league of its own. As quantitatively demonstrated our expert diversity analysis (Appendix B.6, Table [2), all other baselines suffer from high expert representation simi- larity (CKA > 0.92). In stark contrast, GUIDE drastically reduces this similarity 0.75, providing direct evidence that the additional computation is successfully utilized to mitigate homogeneity col- lapse. The direct consequence of this improved diversity dramatic boost in performance on the most challenging tail classes. As shown in Table 14 GUIDE's Few-shot Accuracy reaches 36.0%, remarkable +8.0 absolute point improvement over the next best method (BalPoE). This compelling trade-off validates that the computational overhead is highly effective investment for the significant gains in tail-class performance and overall model robustness.
Table 14: Cost-Beneff Trade-off Analysis CIFAR-100-LE (IR=100)) Cost data for RIDE and SADE are from their original papers. Data for BalPoE and GUIDE are from our precise measure- ments.
Computational Cost
Performance
Params (M)
FLOPs
Time (min)
Mem. (GB)
Latency (ms/batch)
Overall Acc.
Few-shot Acc.
Method
(G)
(%)
(%)
RIDE
0.6
0.08
15
0.8
4.5
48.0
22.3
SADE
0.77
0.10
1.0
5.5
49.8
25.1
BalPoE
1.36
0.75
39
1.4
10.91
52.0
28.0
GUIDE
2.64
1.40
51
1.7
13.00
56.4
36.0
Validation Accuracy
Softmax (CE)
RIDE
BaiPoE
LOS
GUIDE (Ours)
0
25
50
75
100
125
150
175
200
Epoch
Figure 6: Convergence analysis CIFAR-100-L2 (IR=100). Compared to several representative methods, GUIDE not only achieves superior final performance but also shows more stable conver- gence throughout the training process. This validates the effectiveness of our hierarchical disentan- glement and two-timescale optimization.
Convergence Rate. Figure illustrates the convergence behavior GUIDE. The plot shows that GUIDE converges smoothly and stably, direct benefit of two-timescale optimization (Level ). This stability ensures that the training process efficient and reliable, reaching a higher final accuracy without the volatility of other complex methods.
27

---

Published as a conference paper at ICLR 2026
B.10 QUALITATIVE ANALYSIS OF CHALLENGING DEN-IGRINNED EXAMPLES
To provide deeper insight into both the capabilities and limitations of method challenging fine-grained examples, we present qualitative analysis that extends beyond typical failure cases. As shown in Figure 7 we analyze three types of scenarios from the iNaturalist 2018 dataset: one where GUIDE succeeds despite numan-leve difficulty, and two where it fails, revealing the boundaries of its performance.
(a) Model Success (Human-Confusiing
(b) Failure: Extreme Similarity (Model & Human-Confusing)
(c) Failure: Confusing Textures (Model & Human-Confusing)
European Crested
Chinese Pond Heron
Broom Fork-moss
Bridled Titmouse
Javan Pond Heron
Juniper Haircap Moss
Figure 7: Qualitative analysis of challenging fine-grained examples from iNaturalist 2018. (a) A success case where GUIDE correctly distinguishes between two visually similar sparrow species. (b) A failure mode due to extreme inter-species similarity between two types of pond herons in non- breeding plumage. (c) failure mode caused by confusing textures between two different species of moss.
Our analysis reveals the following key insights:
Success in Distinguishing the Indistinguishable: Figure 7 first highlights GUIDE's strength. It shows two highly similar species crested tit (European Crested Tit vs. Bridled Titmouse) that non-expert human might struggle to differentiate. GUIDE, however, correctly identifies each species. This demonstrates that our hierarchical disentanglement approach en- ables the model to learn and focus on subtle, yet critical, discriminative features, validating its effectiveness for expert-level fine-grained tasks.
Limitation 1: Extreme Inter-Class Similarity: Conversely, Figure (b)) illustrates a scenario where this capability reaches its limit. The model misclassifies Chinese Pond Heron Javan Pond Heron. In their non non-breeding plumage, these two species are nearly identical, differing only by subtle variations in bill color and proportions. This represents a classic failure mode in extreme fine- finenneined recognition, where the visual gap between species is minimal.
Limitation 2: Confusing Textures and Patterns: Finally, Figure (c) shows a failure mode driven by textural similarity. The model confuses two different species of moss (Broom Fork- moss and Juniper Haircap Moss), which share similar coloration and growth patterns macroscopic level. these cases, where classification relies on minute textural details rather than distinct object parts, the model can falter. This indicates potential boundary where per- formance limited by the backbone's 's ability to capture extremely fine fine-graine grained textural infor- mation.
28

---

Published as a conference paper at ICLR 2026
These cases collectively indicate that while GUIDE significantly pushes the boundaries of recog- nizing rare classes by learning more diverse and robust representations, its performance ulti- mately bounded by the representation power of the backbone network in handling these extreme fine-grained distinctions. This analysis suggests that future work could benefit from integrating more powerful fine-grained feature extractors part-hased attention mechanisms into the GUIDE framework.
B.11 SENSITIVITY ANALYSIS OF KEY HYPERPARAMETERS
To address the reviewer's query hyperparameter sensitivity, this section provides detailed anal- ysis of the key hyperparameters that govern our framework: the diversity loss weights (Adees Adiv) and the meta-policy learning rate (7²)).
Diversity Loss Weights (Adee Adiv). The sensitivity to the diversity loss weights was analyzed in the main paper (Section 4.3 Figure Bh). As shown the figure, GUIDE's overall performance
remarkably stable across broad range of values for both Adec and Aai- This robustness stems from the nature our competitive specialization objective (Level O). The goal of these losses is to create persistent pressure that pushes experts into distinet functional niches, rather than achieving precise numerical balance. As long as these weights are non-trivial (i.e., not zero), the diversification pressure is effective, making the framework insensitive their exact values.
Meta-Policy Learning Rate (1²).. The learning rate for the meta-policy controllers, Me, is crucial component of our two-timescale optimization (Level @). According to Two-Timesc Stochastic Approximation (TTSA) theory, convergence guaranteed as long as the slow timescale learning rate is substantially smaller than the fast timescale rate (n < No)). While the theory provides broad range of valid options, the specific choice 1) can influence the speed and quality of the meta-policy's convergence.
To empirically investigate this, we conducted an ablation study on CIFAR-100-L1 (IR=100) by vary- ing No while keeping all other parameters fixed. The results, presented in Table 15 demonstrate clear and expected trend. very small learning rate (e.g., le-5) leads to suboptimal performance, likely due to insufficient convergence of the meta-policy within the training schedule. As n7 in- creases, performance improves and then enters stable, near-optimal plateau. An excessively large learning rate (e.g., le-3) destabilizes the meta-learning process, causing sharp drop in performance. Our default choice of П7 = le-4 is situated comfortably within the wide and robust optimal range. This analysis confirms that while the choice ni matters, GUIDE is not sensitive to fine-tuning and performs well across a broad range of reasonable values.
Sensitivity analysis of the meta-policy learning rate (1₂) on CIFAR-100-LT (IR=100). The framework is robust across a wide range of optimal values.
Table 15:
Meta-Policy Learning Rate (11) le-5
5e-5
le-4 (Default)
5e-4
le-3
Overall Accuracy (%)
55.8
56.3
56.4
56.1
54.2
B. 1. COMPREHENSIVE ABLATION STUDY ON THE DERM ARCHITECTURE
To provide more thorough validation of our Dynamic Expert Refinement Module (DERM) design, and in response to valuable reviewer feedback, we conducted a comprehensive ablation study to dis- sect the contributions of its key architectural components. We compared our full Adaptive Residual Mixture against three carefully designed alternatives on CIFAR-100-LT (IR=100).
The configurations are defined as follows:
No Refinement (F Only): The baseline where the DERM is completely removed. The final feature is simply the output of the shared foundation pathway, (x).
Static Residual Mixture (g=0.5): The residual structure is kept, but the gate is fixed to a constant value of ge.c = 0.5, making the refinement policy non-adaptive.
29

---

Published as a conference paper at ICLR 2026
56.6
Overall Top-1 Accuracy (%)
56.2
58.0
GUIDE Performance Trend
Chosen Default (T=1.0)
55.6
Mean Accuracy (3 runs)
0.5
1.0
1.5
2.0
4.0
Uncertainty Temperature
Figure 8: Sensitivity analysis of GUIDE with respect to the temperature T CIFAR-100-LT (IR=100). Overall Top-1 accuracy is stable for E (0.5,2.0) validating our choice of T = 1.0 robust default.
Direct Multiplicative Gating:
The refinement policy adaptive, but the residual connection removed. The formulation is f (2; c) 9e.c F (x)).
Adaptive Residual Mixture (Ours): Our full proposed design as described in Eq. equation 6
The results, presented in Table 16 clearly demonstrate the necessity and synergy of our design choices.
Table 16: Comprehensive ablation study of the DERM architecture CIFAR 100-LT (IR=100).
The results validate the importance of both the adaptive gating and the residual structure.
DERM Architecture Configuration Overall Accuracy (%)
No Refinement (Frvand Only)
53.2
Static Residual Mixture (g=0.5)
53.8
Direct Multiplicative Gating
55.1
Adaptive Residual Mixture (Ours)
56.4
The results tell clear story. Starting from baseline without any refinement (53.2%), simply adding a static residual mixture offers a marginal gain (+0.6%), showing that refinement itself is beneficial adaptive policy, even without the stable residual structure (Direct
but insufficient. Introducing Gating), yields a much larger improvement (+1.9% over the baseline). highlighting the critical role of our uncertainty-guided adaptation. Finally, combining both the adaptive policy and the residual structure in our full model unlocks the best performance (56.4%), outperforming the direct gating alternative by significant 1.3 points.
This comprehensive analysis confirms that both core components of our DERM design the prin- cipled, uncertainty-guide guided adaptive gate and the stable residual mixture formulation are essential for achieving the superior performance and robustness of the GUIDE framework.
B.13 SENSITIVITY TO TEMPERATURE SCALING
In Level @, the temperature T scales the logits before the softmax operation, which influences the sharpness of expert distributions for uncertainty estimation. A higher temperature produces softer distributions, while lower temperature leads to sharper ones. To validate the robustness of our framework this hyperparameter, conducted sensitivity analysis CIFAR-100-L.7 (IR=100)
30

---

Published as a conference paper at ICLR 2026
by varying T while keeping all other settings fixed. For each value of T, report the mean and standard deviation of Top-1 accuracy over three independent runs with different random seeds.
As illustrated in Figure the performance of GUIDE remains remarkably stable across wide range of temperature values. The small error bars indicate low variance across independent runs, confirming the stability of our training procedure. While extremely high temperatures (e.g.. 7 > 4) can cause slight degradation by making expert distributions overly uniform and thus masking true uncertainty, the model achieves statistically indistinguishable, near-optimal performance for any T within the standard range of [0.5, 2.0]. Our choice of T = 1.0 as a default is therefore empirically justified as simple and robust setting that does not require dataset-specific tuning.
C LIMITATIONS AND FUTURE WORK
While GUIDE demonstrates strong performance, identify areas for future exploration. The two- step inference strategy, which relies on an initial prediction, poses theoretical risk of mis- guiddge Our analysis in Appendix [B.5 shows that GUIDE's learned policies are remarkably robust this, often correcting initial errors. However, future work could explore more advanced inference-time strategies, such as self-consistency over top-k predictions, to further enhance robustness.
Furthermore, while our evaluation on skewed test distributions provides strong evidence of robust- ness to prior shifts, more systematic validation on corruption-based benchmarks, such as CIFAR- 100-C, could further delineate the generalization boundaries of our disentanglement approach. We hypothesize that the principled uncertainty handling in Level and the stable optimization in Level would confer significant advantages in such noisy, out-of-distribution scenarios. Verifying this remains a promising direction for future research.
31


Reviews:
[{"review_id": "P64BC9IX3p", "reviewer": "Reviewer_8uEt", "summary": "This paper addresses the problem of long tailed recognition (LTR) in multi-expert architectures via a so-called hierarchical disentanglement (GUIDE). This is done by introducing specific mechanisms that address problems such as entaglement causing homogeneity collapse, entanglement of policy-making from ambiguous signals by decomposing epistemic/aleatoric uncertainty with a dynamic refinement module, and introduces an approach for stabilizing convergence via different timescale updates. Experiments on several dataset demonstrate improvements. Please see below for discussion.", "questions": "- some more ellaboration on design choices; e.g. why in fig 3B only marginal gains are achieved with E=4? Why residual gating? (eq. 6)\n- B.4 - robustness to mis-guided refinement: 21.7% of samples were correctly classified after the refinement step, even when guided by incorrect proxy label. What about the rest? what is the net effect?\n- analysis on computational cost, memory footprint\n- please see also above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Below I list some of the main strengths of the paper:\n- the three leveled framework is theoretically motivated \n- results appear to improve over state of the art\n- particularly strong improvement on few-shot classes\n- comprehensive evaluation (e.g. distribution shift)\n- ablation studies validate the contribution of each component separately, and the impact of combining two components leading to further improvements.", "weaknesses": "- Not sufficient discussion on complexity and convergence rates\n- Discussion on how hyperparameters (inc learning rates and various λ) affect guarantees\n- hierarchical disentanglement can mostly be considered as a smart combination of existing approaches\n- would be interesting to include more analysis of failure cases\n- improve for clarity - be clear on problems that are existing challenges and not new, and on what is proven theoretically vs empirically validated", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qVyFHbaaDX", "reviewer": "Reviewer_Vmze", "summary": "This paper introduces GUIDE, a novel framework conceived from the philosophy of Hierarchical Disentanglement.", "questions": "Please consider to improve the experimental analysis on more datasets.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "A novel framework conceived from the philosophy of Hierarchical Disentanglement is proposed. In addition, this paper systematically addresses these issues at distinct levels.", "weaknesses": "The authors only verify the method on computer vision datasets. If possible, the proposed method is suggested on more long-tailed datasets (e.g., MEET) from more domains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZvvnYHJKhv", "reviewer": "Reviewer_ARjA", "summary": "This paper proposes GUIDE, a novel framework for solving the long-tail identification problem. The authors argue that existing multi-expert architectures suffer from three levels of entanglement: representation-decision entanglement, cause-symptom entanglement, and learning-meta-learning entanglement. GUIDE systematically addresses these problems through hierarchical entanglement resolution methods: (1) competitive specialization objectives achieve representation deentanglement, (2) adaptive policies based on uncertainty decomposition achieve policy deentanglement, and (3) dual-timescale optimization achieves optimization deentanglement. Experiments on five long-tail benchmark datasets demonstrate that GUIDE achieves a new SOTA.", "questions": "How do the inference efficiency and training memory consumption of this method compare to other methods?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper provides a novel and insightful diagnosis of long-tailed recognition challenges.\n2. The module design is well-motivated. The uncertainty decomposition, though lacking rigorous justification, provides a practical heuristic for adaptive refinement that works in practice.\n3. The experimental evaluation is thorough and convincing.", "weaknesses": "### 1. **Overstated Connection Between JSD Maximization and Performance Improvement**\n\nThe paper's Theorem 1(b) claims that \"maximizing JSD serves to tighten this performance bound,\" but this relationship is **not rigorously proven**. The mathematical derivation shows:\n\n-  **Problematic**: The gap in Jensen's bound ≠ JSD\n-  **Unproven**: Maximizing JSD → Tightening the bound\n\nThe authors acknowledge in the proof that *\"the gap in the Jensen bound is not strictly equal to JSD\"* and that maximizing JSD *\"empirically encourages\"* diversity. This is an **empirical heuristic**, not a theoretical guarantee. The authors can provide rigorous proof, or explicitly redefine it as an empirical observation rather than a theoretical contribution.\n\n### 2. **Lack of Theoretical Foundation for Uncertainty Decomposition**\n\nThe epistemic/aleatoric uncertainty decomposition (Equations 4-5) is presented as a principled approach but lacks theoretical justification:\n```\nAleatoric: AleT(x) = (1/E)ΣH(pe,T(·|x))\nEpistemic: EpiT(x) = H(p̄T(·|x)) - AleT(x)\n```\n\n**Issues**:\n- No proof that this decomposition correctly identifies \"model ignorance\" vs \"data ambiguity\"\n- No theoretical guarantee that high epistemic uncertainty → need more refinement\n- No validation that high aleatoric uncertainty → avoid over-refinement\n\nWhile the appendix mentions this is a *\"standard and widely-accepted definition in the Bayesian deep learning community,\"* this is **community convention**, not mathematical fact. The entire Level ❷ policy is built on this unproven assumption. The authors need to clarify that this is an empirical design choice and provide empirical validation (e.g., showing the correlation between these quantities and actual model/data properties), or provide theoretical justification for why this decomposition is suitable for this task.\n\n### 3. **Lack of related work**\n\nMDCS: More diverse experts with consistency self-distillation for long-tailed recognition (ICCV2023) is also a method that uses diversity experts and ensemble learning in long-tail recognition, and it needs to be compared and discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z1F99BXzkL", "reviewer": "Reviewer_LdLA", "summary": "GUIDE aims to improve Long-Tailed Recognition problem by solving homogeneity collapse in multi-expert architectures. Three levels of disentanglement in the learning process is identified as the bottleneck and handled sequentially. **Representation-decision entanglement** is handled by two regularization losses which encourages diversity at feature level and predictive logits level separately. **Cause-symptom entanglement** is handled by decomposing epistemic and aleatoric uncertainty to guide the adaptation policy. **Learning-meta-learning entanglement** in meta learning is handled by using different scales of learning rate for two sets of parameters. Through experiments, GUIDE shows efficacy at medium and few shot classes. The ablations show the necessity and synergy between disentanglements.", "questions": "1. What is motivation of designing the refinement strength a monotonically decreasing function with respect to its aleatoric uncertainty?\n2. In sec 3.2, what are pathways? Why are x as the intput to the pathways?\n3. How to get classwise EU and AU using class-level Exponential Moving Averages of the diagnosed uncertainties? as the entropy in (4) and (5) works on the whole predictive distribution.\n4. In table 5, right, is the total uncertainty the entropy of average predictive distribution? What is the performance of using ambiguous signals like high training loss?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper is well written with clarity and well-illustrated in general.\n2. The insights that different levels of entanglements are novel. Especially the disentanglement for representation-decision establishes a new paradigm beyond traditional logit adjustment.\n3. The necessity of most components in the contributions is examined in the ablation study. Hyperparameter sensitivity also looks stable. \n4. Proof in the theorems are valid and clear.", "weaknesses": "1. Theorem 1 only suggests maximizing JSD, not mentioning minimizing the cosine similarity between the feature vectors. Even though the synergy between two competitive regularizers are shown in the ablation, the motivation of discouraging cosine similarity is heuristic.\n2. The applied approach decomposition of AU and EU is classical and recently shown as entangled [1]. Using separate modules for AU and EU may even boost performance. \n3. adaptive residual mixture is not ablated and there is no motivation\n4. It is bit hard to understand the modle architecture from figure 2 as it contains both models, losses and optimization process. \n\n[1] Mucsányi, B., Kirchhof, M., & Oh, S. J. (2024). Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks. Advances in neural information processing systems, 37, 50972-51038.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)