Title: Unlocker: Disentangle the Deadlock of Learning between Label-noisy and Long-tailed Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: Long-tail learning

Noisy label learning
Abstract: In real world, the observed label distribution of a dataset often mismatches its true distribution due to noisy labels. 
        In this situation, noisy labels learning (NLL) methods directly integrated with long-tail learning (LTL) methods tend to fail due to a dilemma: NLL methods normally rely on unbiased model predictions to recover true distribution by selecting and correcting noisy labels; while LTL methods like logit adjustment depends on true distributions to adjust biased predictions, leading to a deadlock of mutual dependency defined in this paper.  
        To address this, we propose \texttt{Unlocker}, a bilevel optimization framework that integrates NLL methods and LTL methods to iteratively disentangle this deadlock. The inner optimization leverages NLL to train the model, incorporating LTL methods to fairly select and correct noisy labels. The outer optimization adaptively determines an adjustment strength, mitigating model bias from over- or under-adjustment. We also theoretically prove that this bilevel optimization problem is convergent by transferring the outer optimization target to an equivalent problem with a closed-form solution. 
        Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness of our method in alleviating model bias and handling long-tailed noisy label data. Code is available at \url{https://anonymous.4open.science/r/neurips-2025-anonymous-1015/}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of training with both long-tailed and noisy-labeled data, a setting commonly encountered in real-world applications but rarely studied jointly. The authors identify a mutual dependency---or ``deadlock''---between noisy label learning (NLL), which relies on unbiased model predictions, and long-tail learning (LTL), which relies on an accurate class prior. To overcome this, the paper proposes \\textit{Unlocker}, a bilevel optimization framework: the inner loop applies NLL enhanced with logit adjustment, and the outer loop adaptively optimizes the adjustment strength parameter $\\tau$. The method is theoretically justified with convergence proofs and empirically validated on synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper addresses an important but under-explored challenge of jointly handling label noise and long-tailed distributions. The proposed bilevel optimization framework, Unlocker, is conceptually sound, novel, and supported by strong empirical results. However, the method is only evaluated on traditional architectures like ResNet and relies on a clean balanced subset whose reliability is not deeply examined, raising concerns about generalizability. While the core idea is promising, broader evaluation and deeper analysis would strengthen the paper.\n\nQUESTIONS:\n1. How sensitive is the method to errors in the selection of the clean balanced subset $D_{bc}$? Could small amounts of residual noise significantly affect the outer optimization?\n2. What is the computational overhead of the bilevel optimization in practice? Could the method scale to larger models or datasets like ImageNet-1k?\n3. Have the authors tried Unlocker with modern vision backbones (e.g., Vision Transformers, EfficientNet)? How well does it generalize beyond ResNet?\n4. Could you show an ablation analysis for the EMA component or the effect of fixing $\\tau$ vs. learning it?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
The authors present a bi-level optimization framework for learning balanced predictors on noisy data. They demonstrate the tractability of the problem theoretically and show strong results against baselines in real-world and synthetic datasets.

STRENGTHS AND WEAKNESSES:
# Strengths
1. The paper tackles a challenging and relevant problem.
2. The empirical results are promising in some settings.
3. The method introduced is a novel combination of LA and NLL which outperforms naive combinations.

# Weaknesses
1. The paper is not clearly written, and the clearest explanation of the method (the algorithm) is relegated to the appendix. The dependence of the NLL algorithm on the LTL algorithm is nuanced and should be explained more carefully.
2. The results are not dramatically better than SOTA methods on real-world datasets. More could be done to emphasize where Unlocker is strongest (when noise dramatically alters the distribution, perhaps)

QUESTIONS:
1. In table 3, Unlocker is incorrectly highlighted as the best result when GSS outperforms it.
2. Why are LTNLL baselines included in the synthetic results but not the real datasets? And why not the NLL-only baselines in the synthetic? 
3. The flowchart in figure 3 does not provide a clear explanation of the method, specifically in the selection and correction steps. A clearer description may help readers understand the novelty of the work.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address the dual dilemma of the noise-label problem and long-tailed distribution. Existing methods often neglect the true distribution deviation and tend to perform inaccurate optimizations. To overcome this limitation, this work proposes a new method called Unlocker to disentangle the deadlock of the dual dilemma. Unlocker consists of an inner optimization that leverages NLL and LTL methods to ensure fair optimization, and an outer optimization that controls the adjustment strength to alleviate the model bias. Theoretical and empirical results demonstrate the superiority of Unlocker.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This paper has a good motivation. Existing NLL methods rely on unbiased predictions, and LTL methods rely on true distributions. Therefore, the combinations of these two methods indeed cause a deadlock. The authors have adequately supported their motivations by providing detailed explanations and empirical results.\n2. Different from previous works, this work proposes both inner optimizations and outer optimizations. It seems that previous works can be integrated into the inner optimizations of this work, while the outer optimizations can ensure the accuracy at a high level. This framework can also motivate more generalizable works in future research.\n3. Both theoretical and empirical analyses are provided to support the proposed framework. Moreover, the performance gains on popular datasets are constructive. The authors have conducted additional analyses to validate the effectiveness of different modules and hyperparameters.\n\nWeaknesses:\n\n1. It seems that the inner optimization objective and the outer optimization objective are introduced separately. What is the jointly optimized objective? Or could you provide the combined optimization procedure or pseudo-code?\n2. In theoretical analysis, the approximation for $\\tau$ can be solved using the least-squares method. However, in experiments, the parameter $\\tau$ still requires adjustments. Are there any potential explanations?\n3. The distribution estimation plays an important role in the problem. Are there any quantitative metrics for evaluating the effectiveness of distribution estimation? Or how to compare the effectiveness with previous estimating methods?\n\nQUESTIONS:\n1. In lines 77-78, it is said that $\\tau > 0$ is used for post-hoc adjustment, while $\\tau < 0$ can be incorporated as logit adjustment loss during training. What is the intrinsic reason? And what is the value range of $\\tau$ in practice? Is there any guidance?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe problem of  long-tailed label-noise learning is considered. This paper introduces \"Unlocker,\" a novel bilevel optimization framework designed to address the \"deadlock\" dilemma in long-tailed noisy label learning (LTNLL). This deadlock arises because noisy label learning (NLL) methods rely on unbiased predictions to correct labels, while long-tail learning (LTL) methods, like logit adjustment, depend on true label distributions to adjust biased predictions, creating a mutual dependency problem. Unlocker disentangles this by using an inner optimization to train the model with NLL methods and LTL methods for fair noisy label selection and correction, and an outer optimization to adaptively determine an adjustment strength to mitigate model bias. The framework is proven to be convergent through theoretical analysis, and extensive experiments on synthetic and real-world datasets demonstrate its effectiveness in alleviating model bias and handling long-tailed noisy label data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths  \n1. The propose of the deadlock dilemma between NLL and LTL.  \n2. The framework is proven to be convergent through theoretical analysis, and extensive experiments on synthetic and real-world datasets demonstrate its effectiveness in alleviating model bias and handling long-tailed noisy label data.   \n3. Code is publicly available.  \n\nWeaknesses  \n1. The requirement of a balanced clean validation set, which may not be available in real-world scenarios. An ablation study of the validation dataset size is necessary.\n\nQUESTIONS:\nQuestions\n1. It is well-known that DNNs trained with long-tailed data is prone to give miscalibrated class probability estimates. There are some recent advances from the community of probability calibration showing that calibration in long-tailed classification can be significantly improved by some adaptive label smoothing technique e.g. [CVPR'23]. Have you tried better-calibrated DNNs and will it improve  noisy label selection?\n2. Some important works on noisy label selection are missed in the discussion of related works, e.g. [ICML'20]. \n\nClass Adaptive Network Calibration. CVPR'23.  \nLearning with bounded instance and label-dependent label noise. ICML'20.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}