Title: Reduction-based Pseudo-label Generation for Instance-dependent Partial Label Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: instance-dependent

partial label learning

pseudo-label
Abstract: Instance-dependent Partial Label Learning (ID-PLL) aims to learn a multi-class predictive model given training instances annotated with candidate labels related to features, among which correct labels are hidden fixed but unknown. The previous works involve leveraging the identification capability of the training model itself to iteratively refine supervision information. However, these methods overlook a critical aspect of ID-PLL: within the original label space, the model may fail to distinguish some incorrect candidate labels that are strongly correlated with features from correct labels. This leads to poor-quality supervision signals and creates a bottleneck in the training process. In this paper, we propose to leverage reduction-based pseudo-labels to alleviate the influence of incorrect candidate labels and train our predictive model to overcome this bottleneck. Specifically, reduction-based pseudo-labels are generated by performing weighted aggregation on the outputs of a multi-branch auxiliary model, with each branch trained in a label subspace that excludes certain labels. This approach ensures that each branch explicitly avoids the disturbance of the excluded labels, allowing the pseudo-labels provided for instances troubled by these excluded labels to benefit from the unaffected branches. Theoretically, we demonstrate that reduction-based pseudo-labels exhibit greater consistency with the Bayes optimal classifier compared to pseudo-labels directly generated from the training predictive model.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses instance-dependent partial label learning (ID-PLL) by proposing RPLG, a method that generates reduction-based pseudo-labels using a multi-branch auxiliary model. The key insight is that some incorrect candidate labels strongly correlated with features are difficult to distinguish from correct labels, creating a bottleneck in training. To address this, each branch of the auxiliary model is trained in a label subspace excluding certain labels, allowing instances affected by excluded labels to benefit from unaffected branches. The authors provide theoretical analysis showing that reduction-based pseudo-labels can achieve better consistency with the Bayes optimal classifier, and demonstrate consistent improvements across multiple benchmark and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. The identification of \"disturbing incorrect labels\" that are strongly correlated with features represents a genuine challenge in ID-PLL that existing methods overlook. \n2. The reduction-based pseudo-label generation using multi-branch auxiliary models is creative and theoretically motivated, offering a principled way to handle label interference.\n3. Theorems 1 and 2 provide theoretical justification showing that reduction-based pseudo-labels can achieve better consistency with the Bayes optimal classifier under certain conditions.\n\nWeaknesses：\n1. The method directly uses generated pseudo-labels without any confidence estimation or quality control mechanism. Poor-quality pseudo-labels from unreliable branches could introduce more noise than the original candidate labels, but there's no safeguard against this degradation.\n2. Defining subspaces by simply excluding individual labels ignores semantic relationships between labels. Labels with similar meanings or hierarchical relationships should be considered together when defining exclusion sets. The current approach treats all labels as independent, which may not reflect real-world label structures.\n3. The relationship between the \"disturbing incorrect labels\" concept and the practical multi-branch implementation could be clearer\n\nQUESTIONS:\n1. Beyond α, how sensitive is the method to other hyperparameters in the bi-level optimization? The paper mentions \"carefully selected\" parameters but provides limited guidance.\n2. How do you handle cases where auxiliary branches produce worse pseudo-labels than the original candidate sets? Is there any mechanism to detect and mitigate such degradation? What's the failure rate of pseudo-label generation across different datasets?\n3. Since branches share feature extraction layers to save computation, how do you ensure that the learned representations don't carry interference from excluded labels? Wouldn't this shared representation potentially undermine the core premise of label subspace training?\n\nLIMITATIONS:\nThe requirement for training c auxiliary branches makes the approach prohibitively expensive for datasets with hundreds or thousands of classes, limiting its applicability to large-scale modern applications. Besides, I don't believe that the paper has a potential negative social impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses instance-dependent partial label learning (ID-PLL), where training instances are annotated with candidate labels that may include feature-correlated incorrect labels. Existing approaches leveraging the training model's identification capability often encounter a bottleneck: within the originally given label space, the model struggles to distinguish strongly feature-correlated incorrect labels from true labels, leading to poor-quality supervision signals. To tackle this, this paper proposes reduction-based pseudo-labels generated via a multi-branch auxiliary model, each branch trained in a label subspace excluding specific labels. This design explicitly mitigates interference from hard-to-identify incorrect labels. Theoretically, it proves that reduction-based pseudo-labels exhibit higher consistency with the Bayes optimal classifier than pseudo-labels directly derived from the predictive model, validated through rigorous analysis on benchmark and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. **Strong Novelty** \n   - The focus on the ID-PLL learning paradigm is novel. The paper emphasizes candidate labels that are difficult to identify in the original label space, which are clearly defined as ($\\tau$, $f$, $\\epsilon$)-disturbing incorrect labels in the paper.\n   - The paper presents novel theories by being the first to explore the posterior probability related to the Bayes classifier in label subspaces for partial label learning.\n   - The algorithm designed based on this theory is novel. The core idea of leveraging a multi-branch auxiliary model—where each branch is trained in a label subspace excluding specific labels—provides an insightful solution to mitigate interference from feature-correlated incorrect labels. This design explicitly addresses the limitation of prior methods.  \n\n2. **Theoretical Rigor**  \n   The paper makes significant theoretical contributions by proving that reduction-based pseudo-labels generated in label subspaces exhibit stronger consistency with the Bayes optimal classifier than those derived directly from the predictive model. The theorems (Theorem 1 and 2) and their proofs (Appendix A.1–A.2) are meticulously structured, establishing a rigorous theoretical foundation for the method. This theoretical insight validates the algorithm’s superiority in handling label ambiguity.  \n\n3. **Comprehensive Empirical Validation**  \n   The experimental design is robust, spanning diverse benchmark datasets (e.g., FMNIST, CIFAR-100, TinyImageNet) and real-world applications (e.g., Lost, Yahoo!News). RPLG demonstrates significant performance advantages, particularly on complex datasets like TinyImageNet, highlighting its generalizability. The time complexity analysis shows that RPLG achieves state-of-the-art results with only a marginal linear increase in training time, balancing efficiency and accuracy.  \n\n4. **Clarity in Presentation**  \n   The paper is meticulously written and well-structured, with clear notations and detailed algorithm descriptions (e.g., Algorithm 1 in Appendix A.3). Tables and figures (e.g., Tables 1–4, Figure 2) effectively visualize results, while the ablation studies and sensitivity analysis (e.g., on $\\alpha$ and meta-learned weights) enhance reproducibility. \n\nWeaknesses:\n\n1. It is better to further explain the objective optimization function of meta-learning in Eq. (14).\n\n2. It is better to further highlight the advantages of the algorithm in the interpretation of the experimental results section.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nYes. The authors have adequately addressed the limitations and potential negative societal impact of their work in Appendix A.5 and A.6.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed to leverage reduction-based pseudo-labels to alleviate the influence of incorrect candidate labels and train the predictive model to overcome current training for Instance-dependent Partial Label Learning. The proposed method directly tackles an important gap in existing PLL research, offering a compelling motivation. In addition, the theoretical analysis is rigorous and well-integrated with the proposed methodology.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. This paper demonstrates a high standard of scholarly presentation, with clear organization and a coherent logical structure that aligns well with the expectations of top-tier conferences in the field.\n2. This paper addresses a critical challenge in Instance-Dependent Partial Label Learning (ID-PLL) — distinguishing the ground-truth label from misleading candidate labels — through a novel multi-branch auxiliary model architecture. This design directly tackles an important gap in existing PLL research, offering a compelling motivation.\n3. The theoretical analysis is rigorous and well-integrated with the proposed methodology. The mathematical derivations are sound and effectively support the algorithmic design.\n4. The experimental evaluation is thorough and well-executed, showcasing strong empirical performance across multiple benchmark datasets. The results clearly demonstrate the effectiveness of the proposed approach.\n\n**Weaknesses:**\n1. The paper would benefit from a more in-depth discussion of the theoretical component’s significance, particularly how the theory provides insights into or strengthens the proposed method.\n2. Since the multi-branch architecture employed is not entirely novel, this part of the contribution may be perceived as incremental. The authors should clarify what specific innovations differentiate their approach from prior multi-branch frameworks.\n3. The hyperparameter $\\alpha$ plays an important role in the proposed method. A more detailed ablation study focusing on its impact would provide deeper insight into the model's robustness and sensitivity.\n4. The proposed method uses a meta-learner to optimize the sample-wise weight $w$, but only compares it against a uniform weighting strategy (RPLG-NM). Incorporating comparisons with alternative weight learning or fusion strategies (e.g., self-training, attention mechanisms, or ensemble-based reweighting) would further substantiate the claimed advantages.\n\nQUESTIONS:\nPlease see the weaknesses which need for more evidences and explanations.\n\nAdditionally, please answer the following questions:\n1. While the method is applicable to general PLL settings, the authors should evaluate and report its performance under uniformly random candidate label selection (i.e., uniform PLL) to demonstrate broader applicability.\n\nLIMITATIONS:\nThe proposed method uses a meta-learner to optimize the sample-wise weight w, but only compares it against a uniform weighting strategy (RPLG-NM). Incorporating comparisons with alternative weight learning or fusion strategies (e.g., self-training, attention mechanisms, or ensemble-based reweighting) would further substantiate the claimed advantages.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a key challenge in Instance-dependent Partial Label Learning (ID-PLL). The authors identify that existing methods, which rely on the model's self-identification capability, often encounter a training bottleneck because they struggle to distinguish the true label from incorrect candidate labels that are strongly correlated with instance features. To overcome this, the paper proposes a novel method, RPLG (Reduction-based Pseudo-label Generation). The core idea is to train the predictive model using \"reduction-based pseudo-labels\". These are generated by a multi-branch auxiliary model where each branch is trained in a \"label subspace\" that explicitly excludes one specific label, thereby avoiding its interference. The framework further incorporates a meta-learner to perform a weighted aggregation of the outputs from all branches to produce the final pseudo-label. Theoretically, the authors prove that their proposed pseudo-labels exhibit greater consistency with the Bayes optimal classifier.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The concept of \"reduction-based pseudo-labels\" and the use of label subspaces to mitigate interference  is a highly novel contribution. The approach directly targets a well-articulated shortcoming in prior work, namely the model's inability to handle \"disturbing incorrect labels\", rather than offering a mere incremental improvement.\n2. The method is not just heuristic but is supported by a solid theoretical analysis. The authors provide theorems (Theorem 1 and 2) to formally justify the approach, demonstrating that the generated pseudo-labels are more consistent with the Bayes optimal classifier compared to those from the predictive model itself.\n3.The paper presents comprehensive and convincing experimental results. RPLG is shown to significantly outperform a wide range of baselines, including recent state-of-the-art methods, across various benchmark and real-world datasets. The ablation study comparing RPLG with RPLG-NM effectively validates the contribution of the meta-learning component.\n\nWeaknesses:\n1. The multi-branch model design incurs significant overhead when the number of classes is large. For example, TinyImageNet training time is substantially higher.\n2. While the paper uses a weight vector to fuse predictions from each branch, it does not analyze how the branches interact or whether some are redundant.\n3. While a variant without meta-learning (RPLG-NM) is compared, the necessity of using one branch per label is not ablated or justified. Could fewer branches suffice?\n4. All baselines use the same backbone and data augmentation, but many of them are not designed for ID-PLL. It would strengthen the case to compare against ensemble-based or meta-learning based approaches beyond the PLL field.\n\nQUESTIONS:\n1. Must the number of branches equal the number of labels? Have you considered using fewer branches, such as via label clustering or dimensionality reduction, to reduce computation?\n2. How does the method handle labels that are both correct and disturbing in different samples?\n\nLIMITATIONS:\n1. The method trains one branch per label. As the number of classes increases (e.g., 200 in TinyImageNet), the computation and memory cost grows rapidly.\n2. The theoretical guarantees rely on ideal assumptions, such as the model closely approximating the posterior distribution and satisfying the Tsybakov condition.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}