Title: Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Long-tailed Learning

semi-supervised learning
Abstract: Current long-tailed semi-supervised learning methods assume that labeled data exhibit a long-tailed distribution, and unlabeled data adhere to a typical predefined distribution (i.e., long-tailed, uniform, or inverse long-tailed). However, the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. To tackle this challenge, we propose a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. Specifically, CPG operates through a controllable self-reinforcing optimization cycle: (i) at each training step, our dynamic controllable filtering mechanism selectively incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, ensuring that the updated labeled dataset follows a known distribution; (ii) we then construct a Bayes-optimal classifier using logit adjustment based on the updated labeled data distribution; (iii) this improved classifier subsequently helps identify more reliable pseudo-labels in the next training step. We further theoretically prove that this optimization cycle can significantly reduce the generalization error under some conditions. Additionally, we propose a class-aware adaptive augmentation module to further improve the representation of minority classes, and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples. Comprehensive evaluations on various commonly used benchmark datasets show that CPG achieves consistent improvements, surpassing state-of-the-art methods by up to **15.97\%** in accuracy. The code is available at https://github.com/yaxinhou/CPG.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for realistic long-tailed semi-supervised learning (ReaLTSSL), where the distributions of labeled and unlabeled data may differ. To address the challenge of unknown and potentially arbitrary unlabeled distributions, the authors propose to bypass direct distribution estimation. Instead, the method progressively identifies reliable pseudo-labels and adds them to the labeled set, forming an updated dataset with a known class distribution. A Bayes-optimal classifier is then trained using logit-adjusted loss based on this controlled distribution.\n\nThe core of the approach is the Controllable Self-reinforcing Optimization Cycle, which uses agreement between weak and strong augmentation predictions to select consistent pseudo-labels. These labels are treated as reliable and used to update the training distribution in a feedback loop. To further enhance performance, the method introduces a Class-aware Adaptive Augmentation (CAA) module that strengthens minority class representations based on intra-class compactness, and an Auxiliary Branch that enforces prediction consistency across augmentations to better utilize all samples.\n\nThe experimental evaluation is comprehensive. The authors test on four benchmark datasets: CIFAR-10-LT, CIFAR-100-LT, SVHN-LT, and Food-101-LT under a variety of distribution mismatch settings, including long-tailed, inverse, and arbitrary configurations. The method achieves consistent improvements over strong baselines in both aligned and mismatched label/unlabeled distribution settings.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n-The paper targets the mismatch between labeled and unlabeled data distributions,  which reflects real-world data collection scenarios. The proposed self-reinforcing cycle uses consistency between weak and strong views to select pseudo-labels and gradually improve the labeled distribution. This approach is simple to understand and avoids unstable threshold tuning.  \n\n -The method is tested across four standard benchmarks (CIFAR-10/100-LT, SVHN-LT, Food-101-LT) and three types of distribution mismatches (consistent, inverse, arbitrary). The design of diverse imbalance ratios and distribution scenarios strengthens the generality of the results.\n\n-The ablation studies isolate the contributions of each module (CAA, auxiliary branch, and optimization cycle), and the use of pairwise t-tests provides strong statistical support for performance claims.\n\n**Weaknesses**\n\n-The method combines several existing ideas, such as logit adjustment, augmentation-aware consistency, and pseudo-label voting, but **the conceptual novelty is relatively limited**.\n\n-Similar goals have been addressed in earlier works like UPS [1] (uncertainty filtering based on model disagreement) and CADR[2] (reweighting pseudo-labels to correct distribution mismatch). These are not sufficiently discussed.\n\n-The method still relies heavily on a **strong initial supervised phase**, and the assumption that the initial labeled set can provide stable logit-adjusted priors is not fully examined under extreme settings.\n\n\n[1] IN DEFENSE OF PSEUDO-LABELING: AN UNCERTAINTY-AWARE PSEUDO-LABEL SELECTION FRAMEWORK FOR SEMI-SUPERVISED LEARNING \n\n[2] ON NON-RANDOM MISSING LABELS IN SEMI-SUPERVISED LEARNING\n\nQUESTIONS:\nThe proposed method starts training with logit adjustment applied to the limited and imbalanced labeled data, before incorporating any pseudo-labels. Under severe long-tailed settings, where the labeled sample size is extremely small, how reliable is this initial supervised classifier? Has the method been evaluated under more extreme label sparsity (e.g., lower N_max) to stress test this dependency?\n\nHow does the proposed method differ in principle from related realSLL works and other related works that also can handle distribution mismatch? can the authors clarify what is truly novel about the proposed pipeline beyond recombining existing techniques?\n\nThe authors introduce arbitrary unlabeled data distributions to simulate realistic challenges. However, are there real-world datasets or applications where such labeled/unlabeled distribution divergence naturally occurs? \n\nThe experiments use a single backbone (WideResNet-28-2). Has the method been tested on other architectures? \n\nIf these issues are carefully addressed, I would consider raising my score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the long-tailed semi-supervised learning problem. Specifically, it investigates an under-explored and more realistic setting: the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. This is a very challenging problem in semi-supervised learning. To solve this problem, this paper proposes a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. The proposed CPG contains a reliable pseudo-labels filtering mechanism and a Bayes-optimal classifier. This paper also proposes a class-aware adaptive augmentation module to enhance the representation of minority classes, as well as an auxiliary branch to maximize data utilization.\n\nGenerally, the idea is quite novel. Especially, the idea of handling the unknown label distribution of unlabeled samples is quite novel and reasonable. The proposed method is supported by a theory that demonstrates the proposed optimization cycle can significantly reduce the generalization error. The authors conducted extensive experiments to evaluate the proposed method, which suggests that it significantly outperforms previous methods by up to 16.29%. The visual illustration in this paper is informative (like Figure 2), and the writing of this paper is also well done. The code is available in the supplementary material.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths:\n\n1. The investigated problem is interesting, under-explored, and more realistic.\n2. The idea to handle the unknown unlabeled data is novel, and the adopted techniques are reasonable.\n3. The authors theoretically prove that the proposed controllable self-reinforcing optimization cycle can significantly reduce the generalization error.\n4. The experimental performance of the proposed method significantly outperforms the previous methods, and extensive ablation studies validate the effectiveness of each involved component in the proposed method.\n\n#### Weakness:\n\n1. Although the investigated problem is under-explored. I believe the techniques in long-tailed learning will help solve this problem. Therefore, this paper should review the related long-tailed learning methods in the \"Related Work\" section.\n2. Please provide the pseudo-code in the paper, as it will help the authors to reproduce the results.\n\nQUESTIONS:\n1. Different from the previous works in semi-supervised learning, this paper does not use the consistency between the strong augmentation and the weak augmentation. Please explain the reason. \n2. The Bayes-optimal classifier can handle the tailed class samples well, so why is an additional class-aware adaptive augmentation adopted to enhance the minority class representations? \n3. Why is an auxiliary branch adopted in the proposed method?\n4. The proposed method does not estimate the distribution of the unlabeled samples. It just progressively incorporates more unlabeled samples in the labeled dataset. Will this progressive incorporation finally approximate the ground-truth label distribution of the unlabeled dataset?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of semi-supervised learning when the distribution of unlabeled data is arbitrary and unknown.\nThe authors propose the CPG framework, which selectively adds only reliable pseudo-labels to the labeled set from unlabeled set, and applies logit adjustment based on the known class distribution.\nThey further introduce class-aware adaptive augmentation and an auxiliary branch to improve minority class representation and make full use of both labeled and unlabeled data.\nExperiments on several long-tailed benchmarks, including CIFAR-10-LT, CIFAR-100-LT, Food-101-LT, and SVHN-LT, show that CPG significantly outperforms previous methods, with gains of up to 16 percentage points.\nTheoretical analysis also supports the approach by showing that the iterative pseudo-labeling process helps reduce generalization error.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- While previous methods require prior assumptions about the distribution of unlabeled data, CPG can operate robustly even under arbitrary unlabeled data distributions. Moreover, unlike previous ReaLTSSL approaches (e.g., SimPro) that estimate the unlabeled data distribution, CPG only accumulates reliable samples as labeled data, resulting in a more robust training process.\n- The proposed key components (CPG, CAA, auxiliary branch) are thoroughly ablated, and the paper provides detailed analysis across various distribution settings \n- The paper is easy to follow.\n- The proposed method is theoretically justified with formal proofs.\n- The authors provide code to ensure the reproducibility of their results. \n\nWeaknesses: \n- The CPG framework accumulates only reliable pseudo-labels from the unlabeled data into the labeled pool, which poses a risk that tail class samples may be entirely omitted in certain situations. In such cases, the corresponding classes are completely excluded from training and loss balancing (logic-adjustment), potentially resulting in the model failing to learn or classify tail classes at all.\n- Equation (3) uses a confidence threshold to filter reliable pseudo-labels. However, since the performance of the proposed method is likely to be highly sensitive to the choice of this threshold, the paper lacks sufficient ablation analysis regarding this hyper parameter.\n- As mentioned in Section H (Limitation), the current method may encounter problems when noisy labels are present. In real-world scenarios, label noise can exist and potentially harm performance. Therefore, it would be valuable to report experimental results analyzing how CPG performs compared to baseline methods (e.g., FreeMatch, SimPro) under varying levels of label noise in the dataset.\n\nQUESTIONS:\n- Since equation (3) uses a confidence threshold to filter pseudo-labels, it would be helpful to provide an ablation study or sensitivity analysis showing how different threshold values affect the overall performance,\n- As the method assumes noise-free labeled data, it would improve the paper to include experiments evaluating the robustness of CPG under various levels of label noise, as well as comparisons with baseline methods such as FreeMatch and SimPro.\n- It would further strengthen the paper if the effectiveness of the proposed method could be proved on large-scale or real-world datasets beyond standard benchmarks.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of Pseudo-label generation under a Realistic Long-Tailed Semi-Supervised Learning (ReaLTSSL) setting. In this setting, a long-tailed distribution (characterized by an imbalance ratio) is combined with an unlabeled set that has unknown and even arbitrary distribution. \n\nThe authors proposed a iterative method to generate labels for the unlabeled set. The method keeps the pseudo-label for unlabeled images only when its weakly-augmented and strongly-augmented views predict the same class and both predictions exceed a preset confidence threshold. Then the accepted images are moved into the labeled set and the classifier is retrained with logit adjustment. This process is then repeated for more labels. \n\nThe paper also proposed two add-ons CAA (help with minority classes) and an auxiliary FlixMatch-style branch for consistency regularization. \n\nThe method is then evaluated on 4 commonly used benchmarks and achieves a 16.29% performance gain on CIFAR-10-LT outperforming previous methods for long-tailed SSL. The authors also provided theoretical analysis on the optimization cycle.\n\nSTRENGTHS AND WEAKNESSES:\nQuality:\nStrength: The paper did rigorous evaluation on four datasets (CIFAR-10-LT/100-LT, Food-101-LT, SVHN-LT) across multiple imbalance ratios. The paper also conducted ablation studies to isolate the contribution of each module. The paper provided a well-documented code in the supplementary material that is clear and easy to follow. The paper provided a theoretical risk-reduction bound (Theorem 1) and its practical implications are discussed (lines 249-260).\nWeakness: The proposed method CPG is supposed to target the long-tailed distribution with class imbalance. However, only accuracy is reported; macro-F1 or other metrics are not reported.\n\nClarity:\nStrength: The paper is well-written and easy to follow. The algorithm is concisely conveyed in Algorithm 1 together with previously defined equations.\nWeakness: The plots on page 9 feel crowded and break the reading flow. \n\nSignificance:\nStrength: CPG achieves up to 16.29% performance gain on CIFAR-10-LT. The advantage of CPG is clear in Figure 1.\nWeakness: Performance gain on CIFAR-100-LT is lower compared with CIFAR-10-LT. A discussion of why performance drops as class cardinality grows would strengthen the claim of broad significance. The paper only investigated the performance of computer vision datasets. It is not clear how the proposed method performs on other modalities.\n\nOriginality:\nWeakness: The starting filter process uses weak/strong-view confidence. This is similar to FixMatch/FlexMatch. The paper can highlight more clearly how CPG diverges from the other methods conceptually in the appendix.\nStrength: CPG only keeps an unlabeled sample when weak and strong views agree (stricter than Fix/FlexMatch). The accepted samples are promoted to the labeled pool, and then a logit-adjusted head is retrained with pseudo-labels and logit adjustment, which helps with class imbalance.\n\nQUESTIONS:\n1. We see a huge performance gain for CIFAR-10-LT, but the performance gain is less significant for CIFAR-100-LT. Is it possible to provide some reasoning on why the performance gain is less significant for a similar dataset with higher class cardinality? \n\n2. The paper uses confidence interval $\\tau=0.95$ to filter out the pseudo-labels. Is it possible to use a different confidence interval, and how does it affect the performance?\n\n3. The experiments are conducted on long-tailed datasets with class imbalance. Could you also give macro-F1 for the main CIFAR-10-LT and CIFAR-100-LT results? A quick per-class breakdown or confusion matrix would make it clear where CPG’s gains come from.\n\n4. Have you tried CPG on non-vision data (e.g., text, audio)? What components would need to change for other modalities?\n\n5. In line 470, for risk of both the labeled and unlabeled dataset, the data $x_i$ ranges starting from $1$. Maybe it is possible to use notation to differentiate the two or let the latter one start from $N+1$?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}