Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning
Authors: Yaxin Hou
Bo Han
Yuheng Jia
Hui LIU
Junhui Hou
Venue: NeurIPS 2025 poster ()
Keywords: Long-tailed Learning
semi-supervised learning
Primary area: general_machine_learning
TL;DR: We propose an effective long-tailed semi-supervised learning method with unknown unlabeled data distribution.

Abstract: Current long-tailed semi-supervised learning methods assume that labeled data exhibit a long-tailed distribution, and unlabeled data adhere to a typical predefined distribution (i.e., long-tailed, uniform, or inverse long-tailed). However, the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. To tackle this challenge, we propose a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. Specifically, CPG operates through a controllable self-reinforcing optimization cycle: (i) at each training step, our dynamic controllable filtering mechanism selectively incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, ensuring that the updated labeled dataset follows a known distribution; (ii) we then construct a Bayes-optimal classifier using logit adjustment based on the updated labeled data distribution; (iii) this improved classifier subsequently helps identify more reliable pseudo-labels in the next training step. We further theoretically prove that this optimization cycle can significantly reduce the generalization error under some conditions. Additionally, we propose a class-aware adaptive augmentation module to further improve the representation of minority classes, and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples. Comprehensive evaluations on various commonly used benchmark datasets show that CPG achieves consistent improvements, surpassing state-of-the-art methods by up to **15.97\%** in accuracy. The code is available at https://github.com/yaxinhou/CPG.

Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed
Semi-Supervised Learning
Yaxin Hou', Bo Han', Yuheng Jia 1.2.3, 1. Hui Liu", Junhui Hou
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China "School of Computing Information Sciences, Saint Francis University, Hong Kong, China *Department of Computer Science, City University of Hong Kong, Hong Kong, China yaxin@seu. edu. en, hanbo@seu. edu. en, ybjiaßseu. edu. cn, h2liu@sfu edu. jh. houuciiu edw...ueduu...... .kk
Abstract
Current long-tailed semi-supervised learning methods assume that labeled data cxhibit a long-tailed distribution, and unlabeled data adhere to typical predefined distribution (i.e., long-tailed, uniform, inverse long-tailed). However, the dis- tribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. To tackle this challenge, Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. Specifically, CPG operates through a controllable reif-reinnoreing optimization cycle: (i) at each training step. our dynamic control- lable filtering mechanism selectively incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, ensuring that the updated labeled dataset follows known distribution; (ii) we then construct a Bayes-optimal classifier using logit adjustment based on the updated labeled data distribution; (iii) this improved classifier subsequently helps identify more reliable pseudo-labels in the next training step. We further theoretically prove that this optimization cycle can significantly reduce the generalization error under some conditions. Additionally, we propose class-aware adaptive augmentation module to further improve the representation of minority classes, and an auxiliary branch to maximize data utiliza- tion by leveraging all labeled and unlabeled samples. Comprehensive evaluations on various commonly used benchmark datasets show that CPG achieves consistent improvements, surpassing state-of- state-of-the-art the-art methods by up 15.97 % in accuracy. The code is available at https:: //////gttt/////////////////thhht com/ wwt.//cwoo...o.//eeeeeee waayyyaaiuincyea
propose Controllable Pseudo-label
Introduction
Over the past decade, semi-supervised learning (SSL) has emerged mainstream paradigm for enhancing the performance of deep neural networks (DNNs) in label-scarce domains like medical diagnosis [1, 2, 3] by leveraging abundant unlabeled data. The dominant SSL framework utilizes unla- beled samples by generating pssuud-labels from high-confidence model predictions [4, 5, 6]. However, conventional SSL methods [7, 8, 9] rely the unrealistic assumption of balanced and aligned distri-
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

3000
3000
3000
2000
2000
2000
1000
1000
1000
Clo6s Indox
Class Idd
Indox
(a) FreeMatch
(b) SimPro
(c) Ours
Figure 1: Comparison of pseudo-label predictions among Free.atch [8], SimPro [13], and our CPG
under arbitrary unlabeled data distribution. GT denotes the ground-truth unlabeled data distribution.
TP (FP) denotes the predicted true (false) positive pseudo-labels. The dataset is CIFAR-10-LTT with ( [maaz Mmaz: It. Yr) (400, 4600, 50, 50), where Maaz (Mmar) denotes the number of samples in the most frequent class of the labeled (unlabeled) dataset, while (7x) denotes the imbalance ratio of the labeled (unlabeled) dataset. Our CPG can generate more reliable pseudo-labels than FreeMatch and SimPro in both minority classes like class 2, and majority classes like class 8, 9.
butions for labeled and unlabeled data. Long-tailed semi-supervised learning (LTSSLL) [10, 11, 12] relaxes the balance assumption, however, it still suffers from potential distribution mismatches between labeled and unlabeled data.
A motivating example. In urban traffic monitoring systems, vehicle images captured by surveillance cameras are classified into distinct categories (e.g., bicycles, trucks, motorcycles) based on their visual characteristics. This presents a long-tailed semi-supervised learning scenario, where labeled data exhibit long-tailes distribution (with common vehicles like cars dominating and specialized vehicles like ambulances being underrepresented). Mcanwhile, unlabeled data collected across different urban environments share the same vehicle categories but may exhibit unknown distributional shifts. These distributional shifts arise from spatial variations (e.g.. higher proportions of personal vehicles in residential areas) or temporal fluctuations (e.g., increased taxi frequency during rush hours).
Recent advances in realistic long-tailed semi- semi-supervised learning (ReaLTSSL) have attempted to address the distribution mismatch between labeled and unlabeled data. For instance, ACR [14] adjusts logits based on the distance between the estimated unlabeled data distribution and the predefined anchor distributions, while CPE [15] employs multiple classifiers (experts) to model diverse unla- beled data distributions. Although effective, both methods assume that unlabeled data adhere to typical predefined distribution (i.e., long-tailed, uniform, or inverse long-tailen long-tailed), which may not hold in practice. The recent method SimPro [13] alleviates this limitation through an Expectation- Maximization (EM)-based pseudo-labeling method. However, as illustrated in Fig. ( SimPro's distribution estimation ability degrades when unlabeled data follow an unknown arbitrary distribution. In summary, existing methods rely on a confidence threshold to select high-confidence pseudo- for distribution estimation, which subsequently guides pseudo-label generation. However, as shown Figs. and 3, this strategy becomes unreliable when handling the unknown arbitrary unlabeled data distribution, as high-confidence pseudo-labels may still contain substantial errors. These inaccuracies propagate during training, resulting in confirmation bias and model performance degradation.
p. Unlabeled dataset (e):): Updated labeled dataset fi: Constructed Bayes-optimal classitier
Identified reliable pseudo-Jabels (D((1)
D((0
=(0)
Du
(1)
(()
fo on
Add
f on
Ad
ft on
(0)
x(1)
(())
Figure 2: Overview of the controllable self-reinforcing optimization cycle.
To address these challenges, we introduce a Controllable Pseudo-label Generation (CPG) framework that employs controllable self-reinforcing optimization cycle to handle the unknown arbitrary unlabeled data distribution problem in ReaLTSSL, As illustrated in Fig. CPG utilizes dynamic controllable filtering mechanism to identify reliable pseudo-labell These reliable pseudo-labels are combined with the labeled dataset to construct an updated labeled dataset Di(t) with a known distribution ग (t). We then construct a Bayes-optimal classifier ft via logit adjustment [16] based on the known distribution π (1). Note that the Bayes-optimal classifier is unaffected by the unlabeled data distribution. The Bayes-optimal classifier ft, in turn, helps identify more reliable pseudo-labels the next training step. Through this optimization cycle, an increasing number of unlabeled samples progressively incorporated as reliable ones, enabling the framework to capture the unlabeled data

---

Fartarm
Smoro
Freebrch SinPoo
SweWarch
100
100
Bro Rate Mhi
Accurasy (50)
15
60
40-
25
45
65 Epech
85
105
125
145
65 Epoch
85
166
125
165
0.1
3
45
65 105 125 145
Esoch
(a) Pseudo-label error rate
(b) Pseudo-label utilization rate
(c) Testing accuracy
Figure 3: Comparison of pseudo-label error rate (a), pseudo-labe utilization rate (b), and testing accuracy (c) among FreeMatch [8], SimPro [13], and CPG under arbitrary unlabeled data distribution. The dataset is CIFAR-IO-LT with (Nmax Mmaz: "Rs Tu) (400, 4600, 50, 50). The vertical gray dotted line indicates the initiation of pseudo-labeling our method. Our CPG can generate pseudo-labels with a lower error rate and comparable utilization rate, achieving superior testing accuracy compared to both FreeMatch and SimPro.
distribution in later training stages implicitly (See Fig. in Appendix A). Theoretically, prove that this optimization cycle can significantly reduce the generalization error under some conditions. Empirically, Fig. (c) shows the superior pseudo-label prediction ability of method.
To further improve our CPG, we introduce class-aware adaptive augmentation (CAA) module and an auxiliary branch. The CAA module enhances minority class representations by dynamically adjusting augmentation intensity based on class compactness. Specifically, for classes with higher compactness (indicating lower intra-class diversity). CAA applies smaller augmentation radius during representation synthesis to refine decision boundaries more effectively. Meanwhile, the auxiliary branch maximizes data utilization by enforcing prediction consistency between different augmentation views all labeled and unlabeled samples.
In summary, the main contributions of this work are as follows.
We propose Controllable Pscudo-label Generation (CPG) framework to handle the unknown arbitrary unlabeled data distribution problem in ReaLTSSL. The framework expands the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset, and trains the model on the updated labeled dataset with a known distribution, making unaffected by the unlabeled data distribution.
We theoretically prove that our controllable self-reinforcing optimization cycle can significantly reduce the generalization error under some conditions.
We further propose class-aware adaptive augmentation module to enhance the representation of minority classes, and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples.
Comprehensive experiments on four commonly used benchmarks (CIFAR- 10-LT, LIFAR-100-LY Food- 101-L, and SVHN-LT) under various labeled and unlabeled data distributions validate that our CPG achieves state-of-te state-of-the-art the-art performances with 15.97% % maximum improvement.
2 Related Work
Long-Tailed Learning (L.TL) addresses the challenge of training effective models data with long- tailed distributions, where majority of classes are represented by only a few samples. Prevailing LTL methods can be categorized into three groups: re-sampling. logit adjustment, and ensemble learning. Re-sampling [17, 18, 19, 20J balances classes by under-sampling majority classes or over-sampling minority classes. Logit adjustment [21, 22, 23] modifies the predicted logits to compensate for class imbalance. Ensemble learning [24, 25, 26] combines multiple classifiers (experts) to improve the performance and robustness of the model. Despite their success in supervised learning, extending them to long-tailed semi-supervised learning (LTSSL) requires extra effort, as exploiting unlabeled samples is not trivial.
Semi-Supprrised Learning (SSL.) provides powerful framework improving model performance in label-scarce domains by leveraging abundant unlabeled data. Early SSL methods rely self-

---

training with ppeudo-labeling where model refinement depends on the quality of generated pseudo- labels. Recent advances integrate pseudo-labeling [5, 6] with consistency regularization [27, 28], enforcing prediction invariance between weak and strong augmentation views to enhance robustness. Prominent examples include FixMatchh [7], FlexMatch [29], FreeMatch [8], SoftMatch [9], and SemiReward [30], which primarily differ in their confidence thresholding strategies to balance pseudo-label quality and quantity. While effective in settings with balanced labeled and unlabeled data distributions, these methods struggle with long-tailed labeled data or mismatched distributions between labeled and unlabeled data.
Long- Long-Tailed Semi-Supervised Learning (LTSSL) addresses the challenges arising from the long- tailed distribution. Existing LTSSL methods integrate long-tailed learning techniques [31, 32, 33, 34, 35] with frameworks like FixMatch. For example, ABC [10] employs an auxiliary classifier to improve minority class generalization, CReST [36] selectively generates pseudo-labels for minority class samples based on class frequency, and CoSSL [11] introduces feature enhancement strategies to refine classifier learning. However, these approaches overlook distribution mismatches between labeled and unlabeled data, key limitation addressed by realistic long-tailed semi-supervised learning (ReaLTSSL). Recent ReaLTSSL methods include ACR [14], which adjusts logits based on the distance between the estimated unlabeled data distribution and the predefined anchor distributions; CPE [15], which employs multiple classifiers (experts) to model diverse unlabeled data distributions; Meta-Expert [37]. which significantly enhances CPE by incorporating the expertises of different experts, and SimPro [13]. which estimates unlabeled data distributions through high-confidence pseudo-label counts and adapts eseudo-labe generation probabilities correspondingly. Despite their contributions, ACR, CPE, and Meta- Meta-Expert require prior knowledge of the unlabeled data distribution. SimPro, meanwhile, fails to estimate the unlabeled data distribution accurately when the labeled and unlabeled distributions diverge significantly. These limitations restrict their practical applicability.
3 Proposed Method
3.1 Preliminaries
Problem formulation and challenges. In ReaLTSSI, we consider labeled dataset D; = fart v N of size N and an unlabeled dataset D., {a 3-=1 of size M, where both datasets share identical feature and label spaces. Here, 2d denotes the i-th labeled sample with ground-truth label vf E (1..... .1...., C), 2 represents the j-th unlabeled sample, and C is the total number of classes. Let No denote the number of labeled samples in class c, defining the labeled data imbalance ratio as max, No While we can theoretically assume that M, represents the number of samples for class n = min, No in Da., with its imbalance ratio Yo max. M.. in practice the unlabeled data follow an unknown min, M.
arbitrary distribution, making M, and %,. unavailable. The goal of ReaLTSSL is to train classifier F: Rd - [l.... C). parameterized by 0, using D; and D., to achieve robust generalization across all classes. This presents key challenge for SSL and LTSSL methods, most existing methods depend on either balanced or aligned distributions between D2 and D., or require prior knowledge of the unlabeled data distribution. practice, the distribution of D. is typically unknown, limiting the capability of the existing methods.
Logit adjustment. In in models trained on such data exhibiting an inherent bias toward majority classes. To mitigate this issue, numerous long long-tailed learning methods aim to enhance minority class performance without degrading accuracy in majority classes. A prominent approach, logit adjustment [16], promotes larger relative margins between the minority positive and majority negative classes. It has been theoretically established as the Bayes-optimal solution for long-tailed learning. The logit-adjusted softmax cross-entropy loss (LA loss, (1a) is defined as:
scenarios, data typically follow long-tailed distribution, resulting
(o(at))) P.
Cy f(g(z)) -log Eyeloy (o
where g(-) and f(() denote the encoder and classifier, respectively, (((()) is the predicted logit of the y-th class, and P,, denotes the class prior of the y-th class.

---

3.2 Our CPG Framework
Framework overview. To address the above- meove-mmntioned challenges, we introduce CPG, a controllable pseudo-label generation framework tailored for ReaLTSSL that operates without restrictive assump- tions or prior knowledge about the distribution of the unlabeled dataset. Specifically. CPG adopts a controllable self-reinforcing optimization cycle, class-aware adaptive augmentation module, and an auxiliary branch. The first employs a dynamic controllable filtering mechanism to identify reliable pseudo-labels. These pseudo-labels are combined with the original labeled dataset to iteratively update the labeled dataset, and progressively construct a Bayes-optimal classifier through logit ad- justment. The improved classifier, in turn, helps identify more reliable pseudo-labels for subsequent training steps. The second enhances discriminative representations for minority classes within each training step. The last maximizes data utilization by leveraging all labeled and unlabeled samples. Unlike existing methods that estimate the unlabeled data distribution based on high-confidence pseudo-labels, our CPG incorporates reliable leeudo- from the unlabeled dataset into the labeled dataset, and trains the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution.
Controllable seff-reinforcing optimization cycle. The optimization cycle incorporates three key components, i.e., dynamic controllable filtering, iterative labeled dataset construction, and Bayes- optimal classifier construction. We present their details in the following paragraphs.
Dynamic controllable filtering to identify reliable pseudo-labels. Conventional SSL and LTSSL methods typically generate pseudo-labels by propagating predictions from weak augmentation views to their strong augmentation views. Although this strategy improves model robustness, it inherently introduces more error pseudo-labels when handling the unknown arbitrary unlabeled data distribution, as evidenced in Figs. ((a) and 1(b). To overcome these limitations, we propose to identify reliable pseudo-labee leuuls to extend the labeled dataset instead of enforcing prediction consistency between weak and strong augmentation views. For each unlabeled sample I. with weak augmentation view Qu (zw) and strong augmentation view (x.),, we derive predicted pseudo-labele and corresponding confidence scores as:
Two Tw = arg max (ax
Gsy q. = arg max(o(f((h (fxho(((h))),
(2)
where arg max(.) and max(-) extract the pseudo-labels and confidence scores, respectively, and a()) denotes the softmax function. Let h, = (z..) represent the representation produced by g(-) from Pu (z.u,, with Tue and T.e denoting the pseudo-label and its associated confidence score
predicted for hws respectively. h2, G.. and T are defined similarly for f2g (zw). We express the binary sample mask for selecting reliable pseudo-label as:
I= I(@@ T) I(q. r) I(q = qe),
where 7 denotes the confidence threshold and represents element-wise multiplication. Since samples may receive confficting pscudo-labela labels across diffeernnt training steps according to Eq. (3), we introduce a voting strategy to ensure consistent pscudo-label assignments for each sample. As illustrated in Fig. 1(c), the combination of Eq. (3) and the voting strategy effectively controls pseudo-label generation and enhances pseudo-label reliability, demonstrating its strong performance.
Iterative labeled dataset construction. During the training process, we maintain both a labeled dataset and a -seude-labelld dataset (i.e., identified reliable aseudo-labels at each training step. with class frequencies denoted by = {nI,- ..... ne) } and m {mi,.... mc). respectively. The updated labeled dataset's class frequency is defined as Φ = {dI,.... Φc where de = n. + m, denotes the number of samples for class e in the updated labeled dataset. Accordingly, the class distribution for the updated labeled dataset is given by:
de
π = (m...... ". Re Evelcy De
where Te represents the label frequency of the c-th class in the updated labeled dataset. This yields an updated labeled dataset with a known distribution at each training step, enabling the iterative construction of a Bayes-optimal classifier.
Bayes-optimal classifier construction. Leveraging the known distribution of the updated labeled dataset, we construct classifier by minimizing the LA loss (fta) defined in Eq. (1). The resulting classifier minimizes the balanced error and yields Bayes-optimal predictions [16]. By iteratively
5

---

constructing Bayes-optima classifier using the updated labeled dataset (with a known distribution at each training step), our CPG framework generates increasingly reliable pseudo-labels in subsequent training steps. Although our method does not explicitly estimate the unlabeled data distribution, the progressive incorporation of reliable pseudo-labels naturally approximates the ground-trunt truth unlabeled data distribution as training progresses. This is because the model's s increasing confidence enables it to gradually include more diverse samples, including those from minority classes, improving distribution alignment (as demonstrated in Appendix A Fig. 5).
While consistency regularization between augmentation views is commonly used SSL, we avoid it in our controllable self-reiinoriing optimization cycle for two key reasons. First, pseudo-label quality is highly sensitive to the distribution mismatch between labeled and unlabeled data in LTSSL. Second, propagating eтror pseudo-labels from weak augmentation views to strong augmentation views can reinforce the model's learning of incorrect predictions, leading to error accumulation and performance degradation (as demonstrated in Figs. and 3).
Class-aware adaptive augmentation to enhance minority class representations. While the controllable self-reinforcing optimization cycle effectively utilizes unlabeled samples with reliable pseudo-labels and iteratively constructs Bayes-optimal classifiers, enhancing discriminative repre- sentations for minority classes remains crucial in long-tailed learning. To achieve this goal, we enhance the updated labeled dataset by synthesizing minority class representations guided by class compactness. Formally, the compactness of class e is defined as:
(he, m(c)) de 2E|O.] 1/4/l [p(()]
(c)
Σ
(5)
where dh denotes the number of samples in class c, h; = g(zz) denotes the representation produced by encoder g(-). μ(c) denotes the centroid of class in the feature space, and ||-II denotes the Euclidean norm. As minority classes typically exhibit lower intra-class diversity, it is associated with higher compactness a according Eq. (5) and benefits from representation synthesis with a smaller augmentation radius to refine decision boundaries, where r = L/a. The representation synthesis for sample x; from minority class is computed as:
h2
h/ h4 +
T r(c) 55,
(6)
where & is noise vector sampled from d-dimensional standard normal distribution (with d matching the representation dimension). For minority classes in the updated labeled dataset, we synthesize ten augmented representations per original sample by Eq. (6).
Auxiliary branch maximize data utilization. Our controllable self-reinforcing optimization cycle incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset and trains the model on the updated labeled dataset. However, during early training stages when model performance is still weak, only limited number of samples meet the reliability condition for pseudo-labes labeling (as demonstrated in Figs. and 3, and Appendix Fig, 5). To maximize data utilization and provide additional supervision signals to stabilize representation learning during the critical initial phase, we introduce an auxiliary branch to leverage all labeled and unlabeled samples. Specifically, this branch adopts consistency regularization paradigm similar to the conventional SSL methods like FixMatch [7], enforcing prediction consistency between weak and strong augmentation views via an unsupervised consistency regularization loss (Aux loss, lawx). The Aux loss loux is formulated
cfo(o(z))
fau
log Evercy efvr(s()))
where fg(((x)) denotes the logit produced by the strong augmentation view, corresponding to the pseudo-label 1 generated from the weak augmentation view.
3.3 Model Training and Prediction
Our training process consists of an initial stage of thirty epochs using only the labeled dataset, followed by an iterative stage that expands the labeled dataset with reliable lseudo-labels and continues optimization on the updated labeled dataset. Throughout both stages, the auxiliary branch is trained on all labeled and unlabeled samples. We optimize the model by combining the LA loss (lta) applied to both branches, with the Aux loss (faux) applied only to the auxiliary branch. The overall loss Loverotl is formulated as:

---

Algorithm Training process of our CPG
1: Input: Labeled and unlabeled datasets Dj and D,-
2: Output: Encoder g. primary branch (classifier) fpris and auxiliary branch (classifier) Jauz.
3: Initialize the parameters of fpris and faur raaddoly.
4: for epoch=1, do
5: for batch=1, ..... do
6: if epoch > 30 then
7:
Identify reliable pseudo-labels by Eq. (2) and Eq. (3);
8:
Calculate the class distribution of the updated labeled dataset by Eq. (4);
9:
Enhance minority class representations by Eq. (5) and Eq. (6);
10:
end if'
Calculate the LA loss by Eq. (1) for the primary branch;
12:
Calculate the LA loss by Eq. (1) and Aux loss by Eq. (7) for the auxiliary branch;
Obtain the overall loss by Eq. (8);
14:
Update network parameters via gradient descent;
end for
16:
end for
loverall Eia + whaurs
(8)
where w is binary indicator function that equals 1 when Loverati is applied to the auxiliary branch and 0 otherwise. The pseudo-code is detailed Algorithm 1. After training, we can obtain the predicted label of an unseen sample x² by getting the label index with the highest confidence.
3.4 Theoretical Analysis
We prove that our controllable self-reinforcing optimization cycle (including progressively incor- porating reliable pscudo-labels into the labeled dataset and training a Bayes-optimal classifier on the updated labeled dataset with known distribution) can significantly reduce the generalization eπror under conditions. Let Lours N+M, E (x)(f(x:) y.) + Σm (ia(f(z)) 9,) be
the training loss on the N labeled and M, pscudo-labeled samples with sscudo-labeling error rate et for our method at the t-th training step. We begin by defining the model empirical risk R at the t-th training step as:
R₂ R2_1 de Tes + To..
(9)
where RG denotes the model empirical risk at the t-th training step, A 2 0 quantifies the empirical risk reduction achieved, and Le, and lo, represent the impact of noisy pseudo-labels and number of training samples, respectively. Let the function space H, for the label y E {1...., C} be {h: > fu(x) F), where (z) denotes the predicted probability of the y-th class. Let Ro, be the expected Rademacher complexity [38] of Hy with O₂ = N + Mr training samples. Then we have the following theorem.
Theorem 1 (Generalization Error). Suppose that the loss function (fa(f() y) is p-Lipschitz with respect to f(x) for all y E (l,...,C) C. and upper-bounded by U. Given the pseudo-labeling error rate 0 < E < 1, for any U >> O, with probability at least 1 we have:
T
T ΣΣ E1
log 2
R1 S Ro
4AV22
(H)
20 Σ
(10)
20,
where T denotes the total number of training steps, Ry and Rg represent the model empirical risk at the final training step and initial training step (without pseudo-labels), respectively, and represents the pseudo-labeling error rate at the t-th training step.
The proof of Theorem 1 is provided in Appendix B. It can be observed that the model empirical risk Ht depends primarily on three key factors, i.c., the cumulative model empirical risk reduction
Σi= >t, the cumulative pseudo-labeling error rate Σ tt, and the number of training samples Or at each training step. When the training step increases, the number of training samples O, grows while the pscudo-labeling error rate e either decreases remains constant, the Bayes-optimal

---

classifier constructed at the t-th training step can maximize the model empirical risk reduction A i.e.,
as T + oc, Or oc, Ei= Et remains bounded, and E X reaches its theoretical maximum, Theorem demonstrates that the generalization error is minimized. As shown in Figs. 3(a) and 3(b), our CPG framework demonstrates superior performance over baseline methods, achieving both: (i) consistently lower "seudo-labeling error rate <t. and (ii) a growing number of training samples O that gradually match baseline levels during training. Together with the Bayes-optimal classifier, our CPG can significantly reduce the generalization error.
4 Experiments
4.1
Experimental setting
Dataset. We perform our experiments on four widely-used datasets (CIFAR-IO-LT [39], CIFAR- 100-LT [39], 1O1-L [40], and SVHN-LT [41]), following the main experimental settings in FreeMatch [8] and SimPro [13]. More details about those datasets are provided Appendix C, and implementation details are provided in Appendix D.
Baselines. We compare with three SSL algorithms, including FixMatch [7]. FreeMatch [8], and SoftMatch [9], and three ReaLTSSL algorithms, including ACR [14], SimPro [13], and CDMAD [42]. Moreover, we use the supervised learning (SL) setting as a performance upper-bound reference. For a fair comparison, we test these baselines and our CPG on the widely-used codebase USB For the data augmentation strategy, an identical weak augmentation is applied to both labeled and unlabeled data, while reserving a strong augmentation for unlabeled data. We use the same dataset splits with no overlap between labeled and unlabeled training data for all datasets.
Table 1: Comparison of accuracy (%) CIFAR-IO-EE (Nmax 400, Miaz 4600) with class imbalance ratio y E (100, 150, 200) under different unlabeled data distributions. CE and LA denote using softmax cross-entropy loss and logit-adjusted softmax cross-entropy loss under the supervised learning (SL) setting, respectively. t indicates we reproduce ACR without anchor distributions for a fair comparison.
Scctusio Mehhod
150
Aug:
Adbicary
Comistont
Arhinnes
Consisten:
SL
CE L16]
57:00
82.90 12.00
83 10.50
15 20.14
79.14 20.31
81.06 12:4
85.01 $0.56
92.48 0030
$5,99
=0:09
74.51 1b
53.55-0055
34.55 60.51
70.30 11.21
82 -0.38
$1.43 11133
02.50 20
72.
79 iibss
65.22 ::45
FishMach [?]
11:81
57.81 11:71
68.3 11:M1
13.3
56.51
11:31
$4.16 1411
$1.41 15:3
59
SSL
FreetMasch
60:41=333
68.91 #1.78
=0,67
56.09
=2.77
62.35 m0
64.31
04.10:33.11
ACR| (14)
14:33
63.54 1:400
11.001.00
68.32 1800
51.08 1003
10.11-1039
90.131:740
53.40-1.44
65.12 01.00
59 -1-51
ROLLSSS
SimPio [13]
CDMAAD (42]
59.30.1131
65:81:145
62,70:1150
60.34 1315
64.1 11.57
=2.3
61.1
11:17
61.27
61.44
=072
1:11
62.32:241
65.29 1899
$9.11:344
56.90
12:5
58.65 11:90
59.99
59.31 23
50.36
1185
61.68 =3.52
60.90 11:4
Ouss
20034
83.37
76.38
3337
75.19
70.75
12.33
75.18
13255
21:44
65.33
76.41
Table 2: Comparison of accuracy (%) on CIFAR-100-LL (N = 50, Mmar = 450) with class
imbalance ratio y E (10, 15, 20} under different unlabeled data distributions.
Scemeio
Mahool
10
15
Aeg
Artitrary
Inverse
Consisten
Artimrry
Inverse
Coasiteer
Artimary
Inverse
Consiteett
65.12
20.33.0.33
66.68 1..06.00.1
66.50
=0025
62.73.00.3
63.44-2003
FiaMaadh [7]
48.29
43.92.21.6
20.04
42.91221544
11.0
44.60:21446
40.64 21.64.1..00
38.77 20.50
42.11 20.57
2309)
SSI
PhoeMMorchh [5]
4591-100 -1.57
45.94 15
46 56 10-43
40.66 10.00
40 -0-00
18.05 10:17
19:44 10.33
90.11 -0.00
41 97 -b.0
SofMaach []]
47.99:0085
45.17
48.35-0034
42.52:1.34
41.42 :0:45
43.78.21454
40.24
40.17
43.13 20.35
43.82 =0090
REaL
(TMMA (42)
Sinfho [13]
53.95 37:51
35. 02.00.23 31
15.15 50:95
$1.16 11.55
37.00
70 60
41.32
17.63
28,
15.25
34.35
25.1
24.17 61
11.50 15 -3.79
39.91
Oui
$1.48
20.32
$1.40
5.022
51.23 8077
86.36 8: 40
47.088 -0.34
05.04
11.00
48.17
5:130
48.17
1:.00
43.66 30.56
47.25 20075
4.2
Results
Consistent distribution settings. We begin our analysis by evaluating performance under the consistent distribution setting, where labeled and unlabeled data follow an identical long-tailed distribution. Tables 1 and 2 present the primary results for CIFAR- respectively. Across all imbalance ratios, CPG achieves superior classification accuracy compared to previous baselines. For example, given (Nmaz, M Mmazz 7) = (400, 4600, 100), CPG outperforms all baselines by up to 3.73 percentage points (pp) for Food- 101-LL and SVHN-LT in Table 3. On Food-101-L3 with (Nmazz Mmax, 7) = (50, 450, 15), CPG achieves gains of up 2.27 pp.
and CIFAR-100-LT. 100-LT,
1EFAR-10-LT. Similar improvements are observed
hhttps://gi http:p///ggtaa...co//mcrooottttt wwweem--uueiiid-eeeeeeeeee learning
8

---

Table 3: Comparison of accuracy (%) on Food-101-LT (N max = 50, Maa 450) with class
imbalance ratio E f10, 15) and SVHN-LT (Nmaz = 400, Mmax 4600, 7 = 100) under different unlabeled data distributions.
ST
Sectadi
Melao
Avg.
Arz.
Aulidesry
Iarasc
Conaalon
Consittet
Auhtuury
Ienene
Cresolon
41.31 LINTS
20-41
42.02 15:35
16
1.14
44.29
V1.38 08.00
V2
23
V1
94.37:00
FasMasda [1]
2229 2005
21.08 20.90
23.37 20:40
18 44 2300
17.47 20.40
19.77 23036
2042 20.56
$0.57 22.34
$7.29 83
03.3 13.11::551 20:34
00.000 200G
FrecMMach
21112
29.86
18722-9389
2814
2177
55.78 02.97
57.01 36
22.70
12
19.43
MCR [14]
18:1:4975 20725
19:37 12
08 54
17.07 2861
10.25
16.90 2772
$7.67 =0:00
54 56
02 68 08:50
(DMAAD
11.76=1-59
10:11
18:18
11
83.77
Ours
25:95 201.0
25.52
20.45
25.34 20:30
21.88
21.27
2001
22.01
2607
33.67
20035
53.99
20134
94.14
93.45 05:33
94.0% 3003
Inconsistent distribution settings. To assess the real-world scenarios with mismatched distributions, we evaluate CPG under inverse long-tailed and arbitrary unlabeled data distributions. As shown in Tables 1 and 2, CPG consistently surpasses baselines on both CIFAR- 10-LT and CIFAR- 100- LT. Notably, for (Nmaz: Mmaz; 7) = (400, 4600, 100) with arbitrary unlabeled data distribution, CPG achieves performance gains of 15.97 pp over existing methods on CIFAR-10-LIT For realistic benchmark food-101-LTT Table 3, CPG outperforms all competitors by up to 3.22 pp given the setting (NMaa Mmaxx 7) (50, 450, 10) with inverse long-tailed unlabeled data distribution.
When evaluating the overall performance, our CPG framework consistently outperforms baselines across all datasets. Notably, it achieves an average accuracy improvement of 11.14 pp on CIFAR- 10- LT, 3.09 pp on CIFAR- 100-LT, and 4.06 pp on SVHN-LT. Moreover, it maintains a significant gain of 2.57 pp on the challenging realistic benchmark Food- 101-LT.
Note that we observe less significant performance improvement on CIFAR-100-LL than on CIFAR- 10-LT. This can be attributed to the inherent challenges of C0FAR-100. Although CIFAR-100 and CIFAR-10 have the same total training set size, CIFAR-100 divides the data into 100 fine-grained classes (compared to 10 in CIFAR-10), 10), reducing per-class samples by an order of magnitude. The finer class granularity also increases inter-class similarity, making classification inherently more challenging. As a result, the upper-bound performance (c.g., supervised learning) on CIFAR-100-LT (64.62 pp) significantly lower than on CIFAR-10-LE (85.22 pp). Moreover, our method leverages consistency regularization to enhance model stability and prevent severe performance degradation, maintaining a relatively stable nower-bounde performance. The narrower gap between this lower bound and the constrained upper bound on CIFAR- 100-LT inherently limits the absolute performance gains. Despite this, our method still achieves substantial average accuracy improvement of 3.09 pp on 100-T,, as detailed in Table 2.
4.3 Analysis
Evaluation under varying imbalance ratios. Fig. 4 illustrrtes the performance of CPG across
different unlabeled data imbalance ratios, where the labeled data distribution is long-tailed with fixed imbalance ratio and the unlabeled data dis- tribution is arbitrary with varying imbalance ra- tios. The datasets are CIFAR- 10-LT with (n 100, Yo E {1000, 150, 200, 250}) and CIFAR- 100-LT with (7x 10, To (5, 10, 15, 20)). The results consistently indicate that our method outperforms previous baselines. Moreover, as the unlabeled data imbalance ratio increases, the performance of baseline methods degrades sig- nificantly. In contrast, our method maintains nearly consistent performance on (1FAR-10-L11 and exhibits only minor degradation CIFAR-100- LT, demonstrating robust performance across varying unlabeled data imbalance ratios.
Ireebrrc Ratto Unikeed Cats
Iresarc Ratto Urideed Cats
(a) CIFAR-IO-LT
(b) -IFAR-100-LT
Figure 4: Comparison of accuracy (%) among FreeMatch [8], SimPro [13], and our CPG on CIFAR-IO-LT and 1IFAR-100-LL under arbitrary unlabeled data distribution with long-tailed labeled data distribution.
Evaluation under arbitrary labeled data distribution. Real-world scenarios often exhibit dynamic distribution shifts where any class can become dominant, affecting both labeled and unlabeled data. While existing evaluations assume static long-tailes distributions for labeled data, assess

---

CPG under arbitrary labeled data distribu- tions, with unlabeled data following either a consistent or inconsistent distribution relative to labeled data. As shown Ta- ble 4, CPG consistently outperforms base- lines on CIFAR-IO- IO-LT and CIFAR-100- LT, demonstrating exceptional robustness. For instance, with (Nmoay Mmaz: 7) = (400, 4600, 100) and an inconsistent un- labeled data distribution, CPG achieves performance gains of 11.82 pp over exist- ing methods on CIFAR-IO-LTT.
Table 4: Comparison of accuracy (%) CIFAR-1O-LT and CIFAR- 1----10-LTT- under arbitrary labeled and unla- beled data distributions.
CIFAR-IDEET
TEAR-IO0-LT
Seenario Mechod
NN Sangss-deeee Aayae 7) (400, 4600, 100)
Neeaa Maser 30 - (50, 430, 10
Incerstinne
Consinere
Consisten
SL
CB
[A6
87 07.22::4..
82.90333334
85.11 47.4
78.48 1461
64.75 -8.58
66.54:0030
06.75-0.37
-0:77-0331
FiuaMachh 17]
66.27
69.69
:8.50
FreetMauch 62,02
69.73
45.59 20.31
SotMMacch [9]
63.57:a1.68
73007 #2.79
47.79
48.83
ACR] [14)
56.15-5500
71.71 2004
45.30 20.56
42.34
ReLLTSSI
CDMAD
SimPPoo
70.40
60.25
67.54
66.45
44.30 20.45
35,79
Ours K2.22 1:404
76.41 14.33
$1.39 1100
50.39 200K
Ablation study. We conduct ablation studies to validate the effectiveness of key components in our CPG in Table 5. Specifically, we set (Nmaz: Mmar: ?) (400, 4600, 100) for CIFAR- 10-LT and (Nmaa, Mmaz: 7) = (50, 450, 10) for CIFAR-100-LT, respectively. As shown in Ta- ble 5, we can observe that each component brings a significant improvement. For exam- ple, on CIFAR-IO-LT, the auxiliary branch (AB) brings performance gain of 0.80 pp given the arbitrary unlabeled data distribution. Moreover, class-aware adaptive augmentation (CAA) module further boosts performance by 2.99 pp over the AB baseline, while controllable self- reinforcing optimization cycle (CSOC) delivers a more substantial gain of 14.41 pp. Combining CAA and CSOC achieves synergistic improvement of 14.82 pp. underscoring the complementary roles of these components. Table 5: Comparison of accuracy (%) on with and without the key components in the proposed method.
When evaluating the overall per- formance, CAA and CSOC in- dividually contribute average performance gains of 2.35 pp and 6.97 pp, respectively, while their combination yields a 10.65 Pp improvement, reinforcing the effectiveness and robustness of our design.
Ablations
CIFAR-IO-LT
CIFAR-I00-LT
Avg.
CSOC CAA Arbitrary Inverse Consistent Arbitaary Inverse Consistent
65.18
63.94
66. 10
46.32
46.23
45.84
55.60
65.98
64.93
69.28
48.37
47.85
48.13
57.42 f LA2
/
68,97
67.59
75.52
48.90
48.26
49.39
59.77 2
80.85
76.85
49.35
aar
82.33
82.32
78.35
51.85
51.61
51.04
66.25
1ac.s
Statistical significance. Table 6 presents the win/tie/loss counts between our CPG and six base- line methods across different datasets under varying unlabeled data distributions, using a pairwise t-test a 0.05 significance level. The results show that our CPG outperforms baseline methods in all cases, with a significant improvement in 80.25 pp of cases (130 out of 162), demonstrating its effectiveness.
Table 6: Statistical significance of perfor- mance differences assessed with pairwise t- test at 0.05 significance level, reported as win/tie/loss counts.
Scenario
Metbod
Artitrary
Invens
Consistent
FatMaach [7]
7:2/0
9/0/0
4/5/0
20/7/0
SSL
FreetMach [8]
9/0/0
9/0/0
1/2/0
25/2/0
SonMarch [9]
7/2/B
8/1/0
5/4/0
20/7/0
ACRT [14]
8/1/0
8/1/0
5/4/0
21/6/B
ReaLTSSL
SimPro
6/3/0
9/0/0
7/2/0
22/510
More experiment results and ablation studies are pro- vided in Appendix E and Appendix F, respectively.
CDMAD (42)
8/1/0
9/0/0
5/4/0
22/5/0
Total
45/9/0
52/2/0
33/21/0
1307 32//32/0
5 Conclusion
In this work, have presented novel solution the ReaLTSSL problem via controllable pseudo- label generation. We proposed to expand the labeled dataset with reliable pseudo-labels and train the model on the updated labeled dataset with known distribution, making unaffected by the unlabeled data distribution. Our framework introduces a controllable self-reinforcing optimization cycle, combining dynamic controllable filtering for reliable pseudo-label identifying with iterative Bayes-optimal classifier construction through logit adjustment. It is further enhanced by a class- aware adaptive augmentation module for minority class representation learning and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples. We further theoretically proved that this optimization cycle can significantly reduce the generalization error under some conditions. Comprehensive experiments on four commonly used benchmarks show consistent improvements of our method over the sfate-of-the thette-rrtt art methods by up to 15.97 % across various scenarios.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China under Grants U24A20322, 62576094 and 62422118. This work is also supported by Hong Kong UGC under grants UGC/FDS11//03/22 UGC/FDS1 1/E03/25, and Hong Kong Research Grants Council under Grant 11219324. This research work is also supported by the Big Data Computing Center of Southeast University.
References
[1] Aibing Li, Xinyi Zeng, Pinxian Zeng, Sixian Ding, Peng Wang, Chengdi Wang, and Yan Wang. Textmatch: Using text prompts to improve semi-supervised medical image segmentation. In Medical Image Computing and Computer Assisted Intervention (MICCAI), volume 15008, pages 699-709, 2024.
[2] Lei Li, Sheng Lian, Zhiming Luo, Beizhan Wang, and Shaozi Li. Velipseg: Voxel-wise clip- enhanced model for semi- cumi-viiee medical image segmentation. In Medical Image Computing and Computer Assisted Intervention (MICCAI), volume 15009, pages 692-701, 2024.
[3] Yaxiong Chen, Yujie Wang, Zixuan Zheng, Jingliang Hu, Yilei Shi, Shengwu Xiong, Xiao Xiang Zhu, and Lichao Mou. Striving for simplicity: Simple yet effective prior-aware pseudo- labeling for sume-supe ultrasound image segmentation. In Medical Image Computing and Computer Assisted Intervention (MICCAI), volume 15009, pages 604-614, 2024.
[4] Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In International Conference on Machine Learning (ICMI.,, 2013.
[5] Dongdong Chen, Wei Wang, Wei Gao, and Zhi-Hua Zhou. Tri-net for semi-supervised deep learning. In Proceedings of the International Joint Conference Artificial Intelligence (IJCAI), pages 2014-2020, 2018
[6] Paola Cascante- Boniila,, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Curriculum labeling: Revisiting pseudo-labeling labeling for semi-supervised learning. In Proceedings the AAAI Conference on Artificial Intelligence (AAAI), volume 35, pages 6912 6920, 2021.
[7] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi- supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 596-608, 2020.
[8] Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj. Bernt Schiele, and Xing Xie. Freematch: Self- adaptive thresholding for semi-supervised learning. International Conference on Learning Representations (ICLR), 2023.
[9] Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-qualing tradeoff in semi- supervised learning. In International Conference on Learning Representations (ICLR), 2023.
[10] Hyuck Lee, Seungjae Shin, and Heeyoung Kim. ABC: auxiliary balanced classifier for class- imbalanced semi-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 7082-7094, 2021.
[11] Yue Fan, Dengxin Dai, Anna Kukleva, and Bernt Schiele. Cossl: Co-learning of representation and classifier for imbalanced semi-supervised learning. In IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pages 14574-14584, 2022.
[12] Qianhan Feng, Lujing Xie, Shijie Fang, and Tong Lin. Bacon: Boosting imbalanced semi- supervised learning via balanced feature-level contrastive learning. In Proceedings of the AAAI Conference Artificial Intelligence (AAAI), volume 38, pages 11970-11978, 2024.
11

---

[13] Chaoqun Du, Yizeng Han, and Gao Huang. Simpro: A simple probabilistic framework towards realistic long-tailed semi supervisdd learning. In International Conference on Machine Learning (ICML), pages 1666--11 1703, 2024.
[14] Tong Wei and Kai Gan. Towards realistic long-tailed semi-supervised learning: Consistency is all you need. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 3469-3478, 2023.
[15] Chengcheng Ma, Ismail Elezi, Jiankang Deng, Weiming Dong, and Changsheng Xu. Three heads are better than one: Complementary experts for long-tailed semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 38, pages 14229-14237, 2024.
[16] Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In International Conference Learning Representations (ICLR), 2021.
[17] Seulki Park, Youngkyu Hong, Byeongho Heo, Sangdoo Yun, and Jin Young Choi. The majority can help the minority: Context-rich minority oversampling for long-tailed classification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6887-6896. 2022.
[18] Jiang-Xis Xin Shi, Tong Wei, Yuke Xiang, and Yufeng Li. How re-sampling helps for long-tail learning? In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 75669-75687, 2023.
[19] Qihao Zhao, Yalun Dai, Hao Li, Wei Hu, Fan Zhang, and Jun Liu. LTGC: long-tail recognition via leveraging lims-driven generated content. In /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19510-19520, 2024.
[20] Yina He, Lei Peng, Yongcun Zhang, Juanjuan Weng, Shaozi Li, and Zhiming Luo. Long- tailed out-of-distribution detection: Prioritizing attention to tail. In Proceedings of the AAAI Conference Artificial Intelligence (AAAI), volume 39, pages 3446-3454, 2025.
[21] Yuchao Wang, Jingjing Fei, Haochen Wang. Wei Li, Tianpeng Bao, Liwei Wu, Rui Zhao, and Yujun Shen. Balancing logit variation for long-tailed semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19561-19573, 2023.
[22] Jiangpeng He. Gradient reweighting: Towards imbalanced class-incremental learning. In IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16668- 16677, 2024.
[23] Yanbiao Ma, Wei Dai, and Jiayi Chen. Pursuing better decision boundaries for long-taile tailed object detection via category information amount. In International Conference on Learning Representations (ICLR), 2025.
[24] Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X. Yu. Long-tailed recogni- tion by routing diverse distribution experts. In International Conference on Learning Representations (ICLR), 2021.
[25] Emanuel Sanchez Aimar, Arvi Jonnarth, Michael Felsberg, and Marco Kuhlmann. Balanced product of calibrated experts for long-tailed recognition. In IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), pages 19967-19977, 2023.
[26] Qihao Zhao, Chen Jiang, Wei Hu, Fan Zhang, and Jun Liu. MDCS: more diverse experts with consistency self-distillation for long-tailed recognition. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 11597-1 1608,, 2023.
[27] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1199))1979-1993, 2019.
12

---

[28] Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy S. J. Ren, and Rynson W. H. Lau. Dual student: Breaking the limits of the teacher in sumi-supeeeiee learning. IEEE/CVF International Conference Computer Vision (ICCV), pages 0728-6736, 2019.
[29] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervine learning with curriculum pseudo labeling. In Advances in Neural Information Processing Systems (NewrIPS), volume 34, pages 18408-18419, 2021.
[30] Siyuan Li, Weiyang Jin, Zedong Wang, Fang Wu, Zicheng Liu, Cheng Tan, and Stan Z. Li. Semireward: general reward model for semi-supervised learning. In International Conference on Learning Representations (ICLR), 2024.
[31] Chongsheng Zhang, Yaxin Hou, Ke Chen, Shuang Cao, Gaojuan Fan, and Ji Liu. Quality-aware self-training on differentiable synthesis of rare relational data. In Proceedings of the AAAI Conference Artificial Intelligence (AAAI), volume 37, pages 6602-6611, 1, 2023.
[32] Zhixin Li and Yuheng Jia. Commix: Contrastive mixup at representation level for long-tailed deep clustering. In International Conference on Learning Representations (ICLR), 2025.
[33] Yuheng Jia, Xiaorui Peng, Ran Wang, and Min-Ling Zhang. Long-tailed partial label learning by head classifier and tail classifier cooperation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 38, pages 12857-12865, 2024.
[34] Yuhang Li, Zhuying Li, and Yuheng Jia. Boosting class representation via semantically related instances for robust long-tailed learning with noisy labels. In IEEE/CVE International Conference on Computer Vision (ICCV), pages 1516-1525, 2025.
[35] Yaxin Hou, Weiping Ding, and Chongsheng Zhang. imftp: Deep imbalance learning via fuzzy transition and prototypical learning. Information Sciences, 679:121071, 2024.
[36] Chen Wei, Kihyuk Sohn, Clayton Mellina, Alan L. Yuille, and Fan Yang. Crest: A class- rebalancing self-training framework for imbalanced semi-supervised learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10857-10866, 2021.
[37] Yaxin Hou and Yuheng Jia. A square peg in a square hole: Meta-expert for long-tailed semi- supervised learning. In International Conference on Machine Learning (ICML), 2025.
[38] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning. MIT Press, 2018
[39] Alex Krizhevsky. Learning multiple layers of features from tiny images. https://www.csstoromto edu/ -eataamrenn--auurss--22 wwweeeess000-RRPP.P pf. 2009.
[40] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 101 - mining discriminative components with random forests. In European Conference on Computer Vision (ECCV), volume 8694, pages 446-461, 2014.
[41] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In Advances in Neural Information Processing Systems (NeurIPS), 2011.
[42] Hyuck Lee and Heeyoung Kim. CDMAD: blas-distribttio--nismaac-----ed class-imbalanced semi-supervised learning. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23891-2333 123900, 2024.
debiasing for
[43] Andreas Maurer. A vector-contraction inequality for rademacher complexities. In Algorithmic Learning Theory (ALT), volume 9925, pages 3-17, 2016.
[44] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference (BMVC), 2016.
[45] lya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR), 2017.
13

---

[46] Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov. and AbdeIrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451-3460, 2021.
[47] Karol J. Piczak. ESC: dataset for environmental sound classification. Proceedings of the ACM Conference Multimedia Conference, pages 1015-1018, 2015.
[48] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S. Rawat, and Mubarak Shah. In defense of pseudo-labeling: An uncertainty-awarc pseudo-label selection framework for semi-supervised learning. In International Conference on Learning Representations (ICLR), 2021.
[49] Xinting Hu, Yulei Niu, Chunyan Miao, Xian-Sheng Hua, and Hanwang Zhang. On non- random missing labels in comi-supervised learning. In International Conference on Learning Representations (ICLR), 2022.
[50] Emanuel Sanchez. Aimar, Nathaniel Helgesen, Yonghao Xu, Marco Kuhlmann, and Michael Felsberg. Flexible distribution alignment: Towards long-tailed semi-supervised learning with proper calibration. In European Conference on Computer Vision (ECCV), volume 151 pages 307-327, 2024.
[51] Youngtaek Oh, Dong-Jin Kim, and In So Kweon. DASO: distribution-aware semantics-oriented pseudo-label for imbalanced semi uuni-supervised learning. In /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
2022.
[52] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning (ICML), pages 1050-1059, 2016.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. Please see Abstract and Sec. 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of the work performed by the authors. Please see Appendix H.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: For each theoretical result, the paper provides the full set of assumptions and a complete (and correct) proof. Please see Sec. 3.4 and Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversell, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The paper fully discloses all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not). Please see Sec, 4.1, Appendix C, and Appendix D.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The paper provides open access the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
Serriide/CoodeSumiissste
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nipp. htpppuubbuuddeeeeeeeepppppbppeee h
/CodeSubmic wwwbbbisssiooo sesiiisssiniiiiieeeeeioo oniiiniyl) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The paper specifies all the training and test details (e.g., data splits, hyperpa- rameters, how they were chosen, type of optimizer, etc.) necessary understand the results. Please see Sec. 4.1, Appendix C, and Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments. Please see Sec, 4.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
The paper provides sufficient information on the computer resources (type of
compute workers, memory, time of execution) needed reproduce the experiments. Please see Appendix I.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https :/ /wwrreeurppp..e/ httpppplllttiiiteiiiiieee
Answer: [Yes]
Justification: The research conducted in the paper conforms, every respect, with the NeurlPS Code of Ethics https: htaipp::///euriiie /ttp//nuus.../// cc/
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:[Yes
Justification: The paper discusses both potential positive societal impacts and negative socictal impacts of the work performed. Please Appendix J.
18

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our research does not release data models, so the paper has such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual- use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The creators or original owners of assets (c.g., code, data, models), used the paper, are properly credited and are the license and terms of use explicitly mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
19

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //neruurpp.... wcc//ooneeeeccc//22///1/111 for what should or should not be described.
21

---

Table of Contents for the Appendix
A: Evolution of Pseudo-label Predictions
23
B: Proof Theorem
23
C: Dataset Details
26
D: Implementation Details
26
E: More Experiment Results and Analyses
27
E.1: Comparison of Macro-F1 on CIFAR-IO-LT and CIFAR-100-LT
27
E.2: Evaluation on Audio Modality
27
E.3: Evaluation under Noisy Scenarios
27
E.4: Evaluation on Large-scale and Realistic Datasets
28
E.5: Evaluation under Extreme Settings
28
E.6: Evaluation using Other Architectures
28
More Ablation Studies and Analyses
28
F11: Ablation Study on the Confidence Threshold
28
R2: Ablation Study the Auxiliary Branch
29
F3: Ablation Study on the Anchor Distributions
30
G: Relation to Existing Methods
30
H: Limitations
I: Compute Resources
31
J: Broader Impacts
.31
22

---

A Evolution of Pseudo-label Predictions
Fig. 5 shows the evolution of pseudo-label predictions of method under the arbitrary unlaheled data distribution, evaluated on CIFAR-1O-LI. From Fig. 5, we observe that our CPG progressively increases the utilization rate of pseudo-labels while maintaining the high accuracy. The predicted distribution gradually approximates the ground-truth unlabeled data distribution, and the Kullback- Leibler (KL) divergence between the predicted and ground-truth unlabeled data distributions decreases during training.
5000
3000
5000
KL: 0.33
Nimper Simpres
KL: 0L:0.28
Numbor of zumooiss
4000
KL::0
2000
1000
1000
3000-
Cleas Iode
Class Indee
Class intes
(a)
(b)
(c)
5000
Numbat of aampkes
4000
KL:0.33
Rumber sermsiis
4000
4:000
3000
3000
3000
2000
2000
2000
1000
1000
1800
(d)
(c)
(f)
Figure 5: Evolution of pseudo-label predictions of our method under arbitrary unlabeled data distribution. GT denotes the ground-truth unlabeled data distribution. TP (FP) denotes the pre- dicted true (false) positive pseudo-labels. KL denotes the Kullback-Leible divergence between the predicted and ground-truth unlabeled data distributions. The dataset is CIFAR-1O-L] with (Nmaz MM Tu) = (400, 4600, 50, 50). Our CPG can progressively increase the utilization rate of pseudo-labels while maintaining the high accuracy during training, the pscudo-labo distri- bution gradually approximates the ground-trun truth unlabeled data distribution.
B Proof of Theorem 1
We first copy the Theorem here.
Theorem 1. Suppose that the loss function Eio ()) is p-Lipschitz with respect to f(x) for all
y E (1....,C C) and upper-bounded by U. Given the pseudo-labeling error rate < c < 1, for any
> 0, with probability at least 1 U. we have:
I
T
T
C
T
2 log
RI < Ro
E
2p
ΣV (Hg) + 2U 1=
v
(11)
20,
1=1
where T denotes the total number of training steps, Ri and Ro represent the model empirical risk at the final training step and initial training step (without pseudo-labels) respectively, and e represents the leeullg error rate the 1-th training step.
Proof. We first restate the definition of the model empirical risk R at the t-th training step, i.e.,
R₂ R,-1 X Ie, To.:
(12)
where R₂ denotes the model empirical risk at the 1-th training step, A, 2 0 quantifies the empirical risk reduction achieved, and Tr, and To, represent the impact of noisy pseudo-labels and number of training samples, respectively. We derive Le, and lo, below, omitting the subscript t for brevity.
23

---

We define the true risk on the training dataset with respect to the classification model f(z; 0) as:
R(f) = E(z,)) [Lta (x)), v)].
(13)
We aim to learn good classification model by minimizing the empirical risk R(f) = Rf) + M
(f), with (f) = v Ei= lio (f(z), 34) and (f) M Σز= Lia (f(zz))))) representing the empirical risk labeled and unlabeled training dataset, respectively.
Next, we derive the uniform deviation bound Lo between the true risk R(f) and the empirical risk R(f) using the following lemma.
Lemma Suppose that the loss function lia (I(x), 4) is p-Lipschitz with respect to [(x for all y E {I...., C} and upper-bounded by U. For any >> O, with probability at least 1 - U, we have:
log 2
[R(f) - R(fII S 2v2p Σ RN RN- MM (H2) + +U
(14)
2(N M)
where the function space for the label y E {I...., C} is = EF}.
Proof. In order to prove this lemma, we define the Rademacher complexity of the composition of loss function La and model f E F with N labeled and M unlabeled training samples follows:
R RN+M o F)
M
Σ Mi (laa (la((((z)))i)) FEF
Σ us (io (fo (f(zz)) ==1
sup
Sv2p Σ RN+Ms(Hy)
(15)
where o denotes the function composition operator, E(ayaa) denotes the expectation over x, y. and J., μl denotes the Rademacher variable, sup denotes the supremum (or least upper bound) over the function set F of model f. The second line holds because of the Rademacher vector contraction inequality [43].
Then, proceed with the proof by showing that the direction suP R(S) R(F) is bounded with probability at least 1-v2 and the other direction sup - - can be proved similarly, &
Note that replacing a sample (x,, 34) leads to a change of sup R(f)) R(()) at most N+M due
to the fact that &a is bounded by U. According to the McDiarmid's inequality [38], for any v > 0, with probability at least v/2, we have:
log 2
sup R(f) R( E
sup R(f) - R(() + +U
(16)
feF
JEF
2(N + M)
supjer R(f) - R(}} < 2RN+M(F). and further
According the result in [38] that shows EE
considering the other direction sup R(f)) R(f), with probability at least 1 - u. we have:
C
log 2
IR() R(() < 2/2p Σ fEr
sup
RN4M + UV
(17)
2(N + M)
which completes the proof.
In SSL, we cannot minimize the empirical risk Ra( directly, since the ground-truth labels are inaccessible for unlabeled training data. Therefore, we need to train the model with (f)
24

---

M شبh (a(f(zz))05)) where ūJ denotes the pseudo- of unlaheled sample z; and M quan- tifies the pseudo-labels identified by the dynamic controllable filtering. Let f = argmin fer R(f) be the empirical risk minimizer, and f argmin R(f) be the true risk minimizer. Let lours N+M E (f(() ys) + Σソ=1 (xa()(2)) 83) be the training loss on N labeled and M yseudo-labeles samples with pseudo- pseudo-labeling error rate for method. Then, can bound the difference Le, between Rf( and (f) follows.
Lemma Suppose that the loss function lio (f(x), y) is p-Lipschitz with respect to f(x) for all y E (l,...,O) C}. and upper-bounded by U. Given the pseudo-labeling error rute 0 < < 1, we have:
0 S RR(f) - Rf)l S Ue.
(18)
Proof. Let's first expand (f):
1
Σ
la (z)))
M
j=1
M
c
= Σ
(1 (1-6
6
Σ
Tūū + ys U1)
(19)
M
C-1
Next, we show its upper bound:
M
1
R
Σ
Iūū + yj) v,)
M j=l
C
c=l
M
SR, +
1
Σ
Tūū * uU)U
M
C
<R,(f)
(20)
and when 0, it reaches the lower bound (J),,, which concludes the proof.
Based on the above lemmas, for any > O, with probability at least 1 u, at the 1-th training step, we have:
log c 20,
s R,(() + 22
Ro, (Hl) +U
y=1
C
log
< RR,(f)) + Re, (f) 2222p
Ro, (H,,) UU
20,
y=1
C Ro, (H,y) + UV
log 20,
< RR,(()) + R2 (f) +222p
y=1
C
log
$ R., + (f)
Ro, (Hly + +U Σ
20
y=1
log 2
$ RR, + Re. (f) + Ue, + +2v2p Σ Ro. (Hyy) + UV
20,
y=1
log 2
$ RR,()) + UEi 2222p Σ
Ro. (Hl) + +U
20,
25

---

log
$ R((f)+ Ue + 4v Σ Ro. (H4) + 2U
(21)
20
where the first and seventh lines are based on Lemma 1, and three and fifth lines are due to Lemma 2. The fourth line is by the definition of f.
Then, can derive Le + lo, follows:
Ie, + To, = R2( R((sI++1 (R(() Rc( (R,()) (
log
S Uet + 2v2p Σ Ro, (Hyy ( +20
(22)
20,
Substituting Eq. (22) into Eq. (12) yields:
log
R4 S R d + Uc + 4v2p Σ
Ro, +
20
y=1
log 2 20,-1
Re-1 R 2-- + Uee11 4v2p Σ
Ro, (HH) 2U
(23)
y
log
R1 S Ro x + UG + 4v2p
Σ
Ro, (Hyy + 2
201
y=1
Finally, combining the sub- eequaiiuttnn in Eq. (23) through summation yields:
T
log 2
Rr S Ro
Σ
+ 4v2p
Ro, (Hl, + 20
(24)
20,
t=1
1=1
At this point, we have proven Theorem 1.
C Dataset Details
We perform our experiments on four widely-used datasets (CIFAR-IO-LT [39], CIFAR-I00-LT [39], Food-101-LT [40], and SVHN-LT [41]), following the main experimental settings in FreeMatch [8] and SimPro [13], details are as below.
CIFAR-1O-LT: We test nine settings with (NNaaz Mmoz) (400, 4600) and {100, 150, 200). The unlabeled data follows either the same long-tailed distribution as the labeled data (consistent case), inverse long-tailed arbitrary distribution (inconsistent cases).
1MFAR-100-L11 We examine nine settings with (Nmaz: Mmarr (50, 450) and E {10, 15, 20). using the same consistend/inconiisten unlabeled data distribution patterns as above.
101d-10 We evaluate six settings with (N (Nmazz Mmax) (50, 450) and 7 E (10, 15). maintaining the same distribution cases.
SVHN-L.T: We evaluate three settings with (Nmary M Mmaz; 7) (50, 450, 100), following the same distribution scheme.
D Implementation Details
We follow the default settings and hyperparameters in USB, i.e., the batch size of labeled data B, is set to 64, while unlabeled data B,, is set to times B,, and the confidence threshold is set
26

---

to 0.95. Moreover, we use the WRN-28-2 [44] architecture, the SGD optimizer with momentum 0.9, and weight decay 5e-4 for training. We use cosine learning rate decay [45] scheme, which sets the learning rate to 7/ cos training step, and the total number of training steps ''' is set to 2 We repeat each experiment with three different random seeds (i.e., 0, I, and 2) and report the mean and standard deviation of the performances. We conduct the experiments a single NVIDIA RTX 4090 GPU using PyTorch v2.3.1.
16T ), where the initial learning rate rj is set to 0.03, I. is the current 7
E
More Experiment Results and Analyses
E.1
Comparison of Macro-F1 on CIFAR-10-LT and CIFAR-100-LT
To better evaluate our method's
Table Comparison of Macro-F1 (%) on CIFAR-IO-LT
performance under class imbal- ance, include the Macro-F1 metric in our analysis. Table 7 presents the results on CIFAR- 10-LT and CIFAR-100-LT. In both cases, the labeled data fol- lows a long-tailed distribution, while the unlabeled data varies across long- tailed, inverse long- tailed, and arbitrary distributions. The results in Table 7 show that our method achieves significant improvements in Macro-F1 over existing baselines, further validating its effectiveness for imbalance learning. Notably, it yields average Macro-Fl gains of 13.84 pp on CIFAR-10-LT and 4.55 pp on CIFAR-100-LT.
(N max 400, Maa 4600, 7 100) and 1IFAR-100-LT
(Nmaz = 50, Mmar = 450,7 = 10) under different unlabeled
data distributions.
CIFAR-IO-LT
CIFAR 100-T
Sccmario
Method
== 100
Avg.
10
Avp.
Arbitrary Invens Consistent
Artitrary Invense Consistent
FixMaich
SSL
FreetMatchh [8]
66.00
67.00
68.04
67.01
44.13
43.57
43.84
43.85
SoftMatch [9]
62.49
66.66
71.53
66.89
45.67
46.73
46.91
46.44
ACRT [14]
64.03
70.25
64.4%
45.69
48
41. 41.92
45.50
ReaL
SimPro [13]
61.22
62.11
57.81
60.38
39.98
41.54
41.87
41.13
CDMAD [42]
61.73
62.88
62.11
62.24
32.97
35.80
36.68
35.15
Our
81,99
82.31
78.25
80.85
51.31
51,25
50.41
50.99
E.2
Evaluation Audio Modality
While our method is evaluated on vision tasks, the core principles of CPG can be extended to other modalities. Here, we further evaluate our method on the audio modality. Specifically, for audio data, this adaptation requires two key modifications: (i) replacing vision-specific backbones (e.g., WRN-28-2 [44]) with audio-compatible models (e.g., HuBERT [46]), and (ii) substituting image augmentations with audio- specific ones. To demonstrate this adaptability, we perform our CPG on ESC-50-LT [47], a standard benchmark for environmental sound classification. As demonstrated in Table 8, our method achieves state- of-the-art performance on this audio benchmark, confirming its effec- tiveness beyond the visual modality. These results strongly support the generalizability of CPG's framework across different modalities.
Table 8: Comparison of ac-
curacy (%) on ESC-50-LT
(Nmaz
12, Mmaz
12.7 = 1.2) under long-
tailed labeled data and arbi- trary unlabeled data distribu- tions.
FreeMaich [8] SimPro [13] Ours
68.13
68.75
69.13
E.3 Evaluation under Noisy Scenarios
To comprehensively evaluate our method's robust- ness under noisy scenarios, we conduct additional experiments on CIFAR-IO-LT with (Nmox = 400,MMaas
Table 9: Comparison of accuracy (%) on
CIFAR-IO-LT (N = 400, Mmar
4600, 7 = 100) across different noise rate sce- narios.
4600, 7 100), under varying noise rates (i.e., 0%, 10%, 20% and 30%). The la- beled data follows long- tailed distribution, while the unlabeled data adheres to an arbitrary distri- bution. As demonstrated in Table 9, our method consistently outperforms baselines in noisy settings, achieving the highest accuracy across all noise rates with minimal performance degradation. Notably, when noise rate increases from 0% to 30%, our method shows degradation (A) of only 12.46
Noise rate
10%
20%
30%
A (0% 30%)
FreetMatch [8] 66.33 60.33
56.24
51.29
-15.04
SimPro (13) 64.73
59.86
44.54
34,80
-29.93
Ours
82.33 75.95 71.82 69.87
-12.46
27

---

Pp. compared to 15.04 pp for FreeMatch [8] and 29.93 pp for SimPro [13]. This demonstrates our method's strong robustness label noise.
E.4 Evaluation on Large-scale and Realistic Datasets
The Food-101 dataset (evaluated in Table 3) is a widely adopted bench- mark for food image classification. Its training set naturally contains proportion of label noise, making the data distribution implicitly un- known. Thus, distribution mismatches between labeled and unlabeled data occur when splitting the training set into labeled and unlabeled datasets. Despite these challenges, our method achieves significant average accuracy gain of 2.57 pp on this dataset.
Table 10: Comparison of accuracy (%) on ImageNet-
127.
Scetario
Method
32x32 64x64
FisMatch [7]
29.57
37.40
SSI.
FreetMatch [8]
31.37
39.67
31.50
ACR: [14]
38.47
47.61
Real.TSSI SimPlo [13]
43.31 46,93
To further evaluate method's effectiveness in large-scale and real- world scenarios, we conduct experiments on ImageNet-127 (an im- balanced dataset, 127 classes, imbalance ratio 286) with an arbitrary (non-long-tailed) distribution. As shown in Table 10, our method consistently outperforms baselines, achieving gains of 1.27 pp 32x32 resolution and 2.82 pp at 64x64 resolution. These results highlight our method's robustness in large-scale and realistic settings beyond standard benchmarks.
CDMAD [42]
Oars
44.58 50.43
E.5 Evaluation under Extreme Settings
In long-tailed learning scenarios, there exists an inherent trade-off between extremely small Nmaz (number of sam- ples in the most frequent class) and large imbalance ratio values, since Nmin = Nman/ / 2 1. Our initial experi- ments on CIFAR-1O-LT in Table already employed chal- lenging settings (Nmax = 400, Mmax = 4600, y 100) with Wain 4. Notably, even with such extremely few tail class samples, our method still demonstrates signifi- cant performance gains over baselines.
Table 11: Comparison of accuracy (%) on CIFAR-10-LT under extreme scenar- ios.
Method
Setting A1
Setting Setting B
FreetMatch [8]
63.86
45.75
55.92
SimPro [13]
31.01
15.20
50.98
Ours
69.56
49,28
77.78
To further evaluate our method, we conduct additional experiments under three more extreme settings: extreme sparsity setting Al (N 100, Nmi 1, MMaax 4900, 7 = 100), extreme sparsity setting A2 (Nmaz 10, Nmin 1, Mmaa = 4990, 7u = 10. 300). As demonstrated in Table 11, our method achieves tfute-of-thh-ar performance across three settings, attaining 69.56 pp in setting A1, 49.28 pp in setting A2, and 77.78 pp in setting B. These results represent significant improvements over all bascline methods, underscoring our method's exceptional capability to handle extreme settings.
300), and extreme imbalance
setting B (Nmez 400, Nmin M 4600,
E.6 Evaluation using Other Architectures
To evaluate architectural generalization, we conduct experiments using ResNet-50 CIFAR-10-LT with (Nmar 400, Maaz 4600, 7 100). The labeled data follows a long-tailed distribution, while the unlabeled data conforms either an inverse long-tailed or an arbitrary distribution. As evidenced by Table 12, our method demonstrates significant improvements, achieving absolute performance gains of 7.85 pp and 7.23 pp over current state-of-the-ar methods in these respective scenarios. These results highlight our method's consistent robustness when applied to different network architectures and its ability to handle varying distribution patterns the unlabeled data.
Table 12: Comparison of ac- curacy (%) on CIFAR-IO-
LT using ResNet-50.
Method
Arbitrary
Inverse
FreetMatch [8]
47.68
48,28
SimPro (13)
43.67
51.26
Ours
55.53
58.49
F More Ablation Studies and Analyses
F11 Ablation Study on the Confidence Threshold
The confidence threshold - 0.95 a common setting in SSL, which we adopt for fair compari- son. To evaluate the impact of different confidence thresholds model performance, conduct
28

---

additional experiments on CIFAR-IO-LT with (Nmax 400, MMn
Table 13: Ablation study on the confi-
100) and 5 (0.75, 0.85, 0.95). As shown in Table 13, our method's performance variations across different thresholds remain relatively small (within 1-2 pp), and it maintains consis- tently superior performance compared to baseline methods. This threshold-insensiiiie property highlights the stability of our method.
4600,
E
dence threshold.
Method
Arbitrary
Inverse
Consistent
FreetMatch [8]
66.33
68.10
69.42
SimPro [13]
64.73
65.25
64.37
Ours (r 0.75)
81.35
78.26
76.20
Ours (T 0.85)
81.89
80.85
77.79
Ours (T 0.95)
82.33
82.32
78.35
F.2 Ablation Study on the Auxiliary Branch
To evaluate the impact of the auxiliary branch, we integrate into both FreeMatch [8] and SimPro [13] and analyze their performance variations. As demonstrated in Figs. and 7, analysis reveals that the auxiliary branch exhibits divergent effects across methods. FreeMatcht (Free.Match with the auxiliary branch, Fig. 6(a)) selectively improves pseudo-label quality and quantity for both majority classes (e.g., classes 8 and 9) and minority classes (e.g., class 7), unlike the haseline (FreeMatch without the auxiliary branch, Fig. ((a)). In contrast, SimPro suffers consistent degradation in pseudo- label quality and quantity across nearly all classes when integrated with the auxiliary branch. These trends are further supported by the error rate and utilization rate comparisons Figs. and which indicate similar performance variations.
ofmeeroffmme
3000
Tumber
2000
2000
2000
1000
1000
1000
Class Index
Class Index
Class Indox
(a) FreeMatchi
(b) SimProž
(c) Ours
Figure 6: Comparison of pseudo-label predictions among FreeMatcht [8], SimProt [13], and our CPG under arbitrary unlabeled data distribution. GT denotes the ground-truth unlabeled data distribution. TP (FP) denotes the predicted true (false) positive pseudo-labels. The dataset is CIFAR-10-H (Nmazy Mmaax Ti %a) (400, 4600, 50, 50). Our CPG can generate more reliable pseudo-labes labels than FreeMatch, and SimProt in both minority classes like class 1, 2 and majority classes like class 8, 9. $ indicates that we reproduce baseline methods with the auxiliary branch, similar to our CPG.
with
Freehstrra SinPm
OLD
FreeNaect
StrProi
100
80
Accuracy (S4)
50
40-
30-
20-
5.5
185
185
Epoch
(a) Pseudo-label error rate
(b) Pseudo-label utilization rate
(c) Testing accuracy
Figure 7: Comparison of pseudo-labe error rate (a), pseudo- utilization rate (b), and testing accuracy (c) among FreeMatcht [8], SimProt [13], and CPG under arbitrary unlabeled data distribution. The dataset is CIFAR-1O-LT with (Nmaz M "R, 7.) (400, 4600, 50, 50). The vertical gray dotted line indicates the initiation of pseudo-labeling, Our CPG generate pseudo- labels with a lower error rate and comparable utilization rate, achieving superior testing accuracy compared to both FreeMatch and SimProj. indicates that reproduce baseline methods with the auxiliary branch, similar to our CPG.
29

---

F33 Ablation Study on the Anchor Distributions
In the main paper, we compare our CPG with ACRf (without anchor distributions) since anchor distributions are typically unknown in real-world scenarios. Here, we further supplement this analysis by comparing CPG with ACR (with anchor distributions) under the assumption that anchor distributions are known. As shown in Tables 14, 15, and 16, our method outperforms ACR even when anchor distributions are known, while ACR relies on this prior knowledge. This demonstrates the robustness and practical applicability of our method.
Table 14: Comparison of accuracy (%) on CIFAR-IO-LY (N maz 400, M 4600) with class imbalance ratio 7 E (100, 150, 200} under different unlabeled data distributions.
Y=
Avg.
Arbirney
Consisten
Arbimney
Invens
Consistent
Arbitary
Inveme
Consistent
ACR- [14]
+4.21
62.54.a400
73.20 +1.00
18.22:88.07
#1.03
20:35
50.13
ACR [14)
76.18-66:11
75.72 41.05
73:27 10.96
66.69 #2.03
66.68 #1.82
09:34:300
09.91 13.50
67.25
70006
22.19
82.10 #0.74
82.37 #0015
76.93 +2.68
76.38 #3.87
78.19 #1.63
70.75 #0.13
75.18 #3.38
77.45 #1.44
68.33
76.41 #2.21
Table 15: Comparison of accuracy (%) on CIFAR-100-LTT (Nmax = 50, Mmas = 450) with class imbalance ratio y E (10, 15, 20) under different unlabeled data distributions.
Method
7- 10
15
7 20
Avg.
Arbitney
Inverse
Consisten
Arhitray
Invere
Consistent
Arbitrary
Inverve
Consistent
ACR [14)
00.00::: a:1 .M
48. 10.50:40 a0:0
41.33 20.41
35.12 11.36
39.01 44.358
32 32.73+11 #1.74
29:08 $1.11
32.78 11.59
29.69 22:4
=1
ACR [14]
50.92 80096
50.29 20.71
50.57 30.98
44.02 10.63
43.49 20.54
44.68 20.90
41.93 11.64
39.62 #1.3
42.28 1222
45.31 2100
Ours
51.48 #0.32
51.46 #0.2
51.22 #077
46.36 + 1:440
47,88 #0.34
45.94 #1.02
44.17 #1.30
44.17 #1.01
42,66 20.56
47.25 20.75
Table 16: Comparison of accuracy (%) on Food-101-LT (N
50, Maar max 450) with class
imbalance ratio y E (10, 15) and SVHN-LT (Nmax = 400, Maaz = 4600, 7 = 100) under different unlabeled data distributions.
Food 10:-1
SVEN-LT
Mathoo
10
2-20 15
Aug
7 300
Avg.
Arttruay
Bavesse
Consinent
ArNoua
Cossinent
Jubinay
Javesse
Consiscatt
19.27
17.30
1544
13.58
16.90
88.17 60.50
92 12.64
ACR (1:)
21.87 0175113
22.38-5000
21 40.47
19.11
50.5
18.84
20,50
60
83.04
M0,04
89. 19
89.32
Oui
25.98 30.9
25.52 10.4
25.24 30.30
21,88 5085
21.37 36.51
23,11 50.73
2347 30.38
90.99 2034
94.74 19,09
90.45 30:33
94,06 1::45
G Relation to Existing Methods
Here, we summarize the difference between our CPG and existing methods (i.e., FixMatch [7]. FlexMatch [29], UPS [48], CADR [49], FlexDA [50], DASO [51]).
While CPG adopts bonfidence-based filtering similar FixMatch [7] and FlexMatch [29], its key innovation is the controllable self-reinforcing optimization cycle, which incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset rather than employing consistency regularization, distinguishing it from existing methods. This cycle operates in three steps: (i) expanding the labeled dataset with reliable pseudo-labels, (ii) constructing a Bayes-optimal classifier, and (iii) iteratively improving pscudo-label quality, forming a theoretically grounded feedback loop. Additionally. CPG ensures full sample utilization via an auxiliary branch and reduces label noise accumulation through a voting- based stabilization technique, which enforces consistent pseudo- label assignments across training steps. As evidenced by the ablation studies in Table 5, these components individually contribute significant performance improvements, with the auxiliary branch and optimization cycle yielding average gains of 1.82 pp and 6.97 pp, respectively.
The goals of UPS [48] and CADR [49] differ from ours. UPS targets poor model calibration in SSL without distribution mismatch and CADR addresses distribution mismatch due to label missing not at random, while our work focuses on handling the unknown and arbitrary unlabeled data distribution. Specifically, UPS mitigates high-confidence incorrect pseudo-labels caused by poor model calibration through uncertainty estimation techniques like MC- MC-Dropout [52], and CADR focuses on resolving classifier bias due to label missing not at random by employing class-aware propensity estimation and dynamic threshold adjustment for pseudo-labe selection. Unlike these existing techniques that rely on uncertainty estimation or threshold adjustment, we propose a novel controllable self-reinforeing optimization cycle that operates without distribution estimation or correction, introducing distinct conceptual and methodological innovations. More importantly, our framework specifically designed to address real-world scenarios with unknown and arbitrary unlabeled data distributions.
30

---

While FlexDA [50] relies on estimating the unlabeled data distribution for distribution alignment and pseudo-label generation, our core contribution introduces controllable self-reinforcing optimization cycle that entirely bypasses unlabeled data distribution estimation. This approach is fundamentally different from FlexDA's dynamic distribution alignment. Specifically, we iteratively expand the labeled dataset with reliable geeuoollbbll, train a Bayes-optimal classifier via logit adjustment on the updated labeled dataset without using the pseudo-label distribution. More importantly, we further theoretically prove that this optimization cycle can significantly reduce the generalization error, whereas FlexDA lacks such theoretical guarantees. Moreover, our auxiliary branch leverages all available samples (without confidence threshold) to enhance feature learning while preventing the primary branch classifier from error pseudo-labels. Notably, prior methods (e.g., FlexDA [50], SimPro [50], UPS [48], DASO [51]) lack class-at class-aware enhancements for tail classes.
H Limitations
(i) Our CPG assumes that the labeled data is free from noise during training. Consequently, CPG may encounter challenges when adapting scenarios involving noisy labels. (ii) Our CPG operates under the assumption that labeled data follows a long-tailed distribution, with even tail classes retaining minimal supervision. However, its performance may degrade when labeled data follows uniform distribution but suffers from extremely limited supervision. Future work will explore extensions to address these limitations and improve model generalization.
I Compute Resources
CPU: AMD EPYC 7642 48-Core Processor 2
GPU: NVIDIA GeForce RTX 4090 24G x 1
MEM: 500G
Maximum total computing time: training + testing 17h
Broader Impacts
This paper presents work whose goal is to advance the field of Machine Learning. Specifically, propose a new realistic long-tailed semi-supervised learning algorithm to improve performance in unknown arbitrary distribution scenarios by expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with known distribution, making it unaffected by the unlabeled data distribution. We further theoretically prove that this iterative process can significantly reduce the generalization error under some conditions. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
31


Reviews:
[{"review_id": "9CicyKaHZ2", "reviewer": "Reviewer_TLMt", "summary": "The paper studies the problem of Pseudo-label generation under a Realistic Long-Tailed Semi-Supervised Learning (ReaLTSSL) setting. In this setting, a long-tailed distribution (characterized by an imbalance ratio) is combined with an unlabeled set that has unknown and even arbitrary distribution. \n\nThe authors proposed a iterative method to generate labels for the unlabeled set. The method keeps the pseudo-label for unlabeled images only when its weakly-augmented and strongly-augmented views predict the same class and both predictions exceed a preset confidence threshold. Then the accepted images are moved into the labeled set and the classifier is retrained with logit adjustment. This process is then repeated for more labels. \n\nThe paper also proposed two add-ons CAA (help with minority classes) and an auxiliary FlixMatch-style branch for consistency regularization. \n\nThe method is then evaluated on 4 commonly used benchmarks and achieves a 16.29% performance gain on CIFAR-10-LT outperforming previous methods for long-tailed SSL. The authors also provided theoretical analysis on the optimization cycle.", "questions": "1. We see a huge performance gain for CIFAR-10-LT, but the performance gain is less significant for CIFAR-100-LT. Is it possible to provide some reasoning on why the performance gain is less significant for a similar dataset with higher class cardinality? \n\n2. The paper uses confidence interval $\\tau=0.95$ to filter out the pseudo-labels. Is it possible to use a different confidence interval, and how does it affect the performance?\n\n3. The experiments are conducted on long-tailed datasets with class imbalance. Could you also give macro-F1 for the main CIFAR-10-LT and CIFAR-100-LT results? A quick per-class breakdown or confusion matrix would make it clear where CPG’s gains come from.\n\n4. Have you tried CPG on non-vision data (e.g., text, audio)? What components would need to change for other modalities?\n\n5. In line 470, for risk of both the labeled and unlabeled dataset, the data $x_i$ ranges starting from $1$. Maybe it is possible to use notation to differentiate the two or let the latter one start from $N+1$?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality:\nStrength: The paper did rigorous evaluation on four datasets (CIFAR-10-LT/100-LT, Food-101-LT, SVHN-LT) across multiple imbalance ratios. The paper also conducted ablation studies to isolate the contribution of each module. The paper provided a well-documented code in the supplementary material that is clear and easy to follow. The paper provided a theoretical risk-reduction bound (Theorem 1) and its practical implications are discussed (lines 249-260).\nWeakness: The proposed method CPG is supposed to target the long-tailed distribution with class imbalance. However, only accuracy is reported; macro-F1 or other metrics are not reported.\n\nClarity:\nStrength: The paper is well-written and easy to follow. The algorithm is concisely conveyed in Algorithm 1 together with previously defined equations.\nWeakness: The plots on page 9 feel crowded and break the reading flow. \n\nSignificance:\nStrength: CPG achieves up to 16.29% performance gain on CIFAR-10-LT. The advantage of CPG is clear in Figure 1.\nWeakness: Performance gain on CIFAR-100-LT is lower compared with CIFAR-10-LT. A discussion of why performance drops as class cardinality grows would strengthen the claim of broad significance. The paper only investigated the performance of computer vision datasets. It is not clear how the proposed method performs on other modalities.\n\nOriginality:\nWeakness: The starting filter process uses weak/strong-view confidence. This is similar to FixMatch/FlexMatch. The paper can highlight more clearly how CPG diverges from the other methods conceptually in the appendix.\nStrength: CPG only keeps an unlabeled sample when weak and strong views agree (stricter than Fix/FlexMatch). The accepted samples are promoted to the labeled pool, and then a logit-adjusted head is retrained with pseudo-labels and logit adjustment, which helps with class imbalance.", "was_revised": true, "final_justification": "The authors conducted experiments with Macro-F1, which further strengthens the claim that the proposed method works well with class imbalance. The part that CIFAR-100-LT is inherently harder and thus the performance gain is smaller convinced me, and the new audio-modality experiments broaden the method’s demonstrated scope. I also appreciate the clearer discussion of how CPG diverges from related approaches, as well as the improved notation in the final paper. As a result, I maintain my recommendation for acceptance.", "rebuttal": "### **The proposed method is supposed to target the long-tailed distribution. However, only accuracy is reported (W1 & Q3)**\nWe appreciate the reviewer’s insightful comments. To better evaluate our method‘s performance under class imbalance, we have now included the Macro-F1 metric in our analysis. Table R1 presents results on two benchmarks: (i) **CIFAR-10-LT ($N_{max}=400$, $M_{max}=4600$, $\\gamma_l=100$, $\\gamma_u=100$)** in the left four columns, and (ii) **CIFAR-100-LT ($N_{max}=50$, $M_{max}=450$, $\\gamma_l=10$, $\\gamma_u=10$)** in the right four columns. In both cases, the labeled data follows a long-tailed distribution, while the unlabeled data varies across long-tailed, inverse long-tailed, and arbitrary distributions.\n\nThe results in Table R1 show that our method achieves statistically significant improvements in Macro-F1 over existing baselines, further validating its effectiveness for imbalance learning. Notably, it yields average Macro-F1 gains of **13.84%** on CIFAR-10-LT and **4.55%** on CIFAR-100-LT. These results and analysis will be added to **Section 4.2 Results** in the final version.\n##### Table R1\n| Unlabeled distribution | Arbitrary | Inverse long-tailed | Long-tailed | Avg. | Arbitrary | Inverse long-tailed | Long-tailed | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| FixMatch |53.52|54.24|66.47|58.07|44.58|44.85|46.46|45.30|\n| FreeMatch|66.00|67.00|68.04|67.01|44.13|43.57|43.84|43.85|\n| SoftMatch|62.49|66.66|71.53|66.89|45.67|46.73|46.91|46.44|\n| ACR      |59.01|64.03|70.25|64.43|45.69|48.89|41.92|45.50|\n| SimPro   |61.22|62.11|57.81|60.38|39.98|41.54|41.87|41.13|\n| CDMAD    |61.73|62.88|62.11|62.24|32.97|35.80|36.68|35.15|\n| Ours     |**81.99**|**82.31**|**78.25**|**80.85**|**51.31**|**51.25**|**50.41**|**50.99**|\n### **The plots on page 9 feel crowded and break the reading flow (W2)**\nAs suggested, we will improve the clarity and layout of the plots on page 9 to enhance readability in the final version.\n### **Performance drops as class cardinality grows (for example on CIFAR-100-LT) (W3.1 & Q1)**\nWe respond to the reviewer’s concerns regarding the relationship between performance gains and class cardinality.\nCIFAR-100 and CIFAR-10 have the same total training set size, but CIFAR-100 divides the data into 100 fine-grained classes (compared to 10 in CIFAR-10), reducing per-class samples by an order of magnitude. The finer class granularity also increases inter-class similarity, making classification inherently more challenging. As a result, the upper-bound performance (e.g., supervised learning) on CIFAR-100-LT (**64.62%**) is significantly lower than on CIFAR-10-LT (**85.22%**).\n\nMoreover, consistency regularization enhances model stability by encouraging similar predictions for different augmentations of the same sample. This promotes robust representations, preventing severe performance degradation and ensuring that the lower-bound performance on CIFAR-100-LT remains relatively stable.\n\nGiven the narrower performance gap between upper and lower bounds in CIFAR-100-LT (compared to CIFAR-10-LT), the absolute improvement achievable by our method is inherently constrained. Nevertheless, our method still achieves substantial average accuracy gains of **3.09%** on CIFAR-100-LT (as shown in Table 2). These analysis will be added to **Section 4.2 Results** in the final version.\n### **Evaluation on other modalities (W3.2 & Q4)**\nWhile our method is evaluated on vision tasks, **the core principles of CPG can be extended to other modalities**. In this rebuttal, we further evaluate our method on the **audio modality**. Specifically, for audio data, this adaptation requires two key modifications: (i) replacing vision-specific backbones (e.g., WideResNet-28-2) with audio-compatible models like HuBERT, and (ii) substituting image augmentations with audio-specific ones. To demonstrate this adaptability, we perform our CPG on ESC-50, a standard benchmark for environmental sound classification. As demonstrated in Table R2, our method achieves state-of-the-art performance on this audio benchmark, confirming its effectiveness beyond visual modality. These results strongly support the generalizability of CPG's framework across different modalities. These results and analysis will be added to **Section 4.2 Results** in the final version.\n##### Table R2\n| FreeMatch | SimPro | Ours |\n| --- | --- | --- |\n| 68.13 | 68.75 | **69.13** |\n### **Clarify how CPG diverges from other methods like FixMatch/FlexMatch (W4)**\nWhile CPG adopts confidence-based filtering similar to FixMatch/FlexMatch, its key innovation is the **controllable self-reinforcing optimization cycle**, which incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset rather than employing consistency regularization, distinguishing it from existing methods. This cycle operates in three steps: (i) expanding the labeled dataset with reliable pseudo-labels, (ii) constructing a Bayesian classifier, and (iii) iteratively improving pseudo-label quality, forming a theoretically grounded feedback loop. Additionally, CPG ensures full sample utilization via an auxiliary branch and reduces label noise accumulation through a voting-based stabilization technique, which enforces consistent pseudo-label assignments across training steps. As evidenced by the ablation studies in Table 5, these components individually contribute significant performance improvements, with the auxiliary branch and optimization cycle yielding average gains of **1.82%** and **6.97%**, respectively.\n### **Different confidence threshold selection (Q2)**\nThe confidence threshold $t=0.95$ is a common setting in semi-supervised learning, which **we adopt for fair comparison**. To evaluate the impact of different confidence thresholds on model performance, we conduct additional experiments on CIFAR-10-LT (using the same experimental setup as in Table R1) with thresholds of $t=0.85$ and $t=0.75$. As shown in Table R3, our method’s performance variations across different thresholds remain relatively small (within 1-2%), and it maintains consistently superior performance compared to baseline approaches. This threshold-insensitive property highlights the stability of our method, and these results will be incorporated into the final version of the paper in **Section 4.3 Analysis**.\n##### Table R3\n| Unlabeled distribution | Arbitrary | Inverse long-tailed | Long-tailed |\n| --- | --- | --- | --- |\n| FreeMatch |66.33|68.10|69.42|\n| SimPro   |64.73|65.25|64.37|\n| Ours ($t=0.75$) |81.35|78.26|76.20|\n| Ours ($t=0.85$) |81.89|80.85|77.79|\n| Ours ($t=0.95$)|**82.33**|**82.32**|**78.35**|\n### **Clarify equation notation (Q5)**\nTo improve notational clarity, we will modify the indexing in line 470 so that labeled data indices run from 1 to $ N $ and unlabeled data indices run from $N+1$ to $N+M$ in the final version."}, {"review_id": "YLBFtk6NUW", "reviewer": "Reviewer_ydDN", "summary": "This paper addresses the challenge of semi-supervised learning when the distribution of unlabeled data is arbitrary and unknown.\nThe authors propose the CPG framework, which selectively adds only reliable pseudo-labels to the labeled set from unlabeled set, and applies logit adjustment based on the known class distribution.\nThey further introduce class-aware adaptive augmentation and an auxiliary branch to improve minority class representation and make full use of both labeled and unlabeled data.\nExperiments on several long-tailed benchmarks, including CIFAR-10-LT, CIFAR-100-LT, Food-101-LT, and SVHN-LT, show that CPG significantly outperforms previous methods, with gains of up to 16 percentage points.\nTheoretical analysis also supports the approach by showing that the iterative pseudo-labeling process helps reduce generalization error.", "questions": "- Since equation (3) uses a confidence threshold to filter pseudo-labels, it would be helpful to provide an ablation study or sensitivity analysis showing how different threshold values affect the overall performance,\n- As the method assumes noise-free labeled data, it would improve the paper to include experiments evaluating the robustness of CPG under various levels of label noise, as well as comparisons with baseline methods such as FreeMatch and SimPro.\n- It would further strengthen the paper if the effectiveness of the proposed method could be proved on large-scale or real-world datasets beyond standard benchmarks.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- While previous methods require prior assumptions about the distribution of unlabeled data, CPG can operate robustly even under arbitrary unlabeled data distributions. Moreover, unlike previous ReaLTSSL approaches (e.g., SimPro) that estimate the unlabeled data distribution, CPG only accumulates reliable samples as labeled data, resulting in a more robust training process.\n- The proposed key components (CPG, CAA, auxiliary branch) are thoroughly ablated, and the paper provides detailed analysis across various distribution settings \n- The paper is easy to follow.\n- The proposed method is theoretically justified with formal proofs.\n- The authors provide code to ensure the reproducibility of their results. \n\nWeaknesses: \n- The CPG framework accumulates only reliable pseudo-labels from the unlabeled data into the labeled pool, which poses a risk that tail class samples may be entirely omitted in certain situations. In such cases, the corresponding classes are completely excluded from training and loss balancing (logic-adjustment), potentially resulting in the model failing to learn or classify tail classes at all.\n- Equation (3) uses a confidence threshold to filter reliable pseudo-labels. However, since the performance of the proposed method is likely to be highly sensitive to the choice of this threshold, the paper lacks sufficient ablation analysis regarding this hyper parameter.\n- As mentioned in Section H (Limitation), the current method may encounter problems when noisy labels are present. In real-world scenarios, label noise can exist and potentially harm performance. Therefore, it would be valuable to report experimental results analyzing how CPG performs compared to baseline methods (e.g., FreeMatch, SimPro) under varying levels of label noise in the dataset.", "was_revised": true, "final_justification": "The authors have addressed my main concerns in the rebuttal, providing additional experiments that demonstrate the method can learn tail classes effectively, and that it is robust to different confidence thresholds, label noise, and real-world large-scale datasets.\nGiven these clarifications and the strong empirical results, I maintain my rating as borderline accept.", "rebuttal": "### **Potentially fail to learn tail classes (W1)**\nWe sincerely appreciate the reviewer’s insightful comments. We would like to clarify that our model does not fail to learn or classify tail classes. While the labeled data follows a long-tailed distribution, **it does contain some tail class samples in the labeled dataset**. Our approach enhances tail classes through logit adjustment and the proposed class-aware adaptive augmentation module, ensuring robust representation even when initial pseudo-label selection misses tail class samples. As shown in Figure 1, our method can generate more reliable pseudo-labels than FreeMatch and SimPro in tail classes like classes 1 and 2. Moreover, as demonstrated in Appendix Figure 5, our method progressively identifies nearly all samples as reliable pseudo-labels during training, with their final distribution closely approximating the true unlabeled data distribution.\n### **Different confidence threshold selection (W2 & Q1)**\nThe confidence threshold $t=0.95$ is a common setting in semi-supervised learning, which **we adopt for fair comparison**. To evaluate the impact of different confidence thresholds on model performance, we conduct additional experiments on CIFAR-10-LT (using the same experimental setup as in Table R1) with thresholds of $t=0.85$ and $t=0.75$. As shown in Table R4, our method’s performance variations across different thresholds remain relatively small (within 1-2%), and it maintains consistently superior performance compared to baseline approaches. This threshold-insensitive property highlights the stability of our method, and these results will be incorporated into the final version of the paper in **Section 4.3 Analysis**.\n##### Table R4\n| Unlabeled distribution | Arbitrary | Inverse long-tailed | Long-tailed |\n| --- | --- | --- | --- |\n| FreeMatch |66.33|68.10|69.42|\n| SimPro   |64.73|65.25|64.37|\n| Ours ($t=0.75$) |81.35|78.26|76.20|\n| Ours ($t=0.85$) |81.89|80.85|77.79|\n| Ours ($t=0.95$)|**82.33**|**82.32**|**78.35**|\n### **Noisy scenarios (W3 & Q2)**\nTo comprehensively evaluate our method’s robustness under noisy scenarios, we conduct additional experiments on CIFAR-10-LT ($N_{max}$=400, $M_{max}$=4600, $\\gamma_l$=100, $\\gamma_u$=100) under varying noise rates (0%, 10%, 20%, and 30%). The labeled data follows a long-tailed distribution, while the unlabeled data adheres to an arbitrary distribution. As demonstrated in Table R5, our method consistently outperforms baselines in noisy settings, achieving the highest accuracy across all noise rates with minimal performance degradation. Notably, when noise increases from 0% to 30%, our method exhibits a degradation (Δ) of only **−12.46%**, compared to **−15.04%** for FreeMatch and **−29.93%** for SimPro. This demonstrates our method's strong robustness to label noise, and these results will be added to **Section 4.3 Analysis** in the final paper.\n##### Table R5\n| Noise rate | 0% | 10% | 20% | 30% | Δ(0→30%) |\n| --- | --- | --- | --- | --- | --- |\n| FreeMatch |66.33|60.33|56.24|51.29|-15.04|\n| SimPro    |64.73|59.86|44.54|34.80|-29.93|\n| Ours      |**82.33**|**75.95**|**71.82**|**69.87**|**-12.46**|\n### **Large-scale or real-world datasets (Q3)**\nTo further evaluate our method’s effectiveness in large-scale and real-world scenarios, we conduct experiments on ImageNet-127 (an imbalanced dataset, **127 classes**, **imbalance ratio 286**) with an arbitrary (non-long-tailed) distribution. As shown in Table R6, our method consistently outperforms baselines, achieving gains of **+1.27% at 32×32 resolution** and **+2.82% at 64×64 resolution**. These results highlight our method’s robustness in large-scale, realistic settings beyond standard benchmarks. As suggested, these results and corresponding analysis will be incorporated into **Section 4.2 Results** in the final version.\n##### Table R6\n|           | 32 \\*32 | 64 \\*64 |\n| --------- | ----- | ----- |\n| FixMatch  |29.57|37.40|\n| FreeMatch |31.37|39.67|\n| SoftMatch |31.50|39.40|\n| ACR       |38.47|47.61|\n| SimPro    |43.31|46.93|\n| CDMAD     |15.46|21.92|\n| Ours      |**44.58**|**50.43**|"}, {"review_id": "GB7H6sVkvm", "reviewer": "Reviewer_Ckat", "summary": "This paper studies the long-tailed semi-supervised learning problem. Specifically, it investigates an under-explored and more realistic setting: the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. This is a very challenging problem in semi-supervised learning. To solve this problem, this paper proposes a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. The proposed CPG contains a reliable pseudo-labels filtering mechanism and a Bayes-optimal classifier. This paper also proposes a class-aware adaptive augmentation module to enhance the representation of minority classes, as well as an auxiliary branch to maximize data utilization.\n\nGenerally, the idea is quite novel. Especially, the idea of handling the unknown label distribution of unlabeled samples is quite novel and reasonable. The proposed method is supported by a theory that demonstrates the proposed optimization cycle can significantly reduce the generalization error. The authors conducted extensive experiments to evaluate the proposed method, which suggests that it significantly outperforms previous methods by up to 16.29%. The visual illustration in this paper is informative (like Figure 2), and the writing of this paper is also well done. The code is available in the supplementary material.", "questions": "1. Different from the previous works in semi-supervised learning, this paper does not use the consistency between the strong augmentation and the weak augmentation. Please explain the reason. \n2. The Bayes-optimal classifier can handle the tailed class samples well, so why is an additional class-aware adaptive augmentation adopted to enhance the minority class representations? \n3. Why is an auxiliary branch adopted in the proposed method?\n4. The proposed method does not estimate the distribution of the unlabeled samples. It just progressively incorporates more unlabeled samples in the labeled dataset. Will this progressive incorporation finally approximate the ground-truth label distribution of the unlabeled dataset?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "#### Strengths:\n\n1. The investigated problem is interesting, under-explored, and more realistic.\n2. The idea to handle the unknown unlabeled data is novel, and the adopted techniques are reasonable.\n3. The authors theoretically prove that the proposed controllable self-reinforcing optimization cycle can significantly reduce the generalization error.\n4. The experimental performance of the proposed method significantly outperforms the previous methods, and extensive ablation studies validate the effectiveness of each involved component in the proposed method.\n\n#### Weakness:\n\n1. Although the investigated problem is under-explored. I believe the techniques in long-tailed learning will help solve this problem. Therefore, this paper should review the related long-tailed learning methods in the \"Related Work\" section.\n2. Please provide the pseudo-code in the paper, as it will help the authors to reproduce the results.", "was_revised": true, "final_justification": "This rebuttal have addressed all of my concerns. I keep my score.", "rebuttal": "### **Review related long-tailed learning methods (W1)**\nWe sincerely appreciate the reviewer’s insightful suggestion regarding the connection to long-tailed learning. In the revised manuscript, we will expand the **Related Work** section to include a dedicated discussion of long-tailed learning methods, particularly focusing on techniques that could inform solutions to our investigated problem. \n### **Pseudo-code (W2)**\nFor the reviewer's convenience, we would like to highlight that the pseudo-code of our proposed method was already included in Appnedix E of the original submission. We will move it in the main text in the final version.\n### **Without consistency regularization (Q1)**\nWhile consistency regularization between augmentation views is commonly used in semi-supervised learning, we avoid it in our controllable self-reinforcing optimization cycle for two key reasons. First, pseudo-label quality is highly sensitive to the distribution mismatch between labeled and unlabeled data in long-tailed semi-supervised learning. Second, propagating error pseudo-labels from weak augmentation views to strong augmentation views can reinforce the model’s learning of incorrect predictions, leading to error accumulation and performance degradation (as demonstrated in Figures 1 and 3).\n### **Why is an additional class-aware adaptive augmentation necessary (Q2)**\nWhile logit adjustment induced Bayes-optimal classifiers can handle long-tailed distribution, the fundamental challenges of limited sample size and low diversity in minority classes remain unavoidable in long-tailed learning. Our class-aware adaptive augmentation specifically targets these practical limitations by explicitly enhancing minority class representations.\n### **Why use an auxiliary branch (Q3)**\nOur method progressively expands the labeled dataset by identifying reliable pseudo-labels from the unlabeled dataset. However, during early training stages when model performance is still weak, only a limited number of samples meet the reliability condition for pseudo-labeling (as demonstrated in Figures 1 and 3, Appendix Figure 5). To ensure full utilization of all available training samples throughout the learning process, we introduce an auxiliary branch that: (i) guarantees 100% sample participation from the beginning of training, and (ii) provides additional supervision signals to stabilize representation learning during the critical initial phase.\n### **Will the progressive incorporation finally approximate the ground-truth label distribution of the unlabeled dataset (Q4)**\nWhile our method does not explicitly estimate the unlabeled data distribution, the progressive incorporation of reliable pseudo-labels through our controllable self-reinforcing optimization cycle naturally approximates the ground-truth unlabeled data distribution as training progresses. This is because the model’s increasing confidence enables it to gradually include more diverse samples, including those from minority classes, improving distribution alignment. **Empirical results in Appendix Figure 5 demonstrate that the pseudo-label distribution converges toward the ground-truth unlabeled data distribution over time**."}, {"review_id": "05Jj79VmxJ", "reviewer": "Reviewer_HaKG", "summary": "This paper proposes a method for realistic long-tailed semi-supervised learning (ReaLTSSL), where the distributions of labeled and unlabeled data may differ. To address the challenge of unknown and potentially arbitrary unlabeled distributions, the authors propose to bypass direct distribution estimation. Instead, the method progressively identifies reliable pseudo-labels and adds them to the labeled set, forming an updated dataset with a known class distribution. A Bayes-optimal classifier is then trained using logit-adjusted loss based on this controlled distribution.\n\nThe core of the approach is the Controllable Self-reinforcing Optimization Cycle, which uses agreement between weak and strong augmentation predictions to select consistent pseudo-labels. These labels are treated as reliable and used to update the training distribution in a feedback loop. To further enhance performance, the method introduces a Class-aware Adaptive Augmentation (CAA) module that strengthens minority class representations based on intra-class compactness, and an Auxiliary Branch that enforces prediction consistency across augmentations to better utilize all samples.\n\nThe experimental evaluation is comprehensive. The authors test on four benchmark datasets: CIFAR-10-LT, CIFAR-100-LT, SVHN-LT, and Food-101-LT under a variety of distribution mismatch settings, including long-tailed, inverse, and arbitrary configurations. The method achieves consistent improvements over strong baselines in both aligned and mismatched label/unlabeled distribution settings.", "questions": "The proposed method starts training with logit adjustment applied to the limited and imbalanced labeled data, before incorporating any pseudo-labels. Under severe long-tailed settings, where the labeled sample size is extremely small, how reliable is this initial supervised classifier? Has the method been evaluated under more extreme label sparsity (e.g., lower N_max) to stress test this dependency?\n\nHow does the proposed method differ in principle from related realSLL works and other related works that also can handle distribution mismatch? can the authors clarify what is truly novel about the proposed pipeline beyond recombining existing techniques?\n\nThe authors introduce arbitrary unlabeled data distributions to simulate realistic challenges. However, are there real-world datasets or applications where such labeled/unlabeled distribution divergence naturally occurs? \n\nThe experiments use a single backbone (WideResNet-28-2). Has the method been tested on other architectures? \n\nIf these issues are carefully addressed, I would consider raising my score.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n-The paper targets the mismatch between labeled and unlabeled data distributions,  which reflects real-world data collection scenarios. The proposed self-reinforcing cycle uses consistency between weak and strong views to select pseudo-labels and gradually improve the labeled distribution. This approach is simple to understand and avoids unstable threshold tuning.  \n\n -The method is tested across four standard benchmarks (CIFAR-10/100-LT, SVHN-LT, Food-101-LT) and three types of distribution mismatches (consistent, inverse, arbitrary). The design of diverse imbalance ratios and distribution scenarios strengthens the generality of the results.\n\n-The ablation studies isolate the contributions of each module (CAA, auxiliary branch, and optimization cycle), and the use of pairwise t-tests provides strong statistical support for performance claims.\n\n**Weaknesses**\n\n-The method combines several existing ideas, such as logit adjustment, augmentation-aware consistency, and pseudo-label voting, but **the conceptual novelty is relatively limited**.\n\n-Similar goals have been addressed in earlier works like UPS [1] (uncertainty filtering based on model disagreement) and CADR[2] (reweighting pseudo-labels to correct distribution mismatch). These are not sufficiently discussed.\n\n-The method still relies heavily on a **strong initial supervised phase**, and the assumption that the initial labeled set can provide stable logit-adjusted priors is not fully examined under extreme settings.\n\n\n[1] IN DEFENSE OF PSEUDO-LABELING: AN UNCERTAINTY-AWARE PSEUDO-LABEL SELECTION FRAMEWORK FOR SEMI-SUPERVISED LEARNING \n\n[2] ON NON-RANDOM MISSING LABELS IN SEMI-SUPERVISED LEARNING", "was_revised": true, "final_justification": "I acknowledge the authors' detailed rebuttal and appreciate the additional experiments addressing my earlier concerns, especially those related to performance under extreme settings and the comparison with FlexDA. However, after carefully reviewing all responses, my primary concerns regarding the conceptual novelty remain unresolved.\n\nSpecifically, the authors repeatedly emphasize that the novelty of their method is confirmed by other reviewers (e.g., TLMt, ydDN, Ckat), yet upon closely examining these reviewers' comments, Reviewer TLMt explicitly lists originality as a weakness rather than a strength. \n\nThe authors argue their novelty lies in the explicit integration of these methods into a \"self-reinforcing optimization cycle.\" However, iterative training approaches involving pseudo-label updates are common in existing SSL frameworks (e.g., CReST’s iterative distribution adjustment). Label refinement cycles are inherent in many self-learning methods, although not explicitly introduced with the \"self-reinforcing\" terminology presented here. Thus, the claimed novelty is primarily terminological rather than genuinely methodological.\n\nFurthermore, while the proposed method (\"Leash\") integrates established techniques, such as pseudo-label distribution correction, logit adjustment, consistency regularization, and class-aware strategies, these components have individually appeared frequently in prior literature. Moreover, the motivation closely aligns with the previously published method, \"Flexible Distribution Alignment: Towards Long-tailed Semi-supervised Learning with Proper Calibration\" (ECCV 2024), further diluting the originality of this submission. Finally, before the rebuttal, the experiments were restricted to a single backbone (WideResNet-28-2), and inconsistent class imbalance ratios were applied across different datasets. These experimental limitations weaken the generalizability and rigor required for NeurIPS acceptance.\n\nWhile other reviewers have rated this paper highly, my independent evaluation suggests it does not reach the expected novelty and methodological standards required by NeurIPS. Therefore, I maintain my original rating (3: Borderline reject). However, I am OK whether this paper is accepted or not.", "rebuttal": "### **The main novelty of the work (W1 & Q2)**\nThanks for your comments. While our method leverages established components (e.g., logit adjustment, consistency regularization, and pseudo-label voting), its **core innovation is fundamentally different from prior works**.\n\nExisting methods typically employ high-confidence thresholds to filter pseudo-labels, then estimate unlabeled data distribution for subsequent refinement. However, during early training stages, the confidence threshold may completely exclude samples from certain classes. This leads to inaccurate estimation of the unlabeled data distribution, which in turn adversely affects subsequent pseudo-label generation steps that depend on this estimation. As shown in Figure 3, this leads to error accumulation and performance degradation.\n\nOur method introduces a controllable self-reinforcing optimization cycle that: (i) **progressively integrates reliable pseudo-labels from the unlabeled dataset into the labeled dataset**, and (ii) **trains on the updated labeled dataset, which follows a known distribution**. This strategy eliminates dependence on error-prone unlabeled data distribution estimation, effectively circumventing distribution estimation entirely. Meanwhile, **we theoretically prove that this optimization cycle can significantly reduce the generalization error**.\n\nOur method is unaffected by the unlabeled data distribution and addresses the unknown and arbitrary unlabeled data distribution, in contrast to prior works that attempt (and often fail) to estimate the unlabeled data distribution. As demonstrated in Table 5 of the paper, this cycle contributes an average performance improvement of **6.97%**, representing the most significant gain among all components in our framework. **Reviewers TLMt, ydDN, and Ckat all confirm our novelty**.\n### **Discuss related work (W2)**\nThe goals of UPS and CADR differ from ours. UPS targets poor model calibration in SSL without distribution mismatch and CADR addresses distribution mismatch due to label missing not at random, while our work focuses on handling the unknown and arbitrary unlabeled data distribution. Furthermore, our approach introduces distinct conceptual and methodological innovations. We provide a comprehensive comparison with both UPS and CADR below.\n\nUPS mitigates high-confidence incorrect pseudo-labels caused by poor model calibration through uncertainty estimation techniques like MC-Dropout. CADR focuses on resolving classifier bias due to label missing not at random by employing class-aware propensity estimation and dynamic threshold adjustment for pseudo-label selection. Unlike these existing techniques that rely on uncertainty estimation or threshold adjustment, we propose **a novel controllable self-reinforcing optimization cycle** that operates without distribution estimation or correction. More importantly, our framework is specifically designed to address real-world scenarios with unknown and arbitrary unlabeled data distributions. These discussion will be added in the final version.\n### **Evaluation under extreme settings (W3 & Q1)**\nIn long-tailed learning scenarios, there exists an inherent trade-off between extremely small $N_{max}$ (number of samples in the most frequent class) and large imbalance ratio $\\gamma$ values, since $N_{min} = N_{max}/\\gamma \\geq 1$. Our initial experiments on CIFAR-10-LT in Table 1 already employed challenging settings ($N_{max}=400$, $M_{max}=4600$, $\\gamma_l=100$, $\\gamma_u=100$) with **$N_{min}=4$**. Notably, even with such extremely few tail class samples, our method still demonstrates significant performance gains over baselines.\n\nTo further evaluate our method, we conducted additional experiments under **three more extreme settings**: **extreme sparsity setting A1** ($N_{max}=100$, $N_{min}=1$, $M_{max}=4900$, $\\gamma_l=100$, $\\gamma_u=100$), **extreme sparsity setting A2** ($N_{max}=10$, $N_{min}=1$, $M_{max}=4990$, $\\gamma_l=10$,$\\gamma_u=300), and **extreme imbalance setting B** ($N_{max}=400$, $N_{min}\\approx1.33$ (1 in practice), $M_{max}=4600$, $\\gamma_l=300$, $\\gamma_u=300$).\nAs demonstrated in Table R7, our method achieves state-of-the-art performance across three settings, attaining 69.56% accuracy in setting A1, 49.28% in setting A2, and 77.78% in setting B. These results represent significant improvements over all baseline methods, underscoring our method's exceptional capability to handle extreme settings.\n##### Table R7\n|           | Extreme sparsity setting A1 | Extreme sparsity setting A2 | Extreme imbalance setting B |\n| --------- | ----- | ----- | ----- |\n| FreeMatch |63.86|45.75|55.92|\n| SimPro    |31.01|15.20|50.98|\n| Ours      |**69.56**|**49.28**|**77.78**|\n### **Real-world datasets (Q3.1)**\nThe Food-101 dataset (evaluated in Table 3) is a widely adopted benchmark for food image classification. Its training set naturally contains a proportion of label noise, making the data distribution implicitly unknown. Thus, distribution mismatches between labeled and unlabeled data occur when splitting the training set into labeled and unlabeled datasets. Despite these challenges, our method achieves a significant average accuracy gain of **2.57%** on this dataset.\n\nTo further evaluate our method’s effectiveness in large-scale and real-world scenarios, we conduct experiments on ImageNet-127 (an imbalanced dataset, **127 classes**, **imbalance ratio 286**) with an arbitrary (non-long-tailed) distribution. As shown in Table R8, our method consistently outperforms baselines, achieving gains of **+1.27% at 32×32 resolution** and **+2.82% at 64×64 resolution**. These results highlight our method’s robustness in large-scale, realistic settings beyond standard benchmarks. As suggested, these results and corresponding analysis will be incorporated into **Section 4.2 Results** in the final version.\n##### Table R8\n|           | 32 \\*32 | 64 \\*64 |\n| --------- | ----- | ----- |\n| FixMatch  |29.57|37.40|\n| FreeMatch |31.37|39.67|\n| SoftMatch |31.50|39.40|\n| ACR       |38.47|47.61|\n| SimPro    |43.31|46.93|\n| CDMAD     |15.46|21.92|\n| Ours      |**44.58**|**50.43**|\n### **Real-world applications (Q3.2)**\nIn medical applications, **distribution mismatch between labeled and unlabeled data occurs naturally**. Labeled clinical datasets from hospitals typically follow a long-tailed distribution, with abundant cases of common diseases (head classes) but very few cases of rare diseases (tail classes). In contrast, unlabeled data collected from broader populations typically includes a majority of healthy individuals and only a small proportion of diseased individuals, especially those with rare diseases. This creates a fundamental divergence between labeled and unlabeled data distributions.\n\nFurthermore, disease prevalence varies significantly across regions due to differences in climate, healthcare access, and other factors. Consequently, datasets collected from different geographical areas may exhibit arbitrary distribution patterns.\n### **Evaluation using other architectures (Q4)**\nTo evaluate architectural generalization, we conduct experiments using ResNet-50 on CIFAR-10-LT with ($N_{max}$=400, $M_{max}$=4600, $\\gamma_l$=100, $\\gamma_u$=100). The labeled data follows a long-tailed distribution, while the unlabeled data conforms to either an inverse long-tailed or an arbitrary distribution. As evidenced by Table R9, our approach demonstrates significant improvements, achieving absolute performance gains of **7.85%** and **7.23%** over current state-of-the-art methods in these respective scenarios. These results highlight our method's consistent robustness when applied to different network architectures and its ability to handle varying distribution patterns in the unlabeled data. The complete experimental results and detailed analysis will be incorporated into **Section 4.3 Analysis** of the final paper to further support these claims.\n##### Table R9\n|           | Arbitrary | Inverse long-tailed |\n| --------- | ----- | ----- |\n| FreeMatch |47.68|48.28|\n| SimPro    |43.67|51.26|\n| Ours      |**55.53**|**58.49**|"}]

Author rebuttal:

Meta-review: The manuscript addresses the problem of long-tailed semi-supervised learning. The main contributions are a method for controllable pseudo-label generation, its theoretical analysis, a class-aware augmentation, and the experimental evaluation. While the theoretical analysis and the evaluation are the main strengths of the generally well-written manuscript, the novelty of the approach is limited compared to FixMatch/FlexMatch/UPS/CADR/FlexDA. However, on a detail level, the authors argue convincingly in relation to those methods and the results together with the theoretical analysis justify acceptance of the paper.
After rebuttal and discussion three reviewers are rating accept (2) or BA, only HaKG suggest to reject (BR). Most of the pros and cons above a shared among the reviewers, including the criticism regarding novelty. HaKG argues that in particular FlexDA is very similar, but also points out that acceptance would be ok. Thus the AC reads this as a plain borderline and suggests to accept the manuscript.
Decision: Accept (poster)