Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition
Authors: Zi-Hao Zhou
Siyuan Fang
Zi-Jing Zhou
Tong Wei
Yuanyu Wan
Min-Ling Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Semi-supervised learning
Long-tail learning
Weakly-supervised learning
Primary area: machine_vision
TL;DR: This paper introduces a probabilistic framework for long-tailed learning and extends to semi-supervised  learning based on continuous pseudo-labels.

Abstract: Long-tailed semi-supervised learning poses a significant challenge in training models with limited labeled data exhibiting a long-tailed label distribution. Current state-of-the-art LTSSL approaches heavily rely on high-quality pseudo-labels for large-scale unlabeled data. However, these methods often neglect the impact of representations learned by the neural network and struggle with real-world unlabeled data, which typically follows a different distribution than labeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. Our framework derives the class-balanced contrastive loss through Gaussian kernel density estimation. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using *reliable* and *smoothed* pseudo-labels. By progressively estimating the underlying label distribution and optimizing its alignment with model predictions, we tackle the diverse distribution of unlabeled data in real-world scenarios. Extensive experiments across multiple datasets with varying unlabeled data distributions demonstrate that CCL consistently outperforms prior state-of-the-art methods, achieving over 4% improvement on the ImageNet-127 dataset. The supplementary material includes the source code for reproducibility.

Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition
Zi-Hao Zhou 1.2+ Siyuan Fang Zi-Jing Zhou
Tong Wei
Yuanyu Wan Min-Ling Zhang
School of Computer Science and Engineering, Southeast University, Nanjing, China 2 Key Laboratory of Computer Network and Information Integration (Southeast University). Ministry of Education, China
3 Xiaaomi Inc., China
School of Software Technology, Zhejiang University, Ningbo, China {zhouzih, syfang, weit] edu. cn
Abstract
Long-tailed semi-supervee learning poses significant challenge in training mod- els with limited labeled data exhibiting long-tailed label distribution. Current state-of-the ohtt-of-the-art LTSSL approaches heavily rely on high-qualig pseudo-labels for large-scale unlabeled data. However, these methods often neglect the impact of rep- resentations learned by the neural network and struggle with real-world unlabeled data, which typically follows a different distribution than labeled data. This paper introduces novel probabilistic framework that unifies various recent proposals in long-tail learning. Our framework derives the blass-balanced contrastive loss through Gaussian kernel density estimation. We introduce continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels. By progressively estimating the underlying label distribution and optimizing its aligmment with model predictions, we tackle the di- verse distribution of unlabeled data in real-world scenarios. Extensive experiments across multiple datasets with varying unlabeled data distributions demonstrate that CCL consistently outperforms prior state-of-the-art methods, achieving over 4% improvement on the ImageNet-127 127 dataset. Our source code is available at
Introduction
Semi-supervised learning (SSL) serves powerful approach for improving the generalization capabilities of deep neural networks (DNNs) in scenarios where labeled data scarce [37] 59 23 The core concept of SSL methods typically involves assigning pseudo-labels to unlabeled data and utilizing those with high confidence for model training [ [T] However, many existing SSL algorithms presuppose balanced label distribution across both labeled and unlabeled datasets. In real-world applications, datasets commonly exhibit a long-tailed label distribution [65] 28 67 555, leading to biased pseudo-label generation favoring majority classes 440
68 24). This discrepancy
challenges the effectiveness of SSL algorithms in addressing real-world datasets.
The exploration of tailed semi-superrised learning (LTSSL) has gained momentum address the challenge of biased pscudo-label distribution arising from class imbalance in labeled and unlabeled data. Recent LTSSL approaches propose compensating for the learning of minority classes by
"quul contribution corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

distribution alignment [3 63], data rebalancing 12 38 and logit adjustment [64 43 to rectify the pseudo-label distribution. However, existing approaches often assume the equivalence of the unlabeled data distribution with the labeled data or rely on predefined anchor distributions to estimate the unlabeled data distribution (64 43 Furthermore, these methods primarily focus on correcting model outputs without delving into the role of representation learning in improving performance.
This paper explicitly introduces an approach to obtain effective representations for long-tail learning by adopting an Cnformation-theoree view of DNNs. We present a probabilistic framework that utilizes the deep variational information bottleneck method [ learn good representations and demonstrate its unification of recent long-tail learning proposals, such as logit adjustment [44 and balanced softmax [5]], through approximating the density of class-condie caans-ccndiiinnal distribution in different ways. Specifically, our framework encompasses class-balanced supervised contrastive learning (3 via Gaussian kernel density estimation. We extend this framework to address L.TSSI. by adapting the supervised contrastive loss to unlabeled data using "continuous pseudo-labels", derived from model predictions and propagated labels, to mitigate confirmation bias. To account for varying label distribution of unlabeled data, we progressively estimate the label distribution through moving average and adjust model predictions to align with the estimated distribution.
In summary, our contributions are as follows:
We propose a probabilistic framework which unifies many recent proposals in long-tail learning. Specifically, popular blassbbalanced contrastive learning methods can be seen as special cases of our framework when approximating the density using a Gaussian kernel.
2. We generalize the proposed framework to LTSSL and present a continuous contrastive learning method based on reliable and smoothed pseudo-labels to address confirmation bias and improve the quality of learned representations.
3. We conduct extensive experiments across several LTSSL datasets with diverse label distributions of unlabeled data. The results show that proposal substantially outperforms previous state-of- the-art methods.
Probabilistic Framework for Long-Tail Learning
In this section, we first introduce general framework for learning good representations. Then, we expand this framework to long-tail learning and illustrate how recent proposals can be regarded as specific instances of our framework through three ways for density approximation.
Problem setup of long-tail learning. We consider a C-class classification problem with instance space X' and target space y {l..... C}. Let P. and P: denote the source (training) and test distributions on ()).), respectively. We denote by P. and P. the corresponding probability density (or mass) functions. Given training dataset { ys ) i= where 2; R.d is the training sample and y is the ground-truth label.
2.1 Learning good representations from information theoretical view
In this subsection, we rethink one of the most popular approaches to deal with representation learning, ie.. contrastive learning. We derive many recent proposals this branch from an information- theoretic view. Let 7 denote the latent representation of X induced by the encoder enc(-) parameter- ized by O. From an information-theoree about the target Y. and minimally "memorizes" X. The information bottleneck [60] adopts mutual information (.)) to measure information between two variables. Thus, optimal can be obtained by maximizing the following objective:
view, an optimal representation Z is maximally informative
arg max 1(2, -) -1,
(I)
where 5 > 0 is a tradeoff parameter. The variational information bottleneck 1 solves the above objective by variational inference. Based on the definition of T(.), we can rewrite Eq. as:
P(y 2) P(y)
IP(=
x))
T(Z,Y) 8T(2, dydzPP(y 2) log
dzdaP(2 2) log P(z)
(2)
where we omit + for simplicity. Since P(y =) = dxP( / | =)F(v) x) is intractable Eq. ( let
Fly x) be variational approximation to Ply x) and considering that the Kullback-Leibler (KL)

---

divergence [L[ z), Pyy x)] is always positive, we have RHS of Eq. 2 's lower bounded:
P( a:) / (axdyd a T))((= log P(u 2) - dand2((z))P(2 x) log P(2)
(3)
Suppose second term of Eq. ( equals the KL divergence KL P( P(z)) Since P(z x)) and P(z) are normal distributions, can be rewritten as: -
encoder of the form P(z x)) (enc((:)) e and P((2) NN I). the
log E) 8 [[een(())]] where / is the dimension of 2. For deterministic model, 2 is almost unique for each æ, thus assuming ε is a small constant close to O. By integrating out P(z x) and discarding constant terms, maximizing Eq. can be approximated by minimizing:
-f -f dnP((a))))))))))) dyP(y x)) log Ply enc())) + ||eno(=)]]
In the following of this paper. we denote the output of enc(z) as (or ze for a particular sample x) for simplicity. Minimizing Eq. (4) is equivalent to minimizing the following objective in the distribution of test data on each x:
Σ kE(C]
= a log PY k + 0|
Notably, Eq. 55 can be seen general framework for learning good representations. If If, (Y) P.(Y), P. y x) can simply be substituted by the ground-truth labels of training samples. However, in long-tail tail learning, the class-probability function P. y x)) is different from that of the training data due to label distribution shift.
2.2 Probabilistic framework for long-tailed supervised learning
Since PY,Y- = æ) + (Y = y æ), we cannot directly solve Eq. (5). However, since long-tail learning typically assumes that PP (Y') uniform and we work with the label shift assumption, i.e., (2(x Y = y) = Pr(x Y = y), we can obtain P. y x)) by Bayes' theorem:
Y
P. PY x))
P2 = x) = P
(6)
EMErCi P, PY =
Throughout the paper, use the notation P(x Y y) to represent either (x Y y) or (x Y = y). In practice, ||= can be omitted in optimization because normalization is commonly adopted in deep learning. In long-tail learning, minimizing Eq. (5) equals to minimizing:
Σ KE|C
P.(Y = k x:) log P. = k l2).
(7)
P.(Y k))
According Jensen's inequality, Eq. 7 attains its minimum value if and only if P, k x)P, = k) x P. = k x) for k E [C. Hence, Eq. (7) can be replaced as follows:
P(Y = k)
J = Σ kE|Cj
P. = k x)) log IP k 2),
(8)
P.(YY k)
((((=))=)F((=>))) where P(Y y 22) Evercy (Y) and P(Y) is arbitrarily label distribution. Eq.
8
can be seen as an extension sample reweighting [22 and logit adjustment 44 using probabilistic
labels rather than discrete labels when P(Y) is specified as PY) and P. PY). respectively. Besides,
Eq. { presents a unified framework that consolidates existing long-tail learning methods by
estimating P(z Y =y= y) or P. = y z) in different ways. In the following, we discuss three ways to estimate these terms.
Explicitly specify (2 = Y = y) as a prior distribution such as vMF distribution Wrapped Cauchy Distribution 27
Method 1:
34 and
Method 2: Approximate Pz(z Y y) using learnable linear classifier. Let (w, 1
i- denote the
parameters of linear layer, which is followed by a softmax to obtain the normalized probability:
exp
Wy + by))
P( Y = v) x PY = y x) = Σ Exelcy exp (z ww + bb)
(9)
3

---

Table 1: A unified view of popular long-tail learning methods from framework. "_" means that this method does not involve this issue and "x" indicates that the method has not resolved the issue.
Method
Density estimation
Label distribution shift
Mini-batch computation of Eq. 10)
BALMS 5T
Linear layer
Reweighting
LA
44
Linear layer
Logit adjustment
BCL
Gaussian kernel
Reweighting
Class-wise center
GML
Gaussian kernel
Logit adjustment
Class-wiss queue
KCL 30
Gaussian kernel
Balanced resampling
PaCo
15
Gaussian kernel
Logit adjustment
Class-wise center Class-wise vMF distribution
Proco
20
Gaussian kernel
Logil adjustment
T-vMF 34
T-vMF distribution
Logit adjustment
WCDAS
27
Wrapped Cauchy distribution
Logit adjustment
Method 3: Approximate P(( Y y) via the Gaussian kernel. A new sample from class y should be closer to all samples within class y and away from samples from other classes. Using the expected similarity among all samples within the class measure distance, we derive:
Eg [N (Z Zxrr
P(z Y Y x I. =y 22) Exe(cy E BewP((y= [Ar (Zay 2m))
(10)
where r)(:.)) represents the similarity between two samples, when we use Gaussian kernel R(Zx; Rwr) exp( Zx #wr) and approximate expectation through empirical batch B = Ukec Bx. that is Egg Y=y) (z 2m/)] R [Byl EwEB, exp(z= =xr), Eq. (TOp can be instantiated as:
1,1-1 Ex'EB, exp (Z Zar)
Pi y 2) Exejo [B&] ExTES, exp (Zn 2xx)
(11)
Interestingly, we observe that Eq. 11 resembles class-balanced contrastive loss. In the appendix, we also show that the Gaussian kernel approximation is identical to conventional supervised contrastive learning if the training data are class-balanced.
Notably, to ensure the computability of E='-P(()==)) [<(= =xr) )] in Eq. (10 it is essential to ensure that samples are available from each class. Existing methods address this by class-wise queues, class-wise centers, class-wise vMF distribution, details of which are provided in the appendix.
Based on the above three density approximation methods, we find that many recent proposals in long-tail learning can be derived from our framework. In Table we summarize existing methods based on the way they estimate the density, tackle the training/test label shift, and guarantee computation of Eq. (10) in mini-batch training.
3 CCL: Continuous Contrastive Learning
In this section, we introduce the proposed algorithm CCL, which extends the class-balanced con- trastive learning presented in Eq. with Gaussian kernel estimation in Eq. [I to LTSSL.
3.1 Problem setup of loigetailed semi-supervised learning
Let Pt and Pu denote the joint distribution (X) for labeled data and unlabeled data, respectively. We denote by Pt and P., the corresponding probability density (or mass) functions. We possess labeled dataset {<, 1 of size N and unlabeled dataset 1 of size M, where o/, x E Rdd The proportion of labeled data from the entire dataset is = M4N. Denote the number of labeled data for class as N., we have N₁ > N2 > 2 No if the classes are sorted by cardinality in decreasing order. The imbalance ratio of labeled data is given by 7 No while the distribution of the label of the unlabeled data and its imbalance ratio The are unknown. The components of CCL include a feature extractor, two linear classifiers fa(),, fo(-) and contrastive feature projection head g(-).

---

3.2 Balanced classifier training with estimated class prior
We develop our method based on FixMatch [56] following previous works 48
64 and its objective
is: Eas L. + Lu where Eg is a traditional cross-entropy loss. For unlabeled data, the method operates by first generating pscudo-labels for unlabeled data using the model's predictions and selecting unlabeled data whose predicted maximum confidence is higher than a predefined threshold.
The consistency regularizer L. is then applied to two views of each selected sample.
Balanced FixMatch for LTSSL. First, since the labeled data follow long-tailed distribution, which denote x², Li needs to be adjusted by logit adjustment
44 via Eq.
P. (Y = 5 a : f6) T
Ei(w', v/) -log log Exe(c] F. (Y k a fb)
(12)
Second, FixMatch is prone to fit wrong pseudo-labels with high predictive confidence during training [6. However, since the unlabeled data label distribution is inaccessible, pseudo-labels generated by the classifier may be sub-optimal for its adherence to a uniform distribution. By Bayes' theorem, with given estimated unlabeled data label distribution π. the "post-adjusted" model outputs for sample x" can be formulated as:
P (Y = y x= fb) TH
Pu (Y =y|x";
(13)
Eke(oy P. (Y = h xx; fo) k
Given pseudo-label 7 = arg max P. (Y = k )m; where Am denotes the weak
data augmentation, Cu rewritten as:
P, (Y = | mp
-M((") log Exelcy P. (Y= = k A. (x
(14)
fo) m
where M(-) denotes the sample mask to select reliable pseudo-labels. We progressively update π using the exponential moving average (EMA) for each mini-batch by by = (1 a)
18 ExrEx Pa (Y = y x"; fa), where a a momentum updating parameter and B denotes an unlabeled data subset. Directly using confidence selection can lead to a selected B with poor calibration due to model overconfidence 44 45). Thus, the energy score [36] is adopted to filter reliable unlabeled data, which is defined F(x=) = - log Exeror efx(w)/T where T is the temperature and (xx denotes the logits of æ: We select reliable unlabeled data by M =
I(E(x") S () using predefined threshold Ç. and construct {ar x E B" A M= # 0} for the estimation of n..
Dual-branches training. Based on the observation that class-balanced training can be harmful to representation learning, previous works [38] 64 have utilized another branch of the network for standard training. In contrast to the balanced branch, the standard branch, denoted as f-(-), optimizes the cross-entropy loss without employing logit adjustment to fit the original training data distribution. We fuse the predictions of balanced and standard branches reduce the confirmation bias of pseudo-labels by:
1 Pols (Y = y x:") = P. (Y = :b)
1 P = x=:; fa) fa) 3
(15)
2 ExErcy F k x=; f.)
The rationale behind the equation is that the standard branch necessitates the elimination of imbal- anced label prior and then compensates for unlabeled label prior when predicting pseudo-labels,
which is achieved by defining म #" Overall, the classification loss Lels is the combination of losses for learning fx(-) and fo(-)
3.3 Continuous contrastive loss with reliable pseudo-labels
Apart from the classification loss and consistency regularizer, we aim to improve the quality of repre- sentations by extending the framework presented in Section 2 to LTSSL. To achieve the adaptation
5

---

of our framework, a primary obstacle must be addressed. The challenge arises from the unknown ground-truth label (Y y x)) for unlabeled data, which results the calculation of Eq. (8)
infeasible. We propose to utilize the continuous pseudolabel [Plls (Y = y x=) as derived from
the classifier in Eq. (15). Furthermore, (F-I=(() [R (Zm: Zn²)] in Eq. (10) can be extended to
unlabeled data and approximated by an empirical data subset B:
=e--)())) (1s (Za, Zmr)) Ex'eg 16 (Ze, Zwo Pello (Y x')
(16)
Ex'EB Pela (Y x')
Plugging Eq. (16) into Eq. (0),, we obtain the continuous escudo-label P/ y x"; B3) for x". Similar to Eq. 14 logit adjustment is used to handle label shift of unlabeled data by Bayes' theorem, and we can obtain:
Expl
Σ ( le x²) log P. = | =);
(17)
kE|C
where Pu(YY = x= Eneroy 2v So far, the generalized framework using continuous pseudo-labels for LTSSL is derived Eq. (77). the critical issue is how to filter out a reliable unlabeled data subset 1 such that the posterior estimation Fyl((Y = x) in Eq. (I6) calibrated. Similarly, directly using confidence selection may lead to model overconfidence. To mitigate the confirmation bias in pseudo-labels produced by the learned classifier, we propose using
the energy score for data selection to ensure model calibration [26]. Combining with labeled data,
the loss Cipl is obtained with B = {a x E B' V (x E B" A (M #0).
3.4 Continuous contrastive loss with smoothed pseudo-labels
To further mitigate the impact of inaccurate pseudo-labels fCcle(Y y x²"), derive com- plementary contrastive loss with smoothed pseudo-labels. Specifically, we propose aligning the representations of two views of sample by imposing the weak-strong consistency regularization:
Esp1= =- Σ PY = Aw log P (Y
k A. (x")).
(18)
kE(C
In this part, we aim to derive P(Y y x²) by propagating labels from nearby samples in the contrastive space. On the one hand, we take labeled data for B in Eq. (IT) and construct the posterior for unlabeled data, Logit adjustment is employed for tackling label shift of unlabeled data:
1 [18,1 Ex'EB, R (Z axx)) m
P(Y= y ae":B)
(19)
ExElc] [B. Ea'Es, (Z z K
Eq. (99 can be viewed as process of propagating labels from labeled data unlabeled data. On the other hand, we consider label propagation within unlabeled data, i.e., an unlabeled batch B is used to estimate P(Y- x"; B). Assuming there is a sufficient amount of unlabeled data, we have Ex P(Y y xx;; Bd) Re Ry. hence the posterior can be approximated as:
PY y x": B*) Ea'egu Ni (Zz", Zzr P (V = y x BB")
(20)
Exres= 1 (
Let IPY X; represent matrix stacked by [PY = x) x).... P(Y = C' x)] of from B, we can rewrite Eq. (20) in the form of matrix multiplication: PY X": = G P B"),", x(Fe,,
where G is similarity matrix and Gi Σs,es, R(,e It can be interpreted that similar samples possess similar labels. By aggregating the predictions of labeled data and unlabeled data with a fixed hyperparameter 3, we obtain:
P (Y
X = 3G (Y
+ x अ) (Y B":
(21)
P Y x = - (Y x B¹)
Subsequently. Eq.
21 can be plugged into Eq. 18 for calculating Expt. To sum up, the total
objective of CCL is:
Eroaal = Xr + (1 X1) Expt + XxCopl
(22)
where X and A2 are two hyperparameters.

---

Table 2: Test accuracy in consistent setting CIFARIO-LT and CIFAR datasets. The best results are in bold.
CIFARIO-LT
CIFARI0-LLT
7 7
7u 100
7 7
7v 150
7 Tr
7
7 72
%
N; 500
N; = 1500
N; 500
N; 1500
N; 50
N; 150
Ni 50
N; 130
Algoridhm
M, 4000
M, 3000
Mi - 4000
M1 3000
M1 400
300
Mj
M1 300
Supervised
47.3 1:0.5
61.91
40.33
58.29
29.6 0055
46.9 1:0.22
11.14
41.2
w/ LLA
53.3 4:0.4
#0.21
49.5 +:0.0
67.1 #:0.78
30.2
48.7 +0089
26.5 +1.31
44.1 #:0.2
FixMaath
56
67.8
1..5
62.9
1..4
00.5
56.5 20006
40.9
50.7
w/ DARP
33
+40.8
77.8 +-0663
67.2 +0.32
73.6 +0073
49.4 00.00
58. +00.4
+0.87
$2.2 +0.66
w/ CRRST+
63
76.3 #:0.6
78.1 +:0.42
67.5 #00.5
73.7 #:0.44
44.5 0.94
$7.4
40.1 +1128
$2.1 #0021
w/ DASO
43
76.0-+0.37
79.1 +-0.75
70.1 +1.81
75.1 +:0.7
49.8 00.24
59.2 +0035
43.6-+0.00
$2.9 +0042
FixMatch
15.35
80.0.
67.0
18.91
47.3
58.6 1:0.6
41.4-10993
53.4
w/ DARP 33
76.6 70.92
80.8 10.62
68.2 :0.94
11.13
50.5 0.778
59.9 =0.32
44.4 10.65
53.8 40.43
w/ CRRSTT+
76.7
80.57
70,9
1:071
10.55
10.55
52.3
w/ DASO [8
77.9 +:0888
82.5 10.08
70.1 1:668
79.0
50.7 00.51
60.6 =00.1
44.1 :0.611
55. =:0.2
HixMatch ABC 38
78.9
83.86
66.5 40.78
1:0.45
10.5
10.83
53.7
w/ DASO [8
80.1 1:116
83.4
70.6 1:.80
80.4 10056
50.2 20.62
60.0 1:0.2
44.5 11.25
55,3 1:0.3
FixMatch + ACR 64
81.6-10.19
84. 10.39
77.0=1.19
80.9 :0.22
51.1 =0.52
61.0
10.21
55.2
FixMatch 43
FixMatch + CPF
+0.9-
84.4 +-0.29
76.8 +00.53
82.3++0.34 +-0.34
50.3 00.34
89.8 +00166
43.8 +0.28
$5.6++0.15
63.5 10.39
Table 3: Test accuracy under inconsistent setting (y * Tr) on CIFARIO-LT and STL10-LT datasets.
> 100 for CIFARIO-LT, and 10 and 20 for STL10-LT dataset. The best results are in bold.
CIFARIO-LT (y) * %.))
STLI0-LT (%u =NAA)
7/4 (uniform)
Tu 1/100 (reversed)
"
N₁ = 500
N; 1500
N1 500
Ni 1500
Ni 150
N1 150
N; 150
Ni 150
Algorithm
Mi = 2000
M, 3000
M. 4000
M, 3000
100k
100k
M 100k
M 100%
FixMatch
73.0=3.81
62.5 1.15
71.8
12.32
12.7
4..6
64.0
w/ DARP
33
82.5
84.6 10.34
70.1 1:0.2
=0.0=0.93
66.9 1.66
75.6 1.0.5
59.9
72.3 10.60
63
80.8 0.33
+0.88
w/ CReST+
63
82.2
16.4200:2
11.99
72.9 2209-2.00
61.2
71.5 100.6
56.0=3.19
68.5 11.88
DASO
71.0
80.3
75.3
w/ ACR
64
92.1=0.15
90.5
85.0 85.0 :0099
0,17
77.1 10.24
83.0+0.32
75.1 =0.70
81.5 +0.25
CPE
92.3
93.3 10.21
84.8 10.88
0013
73.1 10.47
83.3
69.6
81.7 10.34
w/ CCL
93.1 10021
93.9 10.12
85.0 1:0.00
89.8 0.31
79.1 0043
84.8
77.1 20:33
83.1 10018
Experiments
In this section, we conducted comprehensive experiments to verify the effectiveness of the proposed continuous contrastive learning method (CCL) on CIFARIO-LT, CIFARI00-LT, STLI0-LT, and ImageNet-127 [29 [8] datasets. To simulate real-world unlabeled data, we tested our method on diverse label distributions of unlabeled data. Due limited space, we defer the detailed experimental settings to the appendix.
4.1 Results on CIFAR10/100-LT and STL10-LT
For consistent (71 nw) setting, results are presented Table 2 From the results, we can see that CCL consistently outperforms all comparison methods by large margin. In particular, CCL improves the previous ofate-of-the-rr approach ACR by 2.8% on average. This verifies that the representations learned by proposed contrastive losses more discriminative because both CCL and ACR utilize a dual-branch network.
For inconsistent (y * 7u) setting, we present the results in Table and Table Following prior works, we compare all methods using unlabeled data following uniform and reversed label distributions on CIFARI 100000-LT datasets. On the STL10-LT dataset, the underlying unlabeled data distribution is naturally inaccessible. general, CCL achieves the best results in settings. Particularly, CCL obtains an average performance gain of 1.6% over ACR without using predefined anchor distributions. The results indicate that our method is able to accurately estimate the unlabeled data distribution with calibrated labudo-labels.
7

---

4.2 Results on ImageNet-127
ImageNet- 127 naturally long-tailed dataset and has been used test LTSSL methods in the recent literature [22] 64). Following previous works, we downsample the original images to smaller sizes of x2x32 or 64 x pixels using the box method from the Pillow library and randomly select 10% training samples to construct the labeled data. Learning discriminative representations and balanced classifier is essential to achieve high performance. From the results in Table [ we can see that CCLachieves superior results for image sizes of x2x32 and 64x64. It outperforms ACR by 4.3% and 4.2% in test aceuracy.
Table 4: Test accuracy on LIFAR100-LE in uniform
Table 5: Test accuracy on ImageNet- The best results are in bold.
and reversed settings. The best results are in bold.
74 (uniform)
% 1/10 (reversed)
Algorithm
32 32
64 64
Algorithm
Ni 50
N; 150
Ni 150
FixMatch
29.7
42.3
Mi 400
Mi 300
Mi = 400
M, 300
42.5
FixMarch
40.5
58.1 =0.72
40.2
57.3 10.19
w/ DARP
35
39.7
51.0
w/ DARP
83
43.5 10:95
55.9
30.940.48
51.8 10.92
w/ CReST+
32.5
44.7
w/ CReST 66
43.5+40.0
59.2 0.255
39.0+1.111
56.4
w/ LA
63
40.9
55,9
w/ CReST+ [
43.6+1..0
58.7
39.1 +0.77
30.4
w/ CoSSL.
43.7
53,9
w/ DASO [8
53.9
61.8=009
$1.0
60.0
w/
ACR
57.9-10566
51.7 +0
w/ ACR [6]
57.2
63.6
w/ CCL
59.8
67.9 =0.70
#0.14
64.7 #0.22
4.3 Comprehensive evaluation of the proposed method
Understanding of balanced Fixmatch and dual-branch. First, balanced Fixmatch can be viewed as separate EM algorithm process 717 21]. where the E-step involves estimating suitable pseudo-labels for unlabeled data through カ and the M-step updates the model and obtains a new म' can be seen in Table 6 balanced Fixmatch achieves performance comparable to the recent state-of-ther the-art method ACR. Furthermore, dual-branch significantly enhances the performance of data under highly skewed long-tail distributions (consisten) setting), with averaged 1.5% improvement.
Table 6: Ablation studies of our proposed algorithm. "Con", "Uni", and "Rev" represent consistent, uniform, and reversed, respectively.
CIFARIO-LT
CIFAR100-LT
Dual-branch
Reliable PI.
Smoothed PI.
Energy mask
Con
Uni
Rev
Con
Uni
Rev
81.0
91.8
84.2
57.0
92.0
84.5
92.8
84.7
52.7
83.2
84.8
51.9
58.4
53.2
83.8
84.8
52.7
59.1
53.9
84.3
93.1
85.0
53.5
59.9
54.4
655
derter www.
039
wwy
[ 645
c35
025
125
201
RC
223
137
250
(a) consistent setting
(b) reversed setting
(c) consistent setting
(d) reversed setting
Figure 1: Comparison of class prior estimation error and ECE on CIFAR100-117
How to estimate relatively accurate म. Our method for estimating म. is equivalent to MLLS [53], which is EM process. Accurate estimation of म is only achievable when the model is calibrated (25). Since the confirmation bias is induced by self-training, using confidence selection may result overconfident but wrong pseudo-labels and hurt the calibration [45] 41 In contrast,
energy score leverages the probability density of the predictions, exhibiting reduced vulnerability to overconfidence [39]. Thus, we propose energy selection for a reliable unlabeled data subset on
which the model is calibrated, thereby enabling the accurate estimation of f... We use expected

---

calibration error [26] (ECE) to assess model calibration. The tail of the curve of Figure Ic and Id can be interpreted as overconfidence false pseudo-labels caused by self-training. As can be seen in Figure and lb the In distance between the true class prior of unlabeled data and Tu.. estimated from data subset selected using energy, is significantly smaller compared to when confidence is used for selection, inducing a more balanced classifier training.
Continuous contrastive learning with reliable pseudo-labels. We carried out comparative experiment by removing the continuous reliable pseudo-labels loss. The results reflect an averaged 0.8% drop on CIFARI0/100-LT, demonstrating its efficacy for learning high-quality representation. Moreover, we verified that the data subset filtered by energy selection obtains excellent model calibration. Figure Ind sdlshow energy achieves better calibration than confidence thresholding.
Continuous contrastive learning with smoothed pseudo-labels. Similarly, we conducted a compar- ative experiment by removing the continuous smoothed pseudo-lahels loss. As can be seen in Table [ the performance decreases in all three settings on CIFAR 10/1 10/100-LT datasets, showing the necessity for consistency regularization constraint for feature alignment within the contrastive learning space.
4.4 Results under more class distributions
02
Similar to ACR, to evaluate our method's ef- fectiveness under more imbalanced settings, we conducted further experiments on CIFAR 100- LT, maintaining a fixed ? = 20 and adjusting the imbalance ratio of the unlabeled data from consistent to reversed. We set N₁ 50 and M, = 400 (with Mc = 400 in the reversed scenario) and compared the results with ACR as shown in Figure [ The results demonstrate that our method consistently out-
10
ACR
d0 4) 30 1-38 lo 13 Indukass Uniaoaed
27 15
10
10
1100brre Pabu Ueuubad Eet
(a) CIFARIO-LT " 100 (b) CIFARI00-LT
20
Figure 2: Generalize to more realistic LTSSL settings for ACR and CCL CIFAR10/100-111 dataset in fixed n and various settings.
performs ACR in all scenarios.
5 Related Work
Long-tailed learning (LTL). Early strategies tackling LTL involve two aspects: resampling and reweighting. Resampling methods ( [ 8 [44] either undersample majority classes or oversample minority classes, which may result in information loss or overfitting. Reweighting methods [52 16
12 assign different weights for each class training sample. BBN [72 and Decoupling 31 claim
that re-balancing can negatively impact representation. They propose two-branch structure or a
two-stage paradigm address it. Logit adjustment methods 7
44 learn larger margins for minority
classes by obtaining optimal Bayesian classifiers. Recently, several methods [30] [5] 73 20 57 have
been proposed to improve the representation learning based supervised contrastive learning 32
Long-tailed semi-supervised learning (LTSSL). Most supervipe learning (SSL) methods use unlabeled data by assigning pseudo-labels to unlabeled data [37] 6 566
71 I0 or aligning
predictions of different views of the input by consistency regularization [59]. PAWS leverages
self-supervised representations derived from unlabeled data, and RoPAWS 46 further refines the model predictions using labeled data through kernel density estimation. However, most of these works assume a balanced class distribution of labeled and unlabeled data, which may be violated in real-world applications.
Recently, LTSSL has gained considerable attention due to its applicability in numerous real-life scenarios. Recent works mitigate pseudo-labels bias by distribution aligmment or label refinement [33
63
69
Some others focus on balanced classifier training to overcome long-tailed label distribution
38
22
66). Regrettably, these methods simply assume an identical long-tailed distribution for
labeled and unlabeled data, which may still be unrealistic. Considering the unknown unlabeled data distribution, which can be mismatched with the labeled distribution, DASO
48 mixes the outputs of
linear and semantic classifiers to improve the quality of pseudo-labels. ACR consistency regularization or train multiple expert branches based on predefined anchor distributions. However, how to improve representation learning in LTSSL is ignored in most existing works.
664 and CPE 43 refine

---

6 Conclusion
This paper presents a probabilistic framework that unifies many recent methods in long-tail learning. Our framework is equivalent to supervised contrastive learning when approximating the class- conditional function using the Gaussian kernel. We further extend the contrastive learning objective to LTSSL based continuous pseudo-labels to improve the learned representations. We utilize both reliable pseudo-labels generated by the model and smoothed pseudo-labels propagated from nearby samples to mitigate confirmation bias. Extensive experiments demonstrate that our proposed method achieves state-of-thh-aa performance in all settings. We hope that our work can motivate more research for LTSSL from the perspective of representation learning.
Acknowledgments and Disclosure of Funding
This work was supported by the National Science Foundation of China (62206049, 62225602). and the Big Data Computing Center of Southeast University. We would like to thank anonymous reviewers for their constructive suggestions.
References
[1] Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy, Deep variational information bottleneck. International Conference on Learning Representations, 2017.
[2] Shaden Alshammari, Yu-Xiong Wang, Deva Ramanan, and Shu Kong. Long-tailed recognition via weight balancing. In IEFE Conference on Computer Vision and Pattern Recognition, pages 6897-6907, 2022.
[3] Eric Arazo, Diego Ortego, Paul Albert, Noel O'Connor and Kevin McGuinness. Pseudo-labeling and confirmation bias deep cemi-supervised learning. In International Joint Conference on Neural Networks (IJCNN), pages 1-8, 2020.
[4] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Ballas, and Michael Rabbat. Semi-supervised learning of visual features by non-parametricall predicting view assignments with support samples. In IEEE Conference on Computer Vision and Pattern Recognition. pages 8452, 2021.
[5] Colin Bellinger, Roberto Corizzo, and Nathalie Japkowicz. Calibrated resampling for imbalanced and long-tails in deep learning. Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11-1, 2021, Proceedings 24, pages 242-252. Springer, 2021.
[6] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kibyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. ArXiv Preprint ArXiv: 1911,99785, 2019.
[7] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with lahel-distriuution-awaa margin loss. In Advances Neural Information Processing Systems, volume 32, pages 155771578, 2019.
[8] Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Animashree Anandkumar, Sanja Fidler, Jose M Alvarez. Image-level or object-level? a tale of two resampling strategics for long taig-t detection. In International Conference on Machine Learning, pages 1463- 1472. PMLR, 2021.
[9] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357, 2002.
[10] Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang. Bernt Schiele, Xing Xie, Bhiksha Raj. and Marios Savvides. Softmatch: Addressing the quantity- quality trade-off in semi-supervised leaming, ArXiv Preprint ArXiv:2301 10921, 2023.
[11] Ting Chen, Simon Korbllihh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning visual representations. International Conference on Machine Learning, volume 119, pages 1597-1607, 2020.
[12] Xiaohua Chen, Yucan Zhou, Dayan Wu, Chule Yang, Bo Li. Qinghua Hu, and Weiping Wang. Area: adaptive reweighting via effective area for long-tailed classification. In IEEE International Conference on Computer Vision, pages 19277-19287, 2023.
10

---

[13] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings the Fourteenth International Conference on Artificial Intelligence and Statisticx, pages 213-223. JMLR Workshop and Conference Proceedings. 2011.
[14] Ekin D Cubuk, Barett Zoph, Jonathon Shlens, and Quoe V Le. Randaugment: Practical automated data augmentation with a reduced search space. Advances in Neural Information Processing Systems, 2020.
[15] Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In IEEE International Conference on Computer Vision, pages 715-724, 2021.
[16] Yin Cui, Menglin Jia, Tsung-Yi Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. IEEE Conference on Computer Vision and Puttern Recognition, pages 2019.
[17] AP Dempter. Maximum likelihond from incomplete data via the em algorithm. Journal of Royal Statistical Society, 39:1-22, 1977.
[18] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hierarchical image database. In JEEE Conference on Computer Vision and Puttern Recognition, pages 248-255, 2009.
[19] Terrance DeVries and Graham Taylor. Improved regularization convolutional neural networks with cutout. ArXiv Preprint ArKiv:1708.0455: 2017.
[20] Chaoqun Du, Yulin Wang. Shiji Song, and Gao Huang. Probabilistic contrastive learning for long-tailed visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[21] Marthinus Christoffel Du Plessis and Masashi Sugiyama. Semi-supervised learning of class balance under class-prior change by distribution matching. Neural Networks, 50:1 10-119, 2014.
[22] Yue Fan, Dengxin Dai, Anna Kukleva, and Bernt Schiele. Cossl: Co-learning of representation and classifier for imbalanced supi-super learning. In IEFE Conference on Computer Vision and Pattern Recognition, pages 14574-14584, 2022.
[23] Kai Gan and Tong Wei. Erasing the bias: Fine-tuning foundation models for supe-super learning. In Proceedings of the 41st International Conference on Machine Learning, pages 14453-14470, 2024.
[24] Kai Gan, Tong Wei, and Min-L Ling Zhang. Boosting consistency in dual training for long-tailed semi- supervised learning. arXiv preprint arXiv:2406. 13187, 2024.
[25] Saurabh Garg, Yifan Wu, Sivaraman Balakrishuan, and Zachary Lipton. A unified view of label shift estimation. Advances in Neural Information Processing Systems, 33:3290-3300, 2020.
[26] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference Machine Learning, pages 1321-1330. PMLR, 2017.
[27] Boran Han. Wrapped cauchy distributed angular softmax for long-tailed visual recognition. In International Conference on Machine Learning, pages 12368 -13888 PMLR, 2023.
[28] Youngkyu Hong, Scungju Han, Kwanghee Choi, Scokjun Sco, Beomsu Kim, and Buru Chang. Disentan- gling label distribution for long-tailed visual recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 6626 6636, June 2021.
[29] Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. What makes imagenet good for transfer learning? ArXiv Preprint ArXiv:1608 08614, 2016.
[30] Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng. Exploring balanced feature spaces for representation learning. International Conference on Learning Representations, 2020.
[31] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations, 2020.
[32] Prannay Khosla, Piotr Teterwak. Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola. Aaron Maschinot. Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33: 1331166111867,, 2020.
[33] Jaehyung Kim, Youngbum Hur, Sejun Park, Eunho Yang. Sung Ju Hwang, and Jinwoo Shin. Distribu- tion aligning refinery of pseudo-label for imbalanced semi- supervupe learning. In Advances in Neural Information Processing Systems, 2020.
11

---

[34] Takumi Kobayashi. T-vmf similarity for regularizing intra-class feature distribution. In IEEE Conference Computer Vision and Puttern Recognition. pages 6616-6625, 2021.
[35] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[36] Yann LcCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujic Huang. A tutorial on energy- energy-based learning. Predicting Structured Data, 1(0), 2006.
[37] Dong-Hyun Lee. Pscudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop On Challenges In Representation Learning, ICMI., 2013.
[38] Hyuck Lec, Scungjac Shin, and Hecyoung Kim. Abc: Auxiliary balanced classifier for class- imbalanced semi-supervised learning. Advances in Neural Information Processing Systems, 34:7082-7094, 2021.
[39] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in Neural Information Processing Systems, 33:21464-21 21444,,2215555- 2020.
[40] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X Yu. Large-scale long-tailed recognition in open world. In IEER Conference on Computer Vision and Pattern Recognition, pages 2537-2546, 2019.
[41] Charlotte Loh, Rumen Dangovski, Shivchander Sudalairaj. Scungwook Han, Ligong Han, Leonid Karlinsky, Marin Soljacic, and Akash Srivastava. On the importance of calibration in supervised learning, ArXiv Preprint ArXiv:22/0004783, 2022.
[42] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. ArXiv Preprint ArXiv: 03983, 2016.
[43] Chengcheng Ma, Ismail Elezi, Jiankang Deng, Weiming Dong, and Changsheng Xu. Three heads are better than one: Complementary experts for long-tailed semi-supervised learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14229-14237, 2024.
[44] Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In International Conference on Learning Representations, 2021.
[45] Shambhavi Mishra, Balamurali Murugesan, Ismail Ben Ayed, Marco Pedersoli, and Jose Dolz. Do not trust what you trust: Miscalibration in semi-supervised learning. ArXiv Preprint ArXiv:2403. 15567, 2024.
[46] Sangwoo Mo, Chyi Su, Chih- Cho Ma. Mido Assran, Ishan Misra, Licheng Yu, and Sean Bell. Ropaws: Robust semi-supervised representation learning from uncurated data. ArXiv Preprint ArXhv:2302 14483, 2023.
[47] Yurii Nesterov. A method of solving convex programming problem with convergence rate o(1/k?). Doklady Akademii Nauk SSSR, 269:543, 1983.
[48] Youngtaek Oh, Dong-Jin Jin Kim, and In So Kweon. Daso: Distribution- aware semantics-oriented pseudo- label for imbalanced semi-supervised learning. In IEFF Conference on Computer Vision and Pattern Recognition, pages 9786-9796 2022.
[49] Boris Polyak. Some methods of speeding up the convergence of iteration methods. Ussr Computational Mathematics and Mathematical Physics, 4:1-17, 1964.
[50] Harsh Rangwani, Sumukh Aithal, Mayank Mishra, et al. Escaping saddle points for effective general- ization on class- dlass-imbalanced data. Advances in Neural Information Processing Systems, 35:22791-22805, 2022.
[51] Jiawei Ren, Cunjun Yu, shunan sheng. Xiao Ma, Haiyu Zhao, Shuai Yi, and hongsheng Li. Balanced mcta-softmax for long-tailed visual recognition. In Advonces in Neural Information Processing Systems, volume 33, pages 4175-4188 86, 2020.
[52] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learnnng to reweight examples for robust deep learning. In Internattional Conference on Machine Learning, pages 4334-4343, PMLR, 2018.
[53] Marco Sacrens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of classifier to new a priori probabilities: a simple procedure. Neural Computation, 14(1):21-41, 2002.
[54] Jiang-Xin Xin Shi, Tong Wei, Yuke Xiang, and Yu-Feng Li. How re- sampling helps for long-tail learning? Advances in Neural Information Processing Systems, 011775669-76887 2023.
12

---

[55] Kang-Xin Shi. Tong Wei, Zhi Zhou, Jie-Jing Shao, Yan Han, and Yu-Feng Li. Long-tail learning with foundation model: Heavy fine-tuning hurts. In Proceedings of the 41st International Conference on Machine Learning, pages 45014 2024.
[56] Kihyuk Sohn, David Berthelot. Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang. and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems, 2020.
[57] Min-Koof Suh and Seung- Woo Seo. Long-tailor recognition by mutual information maximization between latent features and ground-trute labels. In International Conference on Machine Learning, pages 32770- 32782. PMLR, 2023.
[58] Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning, In International Conference on Machine Learning, pages 1139-1145 PMLR, 2013.
[59] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems, volume 30, pages 1195-1204, 2017.
consistency
[60] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. ArXiv Preprint Physics/0004057, 2000.
[61] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, (11), 2008.
[62] Xudong Wang, Zhirong Wu, Long Lian, and Stella X Yu. Debiased learning from naturally imbalanced pseudo-labels for zero-shot and supersuper learning. ArXiv Preprint wXXv:2201.01-999 2022.
[63] Chen Wei, Kihyuk Sohn, Clayton Mellina, Alan Yuille, and Fan Yang. Crest: class-rebalnncing self- training framework for imbalanced semi- superviie learning. IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[64] Tong Wei and Kai Gan. Towards realistic long- sumi-super leamingg Consistency is all you need. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3469-3478, 2023.
[65] Tong Wei and Yu- Yu-Feng Li. Does tail label help for large- scale multi Mabli- learning? IEEE transactions on neural networks and learning systems, 31(7):2315 2324, 2019.
[66] Tong Wei, Qian- Yu Liu, Jiang-Xin Shi. Wei-Wei Tu, and Lan-Zhe Guo. Transfer and share: semi-supervised learning from long-tailed tailed data. Machine Learning, pages 1-18, 2022.
[67] Tong Wei, Zhen Mao. Zi-Hao Zhou, Yuanyu Wan, and Min- Zhang. Learning label shift correction for test-agnostic long- tailed recognition. In Proceedings of the 41st International Conference on Machine Learning, pages 5261 1-52631, 2024.
[68] Tong Wei, Jiang- Xin Shi, Wei- Wei Tu, and Yu-Feng Li. Robust long-tailed learning under label noise. ArXiv Preprint 210i 1/56, 2021.
[69] Zhuoran Yu, Yin Li, and Yong Jae Lee. Inpl: Pseudo-labeling the inliers first for imbalanced semi- supervised lcarning. ArXiv Preprint ArXiv:2303.07200 2023.
[70] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference, pages 37.1-87.12, 2016.
[71] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervii learning with curriculum pscudo labeling. Advances in Neural Information Processing Systems, 34:18408 18419, 2021.
[72] Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. Bbu: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9719-9728, 2020.
[73] Jianggang Zhu, Zheng Wang. Jingjing Chen, Yi-Ping Phoebe Chen, and Yu-Gang Jiang. Balanced contrastive learning for toiged visual recognition. In IEEE Conference on Computer Vision and Puttern Recognition, pages 6908-6917 2022.
13

---

A Comparison with Supervised Contrastive Learning
As we have derived in Eq. (10) and use the Gaussian kernel density estimation Eq. (1). if simply assume the data are class-balanced, it simplifies to:
exp (Zx =xx) PP(Y y) P(Y = =y= = ( Ew' Ex'eBu\e May BBu\
Eke(cl (BBi] EovEr, exp (Ex zx/) P(Y k) Ee'EBAMat exp (Z (ao))
(23)
E Ze'EB, (BB (aa) exp (z 2wo
EKE(O) Ex'EB, exp (Z zw²)
Ex'EB exp (Z 2w²
The loss of supervised contrastive learning has two forms, i.e., Lin and Zout which distinguished by the position of summation over positive samples the (.g. Thus, get:
Dan (x. v) = 1 log Ex'EB EB, (ax) exp (=x Fxx)
Ew'er exp (Z Exx)
& log Byl-1 1 E re' (B,((a) exp (E Zxo)
(24)
Ex' EB exp (Zx Raoe)
Jensen
log exp (Zx axr)
Pout (æ) y).
By
E exp (Zn Zx!)
a'EB, (a)
which is consistent with the original paper's derivation.
B Analysis of Existing Long-Tail Learning Methods
As we have derived before, here we dive deep into the analysis of existing methods and demonstrate that they all belong to our unified framework.
B.1 Discussion of Gaussian kernel estimation
Balanced contrastive learning 73 (BCL) was proposed solve long-tailed problems with improved supervised contrastive learning. BCL involves two key techniques: class averaging and class complement. BCL averages out the contributions of different classes in the denominator to ensure class equal distribution, meanwhile, it takes nonlinear mapping of the classifier parameters to form learnable class center to ensure that every class has at least one sample in a mini-batch:
(II")()) [Ks (Zz, 2x))
Σ exp (Zx 2x
(25)
B
2 2'EBLU(cs)
However, BCL ignores the problem of the original long-tailed distribution in the training dataset, necessitating reweighting operation. Let æ denote the class prior, we have:
ExeB, exp (Zxe =xx))
Eael(ar,
(r, u) log
B
(26)
Ty Eke(CT |
E Ee'eB.ueas exp (Zu Zxx)
Gaussian mixture likelihood loss [57 (GML) initiates its approach from the concept of mutual in- formation, employing the Gaussian kernel. GML integrates contrastive learning with logit adjustment enhance its performance.
1,, Exs EB, EB, (Q.A(m) exp (Z 2x') Ty
Egmi(x 2) = log
(27)
Σ [8 +I Ex',, exp (Z Zwo) TT
proposes a class-wise queue Q = Up_ Qu to ensure a balanced class occurrence in mini-batch.
However, it does not propose unified framework, and the understanding not deep enough.
Some other methods: this section, we compare some other methods that use contrastive learning and analyze their mistakes.
14

---

K-positive contrastive learning 30 (KCL) is based on supervised contrastive learning, using K samples of the same class in the molecule ensure balanced feature space. Putting in our unified framework, we can obtain the following:
Bei [K (Zm: mx
~
Σ
exp (Z Zar)
(28)
K
BB' , [8' [8'|
K Σx BB' [B 8y.18' |8 |-K exp (Z #xs
E (xx, (a. y) y)= - log
Σ EB EB exp (zx *xe)
(29)
Jensen
1
E
log exp (Zx =ao)
Ciut (ar v)
K
Σx exp
(Zo= zwx)
"B""
Compared with the above, it regrettably still uses the same unprocessed denominator of SCL and cannot ensure that each mini-batch contains an equal number of samples from each class, nor that each class contributes equally, rendering it suboptimal for long-tailed learning. In addition, Eq. 28 equivalent to the resampling technique.
Parametric contrastive learning [15] (PaCo) introduces set of parametric class-wise learnable centers and uses adjustable parameters a for loss with respect to them. Our framework is used to derive its original form. First, we can obtain:
B
Eg ((( [R (Zw: Zwr)]
Σ
exp (Z= 2xx) (1 exp exp (Zn ck)
(30)
[B
æ'EB,
where B is fixed coefficient. Then, we can derive the PaCo loss as follows.
Epcco (I,
3 Le'EBAlaas Ma)
E
exp (Ex 2x') + 3) exp (zx cy)) P(Y = y)
[8,
log
Exe(c]
[B& Daret, exp (zn zxx) + (1 - 3) exp (Zo cx) IP(Y k)) (31)
a Exx exp (z )w)) + exp (z cy) P(Y =
2 3--log log
ExE(C] (a E. exp (zz 2w/) + exp (Z ck)) P(Y
where 8P(Y=y) (1-B) B,
B. PaCo explicitly uses the parametric class center to ensure balanced class occurrence in a mini-batch. However, ignores the class-equal contribution loss computation,
which can still be suboptimal.
(Proco) simply assumes that the normalized features in contrastive learning follows a mixture of von Mises-Fisher (vMF) distributions on a unit ball, its probability density function has the following form:
Probabilistic contrastive learning 20
fo (zi Myy
ePn
Cp (Kyy
(32)
Cp(p) (2T)p/2I(n I(n/2 (p) 02/2-1 (1)22-1)(2)= =
1
2k+p/2-1
k (p/2
R=
where parameters (Ht, Pg) need to be estimated. The advantage is that Proco can estimate (Hy Pv) using online mini-batch, such that it can be derived closed form of expected contrastive loss. Despite the assumed vMF distribution, it still uses Gaussian kernel estimation:
Cp Cp(A(za,U))
Eg Ex [Nc (Zu; zx') zs E B [Kc (Ze, #x
(33)
Op (Py)
where A(zz, y) represents a fixed function related to 2x, Myy Py- Thus, the loss objective Proco is:
Cp(((za.v)) CplPs)
Eprono (ar, y) log P. (Y y
log
(34)
Σ
(=(
Cp(pz)
However, the assumed distribution of Proco stills needs to be estimated (Fys Pyy using EMA of different batches, which is essentially a similar approach to the momentum queue used in GML. There still exist problems of inconsistent distribution of different mini-batches, and the strong
assumption about (z Y y) which may not follow the vMF distribution.
15

---

B.2
Discussion of explicitly assigning specified distribution
14,
Previous work mainly focused on P(z Y = y) through modeling cos 0g =
|| |=
T-vMF 34 models cos
cos0g Og as vMF distribution:
(cos0y: (g) (py) e.v (0.0008 006 ((
1
('A +C -.yl
(35)
Due to the inherent properties of the exponential function, the posterior quickly converges to despite a large || /4,, |- Such a compact measuring function might hamper model training, since tailed samples hardly enjoy back-prec pooepppppaaation due vanishing gradient. To overcome this problem, T-vMF introduces a family of modeling methods as follows:
1 1- (1 (1-9)) 11=-11y1] 11yl Sg (cos Oyi Pv) = (p,))
C(")))) (10)):)(1==~=ll:ppo) (36)
where sq [|| = Myl Pp)
[1 [1- (1-1(11)) ||2 Myll --. Technically, T-vMF models PY
y x) as follows:
P. =y12) Ene(cr ePPolR)14))
(37)
Pg.p (z, (2) = 8q s/[11= || - 14y 11 8q (2(2.)) -1E1-1,1]
(38)
(0 p) (2,
Thus, the loss objective of T-vMF is:
(n- v) log P. y 2) -logg TyePen(ZitM))
(39)
Exe(c) TTeFF.
WCDAS (27] The accuracy of the posterior approximation is crucial factor influencing the method's performance. Unlike t-vMF, which directly specifies the Pp Y = y) with fixed parameters,
WCDAS seeks optimal parametric probability density function of P(z y)-
Modeling cos By as the Wrapped Cauchy distribution with trainable parametric 0 1......, Ocl.
WCDAS models PY = y x) as follows:
0z f (cos By: 0y) 211(1 + 03 20y cos b,))
(40)
P.(Y 112) 2) ecco 0x)
(41)
Ekeley el (cos Pcide))
Thus, the loss objective of WCDAS is:
T e (cos
ZWCDAS( (x, v) log Pa = 2)) =- -log ExE T
(42)
C Mathematical Notations
To ensure clarity and precision throughout this paper, we provide a comprehensive list and definitions of the key mathematical symbols and terms used in this section. Each symbol is defined with its specific meaning and context to ensure consistency and accuracy across the document.
D Illustration of The Proposed Algorithm
D.1 Illustration of The Overall Proposed Algorithm
CCL consists of two parts: the classification part and the contrastive learning part. The classification part uses logit rectification of the classifier by class prior estimated with a dual-branch. For the contrastive learning part, the energy score is used to select reliable unlabeled data which are merged with labeled data for continuous contrastive loss to ensure calibration. Besides, information of labeled data and unlabeled are used in a decoupled manner while maintaining the constraints of aligning feature in the contrastive learning space, thereby forming smoothed contrastive loss.
16

---

Table 7: List of common mathematical symbols used in this paper.
Symbol
Definition
X R RR. C [C]
Feature space and target space
P.. Pt
Joint distribution of training and test data in LTL, respectively Joint distribution labeled and unlabeled data in LTSSL, respectively Training set in LTL
Pu
{Tos, wi)
(2), v{) 1 ~P
im1 PN
Labeled training set in LTSSL.
{ay)) j=1 ~PM
Unlabeled training set in LTSSLL
[NI,.... Nc}
Number of samples
each class in labeled data
(M,, Mc}
Number of samples for each class in unlabeled data True class prior of labeled data and estimated one of unlabeled data, respectively Random variable of imput, target, and latent feature space, respectively
nτ, π
X,Y,
P,P
Probability density function and its variational approximation, respectively Probability density function of training distribution in LTL Probability density function of test distribution (with uniform label distribution) Probability density function of L, U, respectively
P.
Pt
Pt,Pa
pels
Estimated posterior of unlabeled data with dual-branch Encoder that maps X to Z Parameters of the encoder Mutual information of two random variables Similarity between two latent features
enc(-)
I(-)
M.ME
Sample mask of confidence and energy score, respectively Standard branch and balanced branch, respectively Projection head
f.0...))
9(-)
Data mini-batch
B"
Mini batch of labeled and unlabeled data, respectively
Continous reliable PL
Dual-branch classifier
Labeled data
Energy Mask
Strong Data augmentation
Max
EMA
4pl
Festure Extractor
A%
Feature Extractor
Logit Fusion
Label Propagation
x
Continous smoathed PL
Unlabeled ditta
Projection Head
Forward of
Forward A!!
Forward A,
Logit adjustment
Figure 3: Illustration of the proposed framework.
17

---

Algorithm Continous Contrastive Learning (CCL)
labeled dataset and unlabeled dataset, standard branch ʃa and balanced branch fú. projection head g. class prior of labeled dataset H,, estimated unlabeled dataset class distribution Tus number of iterations in each epoch T' scaling parameter 7.
Input:
Require: Weak augmentation (-)(), strong augmentation A.(+), loss weight coefficients λ1 A2-
for = to T do
[(1) ) 1811 ==1 < Sample a batch of labeled data
{a] Jj=1 + Sample a batch unlabeled data
18"1
# Balanced classifier training with estimated class prior
Calculate pseudo label y arg maXke[C] Pola (Y k (ao())))) via Eq. 15
Calculate classification loss Eets
Update estimated class distribution म" via EMA by energy score selection
Continuous contrastive loss with reliable pseudo-labels
Merge reliable unlabeled data selected based on energy score with labeled data construct B
Calculate loss Enp via Eq. (17) with B
Continuous contrastive loss with smoothed pseudo-labels
Calculate G using unlabeled data
Compute posterior I' (Y A (a")) and P (Y AA(x")) using Eq. 21
Calculate consistency regularization loss Capt via Eq. (18)
Total Objective
Evotal Cus + (1 A1) Enp + >2
Updatef, and fr. and based on V Liotll using SGD
end for
D.2 Illustration of Reliable Pseudo-labels and Smoothed Pseudo-labels
Continous smoothed PL
Labeled data
Continous roliable
Construction
A
J, Logit
Energy score
E(N
E(x) &?
SirongWWeak Data augmentation
Label Propagation
Feeture Extractor
Ac
A
Feature Extracter
Max
Unlabeled
Projection D Forward xf
Forward AA
Forward of AA
Figure 4: Illustration of reliable lseudo-labels and smoothed pseudo- labels in CCL. To generalize the framework in Section 2 LTSSL., the main challenge is unknown (Y
xx"), where " denotes a sample in the unlabeled dataset. We first approximate (Y, = =y xx)) using the output of the calibrated and integrated classifier and energy score to filter reliable unlabeled data, ensuring the model's calibration, which constitutes the reliable pscudo-labels subset. Furthermore, we can also estimate the unknown (Y. = x²) by leveraging the smoothness assumption. Specifically, we construct smoothed pscudo-labels by propagating labels from nearby samples using the Gaussian kernel density estimation.
Pseudo Code of The Proposed Algorithm
Algorithm summmrrizes the whole framework of the proposed CCL, which is clearly divided into three components: balanced classifier, continuous contrastive learning with reliable and smoothed pseudo-labels, respectively.
18

---

F Experimental Setup
Training datasets. Our experimental analysis uses a variety of commonly adopted SSL datasets, including CIFAR 10-LT [35], CIFARI00-LT (35, STLIO-LT [3], and ImageNet-127 22 in various ratios of class imbalance "y and various ratios of the amount of labeled data 7). To create imbalanced versions of these datasets, consider the long- tailed imbalance where the frequency of data points decreases exponentially from the largest to the smallest class, that is, the number of samples in class c
is Ne N₁ X- C-T for labeled data and Mc- M1 X-y c-T for unlabeled data. We use Cutout [9] and Randaugment [14] for strong augmentation on unlabeled data, and we use SimAugtent [[]] on labeled data for continuous contrastive loss with smoothed pseudo-labels. Like recent LTSSL works, we consider three class distribution patterns for unlabeled data, namely, consistent, uniform, and reversed settings.
CIFAR1O-LT: Following ACR 64 we conduct experiments with all comparison methods in settings where N1 = 500, M1 = 4000 and N₁ = 1500, M, = 3000. We adopt imbalance ratios of n Tn = 100 and 1 Tr. = 1 50 for consistent settings, while for uniform and reversed settings, we use "/t = 100, 7/- = and ?/1 = 100, T = 1/100, respectively. CIFAR100-LT: Like CIFARIO 10-LT, we perform experiments in configurations where N1 = M0, 400 and N1 = 150, MM = 300. For the consistent settings, we use imbalance ratios of 72- = The = 10 and 7 = 7/ = 20. In contrast, for the uniform and reversed settings, we apply 71- = 10, Y/a = 1 and ? = 10, 7he = 1/10, respectively.
Given the absence of ground-truth labels for the unlabeled data of the STLIO dataset, we manage the experiments by adjusting the imbalance ratio of the labeled data.
STL10-LT:
Following ACR, we consider the labeled imbalance ratio of 71 = 10 or " 20.
ImageNet-127:
ImageNet127 was first introduced in earlier research [29] and utilized in
LTSSL by CReST. This dataset consolidates the 1000 classes [18) from ImageNet into 127 classes, grouping them according to the WordNet hierarchy. For ImageNet- 127, we follow
the original setting in CoSSL [2
( = Tu = 286).
Implementation details. Our experimental configuration largely aligns with Fixmatch [56] and ACR 664 Specifically, we apply the Wide ResNet-28-2 (70) architecture to implement our method on the
CIFARTO-LT; CIFAR 100-LT and STL10-LT datasets; and ResNet-50 on ImageNet- 127. We adopt the
common training paradigm that the network is trained with standard SGD [47 49 38) for 500 epochs, where each epoch consists of 500 mini-batches, and batch size of 64 for both labeled and unlabeled data. We use cosine learning rate decay (42] where the initial rate is 0.03, we set = 2.0 for logit adjustment on all datasets, except for ImageNet-127 127, where T = 0.1. We set the temperature T = 1 and the threshold < -8.75 for the energy score following [69] and we set \ = 0.7, A2 1.0 on SIFAR10/100-LT and A = 0.7, Ag = 1.5 on STL10-LT and ImageNet- 127 datasets for the final loss. We set 3 = 0.2 in Eq. 21 for smoothed pseudo-labett loss. To show the effectiveness of our approach, we perform a comparative analysis with several existing LTSSL algorithms, including DARP 33 CReST [63] DASO 48), ABC [38], and TRAS (66). We also consider the most popular 43 The performance evaluation of these methods is based
LTSSL methods ACR 66 and CPE on the top-1 1 accuracy metric the test set. We present the mean and standard deviation of the results from three independent runs for each method. In addition, our method is implemented using the PyTorch library and experimented on an NVIDIA RTX A6000 (48 GB VRAM) with an Intel Platinum 8260 (CPU, 2.30GHz, 220 GB RAM).
G In-depth Analysis
G.1 Sensitive analysis of hyperparameters
As outlined figure 5a CCL is relatively robust to the fluctuation of ,3 from 0.1 to 0.4. However, when 3 is set to O, the propagation within unlabeled data is ignored, resulting in a performance decrease of about 0.9%. Thus, the necessity of using Eq. (2I) is verified. In addition, figures 5b| and 5c both demonstrate that CCL is robust to loss weighting coefficients X₁ and X2 within certain range.
'The matrix inversion operation is implemented using torch. inverse O, which utilizes the fast singular value decomposition. The time complexity is (())))).
19

---

However, it is worth noting that when Y = 1.0, the proposed continuous reliable pseudo-labels labels loss ignored, resulting in performance degradation.
Sensitivity CCL on
Sensitivity CCL,
Sensitivity of CCL on
54-
accunayy
accunscy
accurecc
53
53-
53
52-
522
52-
Topo
51-
51
51
50
00
0.
03
64
50
64
0.1
11.
08
89
1.0
50
07
0M
100
12
15
Value of in SPL loss
Vallue of totatl loss
Vallue of 2. in total loss
(a) Sensitivity of CCL on 8
(b) Sensitivity of CCL on A
(c) Sensitivity of CCL on A2
Figure 5: Sensitive analysis of hyperparameters under consi setting of CIFARIOO-LT.
G.2 Time and Space Complexity Analyses
In this section, we conduct analyses of the time and space complexity of the proposed method. Denote feature space dimension D, batch size B and the number of classes C, the time and space complexity of CCL can be seen in Table g
and analysis details as follows.
For time complexity, calculating Lrpt requires two main parts, calculating kernel similarities by multiplying two matrix of size x D and x B with complexity O [² D), and calculating Ba-- [K (Zæ: by multiplying two matrix of size B x B and B C' with complexity C).²C) Calculating Capt requires further calculating part compared to Lrpt: inverse matrix T-BG in 21 with complexity O(B)) by utilizing the fast singular value decomposition.
For space complexity, calculating two losses requires additional storage spaces, first sample pairwise kernel similarity requires space with complexity O(B2), and PP (Y X") requires space with complexity O(BC).
Generally, given B 64, C = 100 and D 256. Compared to the scale of model parameters, CCL adds negligible overhead relative to the neural network's 's computational cost when computing loss. We further report the averaged mini-batch training time with a single 3090 GPU and the GPU memory usage in Table 99] and Table [00] As seen from these tables, the training time and space consumptions of CCL are comparable the existing -fht---rrttt-tt method ACR when CCL applies
additional data augmentations to labeled data for representation learning.
Table 8: Time and space complexity of two continuous contrastive loss of CCL.
CCL loss
Time complexity
Space complexity
Crpl Cxpl
O(B2D + B²C) O BBC B²)
O BC) O(B² +C)++B)
Table 9: Average batch time of each algorithm.
Algorithm
CIFAR-10
CIFAR-100
STL-10
ACR
0.073 secfiter
0.083 sec/iter
0.114 sec/iter
CCL
0.102 sec/iter
0.111 sec/iter
0.143 secfiter
G.3 Confusion matrix
Figure 6 presents the confusion matrix on the test set generated by CCL and ACR, which is
calculated on the CIFARIO-LT dataset under ?/ = 7h = 100 and n = 7n = 150 settings. As we can see in the top row of the figure, ACR often misclassifies the minority class "7" and "8" into the majority class "4" and "0", respectively. comparison, CCL. effectively mitigates this misclassification phenomenon by achieving average improvement of 7.5%. Similarly in the
20

---

Table 10: GPU memory usage of each algorithm.
Algorithm
CIFAR-10
CIFAR-100
STI.-10
ACR
2054M 2230M
2057M 2232M
2236M
CCL
2642M
car OOK 0UC 0:0 DUK dan oue
CP
600 En 600 600 600
660 BOO 600 EOO 8O0 EO- EOR
dos co0 coo 608 601
BOD 660 EOS GOY 800
donn 600
600
600 800
fon 600 608
600 600
0:4 031 0:1 0a0
E15 col 600
8G0 500
660 850 800 E0Y
(a) ACR (? (b) CCL (x=%==10)
(c) ACR (*=)== 150)
(d) CCL (Y=7%=150)
Figure 6: Confusion matrices of the predictions on the test of CIFARIO-LT.
second row, CCL achieved extraordinarily high accuracy of 78% for class "9", which shows significant gain of 37% compared to ACR. CCL. also achieves higher overall accuracy.
G.4 Precision and recall
To conduct more in in-tepth analysis of the effectiveness of pscudo-labels generated by the proposed dual-branch fusion approach, we calculated the precision and recall of the pscudo-labels assigned to unlabeled data by ACR and CCL CIFARIO-LT and CIFAR 1IFAR100-L2 100-LT datasets. Specifically, use // To = 100 and / = You = 150 settings on CIFAR 10-LT dataset and all three cons istent, uniform, reversed settings CIFAR100-LT dataset and grouped the results of CIFAR 100 into 10 categories. each category containing 10 classes, since CIFAR 100 comprises 100 classes. As can be seen in figure 8 CCL achieves significantly improved precision of pseudo-labels for tailed classes "9" and "10" on CIFARIO 10 dataset, while also achieving better recall for head classes. Similarly in Figure 7 CCL achieves overall better precision and recall compared to ACR regardless of the distribution mismatch scenario. It clearly shows that the pseudo- labels generated by CCL are more capable of alleviating the confirmation bias of tailed classes without sacrificing the performance of head classes.
G.5 Visualization
Furthermore, we employ the t-distributed stochastic neighbor embedding (t-SNE) 61 to visualize the representations learned by the CCL method and contrast them with those from the previous ACR method. The comparative results on the test set, under consistent settings, are depicted in Figure 9 The figure demonstrates that the representations derived from CCL provide more distinct classification boundaries.
H Limitation
Our paper examines existing long-tailed learning methods through the lens of information theoretical view, proposing a unified framework. However, we have not established theoretical proof for the convergence of features within this framework. the future, we intend to provide further theoretical analysis from the perspective of neural collapse.
21

---

Precision
Recall
0.6
0.
0.8
0.6
Prodston Recals)
06
Predttio Recal (53
0.0
844
04
04-
0.
02
92-
0.0
5
10
00
5
9
10
00
Class Index
Clsso Indox
Class indox
(a) ACR on
7/u
10
(b) ACR on 72 10, Tu =
(c) ACR
10, 7a 1/10
0.6
Preciiin (Reall))
Preceion (Real())
0.4
0.0
C.555
CCL 7 7h 10
CCL 7V 10, Tu
(1) CCL
7V
100 Tu 1/10
Figure 7: The precision and recall of pseudo-labels for ACR and CCL on CIFARI00-LT dataset in consistent, uniform, reversed settings.
Precision
Recall
10
Closs
Clans Vndoo
Closs Vnde
(a) ACR (x=%)==100
(b) CCL (7=7%,=100)
(c) ACR (7=7%.=150)
(d) CCL (w=7)=150)
Figure 8: The precision and recall of pseudo-labels for ACR and CCL on CIFARIO-LT dataset in consistent settings.
I Broader Impact
The positive impacts of this work are two-fold: 1) It enhances the fairness of the classifier semi- supervised learning, preventing potential biases in deep models, such unfair AI primarily serving the majority, which could lead to discrimination based on gender, race, or religion; 2) It enables the preprocessing.
easy collection of larger image datasets without the need for mandatory class-balancee For example, in training classifiers for real-world natural image scenes using the proposed method, do not need to consider whether the distribution of unlabeled data matches that of the labeled data or if every class in the labeled data has an equal number of samples. However, negative effects might occur if the proposed long-tailed semi-supervised classification technique misused. In the wrong hands, this approach could be exploited for unethical purposes, such as targeting or identifying minority groups for detrimental reasons.
22

---

Classifictioo Boundaries
Classification Boundaries
(a) ACR on " = % = 100
(b) CCL (ours) on 71 = 7 = 100
Classification Boundaries
Classification Boundaries
(c) ACR on = Tu 150
(d) CCL (ours) on = 7v- = 150
Figure 9: The t-SNE visualization of the test set for ACR and CCL on CIFAR-1O-LT dataset in consistent settings.
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly articulated our contributions at the end of the abstract and the introduction. Additionally, the research scope of this paper is introduced at the beginning of both the abstract and the introduction.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations of our method are analyzed in the appendix of this paper.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All experimental results were obtained by implementing the proposed method and rumning it the dataset. All results are reproducible.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have made the source code publicly available via the link described in the abstract.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In the appendix, we provide a detailed description the experimental setup, including the creation of the dataset, hyperparameter settings, well as the pseudocode for our method and diagrams of the model structure.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: In the experimental section, present statistical results, including the mean and variance accuracy, ECE calibration metrics, and line graphs estimating prior errors.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
24

---

Answer: [Yes]
Justification: The computational resources we used are detailed in the appendix.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:///rrrp...cooo/aa//ccccaaaa
Answer: [Yes]
Justification: Our code adheres the NeurIPS Code of Ethies and does not violate any ethical guidelines.
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss societal impacts in the appendix.
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not employ high-risk data models, thus posing such risks.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The assets from several recent works that we utilized are cited in our paper.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicablle as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
25


Reviews:
[{"review_id": "oVZyZVp0KO", "reviewer": "Reviewer_1xLE", "summary": "This paper tackles the long-tailed semi-supervised learning problem. It first reviews recent works with a novel probabilistic framework. Based on this, it proposes a continuous contrastive learning method, CCL, to extend the framework to unlabeled data with pseudo-labels. Experiments show that it outperforms all recent works on the popular long-tailed semi-supervised learning benchmarks.", "questions": "See weaknesses\n\nIn Eq. 22, it mentioned L_{cls}, which is also mentioned in Line 171. Is that the same as P^{cls} in Eq. 15?", "limitations": "Limitations and potential impacts are addressed in Appendix,", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Overall, this paper is good.\n\n1. With a framework from information theory perspective, it unified different approaches on the long-tailed learning topic.\n\n2. It systematically studied the key ideas to tackle the long-tailed learning and semi-supervised learning tasks.\n\n3. By proposing CCL, it unifies different ideas from long-tailed learning and semi-supervised learning, and the result is an effective approach to solve the LTSSL problem.", "weaknesses": "I only have one question:\n\nIn Eq. 22, it mentioned L_{cls}, which is also mentioned in Line 171. Is that the same as P^{cls} in Eq. 15?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 1xLE,\n\nWe sincerely appreciate your thoughtful feedback. We will respond to each of your concerns individually.\n\n**Weakness #1: Misunderstanding between $\\widehat{\\mathcal{L}}_{\\mathrm{cls}}$ in Eq. (22) and $\\widehat{\\mathbb{P}}^{\\mathrm{cls}}\\left(Y=y \\mid \\boldsymbol{x}^u\\right)$ in Eq. (15)**\n\nTo clarify, $\\widehat{\\mathcal{L}}_{\\mathrm{cls}}$ represents **the loss function** for the classification part of the dual-branches $f_s(\\cdot)$ and $f_b(\\cdot)$. \n\nHowever, $\\widehat{\\mathbb{P}}^{\\mathrm{cls}}\\left(Y=y \\mid \\boldsymbol{x}^u\\right)$ denotes **the pseudo-labels** for training dual-branches $f_s(\\cdot)$ and $f_b(\\cdot)$, where we propose to fuse the predictions of balanced and standard branches by Eq. (15) to obtain $\\widehat{\\mathbb{P}}^{\\mathrm{cls}}\\left(Y=y \\mid \\boldsymbol{x}^u\\right)$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4BcWJMLZgG", "reviewer": "Reviewer_biXa", "summary": "This paper proposes a novel contrastive learning method for long-tailed semi-supervised learning (LTSSL). The method is motivated by variational information bottleneck for learning good representations and extends to unlabeled data using continuous pseudo-labels. This paper showcases strong empirical results on multiple LTSSL datasets and label distributions.", "questions": "Please see Weaknesses.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The proposed method is well-motivated. Previous works in LTSSL generally make contributions in learning balanced classifiers, while this paper examines representation learning.\n2.This paper summarizes many existing methods for long-tail learning in a unified framework. It is interesting to show that the equivalence between supervised contrastive learning and Gaussian kernel estimation.\n3.This paper proposes a new LTSSL method by optimizing the contrastive loss using two types of continuous pseudo-labels. The proposed method is novel in LTSSL.\n4.Extensive experiments showcase that the proposed method achieves best results on all datasets. The paper also provides extensive in-depth studies to understand the effectiveness of the method.", "weaknesses": "1.In Table 1, the paper states that long-tail learning methods need to avoid none class samples in each min-batch during training. However, it is unclear how this issue is tackled in the proposed method CCL.\n2.The training time of the proposed method is not reported. It is suggested to compare the training time with ACR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer biXa,\n\nWe sincerely appreciate the reviewer's thoughtful feedback. We will address your concerns individually.\n\n**Weakness #1: How does the proposed method address the issue of having mini-batches with no samples from certain classes during training**\n\nFor our proposed method CCL, to avoid none class samples in each min-batch during training, we maintain a **learnable class prototype** for each class to facilitate the contrastive learning and address the aforementioned issue. The design of the learnable class prototypes is achieved through a nonlinear mapping of the classifier's weights.\n\n**Weakness #2: The comparison of training time overhead between the proposed method and ACR.**\n\nThank you for the suggestion for considering the time overhead of the proposed method! We evaluated the average time taken to process a minibatch with a single 3090 GPU. \n\n###### Table 1: Average batch time of each algorithm\n\n| Algorithm | CIFAR-10       | CIFAR-100      | STL-10         |\n| --------- | -------------- | -------------- | -------------- |\n| ACR       | 0.073 sec/iter | 0.083 sec/iter | 0.114 sec/iter |\n| CCL       | 0.102 sec/iter | 0.121 sec/iter | 0.153 sec/iter |\n\nSince CCL additionally considers representation learning compared to ACR, it applies corresponding data augmentations to labeled data as well, resulting in an overhead approximately 1.5 times that of ACR. However, as seen from the Table 1, the execution speed is nearly identical.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MnVk1LhJJe", "reviewer": "Reviewer_WTLw", "summary": "The author proposes a probabilistic framework that unifies many recent proposals in long-tail learning. Specifically, for long-tailed semi-supervised learning,  a continuous contrastive learning method based on reliable and smoothed pseudo-labels to address confirmation bias and improve the quality of learned representations is proposed. The experiments demonstrate their effectiveness.", "questions": "see weakness", "limitations": "yes", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1) The author notices an important problem that existing approaches often assume the equivalence of the unlabeled data distribution with the labeled data, which is more practical. \n\n2) The proposed probabilistic framework which unifies many recent proposals in long-tail learning is interesting.\n\n3) The paper is well-written and easy to follow.", "weaknesses": "1) The author should discuss explicitly how the proposed method solves the unlabelled data  for diverse label distributions of unlabeled data and why other methods do not. \n\n2) How to interpret \" Furthermore, these methods primarily focus on correcting model outputs without delving into the role of representation learning in improving performance.\"  line 33-34. \n\n3) The author may give an analysis of the relationship between  reliable pseudo-labels and smoothed pseudo-labels and how they play a different role in semi-supervised learning.\n\n4) The time and space complexity should be analyzed in experiments between the proposed method and other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer WTLw,\n\nWe sincerely appreciate the reviewer for thoughtful feedback. We address your concerns one by one.\n\n**Weakness #1: How the proposed method solves the unlabelled data for diverse label distributions of unlabeled data**\n\nHaving an accurate estimation of unlabeled data prior $\\widehat{\\boldsymbol{\\pi}}_u$ helps solve LTSSL like supervised LTL, which can simply achieve balanced training by logit adjustment. We propose to use EMA estimation of dual-branches by selecting data subset from mini-batch via **energy score**.\n\nAccurate estimation of $\\widehat{\\boldsymbol{\\pi}}_u$ requires **calibrated model** [1]. Methods like ACR and CPE, using confidence-based sample selection, producing overconfident but incorrect pseudo-labels [2]. In contrast, **energy score leverages prediction probability density**, mitigating the overconfidence problem [3]. Thus, energy selection is used for a reliable unlabeled data subset, on which model is calibrated, enabling accurate estimation of $\\widehat{\\boldsymbol{\\pi}}_u$.\n\nIn Figure 1 of our paper, we empirically compared the estimation error of confidence and energy score.\n\n**Weakness #2: How to interpret \"Furthermore, these methods primarily focus on correcting model outputs without delving into the role of representation learning in improving performance\"**\n\nRecent LTSSL methods focus on adjusting the classifier output to overcome the long-tail effect. Methods like **DASO**, **ACR**, and **CPE** improve pseudo-label quality through techniques such as classifier output ensembling or logits scaling, ignoring the representation learning.\n\nIn contrast, our method delves into the long-tail effect in **representation learning**. CCL introduces balanced FixMatch for **classifier training** and enhances representation learning with **reliable and smoothed pseudo-labels**, as described in Eq. (17) and Eq. (18).\n\n**Weakness #3: Relationship between reliable pseudo-labels and smoothed pseudo-labels**\n\nThe main challenge in generalizing our supervised learning framework to LTSSL is the unknown of $\\mathbb{P}_{u}(Y = y \\mid \\boldsymbol{x}^u)$ for unlabeled samples.\n\nWe first approximate $\\mathbb{P}_{u}(Y = y \\mid \\boldsymbol{x}^u)$ using integrated classifier output in Eq. (15). We use energy score to filter out reliable unlabeled samples and their pseudo-labels, ensuring that model is calibrated on this subset. Learning from **reliable pseudo-labels** enhances feature discrimination.\n\nHowever, self-training can introduce confirmation bias and negatively impact the representation learning. Thus we propose **smoothed pseudo-labels** for $\\boldsymbol{x}^u$ by propagating labels from nearby samples, leveraging the smoothness assumption in the feature space without classifier information. The purpose of $\\mathcal{L}_{\\mathrm{spl}}$ is to align the smoothed pseudo-labels of $\\mathcal{A}\\_{w}(\\boldsymbol{x}^u)$ and $\\mathcal{A}\\_{s}(\\boldsymbol{x}^u)$, serving as \"consistency regularization\" in SSL.\n\n**Weakness #4: The time and space complexity**\n\nDenote feature space dimension $D$, batch size $B$ and number of classes $C$, the time and space complexity of CCL are as follows.\n\n##### Time Complexity\n\nFor $\\widehat{\\mathcal{L}}\\_{\\mathrm{rpl}}$:\n\n1. Calculating kernel similarities: $\\mathbb{R}^{B \\times D} \\times \\mathbb{R}^{D \\times B}$, with complexity $\\mathcal{O}(B^{2}D)$.\n2. Calculating $\\mathbb{E}\\_{\\boldsymbol{x}^{\\prime} \\sim \\mathbb{P}(\\cdot| Y=y)}\\left[\\kappa\\left(\\boldsymbol{z_x}, \\boldsymbol{z_{x^{\\prime}}}\\right)\\right]$ using Gaussian kernels: $\\mathbb{R}^{B \\times B} \\times \\mathbb{R}^{B \\times C}$, with complexity $\\mathcal{O}(B^{2}C)$.\n\nTotal: $\\mathcal{O}(B^{2}D+ B^{2}C)$.\n\nFor $\\widehat{\\mathcal{L}}\\_{\\mathrm{spl}}$:\n\n1. Calculating kernel similarities: $\\mathbb{R}^{B \\times D} \\times \\mathbb{R}^{D \\times B}$, with complexity $\\mathcal{O}(B^2 D)$.\n2. Calculating $\\widehat{\\mathbb{P}}(Y \\left.=y \\mid \\boldsymbol{x}^u ; \\mathcal{B}^l\\right)$ based on Eq. (19): $\\mathbb{R}^{B \\times B} \\times \\mathbb{R}^{B \\times C}$, with complexity $\\mathcal{O}(B^2 C)$.\n3. Inverse matrix $(\\boldsymbol{I} - \\beta \\boldsymbol{G})^{-1}$ in Eq. (21): $\\mathcal{O}(B^3)$.\n\nTotal: $\\mathcal{O}(B^2 D + B^2 C + B^3)$.\n\n##### Space Complexity\n\nFor $\\widehat{\\mathcal{L}}\\_{\\mathrm{rpl}}$:\n\n1. Pairwise kernel similarity : $\\mathcal{O}(B^2)$.\n2. Calculating $\\mathbb{E}\\_{\\boldsymbol{x}^{\\prime} \\sim \\mathbb{P}(\\cdot| Y=y)}\\left[\\kappa\\left(\\boldsymbol{z_x}, \\boldsymbol{z_{x^{\\prime}}}\\right)\\right]$ using Gaussian kernels: $\\mathcal{O}(BC)$.\n\nTotal: $\\mathcal{O}(B^2 + BC)$.\n\nFor $\\widehat{\\mathcal{L}}\\_{\\mathrm{spl}}$:\n\n1. Computing $\\widehat{\\mathbb{P}}(Y = y \\mid \\boldsymbol{x}^u ; \\mathcal{B}^l)$: $\\mathcal{O}(BC)$.\n2. Pairwise kernel similarity: $\\mathcal{O}(B^2)$.\n\nTotal: $\\mathcal{O}(B^2 + BC)$.\n\nIn Table 2 and Table 3 of the global response (PDF) of our rebuttal, we empirically report the **averaged minibatch training time** with a single 3090 GPU and the **GPU memory usage**.\n\nGenerally, $B=64$, $D=256$, and $C=100$. Compared to the \\# of model parameters, CCL adds negligible overhead relative to the neural network's computational cost. **In Table 2 of our global response (PDF), we should note that the training time of CCL is comparable to existing state-of-the-art method ACR.**\n\nThank you for the valuable feedback again. We will address all the issues mentioned in the next version of the paper.\n\n###### **Reference**\n\n[1] Garg, S., Wu, Y., Balakrishnan, S., & Lipton, Z. (2020). A unified view of label shift estimation. *Advances in Neural Information Processing Systems*, *33*, 3290-3300.\n\n[2] Loh, C., Dangovski, R., Sudalairaj, S., Han, S., Han, L., Karlinsky, L., ... & Srivastava, A. (2022). On the Importance of Calibration in Semi-supervised Learning. *arXiv preprint arXiv:2210.04783*.\n\n[3] Liu, W., Wang, X., Owens, J., & Li, Y. (2020). Energy-based out-of-distribution detection. *Advances in neural information processing systems*, *33*, 21464-21475.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DGaW2FWlvE", "reviewer": "Reviewer_JsM2", "summary": "The paper proposes a novel method to address the long-tail problem in semi-supervised learning by leveraging continuous contrastive learning on both labeled and unlabeled samples to improve model performance, particularly for minority classes.", "questions": "A minor suggestion: the paper can include some demonstrations of unlabeled images with their associated continuous pseudo-labels, including both reliable and smoothed versions. This can help readers identify cases in which the model tends to make mistakes.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novelty: The paper introduces a novel approach to integrate contrastive learning with unlabeled samples into SSL frameworks, which is relatively unexplored in existing literature.\n- Technical Soundness: The theoretical formulation of learning good representations and the unified framework of many existing long-tail learning methods are well-explained and logically sound.\n- Experiments: The experiments are comprehensive, covering various imbalanced ratios and labeled ratios on CIFAR-10-LT and CIFAR-100-LT datasets. The results are statistically significant and show clear improvements over baseline methods.\n- Clarity: The paper is well-structured and clearly written. The introduction, methodology, and experimental sections are easy to follow, and the figures and tables effectively illustrate the results.", "weaknesses": "- In line 189, it is unclear why selecting pseudo-labels using energy score can ensure better model calibration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer JsM2,\n\nWe sincerely appreciate your thoughtful feedback. We will address each of your concerns in detail.\n\n**Weakness #1: Why selecting pseudo-labels using energy score can ensure better model calibration**\n\nSince the confirmation bias is induced by **self-training**, using **confidence selection** may result in overconfident but wrong pseudo-labels and hurt the calibration[1,2]. \n\nIn contrast, the **energy score leverages the probability density of the predictions**, exhibiting reduced vulnerability to overconfidence [3]. Thus, the energy function can alleviate confirmation bias caused by overconfidence, thereby obtaining a reliable data subset on which the model is calibrated.\n\nWe also compare the calibration metric ECE value [4] between confidence threshold and energy score in Figure 1 of the paper, and the results confirm our idea.\n\n**Weakness #2: The suggestion to demonstrate the reliable and smoothed pseudo labels of the unlabeled data for better understanding**\n\n**We demonstrate the generation process of reliable pseudo-label and smoothed pseudo-label in the attached PDF**.\n\nTo generalize the proposed supervised learning framework to LTSSL, the main challenge is unknown $\\mathbb{P}_{u}(Y = y \\mid \\boldsymbol{x}^u)$, where $\\boldsymbol{x}^u$ denote sample in unlabeled dataset. \n\nTo address this issue, we first approximate $\\mathbb{P}_{u}(Y = y \\mid \\boldsymbol{x}^u)$ using **the output of the calibrated and integrated classifier** as expressed in Eq. (15). Additionally, we use **energy score** to filter out reliable unlabeled data, ensuring the model's calibration on this data subset, which constitutes the reliable pseudo-labels subset.\n\nFurthermore, we can also estimate the unknown $\\mathbb{P}_{u}(Y = y \\mid \\boldsymbol{x}^u)$ by leveraging the **smoothness assumption**. Specifically, we construct smoothed pseudo-labels by propagating labels from nearby samples using **Gaussian kernel density estimation**.\n\nWe will include more detailed textual and graphical demonstrations in our next version of the paper.\n\n###### **Reference**\n\n[1] Mishra, S., Murugesan, B., Ayed, I. B., Pedersoli, M., & Dolz, J. (2024). Do not trust what you trust: Miscalibration in Semi-supervised Learning. *arXiv preprint arXiv:2403.15567*.\n\n[2] Loh, C., Dangovski, R., Sudalairaj, S., Han, S., Han, L., Karlinsky, L., ... & Srivastava, A. (2022). On the Importance of Calibration in Semi-supervised Learning. *arXiv preprint arXiv:2210.04783*.\n\n[3] Liu, W., Wang, X., Owens, J., & Li, Y. (2020). Energy-based out-of-distribution detection. *Advances in neural information processing systems*, *33*, 21464-21475.\n\n[4] Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017, July). On calibration of modern neural networks. In *International conference on machine learning* (pp. 1321-1330). PMLR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We extend our gratitude to the reviewers for their constructive and valuable feedback.

We are encouraged by the reviewers' acknowledgment of the novelty of our continuous contrastive learning framework (R2, R4), its significant contribution (R1, R2, R3, R4), and its soundness (R1, R2, R3, R4). Reviewers also consider the theoretical perspective to be interesting (R2, R3) and robust (R4). In addition, the reviewers recognize the practical value of our framework under LTSSL (R3, R4), noting that our approach is effective, with empirical results that outperform existing methods (R1, R2, R3, R4). We are pleased with comments regarding the good presentation (R1, R4), the quality of writing (R3), and the clear motivation and structure (R2) of our work.

(We refer to **Reviewer 1xLE** as R1, **Reviewer biXa** as R2, **Reviewer WTLw** as R3, and **Reviewer JsM2** as R4.)

We have taken the reviewers' comments into consideration and have responded to each of them individually. After carefully reviewing the comments, we have identified two common issues among the reviewers' feedback. To address these concerns, we have included empirical results and illustrations, which can be found in the **attached PDF**.

- **Time and space complexity analysis**

  We provide a detailed analysis of the time and space complexity results for CCL and the existing state-of-the-art method ACR. This includes the average minibatch processing time with a single 3090 GPU and the GPU memory usage during training.

- **Role and relationship of the reliable and smoothed pseudo-labels**

  We illustrate the generation of reliable and smoothed pseudo-labels in the PDF to facilitate a clearer understanding of the roles and relationships of these two types of pseudo-labels.

We will address and resolve these issues in the next version of the paper.

Thank you for taking the time to review our work. We appreciate your efforts and feedback.

Meta-review: The paper initially presents a unified probabilistic framework for long-tail learning and further extends it to a semi-supervised setting through a novel continuous contrastive learning approach. The results demonstrate significant improvements across all benchmarks. The rebuttal effectively addresses the reviewers' concerns, leading to unanimous positive feedback. After reviewing the comments and feedback, the AC  recommends acceptance.
Decision: Accept (poster)