Self-Supervised Logit Adjustment
Authors: Zhihan Zhou
Jiangchao Yao
Feng Hong
Yanfeng Wang
Bo Han
Ya Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: Long-tailed learning
self-supervised learning
logit adjustment
optimal transport

Abstract: Self-supervised learning (SSL) has achieved tremendous success on various well curated datasets in computer vision and natural language processing. Nevertheless, it is hard for existing works to capture transferable and robust features, when facing the long-tailed distribution in the real-world scenarios. The attribution is that plain SSL methods to pursue sample-level uniformity easily leads to the distorted embedding space, where head classes with the huge sample number dominate the feature regime and tail classes passively collapse. To tackle this problem, we propose a novel Self-Supervised Logit Adjustment ($S^2LA$) method to achieve the category-level uniformity from a geometric perspective. Specially, we measure the geometric statistics of the embedding space to construct the calibration, and jointly learn a surrogate label allocation to constrain the space expansion of head classes and avoid the passive collapse of tail classes. Our proposal does not alter the setting of SSL and can be easily integrated into existing works in an end-to-end and low-cost manner. Extensive results on a range of benchmark datasets show the effectiveness of $S^2LA$ with high tolerance to the distribution skewness. 

Under review as a conference paper at ICLR 2023
SELF-SUPERVISER LOGIT ADJUSTMENT
Anonymous authors Paper under double-blind review
ABSTRACT
Self-supervised learning (SSL) has achieved tremendous success various well curated datasets in computer vision and natural language processing. Neverthe- less, it is hard for existing works to capture transferable and robust features, when facing the long-tailed distribution in the real-world scenarios. The attribution is that plain SSL methods to pursue sample-level uniformity easily leads to the dis- torted embedding space, where head classes with the huge sample number domi- nate the feature regime and tail classes passively collapse. To tackle this problem, we propose novel Self-Superrisee Logit Adjustment LS* method to achieve the category-level uniformity from a geometric perspective. Specially, we mea- sure the geometric statistics of the embedding space to construct the calibration, and jointly learn a surrogate label allocation to constrain the space expansion of head classes and avoid the passive collapse of tail classes. Our proposal does not alter the setting of SSL and can be easily integrated into existing works in a low-cost manner. Extensive results on range of benchmark datasets show the effectiveness of LA with high tolerance to the distribution skewness.
INTRODUCTION
Recent years have wimeessed great success of self-supervised learning (Doersch al., 2015 Wang & Gupta 2015 Chen et al.] 2020; Caron et al. 2020). The rapid advances behind this paradigm ben- efit from the elegant training on data without annotations, which can be acquired in a large-volume and low-cost way. However, the real-world natural sources usually exhibit the long tailed distribu- tion (Reed) 2001), and directly applying existing self-supervised learning methods will lead the distorted embedding space, where the majority dominates the feature regime /Zhang et al., 2021) and the minority collapses (Mixon et al. 2022). With the increasing attention on machine learning fairness in the recent years, it becomes a trend to explore self- self-super long-tailed learning (Yang)
& Xu 2020 Liu et al. 2021 Jiang et al. 202 Zhou et al. 2022).
Compared with the flourishing supervised long-tailed learning (Kang al. 2019 Yang & Xu 2020
Menon et al. 202T), the self-supervised counterpart is underexplored as an emerging direction. Existing explorations for self supervieed learning in long-tailed context are from three perspectives: data perspective, model perspective and loss perspective. In the data perspective. BCL (Zhou et al.] 2022 leverages the memorization effect of deep neural networks (DNNs) drive instance-wise augmentation, which learns a better trade-off between head classes and tail classes in representation learning. In the model perspective, SDCLR (Jiang et al. 2022]) contrasts the feature encoder and its pruned counterpart to discover hard examples that mostly covers the samples from tail classes, and efficiently enhance the learning preference towards tail classes. In the loss perspective, the reweighting mechanism like rwSAM (Liu et al., 202T) that adopts data- sharpness aware minimization scheme, can be applied explicitly regularize the loss surface. However, terms of the current performance to self-supervised long-tailed learning, the potential of the loss perspective has not been sufficiently set off, with which in comparison supervised long-tailed learning, logit adjustment (Menon al. 2021 of the same perspective has conquered range of methods.
We dive into the loss perspective and explore to understand "Why the conventional contrastive learn- ing underperforms in self-supervised long-tailed learning?" To answer this question, let us consider two types of representation uniformity: (1) Sample-level uniformity. As proof in (Wang & Isola 2020), contrastive learning targets to distribute the representation of data points uniformly in the embedding space. Then, the feature span of each category is proportional to their corresponding sample number. (2) Category-level uniformity. This uniformity pursues to split the region equally

---

Under review as a conference paper at ICLR 2023
DO
0000
Figure 1: Comparison of LA and the plain SSL method on a 2-D imbalanced synthetic dataset. (Left) Visualization of the 2-D synthetic dataset. (Middle) The embedding distribution of each cat- egory learnt by the plain SSL method is approximately proportional to the sample number. (Right) S reduces the adverse effect of class imbalance and approaches the category-lewe uniformity.
for different categories without considering their corresponding sample number (Papyan al., 2020 Graf et al. 2021). In the case of the class- balanced scenarios, the former uniformity naturally implies the latter uniformity and induces the equivalent separability for classification. However, in the class- imbalanced cases, especially in long-tailed setting, sample-level uniformity leads to the undesired feature regime of head classes due its dominant sample proportion. comparison, category-leve uniformity constrains the greedy expansion of head classes and prevents the passive collapse of tail classes, which is more benign to the downstream classification (Graf et al.) 2021 Fang al. 2021
Li al. 2022). Unfortunately, there is no support regarding category-leve uniformity in contrastive learning losses, which provides an answer to the question arisen at the beginning.
Inspired by logit adjustment (LA) for supervised long-tailed learning, we propose novel method, termed as Self-Supervised Logit Adjustment LA), to calibrate self-supervised long-tailed learning from the geometric perspective. Specially, unlike LA that requires the class distribution available, SALA uses a constant Simplex ETF to measure the geometric characteristics of the embedding space for adjustment. Together with surrogate label allocation to compute the target, plicitly compress the greedy space expansion head classes and avoid the passive collapse of tail classes. Their alternation refers to ordinary balancing and efficient optimal-transport problem, which dynamically approaches towards the category-level uniformity. In Figure [ we give toy experiment to compare the learnt representation without and with LaLA in the embedding space.
then ex-
The contribution can be summarized as follows,
We are among the first attempts to study the drawback of the contrastive learning loss in self-supervised long-tailed context and point out that the resulting sample-level uniformity is an intrinsic limitation, driving our exploration for category-level uniformity (Section 4
2. We develop a novel Self-Supervised Logit Adjustment (Figure D. which dynamically ad- justs the embedding distribution to calibrate the geometric statistics and conducts a surro- gate label allocation for category-level uniformity in an efficient end-to-end manner.
3. Our method can be easily plugged into previous methods of self-supervised long-tailed learning (Eq. 7 Extensive experiments range of benchmark datasets demonstrate the consistent improvement of our LL with high tolerance to the distribution skewness.
2 RELATED WORKS
Self-Supervised Long-tailed Learning. There are several recent explorations devoted to this direc- tions from data, model and loss perspectives. BCL (Zhou al., 2022) leverages the memorization effect of DNNs to drive an instance-win wise augmentation, which enhances the learning of tail samples. SDCLR Diang et al. 2021) constructs a self-contrast between model and its pruned counterpart to learn more balanced representation. Classic Focal loss (Lin et al. (2017) leverages the loss statistics to putting more emphasis on the hard examples, which has been applied to self-supervised long- tailed learning (Zhou al. (022). SeLa (Asano et al. 2020) is the first attempt to cast the unsuper- vised clustering as an optimal transport problem and leverage a uniform prior on the class itaalainbe data. rwSAM (Liu et al. 2021) proposes to penalize loss sharpness in reweighting manner to sim- ilarly calibrate class- inass-imallance learning. However, the potential of the loss perspective has not been set-off due to the intrinsic limitation of the conventional contrastive learning loss.
2

---

Under review as a conference paper at ICLR 2023
Hyperspherical Uniformity. The distribution uniformity has been extensively explored from the physic area, e.g., Thomson problem (Thomson 1904; Smale 1998). to machine learning area like some kernel-based extensions e.g., Riesz s-potential (Hardin & Safi 2005 Liu et al. 2018) Gaus- sian potential (Cohn & Kumar 2007; Borodachov et al. 2019; Wang Isola 2020). Some recent explorations regarding features of IDNNs (Papyan et al. 2020 Fang et al. 2021 Mixon et al. 2022)
discover a terminal training stage when the embedding collapses to the geometric means of the classifier w.E.f each category. Specially, these optimal class means specify maximum separation structure, termed as Simplex Equiangular Tight Frame, and achieve the perfect uniform distribution under some dimensional constraints (Zhu et al., 2021] Yang et al., 2022 Kasarla al. 2022). In this paper, we incorporate Simplex ETF into contrastive learning and leverage the specific structure as a uniform prior to capture the geometric statisties for calibration of the embedding distribution.
PRELIMINARIES
3.1 PROBLEM FORMULATION
Given dataset D. for each (x, y) D, the input R" and the associated label y {l, L-- We define the imbalanced ratio as max; p(y = / p(y = i). In SSL, the ground-truth is not accessed and the goal is to transform an image an embedding via DNNs fo R - Rd. In the linear probing evaluation, we construct a supervised learning task with balanced datasets. A linear classifier g is built top of the feature extractor fe to produce prediction, namely, g(fo(z)).
3.2 LOGIT ADJUSTMENT IN SUPERVISED LONG-TAILED LEARNING
Logit adjustment manipulates the logits of prediction to remove the adverse effect of class- imbalance, which has shown efficiency in various supervised long-tailed explorations (Provost 2000 [Brodersen et al.) 2010: Ren et al. 2020) Menon et al. 2021). Let p(y), (3) be the label distribution of the long-tailed training dataset and the balanced test set, we can derive p(y (z) p(z) and Phal (y(ar)
Pbar)))) based on the Bayes" theorem. Assum- ing class-conditional probabilities p(x|y) are the same on training and test set, we have p(y)2) x Piat (p))) This indicates that learning under long-tailed data leads to prediction shift pro- portional to ply) compared with the optimal classifier on the uniform test set. To this end, logit adjustment adds offset the logits to calibrate the statistical shift, namely,
(1) where s(x) denotes the logits of the training samples and softmax(-) (- is the Softmax function. Logit adjustment are well grounded to be Fisher consistent for minimising the balanced error (Menon et al. 2021), thus extended to a range of class-imbalanced scenarios (Ren et al. 2022).
Phat(yla ()) x p(y) (//ee)((())))) x (oftmax(s(()) - log logg((y)),
3.3 SIMPLEX EQUIANGULAR TIGHT FRAME
Neural collapse (Mixon et al., 2022) describes phenomenon that with the training, the geometric centroid of representation progressively collapses to the optimal classifier parameter w.r.f. each category. The collection of these points builds special geometric structure, termed as Simplex Equiangular Tight Frame (ETF). Some study that shares the similar spirit is also explored regarding the maximum separation structure (Kasarla et al. 2002). We present its formal definition as follows.
Definition 3.1. A Simplex ETF a collection of points in gd specified by the columns of the matrix:
K K K-1
M M
U(KK
where Ix E RRKK is the identity matrix and 1 k is the the patial orthogonal matrix such that U Ix and satisfys d 2 K. All vectors in a Simplex ETF have the same pair-wise angle, i.e., m,m;
K K-dimensional ones vector. U R is
- i + j. The pioneering work (Yang et al. 2022) shows Simplex ETF as a linear classifier combined with neural networks is robust to class-imi mmbalaccd learning in the supervised setting. On the opposite, our motivation is to make self- supervised learning robust the class-imbalance data, which requires the pursuit in the embedding
3

---

Under review as a conference paper at ICLR 2023
space intrinsically switching from the sample-level uniformity to the category-level uniformity. The Simplex ETF is a tool to measure the gap between the category- uniformity and the sample-level uniformity, which is then transformed as the supervision feedback to the training.
4 SELF-SUPERVISED LOGIT ADJUSTMENT
Plain SSL. methods encourage the sample-level uniformity in the embedding space, yielding the undesired property in long-tailed context. Specially, head classes will dominate the feature regime along with the collapse of tail classes. To solve this problem, one straightforward idea is to coun- teract the long-tailed effect like logit adjustment in supervised long-tailed learning. As shown in Figure we observe that the geometric calibration will benefit the feature balancedness. However, the core challenge in self-supervised learning is that there is no class- dissssiissiibtiion prior available to characterize the imbalance degree. To address this issue, propose use Simplex ETF to mea- sure the embedding space, and then the captured geometric statistics are used to properly adjust the embedding distribution alongside a surrogate label allocation. Concretely, we define an constant classifier M E Rdx K following Eq. 2 and then compute the geometric label of each sample:
t = arg max hz())) where h₂(x)) = M. fa
On the basis of such positional indicator, we can further derive population-level statistic to characterize the category-level uniformity, ie., p(t = i) = Eg [1 (t. = i)], i = 1..... K, where (.)) denotes the indicator function with the value 1 when its argument is true and O otherwise. In the balanced setting, we maintain an approximately uniform distribution while in long-tailed context, we face more skewed geometric distribution for the category-level uniformity.
Logit Adjustment. Different from the ordinary logit adjustment that removes the class imbalance in p(t) and exacerbates the space expansion, here we aim to shrink the feature span of head classes for the category- level uniformity, and thus reversely adjust the logits, namely:
= arg max p(t|(=))(()
arg max h(x)) + log
where p(t (x) softmax (x))) denotes the output probabilities of the geometric classifier and is pyper-parameter to temper the adjustment. We can derive p (t|z) x p(z)t))(t))** based on Bayes' theorem, which refers to exponential growth over the original distribution p(t|ar) p( (tt)).) This fulfills an elastic adjustment on the embedding skewness. Logit adjustment is to strengthen the estimation of samples from head classes so that they can be discovered many as possible, which is then used to suppress the feature regime of head samples for the category-leve uniformity. This is different from logit adjustment in supervised long-tailed learning, which is to calibrate the quantity bias towards different classes of samples in the logit space.
However, since there is no ground-truie -ruth annotation of each sample, straightforward instance-level adjustment will lead to performance degradation with unreliable predictions, as empirically evi- denced in Table TO To handle these problems, we learn a surrogate label allocation together with logit adjustment from the population level. Concretely, we first estimate the adjusted statistic (t)
based on Eq. i.e., p (ī i) Ex-D[L [1 (7 = i) 1, 1,..., K, and then propose the following objective to generate label posterior q(t|ar)) with the desired marginal distribution P(t):
Σ (t||=) log ((]x)) s.t. Ca-D(g(((a)) (I) IND
min P-g
(4)
D
where the constraint guarantees the adjusted population-level statistics. Naive jointly optimizing Eq. easily falls into a trivial solution and causes severe performance degeneration, as shown in Table 10 Considering this, we resort to solving an optimal transport problem for label allocation on q(((z) and then optimize the model parameters.
Label Allocation. We first reformulate Eq. aas an optimal transport (OT) problem. Denote the as- signments Q E TR XxN N for distributing N samples to K classes with the minimal transportation cost and let C log P to retain the equivalent objective, where P IR K N are the joint probabilities derived from g (t) We can rewrite Eq. 4 as the following form:
min (Q, C% s.t. Q IN = p (f), QT 1K = AN QERNXN
(5)
4

---

Under review as a conference paper at ICLR 2023
log
Oistr Bution Collbnation
Sample-ivell Uniformitty
cotegory-level Uniformity
min (Q.C)
logp(t)
SSL
L
p(tt)
((
Figure 2: Illustration of S the top of SSL. We utilize the Simplex ETF structure measure the embedding space, and the captured geometric statistics are used for the target calibration logit adjustment. Simultaneously, together with adjustment to avoid the trivial solutions. The additional model parameters introduced by L trained with SSL in an efficient manner.
optimal transport optimization for label allocation is performed
This objective can be efficiently solved by adopting Sinkhorn-Knope algorithm (Cuturi 2013) with an additional entropy regularization to the objective, i.e., ming (Q, C* H()).. Here, we define the entropy of Q as H(Q) = - Σ: Ej Qis log Qiz- We can then obtain the solution as:
Q = diag(u) exp (-XC) diag(v)
(6)
where u, U are two non-negative vectors of Rd We iteratively update u and v to reach the desired row and column marginals of Q. The coefficient controls the smoothness the low-cost assign- ments. As À -> oo, we can approach the optimum of the OT problem at the cost of convergence speed. We keep at relative high level to maintain good clustering results, To explain the merits of the population- level label allocation, we compute the NMI score between é or q and the oracle label in Table II Accorrding to the comparison of their NMIs, we can understand the roughly ad- justed & can be noisy and shows the weak correlation with the oracle label, while q(t|ax) reallocated by optimal transport based on the population statistic of i effectively improves the correlation.
In practice, we estimate (t) and (f) over the training dataset and learn the label allocation ((t[z) in the mini-batch manner. Specifically, we compute the class prior p(t) and the adjusted (t) the beginning of every epoch as the population-Ieve statistic will not change much few mini-batches, Besides, we maintain momentum update mechanism to track the output logits of each sample stabilize the training. i.e., h h²)x)) 3hh" + (1 (xx) Given the surrogate labels q(t)ar), we can directly optimize p(t|ar) by minimising the cross-entropy loss, which can be considered as the alignment to the desired geometric distribution. We illustrate L in Figure 2
and summarize the complete procedure in Algorithm iin the appendix.
Overall Objective. S is compatible with the general self-supervised learning methods as is constructed on the surrogate geometric analysis to rectify the distorted embedding space in long- tailed context. For example, we build SLLA on SimCLR, the overall objective is defined as:
exp (fo(z)] fo(x+)/7)
L
Σ mEX,
log Σ Ex-Ex ( exp (Je( foll= fe(x= )/7)
q(t x log (ttz(())))
B
X
bEB
where x, x is set as different augmented view the same input image and æ denotes the rest images in the mini-batch. Xs is the sampled batch and X is the negative sample set with b as the batch index. 7 represents the temperature
Remark 4.1. Although SaA iteratively infers the surrogate labels and applies them for re-balanced training. it incurs only small amount of computational or memory overhead. Specifically, the stan- dard optimization of deep neural networks requires forward and backward step in each mini-batch batch update with the time complexity as ((MA), where M is the mini-batch size and A is the parame- ter size. At the parameter level, we add an additional classifier with the complexity as O(MK)) where the class number and is the embedding dimension. Besides, Sinkhorn- algorithm + c./lexp(-XC)] u,
only refers to simple matrix-vectee multiplication +- r./ wx/eexp(-CC) whose complexity is O(L(M - K + MK)) with the iteration step L. The complexity incurred in the momentum update is O(MK). Since K, d and L is significantly smaller than the model pa- rameter A of a million scale, the computational overhead involved L2LA is negligible compared to O(MA). The additional storage is the vector hm (x:) RXXM which is also negligible to the
5

---

Under review as a conference paper at ICLR 2023
memory usage. In total, method incurs only small computation or memory overhead and can thus be plugged into baseline methods in a low-cost manner. Moreover, we empirically compare the computational cost in Table O and observe that our method only incurs relatively lightweight overhead compared with the baseline methods on range of datasets.
4.1 DISCUSSION
Relation to Supervised Logit Adjustment. In supervised long-tailed learning, the optimal class- probabilities (hat (z) follow the uniform distribution, which is aligned with the underlying ground- truth distribution Poal (g) on the test set. In contrast, the surrogate label distribution p(t) in SSL encodes the positional information, which is better to be imbalanced. In our algorithm. push away (ff//z from (tar(t) by replacing the division operation of Eq. [ with multiplication, iee, Pirub x p(t))=)p(())' Although it seems that we reversely enlarges the skewness of the geo- metric prediction, the surrogate supervised learning via Eq. Slatually guarantees the space shrinking of head classes and the space expansion of tail classes.
Difference from Unsupervised Clustering, Compared with previous explorations (Asano et al., 2020 Caron et al. 2020), the uniqueness of SLLA lies in the following two aspects: (I) Geomel- ric Classifier. The pioneering works mainly resort to learnable classifier to perform clustering,
which can easily be distorted in the long-taills scenarios (Fang al.] 2021). Built on the maxi- mum separation classifier, our method is capable to provide clustering results with clear geometric interpretations. (2) Adjustable Class Prior. The class prior are assumed to be uniform among the previous attempts. When moving to the long-tailed case, this assumption will cause the undesired sample-level uniformity. In contrast, our methods can potentially cope with any distribution with proper adjustment. In Section 5.4 we empirically demonstrate the priority of our LA.
5
EXPERIMENTS
5.1
EXPERIMENTAL SETUP
Baselines. We choose four representative state-of-the-art methods: one is plain SSL method SimCLR (Chen et al., 2020) and the other three are Focal (Lin et al. (017). SDCLR (Jiang et al., 2021 and BCL (Zhou et 2022) respectively from loss, model and data perspectives.
Implementation Details. Following previous works (Jiang et al. 2021 Zhou et al., 2022). we use ResNet-18 the backbone for small-scale dataset (CIFAR-I00- ITO-LT) and ResNet-50 for large-scale datasets LimageNet-LT/Place--II
For experiments on (LFAR-100LLA we train model with the SGD optimizer for 1000 epochs with the batch size 512, momentum 0.9 and weight decay factor 5 x 10-4 and use the cosine annealing decay with the initial learning rate as 0.5. For experiments on ImageNet-LT and Places-LT, we only train for 500 epochs with the batch size 256 and decrease the weight decay factor to 1 x 10-4 Other pre-training setups like the data augmentation and projector structure follow (Jiang et al. 2021 Zhou et al. 2022). For T in SPLA, we choose from w0w.........00.....
CIFAR- 100-L and 0.02 for Places-LT and ImageNet-LT. We set the dimension K as 100 and the Sinkhorn iteration as 300. As S2IAA is combined with baselines, warm- up of 500 epochs on CIFAR-100-LT and 400 epochs ImageNet-LT and Places-LT is applied.
Evaluation Metrics. Following (Jiang et al. 2021 Zhou et al.) 2022), linear probing on balanced dataset is used for evaluation. We conduct full-shot evaluation on CIFAR-100-LL and few-shot eval- uation on ImageNet-LT and Places-LT. We report the model performance and the standard deviation among three disjoint groups, i.e.. many/medium/feww partitions (Liu et al. 2019)
5.2 SELF-SUPERVISED LONG-TAILED LEARNING PERFORMANCE
CIFAR-100-LT. In Table 1 we compare methods with and without LAA, and analyze follows.
(1) Overall performance. SLLA shows consistent gain in many/medium/ffew groups, yielding a overall performance improvements averaging 2.32%. 2.33% and 1.57% under different imbalance ratios. In particular, compared with previous state-of-the-ee BCL, with S?L, BCL further achieves improvements by 1.20%. 1.82% and 1. 1.22% CIFAR-100-IT-RR 101.TT-RR00////////11/

---

Under review as a conference paper at ICLR 2023
Table 1: Linear probing on CIFAR-100-LT with different imbalanced ratios (100,50,10). Std means the group-level performance standard deviation and Avg is the average accuracy of full test set. Std represents balancedness measure quatify the variance among three specified groups. Here, Many/Med/Few indicate the fine-grae tine-griied groups according to the class cardinality.
Method
CIFAR 100-LT-RIOO
CIFAR- RO0
CIFAR- 100 LT-RIO
Many
Med
Fow
Avg
Many
Med
Fow
Avg
Many
Med
Few
Std
Avg
SimCIR
54.97
49.39
47.67
50.72
56,00
50.48
50.12
3.30
57.85
55.06
54.03
1.98
LA
57.38
52.27
52.12
2.99
53.96
58.88
53.00
54.27
3.09
55.42
59.26
56.91
55.85
1.75
57.36
Improv.
+2.41
+2.88
+4.45
-0.82
+3.24
+2.88
+2.52
+4.15
-0.20
+3.18
+1.41
+1.85
+1.82
-0.23
+1.69
Focal
54.24
49.58
49.21
2,80
51.04
55,40
51.14
50.02
2.84
52.22
58.18
55.82
54.64
1.80
56.23
LA
57.01
52.93
51.74
2.76
53.92
57.97
53.55
53.58
2.54
55.06
60.06
56.79
57.24
1.77
58.05
Improv.
+2.77
-0.04
+2.88
+2.57
+3.56
+2.84
+0.97
-0.03
SDCLR
57.32
50.70
50.45
3.90
52.87
57.50
51.85
52.15
3.18
53.87
58.47
54.79
52.97
2.80
55.44
LS
57.44
52.85
54,06
2.38
54,81
58.47
53.88
53.58
2.74
55.34
59.21
56.06
55,58
1.97
56.97
Improv.
+0.12
+2.15
+1.94
+0.97
+2.03
+1.43
-0.44
+1.47
+0.74
+1,27
+2.61
+1.53
BCL
59.15
$4.82
55.30
2.37
56.45
59.44
54.73
$7.30
2.36
57.18
60.41
57.15
59.76
1.73
59.12
$9.50
55.73
57.67
57.65
60.82
57.58
58.55
1.66
59.00
61.41
59.27
60.30
1.07
60.34
Improv.
+0.35
+0.91
+2.37
-0.49
+1.20
+1.38
+2.85
+1.25
-0.70
+1.82
+1.00
+2.12
+0.54
-0.66
+1.22
Table 2: Linear probing on ImageNet-LT and Places-LT. Similarly, Std means the group-level per- formance standard deviation and Avg is the average accuracy of full test set. Std represents a bal- ancedness measure to quatify the variance among three specified groups Here, Many/Med/Few indicate the fine-grained partitions according to the class cardinality.
Ima
Places-LT
Method
Many
Med
Fow
Std
Avg
Many
Med
Fow
Std
Avg
SimCLR
41.69
33.96
31.82
5:19
36.65
31.98
34.05
1.83
33.61
+S
41.53
36.35
35.84
3.15
38.28
32.46
35.03
36. 14
1.89
34.33
Improv.
40.16
+2.39
+4.02
-2.04
+1.63
+0.48
+0.98
+0.51
+0.06
+0.72
Focal
42.04
35.02
33.32
4.62
37.49
31.69
34.33
35,73
2.05
33.65
LA
42:55
36.75
36.28
3.49
38.92
32.40
35. 14
36.49
2,088
34.42
Impros.
-1.13
+1.43
+0.71
+0.81
+0.03
SDCLR
40.87
33.71
32.07
4.68
36.25
32.17
34.71
69
1.82
33.99
+S
41.92
36.53
36.04
3.26
38.53
32.78
35 60
36,
1.82
34,70
Improw.
+1.05
+2.82
-1.42
+2.28
+0.61
+0.89
+0.49
0.00
+0.71
BCL
42.92
35.89
33.93
4.73
38.33
32.69
35.37
37.18
2.26
34.76
+SPLA
43.22
39.95
33.22
2.23
Improv.
+0.30
+2.27
+3.03
-1.40
+1.62
+0.53
+0.63
+0.44
-0.03
+0.56
(2) Representation balancedness. Previously, we claim LL help compress the expansion of head classes and avoid the passive collapse of tail classes, yielding more balanced representation distri- bution. To certify this aspect, we compute the differenne in performance among many/medium/feww groups, namely, their groupwise standard deviation. Our method provides 1.41% 3.4%, and /1%%///55%/1.8/
improvements w.z.. many/medium/ffww group on CIFAR-100 100-L0-OO------ RA-RITT-ON 100//50/R11 10 with more preference to the tail classes. Therefore, LALA substan- tially improves the standard deviation by 0.72/0.41// 11/20044//0444 on CIFAR-100-LT-R 10000L/T
1.95%
(3) Regarding skewness. Our method be generalized to practical distributions with varying levels of skewness as the algorithm does not refer to any information about the distribution prior.
ImageNet-LT and Places-LT. Table shows the comparison of different methods large- scale dataset ImageNet-LT and PLaces-LT, in which we have similar observations. As can be seen, on more challenging real-world data, LLLA still outperforms other methods in terms of overall accuracy, averaging as 0.58%, 0.44% on ImageNet-LT and Places-LT. Specifically, our method provides 30.%%/...%/335000 and w.F.t. many/medium//iew group on ImageNet-LT and Places-LT, The consistent performance over- head indicates the robustness of our method to deal with long-tailed distribution with different char- acteristics. Moreover, the averaging improvement of standard deviation is 1.50 on ImageNet-LT, indicating the comprehensive merits on the tail classes towards representation balancedness. How- ever, an interesting phenomenon is that the fine-grained performance exhibits a different trend on Places-LT. We observe that the standard deviation of our method does not significantly decrease on Places-LT. A possible explanation is that the head classes suffer from severer over-expansion in the scene-centric scenario. As can be seen, the performance of head classes is even worse than that of tail classes, which requires more effort and exploration for improvement alongside LA
improvements in linear probing
7

---

Under review as a conference paper at ICLR 2023
Table 3: Supervised long-tailed learning by finetuning on CIFAR-100-LT-RI00 and ImageNet-I.T. We compare the performance of four self-supervised learning methods as the pre-training stage for downstream supervised logit adjustment (Menon al. 202T method. Besides, the performance of logit adjustment via learning from scratch is also reported for comparisons.
Dutaset
LA
Logit adjustment pretrained with the following SSL methods
SimCLR
+SALA
Improv.
Hocal
Improv.
SDCLR
+S"LA
Impros.
BCL
L.ALA
Improv.
CIFAR-100-LZ
46.61
49.95
50.67
+0.72
49.87
50.98
+1.11
49.88
50.69
+0.81
50.38
50.99
+0.61
ImageNet-LL
48.27
51.10
51.61
+0.54
51.25
51.52
+0.27
51.01
51.33
+0.32
51.50
51.76
+0.26
5.3
DOWNSTREAM SUPERVISED LONG- TAILED CLASSIFICATION
In supervised long-tailed learning, self- learning has been proved to be beneficial pre-training stage to exclude the explicit bias from the class imbalance (Yang & Xu 2020 Liu et a
2021 Zhou al. 2022). In order validate the representation transferability of L', we conduct self-supervised pre-training as the initialization for downstream supervised classification tasks on CIFAR-100-LT-RIO0 and ImageNet-I.T. The state-of-the-art logit adjustment (Menon et al.) 2021 is chosen as the baseline. The combination of S + LA is interpreted as a method where S aims the re-balanced representation extraction and LA copes with the debiased classifier learning. In Table 3] we can find that the superior performance improvements is achieved by self-suuprvised pre-training over the direct supervised learning baseline. We can also observe that our method consistently outperforms other methods, averaging 0.81% 1% and 0.35% on CIFAR- 1000 and ImageNet-LT. These results demonstrate that S are well designed to facilitate long-tailed representation learning and improve the generalization ability for downstream supervised tasks.
5.4 FURTHER ANALYSIS AND ABLATION STUDIES
Optimal Transport. We conduct a range of experiments verify the effectiveness of our de- sign, which includes ([-1) the logit-adjusted loss with soft assignments, (I-2) the logit-adjusted loss with hard assignments, (I-3) post-hoc logit-adjusted loss with soft assignments, (I-4) post-hoc logit- adjusted loss with hard assignments. Besides, we also add two methods: (J) joint optimization on p, q and (O) oracle ground-trute guidance. From the results, both the logit-adjusted and the post-hoc logit-adjusted manners (I1-14) lead to performance degradation compared with the vanilla SimCLR, while the label allocation by optimal transport LS*LA) significantly improves the performance.
Diverse Pre-defined Skewed Constraints. To further validate S*LA we alternate the es- timated constraints with the fixed imbalanced prior based on the ground-truth label distribu- tion. We explore the effect of the imbalanced prior by adding a temperature factor T, which softens the prior to various scale of skewness. As shown in Figure 3(a) we observe that S outperforms all models pretrained with the im- balanced prior. A possible reason is that our method captures the inherent geometric statistics from the embedding space, which is different from the the label space. This is aligned with the empirical results that the model achieves the worst performance with the vanilla ground-truth prior (I' = 1), whereas achieving the best performance with the prior closest to SPLA (T 0.6). Moreover, we observe that the uniform prior (T = 0) adopted in (Asano et al., 2020 show the expected performance on the long-tailed data.
Table 4: Linear probing results w.rl different se- tups for surrogate labels.
Acronym SimCLR
S*LAA
Accuracy 50.72
2.91
53.96
56.47
Acronym I-1
1-2
1-3
I-4
Accuracy
48.41
49.93
50.38
50.22
Caron et al. (020) does not
Embedding Statistics. In Figure 3(c) we plot the prediction histogram at two stages in the training process, i.e., warm-up stage and the end of training stage. We observe that the adjusted distribution more imbalanced than the vanilla distribution. Specifically, it shows more samples to the instance-rich classes with the shrinking of instance-poor classes. This is aligned with the goal to compress the feature span of head classes and encourage the expansion of tail classes.
assigned
Hapee--paraeetr 7 for L We can adjust the coefficient т in Eq. 3 to control the direction and strength of S'LA, as shown in Figure 3(b) For each т, perform performance evaluation and skewness estimation, providing the inherent characteristics of our method. From the figure, the correlation between the coefficient and the embedding skewness indicates that can ad-
8

---

Under review as a conference paper at ICLR 2023
T=
T=:02
T=:0d
T=dR Tia SPLA
$4.4
Skewnes nesw
Accuracy
1.0
500
Adjustod
Accuracy
54.5
T=0.6
54.2
Sample Number
400
54.0%
Acc
34.0
300
53.5
53.0
Acc:
53.6
200
525
51.4
532
Imbalanced factor
15 2940
100
-0.02
0.30
0.04
20 40
Index
60
50
100
Pre-defined Prior
(b) Ablation studies on 7
(c) Embedding Statistics
Figure 3: (a) Linear probing performance with error bars w.E. the ground-truth prior served as the categorical constraints. We soften the oracle distribution by T to obtain diverse imbalanced priors and report the corresponding imbalanced factor. (b) Linear probing performance and imbalanced factor of the geometric statistic w.r.t. on CIFAR-I00 1-0--T------II 100. (c) Histograms of geometric predictions at the end of the warm-up stage (vanilla) or at the end of training (adjusted).
Table 5: Ablation study on the CIFAR-100-LT, ImageNet-LT and Places-LT.
Table 6: The time cost of mini-batch training (seconds) on
Simplex ETR.
Method
R100
R10
CIFAR-LT
ImageNet-LT
Places- LT
SimCLR
50.72
52.24
55.67
Si
0.379
0.407
0.757
0.787
0.724
0.747
LS
53.96
55.42
57.36
0.423
0.467
0.943
1.008
1.001
1.050
LS'LA(RC)
53.41
54.78
56.54
SDCLR(4S)LA)
0.374
0.397
0.752
0.771
0.759
0.776
+Simplex ETF
51.10
51.99
55,56
BCL(+S²LA)
0.377
0.405
0.756
0.783
0.722
0.745
just the skewness flexibly. Considering t 0 as an anchor point, we can achieve the promising performance with proper strength in the process of increasing skewness. In contrast, the model performance degenerates when we reversely adopt SºLA (decrease skewness). These observations further confirm that our method can realize any desired skewness deal with different levels of data imbalancedness.
Ablation Study on Simplex ETF, To further verify the effectiveness of Simplex ETF, we conduct experiments to investigate our methods with the random linear classifier. In Table we can see that the random classifier SimCLR, but fails to outperforms Simplex ETF. Besides, we also conduct experiments with Simplex ETF the projector. As be seen, if Simplex ETF alone is used to balance the representation learning, the improvement is minor and sometimes degrades. This is because the direct estimation from the Simplex is noisy during training when the representation is not ideally distributed.
measure under the framework of LA does improve the performance of
Computational cost. In Table 6 we provide the mini-batch training time of different methods on 1IFAR-100-L1) ImageNet-I.T and Places-I.T. In our runs, the proposed LLL incurs average 7.8%, 4.2%, 3.4% computational overhead on CIFAR-100-LT, ImageNet-LT and Places-LT, respec- tively, which is relatively lightweight compared to the computational cost of deep neural networks.
6 CONCLUSION
In this paper, we study why the conventional contrastive learning loss underperforms in self- supervised long-tailed learning. motivating our exploration on category-level uniformity instead of sample-level uniformity. Inspired by the power of logit adjustment in supervised long-tailed learning, we correspondingly propose a Self-Superivsed Logit Adjustment algorithm to calibrate the learning in the self-supervised lcarning paradigm. On the geometric level, S²LA gradually rectify the feature span by an alternation between the logit adjustment and the surrogate label allo- cation. Moreover, S2LA is orthogonal to existing self-supervised long-tailed methods and can be easily plugged into these methods in an efficient manner. Extensive experiments demonstrate the consistent efficacy of our proposed LA. We believe that the geometric perspective has the more potential to understand self sulf-supe learning methods, especially when coping with the long- tailed scenarios. In the future, will extend work with several potential directions such as theoretical definition the geometric structure over the training embeddings, and investigations of other clustering methods regularization edit the geometric statistics.

---

Under review as a conference paper at ICLR 2023
7 ETHICS STATEMENT
This paper does not raise any ethics concerns. This study does not involve any human subjects, practices to data releases, potentially harmful insights, methodologies and applications, poten- tial conflicts of interest and sponsorship, iiscrimination/iisffaainn concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of experimental results, we will provide the anonymous repository about our codes in the discussion phase for reviewing purposes. The experimental setups for training and evaluation as well as the hyper- hyper-parameters are detailed in S.. Appendix D
REFERENCES
Yuki M. Asano, Christian Rupprecht, and Andrea Vedaldi. Self-labelling via simultancous clustering and representation learning. In International Conference Learning Representations (ICLR), 2020.
Sergiy V Borodachov, Douglas P Hardin, and Edward B Saff. Discrete energy on rectifiable sets. Springer, 2019.
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international conference on pattern recognition, pp. 31 3121-3124. IEFF, 2010.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with aubel-distribution-awaas margin loss. Advances in Neural Information Processing Systems, 32:1567-1578. 2019.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 33:9912-9924, 2020.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference Machine Learning, pp. 1597-1607. PMIR, 2020.
Henry Cohn and Abhinav Kumar. Universally optimal distribution of points on spheres. Journal of the American Mathematical Society, 20(1):99-148, 2007.
Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 715-724, 2021.
Marco Cuturi. Sinkhorn distances: Lightspeed computttion of optimal transport. Advances in neural information processing systems, 26, 2013.
Carl Doersch, Abhinav Gupta, and Alexci A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision, pp- 1422-1430, 2015.
Cong Fang, Hangfeng He, Qi Long, and Weijie J Su. Exploring deep neural networks via layer- peeled model: Minority collapse in imbalanced training. Proceedings of the National Academy of Sciences, 118(43):e210391 18, 2021.
Florian Graf, Christoph Hofer, Mare Niethammer, and Roland Kwitt. Dissecting supervised con- strastive learning. In International Conference on Machine Learning, pp. 3821-3830. PMLR, 2021.
DP Hardin and EB Saff. Minimal riesz energy point configurations for rectifiable d-dimensional manifolds. Advances in Mathematics, 193(1):17422)) 2005.
10

---

Under review as a conference paper at ICLR 2023
Ziyu Jiang, Tianlong Chen, Bobak J Mortazavi, and Zhangyang Wang. Self-damaging contrastive learning. In International Conference on Machine Learning, pp. 4927-4939. PMLR, 2021.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint MrXivv/910.0925 2019.
Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng. Exploring balanced feature spaces for representation learning. In International Conference on Learning Representations, 2020.
Tejaswi Kasarla, Gertjan J Burghouts, Max van Spengler, Elise der Pol, Rita Cucchiara, and Pascal Mettes. Maximum class separation as inductive bias in one matrix. arXiv preprint arXiv:2206.08700 08704, 2022.
Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio Feris, Piotr Indyk, and Dina Katabi. Targeted supervised contrastive learning for long-tailed recognition. arXiv preprint arXiv:2111.13998, 2021.
Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio S Feris, Piotr Indyk, and Dina Katabi. Targeted supervised contrastive learning for long-tailed recognition. Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 6918-6928, 2022.
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988, 2017.
Hong Liu, Jeff Z HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning is more robust to dataset imbalance. arXiv preprint arXh::2110.05023 2021.
Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learning towards minimum hyperspherical energy. Advances in neural information processing systems, 31, 2018
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large- scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2537-2555 2019.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In International Conference on Learning Representations, 2021.
Dustin Mixon, Hans Parshall, and Jianzong Pi. Neural collapse with unconstrained features. Sampling Theory, Signal Processing, and Data Analysis, 20(2):1-13, 2022.
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40): 24652-24663,, 2020.
Foster Provost. Machine learning from imbalanced data sets 101. In Proceedings of the AAAI'2000 workshop on imbalanced data sets, volume 68, pp. 1-3, AAAI Press, 2000.
William J Reed. The pareto, zipf and other power laws. Economics letters, 74(1):15-19 2001.
Jiawei Ren, Cunjun Yu, Xiao Ma. Haiyu Zhao, Shuai Yi, et al. Balanced meta-softmax for long- tailed visual recognition. Advances neural information processing systems, 33:4175-4186, 2020.
Jiawei Ren, Mingyuan Zhang, Cunjun Yu, and Ziwei Liu. Balanced mse for imbalanced visual regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 7926-7935, 2022.
Steve Smale. Mathematical problems for the next century. The mathematical intelligencer, 20(2): 7-15, 1998.
11

---

Under review as a conference paper at ICLR 2023
Alexander Strehl and Joydeep Ghosh. Cluster ensembles- knowledge reuse framework for com- bining multiple partitions. Journal of machine learning research, 3(Dec):583-611 617, 2002.
Joseph John Thomson. Xxiv. on the structure of the atom: an investigation of the stability and periods of oscillation of number of corpuscles arranged equal intervals around the cireumference of circle; with application of the results to the theory of atomic structure. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 7(39):23-2265, 1904.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International Conference Machine Learning, pp- 9929-9939. PMLR, 2020.
Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In Proceedings of the IEEE International Conference Computer Vision, pp. 2794 2802, 2015.
Yibo Yang, Liang Xie, Shixiang Chen, Xiangtai Li, Zhouchen Lin, and Dacheng Tao, Do really need a learnable classifier at the end of deep neural network? arXiv preprint arXiv:2203.0900 2022.
Yuzhe Yang and 7hi Xu. Rethinking the value of labels for improving class-imbalanced learning- Advances in neural information processing systems, 15119290-19301. 2020.
Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep long-tailed learning: A survey. arXiv preprint arXiv: 10.04596, 2021.
Zhihan Zhou, Jiangchao Yao, Yan-Feng Wang, Bo Han, and Ya Zhang. Contrastive learning with boosted memorization. In International Conference on Machine Learning, pp. 27367-27377. PMLR, 2022.
Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu A ge- ometric analysis of neural collapse with unconstrained features. Advances in Neural Information Processing Systems, 01-29820-29833 2021.
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
A ALGORITHM
We summarize the complete procedure of our S2LA method in Algorithm
Algorithm Our proposed LA.
Input: dataset D, number of epochs E, number of warm-up epochs E geometric classifier M, self-supervised learning method A, logit adjustment temper
Output: pretrained model parameter O Initialize: model parameter 80
1: Warm up model 0 for Ew epochs according to A.
2: for epoch e = Ew, En + l..... do
3: Obtain the geometric prediction t arg max; (x)) for dataset D.
4: Compute the surrogate class prior p(t).
5: Compute (t) using logit adjustment according to Eq.
6:
for mini-batch batch & 1, 2, ... B do
7:
Update the output logits h-'(a) for each sample.
8:
Obtain the label qt xx) using the optimal transport solutions according Eq. 6
9:
Compute LSst. according to A and the proposed LEaLA according to Eq.
Uptate model 0 by minimizing LSst. LSLLA
11: end for
end for
B RELATED WORKS: SUPERVISED LONG-TAILED LEARNING
As the explorations on the classifier learning are orthogonal to the self-supervised learning paradigms, we mainly focus on the representation learning in supervised long-tailed recognition. The pioneering work (Kang et al., 2019) first explored representation and classifier learning with disentangling mechanisms and showed the merits of restance-balancee sampling strategy on the representation learning stage. Subsequently, (Yang & Xu 2020) pointed out the negative impact of label information and proposed to improve the representation learning with semi-supervised learning and self-supervised learning. This motivates stream of research works diving into the represen-
tation learning. Supervised contrastive learning (Kang al. 2020 Cui et al., (02I)) I) is leveraged with rebalanced sampling or prototypical learning design to pursue a more balanced representation space. (Li al. (02T) explicitly regularizes the class centers to maximum separation structure with similar drives to the balanced feature space.
C ON CLASS-BALANCED DATA
According the proof in WWang & Isola 2020). conventional contrastive learning targets to pur- sue the sample-level uniformity. Given the balanced feature subspace, we can naturally obtain the uniform predictions p(t)
based on Simplex ETF, and the logit adjustment will not modify the distribution skewness as f. arg max p(t (t)) arg max p(t]==) = In this case, is reconcilable between contrastive learning and the second term in Eq.(7), only if optimal transport makes the reallocated labels of samples certain and uniformly distributed regarding the categories. The loss term - =+t(==)) log p(t (z) will then make negligible effect the representation learning and the contrastive learning domainates the optimization.
Empirically, conduct experiments in the balanced setting shown in Table 7 From the results, we can see that SLLA shows comparable performance with the baseline methods the balanced setting. This observation is consistent with the aforementioned theoretical analysis.
13

---

Under review as a conference paper at ICLR 2023
Table 7: Linear probing results in the balanced setting CIFAR-I00-LT-RRI0.,
Method
SimCLR
LS
Focal
+SPLA
SDCLR
+SP
BCL
+SPLA
Accuracy 66.75
66.41
66.42
66.79
65.96
66.17
69.16
69.33
D EXPERIMENTAL DETAILS
D.1 DATASET STATISTICS
We conduct experiments on three benchmark datasets for long-tailed learning, including CIFAR- 100-LT (Cao al. 2019), ImageNet-LT (Liu et al. 2019) and Places-LT (Liu al.) 2019). For small-scale datasets, we adopt the widely-used CIFAR R- R-100-LT with the imbalanced factor 100, 50 and 10 (Cao et al. 2019).
In Table we summarize the benchmark datasets used in this paper. Long-tailed versions of CIFAR- 100 constructed following the exponential distribution. For large-scale datasets, ImageNet- LT (Liu et al. 2019) has 115.8K images with 1000 categories, ranging from 1,280 to 5 in terms of class cardinality and Places-LT (Liu et al. 2019) contains 62,500 images with 365 categories, with the sample number per category ranging from 4,980 to 5. The large-scale datasets follow Pareto distribution.
Table 8: Statistics of long-tailed datasets. Exp represents exponential distribution.
Dataset
Class
Type
Imbalanced Ratio
Train data
Test data
CIFAR 100-L-O00-
100
Exp
100
10847
10000
CIFAR-I00-LT-R50
100
Exp
50
12608
10000
CIFAR-100-LT-RI
100
Exp
10
19573
10000
ImageNet-LT
1000
Pareto
256
115846
50000
Places-LT
365
Pareto
996
62500
36500
D.2 LINEAR PROBING STATISTICS ON LARGE-SCALE DATASET
The 100-shot evaluation follows the setting in previous works (Tang et 2021 Zhou al. 2022
As shown in Table 9 full-shot evaluation requires 10x 30x the amount of data compared with the pre-training dataset, which might not be very practical. contrast, the scale of 100-shot data is consistent with the pre-training dataset. We also present full-shot evaluation in Section E.5
Table 9: Statistics of linear probing on large-scale dataset.
Dataset
Class
# Training data
# 100-shot data
# full-shot data
# Test data
ImageNet-LT
1000
115,846 62,500
100,000 36,500
1,261,167 1,803,460
50,000 36,500
Places-LT
365
D.3 IMPLEMENTATION DETAILS
Toy Experiments. (Figure I We 2-Layer ReLU network with 20 hidden units and 2 output units for visualization. The SimCLR algorithm (Chen et al., (2020) is adopted in the warm-up stage with proper Gaussian noise augmentation. After the warm-up stage, train SLLA according 7 Due to the dimensional constraints, we use the orthogonal classifier (0,,))(11)))611))))0.11.)) the suboptimal structure for maximum separation.
Linear Probing Evaluation. We follow (Zhou et al. 2022 to conduct Adam optimizer for 500 epochs based on batch size 128, weight decay factor 5 10 and the learning rate decaying from
14

---

Under review as a conference paper at ICLR 2023
10-2 to 10-6. 6 For few-shot evaluation on ImageNet-LT and Places-LT, we use the same sampled 100-shot subsets in (Zhou et al. 2022).
Computing Infrastructure. Our codes are built PyTorch. We trained all the experiments NVIDIA GeForce RTX 3090.
D.4 FOCAL LOss
Focal loss [Lin al. 2017) is discussed and compared in (Jang al. 2021 Zhou al. 2022 in the context of supf--upe long-tailes learning. Specifically, we use the term inside log(.) of SimCLR loss the likelihood replace the probabilistic term of the supervised Focal loss and obtain the self-supervised Focal loss as:
exp (fo(æ)) (((()t))/)
facol
Ex-Ex, log(p); p Ex Ex, Ufat] exp (fo(x) fo(x=)/7) #EA
E1-
A
where Tf is temperature factor. We defaultly 7, in all experiments.
E MORE COMPREHENSIVE RESULTS
E. ABLATION STUDY ON OPTIMAL TRANSPORT
In the following, conduct range of experiments verify the effectiveness of our design, which includes (I-1) the logit-adjusted loss with soft assignments, (I-2) the logit-adjusted loss with hard assignments, (I-3) Post-hoc logit-adjusted loss with soft assignments. (I-4) Post-hoc logit-adjusted loss with hard assignments. Besides, we also add two methods: joint optimization on p, and oracle ground-truth guidance.
Table 10: Ablation study on optimal transport.
Method
Acronym
Formulation
Accuracy
SimCLR
50.72
+Joint Optimization
minp.
P Exeo ((t))) logp(t|z))
2.91
+Logit-adjusted softmax (soft)
1-1
minp
L p(t(z)) log pl p())))((())
48.41
+Logit-adjuse adjusted softmax (hard)
1-2
minp
m tlogp(t|=)//ll
49.93
P
+Post-hoc (soft)
I-3
min,
aa p(t|)) ((t(())))
50.38
+Post-hoe (hard)
1-4
minp
Znn d fog p(t(z)
50.22
LA
minpa
LanD ((t)z)) log logp(t)()
53.96
+Oracle
minp (x.g)) ((tp))))
56.47
Table 11: Normalized mutual information (NMI) score between the geometric predictions and the oracle labels.
Prediction
i with Post-hoc (CE) 0.056
q(t|æ::) with SLLA
NMI score
0.432
As can be seen, both the logit-adjusted losses and the post-hoc logit-adjusted manners (I1-I4) lead to performance degradation compared with the vanilla SimCLR, while the label allocation by optimal transport LS* significantly improves the performance. To explain this, we compute the NMI score between & and the oracle label, and the NMI score between q({(x;) and the oracle label, sum- marized in Table [1 Accorrding to the comparison of their NMIs, we can understand the roughly adjusted & can be noisy and shows the weak correlation with the oracle label, while q(t(æ; reallo- cated by optimal transport based the population statistic of i effectively improves the correlation.
15

---

Under review as a conference paper at ICLR 2023
E.2 ON DIMENSION OF MAXIMUM SEPARATION STRUCTURE
For Simplex ETF, there is hard dimension constraint in Eq. ie., d < K. However, if this constraint violates, we do not have such structure in the hyperspherical space. Alternatively, we conduct the gradient descent to find approximation of the maximum separation prototypes applied into LA. This refers to minimising the following loss function as demonstrated in LLi
al. 2022
K log Σ>
K
Luniform
m. s.t. Σ m. =0 and Vi E K ||m
(8)
where the first term penalizes the pairwise similarity of different prototypes (Wang & Isola 2020) under the constraints of Simplex ETF. Figure 4 shows the performance of S"LA equipped with different dimension of the geometric structure by analytical and approximate ways. Comparing the
results of Simplex ETF with those of proxy weights, we can see that the comparable performance is achieved in both geometric structures. This indicates that our method is effective to two forms of the geometric structure, relaxing the hard dimensional constraints in Simplex ETF.
Simplex ETF Approximate
53
Accuracy
30
50
60
70
80
101101201304411110 10010000000011000 200210200
Dimension
Figure 4: Linear probing performance w.r.t. the dimensionality d of the pre-defined classifier on 1EAAR-000LLTRR10 We use simplex ETF structure when < or otherwise the approximation learned via Eq. g
E.3 CLUSTERING QUALITY
We track the normalized mutual information (Strehl Ghosh 2002 of the geometric predictions with the ground-truth label in the training stage CIFAR- LIFAR-IO0-LT IOO-LT. As shown in Figure 5 we ob- serve that SLLA significantly improves the NMI score across different methods, indicating that the distorted embedding space calibrated better capture the semantic information. Moreover, find that the NMI improvements of the existing works are marginal compared with LLL,, which verifies the importance of calibrating the distorted embedding space.
SimCLR
0.45
BCL
NMI Score
0.40
SimELR-S'LA
Focai+5'LA
0.35
SDCLR+5TLA
BCL4SRLA
0.30
0.25
0.20
20o
400
600
800
1000
Epoch
Figure 5; Normalized mutual information (NMI) between the geometric predictions and the ground- truth labels throughout training.
16

---

Under review as a conference paper at ICLR 2023
E.4 COMPARISONS WITH SELA
In this part, we conduct empirical comparisons with SeLa (Asano et al. 2020). In the following, we the official code of SeLa with the default training settings (marked SeLa), with our training schedule (marked as SeLa*) and conduct the experiments on LEFAR-100-LT--E 100. From the results, can see that our method can improve the performance of ScLa.
Table 12: Linear probing results of SeLa on CIFAR-I00-LT-RIO0,
Method
SeLa Accuracy 44.45
ScLa* 46.47
SeLa* S
48.10
E.5 FULT-SHOT EVALUATION ON LARGE-SCALE DATASET
Here we provide 100-shot evaluation and full-shot evaluation on ImageNet-LT, as shown Table 13 We observe that the performance improvements and representation balancedness (Std) are consistent with the evaluations, indicating the rationality of the 100-shot evaluation.
Table 13: Full-shot evaluation and 100-shot evaluation on ImageNet-Lii
Evaluation
Method
Many
Medium
Few
Std
Avg
SimCLR
41.69
33.96
31.82
5.19
36.65
100-shot
+S
41.53
36.35
35.84
3.15
38.28
SimCLR
42.86
35.17
33.13
5.13
37.86
Full-shot
LS
44.11
38.59
37.87
3,41
40.62
17


Reviews:
[{"review_id": "svWXd1qxe8L", "reviewer": "Reviewer_jKEp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Given the research progress on logit adjustment for supervised long-tailed learning, this paper presents a new method S^2LA to conduct self-supervised long-tailed learning  from the geometric perspective. In fact, without knowing the class distribution, S^2LA uses a constant simplex ETF to measure the geometric characteristics of the embedding space. Using a surrogate label allocation to represent the target distribution, this paper intends to compress the space of the head classes whilst avoiding the collapse of the tail classes. Treating the above transformation as a balancing and optimal transport problem, the proposed solution is intended to effectively handle the category-level uniformity problem. ", "summary_of_the_review": "In summary, this paper presents an interesting topic, worth of being investigated. The theoretical analysis sounds reasonable but lacks certain explanations to the proposed objective function and its derivation. The experimental results are quite considerate but some of the systematic parameters have not been properly justified in the examination.", "strength_and_weaknesses": "Strengths - \n\n1. The presentation of the proposed method is generally understandable. The discussion on the theoretical analysis and the experimental is clear. \n2. The balance between the proposed approach and the evaluation section look reasonable. Many results have been provided to support the arguments in the theoretical analysis.\n3. The mathematical principles and basics have been clearly defined and explained.  \n\nWeaknesses - \n\n1. The major concern is that the proposed approach seems incremental, compared to the established works and similar technologies. \n2. The evaluations seem to support the arguments that the proposed technology outperforms the other state of the art methods, however, the justification is not comprehensive and requires much additional evidence to support the overall statements.\n3. The experimental results show that the proposed algorithm can only achieve slightly better performance (often < 1%) than the other state of the art technologies.\n\nActions to be taken:\n\n1. Since there is no ground-truth annotation of each sample, and using geometric labels tends to generate trivial unconstrained probabilities, a surrogate label allocation with logit adjustment is used in this paper. Eq. (4) is proposed and then needs to be solved using Eqs. (5)-(6). Finally, the overall objective function is expressed as Eq. (7). However, Eq. (7) has a from similar to that of Zhou et al. (2022). The only difference between them is that the current paper introduces Eq. (4) to its final objective function, which plays as a constraint term. It is not clear if or not this objective function is correctly derived in this paper. If it is, how much difference between the two papers indeed?\n2. In the ablation experiments, Fig. 4(c) is not of much value. This is because presenting different distributions without showing the classification results is not convincing.\n3. In Fig. 5, increasing skewness also leads to higher accuracy. However, the improvement of accuracy is fairly mild, normally less than 1%. If this is a successful mechanism, the increment should be significant, is not it? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Hx8rQlr9-d4", "reviewer": "Reviewer_bvPJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed to solve the self-supervised imbalanced learning by enforcing category-level uniformity instead of sample-level uniformity.", "summary_of_the_review": "The idea of the paper is new and comes with a clear motivation, and the experiment results seem sufficiently strong. However, I find the method part of the paper very difficult to understand which heavily affects my assessment of its correctness. I believe that the method part can be clearer with more polishing efforts, but based on the current version, I would recommend a weak rejection.", "strength_and_weaknesses": "Strength:\n\n- The paper presents a strong motivation, which is to achieve class-level feature uniformity instead of instance-level feature uniformity. The overall method is closely built around this motivation. The toy example in Figure 1. is a good illustration of the paper's idea. The idea is new to the field.\n\n- The empirical results are sufficiently strong. Improvements are observed on multiple datasets (CIFAR-LT, ImageNet-LT, and Places-LT) on top of several SSL methods.\n\n\nWeakness:\n\n- I find the method part convoluted:\n\n  - I have difficulty understanding the importance of Simplex ETF. To my understanding, Simplex ETF is a fixed linear classifier that outperforms the standard learnable linear classifiers in end-to-end __supervised__ imbalanced learning. However, why is it necessary in the unsupervised setting? Should other alternatives to \"measure the geometric characteristics\", e.g., a fixed random linear classifier, be discussed and compared?\n\n  - The motivation for logit adjustment is not very clear. Why is logit adjustment required on top of Simplex ETF? \"All vectors in a Simplex ETF have the same pair-wise angle\", is Simplex EFT alone sufficient to ensure the class-level uniformity already?\n\n  -  Why is Eqn. (4) the objective? Is it to find some surrogate labels $q(t|x)$ to compute the cross-entropy loss with $p(t|x)$, so that the marginal $p(t)$ can be adjusted to $p(\\tilde{t})$? This part may need more clarity.\n\n  - The notions are not clear sometimes. Should the expression $p(\\tilde{t}|x) \\propto p(t|x)p(t)^{1+\\tau}$ be $p(\\tilde{t}|x) \\propto p(\\tilde{t}|x)p(\\tilde{t})^{1+\\tau}$? $q$ is also not defined in Eqn. (4) until later. \n\n  - __Overall__, I believe that the mixing of statistical interpretations (logit adjustment, Bayes) and geometric interpretations (Simplex ETF, geometric span) is what makes the method part hard to understand. Currently, I don't see how the two interpretations can be linked, i.e., the relation between \"skewed geometric distribution\" and the \"skewed statistical distribution\". If the authors are just trying to \"shrink the feature span of head classes for the category-level uniformity\", I would suggest the authors stick to the geometric interpretations and view \"logit adjustment\" as a margin loss.\n\n- I may have missed the part about how focal loss is used for self-supervised learning. It would be better if more related works besides Lin et al. 2017  can be provided.\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GSZh01hG92", "reviewer": "Reviewer_1EQS", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of self-supervised feature learning for downstream tasks where the unsupervised dataset follows a long-tail distribution.\nIn particular, it seeks to address the issue of head classes covering much more of the feature space than the tail classes (expansion/collapse).\nThe proposed method is to obtain initial surrogate class likelihoods for each example, transform these by applying reverse post-hoc logit adjustment based on the surrogate class frequencies, and then obtain final (soft) surrogate labels by solving an optimal transport problem with the Sinkhorn-Knopp algorithm.\nThe initial surrogate class likelihoods are obtained using a Simplex ETF (linear, constant, orthogonal) classifier and the final confidence of the assigned surrogate label is used to weight an additional loss term that encourages the Simplex ETF to be more confident.\nThe marginal likelihoods of the surrogate labels are recomputed every epoch, and the logit adjustment and optimal transport are computed in every minibatch.\nEmpirical studies show that this technique generally improves the accuracy of several baseline algorithms (SimCLR, Focal, SDCLR, BCL) across the Many, Med and Few class subsets for several datasets with a wide range of imbalance (CIFAR100-LT-10, -50, -100, ImageNet-LT, Places-LT).\nIt is applied after an initial warmup phase that uses the baseline algorithm alone.", "summary_of_the_review": "While the results are strong, I find that the description and motivation of the method to be very unclear. I am leaning negative but I'll consider upgrading my review if these concerns can be mitigated in the process of the rebuttal and discussions.", "strength_and_weaknesses": "**Strengths**\n\n1. Dealing with long-tail data in unsupervised pre-training seems like an important and difficult problem which has not received much attention. Indeed, it's hard to even measure the tail since the dataset does not have labels.\n1. Including focal loss as a baseline was good as it may help with class imbalance.\n1. The proposed method is shown to significantly complement a wide range of algorithms for a wide range of datasets.\n1. The method does not incur a high computational cost.\n\n**Weaknesses**\n\n1. The motivation and description of the Simplex ETF classifier, the transformation of its predictions to obtain surrogate labels and the additional loss term is extremely unclear. Similarly, the motivation for applying logit adjustment in reverse is unclear. If I understood correctly, the idea is to make the predictions of head classes even more confident, such that they are moved further from their (fixed) decision boundary. It might be possible to achieve this more elegantly using the logit-adjusted softmax from that same paper (rather than post-hoc logit adjustment)? Also, could the single $q \\log p$ term in the loss be replaced with the more familiar cross-entropy over all labels $t$?\n1. It wasn't clear what the \"trivial unconstrained probabilities\" and \"trivial solution\" to eq. 4 were. One-hot distributions? This was used to motivate the use of Optimal Transport. Can't we just use the (normalized) logit-adjusted likelihoods $p(\\tilde{t})$ as targets? Does this cause the method to fail?\n1. I found it surprising that the proposed approach achieved significant gains for the \"Many\" subset as well as the \"Few\" subset. This was not explained. Perhaps the method is achieving its improvement by some mechanism other than accounting for imbalance? Perhaps it's accounting for imbalance in some unlabelled attribute? This might be revealed by inspecting the clusters. Is it possible to run some experiments in the balanced setting and see if the method still yields gains?\n1. No empirical comparison to the method of Asano et al. (SeLa), which does consider the imbalanced setting.\n1. The values for $\\tau$ seemed much lower than in the logit adjustment paper (0.02-0.07 vs. 1-4). How can this be explained?\n1. While the method is claimed to be end-to-end, it involves non-differentiable computations in each epoch and in each mini-batch.\n1. Does 300 iterations of Sinkhorn-Knopp really not have a significant impact on the training time? How long does this take compared to the typical time for a minibatch?\n1. The number of training epochs seemed very high (e.g. for CIFAR-100-LT, 1000 epochs plus 500 warmup epochs). Is this typical for self-supervised training?\n1. Figure 4(a), which shows a comparison to temperature scaling, would benefit from error-bars, since the values seem noisy and S2LA is not that far from the baseline.\n1. Related work section does not include discussion of Supervised Long-tail Learning. For example, the paper \"Decoupling Representation and Classifier for Long-Tailed Recognition\" seems relevant.\n1. Why was 100-shot evaluation used for ImageNet-LT and Places-LT? Wouldn't this impair the accuracy of the \"Many\" classes compared to a linear classifier trained with all examples?\n1. While Algorithm 1 in the appendix makes the method much more clear, it shouldn't be necessary to refer to the appendix to understand the method. For example, it wasn't clear in the main text that the surrogate class prior was re-estimated once per epoch.\n\n**Nitpicks**\n\n1. Self-supervised learning includes some methods that have labels (i.e. automatically generated labels). The paper often uses \"self-supervised learning\" to refer to *contrastive* methods exclusively. It would be good to be more clear.\n1. The acronyms BCL and SDCLR were not defined.\n1. Needs to be proofread for grammar.\n1. I was confused by the use of bold characters for the single labels $\\boldsymbol{t}$ and $\\tilde{\\boldsymbol{t}}$. This is typically used to denote a vector.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0uMdHyDscUm", "reviewer": "Reviewer_EKLC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper addresses an important problem of self-supervised learning on data with imbalanced classes. SSL trained on sample-level uniformity will learn representation for common classes. However, less represented classes will suffer due to bias for \"head\" classes. The paper introduces a method to achieve category-level uniformity by using surrogate classes. A distinct property of the method is that it can be applied to any SSL method by modifying the loss function along with the training procedure. Authors demonstrate strong empirical results of the method.", "summary_of_the_review": "After addressing the feedback, communication of ideas somewhat improved. I am giving a conditional acceptance, if authors can add a discussion on the limitations of the approach. Such a discussion will bring forth implicit assumption where the method expected to work.\n", "strength_and_weaknesses": "Strength:\n1. An important problem of SSL on imbalanced classes.\n2. An original idea of using surrogate classes to prevent bias due \"head\" classes.\n3. Substantial empirical results.\n4. Authors share code for reproducibility.\n\nWeaknesses:\n1. Mathematical exposition of the material is poor.\n2. Ideas are not communicated clearly. \n3. There is no discussion on the technique's limitations.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject