Improving Barely Supervised Learning by Discriminating Unlabeled Samples with Super-Class
Authors: Guan Gui
Zhen Zhao
Lei Qi
Luping Zhou
Lei Wang
Yinghuan Shi
Venue: NeurIPS 2022 Accept (2022)
Keywords: barely supervised learning

Abstract: In semi-supervised learning (SSL),  a common practice is to learn consistent information from unlabeled data and discriminative information from labeled data to ensure both the immutability and the separability of the classification model.  Existing SSL methods  suffer from failures in barely-supervised learning (BSL), where only one or two labels per class are available, as the insufficient labels cause the discriminative information being difficult or even infeasible to learn. To bridge this gap, we investigate a simple yet effective way to leverage unlabeled samples for discriminative learning, and propose a novel discriminative information learning module to benefit model training. Specifically, we formulate the learning objective of discriminative information at the super-class level and dynamically assign different classes into different super-classes based on  model performance improvement. On top of this on-the-fly process, we further propose a distribution-based loss to learn discriminative information by utilizing the similarity relationship between samples and super-classes. It encourages the  unlabeled samples to stay closer to the distribution of their corresponding super-class than those of others. Such a constraint is softer than the direct assignment of pseudo labels, while the latter could be very noisy in BSL. We compare our method with state-of-the-art SSL and BSL methods through extensive experiments on standard SSL benchmarks. Our method can achieve superior results, \eg, an average accuracy of 76.76\% on CIFAR-10 with merely 1 label per class.

Improving Barely Supervised Learning by Discriminating Unlabeled Data with Super-Class
Guan Gui Nanjing University
Zhen Zhao University of Sydney zhen zhao@sydney. edu au
Lei Qi
Southeast University qilei@seu. edu. cn
guiguandsmail nju. edu. en
Luping Zhou
Lei Wang
Yinghuan Shi" Nanjing University syh@nju. edu. cn
University of Sydney luping. zhou@sydney. edu. au
University of Wollongong leiw@uow edu.au au
Abstract
In semi-supervised learning (SSL), common practice is to learn consistent infor- mation from unlabeled data and discriminative information from labeled data ensure both the immutability and the separability of the classification model. Exist- ing SSL methods suffer from failures in barely-supervised learning (BSL), where only one or two labels per class are available, as the insufficient labels cause the dis- criminative information be difficult or even infeasible to learn. To bridge this gap, we investigate a simple yet effective way to leverage unlabeled data for discrimi- native learning, and propose novel discriminative information learning module to benefit model training. Specifically, we formulate the learning objective of discriminative information the super-class level and dynamically assign different categories into different super-classes based on model performance improvement. On top of this on-the-fly process, we further propose a distribution-base loss learn discriminative information by utilizing the similarity between samples and super-classes. encourages the unlabeled data to stay closer to the distribution their corresponding super-class than those of others. Such constraint is softer than the direct assignment of pseudo labels, while the latter could be very noisy in BSL. We compare method with state-of-tt ofte-of-theeart SSL and BSL methods through exten- sive experiments on standard SSL benchmarks. Our method can achieve superior results, e-g., average accuracy of 76.76% CIFAR-10 with merely label per class. The code is available at https: / thub. http///nnnnnnnjnSSSAAMa
Introduction
As paradigm to reduce the dependency large amount of labeled data, semi sumi-supervieed learning (SSL) has been widely concerned and utilized 2 Although existing advanced SSL methods [3] 4 5,66 could achieve outstanding performance even with less than 1% labels on several datasets (e.g.. CIFAR-10), the labeling process could still be lengthy, especially when there a large number of object categories, which may preclude the deployment of SSL. model in those applications. To tackle this challenge, supervesupervised learning (BSL.), a novel paradigm with rising interest [3] 77, has been proposed recently to explore whether the model can be trained with the extremely scarce label, e.g., only one label per class.
"Corresponding author.
'Guan Gui, Yinghuan Shi are with the National Key Laboratory for Novel Software Technology and the National Institute of Healthcare Data Science, Nanjing University.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

model performance
spochs
after 100 epochs
1.0
100
FicMasc
FixMashh
0.8
uctoll
08
achud
40
ReMicMaach
PicMirch
0.4
CoMizzh
0.2
02
our mechoo
0.0
10 0.
10
20
40
250
5000
2
5
10
Tabel eroOUEt
class Indes
class
(a) performance of SSL methods
(b) predicted class distribution comparison
We conduct experiments on CIFAR-I0 to investigate the test performance of different SSL methods with few labels. (a). Performance comparisons with different amounts of labeled data. (b). The predicted class distributions of FixMatch and our method after training the model for 10 and 100 epochs, respectively. Due to the lack of guidance from discriminative information, FixMatch tends to predict all samples as the same class, while our method alleviates this problem through our proposed discriminative information learning module.
Figure I:
Unfortunately, current state-of-the- art SSL methods cannot well address the label-scarce challenge in BSL. As shown in Figure [[a]] many recent SSL methods can achieve auspicious performance when sufficient labeled data provided, e.g., higher than 90% accuracy with more than 40 labels on CIFAR-10. 10. However, these methods will suffer severe performance degeneration when the label amount is reduced. For example, when only 10 labels available on CIFAR-10, the test accuracy of FixMatch will drop sharply by more than 45% compared to that of 40 labels. In order to explore the reasons for such performance dropping, we then track the predicted class distribution of Fix during the training process. As shown in Figure [(b) FixMatch will end with the model collapse after training for 100 epochs, i.e, the model completely cannot distinguish different samples, and all samples are predicted as single class.
To analyze the reasons for above phenomenon, we prefer to investigate classification models in term of separability and immutaability. Here immutability refers to the capacity of the model to be robust to perturbations. It can be mathematically expressed as arg max Pm (ylus) = arg max pm (y/ a (u:) )), where u, is sample, y is the model prediction, and a(-) is a random perturbation. Separability, on the other hand, refers the capacity of the model to differentiate two different categories of sam- ples, i.c., arg max Pm (y)lu,) # arg max Pm (m (ylu,) where u, Uy are sampled from different categories. Figure[ssoww
a(u;)
model
ang eng pag (all)))
ang my. (9iv)
immutability
separability
Figure 2: Examples on the immutability and the separability.
graphic explanation of these two properties. For SSL classification models, in common practice, the immutability is often achieved by learning the consistent information of the unlabeled data, and the separability is achieved by learning the discriminative information of the labeled data. To achieve good performance, SSL models need to well balance their immutability and separability. However, such balance is destroyed in BSL. The insufficient supervision information from the extremely scarce labels significantly damages the learning for separability, so that the model performance is dominated by the learning towards immutability. That's 's why the model collapse could be observed when all samples are predicted one same class.
Motivated by these observations, in this paper, we aim to enhance the discriminative learning for the model's 's separability under BSL. Since the labeled data are very limited, we explore how to mine additional discriminative supervision from unlabeled data. Although without label information, the unlabeled data could still provide some "latent guidance" to complement the process of learning only from labeled data. We hereby propose a novel module to dynamically form super-classes to "roughly categorize" unlabeled samples, then the discriminative information is learned by measuring the similarity between samples and super-classes, which is realized by our newly proposed loss function on the distribution level. Furthermore, with the improvement of the model, we gradually form more
2

---

Consistent Information Learning Module
aft)
Encoder
Clussifie
A(-)
Ldit
Discriminative Information Learning Module
Siger
Clazn
Super Clacsy
Super Clarg
15
......a
Figure 3: Overview of our method. Two different modules are constructed in our method to learn consistent information and discriminative information, respectively. Each sample u, has two variants, processed by the weak augmentation a and strong augmentation A. In the consistent information learning module, Pm (yl)():)) with confidence above the threshold are then used as the training targets for the corresponding strongly-augmented variants. In the discriminative learning module, these features 2," are clustered into K super-classes. The super-class distribution Te for the k-th super-class is obtained by calculating the average pseudo-labels over unlabeled data gathered in this cluster. We construct contrastive-like loss to use these super-class distributions as guidance to train the model on the encouraged to be closer to its super-class distribution q with u, E Cg than to other super-class distributions.
variants. Specifically, the model's prediction on A(u;)
super-classes for finer categorization unlabeled samples, aiming to provide more fine-grained discriminative information to guide the model training.
In nutshell, our proposed method is a simple yet effective way mine discriminative information from unlabeled data. Compared to directly assigning pseudo labels to each sample 3 4. which could be very noisy in BSL, learning the similarity between super-classes and samples the softer guidance, thus reducing the eгror risk of pseudo labels. We evaluate our method on CIFAR- 10, CIFAR-100, and STL-10, showing that our method outperforms all other SSL and BSL methods by large margin. For example, only using one label per class CIFAR-10, our method successfully avoided occurrence of model collapse and achieved an accuracy of 76.76% with a variance of 6.78%.
2 Method
Similar to the setting of SSL, labeled set X' and an unlabeled set 1 are also given in BSL. X ((11,31) (122, va),.... (Na, J.)), where y: denotes the label of the i-th labeled sample z;. Each sample is classified into one of nı classes denoted as {c1, .... Cay) 2 {u), u2...., where u, denotes i-th unlabeled sample, and typically | < [Z4]. In BSL, a more challenging setting is considered, |X| 4ng, where only few labeled data available. In the implementation, the samples are provided on per batch basis, with a batch of labeled data B, and unlabeled data B.. As discussed before, the key to BSL lies in training robust and stable model by efficiently leveraging the unlabeled data together with such scarce labeled data.
Unlike recent satte-of-the-ae SSL methods that only encourage consistency regularization on unla- beled data, our method aims to learn consistent and discriminative information from the unlabeled data simultaneously. As shown in Figure 3 we construct two modules to leverage unlabeled data accordingly, i.e., the consistent information learning module and the discriminative information learning module. In the consistent information learning module, learn the information from the

---

samples and their corresponding augmented versions, like [3. While in the discriminative information learning module, we develops the super-class distributions by clustering unlabeled samples within mini-batch and then uses them to minimize novel distribution loss on unlabeled samples.
2.1 Consistent information learning module
Like most consistency-based SSL methods, we encourage the model to output the same predictions on two -ateerennty-uummeeeeed versions of the same sample. Specifically, we produce pseudo labels on weakly-augmented samples and use them training targets for their corresponding strongly- augmented variants. Of them, the weak augmentation a(-) includes standard flip and shift operations, while the strong augmentation strategy A(-) consists of RandAugment [8] and CutOut [9] Formally, this consistency-bast -basedd unsupervised loss Leon is defined as,
Bul
Leon
(max(pm.(vlaau))) 2 Tι) H (Ppa(()))())) Pm A(u))) Bu
Σ
where H (P1: P2) denotes the standard cross entropy between P1 and P2, and Ta is pre-defined threshold to retain only high-confidence pseudo labels. As discussed 3 Ti is commonly set as a high value to alleviate the confirmation bias in SSL..
2.2 Discriminative information learning module
In addition to relying labeled data to learn discriminative information, propose novel module, an on-the-fly learning process to first form super-classes and then exploit the similarity between super-classes and samples to improve the model's separability.
One of the most intuitive ways to explore discriminative information to generate class information for unlabeled data by clustering in the feature space. Ideally, samples of the same category will form a separate cluster so that the model can discriminate the samples from all other nu 1 clusters of samples. However, forming such fine- sreeggraied clusters carries considerable risk of errors, especially for tasks with a large number of object categories. What's worse, in the early training stage, due to the weak feature extraction ability, the model inevitably produces wrong discriminative information, resulting in severe accumulated errors. To properly explore the discriminative information for unlabeled data, we propose the following designs,
First, instead of fine-grained clusters, we simplify the clustering task by allowing cluster to contain multiple categories, i.e., a super-class cluster. In this way, the discriminative information relatively weakened but more robust to clustering errors.
Second, can still be noisy to adopt the super-class label as training targets for unlabeled data. Therefore, we tend to utilize the similarity between each sample and the super-classes rather than explicitly assign the training targets for unlabeled data. Concretely, our method encourages the unlabeled samples to stay closer to the predicted class probability distribution of their corresponding super-class than those of others. Such smoothing way can better tolerate the inaccurate prediction of single sample as well as potential clustering errors.
Third, although the discriminative information provided by the coarse- graiined clusters is robust, it will be insufficient when the model's separability is improved. Thus we propose the progressive construction of super-classes gradually increase the clustering number that discriminative information learning module can adapt to the model evolution during the training process. When moderate discriminative information,
the cluster number is small, each super-class provides called low-level super-clase In contrast, a large cluster number can enforce each super-class to abstract more concrete information, and we call high-level super-class.
Super-class representation
As shown Figure 3 we employ standard K-Means on these features 2!"² of weakly-augment samples within a mini-batch. With a given target number K of super-classes, these features are gathered into K clusters, and each cluster denoted by Ck then be represented by the mean distribution of all the samples it contains. Given unlabeled sample
l,,,... K. Fach super-class can

---

Mi and its predicted class probability distribution Pm (u)))( and Pm ()))))) the super-class distribution qi: for each super-class Cg can be calculated by,
ICu
1
4E
Pmo (u ((u)),
with uq E CR
In this way, the super-class distribution can represent the distribution characteristics of the categories contains so that it can be well discriminated from other super-classes. As shown in the lower half of Figure 3] in the automobile-and-airpee
super-class, it is possibly not easy to determine the exact category for a single sample. However, we can find that the sample in this super-class should be closer to the super-class distribution of the automobile-ande waa-oobble-and-irlllam- and-airplaaae super-class compared to those of other super-classes. Additionally, the super-class is more robust to the noisy samples. For samples likely to be misclassified (e.g., samples inside the dashed box), their negative impact on the super-class distribution is well suppressed by other correctly classified samples.
Discriminative distribution loss
To distinguish the sample from other super-classes, this sample is supposed to be more similar to its corresponding super-class on distribution. Inspired by [T0 [.. we design contrastive- like distribution loss to distinguish the sample from other super-classes. Formally, this auxiliary distribution loss is,
Bul Σ
Bu (max(pm 2 72) log (ep(pm (T/T)
1
Lais
(3)
Σ (sp(pmmylA(oe TT/T)
where T is a common temperature parameter, and u; is assigned to super-class Cs. Like in Equation I adopt parameter T2 to control the learned unlabeled data. As mentioned before, the similarity between samples and super-classes is weak constraint, so we are conditioned to use lower threshold learn more samples. We provide an empirical value via extensive ablation studies. Notice that we compute gradients only on strongly-augmentee samples.
Progressive super-class construction
Although small K reduces the clustering error, it comes at the cost that the learned discriminant information is limited. Assuming the extreme case, when K 1, the amount of information is O because all samples belong one super-class, the model will not discriminate against any samples.
With this point of view, we propose the progressive construction of super-class to adapt to the model evolution during training. That is, when the model is not well trained at the beginning, we use small K to form the coarser super-classes to ease the clustering task and thus attain relatively reliable discriminative guidance. When the model is better trained, avoid the training of the model being stagnant due to the limitation of discriminative information, we gradually increase K to provide enhanced discriminative guidance.
In practice, daunting challenge is that we do not know the most appropriate number of super-class for the training samples without prior knowledge. To this end, we design dichotomous method and set the value range K by:
K₁ (2, (2..... m2//44, [nk / /2, n
The above formula restricts the value of K based the principle of dichotomy so that frequent changes of K can be avoided. Especially when there are many classes in the sample set (e.g..100 classes on CIFAR-100), it would be tedious and pointless to learn all K values. Furthermore, to ensure that the clustering task with different K can be performed for a certain period, we adopt a linear-step growth strategy adjust K dynamically:
K = Ki, K,
Kit1,
(5)
a (
where and lg denote the value of the current iteration and the total number of iterations, respectively. E (0, 1) and controls the growth rate of K. With this clustering task, super-classes are dynamically formed at each iteration.
5

---

Algorithm Algorithm of our method
Input: Labeled batch 3₂ {(r,, y;)}, unlabeled batch Ba {ui weak augmentation strategy
a(), strong augmentation strategy A(-)
Parameter: threshold 71, 72, temperature T, loss weight Acon. Adis
compute Leup [+ 1- H(pm(yla(u))) vi):
2: for
ig do
3: for u, E Bi, do
4: 2, = -(coder((((x))
// record features of weakly augmented samples. compute prediction of a(us) and A(us).
5: Pen Pmn(yla(u))) Pm Pm) (y)A()))
6: end for
7: Leon Bul
8: update K.
9: form super-classes by K-Means(K, z(")
10: 1/|Cal E pm (m(yaa(u))), Vui E Ck
exp(pm(vlA((us)) (T(T)
Ldis [..
D. ((ma(pp((vla((a)) T2) ΣØ=1 exp(pm (y/ (g]A((u)) - T5/1))
minimizing the total loss L Loup + Acon Leon + ddielidio.
13:
end for
2.3 Total Loss
Similar most SSL methods, the supervised loss for batch of labeled data BB is obtained by standard cross-entropy loss
[B
T Lup
(6)
B
In summary, the total loss in our method is,
Togup + Aeoo Trcan + Adis Tedis
(7)
where Acon and Adis are the weights of Leon and Liiisn respectively. The full algorithm is provided in Algorithm
Experiments
In this section, we validate the effectiveness of our proposed method by conducting experiments on widely-used SSI. benchmark datasets: CIFAR-10, CIFAR-100 [12, and STL-10 [3
3.1 Implementation details
To face the challenge of BSI., we randomly sample or 2 labels for each class on these data sets. We adopt "WideResNet-28-2"" and "WideResNeet28-8"' [14] as the backbone for CIFAR-10 and CIFAR-100, respectively, while using "ResNet18" [15] for STL-10. For the consistent information learned module, follow the same setting with 3 where T 0.95, [B 64, [., B. And for the discriminative information learned module, we set T = 1, = 0.8. In addition, Since the essence of the three losses of Laup- Leon Ldis in the form of cross entropy, it's prefer set Acon Adis to further reduce of hyperparameters. For CIFAR-10 10 and STL-10 task, we set the K [[m,/3], r2./2 ] (3, 5, 10). For CIFAR-100, considering that the samples of each cluster should be sufficient, we set the K E {ne/20, 20, n2/10, 10, m5/5) (5. 10, 20}. In fact, the specific numerical setting of has little effect on the model performance, and more analysis and experiments
about K will be discussed in the later ablation experiments.

---

Table I: Performance comparisons CIFAR-10, CIFAR- 100, STL-10. Each result is reported as the average runs. The results show our method outperforms other baselines in all settings. On CIFAR-10 10 and STL-10 with 10 labels, our method outperforms other methods by least 10%. For the larger dataset C10AR-100, our method also outperforms baseline methods by at least 6%.
CIFAR-10
CIFAR-100
STL-10
Method
10 labels
20 labels
100 labels
200 labels
10 labels
20 labels
Mean-Teacher
15.48 + 3.19
17.50 + 1.16
5.17 + 2.52
8.26
3.43
11.05 + 6.45
15.99 = 6.45
MixMatch
17.18 4.45
26.45
8.17
12.85
2.21
21.36
+ 4.84
10.94 + 5.18
21.48
= 3.17
ReMisMatch
60.29 1: 15.20
78.56
9.63
26.18
1 3.79
35.90
1 3.66
30.86
1 10.80
45.58
1 8.36
FixMatch
44.47 + 24.99
80.46 5.15
25.49
+ 4.37
35.55
+: 1.59
25.75
1: 8.99
48.98
+ 6.46
MatMhtch (w/DA)
67.79
84.16
9.27
2.29
1.87
6.24
5.44
CoMatch
60.79 +: 12.42
81.19 8.55
27.54
1: 4.25
36.98
1: 2.17
29.11
9.31
50.20
+ 7.57
FlexMatch
06.07 10.58
85.69
6.24
31.50
3.61
38.03 05
2.66
41.17
6.20
54.30
5.65
SLA
65.87 10.83
81.89 6.77
28.45 + 2.16
38.65 + 2.67
32.38 + 8.32
47.50
6.38
LESS
64.40 + 90
81.20 + 5.60
28. 20
+ 3.00
12.50 + 3.20
34.25 + 7.19
18, 98
= 5.19
our method
76.76 6.78
88.49
3.26
37.50 + 1.72
45.62 + 1.39
52.51
8.20
57.98
3.18
The model is trained with total of 22 iterations, and the increased in the first 30% iterations. We use an exponential moving average with decay rate of 0.999 to test our model and repeat the same experiment for five runs with different seeds report the mean accuracy.
3.2 Baseline methods
First, FixMatch [3. Dash [6.. CoMatch [5], FlexMatch 4 are the advanced semi-supervised models in recent years, and we compare these methods under the challenge of
learning. We also use FixMatch with the distribution alignment (DA). SLA [6] and LESS [7] are the latest models on BSL,, and we also use them as our comparison method. In addition to this, we also select some classical semi-supervised methods such as MeanTTeachrr 17 MixMatch [I8] and ReMixMatch [99 for comparison.
3.3 Experimental results
Performance comparisons. In Table [] we compare the test accuracy of our proposed method against recent SSL and BSL. methods. can be seen that our results are state-of-the-art all settings. Especially when there is only one label per class, our method compensates for the shortage of labeled data by mining latent discriminative information from unlaheled data, thus showing enormous superiority. LESS 77], recent work on BSL, since it generates predictions for samples with low confidence and then learns more consistent information, still ignores the learning of discriminative information, it cannot solve the challenges in BSL.
On CIFAR-10 task with 10 labels, our method achieves the mean accuracy of 76.76%, which outperforms other methods by 10%. On STL- 10 task with 10 labels, the recent BSL methods LESS and SLA achieve the accuracy of 34.25% and 32.38%, respectively, while our method achieves the mean accuracy of 52.51%, %, which improved nearly by 20%. For larger dataset CIFAR- our method also outperforms other methods by at least 6% when there 1 label per class. Besides, we can see that, regardless of the dataset, the performance of our method when using only label per class close to or even exceeds the performance of other methods when using 2 labels per class. On CIFAR- 100 task, LESS and SLA achieve the mean accuracy of 42.50% and 38.65% with 100 labels, while our method achieves the mean accuracy of 37.50% with half of the labels they use.
As mentioned by 3 the quality of very few labeled data will significantly affect the performance of the model. Taking the CIFAR-10 10 task with 10 labels as an example, the variance of advanced SSL methods and BSL methods are all more than 10%, while the variance of our method is only 6.78%. These results further illustrate that our method can alleviate the dependence on labeled data by learning discriminative information from unlabeled data.
We also find that the technique of distribution alignment, which forces the alignment of probability distributions is still an effective technique under BSL. Through the result of FixMatch (w/DA), we can see that DA successfully helped FixMatch improve its performance significantly. However, DA is a technique that relies on prior information, and our method does not rely on any prior information and can achieve better performance than it.
7

---

70.23 72.15
80
exp-mode
stop
accuracy(%)
70
75
60
50
40
10
93 0.4
0.5 0.6
0.7
0.95
0.9
0.88
0.7
9.6
(a) K
(b)
(c)
Figure 4: Abalation study on CIFAR-10 with 1 labeled sample per class. (a) Performance using fixed K values without dynamical clustering form super-classes. (b) performance using progressive K with different growth rate a. (c) Confidence threshold T2. Appropriately lowering the threshold can
learn more samples, thereby helping model training.
Stability of the model. First of all, we dis- cuss the phenomenon of model collapse un- der the BSL challenge. For a fair compari- son, we use the same random seed in each trial for FixMatch and our method. Since only I label per class is available, this SSL method that depends on labeled data to learn discriminative information would be volatile. As shown in table we can see that the per- formance of FixMatch extremely unstable, where can achieve very high accuracy of 85. 11% when seed= 3 but obtain an extremely low accuracy of 17.09% when seed= 4. Differently, integrating the proposed super-class distribution to provide more discriminative information, our method can successfully alleviate the model collapse: the accuracy exceeded 70% in all experiments and also exceeded 80% sometimes.
Table 2: Experiments CIFAR-10 with 10 labels (1 label per class). FixMatch exhibits volatile per- formance, while our method greatly improves perfor- mance and is stable.
seed
4
FixMatch method
19.15
85.11
52.52
17.09
48.50
81.28
86.12
70.34
74.90
71.17
Performance under SSL settings. We also analyze our method in stan- dard SSL settings where sufficient la- beled data are provided. As shown in Table 3 we test our method on CIFAR-10 with 40, 250, and 4000 labels. It can be seen that when the number of labels increases, our method is not SOTA, but the gap with other methods within 1%. can be interpreted as these methods us- ing other advanced techniques in the learning consistent information pro- cess. For example, FlexMatch [4] and SLA 6 both leverage prior knowledge of class proportions, contrastive learning, etc. While study independent module for solving BSL, so when the labels are enough, our method does not prevail. However, though our implementation is based on FixMatch (3), the results show that method can still improve slightly when the number of labels is large.
Table Experiments on CIFAR-10 with more labels. When there are enough labeled data, our method performs par with SOTA SSL methods.
Method
40 labels
labels
4000 labels
ReMixMatch
80.90 9.64
94.56 + 0.05
95.02 0.11
FixMatch
90.11 + 3.01
94.21 + 0.61
95.91 = 0.02
CoMatch
93.09 + 1.39
95.09 = 0.33
95.57 0.20
FlexMatch
95.01 0.16
95.20 = 0.06
96.05 0.03
SLA
94.83 0.32
94.98 + 0.28
95.59 0.09
LESS
93.20 + 2.10
95.10 + 0.80
95.64 0.29
method
94.19 + 0.41
91.54 + 0.27
95.78 0.08
and CoMatch (5) leverages graph-
3.4 Ablation study
Performance under different strategies of K. We explore the effect of K for different strategies on the model: (1) fix-mode, the number of super-class remains constant during model training. (2) lonear-mode, the number of super-class increases linearly during model training. (3) exp-mode, the number of super-class increases exponentially. (4) step-mode, step-by-step jump growth based on linear-mode.

---

We fixed different K values for experiments in terms of the fixed strategy. As shown in Figure 4(a) when K is small, the model tends to outperform the larger K. When K is large, the model in the early stage does not provide high-quality features to perform the formation of high-level super- classes, so the discriminative information learned has an extremely high risk of error, leading to the model's failure. On the other hand, when K' is small, our method can learn effective discriminative information from these low-level super-classes. However, as the model performance improves, this limited discriminative information provided by low-level super-classes can no longer help the model learn continuously, so the model's s performance will stagnate. It is worth noting that even if we adopt the fix-mode with K, the model can learn certain degree of discriminative from the super-class to face the challenge of BSL, and its performance also exceeds other methods.
Linear-mode, exp-mode, step- mode can all work well to solve the problem in the fix-mode above, while there is an additional hyperparameter to control the rate of K growth in these modes. As shown in Figure [bb] we conduct experiments for different growth rates, and it turns out that the growth rate of K does not affect the performance of the model too much. In addition, since this mode can explore different levels of discriminative information, the performance is significantly better than that in the fix-mode. Although the performance of these modes is exceptionally close, we prefer step-mode as it can be more suitable for large data sets, e.g., it is impractical to increase K from 3 to 100 sequentially when we test on CIFAR-100.
Performance under different T2. We investigate 5 different T2 values on CIFAR-10 10 datasets with 10 labels. As shown in Figure [[)] the test performance achieve the best when 722 0.8. It shows that appropriately lowering the threshold can learn discriminative information from more samples, thereby helping model training. However, if the threshold 72 is too low, the noise of the sample will increase, which is not conducive to the training of the model.
Related Work
Recent popular semi-supervised learning studies can be classified into entropy minimization (ER) based methods and consistency regularization (CR) based methods. Self training is the typical representative of ER-based methods. In these methods [[] the model is first trained on the provided labeled data and then used to generate pseudo-labels for unlabeled data. After that, such methods add these unlabeled data with high-confidence predictions into the labeled set retrain the model, repeating this process until all unlabeled data are involved (20 Recent studies tend to involve more advanced techniques in this framework enhance the SSL performance. [2I introduces multiple views to provide more robust pseudo- LaSSL [22] and Curriculum Labeling [23] integrate the
contrastive learning and curriculum learning techniques to improve the accuracy of pseudo-labels further.
As the most widely-usee and successful technique in recent SSL methods, CR is the semantics of a sample should be consistent after data perturbations [24] 17 18 25 26). FixMatch (3) combines strong augmentation technology (8] 9 and the labels of weakly augmented samples with high confi- dence are used to guide the learning of strong augmented samples. Although a major breakthrough has been made in conventional semi- supesvieee learning, it still cannot avoid model collapse. 16 further dynamically adjusts the confidence threshold based on FixMatch. Although it can learn more low-confidence samples to improve the performance of the model, it cannot cope with the challenge of lack of discriminative information under BSL.
In the literature, there have been few works on barely-supervised learning. Fix initially came up with the concept of BSL and emphasized that the quality of the labeled data played a crucial role in the test performance. Our experiment results also demonstrate that its testing results have a very high variance under the BSL settings. Recent SLA [6] also achieved better performance in BSL by formulating an optimal transportation problem between samples and labels. introduced many extra syper-parameters and adopted the Bankhorn-Knopp algorithm to solve the optimization problem approximately. Differently, our method gets rid of complicated operations but can effectively improve the BSL performance. Another work [7] argues that the dilemma in BSL is that there are no pseudo-labets that can be predicted with high confidence, so online deep clustering is used to supplement the pseudo-labels predicted by the model. Although more pseudo-labels can be used, still learns consistent information. [27] is a very recent work that uses the coarse-graine class labels to guide the SSL model. However, it requires strong prior knowledge about the class hierarchy

---

structure in advance; while we are faced with BSL scenarios without any prior knowledge, even the hierarchy structure may not exist.
Conclusion
In this paper, analyze the failure of SSL methods in the face of BSL. insufficient discriminative information learning. To tackle this problem, we design a discriminative learning module to leverage unlabeled data for additional discriminative supervision. In this module, super-classes dynamically reformed with the model training, and then the discriminative information is learned by measuring the similarity between samples and super-classes. We conduct our methods on several SSL benchmarks, and it shows that our method outperforms other methods in BSL.
6
Acknowledgment
This work was supported by the Science and Technology Innovation 2030 New Generation Artificial Intelligence Major Project (2021ZD01 13303), the NSFC Program (62222604, 62206052, 62192783), CAAl-Huawei MindSpore Project (CAAIXSJLJJ-22) 2021-222), China Postdoctoral Science Foun- dation Project 1690669),,, Jiangsu Natural Science Foundation Project (BK20210224), and CCF-Lenovo Bule Ocean Research Fund.
References
[1] Yassine Ouali, Céline Hudelot, and Myriam Tami. An overview of deep semi comi-supevvised learning. arXiv preprint arXiv:2006.00 05278, 2020.
[2] Jesper E Van Engelen and Holger H Hoos. A survey on semi-supervised learning. Machine Learning, 109(2):333-440, 2020.
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying supe-supe learning with consistency and confidence. Advances in Neural Information Processing Systems, 2020.
[4] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised lcarning with curriculum pscudo labeling. Advances in Neural Information Processing Systems, 2021.
[5] Junnan Li, Caiming Xiong, and Steven CH Hoi. Comatch: Semi-supervinn learning with contrastive graph regularization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 99744, 2021.
[6] Kai Sheng Tai, Peter Bailis, and Gregory Valiant. Sinkhorn label allocation: Semi-supervised classification via annealed self-training. Workshop on challenges in representation learning, ICML, 2021.
[7] Thomas Lucas, Philippe Weinzaepfel. Gregory Rogez. Sarely-uuperviee learning: supervupereieed learning with very few labeled images. the 36th AAAI Conference on Artificial Intelligence. 2022.
[8] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoe V Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
[9] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint urXivv:708.044: (tt.:1708804552,, 2017.
[10] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning visual representations. International conference on machine learning, pages 1597-1607, 2020.
[11] Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin. Unsupervised feature learning non- parametric instance discrinination. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 3733-3742, 2018
[12] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4). 2009.
10

---

[13] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statisticx, pages 213-223. JMLR Workshop and Conference Proceedings. 2011.
[14] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference, 2016.
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[16] Yi Xu, Lei Shang, Jinxing Ye, Qi Qian. Yu-Feng Li, Baigui Sun, Hao Li, and Rong Jin. Dash: Semi- supervised learning with dynamic thresholding. In Proceedings of the 38th International Conference on Machine Learning, 2021.
[17] Antti Tarvainen and Harri Valpola. Mcan teachers are hetter role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances Neural Information Processing Systems, 2017.
[18] David Berthelot, Nicholas Carlini, Ian Goodfellow, Avital Oliver, Nicolas Papernot, and Colin Raffel. Mixmatch: holistic approach to semi- superviie learning. Advances Neural Information Processing Systems, 32, 2019.
[19] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. In Eighth International Conference on Learning Representations, 2020.
[20] Dong-HYuu Lee. Pseudo-label: The simple and efficient seni-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, 2013.
[21] Dongdong Chen, Wei Wang, Wei Gao, and Zhihua Zhou. Tri-net for supiruuee deep learning. In International Joint Conferences on Artificial Intelligence, 2018.
[22] Zhen Zhao, Luping Zhou, Lei Wang, Yinghuan Shi, and Yang Gao. Lassl: Label-guided self-training for semi-supervised learning. In the 36th AAAI Conference on Artificial Intelligence. 2022.
[23] Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Curriculum labeling: Revisiting lseudo-labeling for semi-supervised learning. In the 35th AAAI Conference on Artificial Intelligence.
2021.
[24] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In Fifth International Conference on Learning Representations, 2017.
[25] Chia-Wen Kuo, Chih-Yao Ma, Jia-Bin Huang, and Zsolt Kira. Featmatch: Feature-based augmentation for semi-supervised learning. In European Conference on Computer Vision, pages 479-495, 2020.
[26] Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, and David Lopez-Paz Paz. Interpolation consisteney training for sepi-supereised learning. In International Joint Conference on Artificial Intelligence, pages 3635-3641, 2019.
[27] Ashima Garg, Shaurya Bagga, Yashvardhan Singh, and Saket Anand. Hiermatch: Leveraging label hierarchies for improving seme-supervised learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1015-1024, 2022.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you descrihe the limitations of your work? [Yes]
(c) Did you discuss any potential negative socictal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments...

---

(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Ys]]
(c) Did you report
bars (e.g.. with respect the random seed after running experiments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e-g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) your work uses existing assels, you cite the creators? [Yes]
(h) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're re us- ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable informa- tion offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) you describe any potential participant risks, with links to Institutional Review Board (IRB)
approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "owt5AA7QS-", "reviewer": "Reviewer_FJpN", "summary": "From the notion and empirical analyses from barely supervised learning, it introduces super class based regularization which improves the discriminability of unlabeled data instances. It largely improves the performances with various benchmark datasets with high confidences. The introduction of clustered super class is somewhat novel in the semi-supervised learning.", "questions": "Stated in weaknesses session.", "limitations": "Stated in weakness session.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths :\n\nIntroduction of super class is the main novelty of this paper. It mitigates the imbalanced ability of barely supervised learning from the side of immutability and separability, which is sound. \n\nWeaknesses :\n\nApplication on the semi-supervised learning with more labels provides ineffective performances, which is not explainable from the view of immutability and separability. If it really matters, the performances can be calibrated from the hyper-parameter tuning between consistent learning and the introduced discriminative learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "igVKNu4fTpO", "reviewer": "Reviewer_vR2e", "summary": "This paper presents a semi-supervised learning method for barely supervised setting. The core contribution is the design of discriminative information learning module with clustered super classes. The experiment on barely supervised setting demonstrates the consistently improved results on small datasets. ", "questions": "Questions:\nFor eq.3, why a constative-like loss is exploited. Is simple CE loss on the super classes not working? Cosine similarity is adopted in eq.3, but for softmax probabilities.\n\nIt important to show the training speed with the proposed discriminative information learning module. Would running K-means for each training iteration slow down the training? Comparison between the training speed would be expected.\n\nFor barely supervised setting, random seed often plays important roles for the distribution of the labeled data. Can you provide more details on how the random seed is set? Is it randomly selected or with more sophisticated selection? For example, in FixMatch, they expain how they choise the 10 labels for CIFAR-10\n\nAll experiments are run with small datasets. Also, a challenging dataset SVHN is not shown. SVHN is more challenging because its noisy background. It would be interesting to see the performance on SVHN and larger datasets such as (Mini)-ImageNet.\nTo my understanding, for barely supervised setting, the quantity of the samples accepted for calculating the loss is especially important for early training. If a high threshold (tau_1, tau_2) is employed, the SSL algorithms would collapse to one class as shown in Fig.1 because not diverse enough samples are enrolled during training. This is also demonstrated in Fig.4(c), when tau_2 is higher, the performance drops more significantly than when tau_2 is lower. In that case, compare the enrolled rate of samples and the pseudo-label accuracy to FixMatch and FlexMatch is important to demonstrate the effectiveness of the proposed method.\n\nThe Super class concept is very similar to SimMatch (https://arxiv.org/abs/2203.06915) yet no discuss and comparision is provided. ", "limitations": "see above", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe paper is well-written and organized, easy to follow.\nThe improvement on the studied settings is significant and consistent.\n\nWeakness:\nThe design choice on auxiliary distribution loss (eq.3) is not shown\nThere is a mismatch of EMA of model parameter in main paper 0.999 (L201) versus 0.99 in appendix 0.99 (in table1)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "h9dHwfOQ2vF", "reviewer": "Reviewer_5pm6", "summary": "The paper tackles the problem of barely-supervised learning – an SSL setting in which only a few-labels per class are annotated. The authors propose a novel approach that extends the FixMatch with a discriminative information learning module. The key idea of this module is to form super-classes by clustering the data using K-means clustering and then rely on the similarity between super-classes and samples to guide the training. The number of clusters is gradually increased during the training. In that way, super-classes are dynamically formed and gradually become more fine-grained. The final objective function consists of standard cross-entropy loss on the labeled examples, standard consistency loss based on the weakly and strongly augmented versions of the data and the newly introduced discriminative distribution loss based on the formed super-classes. The performance of the approach is compared to state-of-the-art SSL and barely-supervised methods on CIFAR-10, CIFAR-100 and STL-10 datasets.", "questions": "- Why K ∈ {5,10,20} for CIFAR-100? What is the performance on CIFAR-100 when the K goes to the maximum of 100 classes?\n- Why is K increased only in the 30% of iterations? How is that determined and how does a different setup affect the performance?\n- How does the algorithm compare to others on the SVHN benchmark dataset on the barely-supervised learning as well as standard SSL setting?\n- How does the algorithm perform on the unbalanced dataset?\n- Authors motivate super-class idea by the fact that the pseudo-labelling approach can be very noisy. But how accurate are the resulting clusters (super-classes) during model training? It would be beneficial to compare the accuracy of pseudo-labels of FixMatch with the the quality of generated clusters in the proposed approach (e.g., whether examples that belong to the same ground-truth class are assigned to the same super-class during training)\n\n**Minor comments:**\n- Line 194: it seems it should be \\lambda_con=\\lambda_dis=1 not L\n- Line 294: sentence is not finished\n", "limitations": "No. I suggest authors comment on the limitations of their reliance on the K-means clustering.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- Paper is well written, well structured and it was a pleasure to read it. The explanation with immutability and separability motivates the paper and the proposed approach really well. \n- Barely-supervised learning is a challenging setting, and the proposed method solves the challenge in a simple, elegant and effective way\n- The proposed  method achieves state-of-the art performance on the barely-supervised learning setting on three datasets, and comparable performance on the standard SSL setting.\n\n**Weaknesses:**\n- (Transductive) few-shot learning should be discussed in the related work and in the introduction and the differences between the settings need to be explained. Barely supervised learning should be better defined in the introduction and put in the relation with the more studied problem of the few-shot learning.\n- The performance does depend on K and it is not clear how to set it. For example, it is not clear why for the CIFAR-100 K was set in the range {5,10,20} and differently then for the CIFAR-10 and STL-10 datasets. How does setting K differently affect performance on the more challenging CIFAR-100 dataset? The analysis is only done on the CIFAR-10 dataset.\n- Reliance on the K-means clustering. Clusters can only be sperical in shape and the method may fail for the unbalanced dataset.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "HOqtMDPY__f", "reviewer": "Reviewer_B558", "summary": "This paper addresses Barely Supervised Learning (BSL), an SSL problem when only a few labeled samples are available.\nThe key is the introduction of a new loss function to the general consistency-based framework of SSL; the new loss measures the consistency between unlabeled samples and the centroids of \"superclasses,\" which are obtained by applying K-means to the unlabeled data with $K <=$ # classes. K is gradually increased as the learning progresses, like a “curriculum learning” idea.\nExperiments with CIFAR-10, CIFAR-100, and STL-10 have shown that the proposed method outperforms the existing SoTA SSL/BSL methods under BSL settings.\n", "questions": "There are several unclear points. In particular, I would like the authors to clarify C1-C3.", "limitations": "I did not find any discussion on limitations. Related to C2 above, if the training time increases when running K-means on a batch basis, it would be a clear weakness.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "--------------------------------------Pros--------------------------------------\n\nP1. The paper addresses BSL, an SSL problem with only a few labels. While it is important and practical, few studies have been reported in the literature.\n\nP2. This paper provides a good insight that BSL can be improved by a very simple method of adding consistency loss with K-means centroids.\n\nP3. The method outperforms SoTA SSL/BSL methods.\n\nP4. Ablation study on $K$ is reported.\n\n--------------------------------------Cons--------------------------------------\n\nC1. Batch size and $K$: Algorithm 1 says that K-means is performed batch by batch. Just to make sure, does this mean that K-means is applied to the samples within a batch, i.e., 64 samples? If yes, I wonder the resulting clusters will be quite different between iterations and may not be meaningful when the number of clusters is large.\n\nC2. Computation Time: Since every iteration K-means is performed, the training time may increase. There is no discussion on this point.\n\nC3. Results: I have some questions about the results. For example, in the CIFAR-100 100-label setting in Table 1, the score of LESS [4] is 28.2 $\\pm$ 3.0, which is consistent with the number reported in [4]. However, in the 200-label setting, there is a gap between the numbers reported in this paper (39.5 $\\pm$ 3.2) and those given in the original [4] (42.5 $\\pm$ 3.2). It is unclear what is causing this difference. Such inconsistencies are also observed in Table 3.\n\nC4. Analysis: The ablation study on the proposed loss term $L_{dis}$ would be essential, as it is the main idea of this paper. Some sensitivity analysis and discussions on the weight for $L_{dis}$, $\\lambda_{dis}$, would be necessary.\n\nC5. Novelty: The method looks a little incremental. The proposed method only adds the consistency loss with K-means centroids to the typical SSL framework (i.e., consistency between weak and strong augmented samples).\n\nC6. The discussion of immutability and separability given in the introduction is interesting, but its significance is not clear. The relationship to the idea of the proposed method seems somewhat tenuous, and these measures are not used in the evaluations.\n\nC7. Typos：\n- L37 and others: “model collapse” -> \"mode collapse\" would be better.\n- L49: “two different samples” -> \"two samples from different classes\" would be what the authors want to mean.\n- L140 and others: “super-class super-class” -> “super-class”\n- L190: “the consistent information learned model” -> \"consistent information learning module\" (the same is applied to L192: “the discriminative information learned model”)\n- L221: “52.52%” -> “76.76%”\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept