On Pseudo-Labeling for Class-Mismatch Semi-Supervised Learning
Authors: Lu Han
Han-Jia Ye
De-Chuan Zhan
Venue: Accepted by TMLR ()

Abstract: When there are unlabeled Out-Of-Distribution (OOD) data from other classes, Semi-Supervised Learning (SSL) methods suffer from severe performance degradation and even get worse than merely training on labeled data. In this paper, we empirically analyze Pseudo-Labeling (PL) in class-mismatched SSL. PL is a simple and representative SSL method that transforms SSL problems into supervised learning by creating pseudo-labels for unlabeled data according to the model's prediction.  We aim to answer two main questions: (1) How do OOD data influence PL? (2) What is the proper usage of OOD data with PL?  First, we show that the major problem of PL is imbalanced pseudo-labels on OOD data. Second, we find that OOD data can help classify In-Distribution (ID) data given their OOD ground truth labels. Based on the findings, we propose to improve PL in class-mismatched SSL with two components -- Re-balanced Pseudo-Labeling (RPL) and Semantic Exploration Clustering (SEC). RPL re-balances pseudo-labels of high-confidence data, which simultaneously filters out OOD data and addresses the imbalance problem. SEC uses balanced clustering on low-confidence data to create pseudo-labels on extra classes, simulating the process of training with ground truth. Experiments show that our method achieves steady improvement over supervised baseline and state-of-the-art performance under all class mismatch ratios on different benchmarks.

Published in Transactions on Machine Learning Research (11/2022)
On Pseudo-Labeling for Class-Mismatch Semi-Supervised Learning
Lu Han State Key Laboratory for Novel Software Technology, Nanjing University
Sanhullamd.nju.ettu
Han-Jia Ye
yehxjälamda.rju.ccu..cm
State Key Laboratory for Novel Software Technology, Nanjing University
De-Chuan Zhan
shando@nju.edue edu.cc
State Key Laboratory for Novel Software Technology, Nanjing University
Reviewed on OpenReview: https: // wwtp://eepervvee....m net/ farmm? id= tLG26QuoD8
Abstract
When there are unlabeled Out-Off-Distriuttion (OOD) data from other classes, Semi- Supervised Learning (SSL) methods suffer from severe performance degradation and even get worse than merely traiming labeled data. In this paper, we empirically analyze Pseudo- Labeling (PL) in
SSL. PL is simple and representative SSL method that transforms SST, problems into supervised learning by creating pseudo-labels for unlabeled data according to the model's prediction. We aim to answer two main questions: (1) How do OOD data inffuence PL? (2) What is the proper usage OOD data with PL? First, we show that the major problem PL is imbalanced pseudo-labels on OOD data. Second, we find that OOD data can help classify In-Distribution (ID) data given their OOD ground truth labels. Based the findings, we propose to improve PL in class-mist missmachmmt SSL with two components Re-balanced Pseudo- Labellin (RPL) and Semantic Exploration Clustering (SEC). RPL re-balances pseudo-labels of high-confidence data, which simultancously filters out OOD data and addresses the imbalance problem. SEC uses balanced clustering on low-confidence data to create pseudo-labels on extra classes, simulating the process training with ground truth. Experiments show that our method achieves steady improvement over supervised bascline and stato-of-the-art performance under all class mismatch ratios on different benchmarks.
1 Introduction
Deep Sunir-Suer Learning (SSL) methods are proposed to reduce dependency on massive labeled data by utilizing a number of cheap, accessible unlabeled data. Pseudo-Labeling (PL) (Lee) 2013) is a widely used SSL method. PL is simple yet effective, which creates pseudo-labels according to predictions of the training model itself. Then SST. can be transformed into standard supervised learning. Other representative SSL methods include consistency regularization (Laine Aila 2017 Tarvainen Valpola, 2017 Miyato al. 2019), holistic methods (Berthelot et al. 2019) [Sohn et al. 2020), and generative methods (Kingma al., 2014. The recent development of SSL shows that these methods have achieved competitive performance to supervised learning methods.
However, these SSL methods achieve their good results based on an assumption that unlabeled data are drawn from the same distribution as the labeled data. This assumption can be easily violated real-world applications. One of the common cases is that some unlabeled data come from unseen classes. As is illustrated in Figure [ in image classification, we can collect lot of unlabeled images from the internet but usually, they cover broader category concepts than labeled data. Oliver et al. (2018) have shown

---

Published in Transactions on Machine Learning Research (11/2022)
that under such class-mismatched conditions, the performance of traditional SSL methods is damaged. Several methods are proposed for class-mismatched SSL, including filtering out OOD data (Yu et al. 2020; Chen et al.] 22200), down weighting OOD data (Chen et al., 2020), and re-using OOD data by neural style transfer (Luo et al. 2021 /eef///ssssssurisss/e learning (Huang et al. 2021). Although these methods achieve good results, why OOD data damage performance and how will OOD data help remain unclear.
labeled data
unlabeled data
In this paper, we focus empirically analyzing one representative family of the SSL method - PL in daas-miisttthhe SSL and give some answers to these two questions. (1) How do OOD data influence PL? (2) What are suitable pseudo-labels for OOD data? For question (1), we investigate pseudo-labels created by PL. The main finding that pscudo-labels on OOD data tend be imbalanced while on ID data, they remain balanced. We further show that PL's performance damaged due to such an imbalance in OOD data. For question (2), several strategies for labeling OOD data are investigated. We conclude that it is beneficial when labeling OOD data as class different from ID data, and the performance can be further improved when pseudo-labels partition unlabeled OOD data into their semantic clusters.
cat
dog
cat
dog
plane
car
ID data
OOD data
Figure 1: Realistic Supi-vuper
Learning may simultancously contain un- labeled ID and OOD data. ID) data come from the same classes as labeled data while OOD data come from classes that are not seen in labelod data.
Based on the experimental analyses, we propose two-branched model called Y-Model, which processes unlabeled data according
to their confidence score on ID classes. The first branch performs Re-balanced Pseudo-Labeling (RPL) on high-confidence data. It utilizes the property of imbalanced pseudo-labels on OOD data, truncating the number of pseudo-labeled data for each class to their minimum. This procedure filters out many OOD data and also prevents the negative effect of imbalanced pscudo-labels. For the other branch, Semantic Exploration Clustering (SEC) is performed on low-confidence data. They are considered OOD data and their semantics will be mined by clustering into different partitions on extra classes. The clustering result provides better pscudo-labels for these OOD data than vanilla PL. Experiments on different SSL benchmarks show that our model can achieve steady improvement in comparison to the supervised baseline. Our contributions are:
We analyze PL for ID and OOD data. The findings lead two primary conclusions: (1) Imbalance of pseudo-labels OOD data damages PL.s performance. (2) Best. pseudo-labels for unlabeled OOD data are those different from ID classes and partitioning them into their semantic clusters.
We propose our two-branched T-Model. One branch re-balances pseudo-labels on ID classes and filters out OOD data. The other branch explores the semantics of OOD data by clustering on extra classes.
Experiments on different SSL benchmarks empirically validate the effectiveness of our model.
2 Preliminary
2.1 Class Mismatchhh SSL
Similar to the SSL problem, the training dataset of the class-mismatched SSL problem contains n ID labeled samples D; = {(w,, H and unlabeled samples D., {w Im (usually, m > n.) yis E VID = (1...., KID), while different from SSL, the underlying ground truth ya of unlabeled data may be different from labeled data. i.e, Yuj E VID U YOOD,YOOD = {KiD + +,...,, clast-ems SSL is to correctly classify ID samples into VID using labeled set with ID samples and unlabeled set possibly with OOD samples.
KID +
KoOD). The goal of
2.2 Pseudo-Labeling
Exeudo-Labeling (PL) leverages the idea that we can use the model itself obtain artificial labels for unlabeled data [Lee 2013). PT. first. performs supervised learning on labeled data to get pre-trained model f, which
2

---

Published in Transactions on Machine Learning Research (11/2022)
outputs the probability of belonging to each ID class. Given c(z) is the confidence score for
<(x) = max f(ylæ),
(1)
vEVrD
PL creates the pscudo-labels for each unlabeled sample:
3/ arg max f(/az)
c(x) >T
(2)
reject
otherwise
All pseudo-labeled unlabeled data will be treated labeled data for the next supervised learning generation. PL iteratively performs supervised learning and pseudo-label creation until stops.
3 Analysis of Pseudo-Labeling in Class MissstMismacche SSL
Pseudo-labels on ID data
Pseudo-labels OOD data
Cee
3.79
Frequency
Frequenes
Imbalance
1101
Class
Class
(a) Pseudo-labels
(b) Pscudo-labels on OOD
(c) Imbalance ratio
Figure 2: Analysis of the pseudo-label distributions created by the pre-trained model. (a) On ID data, pseudo-label distribution balanced since they share the same distribution with the labeled data. (b) On OOD data, the pscudo-label distribution imbalanced. (c) The imbalance ratio of pscudo-labels on ID and OOD data other datasets. The imbalance ratio is computed by KL-divergence with uniform distribution.
In class-mismattee SSL, vanilla PL can only create pseudo-labels on ID classes even for OOD data. We will analyze how these OOD data influence vanilla PL and what are the better pseudo-labels for them in this section. Experiments are carried out on totally five kinds of datasets. (We use (n/m) to represent n ID classes and OOD classes.)
CIFAR10 (6/4) created from CIFAR10 (Krizhevsky & Hinton 2009 takes the animal classes ID classes and 4 vehicle classes OOD classes. We select 400 labeled samples for each ID class and totally 20,000 unlabeled samples from ID and OOD classes.
SVHN (6/4): We select the first "0"_"5" as ID classes and the rest as OOD. We select 100 labeled samples for each ID class and totally 20,000 unlabeled samples.
CIFAR100 (50/50): created from CIFAR100 (Krizhevsky & Hinton 2009 The first 50 classes are taken as ID classes and the rest as OOD classes. We sclect 100 labeled samples for cach ID class and a Lotal of 20,000 unlabeled samples.
Tiny ImageNet (100/100): created from Tiny ImageNet, which is a subset of ImageNet (Deng ct al. 2009) with images downscaled to 64 x 64 from 200 classes. The first 100 classes taken ID classes and the rest OOD classes. We select 100 labeled samples for each ID class and 40,000 unlabeled samples.
ImageNet100 (50/50): created from the 100 class subset of ImageNet (Deng et al. 2009 The first 50 classes are taken as ID classes and the rest as OOD classes. We select 100 labeled samples for each TD class and total of 20,000 unlabeled samples.
3

---

Published in Transactions on Machine Learning Research (11/2022)
Here use C to represent CIFAR, TIN to represent Tiny ImageNet, IN to represent ImageNet for short. We vary the ratio of unlabeled images to modulate class distribution mismatch. For example, the extent is 50% means half of the unlabeled data comes from ID classes and the others come from OOD classes. We use Wide-RosNet-28-- (Zagoryyko & Komodakis 2016) as our backbone. We also adopt data augmentation techniques including random resized crop, random color distortion and random horizontal flip. For each epoch, we iterate over the unlabeled set and random sample labeled data, each unlabeled and labeled mini-batch contains 128 samples. We adopt Adam the optimization algorithm with the initial learning rate 3 x 10 and train for 400 epochs. Averaged accuracies of the last 20 epochs are reported, pretending there is no reliable (too small) validation set to perform early stop (Oliver et al. 2018
3.1 Imbalance Pseudo-labels on OOD Data
In this section, we analyze the pre-trained model that creates the first set of pscudo-labels, and the final model trained by Pseudo-Labeling.
Pre-trained model. Like what is concluded in OOD delection (Hendrycks Gimpel 2017 ID data lend have higher confidence than OOD data, but there are still considerable OOD data with high confidence. In class-mismatched SSL, the unlabeled data in much larger quantities. When the class mismatch ratio is large, there quite few OOD data with high confidence scores. We will show in the final model experiments that these high-confidence OOD data damage performance. Secondly, we study pseudo-labels on both ID data and OOD data. Figure (a) shows that pseudo-labels ID data is balanced. However, they are rather imbalanced on OOD data (Figure 2bb].. Such difference in created pscudo-labels is attributed to the different distribution they are drawn from. Samples with certain pattern bias to certain classes. ID data bias ID classos uniformly because they are sampled by the same distribution. However, with little probability, OOD data will also bias to ID classes uniformly since they have little relevance to ID data
PL model performance
Confusion Matrix of PL model
PL
Supervisad
(%)
Prediction
12
15
Class MisMatch (%)
Ground Truth
(a) Performance of PT.
(b) Confusion Matrix of PT, model
(b) Confusion matrix PI, model when the mismatch ratio 100%. demonstrates that the imbalance of pseudo labels on OOD data affects PL's performance. A lot of ID samples with class 1-5 are misclassified into class 0. Also, the PL process continues, the imbalance of pseudo-labels
Figure 3: (a) PL model degrades as the mismatch ratio increases.
OOD data get even worse
Final Pseudo-Labeling model. As an old saying goes, good beginning is half done. However, such imbalance of the first set of pseudo-labels starts PT. model badly when there is large portion of OOD data, putting the model in danger of imbalanced learning. We run vanilla PL and show that the imbalance of pseudo-labels harms the performance. Figure 3(a) shows the performance of PL model with different OOD ratios. In accord with OOliver et al. 2018). Pl. model degrades as the portion of OOD data gets larger. Figure 3(b) displays the confusion matrix of the PL model on the whole test set containing both ID and OOD data. Since only 6 classes are known to us, the confusion matrix is a rectangle. We can almost.
1 How OOD data are generated affects imbalance ratio. experiment on different OOD class settings Appendix C.2 show that imbalanced pseudo-labels OOD data general phenomenon non-curated natural datasets.
4

---

Published in Transactions on Machine Learning Research (11/2022)
all the OOD samples (class 6-9) are classified as class 0, which means the imbalance effect on OOD data gets even worse as the PL training goes on. The possible reason is that, unlike Pseudo-Labellig on ID data, supervision of labeled data can not help correct pseudo-labels on OOD data. Thus the imbalance contimuuully deteriorates. The imbalance on OOD data also inffuences classification performance on ID data. Samples of major classes (class 0) overwhelm the loss and gradient, leading to a degenerate model (Lin et al. 2017). We can see the PL model mistakenly classifies many of data with class 1-5 into class 0.
3.2 Pseudo- Pseudo-Labeling Labeling Strategy for OOD data
The previous section shows OOD data hurt the performance of vanilla PL. Then here comes the question: Assuming that we already know which data are OOD, how do we use these OOD data? Is omitting them the best way? If not, what are the better pseudo-labels for them? To answer these questions, we investigate four strategies to create pseudo-labels for OOD data:
Baseline. This baseline omits all the OOD data and only trains on the labeled ID data.
Re- Assignedd Labeling This strategy assigns data of each OOD class to an ID class. It ensures that different OOD class is assigned to different ID class, keeping the semantics unchanged between OOD classes. For example, (ship, trunk, airline, automobile) can be assigned to (bird, cat, deer, dog). This strategy can be seen as training a classifier of "super-classes".
Open-Set Labeling. This strategy is named after the related setting Open-Set Recognition (Scheirer let al.; 2013: Bendale & Boult 2016). This strategy treats all OOD data as one unified class KjD + 1. Thus this model outputs probability over KID + classes.
Oracle Labeling. This strategy uses the ground truth of OOD data. Thus this model outputs probability over Kin + Koon classes.
Note that Open-Set Labeling and Oracle Labeling can classify samples into more than KID classes. However, during evaluation, we only classify samples into KID ID classes. For these models, the predicted label ğ of a test sample x is calculated
(xx) = arg max f(yl=)
(3)
DEVID
the overall comparison of the four strategics is illustrated in Figure We also report test accuracy on the five datasets when the class-mismatchedd ratio is 100% in Table 1 From the results, we can get several important conclusions. (1) Re- Assigned Labeling underperforms baseline a little. This indicates that assigning samples with OOD classos to ID classes does not help the model distinguish between ID classes even if we somehow know which OOD data are semantically different. It also reveals that performing vanilla PL on OOD data may never help even if we do it perfectly. (2) Open-Set Labeling outperforms baseline, which indicates it improves the performance if we label the OOD data
(3) We can see Oracle Labeling improves performance and achieves the best results among the four strategies. It means that in addition to labeling OOD data as extra classes, if we can further assign OOD data with different semantics to different classes, the model will achiove better results.
class other than ID classes.
Discussion. Why does Oracle Labeling consistently outperform Open-Set Labeling by large margin? We think the most important reason is that Oracle Labeling utilizes information among OOD classes. For example, in the experiment of CIFAR10(6/4) open-set labeling takes all the vehicle samples as one class, ignoring the fact that they come from four categories airplane, automobile, ship and truck. The difference among data of these classes may provide useful information not covered by labeled data of ID classes, especially when the labeled data is not plenty. For example, learning to distinguish between airplane and truck helps distinguish between bird and dog by judging whether having wings not. But the model trained by open-set labeling loses such benefit. In contrast, Oracle labeling can help the model capture this information by utilizing the ground truth labels. Consequently, oracle labeling performs better than open-set labeling.
Note that Re-Assigned Labeling has many possibilitics. If there are ID classes and OOD classes, AT possible assignments exist. impossible to experiment of them. To deal with it, we randomly choose possihle assignment and pick the maximum performance among them.
5

---

Published in Transactions on Machine Learning Research (11/2022)
ID data
OOD data
Decision Boundayy
(a) Baseline
Re-Assigned Labeling
(c) Open Labeling
(d) Oracle Labeling
Figure 4: Four strategies of how to label OOD data. Different shapes represent different ground truths. Data with the same color are labeled as the same classes. A shape with a solid outline means is an ID sample while OOD data are represented with a dashed line. (a) No Labeling acts as a baseline where OOD data are omitted. (b) Re-isneded Labeling re-labels OOD data certain IDD classes. (c) Open-Set Labeling labels all the OOD data a unified class. (d) Oracle Labeling uses the ground truths of OOD data.
Table I: Performance of five different pseudo-labeling strategies on different datasets. It can be concluded: (1) Re- Assigned Labeling underperforms baseline. (2) Open-Set Labeling outperforms baseline little. (3) Oracle Labeling improves performance and achieves the best results.
C10 (6/4)
SVHN (6/4)
C100 (50/50)
TIN (100/100)
IN (50/50)
Bascline
76.21 (-)
88.33 (-)
58.68 (-)
39.08 (-)
48.12 (-)
Re-Assigned
75.99(-0.22)
84.40(-3.93)
50.52(-8.16)
34.90(-4.76)
45.60 (-2.52)
Open-Set
77.95(+1.74)
88.31(-0.02)
58.76(+0.08)
40.06(+0.86)
49.56(+1.44)
Oracle
79.25(+3.00)
92.07(+3.74)
63.90(+3.73)
45.28(+6.78)
55.96(+7.84)
3.3
Summary of Section
In this section, we study the behavior of the Pseudo-Labeling model in clas--mismatched SSL. We summarize several important conclusions here:
Conclusion Classífication model trained with labeled TD data creates imbalanced pseudo-labels on OOD data while balanced pseudo-labels on ID data.
Conclusion 2: The vanilla PL makes the imbalance of pseudo-labels deteriorate, damaging the classification performance on ID data.
Conclusion Labeling OOD data ID classes does not help and may even perform
Conclusion is beneficial to label OOD data as extra classes differont from ID classos. If we can further label semantically different OOD data as different classes, the performance can be further improved.
4
Method
Based on the findings in Section 3 we proposed T-Model (named after its shape) for class- mismatched SSL. T-Model trains a classifier that will output the posterior distribution over KID + K classes, í.e, f(y (z)) 1. K is the number of extra classes, which be known in advance (i.e, K = KoOD) or be set
(yy2:) K 1
hyper-parameter Similar to vanilla. PL, define confidence with the same form as Equation [ However, this confidence is little different from its original definition in Hendrycks & Gimpel (2017), because we only calculate the maximum probability of the KjD classes instead of all. Therefore, we rename it to In-Distribution confidence (ID confidence). For evaluation, we predict labels using Equation T-Model aims to solve the following questions:
Problem 1: how to avoid imbalanced pseudo-labels in PL model? (Conclusion
Problem 2: how to avoid labeling OOD data ID? (Conclusion 3

---

Published in Transactions on Machine Learning Research (11/2022)
distribution of pseudo-lahels
clustering on extra K. clusses
<(x) > z
c(x) y
ID confident c(x)
data
0CD data
Dulance pseude-labels
truncatl
Kip
pseudo-labels for high-coonfddence data
pecudo-lahels
low-confidence data
Kip
unlabeled data (b) Y-model
(a) Re-balanced Pseudo-Labeling
(c) Semantic Exploration Clustering
Figure Illustration of T-Model and its two main branches. (b) is the main structure of T-Model where judge by the TD confidence if certain unlabeled data. belongs to ID classes not. The data with high confidence will perform Ro-balanced Pseudo-Labeling, while those with low confidence will get their pseudo-labels by Semantic Exploration Clustering. (a) Re-balanced Pseudo-Labeling truncales the number of pseudo-labeled data to the minimum, making the pseudo-labels balanced and filtering out OOD data. (c) Semantic Exploration Clustering simulates the process of learning from ground truth labels of OOD data, creating pseudo-labels on extra K classes by clustering.
Problem 3: how to create proper pseudo-labels for unlabeled OOD data? (Conclusion
Y-Model consists of two main branches Re-balanced Pseudo-Tabeling (RPI.) and Semantic Exploration Clustering (SEC). RPL acts high-confidence data to solve Problem | SEC acts on low-confidence data to solve Problem We describe the two branches in the following sections. The overview of T-Model is
g
illustrated in Figure 5
4.1 Re-balanced Pseudo-Labeling
As illustrated in Section 3.2 the main problem of vanilla PL that large number of OOD data with high confidence scores have imbalanced pseudo-labels. One possible solution is se-weighting the unlabeled sample (Guo et al. 2020) or using other methods in the imbalance lcarning field. However, even if we solve the problem of inbalance learning, labeling OOD data ID classes also may damage the performance (Conclusion 3 In this paper, we use a simple method - Re-balanced Pseudo Labeling to simultancously solve imbalance (Problem and incorrect recognition (Problem 2 It produces a set P of pseudo-labeled samples in three steps:
N min 1 E D. f x) r}}, VEVID
(4)
Ty select N-th((f(y) =) D,)),
(5)
p U (1x,)) f(y x)) Ty. E D.).
(6)
DEVID
where _eloct__Naaa returns the N-th biggest value of the given set. RPL first calculates the minimum number of psendo-labeled samples for each ID class by Equation Then it truncates the number of pseudo-labels of
each ID class to that number by Equation 5 The process of RPL is illustrated in Figure 5 aa) First, it
enforces the pseudo labels on ID classes to be balanced, solving Problem D Second, as shown Section 3 the of high-confidence data is a mixture of ID and ODD data. Due to Conclusion [ the pscudo-label distribution of such a set sum of imbalanced and balanced ones, thus still imbalanced. However, by selecting only top-N confident samples for each ID class, will keep ID data and omit many OOD data
since confidence on ID data tends to be higher than OOD data (Hendrycks & Gimpel 2017). This process solves Problem 2
4.2 Semantic Exploration Clustering
As demonstrated in Section 3.2, if we know a set of samples is OOD, will improve the performance we label them unified class K + 1. But the best way is to use their ground truths (Conclusion 4 However,
7

---

Published in Transactions on Machine Learning Research (11/2022)
it is impossible to access their ground truths since they are unlabeled. We resort to using Deep Clustering methods (Caron et al. 2018 Asano et. al.
2020 to mine their semantics and approximate the process of learning with the ground truths. Modern Deep Clustering can learn semantically meaningful clusters and achieves competitive results against supervised learning [Gansbeke et al.) 2020). Here, we use the balanced clustering method in Asano el al. (2020): Caron et al. (2020) to create pseudo-labels for these OOD data.
Assuming there are . samples recognized as OOD, we first compute their soft targets:
min (Q. 1 log P) QEU(K,M)
(7)
1 U(K, M) { IQ1 - 1, Q,
M M/)
where P E RRK fKID + i/00)). f is the normalized posterior distribution on extra classes, i. f(KID+ilz)) = ( Ek=1 f(KID + k|æ,), We use the Sinkhorn-Knopy algorithm |Cuturi 2013
to optimize Q. Once we get Q, we harden the label by picking the class with the maximum predicted probability and mapping it to the extra K classes:
Ú; = KID arg max Qij-
(8)
3j is used as the pseudo-label for Vy. We perform SEC on the set of data with ID confidence lower than a threshold 7, i.e, {æle(()) < <. It may be the concern that introducing a clustering component makes the T-Model too computationally expensive to be practical. We give analyses of time complexity in Appendix
5 Related Work
Class-Mismatched Supervupe Learning. Deep Supervsuerr Learning suffers from perfor- mance degradation when there are unseen classes unlabeled data (Oliver et al. 2018). As the proportion of such out-of-distribution (OOD) data get larger, the performance drop more. To cope with such a class-
mismatched problem, several methods are proposed. Chen et al. (2220) formulate sequence of ensemble models aggregated accumulatively on-the-fly for joint self-distllation and OOD filtering. Guo et al. (0020)
ro-weight the unlabeled data by meta-learning to decrease the negative effect of OOD data. Huang et al.
(2020) recycle transferable OOD data employing adversarial learning. Recto et al. (2021) proposed open-set consistency regularization to improve outlier detection. Cao et al. (2021) proposed open-world semi-supervised learning, where the classes of unlabeled data need to be discovered. From a methodological
perspective, both [ao et al. (2021) and our method use cluster to mining semantics. However, their method is originated from the demand of discovering novel classes, while ours is based on analyses of best labeling solve the imbalanced pseudo-labels
strategies. ORCA does not give such important analyses. Additionally, by RPL while they do not reveal this problem. Different from all these methods, we conduct a comprchensive study on Pseudo-Labeling (PL) and give useful guidance on how to do betler in misssmmisete SSL. In addition to that, we reveal the imbalance phenomenon and propose RPL.
Pseudo-Labeling. The method of Pseudo- Leoudo-Labeling, also known as self-training is a simple and effective way for Deep SSL [Le| 2013; Shi et al. 2018 Arazo et al.) 2020 Iscen et al.) 2019 ). Despite its simplicity, it has been widely applied to diverse fields such as image classification (Xie el al. 2020), natural language processing (He et al.) (2200) and object detection (Rosenberg et al.) (005). The use of a hard label makes Pscudo-Labeling closely related to entropy minimization (Grandvalet & Bengio 2004
Deep Clustering and Novel Class Discovery. Deep clustering methods improve the ability of traditional cluster methods by leveraging the representation power of DNNs. A common means is to transform data into low-dimensional feature vectors and apply traditional clustering methods (Yang et al. 2017 Caron
ct al. 2018). In Self-Supervieed Learning, clustering methods are used to learn mcaningful representation for downstream (asks (Caron et al. 2018 Asano et al.) 2020 Caron et al. 2020). Modern Deep Clustering can learn semantically meaningful clusters and achieves competitive results against supervised learning (Gansbeke ct al. 2020). With the help of Deep Clustoring, the concept of Novel Class Discovery (NCD) was first formally
introduced by Han et al. (2019). There are two main differences between NCD and our setting. First, the goal of NCD is to correctly cluster OOD data while semi-supervised learning aims to correctly classify ID
8

---

Published in Transactions on Machine Learning Research (11/2022)
data. Second, NCD knows which data are OOD in advance while llas-mmismtthhed SSL does not. Therefore, NCD methods can not direclly apply to this setling. A recent method, UNO [1], uses a similar cluster method to discover semantics among OOD data like our SEC component. However, since they mainly focus on cluster OOD data, they do not reveal the benefit for classifying ID data when mining the semantics of OOD data, which is one of the contributions of this paper.
6 Experiments
Dataset. We test our methods the five datasets as in Section 3 i.e, CIFAR10 (6/4), SVHN (6/4), CIFAR100 (50/50), Tiny ImageNet (100/100) and ImageNet100 (50/50). The class-mismatched ratio is set as (0%, 25%, 50%, 75%, 100%].
Implementation Details. We use the same network and training protocol Section 3 classification model only on labeled data for 100 epochs without RPL and SEC. We update pscudo-labels every 2 epochs. For both datasets, we set T = 0.95, 0.3,K = 4. We use an exponential moving average model for final evaluation as in Athiwaratkun et al. (019))
We first train a
CIFARIB, Labele Each
SVHNN Labela
CIFARIO, 408
Yododd uemo
Yolall Scand
FL
Toi Accenncy (50)
Tee Aocerna
VAT
Saperad
Too Accerpoc
Fl.
RM0) Apecial
Sepnsie
Cbas MicMatch (%)
Clas MixMarch (%)
Clan MhMaath (%)
Class Mietaach (%)
(a)
(b)
(c)
(d)
Figure 6: Comparison with existing methods CIFAR10 and SVHN dataset with Nest--22 network. Class mismatch ratios varied. The shaded regions with the curves indicate the standard deviations of
the accuracies over five runs. (a) (b) Comparison with traditional SSL methods. These methods sulfer from
performance degradation as the mismatch ratio increases. (c) (d) Comparison to two existing class- mass-mmssttthed
SSL methods UASD and D Our methods perform better in almost all the experimental setups.
6.1
Compare with Traditional SSL methods
In this subsection, compare our methods with four traditional SSL methods Pseudo-Tabbllng Lee
2013), MI-Model (Laine & Aila 2017), Mean Teacher [Tavvainnn & Valpola, 2017) and VAI (Miyato et al, b(b) show the results. Traditional mothods suffer from performance degradation as the mismatch ralio increases. They usually get worse than the supervised baseline when the mismatch ratio is larger than 50% on CTFARIO and SVHN. In contrast, our methods get steady improvement under all class mismatch ratios. The reasons can be attributed as follows. First, our method is aware of the existence of OOD data. We do not treat OOD data like ID data, which can hurt performance. Second, reuse OOD data by exploring their semantics which proves to be useful in Section 3.2] Therefore, even when the clase-mismatcher ratio gets 100%, the performance of T-Model is still better than the supervised baseline.
2019 Figure 6(a)
6.2 Compare with MissaMMis SSL methods
In this subsection, we compare our method with two existing class- mismatched SSL methods UASD Chhen et al. 2020) and D (uuo al.
2020 For fair comparison, we Pseudo-Labeling the base method
of D LL. From Figure [(c) Figure 6(d) we see methods are superior to these two methods in all
settings. It is noticeable that L underperforms supervised baseline when all the unlabeled data are drawn
WWe note that there also other methods like FixMatch (ooh al. 2020 and MixMatch (Berthclot al. 2019 focus augmentation other tricks. Their contrihution is orthogonal ours. Also, FixMatch is unstable in this setting. We provide comprehensive comparisons with these mothods in A Appendix
9

---

Published in Transactions on Machine Learning Research (11/2022)
Table 2: Comparison three hid/llrge-scale datasets with different miiasmmsmmttced ratios. The backbone is Mide-ResNet-28-2 for all experiments. The standard deviations of the accuracies over five runs are also reported. The best results are highlighted in bold. The second-best results are highlighted in underline. Baseline and Oracle have the same meaning in Section 3.2
Mismatch Ratio
0%
25%
50%
75%
100%
CIFAR100 (50/50)
PL
61.68 + 0.30
60.20 + 0.25
60.12 + 0.30
57.79 = 0.27
57.62 0.57
UASD
60.30 + 0.26
59.42 + 0.61
59.92 + 0.53
58.94 + 0.63
58.74 + 0.50
D
60.68 + 0.67
60.60 + 0.40
59.22 + 0.33
59.74 = 0.37
59.56 + 0.57
T-Model
62.10 + 0.30
61.26 + 0.40
60.68 + 0.24
60.12 = 0.38
59.46 + 0.36
ORCA
58.94 + 0.24
59.98 + 0.35
60.14 + 0.40
58.84 = 0.27
44.00 + 0.42
OpenMatch
62.08 + 0.26
60.94 + 0.36
60.92 + 0.23
60.36 + 0.47
59.22 + 0.50
Baseline
58.68 + 0.25
Oracle
75.46 + 0.20
73.36 + 0.29
72.94 + 0.25
69.26 + 0.26
63.90 0.30
Tiny ImageNet (100/100)
PL
43.42 + 1.03
42.88 + 1.51
41.94 + 1.49
39.72 + 2.30
38.94 2.41
UASD
43.34 + 0.78
42.34 + 0.61
41.80 + 1.33
41.08 + 1.16
36.16 1.05
DS
T-Model
44.42 + 0.43
43.48 + 0.40
42.42 + 0.95
43.22 + 0.60
41.76 + 0.63
ORCA
42.16 + 0.41
40.22
+ 0.37
41.58 + 1.24
42.92 + 0.57
40.82 + 0.36
OpenMatch
43.42 + 0.31
42.00 + 0.37
42.26 + 0.27
42.5 + 0.31
41.03 + 0.38
Baseline
39.66 + 0.52
Oracle
56.18 + 0.31
53.8
0.28
51.82 + 0.20
48.3 + 0.30
45.28 0.31
ImageNet (50/50)
PL
50.04 + 0.11 50.24 + 0.20
49.32 + 0.15
48.36 + 0.19
47.40 + 0.13
46.96 0.19
UASD
48.68 + 0.26
48.12 + 0.17
48.00 = 0.11
47.04 + 0.25
DS3L
T-Model
50.6 + 0.19
49.78 + 0.16
48.88 + 0.30
48.36 + 0.11
47.64 + 0.22
ORCA
41.56 + 0.27
43.32 + 0.38
46.60 + 0.23
47.08 + 0.45
46.88 + 0.36
OpenMatch
49.92 + 0.11
49.88 + 0.15
48.82 + 0.19
48.01 - 0.21
46.84 + 0.27
Baseline
48.12 12 + 0.29
Oracle
62.92 + 0.26
62.00 + 0.28
61.04 + 0.26
57.92 + 0.26
55.96 0.30
We cannot finish D on mid-scale or large-scale datasets like Tiny ImageNet or ImageNet100 within reasonable time.
from OOD classes. This attributed to the fact that L uses down weighting strategy alleviate the negative effect of OOD data and does not change the form of unsupervised loss. But we have shown in Section 3.2 that labeling OOD data ID classes damages performance anyhow. On the contrary, T-Model uses the OOD data in the right way simulatiin the process of training them with their ground truth labels. As a result, our method shows superiority especially under large class- moassmmmmmtthhd ratio. We also notice that the performance curve of T-Model appears a U-shape (obvious on CIFAR10). A possible reason is that RPT. and SEC compete with each other. RPT tends make samples get a high prediction on ID classes while SEC tends to make samples get a high prediction on OOD classes. When the dass-mismatchhl ratio reaches 0% (100%), RPL (SEC) dominates the other. In this circumstance, one works without any disturbance to the other. However, when the class-mist masamachstt ratio is 50% they compete fiercely with each other, causing many incorrectly recognized ID OOD samples.
Additionally, we compare our method to OpenMatch (Saito et al. 2021 and ORCA (Cao et. al. 2021 on mid- llrge-scalee datasets in Section Again we use the same augmentation for both labeled and unlabeled data to avoid unfairness for baseline. ORCA is an open-world semi-supervised lcarning method, which not only aims correctly classify ID data but also to cluster OOD. Due to this, its ID classification performance is lower than others. OpenMatch is competitive method, but it can not surpass ours in most of the settings.
10

---

Published in Transactions on Machine Learning Research (11/2022)
Mis-laheled Ratio
Confusion Matrix of Y-Model
Proportion (%)
Ground Truth
Test Accuracy
16
25%
LDG
22
200
75%,
100%
RPL.
Supersisol
Epoch
Prediction
Number Extra Classes
(b)
(c)
Figure 7: Two experiments on CIFAR10 benchmark to validate the functionality of RPL and SEC. (a) The proportion of OOD data pseudo-labeled as ID classes with the training goes on. Vanilla PL makes this ratio keep increasing while RPL makes drop to O. It demonstrates that RPL help filter out OOD data. (b) Confusion matrix the full test set of CIFAR10. RPL solves the imbalance problem.
6.3 Ablation Study
In this section, validate the functionality of RPL and SEC, We conduct experiments CIFARIO benchmark as in the analysis section 3
Table 3: Validation of RPL and SEC. The experiments are conducted on CIFARIO with a Wide-ResNet-28-- backbone. Class-mismatched ratio varies from O to 100%. Check RPL, or not means using RPL, or vanilla PL. K = 0 means we do not use SEC. K = means we label all the low confidence data as a unified class KID + 1.
Mlass-Missatched Ratio (%)
RPL SEC
0
25
50
75
100
78.23 + 0.39
76.50 + 0.30
75.85 + 0.35
74.60 + 0.27
73.97 + 0.30
78.76 + 0.49
76.92 = 0.32
76.80
0.31
75.48 + 0.27
75.38 + 0.27
78.86 + 0.37
77.17 + 0.55
77.12
0.37
76.88 + 0.35
77.27 + 0.39
78.11 = 0.45
77.43 = 0.29
77.46
0.37
76.38 0.37
74.18 + 0.32
/
78.98 + 0.49
78.93 + 0.28
77.78 + 0.31
77.90 = 0.21
78.50 + 0.27
Baseline
76.21 + 0.21
Validation of effectiveness of RPL and SEC. We conduct ablation studies under different class- mismatched ratios and report the averaged test accuracy and standard deviation of five runs. As usual, we vary the class-mismatchedd ratio. Table 3 displays the results. Firstly, comparing the first line and second line the table, RPL. not only outperforms vanilla PI. in high class-mismatched ratio scenarios but also improves low class-misnn mismatched ratio scenarios. This reveals that balanced pseudo-labels always help since once the model creates imbalanced pscudo-labels, will deteriorate when there are not enough measures correct it. Secondly, comparing the second and third line, it shows that RPL alone alleviate the performance degradation but it not prevent it, in accord with Conclusion 3 When using SEC, T-Model gets better results than supervised baseline when the class-mismatcherr ratio is high. Besides, comparing the third and last lines, we see that when clustering OOD data and exploring their semantics instead of using unified class to label them, the performance improves.
RPL helps filter out OOD data and solve the imbalance problem. It is noticeable that the last. two lines Table 3 show that without RPL, SEC alone can not achieve better performance than supervised bascline. We show the reason here. Figure 7(a) plots the proportion of OOD data that are pscudo-labeled as ID classes. Without RPL, í.e, using vanilla PL, the number of incorrectly recognized OOD data keep increasing the training proceeds. While with RPI, this ratio rapidly drops to 0. This proves that RPT.
11

---

Published in Transactions on Machine Learning Research (11/2022)
help filter out OOD data by utilizing the imbalance property of OOD data. Further, we present the confusion matrix of T-Model on the full test set (all the 10 classes) of CIFAR10. Compared to vanilla PL, in Figure 3(b) Y-Model does not suffer from imbalance problem, as result of which, its performance is not degraded.
Effect of extra class number K. We vary the number of extra classes K. Figure 7 shows the result on CIFARIO with various class mismatch ratios. Without SEC (K = 0); Y-Model underperforms the supervised baseline. Using SEC (K 2 1), T-Model is always better than baseline. Also, it reaches its best performance when K is roughly the actual number of OOD classes. This demonstrates that by simulating the process of training on OOD data with ground truth, SEC helps the classification model on ID data. Also, our model benefits from SEC but is not sensitive to the selection of K. SEC helps explore the semantic structure of OOD data, but does not affect the learning on ID data since they perform tasks on different classifiers.
Table 4: Comparison of PL and T-Model on the constructed clase-mismatched CIFARIO dataset with imbalanced OOD data.
Class- MissstMisttc Ratio (%)
25
50
75
100
PL
76.42
75.78
74.58
73.33
Y-Model (Ours)
77.57
77.42
77.13
77.26
Effectiveness on imbalanced OOD data. It might be a question that whether it is suitable to use balanced clustering since the OOD data are usually imbalanced in the real world. We note that we use cluster methods only to mine the semantics among OOD data. We do not aim to correctly recognize them. The effectiveness of using clustering methods to learn semantically meaningful representations has been proved in many works (Asano al. 2020; Caron al., 2020). To prove the effectiveness of our method, we conduct experiments on additional benchmark where the OOD data is imbalanced. The OOD data are subsampled like CIFAR-1O-LT (Cao et al. 2019) with imbalance ratio of 10 the OOD classes. Table 4 displays the results. We can see that our model can handle situations where OOD data are even imbalanced.
Other imbalance methods and uncertainty measures. We present results of an imbalanced method applied to PL: PL-Reweight and two uncertainty measures applied to model: T-Model (Ent) and Y-Model (SD). Ent and SD detect OOD data via entropy and score difference. Ent uses negative entropy of the output distribution as the confident measure. SD use the difference between the largest and the second-largest output probability as the confidence score.
Table 5: Experiments on combining different imbalance and OOD detection methods with PL/T-Model. PL-Reweight uses reweight strategy to balance pseudo-labels. Ent (Entropy) uses negative entropy of the output distribution as the confident measure. SD (Score Difference) uses the difference between the largest and the larges- output probability as the confidence score.
Class- Mismatch Ratio (%)
25
50
75
100
PL
76.5
75.85
74.6
73.97
PL. PL-Reeeighh
76.98
76.33
74.52
diverge
T-Model (Ent)
77.78
77.97
76.32
77.12
T-Model (SD)
77.70
78.25
78.02
76.72
M-Model (Ours) 78.93
77.78
77.90
78.50
Common imbalance methods like resample and reweight will be unstable in this setting since there is the possibility that the number of pseudo-labels is zero in certain classes. Also, in Section 3.3] we have concluded that assigning ID labels for OOD data will degrade the model. Therefore, solving the imbalance problem is only partial solution for this setting. In contrast, our RPL is stable this situation and uses the imbalance phenomenon to simultancously filter out OOD data. Also, uncertainty measures other than confidence score are applicable to method.
12

---

Published in Transactions on Machine Learning Research (11/2022)
7 Conclusion
In this paper, analyze Pseudo-Labeling in class-mssmacchdd semi-supervised learning where there are unlabeled OOD data from other classes. We show that Pseudo- Labellig suffers from performance degradation due to imbalanced pseudo-labels on OOD data. The correct way to use OOD data is to label them as classes different from classes while also partitioning them according to their semantics. Based on the analysis, we proposed T-Model and empirically validate effectiveness.
In future work, will explore whether other forms of semi-supervised learning methods like the consistency- based method suffer from the same problems. Also, currently our model is based on Pseudo-labeling, But we see the possibility of extending it to other SSL methods. For example, there may be re-balanced consistency methods and SEC may be alug-and-play component for all SSL methods.
8 Acknowledgments
This work is partially supported by The National Key R&D Program of China (2020AAA01094001) NSFC (62006112, 61773198, 61921006), NSF of Jiangsu Province (BK20200313), CCF-Baidu Open Fund (N0.2021P15000000)
References
Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo-labeling and confirmation bias in deep semi-supervised learning. In IJCNN, pp. 1-8. IEEE, 2020.
Yuki Markus Asano, Christian Rupprocht, and Andrea Vedaldi. Self-labelling via simultancous clustering and representation learning. In ICLR, 2020.
Ben Athiwaratkun, Mare Finzi, Pavel Izmailov, and Andrew Gordon Wilson. There are many consistent explanations of unlabeled data: Why you should average. In ICLR, 2019.
Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In CVPR, pp. 1563-1572, 2016.
David Berthelot, Nicholas Carlini, Ian J. Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. In NewrIPS, pp. 5050-5060 2019.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Aréchiga, and Tengyu Ma. Learning imbalanced datasets with label-distrrbutionnaaaar margin loss. In NewrIPS, pp. 1565-1576, 2019.
Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. CoRR, abs abs/2102.03326, 2021.
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustcring for unsupervised learning of visual features. In ECCV, pp. 139 156, 2018.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. NewrIPS, 2020.
Yanbei Chen, Xiatian Zhu, Wei Li, and Shaogang Gong. Semi-supervised learning under class distribution mismatch. In AAAI, pp. 3569-3576, 2020.
Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with reduced search space. In NeurIPS, 2020.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In NIPS, pp. 2292-2300, 2013.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248 255, 2009.
13

---

Published in Transactions on Machine Learning Research (11/2022)
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Mare Proesmans, and Luc Van Gool. SCAN: learning to classify images without labels. In ECCV, pp. 268 218-285,-- 2020.
Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In NIPS, pp. 529-536, 2004.
Lan-Zhe Guo, Zhenyu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. Safe deep semi-supervised learning for unsoen-class unlabeled data. In ICML, pp. 3897-3906, 2020.
Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustcring. In ICCV, pp. 8400-8408, 2019.
Junxian He, Jiatao Gu, Jiajum Shen, and Marc Aurelio Ranzato. Revisiting self-training for neural sequence generation. In ICLR, 2020.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Junkai Huang, Chaowei Fang, Weikai Chen, Zhenhua Chai, Xiaolin Wei, Pengxu Wei, Liang Lin, and Guanbin Li. Trash to treasure: Harvesting OOD data with cross-modal matching for open-set cuni-supervised learning. In ICCV, pp. 8290-8299, 2021.
Zhuo Huang, Ying Tai, Chengjie Wang, Jian Yang, and Chen Gong. They are not completely useless: Towards recycling transferable unlabeled data for class- dass-mismatched semi-supervised learning. CoRR, abs/2011.13529, 2020.
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi-supervised learning. In CVPR, pp. 5070 5079, 2019.
Diederik P. Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised lcarning with deep generative models. In NIPS, pp. 3581-3589, 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In ICLR, 2017.
Dong-Hyun Lec. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896, 2013.
Tsung-Yi Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pp. 2999 3007, 2017.
Huixiang Luo, Hao Cheng, Yuting Gao, Ke Li, Mengdan Zhang, Fanxu Meng, Xiaowei Guo, Feiyue Huang, and Xing Sun. On the consistency training for open-set semi-supervised learning. CoRR, abs/2101.08237, 2021.
Takeru Miyalo, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regular- ization method for supervisod and semi-supervised learning. TPAMT, (1(8):1979-1998, 2019.
Avital Oliver, Augustus Odena, Colin Raffel, Ekin Dogus Cubuk, and Tan J. Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. In NeurIPS, pp. 2239-3250, 2018.
Chuck Rosenberg, Martial Hebert, and Henry Schneiderman. Semi-supervised self-training of object detection models. In WACV, pp. 29-36, 2005.
Kuniaki Saito, Donghyun Kim, and Kate Saenko. Openmatch: Open-set semi-supervised learning with open-set consistency regularization. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jenuifer Wortman Vaughan (eds.), NeurIPS, pp. 15956-25966 2021.
Walter Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E. Boult. Toward open set recognition. TPAMI, (5(7) 1772, 2013.
14

---

Published in Transactions on Machine Learning Research (11/2022)
Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng Ma, Xiaoyu Tao, and Nanning Zheng. Transductive semi-supervised deep learning using min-max features. In ECCV, pp. 311-327, 2018.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NewrIPS, 2020.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NIPS, pp. 1195-1204, 2017.
Qizhe Xie, Minh-Thang Luong, Eduard Н. Hovy, and Quoe V. Le. Self-training with noisy student improves imagenet classification. In CVPR, pp. 10684-10695, 2020.
Bo Yang. Xiao Fu, Nicholas D. Sidiropoulos, and Mingyi Hong. Towards k-means-friendly spaces: Simultancous deep learning and clustering. In TCML, pp. 3861-3870) 2017.
Qing Yu, Daiki Ikami, Go Irie, and Kiyoharu Aizawa. Multi-task curriculum framework for open-sel. semi-supervised learning. In ECCV, pp. 438 084,45,,40 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. Tn BMVC, 2016.
15

---

Published in Transactions on Machine Learning Research (11/2022)
A
Algorithm
Algorithm T-Model algorithm
Require: Labeled dataset Dj {( M= and unlabeled dataset D,, { Classification model
fo parameterized with Φ, ID class number KDD, extra class number K, total epoch number E, pretrain epochs Ept, interval to update pseudo-labels Ept, pseudo-labeled set P, confidence caleulation function f)
1: function
2: N < mon xx E Du f x) 7
3: f Nth (i a)
æ E Du)), = 1,2,..., KID
UpE ((x, v) fly x) 2 Ty. D.)
return P
function SARANTTOEPPOORRTOOOCCSSSEEEEE
S St {x]](())= < <}}
8:
M + |S|
P.. f(KID + f/z,)/ Ex f(Ki + k|æ,), i= 1. K, j= 1...
10:
Solve by Sinkhorn-Knopp algorithm and get Q
11:
Új KID + arg max; Qij- j = 1.2...... M
12:
C C {a0j, )5)
13:
return
14: for == to E do
15:
if < Epp then
16:
train So with standard supervised learning on Dr
Pre-training phase
17:
else
18:
train fo with standard supervised learning on UPP
PL training phase
19:
c S Ept and e Epp then
20:
P p+0 0
21:
P. + PRRALL/NCppPPErrO.....LIINN...,D P, SSAATTTIEEPPORRRTTOOCCTTTTETEE.E
b. Perform RPL Perform SEC
22:
23:
P P. UP,
24:
return classification model fo
B Embedding visualization
We visualize the embedding of supervised baseline, vanilla PL and v-Model on CIFAR10's test set with all the classes by (-SNE. Figure S shows the result. OOD data are mixed with ID data since the supervised baseline does not see unlabeled OOD data. PL mixes OOD data with samples of certain classes (class 0). This is attributed to their pscudo-labels being biased toward this class. Also, we can not clearly distinguish between OOD classes. In contrast, T-Model can not only make ID data distinguishable but also forms meaningful clusters on OOD data.
100
-203
-200
-200 200
283
(a) Bascline
(b) PL
(c) T-Model
Figure 8: t-SNE visualization supervised baseline, vanilla PL and T-Model on CIFAR10's test set. Class 0-5 are ID classos, shown by non-transparent circles. Class 6-9 are OOD classes, represented by semi-transparent circles.
16

---

Published in Transactions on Machine Learning Research (11/2022)
C Imbalance of Pseudo-Labels
C.1 Metrics
To demonstrate the imbalance of pseudo-labols on OOD data, we compute two metrics to mcasure the extent. of imbalance.
The KL divergence of pseudo-label distribution q and the uniform distribution u:
kl = KL(gllu) = Σ qs log qi
The ratio of the 'majority class' and 'minority class':
max; qs min;
The results on these datasets are displayed Table
Table 6: Illustration of the extent of imbalance of pseudo-labels different. datasets.
C10 (6/4)
C10 (5/5)
SVHN (6/4)
C100 (50/50)
TIN (100/100)
kl
ID OOD
0.0131
0.0169
0.0006
0.0771 0.4177
0.0728
0.3636
0.0429
0.1031
0.4089
r
ID
1.5390 18.3183
1.6235 2.4878
1.1047
5.1834
3.6609
OOD
4.4797
213.7661
187.2124
C.2 Different Selected OOD Classes
In this section, show that the imbalance phenomenon is general in real-world datasets. We conduct experiments on CIFAR10(6/4), SVHN(6/4), CIFAR100(50/50) and ImageNet ImageNet100(50/5000 The three datasets cover both small-scale and large-scale datasets, cover datasets with both small and large number of classes, and also cover datasets with hierarchical (CIFAR100) and non-hierarchical (SVHN) classes. We also add a datasct with 6 random ID classes from CIFAR10 and random OOD classes from SVHN, donoted as (IPARIO())SVHNN()) This mixed dataset simulates the condition where all the OOD data come from domain largely different from the ID data. For each dataset, we randomly select different set of classes as the ID classes, and the remaining classes as OOD classes except CIFAR10(6)/SHHII /SHH((4) A classification model is trained on ID classes and the imbalance ralio on both ID and OOD classes is reported. We repeat for 50 times each dataset.
Figure g display the results on these datascts. We can draw some conclusions from this figure. First, each trial, the imbalance ratio on ID classes is always lower than OOD by a large margin. proves that the imbalanced pscudo labels are common on OOD data. Second, imbalance ratios have much lower deviation on ID classes than on OOD classes. is natural since ID data are selected from the same distribution as the training data, while the OOD data come from irrelevant domain. Therefore, pseudo-labels these OOD data appear with high randomization, which means high imbalance ratio and high deviation.
We agree that whether pseudo-labels on OOD data are imbalanced depends on how they are generated. is always possible to manually pick a set of OOD data that have balanced pseudo-labels, However, we are here show that for a non-curated natural dataset, the imbalance on OOD data general phenomenon. With little possibility, the pseudo-labels happen to be balanced on these data. As result of it, method is effective in most conditions.
Discussion.
17

---

Published in Transactions on Machine Learning Research (11/2022)
Iroghii6
CHAMBID SICEN0
00E
0c0
000
15
Inbaleesees 07
Petio Patio
Intraloe Stato
0.0
Intasaan
Ittaobee
022
Eiperimnnt
Reperiment
Eapetiment
Begattmen
Figure 9: Imbalance ratio ID and OOD classes with different generations of ID and OOD data. It be seen that pseudo-labels ID classes always are always (relatively) balanced. The imbalanced ratios of ID classes concentrate around a certain low value. In contrast, pscudo-labels on OOD data have a much higher imbalanced ratio. Among different OOD settings, the imbalance ratios vary much but all of them lie higher than ratio classes by a large margin.
D Comparison with FixMatch and MixMatch
We display the results of FixMatch (Sohn et al. 2020 and MixMatch (Berthelot. et al. 2019). For fair comparison, we use the same augmentation for labeled and unlabeled data here. The augmentation 'paper'
means what we used in the paper. is commonly used in class-mismatched settings (Oliver et al. 2018 Guo
et al. 2020 RandAug (Cubuk et al. 2020) is the augmentation used in FixMatch.
Table 7: Results of FixMatch, MixMatch, Pseudo-Labeling (PL) and our method. The results are reported on CIFAR-10 (6/4) with different class mismatch ratios.
Augmentation
25%
50%
75%
100%
MixMatch
paper
77.88
77.15
76.83
75.9
75.85
PI.
paper
78.23
76.5
75.85
74.6
73.97
FixMatch
paper
78.75
76.25
diverge
diverge 77.9
diverge 78.5
T-Model (ours)
paper
78.98
78.93
77.78
Baseline
paper
76.21
MixMatch
RandAug
81.98
80.08
76.75
73.45
69.33
PL
RandAug
88.30
86.80
85.61
83.72
81.62
FixMatch
RandAug
87.35
84.314
82.367
78.733
diverge
T-Model (ours) RandAug
88.35
86.83
85.91
85.25
84.31
Bascline
RandAug
83.75
Some conclusions can be drawn from Table 7 FixMatch unstable in the class- mass-mismattchd setting while MixMatch is more stable, The former is mainly caused by the imbalance of pseudo-labels, MixMatch's 's stability may be brought about by its mixup operation. FixMatch and MixMatch all suffer from performance degradation in such a setting, The strong augmentation strategy in FixMatch can bring improvement to our method. It brings improvement to all the methods, but we also emphasize that it also pulls up the baseline.
E
Hyperparameters on Different Datasets
We compare our method with vanilla PL and the two class-mismatein methods in Section following hyperparameters:
6.2 We use the
CIFARI0 (6/4): 0.95, = 0.3, Ept 50, Ept = 2, K =
SVHN (6/4): 0.95, 0.3, Fppe 50, Fpp. 2, K
18

---

Published in Transactions on Machine Learning Research (11/2022)
CIFAR100 (50/50):7 0.95, 7 = 0.18, Ept = 2, = 20
Tiny ImageNet (100/100): 0.9. 0.15, Ept 50, = 20
ImageNet (50/50): T 0.9, 7 0.20, Ept 50, Ep 2, K 20
For CIFAR100 (50/50) Tiny ImageNet (100/100) and ImageNet (50/50), we use weight factor À to trade off the loss on labeled set Di and pseudo-labeled set P, which ramps up with function -1))=)1))))))))))))
x = exp -5 x
min
iter 40,000
where iter is the number of training steps from Ept-
F Analysis of Time Complexity
It may be the concern that introducing an extra clustering component causes the T-Model to be too computationally expensive to be practical. We want to emphasize that it is not the case. We have empirically tested the run time Y-Model compared with vanilla. Pseudo-Tabelling We display the comparison results in Figure 10 The upper two figures show the test accuracy and relative runtime (compare to vanilla Pseudo-Labeling with varying numbers of extra classes K. can be shown that the time complexity is almost irrelevant K. The lower two figures show the relationship to the Sinkhorn iterations, which control the quality of clustering (Cuturi, 2013 Asano et al. 2020). It be seen that both the performance and time complexity increases with the number of iterations. However, we want to note that the X-axis is in log scale. When the number increases ex exponentially, the time increases nearly linearly. Even though we use 32 iterations, which is 6x the number in Asano et al. (2020), it cost not more than 1.7x time of vanilla Pseudo-Labeling.
SSarsa Beto 20%
Miracc- 100)
Tex Nocanc
11
The test accuracy and relative runtime (compare to vanilla Pseudo-Labeling! v.s. the number of clusters K and Sinkhorn iteration. The time complexity is not relevant to K, but it affects the test accuracy. Both the accuracy and time complexity increase with the number of iterations. But when the number of iterations increases exponentially, the time increases nearly linearly.
Figure 10:
Our method is not inefficient as it may seem. First, the OT-based clustering is fast by the Sinkhorn-Knopp algorithm (Cuturi 2013). Several works using clustering have been proved efficient even on large-scale datasets (Asano et al. 2020 Caron et al., 2020). Our SEC component has similar functionality and setup in practice. Second, we perform clustering periodically. We do clustering every 2 epochs. Compared to the network updating cost, clustering spends an acceptable time.
G Discussion about Limitation and Social Impacts
Currently, our model is based on Pseudo-labeling. In future work, we will explore whether other forms of semi-supervised learning methods like the consistency-bu based method suffer from the same problems. And we will investigate whether the rebalance and semantic explore strategy can also benefit other forms of semi-supervised learning methods.
This paper explores a way of saving Pseudo-labeling methods from the harm of class mismatched unlabeled data. The results of this paper can spare the effort of clcaning unlabeled data, which can benefit the development and application of semi-supervised learning. Currently, we do not see a direct negative impact society.
19


Reviews:
[{"review_id": "r97E1knrer", "reviewer": "Reviewer_hokU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nThe paper is a good try on analysing the relationship between a particular SSL method (pseudo labeling) and OOD test samples. There are some interesting findings from the experiments. The paper then proposes new algorithms based on the interesting findings. The proposed new algorithm is a straightforward modification of the existing method but does take effect as shown in the experiments.\n\nThere are some concerns on the “reason analysis” part of the paper. On the finding of imbalance, the actual imbalance ratio depends heavily on the original distribution of the data, and how the OOD data is generated. I do not see sufficient discussion and experiments on this aspect yet in the paper. I think to make the observation general instead of specially on some datasets, different data generation and distribution should be tested to see whether the pseudo labels for OOD data are imbalanced. The paper needs extension and clarity on this issue. \n\nA minor concern would be on the labeling strategy part as well. Between open-set and oracle, there is another method: pseudo labeling to each OOD class instead of one whole class for OOD data. Since the paper uses clustering in the proposed method, I believe labeling the OOD data to each individual OOD class could be a better choice, and would like to see more experimental results on them. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents the effect of OOD sample on the pseudo labelling method for SSL, and analyses by experiments on why such effect happens. The conclusion is that when models pseudo label the unlabelled data, the class distribution on OOD data is more imbalanced on ID data. For this reason, the paper proposes to balance the pseudo labelled data. The paper also found by experiments that label those OOD data as one additional label could best benefit the model and proposes to use clustering method to find those OOD data and pseudo label them. Experiments demonstrate the effectiveness of the method. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. More data distribution and generation settings for showing that the \"imbalance\" finding is general.\n2. If we pseudo-label the OOD data to each class instead of one whole class, what will be the results? ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IBAJhjdq0e", "reviewer": "Reviewer_FPtq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### Strengths. \n1. Their finding on imbalanced pseudo-labels for OOD data is interesting and may be worth sharing in the research community. \n2. Their simple proposed method is robust to different degrees of class-mismatch, which seems to support their claim. \n3. They have conducted an extensive analysis with respect to the datasets, different degrees of class-mismatch, and ablation study. \n\n#### Weakness. \nComparison to existing work is limited. As seen in related work, Saito et al. proposed a method in a similar setting, but they have not done any comparison or analysis on this kind of approach. Other than this work, there are several methods for open-set semi-supervised learning. Second, in the literature on open-world semi-supervised learning such as Cao et al. and [1], the use of clustering algorithm is also explored. Comparison or discussion on these works is quite limited. The authors need to see if these existing works have similar findings or not. Also, from a methodological perspective, they have to discuss the novelty of the proposed method. \n\n\n[1] A Unified Objective for Novel Class Discovery (ICCV 2021)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "They tackle the semi-supervised learning scenario where unlabeled data can contain classes unseen in the training set. Under this scenario, they discover the following things: 1. a model trained with labeled ID data creates imbalanced pseudo-labels on OOD data while creating balanced pseudo-labels on ID data, which can degrade the performance of the model, 2. It is beneficial to label OOD data as extra classes different from ID classes and applying a clustering algorithm can help to improve the performance. \nThey studied the trend in 5 datasets and presented their method following intuition. Their approach first balances the label distributions assigned to unlabeled data. This is to avoid the imbalanced pseudo-labels to OOD data. Second, they applied the clustering objective to unlabeled samples which are supposed to be OOD. \nIn experiments, they have shown that their proposed approach can improve over supervised baseline even when there is a significant class mismatch between training classes and unlabeled data while baseline approaches are not very robust to the setting. \n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "As mentioned in the weakness, discussion and comparison to existing work are limited. They have to add discussion and empirical comparison to these work to clarify their novelty. ", "broader_impact_concerns": "They do not have broader impact section. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "b2jkyyzRmP", "reviewer": "Reviewer_vT6F", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths are the following:\n- Performs better than other class-mismatched SSL methods. Traditional SSL methods along with FixMatch and MixMatch are also compared in Figure 6 and in the Appendix. Ablation studies provide enables us to understand the behavior of the proposed method in depth.\n- Many datasets are used in the experiments: CIFAR-10, CIFAR-100, SVHN, Tiny ImageNet and ImageNet.\n- I like how the paper is organized: in the first half, the issues are studied with experiments, and in the latter half, the paper proposes some algorithms that can overcome those issues or take advantage of the findings.\n- The experimental results shown in Section 3 are simple but provides insights for future research with pseudo-labels and OOD.\n\nWeaknesses are the following:\n- The paper is focused on studying pseudo-labelling, but it would be interesting to explore how other SSL methods would behave. However, this is already discussed in Section 7 and I agree that it can be left for future work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "I believe the contribution of this paper is two-fold. The first one is an empirical contribution for studying the behavior of pseudo labels in a class-mismatch scenario. The experiments that shows imbalance of pseudo-labels and the experiments with four strategies of how to label OOD data provides insights that may be helpful for the research community. The second contribution is an algorithmic contribution, where the paper provides a new method that can solve the issues discussed and perform better than previous class-mismatch SSL and traditional/ordinary SSL methods, by rebalancing the pseudo-labels of high-confidence data and a clustering component.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- It is somehow surprising that oracle labelling performs better than open-set labelling, because the additional ground truth OOD labels are not directly necessary for the end task (where we don't need to distinguish between OOD classes). Intuitively we would expect similar results, but it seems that oracle labelling is much better. This seems like a nice discovery/contribution of the paper, so perhaps the paper can emphasize this point a bit more. (If similar results are already discussed in the OOD literature, additional discussions in the paper under review would be helpful.)\n- I wasn't sure if there was a comparison between providing ground truth OOD labels and clustering in the paper. It would be helpful to have the oracle baseline in Table 2. (I don't expect the proposed method to be better than oracle, but it would be helpful to see how close it is.)\n- In Section 4.1: \"As illustrated in Section 3.2\" => Should this be section 3.1?\n- Why is it hard to do mid/large scale experiments with DS3L?\n- Consistency in the reference section: \"et al.\" is used for some entries, such as for Lee (ICML WS 2013) while others have all author names.\n- $\\Upsilon$ is missing in Table 2, 2nd line from the end.", "broader_impact_concerns": "I did not find any issues with ethics. In my opinion, I do not see immediate significant negative societal impact in the future for a new algorithm with class-mismatch SSL.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors study a common and popular approach to semi-supervised learning, self-training/pseudo-labeling, in the scenario where the unlabeled data might be out-of-distribution. In particular, they work on class mismatch, where the latent classes of the unlabeled data might be quite different from the distribution set.

They perform both an empirical study to deal with imbalance on OOD data and introduce a number of strategies for rebalancing, achieving good empirical performance. 

The paper makes clear and convincing contributions and meets the bar for. During the review phase, the reviewers made a number of suggestions for improved clarity, which the authors have taken. The reviewers suggest accepting, and I agree.
Decision: Accept (poster)