Year: 2021

Abstract:
Semi-Supervised Learning (SSL) methods have shown superior performance when unlabeled data are drawn from the same distribution with labeled data. Among them, Pseudo-Labeling (PL) is a simple and widely used method that creates pseudo-labels for unlabeled data according to predictions of the training model itself. However, when there are unlabeled Out-Of-Distribution (OOD) data from other classes, these methods suffer from severe performance degradation and even get worse than merely training on labeled data. In this paper, we empirically analyze PL in class-mismatched SSL. We aim to answer the following questions: (1) How do OOD data influence PL? (2) What are the better pseudo-labels for OOD data? First, we show that the major problem of PL is imbalanced pseudolabels on OOD data. Second, we find that when labeled as their ground truths, OOD data are beneficial to classification performance on In-Distribution (ID) data. Based on the findings, we propose our model which consists of two components – Re-balanced Pseudo-Labeling (RPL) and Semantic Exploration Clustering (SEC). RPL re-balances pseudo-labels on ID classes to filter out OOD data while also addressing the imbalance problem. SEC uses balanced clustering on OOD data to create pseudo-labels on extra classes, simulating the process of training with their ground truths. Experiments show that our method achieves steady improvement over supervised baseline and state-of-the-art performance under all class mismatch ratios on different benchmarks.

text: 1 INTRODUCTION
labeled data
cat dog
unlabeled data
cat dog plane car
ID data OOD data
Figure 1: Realistic Semi-Supervised Learning may simultaneously contain unlabeled ID and OOD data. ID data come from the same classes as labeled data while OOD data come from classes that are not seen in labeled data.
Deep Semi-Supervised Learning (SSL) methods are proposed to reduce dependency on massive labeled data by utilizing a number of cheap, accessible unlabeled data. Pseudo-Labeling (Lee et al., 2013) is a simple but effective and widely used method that creates pseudo-labels according to predictions of the training model itself. Then SSL can be transformed to standard supervised learning. Other representative SSL methods are consistency regularization (Laine & Aila, 2017; Tarvainen & Valpola, 2017; Miyato et al., 2019), holistic methods (Berthelot et al., 2019; Sohn et al., 2020) and generative methods (Kingma et al., 2014). The recent development of SSL shows that these methods have achieved competitive performance to supervised learning methods.
However, all of these SSL methods achieve their good results based on an assumption that the unlabeled data are
drawn from the same distribution as the labeled data. This assumption can be easily violated in real-world applications. One of the common cases is that some unlabeled data come from unseen classes. For example, as is illustrated in Figure 1, in image classification, we can collect a lot of unlabeled images from the internet but usually they cover broader category concepts than labeled data. Oliver et al. (2018) have shown that on such class-mismatched conditions, performance of traditional SSL methods is damaged. To deal with this problem, several methods have been proposed. These methods include filtering out OOD data (Yu et al., 2020; Chen et al., 2020), down weighting
OOD data (Chen et al., 2020) and re-use OOD data by neural style transfer (Luo et al., 2021) or self-supervised learning (Huang et al., 2021). Although these methods achieve good results, why do OOD data damage performance and how will OOD data help remain unclear. Here, we focus on analyzing Pseudo-Labeling (PL) in class-mismatched SSL and give some answers to these two questions.
In this paper, we empirically analyze PL in class-mismatched SSL setting. These experiments aim to answer the following questions: (1) How do OOD data influence PL? (2) What are the better pseudo-labels for OOD data? For question (1), we investigate pseudo-labels created by PL. The main finding is that pseudo-labels on OOD data tend to be imbalanced while on ID data, it remains balanced. We further show that PL’s performance is damaged due to such imbalance on OOD data. For question (2), several strategies for labeling OOD data are investigated. We conclude that it is beneficial when labeling OOD data as a class different from ID data, and the performance can be further improved when the pseudo-labels partition unlabeled OOD data into their semantic clusters.
Based on the experimental analysis, we propose a two-branched model called Υ-Model, which processes unlabeled data according to their confidence score on ID classes. The first branch performs re-balanced pseudo-labeling on high-confidence data. It utilizes the property of imbalanced pseudolabels on OOD data, truncating the number of pseudo-labeled data for each class to their minimum. This procedure filters out many OOD data and also prevents the negative effect of imbalanced pseudo-labels. For the other branch, semantic exploration clustering is performed on low-confidence data. They are considered as OOD data and their semantics will be mined by clustering into different partitions on extra classes. The clustering result provides better pseudo-labels for these OOD data than vanilla PL. Experiments on different SSL benchmarks show that our model can achieve steady improvement in comparison to supervised baseline. We summarize our contributions as follows:
• We analyze the Pseudo-Labeling model for ID and OOD data. The findings lead to two primary conclusions: (1) Imbalance of pseudo-labels on OOD data damages PL’s performance. (2) Best pseudo-labels for unlabeled OOD data are those different from ID classes and partitioning them into their semantic clusters.
• We propose our two-branched Υ-Model. One branch re-balances pseudo-labels on ID classes and filter out OOD data. The other branch explores semantics of OOD data by clustering on extra classes.
• Experiments on different SSL benchmarks empirically validate effectiveness of our model.

heading: 2 PRELIMINARY

heading: 2.1 CLASS-MISMATCHED SSL
text: Similar to the SSL problem, the training dataset of the class-mismatched SSL problem contrains n ID labeled samples Dl = {(xli, yli)}ni=1 and m unlabeled samples Du = {xui}mi=1, (usually, m ≫ n,) yli ∈ YID = {1, . . . ,KID}, while different from SSL, the underlying ground truth yu of unlabeled data may be different from labeled data. i.e., yuj ∈ YID ∪ YOOD,YOOD = {KID + 1, . . . ,KID + KOOD}. The goal of class-mismatched SSL is to correctly classify ID samples into YID using labeled set with ID samples and unlabeled set possibly with OOD samples.

heading: 2.2 PSEUDO-LABELING
text: Pseudo-Labeling (PL) leverages the idea that we can use the model itself to obtain artificial labels for unlabeled data (Lee et al., 2013). PL first perform supervised learning on labeled data to get a pre-trained model f , which outputs the probability of belonging to each ID class. It then creates the pseudo-labels for each unlabeled sample:
y′ = { argmaxy∈YID f(y|x) , c(x) > τ reject , otherwise , (1)
c(x) = max y∈YID
f(y|x), (2)
where c(x) is the confidence score for x. All the pseudo-labeled unlabel data will be treated as labeled data for the next supervised learning generation. PL iteratively performs supervised learning and pseudo-label creation until stop condition.

heading: 3 ANALYSIS OF PSEUDO-LABELING IN CLASS-MISMATCHED SSL
text: In class-mismatched SSL, vanilla PL can only create pseudo-labels on ID classes even for OOD data. We will analyze how these OOD data influence vanilla PL and what are the better pseudo-labels for them in this section.

heading: 3.1 SETUP
text: We use CIFAR-10 (Krizhevsky et al., 2009) as our experimental dataset. The data set contains 10 categories – 6 animal classes and 4 vehicle classes. Following Guo et al. (2020), we perform a classification task on animal classes (denoted as class 0-5) and select 400 images per class to construct the labeled data set, i.e., 2,400 labeled examples. The other 4 vehicle classes are taken as OOD classes (denoted as classes 6-9). 20,000 images are randomly selected from all the 10 classes as the unlabeled data set. We vary the ratio of unlabeled images to modulate class distribution mismatch. For example, the extent is 50% means half of the unlabeled data comes from animal classes and the others come from vehicle classes. We use Wide-ResNet-28-2 (Zagoruyko & Komodakis, 2016) as our backbone. We also adopt data augmentation techniques including random resized crop, random color distortion and random horizontal flip. We train our network for 400 epochs. For each epoch, we iterate over the unlabeled set and random sample labeled data, each unlabeled and labeled minibatch contains 128 samples. We adopt Adam as the optimization algorithm with the initial learning rate 3× 10−3. We report the averaged accuracy of the last 20 epochs, pretending there is no reliable (too small) validation set to perform early stop (Oliver et al., 2018).

heading: 3.2 IMBALANCE OF PSEUDO-LABELS ON OOD DATA
text: In this section, we analyze the pre-trained model that creates the first set of pseudo-labels, and the final model trained by Pseudo-Labeling.
Pretrained model. First, we draw the distribution of confidence score on OOD data and ID data. Figure 2(a) tells that, like what is concluded in OOD detection (Hendrycks & Gimpel, 2017), proportion of high-confidence data in ID data is larger than OOD data. However, in class-mismatched SSL, the unlabeled data are in much larger quantities. When the class mismatch ratio is large, there are quite a few OOD data with high confidence scores. We will show in the final model experiments that these high-confidence OOD data damages performance. Secondly, we study pseudo-labels on both ID data and OOD data. Figure 2(b) shows that pseudo-labels ID data is balanced. However, they are rather imbalanced on OOD data (Figure 2(c)). This is attributed to the different distribution they are drawn from. Samples with certain pattern bias to certain classes. ID data bias to ID classes uniformly because they are sampled by the same distribution. However, with little probability, OOD data will also bias to ID classes uniformly since they have little relevance to ID data.
Final Pseudo-Labeling model. As an old saying goes, a good beginning is half done. However, such imbalance of the first set of pseudo-labels starts PL model badly when there is a large portion of OOD data, putting the model in danger of imbalanced learning. We run vanilla PL and show that the imbalance of pseudo-labels harms the performance. Figure 3(a) shows the performance of PL model with different OOD ratios. In accord with Oliver et al. (2018), PL model degrades as the portion of OOD data gets larger. Figure 3(b) displays the confusion matrix of the PL model on the whole test set containing both ID and OOD data. Since only 6 classes are known to us, the confusion matrix is a rectangle. We can see almost all the OOD samples (class 6-9) are classified as class 0, which means the imbalance effect on OOD data gets even worse as the PL training goes on. The possible reason is that, unlike Pseudo-labeling on ID data, supervision of labeled data can not help correct pseudo-labels on OOD data. Thus the imbalance continuously deteriorates. The imbalance on OOD data also influences classification performance on ID data. Samples of major classes (class 0) overwhelm the loss and gradient, leading to a degenerate model (Lin et al., 2017). We can see the PL model mistakenly classifies many of data with class 1-5 into class 0.

heading: 3.3 PSEUDO-LABELING STRATEGY FOR OOD DATA
text: The previous section shows OOD data hurt performance of vanilla PL. Then here comes the question: Assuming that we already know which data are OOD, how do we use these OOD data? Is omitting them the best way? If not, what are the better pseudo-labels for them? To answer these questions, we investigate four strategies to create pseudo-labels for OOD data:
• Baseline. This baseline omits all the OOD data and only trains on the labeled ID data. • Re-Assigned Labeling. This strategy assigns data of each OOD class to an ID class. It ensures
that different OOD class is assigned to different ID class, keeping the semantics unchanged between OOD classes. For example, (ship, trunk, airline, automobile) can be assigned to (bird, cat, deer, dog). This strategy can be seen as training a classifier of “super-classes”.
• Open-Set Labeling. This strategy is named after the related setting – Open-Set Recognition (Scheirer et al., 2013; Bendale & Boult, 2016). This strategy treats all OOD data as one unified class KID + 1. Thus this model outputs probability over KID + 1 classes.
• Oracle Labeling. This strategy uses the ground truth of OOD data. Thus this model outputs probability over KID +KOOD classes.
Note that Open-Set Labeling and Oracle Labeling can classify samples into more than KID classes. However, during evaluation, we only classify samples into KID ID classes. For these model, the predicted label ŷ of a test sample x is calculated as:
ŷ(x) = argmax y∈YID
f(y|x) (3)
The overall comparison of the four strategies is illustrated in Figure 4. We also report test accuracy when class mismatch ratio is 100%. From Figure 4, we can get several important conclusions. (1)
Re-Assigned Labeling underperforms baseline a little1. This indicates that assign samples with OOD classes to ID classes does not help the model distinguish between ID classes even if we somehow know which OOD data are semantically different. It also reveals that performing vanilla PL on OOD data may never help even if we do it perfectly. (2) Open-Set Labeling outperforms baseline, which indicates it improves the performance if we label the OOD data as a class other than ID classes. (3) We can see Oracle Labeling improves the performance and achieves the best result among the four strategies. It means that in addition to label OOD data as extra classes, if we can further assign OOD data with different semantics to different classes, the model will achieve better results.

heading: 3.4 SUMMARY OF SECTION
text: In this section, we study the behavior of Pseudo-Labeling model in class-mismatched SSL. We summarize several important conclusions here:
Conclusion 1: Classification model trained with labeled ID data creates imbalanced pseudo-labels on OOD data while on ID data, it remains balanced.
Conclusion 2: The vanilla PL process makes the imbalance of pseudo-labels deteriorate, damaging the classification performance on ID data.
Conclusion 3: Labeling OOD data as ID classes does not help and may even perform a little worse.
Conclusion 4: It is beneficial to label OOD data as extra classes different from ID classes. If we can further label semantically different OOD data as different classes, the performance can be further improved.

heading: 4 METHOD
text: Based on the findings in Section 3, we proposed Υ-Model (named after its shape) for classmismatched SSL. Υ-Model trains a classifier f that will output the posterior distribution over KID + K classes, i.e., f(y|x) ∈ RKID+K ,1⊤f(y|x) = 1. K is the number of extra classes, which can be known in advance (i.e., K = KOOD) or be set as a hyper-parameter. Similar to vanilla PL, we define confidence with the same form as Equation 2. However, this confidence is a little different from its original definition in Hendrycks & Gimpel (2017), for we only calculate the maximum probability of the KID classes instead of all. Therefore, we rename it to In-Distribution confidence (ID confidence). For evaluation, we predict labels using Equation 3. Υ-Model aims to solve the following questions:
Problem 1: how to avoid imbalanced pseudo-labels in PL model? (Conclusion 1, 2)
Problem 2: how to avoid labeling OOD data as ID? (Conclusion 3)
Problem 3: how to create proper pseudo-labels for unlabeled OOD data? (Conclusion 4)
1Assign 4 OOD classes to 4 ID classes of 6 causes imbalance. But we test the accuracy on selected 4 classes and find they show a similar result.
Υ-Model consists of two main branches – Re-balanced Pseudo-Labeling (RPL) and Semantic Exploration Clustering (SEC). RPL acts on high-confidence data to solve Problem 1, 2. SEC acts on low-confidence data to solve Problem 3. We describe the two branches in the following sections. The overview of Υ-Model is illustrated in Figure 5.

heading: 4.1 RE-BALANCED PSEUDO-LABELING
text: As is illustrated in Section 3.3, the main problem of vanilla PL is that a large number of OOD data with high confidence scores have imbalanced pseudo-labels. One possible solution is re-weighting the unlabeled sample (Guo et al., 2020) or using other methods in the imbalance learning field. However, even if we solve the problem of imbalance learning, labeling OOD data as ID classes also may damage the performance (Conclusion 3). In this paper, we use a simple method – Re-balanced Pseudo Labeling – to simultaneously solve imbalance (Problem 1) and incorrect recognition (Problem 2). It produces a set P of pseudo-labeled samples in three steps:
N = min y∈YID
|{x ∈ Du | f(y | x) > τ}| , (4)
τy = Nth biggest({f(y | x) | x ∈ Du}), y = 1, 2, . . . ,KID, (5) P = ⋃
y∈YID
{(x, y) | f(y | x) ≥ τy,x ∈ Du}, (6)
where Nth biggest represents the N–th biggest value of the given set. RPL first calculates the minimum number of pseudo-labeled samples for each ID class by Equation 4. Then it truncates the number of pseudo-labels of each ID class to that number by Equation 5, 6. The process of RPL is illustrated in Figure 5(a). First, it enforces the pseudo labels on ID classes to be balanced, solving Problem 1. Second, as is shown in Section 3, the set of high-confidence data is a mixture of ID and ODD data. Due to Conclusion 1, the pseudo-label distribution of such a set is a sum of imbalanced and balanced ones, thus still imbalanced. However, by selecting only top-N confident samples for each ID class, we will keep ID data and omit many OOD data since confidence on ID data tends to be higher than OOD data (Hendrycks & Gimpel, 2017). This process solves Problem 2.

heading: 4.2 SEMANTIC EXPLORATION CLUSTERING
text: As is demonstrated in Section 3.3, if we know a set of samples is OOD, it will improve the performance if we label them as a unified class KID + 1. But the best way is to use their ground truths (Conclusion 4). However, it is impossible to access their ground truths since they are unlabeled. We resort to using Deep Clustering methods (Caron et al., 2018; Asano et al., 2020) to mine their semantics and approximate the process of learning with the ground truths. Here, we use the balanced clustering method in Asano et al. (2020); Caron et al. (2020) to create pseudo-labels for these OOD data. Assuming there are M samples recognized as OOD, we first compute their soft targets:
min Q∈U(K,M)
⟨Q,− logP ⟩, U(K,M) := { Q ∈ RK×M+ | Q1 = 1
K 1, Q⊤1 =
1
M 1
} , (7)
where P ∈ RK×M+ , Pij = f̂(KID + i|xj). f̂ is the normalized posterior distribution on extra classes, i.e., f̂(KID+ i|xj) = f(KID+ i|xj)/ ∑K k=1 f(KID+k|xj). We use the Sinkhorn-Knopp algorithm Cuturi (2013) to optimize Q. Once we get Q, we harden the label by picking the class with the maximum predicted probability and mapping it to the extra K classes:
ŷj = KID + argmax i Qij . (8)
ŷj is used as the pseudo-label for xj . We perform SEC on the set of data with ID confidence lower than a threshold γ, i.e., {x|c(x) < γ}.

heading: 5 RELATED WORK
text: Class-Mismatched Semi-Supervised Learning. Deep Semi-Supervised Learning suffers from performance degradation when there are unseen classes in unlabeled data (Oliver et al., 2018). As the proportion of such out-of-distribution (OOD) data get larger, the performance drop more. To cope with such a class-mismatch problem, several methods are proposed. Chen et al. (2020) formulate a sequence of ensemble models aggregated accumulatively on-the-fly for joint self-distillation and OOD filtering. Guo et al. (2020) re-weight the unlabeled data by meta-learning to decrease the negative effect of OOD data. Huang et al. (2020) recycle transferable OOD data by means of adversarial learning. Different from all these methods, we conduct a comprehensive study on Pseudo-Labeling (PL) and give useful guidance on how to do better in class-mismatched SSL. Pseudo-Labeling. The method of Pseudo-Labeling, also known as self-training, is a simple and effective way for Deep Semi-Supervised Learning (Lee et al., 2013; Shi et al., 2018; Arazo et al., 2020; Iscen et al., 2019). Despite its simplicity, it has been widely applied to diverse fields such as image classification (Xie et al., 2020), natural language processing (He et al., 2020) and object detection (Rosenberg et al., 2005). The use of a hard label makes Pseudo-Labeling closely related to entropy minimization (Grandvalet & Bengio, 2004). Deep Clustering. Deep clustering methods improve ability of traditional cluster methods by leveraging the representation power of DNNs. A common means is to transform data into lowdimensional feature vectors and apply traditional clustering methods (Yang et al., 2017; Caron et al., 2018). In Self-Supervised Learning, clustering methods are used to learning meaningful representation for downstream tasks (Caron et al., 2018; Asano et al., 2020; Caron et al., 2020). Modern Deep Clustering can learn semantically meaningful clusters and achieves competitive results against supervised learning (Gansbeke et al., 2020).

heading: 6 EXPERIMENTS
text: To validate the effectiveness of our Υ-Model, we conduct experiments on different benchmarks. Dataset. We test our methods on two datasets as in Oliver et al. (2018): (1) CIFAR10: we use the same configuration as in Section 3.1. SVHN: The data set contains 10 categories – digits “0”-“9”. We select the first “0”-“5” as ID classes and the rest as OOD. For each class, we randomly select 100 images as labeled data. Meanwhile, 20,000 images are randomly selected from all the 10 classes as the unlabeled data set. The class-mismatch ratio is set as {0%, 25%, 50%, 75%, 100%}. Implementation Details. We use the same network and training protocol as Section 3.1. We first train a classification model only on labeled data for 100 epochs without RPL and SEC. We update pseudo-labels every 2 epochs. For both datasets, we set τ = 0.95, γ = 0.3,K = 4. We use an exponential moving average model for final evaluation as in Athiwaratkun et al. (2019).

heading: 6.1 COMPARE WITH TRADITIONAL SSL METHODS
text: In this subsection, we compare our methods with four traditional SSL methods – PseudoLabeling (Lee et al., 2013), Π-Model (Laine & Aila, 2017), Mean Teacher (Tarvainen & Valpola, 2017) and VAT (Miyato et al., 2019). Figures 6(a), 6(b) show the results. Traditional methods suffer from performance degradation as the mismatch ratio increases. They usually get worse than the supervised baseline when the mismatch ratio is larger than 50% on CIFAR10 and SVHN. In contrast, our methods get steady improvement under all class mismatch ratios. The reasons can be attributed as follows. First, our method is aware of the existence of OOD data. We do not treat OOD data
like ID data, which can hurt performance. Second, we reuse OOD data by exploring their semantics which proves to be useful in Section 3.3. Therefore, even when the class-mismatch ratio gets 100%, performance of Υ-Model is still better than supervised baseline.

heading: 6.2 COMPARE WITH CLASS-MISMATCHED SSL METHODS
text: In this subsection, we compare our method with two existing class-mismatched SSL methods – UASD (Chen et al., 2020) and DS3L (Guo et al., 2020). For a fair comparison, we use PseudoLabeling as the base method of DS3L. From Figures 6(c), 6(d), we can see our methods are superior to these two methods in all settings. It is noticeable that DS3L underperforms supervised baseline when all the unlabeled data are drawn from OOD classes. This is attributed to the fact that DS3L uses a down weighting strategy to alleviate the negative effect of OOD data and does not change the form of unsupervised loss. But we have shown in Section 3.3 that labeling OOD data as ID classes damage performance anyhow. On the contrary, Υ-Model uses the OOD data in the right way – simulating the process of training them with their ground truth labels. As a result, our method shows superiority especially under a large class-mismatch ratio. We also notice that the performance curve of Υ-Model appears a U-shape (obvious on CIFAR10). A possible reason is that RPL and SEC compete with each other. RPL tends to make samples get a high prediction on ID classes while SEC tends to make samples get a high prediction on OOD classes. When the class-mismatch ratio reaches 0% (100%), RPL (SEC) dominants the other. In this circumstance, one works without any disturbance of the other. However, when the class-mismatch ratio is 50%, they compete fiercely with each other, causing many incorrectly recognized ID or OOD samples.

heading: 6.3 ABLATION STUDY
text: In this section, we validate the functionality of RPL and SEC. We conduct experiments on CIFAR10 benchmark as in the analysis section 3. Validation of effectiveness of RPL and SEC. We conduct ablation studies under different classmismatch ratios and report the averaged test accuracy and standard deviation of five runs. As usual, we vary the class-mismatch ratio. Table 1 displays the results. Firstly, comparing the first line and second line of the table, RPL not only outperforms vanilla PL in high class-mismatch ratio scenario but also improves in low class-mismatch ratio scenario. This reveals that balanced pseudo-labels always help since once the model creates imbalanced pseudo-labels, it will deteriorate when there are not enough measures to correct it. Secondly, comparing the second and third line, it shows that RPL alone alleviate the performance degradation but it can not prevent it, in accord with Conclusion 3. When using SEC, Υ-Model gets better results than supervised baseline when the class-mismatch ratio is high. Besides, comparing the third and last lines, we see that when cluster OOD data and exploring their semantics instead of using a unified class to label them, the performance improves. RPL helps filter out OOD data and solve imbalance problem. It is noticeable that the last two lines of Table 1 show that without RPL, SEC alone can not achieve better performance than supervised baseline. We will show the reason here. Figure 7(a) plots the proportion of OOD data that are pseudo-labeled as ID classes. It reveals that without RPL, i.e., using vanilla PL, the number
of incorrectly recognized OOD data keep increasing as the training proceed. While with RPL, this ratio rapidly drops to 0. This proves that RPL help filter out OOD data by utilizing the imbalance property of OOD data. Further, we present the confusion matrix of Υ-Model on the full test set (all the 10 classes) of CIFAR10. Compared to vanilla PL in Figure 3(b), Υ-Model does not suffer from imbalance problem, as a result of which, its performance is not degraded.
Effect of extra class number K. We vary the number of extra classes K. Figure 7(c) shows the result on CIFAR10 with class mismatch ratio 100%. The gray dashed line is the supervised baseline. The red dashed line is the Oracle Labeling strategy in Section 3.3. It is the upper bound of Υ-Model in this setup. Without SEC (K = 0), Υ-Model underperform supervised baseline. Using SEC (K ≥ 1), Υ-Model is always better than baseline. Also, it reaches its best performance when K equals the actual number of OOD classes. This demonstrates that by simulating the process of training on OOD data with ground truth, SEC helps classification model on ID data.

heading: 7 CONCLUSION
text: In this paper, we analyze Pseudo-Labeling in class-mismatched semi-supervised learning where there are unlabeled OOD data from other classes. We show that Pseudo-Labeling suffers from performance degradation due to imbalanced pseudo-labels on OOD data. The correct way to use OOD data is to label them as classes different from ID classes while also partitioning them according to their semantics. Based on the analysis, we proposed Υ-Model and empirically validate its effectiveness. We believe our findings are not only beneficial to PL methods, but also inspiring to other methods like consistency regularization and holistic methods on how to effectively use OOD data.

heading: A ALGORITHM
text: Algorithm 1 Υ-Model algorithm Input: Labeled dataset Dl = {(xli, yli)}ni=1, and unlabeled dataset Du = {xui}mi=1; Classification
model fϕ parameterized with ϕ, ID class number KID, extra class number K, total epoch number E, pretrain epochs Ept, interval to update pseudo-labels Epl, pseudo-labeled set P , confidence calculation function c.
1: function REBALANCEDPSEUDOLABELING(D, f, τ ) 2: N ← miny∈YID |{x ∈ Du | f(y | x) > τ}| 3: τy ← Nth biggest({f(y | x) | x ∈ Du}), y = 1, 2, . . . ,KID 4: P ← ⋃ y∈YID{(x, y) | f(y | x) ≥ τy,x ∈ Du} 5: return P 6: function SEMANTICEXPLORATIONCLUSTERING(D, f, γ) 7: S ← {x|c(x) < γ} 8: M ← |S| 9: Pij ← f(KID + i|xj)/ ∑K k=1 f(KID + k|xj), i = 1, 2, . . . ,K, j = 1, 2, . . . ,M
10: Solve 7 by Sinkhorn-Knopp algorithm and get Q 11: ŷj ← KID + argmaxi Qij , j = 1, 2, . . . ,M 12: C ← {(xj , ŷj)}Mj=1 13: return C 14: for e = 1 to E do 15: if e < Ept then 16: train fϕ with standard supervised learning on Dl ▷ Pre-training phase 17: L = 18: else 19: train fϕ with standard supervised learning on Dl ∪ P ▷ PL training phase 20: if e ≤ Ept and e %Epl = 0 then 21: P ← ∅ 22: Pτ← REBALANCEDPSEUDOLABELING(Du, fϕ, τ ) ▷ Perform RPL 23: Pγ← SEMANTICEXPLORATIONCLUSTERING(Du, fϕ, γ) ▷ Perform SEC 24: P ← Pτ ∪ Pγ 25: return classification model fϕ

heading: B EMBEDDING VISUALIZATION
text: We visualize the embedding of supervised baseline, vanilla PL and υ-Model on CIFAR10’s test set with all the classes by t-SNE. Figure 8 shows the result. OOD data are mixed with ID data since the supervised baseline does not see unlabeled OOD data. PL mixes OOD data with samples of certain classes (class 0). This is attributed to that their pseudo-labels are biased toward this class. Also, we can not clearly distinguish between OOD classes. In contrast, Υ-Model can not only make ID data distinguishable but also forms meaningful clusters on OOD data.

heading: C UNIVERSALITY OF ANALYSIS CONCLUSIONS
text: To illustrate the universality of the conclusion in Section 3. We experiment on totally five kinds of datasets. ( We use (n/m) to represent n ID classes and m OOD classes.)
• CIFAR10 (6/4) and CIFAR10 (5/5) : both created from CIFAR10 (Krizhevsky et al., 2009).CIFAR10 (6/4) takes the 6 animal classes as ID classes and 4 vehicle classes as OOD classes. CIFAR10 (5/5) has 3 animal and 2 vehicle classes for both ID and OOD classes. We select 400 labeled samples for each ID classes and totally 20,000 unlabeled samples from ID and OOD classes.
• SVHN (6/4): We select the first “0”-“5” as ID classes and the rest as OOD. We select 100 labeled samples for each ID class and totally 20,000 unlabeled samples.
• CIFAR100 (50/50): created from CIFAR100 (Krizhevsky et al., 2009). The first 50 classes are taken as ID classes and the rest as OOD classes. We select 100 labeled samples for each ID classes and totally 20,000 unlabeled samples.
• Tiny ImageNet (100/100): created from Tiny ImageNet, which is a subset of ImageNet (Deng et al., 2009) with images downscaled to 64 × 64 from 200 classes. The first 100 classes are taken as ID classes and the rest as OOD classes. We select 100 labeled samples for each ID class and totally 40,000 unlabeled samples.
Here we use C to represent CIFAR, TIN to represent Tiny ImageNet for short. For each dataset, we use the same experimental setup as in Section 3.
C.1 IMBALANCE OF PSEUDO-LABELS
To demonstrate the imbalance of pseudo-labels on OOD data, we compute two metrics to measure the extent of imbalance.
• The KL divergence of pseudo-label distribution q and the uniform distribution u: kl = KL(q||u) = ∑ i qi log qi ui
• The ratio of the ‘majority class’ and ‘minority class’:
r = maxi qi mini qi
The results on these datasets are displayed in Table 2.
C.2 PSEUDO-LABELING STRATEGY FOR OOD DATA
We report the test accuracy of the four pseudo-labeling strategies in Section 3.3. The class-mismatch ratio is set to 100%. Table 3 shows the results.
Note that Re-Assigned Labeling has many possibilities. If there are n ID classes and m OOD classes, Amn possible assignments exist. It is impossible to experiment on all of them. To deal with it, we randomly choose 10 possible assignment and pick the maximum performance among them.

heading: D COMPARISON ON MORE DATASETS
text: We compare our method with vanilla PL and the two class-mismatched methods in Section 6.2. We use the following hyperparameters:
• CIFAR10 (6/4): τ = 0.95, γ = 0.3, Ept = 50, Epl = 2,K = 4 • SVHN (6/4): τ = 0.95, γ = 0.3, Ept = 50, Epl = 2,K = 4 • CIFAR100 (50/50):τ = 0.95, γ = 0.18, Ept = 50, Epl = 2,K = 20 • Tiny ImageNet (100/100): τ = 0.9, γ = 0.15, Ept = 50, Epl = 2,K = 20
For CIFAR100 (50/50) and Tiny ImageNet (100/100), we use a weight factor λ to trade off the loss on labeled set Dl and pseudo-labeled set P , which ramps up with function λ =
exp ( −5× ( 1−min ( iter 40,000 , 1 ))2) , where iter is the number of training steps from Ept.

Review summaries:
This paper works on the class-mismatch semi-supervised learning problem, where the assumption is that the unlabeled samples include class labels that do not appear in the labeled data, i.e., out-of-distribution (OOD) data, in addition to the class labels that appear in the labeled data, i.e., in-distribution (ID) data. It is known that if there are OOD data included in the unlabeled data, it can degrade performance of semi-supervised learning algorithms. This paper focuses on one of the semi-supervised learning methods, which is the pseudo-labeling method. The paper first investigates how using pseudo-labels in the class-mismatch semi-supervised learning setup can be problematic. The experiments show that proportions of high-confidence data in ID data is larger than OOD data. Experiments also show that pseudo-labels (w.r.t. class predictions) are more balanced in ID data while it is unbalanced in OOD data at the beginning of training (with a pretrained model). This phenomenon becomes exaggerated at the end of training, where almost all of the OOD data is predicted into a single ID class. Assuming we know the labels of the unlabeled data, the paper performs further experiments that compare a few methods, and finds out that it is ideal if we have access to the underlying single class labels of OOD data, and solve the problem as a multi-class classification problem with K_id + K_ood (number of classes of ID and OOD) classes. Based on these observations, the paper proposes an Upsilon model, that consists of a re-balanced pseudo-labelling (RPL) and semantic exploration clustering (SEC). RPL aims to balance the distributions of pseudo-labels based on the observation that pseudo-labels become balanced/imbalanced with ID/OOD data, respectively. SEC is a workaround for not having ground truth labels of OOD data.  Experiments show comparison with both traditional SSL methods and class-mismatch SSL methods, and show how the proposed Upsilon model works better than these baselines. Ablation study shows why the components within the Upsilon model is necessary.

This paper focuses on the problem of semi-supervised learning (SSL) with class miss-match. The authors first empirically analyze Pseudo-Labeling (PL) in class-mismatched SSL and proposed a new method that consists of two components – Re-balanced Pseudo-Labeling (RPL) and Semantic Exploration Clustering (SEC). Experiments show that the proposed method achieves steady improvement over supervised baseline and state-of-the-art performance under all class mismatch ratios on different benchmarks.  

This paper researched on the semi-supervised learning task when there are unlabeled out of distribution data from other classes. Several interesting issues of class mismatch SSL are studied, including the reasons for the performance degradation of PL in OOD data and how to better pseudo-label OOD data to provide a more balanced semantic distribution. To address above-mentioned problems, Y-model, consisting of two components – Re-balanced Pseudo-Labeling (RPL) and Semantic Exploration Clustering (SEC), are proposed. Authors conducted several experiments to show the effectiveness of the proposed method, such as CIFAR-10 and SVHN.