Title: SELECTIVE SELF-TRAINING FOR SEMI-SUPERVISED LEARNING

Abstract: Semi-supervised learning (SSL) is a study that efficiently exploits a large amount of unlabeled data to improve performance in conditions of limited labeled data. Most of the conventional SSL methods assume that the classes of unlabeled data are included in the set of classes of labeled data. In addition, these methods do not sort out useless unlabeled samples and use all the unlabeled data for learning, which is not suitable for realistic situations. In this paper, we propose an SSL method called selective self-training (SST), which selectively decides whether to include each unlabeled sample in the training process. It is also designed to be applied to a more real situation where classes of unlabeled data are different from the ones of the labeled data. For the conventional SSL problems which deal with data where both the labeled and unlabeled samples share the same class categories, the proposed method not only performs comparable to other conventional SSL algorithms but also can be combined with other SSL algorithms. While the conventional methods cannot be applied to the new SSL problems where the separated data do not share the classes, our method does not show any performance degradation even if the classes of unlabeled data are different from those of the labeled data.

1. INTRODUCTION:

Recently, machine learning has achieved a lot of success in various fields and well-refined datasets are considered to be one of the most important factors (Everingham et al., 2010; Krizhevsky et al., 2012; Russakovsky et al., 2015) . Since we cannot discover the underlying real distribution of data, we need a lot of samples to estimate it correctly (Nasrabadi, 2007) . However, creating a large amount of dataset requires a huge amount of time, cost and manpower (Odena et al., 2018) .

Semi-supervised learning (SSL) is a method relieving the inefficiencies in data collection and annotation process, which lies between the supervised learning and unsupervised learning in that both labeled and unlabeled data are used in the learning process (Chapelle et al., 2009; Odena et al., 2018) . It can efficiently learn a model from fewer labeled data using a large amount of unlabeled data (Zhu, 2006) . Accordingly, the significance of SSL has been studied extensively in the previous literatures (Zhu et al., 2003; Rosenberg et al., 2005; Kingma et al., 2014; Rasmus et al., 2015; Odena, 2016) . These results suggest that SSL can be a useful approach in cases where the amount of annotated data is insufficient.

However, there is a recent research discussing the limitations of conventional SSL methods (Odena et al., 2018) . They have pointed out that conventional SSL algorithms are difficult to be applied to real applications. Especially, the conventional methods assume that all the unlabeled data belong to one of the classes of the training labeled data. Training with unlabeled samples whose class distribution is significantly different from that of the labeled data may degrade the performance of traditional SSL methods. Furthermore, whenever a new set of data is available, they should be trained from the scratch using all the data including out-of-class1 data.

In this paper, we focus on the classification task and propose a deep neural network based approach named as selective self-training (SST) to solve the limitation mentioned above. Unlike the conventional self-training methods in (Chapelle et al., 2009) , our algorithm selectively utilizes the unlabeled data for the training. To enable learning to select unlabeled data, we propose a selection network, which is based on the deep neural network, that decides whether each sample is to be added or not. Different from (Wang et al., 2018) , SST does not use the classification results for the data selection. Also, we adopt an ensemble approach which is similar to the co-training method (Blum & Mitchell, 1998) that utilizes outputs of multiple classifiers to iteratively build a new training dataset. In our case, instead of using multiple classifiers, we apply a temporal ensemble method to the selection network. For each unlabeled instance, two consecutive outputs of the selection network are compared to keep our training data clean. In addition, we have found that the balance between the number of samples per class is quite important for the performance of our network. We suggest a simple heuristics to balance the number of selected samples among the classes. By the proposed selection method, reliable samples can be added to the training set and uncertain samples including out-of-class data can be excluded.

SST is a self-training framework, which iteratively adopts the newly annotated training data (details in Section 2.1). SST is also suitable for the incremental learning which is frequently used in many real applications when we need to handle gradually incoming data. In addition, the proposed SST is suitable for lifelong learning which makes use of more knowledge from previously acquired knowledge (Thrun & Mitchell, 1995; Carlson et al., 2010; Chen & Liu, 2018) . Since SSL can be learned with labeled and unlabeled data, any algorithm for SSL may seem appropriate for lifelong learning. However, conventional SSL algorithms are inefficient when out-of-class samples are included in the additional data. SST only add samples having high relevance in-class data and is suitable for lifelong learning. The main contributions of the proposed method can be summarized as follows:

• For the conventional SSL problems, the proposed SST method not only performs comparable to other conventional SSL algorithms but also can be combined with other algorithms. • For the new SSL problems, the proposed SST does not show any performance degradation even with the out-of-class data. • SST requires few hyper-parameters and can be easily implemented.

• SST is more suitable for lifelong learning compared to other SSL algorithms.

To prove the effectiveness of our proposed method, first, we conduct experiments comparing the classification errors of SST and several other state-of-the-art SSL methods (Laine & Aila, 2016; Tarvainen & Valpola, 2017; Luo et al., 2017; Miyato et al., 2017) in conventional SSL settings. Second, we propose a new experimental setup to investigate whether our method is more applicable to realworld situations. The experimental setup in (Odena et al., 2018) samples classes among in-classes and out-classes. In the experimental setting in this paper, we sample unlabeled instances evenly in all classes. (details in Section 6.6 of the supplementary material). We evaluate the performance of the proposed SST using three public benchmark datasets: CIFAR-10, CIFAR-100 (Krizhevsky & Hinton, 2009) , and SVHN (Netzer et al., 2011) .

2. BACKGROUND:

In this section, we introduce the background of our research. First, we introduce some methods of self-training (McLachlan, 1975; Zhu, 2007; Zhu & Goldberg, 2009) on which our work is based. Then we describe consistency regularization-based algorithms such as temporal ensembling (Laine & Aila, 2016) .

2.1. SELF-TRAINING:

Self-training method has long been used for semi-supervised learning (McLachlan, 1975; Rosenberg et al., 2005; Zhu, 2007; Zhu & Goldberg, 2009) . It is a resampling technique that repeatedly labels unlabeled training samples based on the confidence scores and retrains itself with the selected pseudo-annotated data. Our proposed method can also be categorized as a self-training method. Figure 1 shows an overview of our SSL system. Since our proposed algorithm is based on the selftraining, we follow its learning process. This process can be formalized as follows. (i) Training a However, most self-training methods assume that the labeled and unlabeled data are generated from the identical distribution. Therefore, in real-world scenarios, some instances with low likelihood according to the distribution of the labeled data are likely to be misclassified inevitably. Consequently, these erroneous samples significantly lead to worse results in the next training step. To alleviate this problem, we adopt the ensemble and balancing methods to select reliable samples.

2.2. CONSISTENCY REGULARIZATION:

Consistency regularization is one of the popular SSL methods and has been referred to many recent researches (Laine & Aila, 2016; Miyato et al., 2017; Tarvainen & Valpola, 2017) . Among them, Π model and temporal ensembling are widely used (Laine & Aila, 2016) . They have defined new loss functions for unlabeled data. The Π model outputs f (x) and f (x) for the same input x by perturbing the input with different random noise and using dropout (Srivastava et al., 2014) , and then minimizes the difference ( f (x) -f (x) 2 ) between these output values. Temporal ensembling does not make different predictions f (x) and f (x), but minimizes the difference ( f t-1 (x) -f t (x) 2 ) between the outputs of two consecutive iterations for computational efficiency. In spite of the improvement in performance, they require lots of things to consider for training. These methods have various hyperparameters such as 'ramp up', 'ramp down', 'unsupervised loss weight' and so on. In addition, customized settings for training such as ZCA preprocessing and mean-only batch normalization (Salimans & Kingma, 2016) are also very important aspects for improving the performance (Odena et al., 2018) .

Algorithm 1 Training procedure of the proposed SST Require: x i , y i : training data and label Require: L, U: labeled and unlabeled datasets Require: I U : set of unlabeled sample indices Require:

f n (•; θ n ), f cl (•; θ c ) and f sel (•; θ s ): trainable SST model Require: α, , K, K re : hyper-parameters, 0 ≤ α < 1, 0 ≤ < 1 1: randomly initialize θ n , θ c , θ s 2: train f n (•; θ n ), f cl (•; θ c ) and f sel (•; θ s ) for K epochs using L 3: repeat 4: initialize r t i = -1, I S = ∅ 5:

for each i ∈ I U do 6:

r t-1 i ← r t i , r t i ← f cl (f n (x i ; θ n ); θ c ), s i ← f sel (f n (x i ; θ n ); θ s ) 7: if r t-1 i = r t i then 8: z i ← 0 9: end if 10: z i ← αz i + (1 -α)s i 11: if z i > 1 -then 12: I S ← I S ∪ {i} 13:

assign label for x i using r i 14:

end if 15:

end for 16:

update U S with data balancing 17:

T ← L ∪ U S 18: retrain f n (•; θ n ), f cl (•; θ c ) and f sel (•; θ s ) for K re epochs using T 19: until stopping criterion is true

3. METHOD:

In this section, we introduce our selective self-training (SST) method. The proposed model consists of three networks as shown in the bottom part of Figure 1 . The output of the backbone network is fed into two sibling fully-connected layers -a classification network f cl (•; θ c ) and a selection network f sel (•; θ s ), where θ c and θ s are learnable parameters for each of them. In this paper, we define the classification result and the selection score as

r i = f cl (f n (x i ; θ n ); θ c ) and s i = f sel (f n (x i ; θ n ); θ s ),

respectively, where f n (•; θ n ) denotes the backbone network with learnable parameters θ n . Note that we define r i as the resultant label and it belongs to one of the class labels

r i ∈ Y = {1, 2, • • • , C}.

The network architecture of the proposed model is detailed in Section 6.2 in the supplementary material. As shown in Figure 1 , the proposed SST method can be represented in the following four steps. First, SST trains the network using a set of the labeled data

L = {(x i , y i ) | i = 1, • • • , L}, where x i and y i ∈ {1, 2, • • • , C}

denote the data and the ground truth label respectively, which is a standard supervised learning method. The next step is to predict all the unlabeled data U = 

{x i | i = L + 1, • • • , N }

I U = {L + 1, • • • , N }.

Then, we annotate the selected samples with the pseudo-categories evaluated by the classification network and construct a new training dataset T composed of L and U S = {(x i , ŷi )|i ∈ I S }. After that, we retrain the model with T and repeat this process iteratively. The overall process of the SST is described in Algorithm 1 and the details of each of the four steps will be described later.

3.1. SUPERVISED LEARNING:

The SST algorithm first trains a model with supervised learning. At this time, the entire model (all three networks) is trained simultaneously. The classification network is trained using the softmax function and the cross-entropy loss as in the ordinary supervised classification learning task. In case of the selection network, the training labels are motivated by discriminator of generative adversarial networks (GAN) (Goodfellow et al., 2014; Yoo et al., 2017) . When i-th sample x i with the class label y i is fed into the network, the target for the selection network is set as:

EQUATION

where

I L = {1, • • • , L} represents a set of labeled sample indices.

The selection network is trained with the generated target g i . Especially, we use the sigmoid function for the final activation and the binary cross-entropy loss to train the selection network. Our selection network does not utilize the softmax function because it produces a relative value and it can induce a high value even for an out-of-class sample. Instead, our selection network is designed to estimate an absolute confidence score using the sigmoid activation function. Consequently, our final loss function is a sum of the classification loss L cl and the selection loss L sel :

L total = L cl + L sel .

(2)

3.2. PREDICTION AND SELECTION:

After learning the model in a supervised manner, SST takes all instances of the unlabeled set U as input and predicts classification result r i and the selection score s i , for all i ∈ I U . We utilize the classification result and selection score (r i and s i ) to annotate and choose unlabeled samples, respectively. In the context of self-training, removing erroneously annotated samples is one of the most important things for the new training dataset. Thus, we adopt temporal co-training and ensemble methods for selection score in order to keep our training set from contamination. First, let r t i and r t-1 i be the classification results of the current and the previous iterations respectively and we utilize the temporal consistency of these values. If these values are different, we set the ensemble score z i = 0 to reduce uncertainty in selecting unlabeled samples. Second, inspired by (Laine & Aila, 2016) , we also utilize multiple previous network evaluations of unlabeled instances by updating the ensemble score z i = αz i + (1 -α)s i , where α is a momentum weight for the moving average of ensemble scores. However, the aim of our ensembling approach is different from (Laine & Aila, 2016) . They want to alleviate different predictions for the same input, which are resulted from different augmentation and noise to the input. However, our aim differs from theirs in that we are interested in selecting reliable (pseudo-)labeled samples. After that, we select unlabeled samples with high ensemble score z i . It is very important to set an appropriate threshold because it decides the quality of the added unlabeled samples for the next training. If the classification network is trained well on the labeled data, the training accuracy would be very high. Since the selection network is trained with the target g i generated from the classification score r i , the selection score s i will be close to 1.0. We set the threshold to 1and control it by changing . In this case, if the ensemble score z i exceeds 1 -, the pseudo-label of the unlabeled sample ŷi is set to the classification result r i .

3.3. NEW TRAINING DATASET:

When we construct a new training dataset, we keep the number of samples of each class the same. The reason is that if one class dominates the others, the classification performance is degraded by the imbalanced distribution (FernáNdez et al., 2013) . We also empirically found that naively creating a new training dataset fails to yield good performance. In order to fairly transfer the selected samples to the new training set, the amount of migration in each class should not exceed the number of the class having the least selected samples. We take arbitrary samples in every class as much as the maximum number satisfying this condition. The new training set T is composed of both a set of labeled samples L and a set of selected unlabeled samples U S . The number of selected unlabeled samples is the same for all classes.

3.4. RE-TRAINING:

After combining the labeled and selected pseudo-labeled data, the model is retrained with the new dataset for K re epochs. In this step, the label for the selection network is obtained by a process similar to Eq. (1). Above steps (except for Section 3.1) are repeated for M iterations until (near-) convergence. 

4. EXPERIMENTS:

To evaluate our proposed SST algorithm, we conduct two types of experiments. First, we evaluate the proposed SST algorithm for the conventional SSL problem where all unlabeled data are in-class.

Then, SST is evaluated with the new SSL problem where some of the unlabeled data are out-of-class.

In the case of in-class data, gradually gathering highly confident samples in U can help improve the performance. On the other hand, in the case of out-of-class data, a strict threshold is preferred to prevent uncertain out-of-class data from being involved in the new training set. Therefore, we have experimented with decay mode that decreases the threshold in log-scale and fixed mode that fixes the threshold in the way described in Section 4.2. We have experimented our method with 100 iterations and determined epsilon by cross-validation in decay modes. In case of fixed modes, epsilon is fixed and the number of iteration is determined by cross-validation. The details about the experimental setup and the network architecture are presented in Section 6.1, 6.2 of the supplementary material.

4.1. CONVENTIONAL SSL PROBLEMS WITH IN-CLASS UNLABELED DATA:

We experiment with a couple of simple synthetic datasets (two moons, four spins) and three popular datasets which are SVHN, CIFAR-10, and CIFAR-100 (Netzer et al., 2011; Krizhevsky et al., 2014) .

The settings of labeled versus unlabeled data separation for each dataset are the same with (Laine & Aila, 2016; Miyato et al., 2017; Tarvainen & Valpola, 2017) . More details are provided in Section 6.3 in the supplementary material. The experimental results of the synthetic datasets can be found in Section 6.4 of the supplementary material.

4.1.1. ABLATION STUDY:

We have performed experiments on CIFAR-10 dataset with the combination of three types of components. As described in Table 1 , these are whether to use data balancing scheme described in Section 3.3 (balance), whether to use selection score ensemble in the 10th line of Algorithm 1 (ensemble) and whether to multiply the selection score with the top-1 softmax output of the classifier network to set a new selection score for comparison with the threshold (multiplication). First, when SST does not use all of these, the error 21.44% is higher than that of the supervised learning which does not use any unlabeled data. This is due to the problem of unbalanced data mentioned in subsection 3.3. When the data balance is used, the error is 14.43%, which is better than the baseline 21.44%. Adding the ensemble scheme results in 11.82% error, and the multiplication scheme shows a slight drop in performance. Since all of the experiments use the same threshold, the number of candidate samples to be added is reduced by the multiplication with the top-1 softmax output and the variation becomes smaller because only confident data are added. However, we have not used the multiplication scheme in what follows because the softmax classification output is dominant in multiplication. Therefore, we have used only balance and ensemble schemes in the following experiments.

4.1.2. EXPERIMENTAL RESULTS:

Table 2 shows the experiment results of supervised learning, conventional SSL algorithms and the proposed SST on CIFAR-10, SVHN and CIFAR-100 datasets. Our baseline model with supervised learning performs slightly better than what has been reported in other papers (Laine & Aila, 2016; Tarvainen & Valpola, 2017; Luo et al., 2017) because of our different settings such as Gaussian noise Table 2 : Classification error on CIFAR-10 (4k Labels), SVHN (1k Labels), and CIFAR-100 (10k Labels) with 5 runs using in-class unlabeled data (* denotes that the test has been done by ourselves).

Method CIFAR-10 SVHN CIFAR-100 Supervised (sampled)* 18.97 ± 0.37% 13.45 ± 0.92% 40.24 ± 0.45% Supervised (all)* 5.57 ± 0.07% 2.87 ± 0.06% 23.36 ± 0.27% Mean Teacher (Tarvainen & Valpola, 2017) 12.31 ± 0.28% 3.95 ± 0.21% -Π model (Laine & Aila, 2016) 12.36 ± 0.31% 4.82 ± 0.17% 39.19 ± 0.36% TempEns (Laine & Aila, 2016) 12.16 ± 0.24% 4.42 ± 0.16% 38.65 ± 0.51% TempEns + SNTG (Luo et al., 2017) 10.93 ± 0.14% 3.98 ± 0.21% 40.19 ± 0.51%* VAT (Miyato et al., 2017) 11.36 ± 0.34% 5.42 ± 0.22% -VAT + EntMin (Miyato et al., 2017) 10.55 ± 0.05% 3.86 ± 0.11% pseudo-label (Lee, 2013; Odena et al., 2018) on inputs, optimizer selection, the mean-only batch normalizations and the learning rate parameters. For all the datasets, we have also performed experiments with a model of SST combined with the temporal ensembling (TempEns) and SNTG, labeled as SST+TempEns+SNTG in the table. For the model, the pseudo-labels of SST at the last iteration is considered as the true class label. Figure 2 shows the number of samples used in the training and the corresponding accuracy on the test set for each dataset.

CIFAR-10: The baseline network yields the test error of 18.97% and 5.57% when trained with 4,000 (sampled) and 50,000 (all) labeled images respectively. The test error of our SST method reaches 11.82% which is comparable to other algorithms while SST+TempEns+SNTG model results 1.83% better than the SST-only model.

SVHN::

The baseline model for SVHN dataset is trained with 1,000 labeled images and yields the test error of 13.45%. Our proposed method has an error of 6.88% which is relatively higher than those of other SSL algorithms. Performing better than SST, SST+TempEns+SNTG reaches 4.74% of error which is worse than that of TempEns+SNTG model. We suspect two reasons for this. The first is that SVHN dataset is not well balanced, and the second is that SVHN is a relatively easy dataset, so it seems to be easily added to the hard labels. With data balancing, the SST is still worse than other algorithms. More details are provided in Section 6.5 in the supplementary material. We think this phenomenon owes to the use of hard labels in SST where incorrectly estimated samples deteriorate the performance.

CIFAR-100 : While the baseline model results in 40.24% of error rate through supervised learning with sampled data, our method performs with 34.89% of error, enhancing the performance by 5.3%.

We have observed that the performance of TempEns+SNTG is lower than TempEns, and when TempEns+SNTG is added to SST, performance is degraded slightly. Although TempEng+SNTG shows better performance than TempEng without augmentation in (Luo et al., 2017) , its performance is worse than that of TempEng with augmentation in our experiment. 2 . The reason for this can be conjectured that the hyper-parameter in the current temporal ensembling and SNTG may not have been optimized.

4.2. NEW SSL PROBLEMS WITH OUT-OF-CLASS UNLABELED DATA:

We have experimented with the following settings for real-world applications. The dataset is categorized into six animal and four non-animal classes as similarly done in (Odena et al., 2018) . In CIFAR-10, 400 images per animal class are used as the labeled data (total 2,400 images for 6 animal classes) and a pool of 20,000 images with different mixtures of both animal and non-animal classes are experimented as an unlabeled dataset. In CIFAR-100, 5,000 labeled data (100 images per animal class) and a total of 20,000 unlabeled images of both classes with different mixed ratios are utilized. Unlike the experimental setting in (Odena et al., 2018) , we have experimented according to the ratio (%) of the number of out-of-class data in the unlabeled dataset. More details are provided in Section 6.6 in the supplementary material.

As mentioned in Section 4, in the presence of out-of-class samples, a strict threshold is required. If all of the unlabeled data is assumed to be in-class, the decay mode may be a good choice. However, in many real-applications, out-of-class unlabeled data is also added to the training set in the decay mode and causes poor performance. In avoidance of such matter, we have experimented on a fixed mode of criterion threshold on adding the unlabeled data. Unlike the decay mode that decrements the threshold value, SST in the fixed mode sets a fixed threshold at a reasonably high value throughout the training. Our method in the fixed mode should be considered more suitable for real-applications but empirically shows lower performances in Figure 3 and Table 3 than when running in the decay mode. The difference between the decay mode and the fixed mode are an unchangeable and the initial ensemble.

Setting a threshold value for the fixed mode is critical for a feasible comparison against the decay mode. Figure 3 shows the average of the results obtained when performing SST five times for each ratio in CIFAR-10. As shown in Figure 3 (a), as the number of iteration increases, the threshold in the decay mode decreases and the number of additional unlabeled data increases. Obviously, while the different percentage of the non-animal data inclusion show different trends of training, in the cases of 0 ∼ 75% of non-animal data included in the unlabeled dataset, the additionally selected training data shows an initial increase at 30 th ∼ 40 th iteration. On the other hand, when the unlabeled dataset is composed of only the out-of-class data, selective data addition of our method initiates at 55 th ∼ 65 th training iteration. This tendency has been observed in previous researches on classification problems and we have set the threshold value fixed at a value between two initiating points of data addition as similarly done in the works of (Viola & Jones, 2001; Zhang & Viola, 2008) . We have set the fixed threshold based on 47th iteration (between 40 and 55). For a more reliable selection score, we have not added any unlabeled data to the new training set and have trained our method with the labeled data only for 5 iterations.

As it can be seen in Table 3 , in the case of SST in the decay mode, the performance has been improved when the unlabeled dataset consists only in-class animal data, but when the unlabeled pool is filled with only out-of-class data, the performance is degraded. For the case of SST with a fixed threshold value, samples are not added and the performance was not degraded at 100% nonanimal ratio as shown in Figure 3(c) . Furthermore, at 0% of out-of-class samples in the pool, there is a more improvement in the performance than at 100 % of out-of-class samples while still being inferior to the improvement than the decay mode. Because less but stable data samples are added by SST with a fixed threshold, the performance is improved for all the cases compared to that of supervised learning. Therefore, it is more suitable for real applications where the origin of data is usually unknown.

5. CONCLUSION:

We proposed selective self-training (SST) for semi-supervised learning (SSL) problem. Unlike conventional methods, SST selectively samples unlabeled data and trains the model with a subset of the dataset. Using selection network, reliable samples can be added to the new training dataset. In this paper, we conduct two types of experiments. First, we experiment with the assumption that unlabeled data are in-class like conventional SSL problems. Then, we experiment how SST performs for out-of-class unlabeled data.

For the conventional SSL problems, we achieved competitive results on several datasets and our method could be combined with conventional algorithms to improve performance. The accuracy of SST is either saturated or not depending on the dataset. Nonetheless, SST has shown performance improvements as a number of data increases. In addition, the results of the combined experiments of SST and other algorithms show the possibility of performance improvement.

For the new SSL problems, SST did not show any performance degradation even if the model is learned from in-class data and out-of-class unlabeled data. Decreasing the threshold of the selection network in new SSL problem, performance degrades. However, the output of the selection network shows different trends according to in-class and out-of-class. By setting a threshold that does not add out-of-class data, SST has prevented the addition of out-of-class samples to the new training dataset. It means that it is possible to prevent the erroneous data from being added to the unlabeled dataset in a real environment. As synthetic datasets, two moons and 4 spins were tested in the same manner as SNTG (Luo et al., 2017) . Each dataset has 6,000 training and 6,000 test samples. In the case of two moons, there are two classes y ∈ {0, 1}, and in case of 4 spins, y ∈ {0, 1, 2, 3}. In 6,000 training data, there are 12 labeled data and 5,988 unlabeled data. Thus, for two moons, each class has 6 points and for 4 spins, each class has 3 points. Because the number of labeled datapoints are too small, random sampling can lead to sample similar points. Therefore, we randomly sampled the labeled data with a constraint that the Euclidian distance of each data point is greater than 0.7. For these datasets, total iteration was performed 50 times, and the was increased from 10 -7 to 10 -4.5 on a log scale. 1975; Zhu, 2007; Zhu & Goldberg, 2009) without threshold does not improve performance even at 0%, and performance at 100% is degraded. When SST is applied to the softmax output as a threshold without selection network, the performance is improved at 0%, but the performance is degraded at 100%. Although the threshold was 0.9999, unlabeled data was added in 100% of the non-animal data.

6.8. NEW SSL PROBLEMS IN CIFAR-100:

In the new SSL problem, the experiment in decay mode is to find a gap between two initiating points of data addition in 0 ∼ 75% of non-animal data and 100% of non-animal data. In our experiment, the growth rate of epsilon in CIFAR-10 is applied to CIFAR-100. (The smaller the growth rate of epsilon, the less the difference in between iterations. Therefore, although the difference in the between intervals is the same, depending on the growth rate of , the difference in the iteration can be greater.) In the case of 0 ∼ 75%, the number of data shows a slight increase from about 30 iterations.

On the other hand, in the case of 100%, selected samples are added from about 40 iterations. The fixed threshold set to the threshold of 35 iterations. In the decay mode, the performance is much improved at 0%, and at 100%, the performance is degraded. On the other hand, in the fixed mode, there was no performance degradation from 0% to 100%. In CIFAR-100, the difference between 0% and 100 % is less than CIFAR-10, because the gap between animal and non-animal is small and additional data is small. Figure 7 shows the experimental results.

:

The term out-of-class is used to denote the situation where the new dataset contains samples originated from different classes than the classes of the old data. On the other hand, the term in-class is used when the new data contain only the samples belonging to the previously observed classes.

We have reproduced tempEns+SNTG model with a Pytorch implementation, and have verified of its performance on CIFAR-10 and SVHN akin to what is reported in(Luo et al., 2017). However, for CIFAR-100 dataset, since the experimental result when data augmentation is not used is not reported, we thus report our reproduced result.

Figures and Tables:

Figure 1: An overview of the proposed SST. Different shapes represent the input data with different underlying distribution, and different colors (orange, blue, and green) are for different classes. In the initial training dataset, only three classes with their corresponding distributions ( , , ) exist and are used for initial training. Then the unlabeled data which include unseen distribution ( , ) are inputted to the classification as well as the selection network. At the bottom right, unlabeled samples with higher selection network output values than a certain threshold are denoted by yellow and selected to be included in the training process for the next iteration, while the remaining are not used for training.

and select a subset of the unlabeled data {x i |i ∈ I S } whose data have high selection scores with the current trained model, where I S denotes a set of selected sample indices from

Figure 2: SST result on CIFAR-10, SVHN, and CIFAR-100 datasets with 5 runs. The x-axis is the iteration, the blue circle is the average of the number of data used for training, and the red diamond is the average accuracy.

Figure 3: Result of new SSL problems on CIFAR-10 dataset with 5 runs. (a) number of data with iteration in decay mode (b) accuracy with iteration in decay mode (c) number of data with iteration in fixed mode(d) accuracy with iteration in fixed mode. % means the ratio of the number of non-animal classes in the unlabeled data.

Figure 4: Synthetic datasets (Top : Two moons, Bottom : 4 spins) (a): training dataset (black point : unlabeled data point, color point : labeled data point), (b) test dataset result.

Figure 5: Synthetic datasets (Two moons) Top : training dataset, Bottom : test result

Figure 7: Result of new SSL problems on CIFAR-100 dataset with 5 runs. (a) number of data with iteration in decaying threshold (b) accuracy with iteration in decaying threshold (c) number of data with iteration in fixed threshold (d) accuracy with iteration in fixed threshold. % means the ratio of non-animal classes in the unlabeled data.

Ablation study with 5 runs on the CIFAR-10 dataset. 'balance' denotes the usage of data balancing scheme during data addition as described in Sec. 3.3, 'ensemble' is for the usage of previous selection scores as in the 10th line of Algorithm 1, and 'multiplication' is the scheme of multiplying top-1 softmax output of the classifier network to the selection score and use it as a new selection score.

Table content: <table><tr><td>method</td><td colspan="3">balance ensemble multiplication</td><td>error</td></tr><tr><td>supervised learning</td><td/><td/><td/><td>18.97 ± 0.37%</td></tr><tr><td/><td>x</td><td>x</td><td>x</td><td>21.44 ± 4.05%</td></tr><tr><td>SST</td><td>o o</td><td>x o</td><td>x x</td><td>14.43 ± 0.43% 11.82 ± 0.40%</td></tr><tr><td/><td>o</td><td>o</td><td>o</td><td>11.86 ± 0.15%</td></tr></table>



Table content: <table><tr><td>dataset</td><td colspan="2">CIFAR-10</td><td colspan="2">CIFAR-100</td></tr><tr><td>method</td><td>SST(decay)</td><td>SST(fixed)</td><td>SST(decay)</td><td>SST(fixed)</td></tr><tr><td>supervised</td><td colspan="2">22.27 ± 0.47%</td><td colspan="2">34.62 ± 1.14%</td></tr><tr><td>0%</td><td colspan="4">14.99 ± 0.54% 17.84 ± 0.39% 28.01 ± 0.44% 32.16 ± 0.64%</td></tr><tr><td>25%</td><td colspan="4">17.93 ± 0.33% 18.38 ± 5247% 29.94 ± 0.45% 32.28 ± 0.58%</td></tr><tr><td>50%</td><td colspan="3">20.91 ± 0.53% 19.04 ± 0.63% 31.78 ± 0.62%</td><td>32.6 ± 0.67%</td></tr><tr><td>75%</td><td colspan="4">22.72 ± 0.42% 20.07 ± 0.98% 34.44 ± 0.85% 32.32 ± 0.52%</td></tr><tr><td>100%</td><td colspan="4">26.78 ± 1.35% 20.24 ± 0.15% 37.17 ± 1.08% 32.62 ± 0.63%</td></tr></table>

Network structure for CIFAR-10, SVHN, and CIFAR-100 datasets. SVHN : 1k labeled data ( 100 images per class), 72,257 unlabeled data (it is not well balanced), and 26,032 test data. CIFAR-100 :10k labeled data (100 images per class ), 40k unlabeled data ( 400 images per class ), and 10k test data.

Table content: <table><tr><td>Type</td><td>Filter Shape / pad</td><td>Output Size</td></tr><tr><td>input</td><td>RGB channel images</td><td>32 × 32 × 3</td></tr><tr><td>conv2d</td><td>3 × 3 × 3 × 128 / same</td><td>32 × 32 × 128</td></tr><tr><td>conv2d</td><td colspan="2">3 × 3 × 128 × 128 / same 32 × 32 × 128</td></tr><tr><td>conv2d</td><td colspan="2">3 × 3 × 128 × 128 / same 32 × 32 × 128</td></tr><tr><td>pool</td><td>maxpool 2 × 2</td><td>16 × 16 × 128</td></tr><tr><td>dropout</td><td>p = 0.5</td><td>16 × 16 × 128</td></tr><tr><td>conv2d</td><td colspan="2">3 × 3 × 128 × 256 / same 16 × 16 × 256</td></tr><tr><td>conv2d</td><td colspan="2">3 × 3 × 256 × 256 / same 16 × 16 × 256</td></tr><tr><td>conv2d</td><td colspan="2">3 × 3 × 256 × 256 / same 16 × 16 × 256</td></tr><tr><td>pool</td><td>maxpool 2 × 2</td><td>8 × 8 × 256</td></tr><tr><td>dropout</td><td>p = 0.5</td><td>8 × 8 × 256</td></tr><tr><td>conv2d</td><td>3 × 3 × 256 × 512 / valid</td><td>6 × 6 × 512</td></tr><tr><td>conv2d</td><td>3 × 3 × 512 × 256 / same</td><td>6 × 6 × 256</td></tr><tr><td>conv2d</td><td>3 × 3 × 256 × 128 / same</td><td>6 × 6 × 128</td></tr><tr><td>pool</td><td>average pool 6 × 6</td><td>1 × 1 × 128</td></tr><tr><td>classification outputs</td><td>128 × classes (softmax)</td><td>1 × 1 × classes</td></tr><tr><td>selecting outputs</td><td>128 × 1 (sigmoid)</td><td>1 × 1 × 1</td></tr><tr><td>and 10k test data.</td><td/><td/></tr><tr><td>6.4 SYNTHETIC DATASETS</td><td/><td/></tr></table>

When the network were trained with 1k and 2k images, the test error were 38.71% and 26.99% respectively. The test errors in the SST algorithm were 23.15% and 15.72%, the SST has better performance than Π model but worse than Mean Teacher in 1k test. In 2k test, the SST has better performance than Π model and similar with Mean Teacher.

Table content: <table><tr><td>6.5 FURTHER EXPERIMENTS</td><td/></tr><tr><td colspan="2">CIFAR-10 : Table 6: Classification error on CIFAR-10 (1k and 2k Labels) with 5 runs using in-class unlabeled</td></tr><tr><td>data</td><td/></tr><tr><td>Method</td><td>CIFAR-10 (1k) CIFAR-10 (2k)</td></tr><tr><td>supervised (sampled)</td><td>38.71 ± 0.47% 26.99 ± 0.79%</td></tr><tr><td>Π model (Laine &amp; Aila, 2016)</td><td>27.36 ± 1.20% 18.02 ± 0.60%</td></tr><tr><td colspan="2">Mean Teacher (Tarvainen &amp; Valpola, 2017) 21.55 ± 1.48% 15, 73 ± 0.31%</td></tr><tr><td>Proposed method (SST)</td><td>23.15 ± 0.61% 15.72 ± 0.50%</td></tr></table>

Classification error on SVHN (balanced & all unlabeled data) with 5 runs using in-class unlabeled data.Odena et al., 2018) adds only four unlabeled classes and tests according to the radio of unlabeled class. For example, at 50%, two classes are in-class, and two classes are out-of-class. However, we experimented with the ratio of the number of non-animal data. Thus at 50% in CIFAR-10, unlabeled data consists of 50% in-class and 50% out-of-class. The data for each ratio are shown in Table8, and the data category for animal and non-animal is shown in Table9.

Table content: <table><tr><td>Method</td><td>SVHN (balanced)</td><td>SVHN (all)</td></tr><tr><td>supervised (sampled)</td><td colspan="2">13.45 ± 0.92%</td></tr><tr><td>Π model (Laine &amp; Aila, 2016)</td><td>5.09 ± 0.31%</td><td>4.82 ± 0.17%</td></tr><tr><td>TempEns (Laine &amp; Aila, 2016)</td><td>5.01 ± 0.15%</td><td>4.42 ± 0.16%</td></tr><tr><td>Proposed method (SST)</td><td>6.75 ± 0.28%</td><td>6.88 ± 0.59%</td></tr><tr><td>6.6 DATA SETTING FOR NEW SSL PROBLEM</td><td/><td/></tr><tr><td>(</td><td/><td/></tr></table>

Number of each class data for new SSL problems.

Table content: <table><tr><td>Dataset</td><td>Ratio</td><td>Labeled data Animal</td><td colspan="2">Unlabeled data Animal Non-Animal</td></tr><tr><td/><td>0%</td><td>400</td><td>3334 or 3333</td><td>0</td></tr><tr><td/><td>25%</td><td>400</td><td>2500</td><td>1250</td></tr><tr><td>CIFAR-10</td><td>50%</td><td>400</td><td>1667 or 1666</td><td>2500</td></tr><tr><td/><td>75%</td><td>400</td><td>834 or 833</td><td>3750</td></tr><tr><td/><td>100%</td><td>400</td><td>0</td><td>5000</td></tr><tr><td/><td>0%</td><td>100</td><td>400</td><td>0</td></tr><tr><td/><td>25%</td><td>100</td><td>300</td><td>100</td></tr><tr><td>CIFAR-100</td><td>50%</td><td>100</td><td>200</td><td>200</td></tr><tr><td/><td>75%</td><td>100</td><td>100</td><td>300</td></tr><tr><td/><td>100%</td><td>100</td><td>0</td><td>400</td></tr></table>

Data category for new SSL problems.

Table content: <table><tr><td>Type</td><td>Animal</td><td>Non-Animal</td></tr><tr><td>CIFAR-10 (CLASS)</td><td colspan="2">bird, cat, deer, dog, frog, horse airplane, automobile, ship, truck</td></tr><tr><td/><td>aquatic mammals, fish,</td><td>flowers, food containers,</td></tr><tr><td/><td>insects, large carnivores,</td><td>fruit and vegetables,</td></tr><tr><td/><td>large omnivores and herbivores,</td><td>household electrical devices,</td></tr><tr><td>CIFAR-100 (SUPERCLASS)</td><td>medium-sized mammals,</td><td>household furniture,</td></tr><tr><td/><td>non-insect invertebrates,</td><td>large man-made outdoor things,</td></tr><tr><td/><td>people, reptiles,</td><td>large natural outdoor scenes,</td></tr><tr><td/><td>small mammals</td><td>trees, vehicles 1, vehicles 2</td></tr><tr><td colspan="2">6.7 OTHER ALGORITHMS FOR NEW SSL PROBLEMS</td><td/></tr></table>

shows the results of a general test on other algorithms. First, self-training (McLachlan,

Table content: <table/>

Other algorithms for new SSL problems of increased classes with 5 runs. method self-training SST (softmax) SST (sigmoid) supervised 22.27 ± 0.47% 0% 21.97 ± 0.24% 18.27 ± 0.52% 17.84 ± 0.39% 25% 22.80 ± 0.39% 18.35 ± 0.86% 18.38 ± 0.52% 50% 23.93 ± 0.71% 18.72 ± 0.36% 19.04 ± 0.63% 75% 25.45 ± 0.47% 20.33 ± 0.82% 20.07 ± 0.98% 100% 27.31 ± 0.57% 20.71 ± 0.19% 20.24 ± 0.15%

Table content: <table><tr><td>(a)</td><td>(b)</td></tr><tr><td>(c)</td><td>(d)</td></tr></table>

Peer review (iclr2019):
 This paper describes the method for performing self-training where the unlabeled datapoints are iteratively added to the training set only if their predictions by the classifier are confident enough. The contributions of this paper are to add datapoints based on the prediction of the confidence level by a separate selection network and a number of heuristics applied for better selection. On the experimental side, the contribution is to test the scenario where datapoints from irrelevant classes are included in the unlabeled dataset. The paper is written in a way that makes following it a bit difficult, for example, the experimental setups. Also, the writing can be improved by making the writing more concise and formal (examples of informal: "spoil the network", "model is spoiled", "problem of increased classes", "many recent researches have been conducted", "lots of things to consider for training", "supervised learning was trained" etc.). The contributions of the method could also be underlined more clearly in the abstract and introduction. The description of consistency regularisation methods in section 2.2 is not very clear and I would like to get better understanding of temporal ensembling and SNTG methods here as they play an important role in the experiments. The idea of selective sampling for self-training is promising and the investigated questions are interesting. As far as I understand, the main contribution of this paper is the use of separate "selection network" to estimate the confidence of predictions by "classification network". However, as the "selection network" uses exactly the same input as "classification network", it is hard to imagine how it can learn additional information. For example, imagine the case of binary classification. If the selection network predicts 0 in come cases, it can be used to improve the result of "classification network" by flipping the corresponding label. How can you interpret such a thought experiment? One could understand the use of "selection network" as a way to automatically select a threshold of what to consider confident, however, in this case, the prediction of "selection network" should be thresholded at 0.5 (correct prediction or not), but the experiments use complex thresholds. Could you elaborate more on why the selection network is needed? How would it compare to a simple strategy of only including the datapoints whose top-1 prediction of "classification network" is greater than some threshold? Finally, could you show a plot of top-1 prediction of "classification network" vs score of "selection network" and elaborate on that? Then, in sections 3.2 and 3.3 the authors introduce a few additional tricks for self-training: exclude datapoints whose predictions are changing and balance the classes. Intuitively, these criteria are well motivated, but unfortunately, the combination of all the intuitions (including "selection network" with threshold) is not very principled. Ablation study shows that the use of the "selection network" strategy does not improve the results without these heuristics. It would be interesting to see how these heuristics would do without "selection network", for example, either by doing simple self-training with thresholding on the score of the classifier or by applying only these heuristics in combination with TempEns+SNTG. In the current form of evaluation, it is hard to say if there is any benefit of using the "selection network" that is the main novelty of the paper. It is very valuable that the experimental results include many recently proposed methods. Besides, the settings are described in details that could help for the reproducibility of the results. However, I have a few concerns about the results. First of all, the proposed SST algorithm alone only performs better than baselines in 1 case, equal to them in 1 case and worse in 1 (table 3). Besides, as the base classifier is different for various baselines, it is hard to compare the methods. Then, the important hyperparameter of the method---threshold---seems to be hard to select (both in sections 4.1 and 4.2). How did you chose the current values? How sensitive is it? Why various datasets need different settings? How the threshold value can be set in practice? Another important parameters is the number of iterations of the algorithm. How was it chosen? Concerning the experiments of section 4.2, how would the baseline methods of section 4.1 do in this case? Why did you select to study animal vs non-animals sets of classes? What would happen if you use random class splits or split animal classes (like in a more realistic scenario)? To conclude, while I find the studied problem quite interesting and intuitions behind the method very reasonable, the current methodology is not very principled and the experiment evaluation did not convince me that such an elaborate strategy is needed. Some questions and comments: - The setting of including unrelated classes in the unlabeled data resembles transfer learning setting. Could you explain why the ideas from transfer learning are not applicable in your case? - In the training procedure of "selection network" of Sections 3.1, do you use the same datapoints to train a "classification network" and "selection network"? If it is the case, how do you insure that the "classification network" does not learn to fit the data perfectly and thus all labels s_i are 1? - In the last sentences of the first paragraph on p.2 you make a contrast between using softmax and sigmoid functions, however, normally the difference between them is their use in binary or multiclass classification. Is there anything special that you want to show in you case? - What do you mean in section 3.3 by "if one class dominates the dataset, the model tends to overfit"? - I think parameters of training the networks from the beginning of section 4 could be moved to the supplementary materials. - Figure 3: wouldn*t the plot of accuracy vs amount of data be more suitable here? - Synthetic experiments of supplementary materials: the gains of the methods seem to be small. What are the numerical results? What would happen if you allow to select starting point at random (a more realistic case)? - Can you explain the sentence "To prevent data being added suddenly, no data was added until 5 iterations"? - How was it possible to improve the performance in experiment of section 4.2 with 100% of irrelevant classes?