Selective Self-Training for semi-supervised Learning

Semi-supervised learning ( SSL ) is a study that efficiently exploits a large amount of unlabeled data to improve performance in conditions of limited labeled data . Most of the conventional SSL methods assume that the classes of unlabeled data are included in the set of classes of labeled data . In addition , these methods do not sort out useless unlabeled samples and use all the unlabeled data for learning , which is not suitable for realistic situations . In this paper , we propose an SSL method called selective self-training ( SST ) , which selectively decides whether to include each unlabeled sample in the training process . It is also designed to be applied to a more real situation where classes of unlabeled data are different from the ones of the labeled data . For the conventional SSL problems which deal with data where both the labeled and unlabeled samples share the same class categories , the proposed method not only performs comparable to other conventional SSL algorithms but also can be combined with other SSL algorithms . While the conventional methods can not be applied to the new SSL problems where the separated data do not share the classes , our method does not show any performance degradation even if the classes of unlabeled data are different from those of the labeled data . 1 INTRODUCTION . Recently , machine learning has achieved a lot of success in various fields and well-refined datasets are considered to be one of the most important factors ( Everingham et al. , 2010 ; Krizhevsky et al. , 2012 ; Russakovsky et al. , 2015 ) . Since we can not discover the underlying real distribution of data , we need a lot of samples to estimate it correctly ( Nasrabadi , 2007 ) . However , creating a large amount of dataset requires a huge amount of time , cost and manpower ( Odena et al. , 2018 ) . Semi-supervised learning ( SSL ) is a method relieving the inefficiencies in data collection and annotation process , which lies between the supervised learning and unsupervised learning in that both labeled and unlabeled data are used in the learning process ( Chapelle et al. , 2009 ; Odena et al. , 2018 ) . It can efficiently learn a model from fewer labeled data using a large amount of unlabeled data ( Zhu , 2006 ) . Accordingly , the significance of SSL has been studied extensively in the previous literatures ( Zhu et al. , 2003 ; Rosenberg et al. , 2005 ; Kingma et al. , 2014 ; Rasmus et al. , 2015 ; Odena , 2016 ) . These results suggest that SSL can be a useful approach in cases where the amount of annotated data is insufficient . However , there is a recent research discussing the limitations of conventional SSL methods ( Odena et al. , 2018 ) . They have pointed out that conventional SSL algorithms are difficult to be applied to real applications . Especially , the conventional methods assume that all the unlabeled data belong to one of the classes of the training labeled data . Training with unlabeled samples whose class distribution is significantly different from that of the labeled data may degrade the performance of traditional SSL methods . Furthermore , whenever a new set of data is available , they should be trained from the scratch using all the data including out-of-class1 data . 1The term out-of-class is used to denote the situation where the new dataset contains samples originated from different classes than the classes of the old data . On the other hand , the term in-class is used when the new data contain only the samples belonging to the previously observed classes . In this paper , we focus on the classification task and propose a deep neural network based approach named as selective self-training ( SST ) to solve the limitation mentioned above . Unlike the conventional self-training methods in ( Chapelle et al. , 2009 ) , our algorithm selectively utilizes the unlabeled data for the training . To enable learning to select unlabeled data , we propose a selection network , which is based on the deep neural network , that decides whether each sample is to be added or not . Different from ( Wang et al. , 2018 ) , SST does not use the classification results for the data selection . Also , we adopt an ensemble approach which is similar to the co-training method ( Blum & Mitchell , 1998 ) that utilizes outputs of multiple classifiers to iteratively build a new training dataset . In our case , instead of using multiple classifiers , we apply a temporal ensemble method to the selection network . For each unlabeled instance , two consecutive outputs of the selection network are compared to keep our training data clean . In addition , we have found that the balance between the number of samples per class is quite important for the performance of our network . We suggest a simple heuristics to balance the number of selected samples among the classes . By the proposed selection method , reliable samples can be added to the training set and uncertain samples including out-of-class data can be excluded . SST is a self-training framework , which iteratively adopts the newly annotated training data ( details in Section 2.1 ) . SST is also suitable for the incremental learning which is frequently used in many real applications when we need to handle gradually incoming data . In addition , the proposed SST is suitable for lifelong learning which makes use of more knowledge from previously acquired knowledge ( Thrun & Mitchell , 1995 ; Carlson et al. , 2010 ; Chen & Liu , 2018 ) . Since SSL can be learned with labeled and unlabeled data , any algorithm for SSL may seem appropriate for lifelong learning . However , conventional SSL algorithms are inefficient when out-of-class samples are included in the additional data . SST only add samples having high relevance in-class data and is suitable for lifelong learning . The main contributions of the proposed method can be summarized as follows : • For the conventional SSL problems , the proposed SST method not only performs comparable to other conventional SSL algorithms but also can be combined with other algorithms . • For the new SSL problems , the proposed SST does not show any performance degradation even with the out-of-class data . • SST requires few hyper-parameters and can be easily implemented . • SST is more suitable for lifelong learning compared to other SSL algorithms . To prove the effectiveness of our proposed method , first , we conduct experiments comparing the classification errors of SST and several other state-of-the-art SSL methods ( Laine & Aila , 2016 ; Tarvainen & Valpola , 2017 ; Luo et al. , 2017 ; Miyato et al. , 2017 ) in conventional SSL settings . Second , we propose a new experimental setup to investigate whether our method is more applicable to realworld situations . The experimental setup in ( Odena et al. , 2018 ) samples classes among in-classes and out-classes . In the experimental setting in this paper , we sample unlabeled instances evenly in all classes . ( details in Section 6.6 of the supplementary material ) . We evaluate the performance of the proposed SST using three public benchmark datasets : CIFAR-10 , CIFAR-100 ( Krizhevsky & Hinton , 2009 ) , and SVHN ( Netzer et al. , 2011 ) . 2 BACKGROUND . In this section , we introduce the background of our research . First , we introduce some methods of self-training ( McLachlan , 1975 ; Zhu , 2007 ; Zhu & Goldberg , 2009 ) on which our work is based . Then we describe consistency regularization-based algorithms such as temporal ensembling ( Laine & Aila , 2016 ) . 2.1 SELF-TRAINING . Self-training method has long been used for semi-supervised learning ( McLachlan , 1975 ; Rosenberg et al. , 2005 ; Zhu , 2007 ; Zhu & Goldberg , 2009 ) . It is a resampling technique that repeatedly labels unlabeled training samples based on the confidence scores and retrains itself with the selected pseudo-annotated data . Our proposed method can also be categorized as a self-training method . Figure 1 shows an overview of our SSL system . Since our proposed algorithm is based on the selftraining , we follow its learning process . This process can be formalized as follows . ( i ) Training a model with labeled data . ( ii ) Predicting unlabeled data with the learned model . ( iii ) Retraining the model with labeled and selected pseudo-labeled data . ( iv ) Repeating the last two steps . However , most self-training methods assume that the labeled and unlabeled data are generated from the identical distribution . Therefore , in real-world scenarios , some instances with low likelihood according to the distribution of the labeled data are likely to be misclassified inevitably . Consequently , these erroneous samples significantly lead to worse results in the next training step . To alleviate this problem , we adopt the ensemble and balancing methods to select reliable samples . 2.2 CONSISTENCY REGULARIZATION . Consistency regularization is one of the popular SSL methods and has been referred to many recent researches ( Laine & Aila , 2016 ; Miyato et al. , 2017 ; Tarvainen & Valpola , 2017 ) . Among them , Π model and temporal ensembling are widely used ( Laine & Aila , 2016 ) . They have defined new loss functions for unlabeled data . The Π model outputs f ( x ) and f̂ ( x ) for the same input x by perturbing the input with different random noise and using dropout ( Srivastava et al. , 2014 ) , and then minimizes the difference ( ‖f ( x ) − f̂ ( x ) ‖2 ) between these output values . Temporal ensembling does not make different predictions f ( x ) and f̂ ( x ) , but minimizes the difference ( ‖ft−1 ( x ) −ft ( x ) ‖2 ) between the outputs of two consecutive iterations for computational efficiency . In spite of the improvement in performance , they require lots of things to consider for training . These methods have various hyperparameters such as ‘ ramp up ’ , ‘ ramp down ’ , ‘ unsupervised loss weight ’ and so on . In addition , customized settings for training such as ZCA preprocessing and mean-only batch normalization ( Salimans & Kingma , 2016 ) are also very important aspects for improving the performance ( Odena et al. , 2018 ) . Algorithm 1 Training procedure of the proposed SST Require : xi , yi : training data and label Require : L , U : labeled and unlabeled datasets Require : IU : set of unlabeled sample indices Require : fn ( · ; θn ) , fcl ( · ; θc ) and fsel ( · ; θs ) : trainable SST model Require : α , , K , Kre : hyper-parameters , 0 ≤ α < 1 , 0 ≤ < 1 1 : randomly initialize θn , θc , θs 2 : train fn ( · ; θn ) , fcl ( · ; θc ) and fsel ( · ; θs ) for K epochs using L 3 : repeat 4 : initialize rti = −1 , IS = ∅ 5 : for each i ∈ IU do 6 : rt−1i ← rti , rti ← fcl ( fn ( xi ; θn ) ; θc ) , si ← fsel ( fn ( xi ; θn ) ; θs ) 7 : if rt−1i 6= rti then 8 : zi ← 0 9 : end if 10 : zi ← αzi + ( 1− α ) si 11 : if zi > 1− then 12 : IS ← IS ∪ { i } 13 : assign label for xi using ri 14 : end if 15 : end for 16 : update US with data balancing 17 : T ← L ∪ US 18 : retrain fn ( · ; θn ) , fcl ( · ; θc ) and fsel ( · ; θs ) for Kre epochs using T 19 : until stopping criterion is true

Summary: This paper describes the method for performing self-training where the unlabeled datapoints are iteratively added to the training set only if their predictions by the classifier are confident enough. The contributions of this paper are to add datapoints based on the prediction of the confidence level by a separate selection network and a number of heuristics applied for better selection. On the experimental side, the contribution is to test the scenario where datapoints from irrelevant classes are included in the unlabeled dataset.