Consistency-Based Semi-Supervised Active Learning: Towards Minimizing Labeling Budget

Active learning ( AL ) integrates data labeling and model training to minimize the labeling cost by prioritizing the selection of high value data that can best improve model performance . Readily-available unlabeled data are used for selection mechanisms , but are not used for model training in most conventional pool-based AL methods . To minimize the labeling cost , we unify unlabeled sample selection and model training based on two principles . First , we exploit both labeled and unlabeled data using semi-supervised learning ( SSL ) to distill information from unlabeled data that improves representation learning and sample selection . Second , we propose a simple yet effective selection metric that is coherent with the training objective such that the selected samples are effective at improving model performance . Experimental results demonstrate superior performance of our proposed principles for limited labeled data compared to alternative AL and SSL combinations . In addition , we study an important problem – “ When can we start AL ? ” . We propose a measure that is empirically correlated with the AL target loss and can be used to assist in determining the proper start point . 1 INTRODUCTION . Deep learning models are improved when trained with more labeled data ( Goodfellow et al. , 2016 ) . A standard deep learning development procedure involves constructing a large-scale labeled dataset and optimizing a model with it . Yet , in many real-world scenarios , large-scale labeled datasets can be very costly to acquire , especially when expert annotators are required , as in medical diagnosis and loan prediction . An ideal framework would integrate data labeling and model training to improve model performance with minimal amount of labeled data . Active learning ( AL ) ( Balcan et al. , 2009 ) assists the learning procedure by judicious selection of unlabeled samples for labeling , with the goal of maximizing the ultimate model performance with minimal labeling cost . We focus on pool-based AL where an unlabeled data pool is given initially and the AL mechanism iteratively selects batches to label in conjunction with training . As the name “ learning-based AL selection ” suggests , each batch is selected with guidance from the previouslytrained model , is labeled , and then added into the labeled dataset on which the model is trained . Maximization of performance with minimal labeled data requires properly leveraging model learning and AL sample selection , especially in early AL cycles . While using the unlabeled pool as the candidates in the AL sample selection phase , it is natural for pool-based AL methods to integrate SSL objectives to improve performance by learning meaningful data representations from the unlabeled pool ( Zhu et al. , 2003 ; Tomanek & Hahn , 2009 ) . However , fewer existing AL methods consider SSL during training ( Drugman et al. , 2019a ; Rhee et al. , 2017 ; Zhu et al. , 2003 ; Sener & Savarese , 2018 ) compared to those utilizing only labeled samples . Moreover , we believe that AL selection criterion should be in coherence with the SSL objectives to select the most valuable samples , since ( 1 ) unsupervised losses could alter the learned representation and decision manifolds significantly ( Oliver et al. , 2018 ) , and the AL sample selection should reflect that ; ( 2 ) SSL already results in the embodiment of knowledge from unlabeled data in a meaningful way ; thus AL selection should reflect the extra value of the labeled data on top of it . Motivated by these observations , we propose an AL framework that combines SSL with AL and also a selection metric that is strongly related to the training objective . In the absence of labeled data , a common practice to initiate AL is to uniformly select a small starting subset of data for labeling . Learning-based AL selection is then used in subsequent cycles . The size of the starting subset affects AL performance – when the start size is not sufficiently large , the models learned in subsequent AL cycles are highly-skewed and result in biased selection , a phenomenon commonly known as the cold start problem ( Konyushkova et al. , 2017 ; Houlsby et al. , 2014 ) . When cold start issues arise , learning-based selection yields samples that lead to lower performance improvement than using naive uniform sampling ( Konyushkova et al. , 2017 ) . Increasing the start size alleviates the cold start problem , but consumes a larger portion of the labeling budget before learning-based AL selection is utilized . With better understanding of data , our method relieves this problem by allowing learning-based sample selection to be initialized from a much smaller start size . However , an ideal solution is determining a proper start size that is large enough in avoiding cold start problems , yet sufficiently small to minimize the labeling cost . To this end , we propose a measure that is empirically shown to be helpful in estimating the proper start size . Contributions : We propose a simple yet effective selection metric that is in coherent with training objectives in SSL . The proposed AL method is based on an insight that has driven recent advances in SSL ( Berthelot et al. , 2019 ; Verma et al. , 2019 ; Xie et al. , 2019 ) : a model should be consistent in its decisions between a sample and its meaningfully-distorted versions . This motivates us to introduce an AL selection principle : a sample along with its distorted variants that yields low consistency in predictions indicates that the SSL model is incapable of distilling useful information from that unlabeled sample , thus human labeling is needed . Experiments demonstrate that our proposed metric outperforms previous methods integrated with SSL . With various quantitative and qualitative analyses , we demonstrate the rationale behind why such a selection criteria is highly effective in AL . In addition , in an exploratory analysis we propose a measure that can be used to assist in determining the proper start size to mitigate cold start problems . 2 RELATED WORK . Extensive research has been done in AL ( Dasgupta et al. , 2008 ; Dasgupta & Hsu , 2008 ; Balcan et al. , 2009 ; Cortes et al. , 2019a ) . Traditional AL methods can be roughly classified into three categories : uncertainty-based methods , diversity-based methods and expected model change-based methods . Among uncertainty-based ones , methods based on max entropy ( Lewis & Catlett , 1994 ; Lewis & Gale , 1994 ) and max margin ( Roth & Small , 2006 ; Balcan et al. , 2007 ; Joshi et al. , 2009 ) are popular for their simplicity . Some other uncertainty-based methods measure distances between samples and the decision boundary ( Tong & Koller , 2001 ; Brinker , 2003 ) . Most uncertainty-based methods use heuristics , while recent work ( Yoo & Kweon , 2019a ) directly learns the target loss of inputs jointly with the training phase and shows promising results . Diversity-based methods select diverse samples that span the input space the most ( Nguyen & Smeulders , 2004 ; Mac Aodha et al. , 2014 ; Hasan & Roy-Chowdhury , 2015 ; Sener & Savarese , 2018 ) . There are also methods that consider both uncertainty and diversity ( Guo , 2010 ; Elhamifar et al. , 2013 ; Yang et al. , 2015 ) . The third category estimates the future model status and selects samples that encourage optimal model improvement ( Roy & McCallum , 2001 ; Settles et al. , 2008 ; Freytag et al. , 2014 ) . Both AL and SSL aim to improve learning with limited labeled data , thus they are naturally related . Only a few works have considered combining AL and SSL in different tasks . In Drugman et al . ( 2019b ) , joint application of SSL and AL is considered for speech understanding , and significant error reduction is demonstrated with limited labeled speech data . For AL , their selection criteria is based on a confidence score that quantifies the observed probabilities of words being correct . Rhee et al . ( 2017 ) propose an active semi-supervised learning system which demonstrates superior performance in the pedestrian detection task . Zhu et al . ( 2003 ) combine AL and SSL using Gaussian fields and validate their method on synthetic datasets . Sener & Savarese ( 2018 ) also consider SSL during AL cycles . However , in their setting , the performance improvement is marginal when adding SSL in comparison to their supervised counterpart , potentially due to the suboptimal SSL method . Agreement-based methods , also referred as “ query-by-committee ” , base the selection decisions on the opinions of a committee which consist of independent AL metrics or models ( Seung et al. , 1992 ; Cohn et al. , 1994 ; McCallumzy & Nigamy , 1998 ; Iglesias et al. , 2011 ; Beluch et al. , 2018 ; Cortes et al. , 2019b ) . Our method is related to agreement-based AL where samples are determined based Algorithm 1 A semi-supervised learning based AL framework Require : Unlabeled data poolD , the total number of steps T , AL batch size K , start size K0 |D| B0 ← uniformly sampling from D with |B0| = K0 U0 ← D\B0 L0 ← { ( x , J ( x ) ) : x ∈ B0 } , where J ( x ) stands for the assigned label of x. for t = 0 , . . . , T − 1 do ( training ) Mt ← argminM { 1 |Lt| ∑ ( x , y ) ∈Lt Ll ( x , y , M ) + 1 |Ut| ∑ x∈Ut Lu ( x , M ) } ( selection ) Bt+1 ← argmaxB⊂Ut { C ( B , Mt ) , s.t . |B| = K } ( labeling ) Lt+1 ← Lt ∪ { ( x , J ( x ) ) : x ∈ Bt+1 } ( pool update ) Ut+1 ← Ut \Bt+1 end for MT ← argminM { 1 |LT | ∑ ( x , y ) ∈LT Ll ( x , y , M ) + 1 |UT | ∑ x∈UT Lu ( x , M ) } return MT on the conformity of different metrics or models . Specifically , our method selects data that mostly disagrees with the predictions of its augmentations . 3 CONSISTENCY-BASED SEMI-SUPERVISED AL . 3.1 PROPOSED METHOD . We consider the setting of pool-based AL , where an unlabeled data pool is available for selection of samples to label . To minimize the labeling cost , we propose a method that unifies selection and model updates , overviewed in Algorithm 1 . The proposed method has two key aspects . Most conventional AL methods base model learning only on the available labeled data , which ignores the useful information in the unlabeled data . Our first contribution is incorporating a semisupervised learning ( SSL ) objective in the training phases of AL . Specifically , as shown in Algorithm 1 , each model Mt is learned by minimizing an objective loss function of the form Ll + Lu . The model should both fit the labeled data well and obtain a good representations of the unlabeled data . The design of the selection criterion plays a crucial role in integrating SSL and AL . To this end , our second contribution is a selection criterion C to better integrate AL selection mechanism in the SSL training framework . It has been observed that predictions of deep neural networks are sensitive to small perturbations on the input data ( Zheng et al. , 2016 ; Azulay & Weiss , 2018 ) . Recent successes in SSL ( Athiwaratkun et al. , 2019 ; Berthelot et al. , 2019 ; Verma et al. , 2019 ) are based on minimizing the notion of sensitivity to perturbations with the idea of inducing “ consistency ” , i.e. , imposing similarity in predictions when the input is perturbed in a way that would not change its perceptual content . For consistency-based semi-supervised training , a common choice of loss is Lu ( x , M ) = D ( P ( Ŷ = ` |x , M ) , P ( Ŷ = ` |x̃ , M ) ) , where D is a distance function such as KL divergence ( Xie et al. , 2019 ) , or L2 norm ( Laine & Aila , 2017 ; Berthelot et al. , 2019 ) and x̃ denotes a perturbation ( augmentation ) of the input x . Our proposal is motivated by the following intuition . First , the unsupervised objective exploits unlabeled data by encouraging consistent predictions across slightly distorted version of each unlabeled sample . Labeling samples with highly inconsistent predictions is valuable , since these samples are hard to be minimized using Lu . Thus , they need human annotations to provide further useful supervision for model training . Second , the data that yields large model performance gain is not necessarily the data with the highest uncertainty , since neural network prefers learning with a particular curriculum ( Bengio et al. , 2009 ) . The most uncertain data could be too hard to learn , and including them in training would be misleading . Thus , we argue that labeling samples that can be recognized to some extent but not consistently should benefit learning more compared to the most uncertain ones . Specifically , we propose a simple metric C measures the inconsistency across perturbations . There are various ways to quantify consistency . Due to its empirically-observed superior performance , we choose C ( B , M ) = ∑ x∈B E ( x , M ) , where E ( x , M ) = J∑ ` =1 Var [ P ( Ŷ = ` |x , M ) , P ( Ŷ = ` |x̃1 , M ) , ... , P ( Ŷ = ` |x̃N , M ) ] , ( 1 ) J is the number of response classes and N is the number of perturbed samples of the original input data x , { x̃1 , ... , x̃N } , which can be obtained by standard augmentation operations 1 . Our method selects data samples with high C values for labeling , which may possess varying level of difficulty for the model to classify .

Summary: This paper proposes a semi-supervised active learning method to reduce the labeling cost. In the proposed method, a selection criterion to better integrate AL selection mechanism in SSL training framework is designed. The simple metric that aims to measure the inconsistency across a certain number of meaningful perturbations. It considers N perturbed samples of the original input data x, which can be obtained by standard augmentation operations (e.g. random crops and horizontal flips for image data). Then the variance is adopted to quantify consistency.  In this way, the proposed method prefers data samples with high values, which may possess varying level of difficulty for the model to classify. To verify the effectiveness of the proposed method, several baseline methods are compared on several benchmark data sets, and the proposed method has achieved better performance. Meanwhile, to deal with the “cold start” problem, a measure that is found to be empirically correlated with the AL target loss is proposed, and this measure can be used to assist in determining the proper start size. However, there are some minor concerns: