Discriminative Active Learning

We propose a new batch mode active learning algorithm designed for neural networks and large query batch sizes . The method , Discriminative Active Learning ( DAL ) , poses active learning as a binary classification task , attempting to choose examples to label in such a way as to make the labeled set and the unlabeled pool indistinguishable . Experimenting on image classification tasks , we empirically show our method to be on par with state of the art methods in medium and large query batch sizes , while being simple to implement and also extend to other domains besides classification tasks . Our experiments also show that none of the state of the art methods of today are clearly better than uncertainty sampling , negating some of the reported results in the recent literature . 1 INTRODUCTION . In the past few years deep neural networks achieved groundbreaking results in many machine learning tasks , ranging from computer vision to speech recognition and NLP . While these successes can be attributed to the increase in computational power and to the expressiveness of neural networks , they are also a clear result of the use of massive datasets which contain millions of samples . The collection and labeling of such datasets is highly time consuming and often expensive , and much research has been done to try and reduce the cost of this process . Active learning is one field of research which tries to address the difficulties of data labeling . It assumes the collection of data is relatively easy but the labeling process is costly , which is the case in many language , vision and speech recognition tasks . It tackles the question of which samples , if labeled , will result in the highest improvement in test accuracy under a fixed labeling budget . The most popular active learning setting is pool-based sampling ( Settles , 2012 ) , where we assume that we have a small labeled dataset L and access to a large unlabeled dataset U from which we choose the samples to be labeled next . We then start an iterative process where in every step the active learning algorithm uses information in L and U to choose the best x ∈ U to be labeled . x is then labeled by an oracle and added to L , and the process continues until we have reached the desired amount of samples or test accuracy . Many heuristic methods have been developed over the years ( Settles , 2012 ) , mostly for classification tasks , that have been shown to reduce the sample size by up to an order of magnitude . These methods fit a model to the labeled dataset and then choose a single sample to be labeled according to the performance of the model on the samples in U . While these methods are very successful , they have a few shortcomings that make them less suited for large scale data collection for deep learning . First , it is highly impractical to query a single sample from U in every iteration , since we need a human in the loop for every one of those iterations . Adding to that the time needed to fit a neural network , collecting a large dataset using this setting simply takes too long . Second , many of these methods are designed solely for classification tasks , and extending them to other domains is often non trivial . To tackle the first problem , the batch-mode active learning setting was introduced , allowing for the querying of a batch of examples at every iteration . This change adds complexity to the problem since it might not be enough to greedily choose the top-K samples from U according to some score , since they may be very similar to each other . Choosing a batch of diverse examples is often better than choosing one containing very similar examples , which may lead to performance that is worse than random sampling . Several methods were developed with this problem in mind ( Chen & Krause , 2013 ) , ( Guo & Schuurmans , 2008 ) , ( Azimi et al. , 2012 ) . The second problem however has not been explicitly addressed to the best of our knowledge , and active learning solutions to other domains have had to be adapted separately ( Settles , 2008 ) , ( Vezhnevets et al. , 2012 ) . To address the second problem and make the method agnostic to the specific task being performed , we do not assume access to a classifier trained on the labeled set when choosing the batch to query and instead only assume access to a learned representation of the data , Ψ ( x ) . We pose the active learning problem as a simple binary classification problem and choose the batch to query according to a solution to said problem . We compare our method empirically to today ’ s state-of-the-art active learning methods for image classification , showing competitive results while being simple to implement and conceptually simple to transfer to new tasks . 2 RELATED WORK . There has been extensive work on active learning prior to the advances in deep learning of the recent years . This work is summarized in ( Settles , 2012 ) which covers the many different heuristics developed for active learning , along with adaptation made for other tasks besides classification . A large empirical study of different heuristics applied to classification using the logistic regression model can be found in ( Yang & Loog , 2018 ) . This study shows uncertainty-based methods ( Lewis & Gale , 1994 ) perform well on a large and diverse set of datasets while being very simple to implement . These methods score the unlabeled examples according to the classifier ’ s uncertainty about their class , and choose the example with highest uncertainty . While neural networks for classification tasks output a probability distribution over labels , they have been shown to be over-confident and their softmax scores are unreliable measures of confidence . Recent work ( Gal et al. , 2017 ) shows an equivalence between dropout , a widely used neural network regularization method , and approximate Bayesian inference . This allows for the use of dropout at test time in order to achieve better confidence estimations of the network ’ s outputs . Our empirical analysis shows this method performs very similarly to uncertainty sampling using the regular softmax scores . In addition to uncertainty-based approaches to active learning , another approach that has been actively studied in the past is the margin-based approach , which has a similar flavor . Instead of scoring an example based on the output probabilities of the model , these methods choose the examples which are closest to the decision boundary of the model . While in linear models like logistic regression for binary classification problems these two approaches are equivalent , this isn ’ t the case for neural networks that have an intractable decision boundary . ( Ducoffe & Precioso , 2018 ) approximates the distance to the decision boundary using the distance to the nearest adversarial example ( Szegedy et al. , 2013 ) . Our empirical analysis shows that while this method is conceptually very different from uncertainty sampling , it ranks the unlabeled examples in a very similar way . Another active learning approach is based on expected model change , choosing examples which will cause the biggest effect to our model when labeled . ( Settles et al. , 2008 ) and ( Huang et al. , 2016 ) use this approach by calculating the expected gradient length ( EGL ) of the loss with respect to the model parameters , when the expectation is over the posterior distribution over the labels induced by the trained classifier . While their results show significant improvements over random sampling for text and speech recognition tasks , our empirical analysis along with the analysis of ( Ducoffe & Precioso , 2018 ) show that this method is not suited for image classification using CNNs . Finally , ( Sener & Savarese , 2018 ) and ( Geifman & El-Yaniv , 2017 ) address the challenge posed by the batch-mode setting by formulating the active learning task as a core set selection problem , attempting to choose a batch which -covers the unlabeled set as closely as possible and obtains a diverse query in the process . This is done robustly by formulating the -cover as an integer program which allows a small subset of the examples ( outliers ) not to be covered . The metric space over which the -cover is optimized is the learned representation of the data . While not discussed in their paper , the core set approach is similar to our method in that it only requires access to a learned representation of the data and not a trained classifier , which allows their method to easily extend to other tasks as well . However , their robust formulation is not as straightforward to implement as our method and does not scale as easily to large unlabeled datasets . 3 DAL - DISCRIMINATIVE ACTIVE LEARNING . We motivate our proposed method with a simple idea - when building a labeled dataset out of a large pool of unlabeled examples , we would like our dataset to represent the true distribution of the data as well as possible . In high dimensional and multi-modal data , this would ensure our labeled set has samples from all of the different modalities of the distribution . A naive solution would be to model the distribution of the unlabeled pool and use the learned distribution to generate a subsample for labeling , but the generative models of today are either hard to train or break down in high dimensional problems , so a different solution is needed . Assuming the unlabeled pool is large enough to represent the true distribution , we can instead ask for every example how certain we are that it came from the unlabeled pool as opposed to our current labeled set . If the examples from the labeled set are indistinguishable from the unlabeled pool , then we have successfully represented the distribution with our labeled set . Another intuitive motivation is that if we can say with high probability that an unlabeled example came from the unlabeled set , then it is different from our current labeled examples and labeling it should be informative . This proposition allows us to pose the active learning problem as a binary classification problem between the unlabeled class , U , and the labeled class , L. Given a simple binary classification task , we can now leverage powerful modern classification models to solve our problem . Formally , with Ψ : X → X̂ being the mapping from the original input space to the learned representation , we define a binary classification problem with X̂ as our input space and Y = { l , u } as our label space , where l is the label for a sample being in the labeled set and u is the label for the unlabeled set . For every iteration of the active learning process , we solve the classification problem over U ∪ L by minimize the log loss and obtaining a model P̂ ( y|Ψ ( x ) ) . We then select the top-K samples that satisfy argmax x∈U P̂ ( y = u|Ψ ( x ) ) . 3.1 RELATION TO DOMAIN ADAPTATION . We next show a connection to domain adaptation in the binary classification setting . This connection gives some theoretical motivation to our method . Domain adaptation deals with the problem of bounding the error of a classifier trained on a source distribution DS but then tested on a different target distribution DT . Naturally , we are interested in how different the two distributions are . Following ( Kifer et al. , 2004 ) , ( Ben-David et al. , 2010 ) suggests theH-divergence as a relevant distance : Definition 3.1 Given a domain X and two distributions , DS and DT , over X , and given a hypothesis classH over X , theH-divergence between DS and DT is dH ( DS , DT ) = 2 sup h∈H |Px∼DS [ h ( x ) = 1 ] − Px∼DT [ h ( x ) = 1 ] | Furthermore , ( Ben-David et al. , 2010 ) relates the domain adaptation error to dH ( DS , DT ) , showing that the test error overDT of a classifier which was trained onDS is bounded by a term that depends on dH ( DS , DT ) . The similarity to our case is straightforward : we train a predictor based on the labeled examples , and expect it to also work well on the unlabeled examples . To do so , we would like the distributions over the labeled and unlabeled examples to be as similar as possible . Formally , we will construct a source and target distributions as follows : the “ target ” distribution will be the distribution which is focused evenly on the examples in the unlabeled set , namely , DT = 1 |U| ∑ x∈U δ ( x ) . Similarly , the “ source ” distribution will be DS = 1 |L| ∑ x∈L δ ( x ) . Observe that the H divergence becomes dH ( DS , DT ) = 2 sup h∈H ∣∣∣∣∣ 1|L|∑ x∈L h ( x ) − 1 |U| ∑ x∈U h ( x ) ∣∣∣∣∣ Solving this problem is equivalent to finding a classifier in H that discriminates between L and U . So , if we want to minimize the difference in performance between source and target distributions , we should minimize dH ( DS , DT ) . DAL can be viewed as a proxy to minimizing dH ( DS , DT ) in a greedy manner — at each iteration , we first ( approximately ) find the h that achieves the supremum in the definition of dH ( DS , DT ) and then we move an example from U to L so as to decrease dH ( DS , DT ) .

Summary: The paper is proposing a distribution matching as a metric for active learning. Basic intuition is: if we can make the distribution of labelled and unlabelled examples similar to each other, training error in one will approximate the training error in the other. Hence, a model learned using labelled ones will do well in unlabelled ones. The main tool to enforce this distributional distance is using adversarial learning similar to GANs or gradient reversal network for domain adaptation.