Information Condensing Active Learning

1 Introduction . Machine learning models are widely used for a vast array of real world problems . They have been applied successfully in a variety of areas including biology ( Ching et al. , 2018 ) , chemistry ( SanchezLengeling and Aspuru-Guzik , 2018 ) , physics ( Guest et al. , 2018 ) , and materials engineering ( AspuruGuzik and Persson , 2018 ) . Key to the success of modern machine learning methods is access to high quality data for training the model . However such data can be expensive to collect for many problems . Active learning ( Settles , 2009 ) is a popular methodology to intelligently select the fewest new data points to be labeled while not sacrificing model accuracy . The usual active learning setting is pool-based active learning where one has access to a large unlabeled dataset DU and uses active learning to iteratively select new points from DU to label . Our goal in this paper is to develop an active learning acquisition function to select points that maximize the eventual test accuracy which is also one of the most popular criteria used to evaluate an active learning acquisition function . In active learning , an acquisition function is used to select which new points to label . A large number of acquisition functions have been developed over the years , mostly for classification ( Settles , 2009 ) . Acquisition functions use model predictions or point locations ( in input feature or learned representation space ) to decide which points would be most helpful to label to improve model accuracy . We then query for the labels of those points and add them to the training set . While the past focus for acquisition functions has been the acquisition of one point at a time , each round of label acquisition and retraining of the ML model , particularly in the case of deep neural networks , can be expensive . Furthermore in several applications like biology , it can be much faster to do acquisition of a fixed number of points in parallel versus sequentially . There have been several papers , particularly in the past few years , that try to avoid this issue by acquiring points in batch . As our goal is to apply AL in the context of modern ML models and data , we focus in this paper on batch-mode AL . Acquisition functions can be broadly thought of as belonging to two categories . The ones from the first category directly focus on minimizing the error rate post-acquisition . A natural choice of such an acquisition function might be to acquire labels for points with the highest uncertainty or points closest to the decision boundary ( Uncertainty sampling can be directly linked to minimizing error rate in the context of active learning Mussmann and Liang ( 2018 ) ) . In the other category , the goal is to get as close as possible to the true underlying model . Thus here , acquisition functions select points which give the most amount of knowledge regarding a model ’ s parameters where knowledge is defined as the statistical dependency between the parameters of the model and the predictions for the selected points . Mutual information ( MI ) is the usual choice for the dependency , though other choices are possible . For well-specified model spaces , e.g . in physics , such a strategy can identify the correct model . In machine learning , however , models are usually mis-specified , and thus the metric of evaluation even for model-identification acquisition functions is how successful they are at reducing test error . Given this reality , we follow the viewpoint of trying to minimize the error rate of the model post-acquisition . Our strategy is to select points that we expect would provide substantial information about the labels of the rest of the unlabeled set , thus reducing model uncertainty . We propose acquiring a batch of points B such that the model ’ s predictions on B have as high a statistical dependency as possible with the model ’ s predictions on the entire unlabeled set DU . Thus we want a batch B that condenses the most amount of information about the model ’ s predictions on DU . We call our method Information Condensing Active Learning ( ICAL ) . A key desideratum for our acquisition function is to be model agnostic . This is partly because the model distribution can be very heterogeneous . For example , ensembles which are often used as a model distribution can consist of just decision trees in a random forest or different architectures for a neural network . This means we can not assume any closed form for the model ’ s predictive distribution , and have to resort to Monte Carlo sampling of the predictions from the model to estimate the dependency between the model ’ s predictions on the query batch and the unlabeled set . MI , however , is known to be hard to approximate using just samples ( Song and Ermon , 2019 ) . Thus to scale the method to larger batch sizes , we use the Hilbert-Schmidt Independence Criterion ( HSIC ) , one of the most powerful extant statistical dependency measures for high dimensional settings . Another advantage of HSIC is that it is differentiable , which as we will discuss later , can allow applications of the acquisition function to areas where MI would be difficult to make work . To summarize , we introduce Information Condensing Active Learning ( ICAL ) which maximizes the amount of information being gained with respect to the model ’ s predictions on the unlabeled set of points . ICAL is a batch mode acquisition function that is model agnostic and can be applied to both classification and regression tasks . We then develop an algorithm that can scale ICAL to large batch sizes when using HSIC as the dependency measure between random variables . As our method only needs samples from the posterior predictive distribution which can be obtained for both regression and classification tasks , it is applicable to both . 2 Related work . A review of work on acquisition functions for active learning prior to the recent focus on deep learning is given by Settles ( 2009 ) . The BALD ( Bayesian Active Learning by Disagreement ) ( Houlsby et al. , 2011 ) acquisition function chooses a query point which has the highest mutual information about the model parameters . This turns out to be the point on which individual models sampled from the model distribution are confident about in their prediction but the overall predictive distribution for that point has high entropy . In other words this is the point on which the models are individually confident but disagree on the most . In Guo and Schuurmans ( 2008 ) which builds on Guo and Greiner ( 2007 ) , they formulate the problem as an integer program where they select a batch such that the post acquisition model is highly confident on the training set and has low uncertainty on the unlabeled set . While the latter aspect is related to what we do , they need to retrain their model for every candidate batch they search over in the course of trying to find the optimal batch . As the total number of possible batches is exponential in the size of the unlabeled set , this can get too computationally expensive for neural networks limiting the applicability of this approach . Thus as far as we know , Guo and Schuurmans ( 2008 ) has only been applied to logistic regression . BMDR ( Wang and Ye , 2015 ) queries points that are as close to the classifier decision boundary as possible while still being representative of the overall sample distribution . The representativeness is measured using the maximum mean discrepancy ( MMD ) ( Gretton et al. , 2012 ) of the input features between the query batch and the set of all points , with a lower MMD indicating a more representative query batch . However this approach is limited to classification problems , as it is based on a decision boundary . BMAL ( Hoi et al. , 2006 ) selects a batch such that the Fisher information matrices for the total unlabeled set and the selected batch are as close as possible . The Fisher information matrix is however quadratic in the number of parameters and thus infeasible to compute for modern deep neural networks . FASS ( Filtered Active Subset Selection ) ( Wei et al. , 2015 ) picks the most uncertain points and then selects a subset of those points that are as similar as possible to the whole candidate batch which favors points that can represent the diversity of the initial set of most uncertain points . Recently active learning methods have been extended to the deep learning setting . Gal et al . ( 2017 ) adapts BALD ( Houlsby et al. , 2011 ) to the deep learning setting by using Monte Carlo Dropout ( Gal and Ghahramani , 2016 ) to do inference for their Bayesian Neural Network . They extend BALD to the batch setting for neural networks with BatchBALD ( Kirsch et al. , 2019 ) . In Pinsler et al . ( 2019 ) , they adapt the Bayesian Coreset ( Campbell and Broderick , 2018 ) approach for active learning , though their approach requires a batch size that changes for every acquisition . As the neural network decision boundary is intractable , DeepFool ( Ducoffe and Precioso , 2018 ) uses the concept of adversarial examples ( Goodfellow et al. , 2014 ) to find points close to the decision boundary . However this approach is again limited to classification tasks . FF-Comp ( Geifman and El-Yaniv , 2017 ) , DAL ( Gissin and Shalev-Shwartz , 2019 ) , Sener and Savarese ( 2017 ) , and BADGE ( Ash et al. , 2019 ) operate on the learned representation , as that is the only way the methods incorporate feedback from the training labels into the active learning acquisition function , and they are thus not model-agnostic , as they are not extendable to any model distribution where it is difficult to have a notion of a common representation – as in a random forests or ensembles , etc . where the learned representation is a distribution and not a single point . This is also the case with the model distribution – MC-dropout – we use in this paper . There is also extensive prior work on exploiting Gaussian Processes ( GPs ) for Active Learning ( Houlsby et al. , 2011 ; Krause et al. , 2008 ) . However GPs are hard to scale especially for modern image datasets . 3 Background . Statistical background The entropy of a distribution is defined as HpY q “ ´ ř xPX px logppxq , where px is the probability of the x . Mutual information ( MI ) between two random variables is defined as IrX ; Y s “ ř xPX ř yPY ppx , yq logp ppx , yq ppxqppyq q , where ppx , yq is the joint probability of x , y . Note that IrX ; Y s “ HpY q´HpY |Xq “ HpXq´HpX|Y q . By posterior predictive distribution yx we mean ş θ ppy|x , θqppθ|Dqdθ where y is the prediction , x the input point , θ the model parameters , and D the training data . M is the distribution of models ( parametrized by θ ) we wish to choose from via active learning . As mentioned before , we use MC-dropout for our model distribution by sampling random dropout masks and use the same set of dropout masks across points to generate joint predictions . Hilbert-Schmidt Independence Criterion ( HSIC ) Suppose we have two ( possibly multivariate ) distributions X , Y and we want to measure the dependence between them . A well known way to measure it is using distance covariance which intuitively , measures the covariance between the distances of pairs of samples from the joint distribution PXY and the product of marginal distributions P pXq , P pY q ( Székely et al. , 2007 ) . HSIC can simply be thought of as distance covariance except in a kernel space ( Sejdinovic et al. , 2013b ) . A ( sample ) kernel matrix kX is a matrix whose ijth element is kpxi , xjq where k is the kernel function and xi , xj are the i , jth samples from X . Further details are in the Appendix . Acquisition Function Let the batch to acquire be denoted by B with B “ |B| . Given a model distribution M , training data Dtrain , unlabeled data DU , input space X , set of labels Y and an acquisition function αpx , Mq , we decide which batch of points to query next via : B˚ “ arg max B αpB , Mq

Summary: This paper proposes a way to do batch mode model agnostic active learning. In this task, the agent has to query a batch of data points from a set of unlabeled examples for which it will get labels. The paper puts an additional requirement that the algorithm is model-agnostic. The key idea here is to sample a batch of points that provide the most "information" about the remaining unlabeled examples.  Authors argue that this will result in higher performance on the unlabeled examples. The proposed approach called ICAL (Information Condensing Active Learning) uses Hilbert-Schmidt Independence Criterion (HSIC) to measure dependence between a chosen batch and the unlabeled examples. The goal is to pick a batch with a maximum value of HSIC which should intuitively give us a batch which is representative of the unlabeled set. HSIC can be easily estimated unlike other dependence measures such as mutual information. Given a batch size $|B|$, a dataset of unlabeled examples $D_u$  and $m$ samples to estimate HSIC, the ICAL algorithm computes a batch for label acquisition in $O(|D_U|m^2|B|)$ steps, where a greedy strategy is used to search over batch.  Results are presented over MNIST, variants of MNIST and CIFA and show improvements over five previous active learning approaches and a random acquisition baseline.