Improving greedy core-set configurations for active learning with uncertainty-scaled distances

1 INTRODUCTION . Active learning aims to identify the most informative data to label and include in supervised training . Often , these algorithms focus on reducing model variance , representing distributional densities , maximizing expected model change , or minimizing expected generalization error ( Kirsch et al. , 2019 ; Sener & Savarese , 2018 ; Settles , 2009 ; Shen et al. , 2018 ; Sinha et al. , 2019 ) . A unifying theme is efficient data collection , which is measured by the rate in improvement as more data are labelled . This is important when we want to identify only the most promising samples to be labelled , but also for tasks that require slow or expensive labelling ( Ducoffe & Precioso , 2018 ; Ma et al. , 2020 ; Settles , 2009 ) . We describe active learning with the same notation as Sener & Savarese ( 2018 ) . Suppose we wish to classify elements of a compact space X into labels Y = { 1 , . . . , C } . We collect n data points { xi , yi } i∈ [ n ] ∼ PX×Y , but only have access to the labels of m of these , denoted by their indices s = { s ( i ) ∈ [ n ] } i∈ [ m ] . We use the learning algorithmAs on the labelled set s to return the optimized parameters of the classifier , and measure performance with the loss function l ( · , · ; As ) : X×Y → R. The goal of active learning is to produce a set of indices s+ whose cardinality is limited by the labelling budget b , such that expected loss is minimized upon labelling and training on these elements : arg mins+ : |s+|≤b Ex , y∼PX×Y [ l ( x , y ; As∪s+ ) ] ( Sener & Savarese , 2018 ) . In practice , we use the test set { xi , yi } i∈ [ t ] ∼ PX×Y to approximate the expectation . The typical way to assess the dataefficiency of any particular active learning algorithm is to compare its trend of test performance across increasing labels compared to random and other sampling baselines ( Kirsch et al. , 2019 ; Sener & Savarese , 2018 ; Settles , 2009 ; Shen et al. , 2018 ; Sinha et al. , 2019 ; Ducoffe & Precioso , 2018 ; Ma et al. , 2020 ) . Sener & Savarese ( 2018 ) suggested that we can improve data-efficiency by minimizing the core-set radius , δ , defined as the maximum distance of any unlabelled point from its nearest labelled point : δ = maxi∈ [ n ] minj∈s δ ( xi , xj ) . Given the generalization error ζn of all labelled and unlabelled data , and zero training error , expected error converges linearly with respect to δ ( Sener & Savarese , 2018 ) : E x , y∼PX×Y [ l ( x , y ; As ) ] ≤ ζn + 1 n ∑ i∈ [ n ] l ( xi , yi ) ≤ ζn +O ( Cδ ) +O ( √ 1 n ) ( 1 ) Sener & Savarese ( 2018 ) argued that generalization error for neural networks has well-defined bounds , so optimizing the rest of Equation 1 , referred to as core-set loss , is critical for active learning . Indeed , their algorithms for optimizing core-sets consistently improved over their baselines ( Sener & Savarese , 2018 ) . Uncertainty-based sampling is particularly valuable for identifying support vectors , leading to finer classification boundaries ( Kirsch et al. , 2019 ; Settles , 2009 ) . However , these methods may catastrophically concentrate their labelling budget on difficult , noisy regions between classes , as shown in Figure 1 . We present a two-part solution for incorporating uncertainty into core-sets : 1 . Scale distances between points by doubt ( I ) ( Settles , 2009 ; Shen et al. , 2018 ) before computing core-set radii : δ̂i = δi I ( xi ) , where I ( x ) = 1−max y P ( y|x ) ( 2 ) 2 . Apply beam search to greedily identify the core-set configuration among K-candidates with the lowest maximum log-confidence to reduce the variance of core-set trajectories . 2 BACKGROUND . Greedy versus optimal core-set for active learning . Core-set radius δ is the maximum of all distances between each data point in xu = { xi : ∀i ∈ [ n ] } and its closest labelled point in xl = { xi : ∀i ∈ s } ( Sener & Savarese , 2018 ) . The optimal coreset achieves linear convergence of core-set loss in respect to δ by finding the acquisition set s+ with optimal core-set radii δOPT shown in Equation 3 ( Sener & Savarese , 2018 ) . Sener and Savarese used l2-norm between activations of the last layer of VGG16 as ∆ . δOPT = min s+ max i∈ [ n ] min j∈s+∪s ∆ ( xi , xj ) ( 3 ) δg = max i∈ [ n ] min j∈ŝ+∪s ∆ ( xi , xj ) ≤ 2 δOPT ( 4 ) Since this problem is NP-hard ( Cook et al. , 1998 ) , Sener & Savarese ( 2018 ) proposed a greedy version shown in Algorithm 1 with acquisitions ŝ+ bounded above by Equation 4 . It returns a selection mask over the data pool to signal labelling requests for elements that greedily minimize the maximum distance between any point and its nearest labelled point . Algorithm 1 : Greedy core-set ( Sener & Savarese , 2018 ) 1 def greedy core-set ( xu , xl , budget ) : 2 selection = [ 0 : ∀i ∈ [ |xu| ] ] ; 3 for t = 0 , . . . , budget do 4 i = arg maxi∈ [ |xu| ] minj∈ [ |xl| ] ∆ ( x ( i ) u , x ( j ) l ) ; 5 selection ( i ) = 1 ; 6 return selection ; Figure 2 shows how δ varies compared to closest-K-means core-sets , where the core-set consists of the closest points to optimized K-means . Related techniques for batched acquisition . Batch active learning by diverse gradient embeddings acquires batches in two steps . First , we compute loss gradients in respect to the parameters of the last layer of the classifier for each unlabelled point and its most probable label ( Ash et al. , 2020 ) . Then , we sample from clusters of these gradients using , for instance , K-means++ to avoid catastrophic concentration ( Ash et al. , 2020 ) . We share similar intuitions about classifier confidence and intra-batch diversity being important sources of information that may enhance active learning , but differ in that we do not optimize for model change and use core-sets for diversification because of its theoretical foundations . BatchBALD acquires batches that maximize the mutual information between the joint data and model parameters and was intended to overcome redundant sampling of repeated BALD ( Kirsch et al. , 2019 ) . We tried combining this with a probabilistic technique of estimating likely core-set locations ( see Appendix A.2 ) , but it appeared that core-sets mainly require δ minimization for coreset loss convergence . 3 METHODS . Algorithm 2 and its dependence on Algorithm 3 implement doubt-weighted greedy core-set to run on GPU . To incorporate uncertainty information , we make two key changes to the original greedy core-set algorithm . First , we compute core-sets in a warped space where distances originating from any unlabelled point diminish to zero with classification confidence . Given inputs xu and xl , which represent the unlabelled and labelled data , Line 2 of Algorithm 2 calls Algorithm 3 to pre-compute distances of each unlabelled datum to their nearest labelled data . We scale these distances by the doubt of the classifier on the respective unlabelled data on Lines 3 and 11 . Each acquisition is removed from the existing unlabelled pool , and Line 12 updates new core-set radii . Figure 3 illustrates how core-sets in these spaces preferentially cover regions of low confidence . Algorithm 2 : Doubt-weighted core-set 1 def doubted core-set ( xu , xl , batch size , budget ) : 2 min δ = compute min δ ( xu , xl , batch size ) ; 3 min δ̂ = [ min δ ( i ) · I ( x ( i ) u ) : ∀i ∈ [ |xu| ] ] ] ; 4 xu = memory-copy ( xu ) ; 5 index = [ i : ∀i ∈ [ |xu| ] ] ; 6 selection = [ 0 : ∀i ∈ [ |xu| ] ] ; 7 for t = 0 , . . . , budget do 8 i = arg max min δ̂ ( i ) ; 9 selection ( index [ i ] ) = 1 ; 10 x = x ( i ) u ; splice out : index ( i ) , x ( i ) u , min δ ( i ) 11 δ̂ = ∆ ( x , xu ) · I ( x ) ; 12 min δ̂ = [ min { min δ̂ ( k ) , δ̂ ( k ) } : ∀k ∈ [ |xu| ] ] 13 return selection ; We choose the same ∆ as Sener & Savarese ( 2018 ) , which is l2-norm between activations of the last layer of VGG16 . Given unlabelled data of size U = |xu| , labelled data of size L = |xl| , feature size ∀i ∈ [ U ] , ∀j ∈ [ L ] , D = dimx ( i ) u = dimx ( j ) l , batch size B min { U , L } and labelling budget b , Algorithm 3 costs Θ ( ULD ) steps and Θ ( B2D ) memory with the bottleneck on line 5 . Excluding line 2 , Algorithm 2 costs O ( bUD ) in both computation and memory with the bottleneck on line 11 . Since both the original core-set algorithm and our modification requires fine-tuning VGG16 per addition to the training set , and computing the class probabilities requires only a single linear transformation , the final computational complexity is the same as the original core-set search . For core-set sizes 5k to 15k , compute time scales linearly from 25 s to 50 s on a NVIDIA Titan GPU . Algorithm 3 : Memory-efficient core-set radii 1 def compute min δ ( xu , xl , b ) : 2 min δ = [ ∞ : ∀k ∈ [ |xu| ] ] ; 3 for i = 0 , . . . , |xu| b do 4 for j = 0 , . . . , |xl| b do 5 δ̃i : i+b , j : j+b = ∆ ( x ( i : i+b ) u , x ( j : j+b ) l ) ; 6 min δ̃i : i+b = [ min c∈ [ j , j+b ] δ̃ ( k , c ) i : i+b , j : j+b : ∀k ∈ [ i , i+ b ] ] ; 7 min δ ( i : i+b ) = [ min { min δ ( k ) , min δ̃ ( k ) i : i+b } : ∀k ∈ [ i , i+ b ] ] 8 return min δ ; Second , we use beam search to greedily prune and keep track of the top resulting core-set configurations with the lowest overall confidence . Since there is no guarantee for the optimality of greedy core-sets ( Sener & Savarese , 2018 ) , we seek an orientation with the most points near classification regions of high uncertainty at the cost of increasing compute and memory complexity by a factor of the beam width . We modify maximum normalized log probability ( Shen et al. , 2018 ) to rank overall classifier uncertainty U of core-set s : U ( s ) = − 1 |s| ∑ x∈s log ( 1− I ( x ) ) ( 5 ) Figure 4 shows a sample ranking of the configurations found during beam search with widthK = 4 . Active learning pipeline . For each active learning experiment , we start by randomly partitioning the full training set into an initial pool of labelled data and an unlabelled pool of features . We finetune the parameters of a ImageNet-1-pretrained VGG16 on this initial dataset . For each training batch size of 64 , we optimize for cross-entropy loss using Adam ( Kingma & Ba , 2015 ) under default hyperparameters from PyTorch ( Paszke et al. , 2019 ) and a learning rate of 0.01 for CIFAR10/100 and 0.005 for SVHN . We then use either random acquisition , the original greedy core-set algorithm , or variations of Algorithm 2 with the trained model to produce a selection mask over the unlabelled data . We enforce that the number of selected elements is equal to the labelling budget per iteration . The selected features and their labels join the training set , the model retrains with a re-initialized optimizer , and the process is repeated until the number of the labelled data reaches the specified ceiling for the experiment . Note that we do not compare with the other baselines used in the original core-set experiments . Since the original core-set algorithm improved significantly from those baselines , we expect improvement over the original core-set algorithm to imply similar or greater improvement as well . Table 1 shows the iterations that we found were necessary to roughly meet zero training error on the initial dataset ( “ First-pass ” ) and all additions to the dataset per active learning iteration ( “ Thereafter ” ) . Note that validation error is not required to satisfy the convergence requirement of core-sets , so we ignore it in our experiments . For the ablation studies , we tune hyperparameters and conduct ablation studies on CIFAR10 ( Krizhevsky , 2009 ) using a budget of 400 labels per active learning iteration and an initial dataset size of 1000 samples . We use the same hyperparameters as the ablation studies in the main experiments on CIFAR10/100 ( Krizhevsky , 2009 ) and SVHN ( Netzer et al. , 2011 ) , which uses a budget of 5000 labels per iteration and a starting dataset size of 5000 samples .

Summary: In this paper, the authors propose to improve the vanilla greedy Core-set active learning algorithm by (1) weighting the distance with uncertainty (measured by doubt, $1-\max_yP(y|x)$) and (2) use beam search instead of greedy search where the beams are selected by average uncertainty. They show with several toy examples that this way the samples are concentrated closer to low-confidence region. They further try to find theoretical groundings for the advantage of the proposed algorithm with several assumptions. Finally they show that the proposed algorithm outperform vanilla coreset on CIFAR and  SVHN, and provide some ablation studies to showcase the effect of beam search and uncertainty weighting.