SABAL: Sparse Approximation-based Batch Active Learning

1 INTRODUCTION . Over the last decade , deep neural networks have achieved promising results in various learning tasks . However , obtaining labels for a complex training dataset can be challenging in practice , as the data annotation is usually a time-consuming process that may require professional knowledge in certain applications such as in medicine ( Hoi et al. , 2006 ; Shen et al. , 2021 ) . Active Learning ( AL ) ( Settles , 2009 ) is commonly employed to mitigate the problem of scarce labeled data – enabling efficient model training with limited annotation costs . Given a partially labeled dataset , active learning ideally selects data samples that are the best for learning . Specifically , it aims to iteratively query the most helpful data to ask an oracle ( human annotator ) to annotate . The queried data samples can be added back to the labeled data pool , and the model is updated . This process is repeated until the model has achieved the desired performance . Intelligently identifying the most valuable data for annotation , also known as the query strategy , is the key problem in active learning . A common strategy is to take the prediction uncertainty or data representation as the metric for data query . This uncertainty-based approach ( Settles , 2009 ; Tong & Koller , 2001 ; Gal et al. , 2017 ; Beluch et al. , 2018 ) works by querying samples with high uncertainty , but often results in selecting correlated and redundant data samples in each batch ( Kirsch et al. , 2019 ; Ducoffe & Precioso , 2018 ) . Representation-based approaches ( Sener & Savarese , 2017 ; Yang & Loog , 2019 ) aim to select a subset of data that represents the whole unlabeled dataset , but tend to be computationally expensive and sensitive to batch sizes ( Ash et al. , 2019 ; Shui et al. , 2020 ) . More recently , several hybrid approaches that try to take both uncertainty and representation into consideration have shown advantages ( Ash et al. , 2019 ; Shui et al. , 2020 ; Sinha et al. , 2019 ) . This paper takes this hybrid view towards an active learning framework that balances the trade-off between uncertainty and representation . Besides hybrid approaches , deep Bayesian active learning has also gained attention due to recent advances in Bayesian deep learning . Several Bayesian approaches ( Gal et al. , 2017 ; Kirsch et al. , 2019 ) leverage model uncertainty measurements ( Gal & Ghahramani , 2015 ; 2016 ) determined by Bayesian neural networks , while other works ( Pinsler et al. , 2019 ) leverage progress in Bayesian Coreset problems ( Zhang et al. , 2021 ; Huggins et al. , 2016 ; Campbell & Broderick , 2019 ) . However , as most existing Bayesian approaches are explicitly designed for Bayesian neural networks , another goal of this paper is to propose a general method for both Bayesian and non-Bayesian models . For deep models , it is reasonable to query a large batch of data simultaneously to reduce model update frequency . The batch selection approach is known as batch active learning . Taking an optimization perspective , finding the best batch is NP-hard in general . Two common approaches for such combinatorial problems are the greedy and clustering approaches . Greedy algorithms select one data sample in sequence until the batch budget is exhausted ( Kirsch et al. , 2019 ; Bıyık et al. , 2019 ; Chen & Krause , 2013 ) . Here , specific conditions of the acquisition function such as submodularity ( Nemhauser et al. , 1978 ) are required to guarantee a good optimization result . Clustering algorithms regard cluster centers as their queried batch ( Sener & Savarese , 2017 ; Ash et al. , 2019 ) , but can be computationally expensive . To our knowledge , except for Pinsler et al . ( 2019 ) that focus on the Bayesian models , so far active learning has rarely been studied from a sparse approximation perspective . This is despite the ubiquity of sparse approximation in signal processing for tasks such as dictionary learning ( Aharon et al. , 2006 ) and compressed sensing ( Donoho , 2006 ) due to its performance for discovering a sparse representation while avoiding redundant information . Here we employ sparse approximation methods for batch active learning tasks . Our main contributions are summarized in the following . We propose a novel and flexible Sparse Approximation-based Batch Active Learning framework , i.e. , SABAL . We show how SABAL generalizes batch active learning as a sparse approximation problem and can adapt to different settings and models . The central intuition of SABAL is finding a weighted subset from the unlabeled data pool so that its corresponding training loss approximates the full-set loss function in a function space . We realize the SABAL framework as an efficient finite-dimensional optimization problem : First , we derive an upper bound to balance the trade-off between uncertainty and representation in a principled way . Second , we approximate the loss functions using finite-dimensional approximation . This results in a sparsity-constrained discontinuous optimization problem , for which we propose several efficient optimization algorithms . We demonstrate the advantages of SABAL in experiments for both Bayesian and non-Bayesian batch active learning settings . The structure of this manuscript is as follows . In Section 2 , we formulate the general framework of SABAL , and in Section 3 , we realize the framework into a finite-dimensional discontinuous sparse optimization problem . To solve the resulting optimization problem , we propose two optimization algorithms in Section 4 . Related work are discussed in Section 5 and Appendix Section B . Results of our experiments are presented in section 6 , and all proofs are provided in Appendix Section C . 2 BATCH ACTIVE LEARNING AS SPARSE APPROXIMATION . This section introduces the preliminaries and the general formulation of batch active learning as a sparse approximation problem . Preliminaries Vectors are denoted as bold lower case letters , e.g. , w ∈ Rn . The l0 pseudonorm of a vector w is denoted as ‖w‖0 , i.e. , the number of non-zero elements of w. We denote R+ : = [ 0 , +∞ ) . Distributions are denoted in script , e.g. , P , and a random variable is denoted by tilde , e.g. , ỹ ∼P . We denote sets in calligraphy or in uppercase Greek alphabet ( e.g. , D , Θ ) , and additionally we denote [ n ] : = { 1 , 2 , . . . , n } . In supervised learning , given a labeled training dataset Dl : = { ( xi , yi ) } nli=1 , where we denote their domain to be x ∈ X and y ∈ Y , the empirical goal is to minimize a loss function Ll ( θ ) : = ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) formed by the training dataset , where θ ∈ Θ ⊂ Rm is the parameter of the model and ` is a loss function evaluated on individual pairs of data . Without loss of generality , we assume Θ ⊂ Rm is compact and ` ( x , y ; · ) : Θ→ R is in a normed space ( L ( Θ , R ) , ‖ · ‖† ) for all x , y . We further assume the constant function f : Θ→ 1 is included in L ( Θ , R ) . The “ † ” in the norm ‖ · ‖† : L ( Θ , R ) → R+ , representing its definition is a placeholder that will be discussed later . Batch Active Learning Besides the labeled dataset Dl , there is an unlabeled dataset Du : = { xj } nuj=1 where the labels are unknown but could be acquired at a high cost through human labeling . Combining two datasets , the ideal loss function to minimize w.r.t . θ is∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈Du ` ( xj , y ? j ; θ ) , ( 1 ) where y ? j is the unknown true label corresponding to the data xj . Since acquiring true labels could be costly , we have to impose a budget b ( b < nu ) on the number of label acquisitions . Therefore , the batch active learning problem is to find a subset S ⊂ Du such that we can obtain a good model by optimizing the following loss function w.r.t . θ , ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈S ` ( xj , y ? j ; θ ) , where |S| = b . ( 2 ) Generalized Batch Active Learning We start our method by generalizing the classical formulation ( equation 2 ) by considering an importance weight for each unlabeled data . That is , we aim to find a sparse non-negative vector w ∈ Rnu+ such that we can obtain a good model by optimizing the following loss function w.r.t . θ : ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈Du wj ` ( xj , y ? j ; θ ) , where ‖w‖0 = b . ( 3 ) A key question now is—what is the criterion for a good w ? Comparing the ideal loss function ( equation 1 ) and the sparse importance weighted loss ( equation 3 ) , the only difference is their unlabeled data loss functions . Therefore , a straight-forward informal criterion for a good importance weight w is that the two unlabeled data loss functions are close to each other , i.e. , L ? w ( θ ) : = 1 b ∑ xj∈Du wj ` ( xj , y ? j ; θ ) ≈ L ? ( θ ) : = 1 nu ∑ xj∈Du ` ( xj , y ? j ; θ ) . However , as the true labels are unknown , we can not compute L ? w and L ? . Luckily , we can have an estimator for the true labels , i.e. , estimation based on the labeled data p ( ỹj | xj , Dl ) or an approximation of it . Denote P ( xj ) as an estimated distribution , so ỹj ∼P ( xj ) , then the informal criterion for a good importance weight w then becomes L̃w ( θ ) : = 1 b ∑ xj∈Du wj ` ( xj , ỹj ; θ ) ≈ L̃ ( θ ) : = 1 nu ∑ xj∈Du ` ( xj , ỹj ; θ ) . ( 4 ) Thus , we are one step closer to evaluating the quality of a weighted selection . The next question is how to measure the difference between L̃ and L̃w . Difference Between Two Loss Functions Given the two loss functions L̃ , L̃w ∈ L ( Θ , R ) , where L ( Θ , R ) is equipped with the norm ‖ · ‖† , a straight-forward measurement of the difference between them is ‖L̃− L̃w‖† . However , observing that the optimization of a loss function is shift-invariant , the difference between two loss functions should also be shift-invariant . For example , for ∀L ∈ L ( Θ , R ) we have arg minθ∈Θ ( L ( θ ) + c ) = arg minθ∈Θ L ( θ ) for ∀c ∈ R , implying that L + c should be treated the same as L. Therefore , to account for the shift-invariance , we define q : L ( Θ , R ) → R+ as q ( L ) : = inf c∈R ‖L+ c‖† , ∀L ∈ L ( Θ , R ) . ( 5 ) Note that we abuse the notation a bit , i.e. , the c in L+ c should be the constant function that maps every θ ∈ Θ to c. The above definition has some nice properties that make it a good difference measurement of two loss functions , as proved in proposition C.1 in the appendix . In particular , q ( · ) satisfies the triangle inequality , and q ( L+ c ) = q ( L ) for any constant c. Therefore , we can formulate the generalized batch active learning problem as the following sparse approximation problem . Problem 1 ( Sparse Approximation-based Batch Active Learning ) . Given the shift-invariant seminorm q induced by the norm ‖ · ‖† ( equation 5 ) , and a label estimation distribution P , the generalized batch active learning problem ( equation 4 ) is formally defined as arg min w∈Rnu+ EP [ q ( L̃− L̃w ) ] s.t . ‖w‖0 = b , ( 6 ) where EP stands for the expectation over ỹj ∼P ( xj ) for ∀j ∈ [ nu ] . Problem 1 ( SABAL ) offers a general framework for batch active learning and can be applied with various settings , i.e. , both the norm ‖ · ‖† and the individual loss function ` can be chosen based on specific problems and applications . In the next section , we introduce two practical realizations of equation 6 for Bayesian and non-Bayesian active learning respectively .

Summary: This paper provides a framework which attempts to find a weighted subset of the unlabeled pool that matches the loss on all the points. This paper introduces a seminorm on the loss functions and proposes an optimization method to minimize this. Somewhat favorable results are shown for vision datasets.