SABAL: Sparse Approximation-based Batch Active Learning

1 INTRODUCTION . Over the last decade , deep neural networks have achieved promising results in various learning tasks . However , obtaining labels for a complex training dataset can be challenging in practice , as the data annotation is usually a time-consuming process that may require professional knowledge in certain applications such as in medicine ( Hoi et al. , 2006 ; Shen et al. , 2021 ) . Active Learning ( AL ) ( Settles , 2009 ) is commonly employed to mitigate the problem of scarce labeled data – enabling efficient model training with limited annotation costs . Given a partially labeled dataset , active learning ideally selects data samples that are the best for learning . Specifically , it aims to iteratively query the most helpful data to ask an oracle ( human annotator ) to annotate . The queried data samples can be added back to the labeled data pool , and the model is updated . This process is repeated until the model has achieved the desired performance . Intelligently identifying the most valuable data for annotation , also known as the query strategy , is the key problem in active learning . A common strategy is to take the prediction uncertainty or data representation as the metric for data query . This uncertainty-based approach ( Settles , 2009 ; Tong & Koller , 2001 ; Gal et al. , 2017 ; Beluch et al. , 2018 ) works by querying samples with high uncertainty , but often results in selecting correlated and redundant data samples in each batch ( Kirsch et al. , 2019 ; Ducoffe & Precioso , 2018 ) . Representation-based approaches ( Sener & Savarese , 2017 ; Yang & Loog , 2019 ) aim to select a subset of data that represents the whole unlabeled dataset , but tend to be computationally expensive and sensitive to batch sizes ( Ash et al. , 2019 ; Shui et al. , 2020 ) . More recently , several hybrid approaches that try to take both uncertainty and representation into consideration have shown advantages ( Ash et al. , 2019 ; Shui et al. , 2020 ; Sinha et al. , 2019 ) . This paper takes this hybrid view towards an active learning framework that balances the trade-off between uncertainty and representation . Besides hybrid approaches , deep Bayesian active learning has also gained attention due to recent advances in Bayesian deep learning . Several Bayesian approaches ( Gal et al. , 2017 ; Kirsch et al. , 2019 ) leverage model uncertainty measurements ( Gal & Ghahramani , 2015 ; 2016 ) determined by Bayesian neural networks , while other works ( Pinsler et al. , 2019 ) leverage progress in Bayesian Coreset problems ( Zhang et al. , 2021 ; Huggins et al. , 2016 ; Campbell & Broderick , 2019 ) . However , as most existing Bayesian approaches are explicitly designed for Bayesian neural networks , another goal of this paper is to propose a general method for both Bayesian and non-Bayesian models . For deep models , it is reasonable to query a large batch of data simultaneously to reduce model update frequency . The batch selection approach is known as batch active learning . Taking an optimization perspective , finding the best batch is NP-hard in general . Two common approaches for such combinatorial problems are the greedy and clustering approaches . Greedy algorithms select one data sample in sequence until the batch budget is exhausted ( Kirsch et al. , 2019 ; Bıyık et al. , 2019 ; Chen & Krause , 2013 ) . Here , specific conditions of the acquisition function such as submodularity ( Nemhauser et al. , 1978 ) are required to guarantee a good optimization result . Clustering algorithms regard cluster centers as their queried batch ( Sener & Savarese , 2017 ; Ash et al. , 2019 ) , but can be computationally expensive . To our knowledge , except for Pinsler et al . ( 2019 ) that focus on the Bayesian models , so far active learning has rarely been studied from a sparse approximation perspective . This is despite the ubiquity of sparse approximation in signal processing for tasks such as dictionary learning ( Aharon et al. , 2006 ) and compressed sensing ( Donoho , 2006 ) due to its performance for discovering a sparse representation while avoiding redundant information . Here we employ sparse approximation methods for batch active learning tasks . Our main contributions are summarized in the following . We propose a novel and flexible Sparse Approximation-based Batch Active Learning framework , i.e. , SABAL . We show how SABAL generalizes batch active learning as a sparse approximation problem and can adapt to different settings and models . The central intuition of SABAL is finding a weighted subset from the unlabeled data pool so that its corresponding training loss approximates the full-set loss function in a function space . We realize the SABAL framework as an efficient finite-dimensional optimization problem : First , we derive an upper bound to balance the trade-off between uncertainty and representation in a principled way . Second , we approximate the loss functions using finite-dimensional approximation . This results in a sparsity-constrained discontinuous optimization problem , for which we propose several efficient optimization algorithms . We demonstrate the advantages of SABAL in experiments for both Bayesian and non-Bayesian batch active learning settings . The structure of this manuscript is as follows . In Section 2 , we formulate the general framework of SABAL , and in Section 3 , we realize the framework into a finite-dimensional discontinuous sparse optimization problem . To solve the resulting optimization problem , we propose two optimization algorithms in Section 4 . Related work are discussed in Section 5 and Appendix Section B . Results of our experiments are presented in section 6 , and all proofs are provided in Appendix Section C . 2 BATCH ACTIVE LEARNING AS SPARSE APPROXIMATION . This section introduces the preliminaries and the general formulation of batch active learning as a sparse approximation problem . Preliminaries Vectors are denoted as bold lower case letters , e.g. , w ∈ Rn . The l0 pseudonorm of a vector w is denoted as ‖w‖0 , i.e. , the number of non-zero elements of w. We denote R+ : = [ 0 , +∞ ) . Distributions are denoted in script , e.g. , P , and a random variable is denoted by tilde , e.g. , ỹ ∼P . We denote sets in calligraphy or in uppercase Greek alphabet ( e.g. , D , Θ ) , and additionally we denote [ n ] : = { 1 , 2 , . . . , n } . In supervised learning , given a labeled training dataset Dl : = { ( xi , yi ) } nli=1 , where we denote their domain to be x ∈ X and y ∈ Y , the empirical goal is to minimize a loss function Ll ( θ ) : = ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) formed by the training dataset , where θ ∈ Θ ⊂ Rm is the parameter of the model and ` is a loss function evaluated on individual pairs of data . Without loss of generality , we assume Θ ⊂ Rm is compact and ` ( x , y ; · ) : Θ→ R is in a normed space ( L ( Θ , R ) , ‖ · ‖† ) for all x , y . We further assume the constant function f : Θ→ 1 is included in L ( Θ , R ) . The “ † ” in the norm ‖ · ‖† : L ( Θ , R ) → R+ , representing its definition is a placeholder that will be discussed later . Batch Active Learning Besides the labeled dataset Dl , there is an unlabeled dataset Du : = { xj } nuj=1 where the labels are unknown but could be acquired at a high cost through human labeling . Combining two datasets , the ideal loss function to minimize w.r.t . θ is∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈Du ` ( xj , y ? j ; θ ) , ( 1 ) where y ? j is the unknown true label corresponding to the data xj . Since acquiring true labels could be costly , we have to impose a budget b ( b < nu ) on the number of label acquisitions . Therefore , the batch active learning problem is to find a subset S ⊂ Du such that we can obtain a good model by optimizing the following loss function w.r.t . θ , ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈S ` ( xj , y ? j ; θ ) , where |S| = b . ( 2 ) Generalized Batch Active Learning We start our method by generalizing the classical formulation ( equation 2 ) by considering an importance weight for each unlabeled data . That is , we aim to find a sparse non-negative vector w ∈ Rnu+ such that we can obtain a good model by optimizing the following loss function w.r.t . θ : ∑ ( xi , yi ) ∈Dl ` ( xi , yi ; θ ) + ∑ xj∈Du wj ` ( xj , y ? j ; θ ) , where ‖w‖0 = b . ( 3 ) A key question now is—what is the criterion for a good w ? Comparing the ideal loss function ( equation 1 ) and the sparse importance weighted loss ( equation 3 ) , the only difference is their unlabeled data loss functions . Therefore , a straight-forward informal criterion for a good importance weight w is that the two unlabeled data loss functions are close to each other , i.e. , L ? w ( θ ) : = 1 b ∑ xj∈Du wj ` ( xj , y ? j ; θ ) ≈ L ? ( θ ) : = 1 nu ∑ xj∈Du ` ( xj , y ? j ; θ ) . However , as the true labels are unknown , we can not compute L ? w and L ? . Luckily , we can have an estimator for the true labels , i.e. , estimation based on the labeled data p ( ỹj | xj , Dl ) or an approximation of it . Denote P ( xj ) as an estimated distribution , so ỹj ∼P ( xj ) , then the informal criterion for a good importance weight w then becomes L̃w ( θ ) : = 1 b ∑ xj∈Du wj ` ( xj , ỹj ; θ ) ≈ L̃ ( θ ) : = 1 nu ∑ xj∈Du ` ( xj , ỹj ; θ ) . ( 4 ) Thus , we are one step closer to evaluating the quality of a weighted selection . The next question is how to measure the difference between L̃ and L̃w . Difference Between Two Loss Functions Given the two loss functions L̃ , L̃w ∈ L ( Θ , R ) , where L ( Θ , R ) is equipped with the norm ‖ · ‖† , a straight-forward measurement of the difference between them is ‖L̃− L̃w‖† . However , observing that the optimization of a loss function is shift-invariant , the difference between two loss functions should also be shift-invariant . For example , for ∀L ∈ L ( Θ , R ) we have arg minθ∈Θ ( L ( θ ) + c ) = arg minθ∈Θ L ( θ ) for ∀c ∈ R , implying that L + c should be treated the same as L. Therefore , to account for the shift-invariance , we define q : L ( Θ , R ) → R+ as q ( L ) : = inf c∈R ‖L+ c‖† , ∀L ∈ L ( Θ , R ) . ( 5 ) Note that we abuse the notation a bit , i.e. , the c in L+ c should be the constant function that maps every θ ∈ Θ to c. The above definition has some nice properties that make it a good difference measurement of two loss functions , as proved in proposition C.1 in the appendix . In particular , q ( · ) satisfies the triangle inequality , and q ( L+ c ) = q ( L ) for any constant c. Therefore , we can formulate the generalized batch active learning problem as the following sparse approximation problem . Problem 1 ( Sparse Approximation-based Batch Active Learning ) . Given the shift-invariant seminorm q induced by the norm ‖ · ‖† ( equation 5 ) , and a label estimation distribution P , the generalized batch active learning problem ( equation 4 ) is formally defined as arg min w∈Rnu+ EP [ q ( L̃− L̃w ) ] s.t . ‖w‖0 = b , ( 6 ) where EP stands for the expectation over ỹj ∼P ( xj ) for ∀j ∈ [ nu ] . Problem 1 ( SABAL ) offers a general framework for batch active learning and can be applied with various settings , i.e. , both the norm ‖ · ‖† and the individual loss function ` can be chosen based on specific problems and applications . In the next section , we introduce two practical realizations of equation 6 for Bayesian and non-Bayesian active learning respectively .

Summary: The paper proposes SABAL as a framework to formulate batch active learning as a sparse approximation problem. The paper considers the SABAL framework as a finite-dimensional optimization problem, efficiently solvable by the proposed greedy or proximal IHT algorithms.   Numerical experiments demonstrate the strong performance of SABAL, comparable to the state-of-the-art with lower time complexity.  Since directly optimising the original objective function is intractable, SABAL proposes to optimise (minimise) the upper bound which is obtained by the triangle inequality including the (i) variance and (ii) approximation bias.