LADA: Look-Ahead Data Acquisition via Augmentation for Deep Active Learning

1 Introduction . Large-scale datasets in the big data era have opened the blooming of data science , but the data labeling requires significant efforts from human annotators , or oracle . Therefore , an adaptive sampling by an acquisition function , i.e . active learning , has been developed to select the most informative data instances in learning the decision boundary [ 1–3 ] . This selection is difficult because it is influenced by the learner and the dataset at the same time . Hence , the understanding of the relation between the two has become the components of active learning , which queries the next training example by the informativeness , assessed by acquisition function . Besides active learning , data augmentation is another data source for learning models that provides virtual data instances generated from the training dataset [ 4 ] . Conventional data augmentation has been a simple transformation of labeled data instances , e.g. , flipping , rotating , etc [ 5 ] . Recently , the data augmentation has expanded to become a deep neural model generating virtual instances , such as Generative Adversarial Networks ( GAN ) [ 6 ] or Variational Autoencoder ( VAE ) [ 7 ] . Spatial Transform Networks ( STN ) [ 8 ] also generate spatially transformed instances for learning the classifier . Since the conventional and the deep neural model-based augmentations perform the Vicinal Risk Minimization ( VRM ) [ 9 ] , they preserve labels of virtual instances and limit the feasible vicinity . To overcome the limited vicinity of VRM , Mixup [ 10 ] and its variants have been proposed by interpolating multiple data instances . The pair of interpolated features and labels , or the Mixup instances , become virtual instances to enlarge the support of the data distribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Given the scarce labeled dataset , it is natural to consider combining active learning and data augmentation . One possible way is a pipelined approach , which selects data instances by an acquisition function and generates virtual instances from the selected instances by an augmentation model afterward [ 11 ] . However , the acquisition function does not consider the potential gain from the augmentation in the assessment of the informativeness . Hence , without any feedback or integration effort at the acquisition level , the virtual instances generated by data augmentation would not guarantee the informativeness . Figure 1a illustrates the pipelined combination , where the averaged entropy value of the virtual instances is 1.61 . This paper proposes the Look-Ahead Data Acquisition via augmentation , or LADA framework . LADA looks ahead the effect of data augmentation in advance of the actual acquisition process , by selecting data instances according to the acquisition score of both unlabeled real instances and their augmented virtual instances , at the same time . The acquisition algorithm of LADA enables us to train the classifier with the instances that are informative 1 ) when labeled by oracle and 2 ) when augmented via data augmentation . Furthermore , the data augmentation policy in LADA is trained to maximize the acquisition score of the virtual instances . Figure 1b illustrates the different behavior of LADA with Max Entropy and Mixup , where the averaged entropy value of the virtual instances is 2.12 , which is higher than the pipelined combination in Figure 1a . Here are our contributions . First , we propose a generalized framework , named LADA , that looks ahead the acquisition score of the virtual data instances to be augmented , in advance of the acquisition . Second , we train the data augmentation policy to maximize the acquisition score , hence generate informative virtual instances . Particularly , we propose two data augmentation methods , InfoSTN and InfoMixup , which are trained by the feedback of acquisition scores . Third , we instantiate the proposed framework with various combinations of acquisition-augmentation of known methods . There have been some prior works that suggest the concept of look-ahead , without acknowledging the value of augmentation [ 12 , 3 , 13 ] . We claim our novelty for look-ahead in conjunction with the augmentation of virtual instances . Moreover , look-ahead is a necessary concept in any active learning scheme because the active learning requires an active seeking on high-value data instances which will impact the classifier if they are used in the inference , so this assessment on the impact becomes the look-ahead in such active learning concept . 2 Preliminaries . 2.1 Problem Formulation . This paper trains a classifier network parameterized by θ , i.e. , fθ , with dataset X while our scenario is differentiated by assuming X = XU ∪XL and |XU | |XL| . Here , XU is a set of unlabeled data instances , and XL is a set of labeled data instances . Given these notations , a data augmentation function , faug ( x ; τ ) : X → T ( X ) , transforms a data , x ∈X , into a modified data , x̃ ∈ T ( X ) ; where τ is a parameter describing the policy of transformation , and T ( X ) is the transformed set of X . On the other hand , a data acquisition function , facq ( x ; fθ ) : XU → R , calculates a score of each data instance , x ∈XU , based on the current classifier , fθ . facq provides the criteria of selection strategy in the learning procedure of fθ with the instance , x ∈ XU . We categorize the acquisition functions and the augmentation functions by placing the name of the algorithm as superscript . 2.2 Data Augmentation . In the conventional data augmentations , τ in faug ( x ; τ ) indicates the predefined degree of rotating , flipping , cropping , etc . τ is manually chosen to describe the vicinity of each data instance . Another approach of modeling τ is utilizing the feedback from the current classifier network , fθ . Spatial Transformer Network ( STN ) transforms a data using a grid sampler generated through a localisation network parameterized by τ , or fτ [ 8 ] . The augmentation policy , τ , in STN is trained by the cross-entropy ( CE ) loss of the transformed data with the ground-truth label , y , resulting in τ∗ = argminτ CE ( fθ ( f STN aug ( x ; τ ) ) , y ) . In a recent work , Mixup-based data augmentations generate a virtual data instance in between a pair of data instances to overcome the limited vicinity . In Mixup , τ becomes the mixing policy of two data instances , xi and xj , as fMixupaug ( xi , xj ; τ ) = λxi + ( 1− λ ) xj , λ ∼ Beta ( τ , τ ) , where the labels are also mixed by the proportion λ [ 10 ] . Further from mixing input features , Manifold Mixup mixes the hidden features from the multiple middle layers of neural networks to learn a smoother decision boundary [ 14 ] . Whereas τ is a fixed value without learning process so far , AdaMixup learns τ by adopting a discriminator , ϕada , as τ∗ = argmaxτ [ log P ( ϕ ada ( fMixupaug ( xi , xj ; τ ) ) = 1 ) + log P ( ϕada ( xi ) = 0 ) + log P ( ϕada ( xj ) = 0 ) ] [ 15 ] . 2.3 Active Learning . We focus on the pool-based active learning with an uncertainty score [ 3 ] . Given this scope of active learning , the data acquisition function measures the utility score of the unlabeled data instances , i.e . x∗U = argmaxxU∈XU facq ( x ; fθ ) . The traditional acquisition functions measure the predictive entropy , fEntacq ( xU ; fθ ) = H [ ŷ|xU ; fθ ] [ 16 ] , where H : = − ∑ c P ( ŷ = c|xU ; fθ ) log2 P ( ŷ = c|xU ; fθ ) and ŷ = fθ ( xU ) ; or the variation ratio , fV aracq ( xU ; fθ ) = 1 − maxŷP ( ŷ|xU ; fθ ) [ 17 ] . Bayesian approaches for active learning are also proposed as fBALDacq ( xU ; fθ ) = H [ ŷ|xU ; fθ ] − EP ( θ|Dtrain ) [ H [ ŷ|xU ; fθ ] ] [ 18 ] . Additional modules are also applied to measure the acquisition score . Variational Adversarial Active Learning ( VAAL ) introduces a discriminator , ϕV AAL , to estimate the probability of xU belonging to XU , as fV AALacq ( xU ; ϕ V AAL ) = P ( xU ∈XU ; ϕV AAL ) [ 19 ] . Learning Loss for Active Learning ( LL4AL ) adopts a simple neural network called a loss prediction module , or fLPM [ 20 ] . fLPM is trained to predict the loss of each data , and the instance with the highest predictive loss is selected , as fLL4ALacq ( xU ; fLPM ) = fLPM ( f k θ ( xU ) |k ∈ K ) . Here , fkθ ( xU ) represents the kth hidden representation of xU , and K represents the set of hidden layers of the classifier , fθ . 2.4 Active Learning with Data Augmentation . There have been prior researches on leveraging data augmentation for active learning . Bayesian Generative Active Deep Learning ( BGADL ) combines acquisition and augmentation in a pipelined approach [ 11 ] ; BGADL selects data instances via facq , and BGADL augments the selected instances via faug , which is VAE-ACGAN . However , BGADL limits the vicinity to preserve the label validity . Also , a large number of labeled instances are demanded to train the generative model , VAE-ACGAN , of BGADL at every acquisition round . More importantly , BGADL does not consider the potential gain from data augmentation in the process of acquisition . In comparison with BGADL , Consistency-based Active Learning ( CAL ) algorithms consider data augmentation in the acquisition process , by replacing the uncertainty with augmentation-based inconsistency , resulting in fCALacq ( x ; fθ ) = D [ P ( Ŷ |x , fθ ) , P ( Ŷ |x̃ , fθ ) ] [ 21 ] . Here , D denotes the L2 norm [ 22 ] or KL divergence [ 23 ] that represents the inconsistency of the predictions from the transformation of the data instance x into x̃ . The algorithm in [ 21 ] selects a data instance that has the highest variance of class-wise predictions when it is transformed over a random set of data augmentations . However , the augmentation in [ 21 ] is not learnable , i.e. , not optimized to enhance the informativeness of the augmented instance . Also , the augmentation is restricted to label-preserving transformations , such as random cropping or horizontal flipping , to measure the dissimilarity in predictions when the input is perturbed with the perceptual content of the instance being preserved . 3 Methodology . This paper differentiates itself from the previous acquisition-augmentation integration by presenting the learnable augmentations in conjunction with the potential acquisition scores of the virtual instances . Therefore , we start by formulating such a learnable framework in Section 3.1 . Afterward , we propose an integrated function for acquisition and augmentation as the implementation of the framework in Section 3.2 and Section 3.3 . Particularly , we propose adaptive versions of augmentation , i.e. , InfoSTN and InfoMixup , whose policies are learned by the feedback of data acquisition score , i.e . Max Entropy . It should be noted that InfoMixup , as well as InfoSTN , can adopt various types of acquisition functions , other than Max Entropy , for feedback to train , see Appendix D and Section 4 .

Summary: The paper proposes to combine data augmentation with active learning to improve the performance of Active Learning approaches. Instead of a pipelined approach, where the augmentation is applied on top of the data selected by the acquisition function, the paper aims to involve data augmentation in the data acquisition step too. Data augmentation policy is optimized to increase the acquisition function score, and the resulting augmented data is then used in the search process.