Deep Active Learning with Noise Stability

1 INTRODUCTION . The success of training a deep neural network highly depends on a huge amount of labeled data . Nevertheless , in the era of big data , much data is unlabeled . This imposes a challenge on exploiting appropriate learning fashions . For instance , given a pool of unlabeled data , one can employ unsupervised manner to learn a model . However , the performance of unsupervised or semi-supervised learning is still bound to that of supervised learning ( Yoo & Kweon , 2019 ) . An ideal option could be annotating all the given data , so as to use the supervised way to learn a model . Nevertheless , the cost for data annotation could be extremely high , such as in medical area ( Gorriz et al. , 2017 ; Konyushkova et al. , 2017 ) . Due to the limited labeling budget , it may be more feasible to annotate a portion of the given data . This is also aligned with the prerequisite of semi-supervised learning . Then one can ask : which data is deserved for being annotated ? Active learning was proposed to solve this question . One of the main ideas , namely uncertaintybased method , aims to select a portion of the most uncertain or informative data from an unlabeled pool for annotation . The newly annotated data are then used to train a task model . Note that uncertainty estimation with deep neural networks ( DNNs ) remains challenging , due to the potential over-confidence of deep models . That is , the primitive softmax output1 as a score does not necessarily reflect a reliable uncertainty or confidence for the prediction . Several methods have been proposed to address this challenge . For example , MC-Dropout ( Gal & Ghahramani , 2016 ) implements a Single-Training Multi-Inference algorithm by aggregating multiple confidences sampled by Monte Carlo dropout . The algorithm has been shown to be equivalent to the approximate Bayesian inference . Another typical idea is query-by-committee ( Freund et al. , 1997 ; Gorriz et al. , 2017 ) , or QBC in short , which requires training multiple auxiliary models as a committee . Besides , many other efforts suggest to use specially designed auxiliary modules or training fashions , such as variational auto-encoder ( Sinha et al. , 2019 ) , adversarial learning ( Ducoffe & Precioso , 2018 ; Mayer & Timofte , 2020 ) and Graph Convolutional Network ( Caramalau et al. , 2021 ) . In this work , we study the problem of deep active learning from a different viewpoint , inspired by a relevant topic of noise stability . Noise stability , in terms of the output stability w.r.t . the input 1Considering classification problems noise , is well studied in literature . For example , the authors in ( Bishop , 1995 ) propose to train a model with perturbed input samples as a regularizer to improve the model performance . A recent theoretical work ( Arora et al. , 2018 ) shows that , the stability of each layer ’ s computation to noise injected at lower layers acts as a good indicator of the generalization bounds for DNNs . Behind these achievements is an intuitive implication that , a model ( or more exactly , its parameters ) robust to perturbed input examples tends to be easy to recognize unseen examples . For the active learning problem , we give an analogous intuition by exchanging the role of the input and parameter that , an example robust to the perturbed current parameters tends to be easy to be recognized by a future model . In other words , examples less robust to the parameter perturbation would be regarded as having higher uncertainty . Specifically , we introduce a simple algorithm of uncertainty estimation by measuring how far does the output deviate from the original value , when imposing a small noise on the parameters . The induced distance is used as a measurement of uncertainty used to select unlabeled examples in active learning . We provide a theoretical analysis under the condition that the noise magnitude is small . By imposing standard multivariate Gaussian noise to the parameter , we prove that the proposed noise stability is equivalent to the parameter-output Jacobian norm under the first order Taylor assumption . Furthermore , we show that , our algorithm has a close connection with variance reduction ( Cohn et al. , 1996 ) . That is , selecting unlabeled samples with low noise stability would yield the same effect of reducing the prediction variance of existing training samples . Our method is easy to implement and free of customized auxiliary models . Therefore , it can be exploited in various tasks , such as image classification and semantic segmentation , leading to its task-agnostic nature . We conduct extensive experiments to evaluate our method on various datasets including Cifar10 ( Krizhevsky et al. , 2009 ) , Cifar100 ( Krizhevsky et al. , 2009 ) , SVHN ( Netzer et al. , 2011 ) , Caltech101 ( Fei-Fei et al. , 2006 ) , Cityscapes ( Cordts et al. , 2016 ) , and cryo-ET ( Chen et al. , 2017 ) . The performance of our method significantly exceeds that of the state-of-the-art . The contributions of this work are summarized as follows . 1 . We propose a novel effective method of noise stability to select unlabeled data for active learning . The proposed method is free of any auxiliary models or special training fashions . 2 . We provide a theoretical analysis to show that , selecting unlabeled data with higher noise stability is equivalent to selecting that with higher Jacobian norm w.r.t . the parameter . We also establish a connection between noise stability and variance reduction for the existing training set . 3 . We conduct extensive experiments on benchmark datasets of image classification , semantic segmentation , and 3D cryo-ET subtomogram classification . The results demonstrate the effectiveness of the proposed method . 2 ACTIVE LEARNING WITH NOISE STABILITY . 2.1 PROBLEM DEFINITION . Here we formulate the active learning problem . Given a pool of unlabeled data { XU } , and a labeled pool { XL , YL } which is initially empty . Active learning aims to select a portion of data from { XU } depending on labeling budget ( e.g . select 2500 samples out of 50000 at a time ) . The selected data XN is then annotated by human oracles ( or equivalent ) , and added to the labeled pool . That is , { XL , YL } ← { XL , YL } + { XN , YN } , where YN is the new annotation for XN . The unlabeled pool is then updated by removing the selected data : { XU } ← { XU } − { XN } . { XL , YL } is then used to train a task model f ( . ; θ ) in supervised fashion by minimizing an empirical loss L ( e.g . crossentropy loss for classification ) , where f ( . ; θ ) denotes a neural network f ( . ) parameterized with θ . Note that in this paper , 2.2 UNCERTAINTY ESTIMATION WITH NOISE STABILITY . We use the noise stability to estimate uncertainty for unlabeled data . Specifically , for an input sample x , we explore to what degree will the model ’ s output deviate from the original observation f ( x ; θ ) , when adding random noise to the model parameter θ . Let ξ∆θ denote the added noise , Algorithm 1 : Active learning with noise stability as uncertainty estimation . Input : T : random initialized task model , U : unlabeled pool of training data , L : labeled pool of training data , C : number of cycles in active learning ; Output : θ : Final learned parameter of T ; 1 begin 2 for i← 1 to C do 3 train T with L , obtaining the current parameter θ ; 4 for k ← 1 to K do 5 sample ∆θ ∼ N ( 0 , σ2I ) ; 6 create a perturbed model T ′ parameterized with θ + ξ∆θ ; 7 for every x in U do 8 calculate R ( k ) ( x ) = ‖T ( x ) − T ′ ( x ) ‖2 ; 9 compute every R ( x ) by averaging all R ( k ) ( x ) ; 10 select samples in U with top N largest R ( x ) as { XN } , and obtain their labels { YN } ; 11 update L with L = L ⋃ { XN , YN } ; 12 update U with U = U \ { XN } ; 13 return θ ; where ξ controls the magnitude of the noise . If ∆θ conforms to the standard multivariate Gaussian distribution2 N ( 0 , σ2I ) , the criterion to quantify the degree of noise instability can be formulated as R ( x ) = E ∆θ∼N ( 0 , σ2I ) ‖f ( x ; θ ) − f ( x ; θ + ξ∆θ ) ‖2 , ( 1 ) where ‖.‖2 is the L2 norm of a vector . In active learning , we select samples with largestR ( x ) for annotation . To approximate the expectation , we perform a Monto-carlo sampling from the multivariate Guassian distribution to generate multiple ∆θ ( i ) . Then we get the approximated noise stability by R ( x ) = 1 K K∑ i=1 ‖f ( x ; θ ) − f ( x ; θ + ξ ( i ) ∆θ ( i ) ) ‖2 . ( 2 ) The noise magnitude should be small relative to that of the original parameter ( e.g . ξ ( i ) = 10−3 ‖θ‖‖∆θ ( i ) ‖ ) in order to avoid catastrophic perturbation to the clean model . Note that , we use a single ξ ( i ) for the entire ∆θ ( i ) , i.e . each element in ∆θ ( i ) is equally re-scaled by ξ ( i ) . As for the sampling number , we find the setting K = 5 or 10 works well in practice . A complete procedure of our method is presented in Algorithm 1 . 3 THEORETICAL UNDERSTANDINGS . In this section , we provide theoretical understandings about our simple method for uncertainty estimation . Our main conclusions are summarized as follows . • When the noise magnitude ξ is sufficiently small , selecting data according toR ( x ) in Eq ( 1 ) is equivalent to selecting data by the Frobenius norm of parameter-output Jacobian w.r.t . f . • Based on the above conclusion , selecting new samples with higher R ( x ) are expected to contribute more in reducing the prediction variance of existing training samples . 3.1 NOISE STABILITY AS JACOBIAN NORM . Let f ( x ; θ ) be differentiable ( w.r.t . θ ) at the point θ given x as the input . When the imposed noise ξ∆θ in Eq ( 1 ) has a sufficiently small magnitude , we use the first-order Taylor expansion to estimate f ( x ; θ + ξ∆θ ) as f ( x ; θ + ξ∆θ ) ≈ f ( x ; θ ) + Jθ ( x ; θ ) ξ∆θ , ( 3 ) 2See Appendix A.1 for preliminaries about the standard multivariate Gaussian distribution where Jθ ( x ; θ ) is the Jacobian matrix of f with respect to the parameter θ as Jθ ( x ; θ ) ( i , j ) = ∂f ( x ; θ ) ( i ) /∂θ ( j ) . By substituting the Taylor approximation into Eq ( 1 ) , the uncertainty estimation can be reformulated as R ( x ) = E ∆θ∼N ( 0 , σ2I ) ‖Jθ ( x ; θ ) ξ∆θ‖2 . ( 4 ) Since the elements of ∆θ are independent with each other , it ’ s easy to derive that the expected uncertainty in Eq ( 4 ) is in proportion with the Frobenius norm of Jacobian as R ( x ) = ξ2σ2‖Jθ ( x ; θ ) ‖2F . ( 5 ) In fact , Eq ( 5 ) stands for any appropriate noise distribution with the above mentioned independence prerequisite and zero mean . We put a detailed proof of Eq ( 5 ) in Appendix A.2 .

Summary: This paper provides a new method for solving the active learning problem, i.e. choosing from a set of unlabeled examples that should be labelled and added to the training set. The proposed method makes use of 'noise stability' of inputs–specifically if the model output corresponding to an input changes to a large extent when small amounts of noise are added to the model parameters, then this input should be labelled. The authors compare their method to a range of existing AL methods and draw theoretical connections to predictive variance reduction.