Deep Active Learning with Noise Stability
Authors: Xingjian Li
Pengkun Yang
Tianyang Wang
Min Xu
Dejing Dou
Cheng-zhong Xu
Venue: ICLR 2022 Submitted (2022)
Keywords: deep learning
active learning
noise stability

Abstract: Uncertainty estimation for unlabeled data is crucial to active learning. With a deep neural network employed as the backbone model, the data selection process is highly challenged due to the potential over-confidence of the model inference. Existing methods usually resort to multi-pass model training or adversarial training to handle this challenge, resulting in complex and inefficient pipelines, which prevent the deployment in practice. To address such an issue, in this work we propose a novel Single-Training Multi-Inference algorithm that leverages noise stability to estimate data uncertainty. Specifically, it is measured by to what degree the output deviates from the original observation when the model parameters are randomly perturbed by noise. We provide theoretical analysis of using small Gaussian noise, showing that our method has a solid connection with the classical theory of variance reduction, i.e. labelling a data sample of higher uncertainty, indicated by the inverse noise stability, contributes more to reducing the variance of existing training samples. Despite its simplicity and efficiency, our method outperforms the state-of-the-art active learning baselines in image classification and semantic segmentation tasks.

Under review as a conference paper at ICLR 2022
DEEP ACTIVE LEARNING WITH NOISE STABILITY
Anonymous authors
Paper under double-blind review
ABSTRACT
Uncertainty estimation for unlabeled data crucial to active learning. With a deep neural network employed as the backhone model, the data selection pro- cess is highly challenged due to the potential over-confidence of the model infer- ence. Existing methods usually resort to multi-pass model training ad adversarial training to handle this challenge. This tends to result in complex and inefficient pipelines, which would render the methods impractical. In this work, we propose a novel Single-Training estimate data uncertainty. A key idea behind is magnitude measurement of output derivation from the original observation when the model parameters are randomly perturbed by noise. We provide analysis by using the small Gaussian noise theory and show that our method has a strong connection with the classical theory of variance reduction. That is, by labelling a data sample of higher un- certainty, as indicated by the inverse noise stability, our method contributes more to reducing the variance of existing training samples. Despite its simplicity, our method outperforms the state-of-the-art active learning baselines in image classi- fication and semantic segmentation tasks.
Multi-Inference algorithm that leverages noise stability
INTRODUCTION
The success of training a deep neural network highly depends on a huge amount of labeled data. Nevertheless, in the era of big data, much data is unlabeled. This imposes a challenge on exploiting appropriate learning fashions. For instance, given a pool of unlabeled data, one can employ unsu- pervised manner to learn model. However, the performance of unsupervised or semi-supervised learning is still bound to that of supervised learning YYo & Kweon, 2019). An ideal option could be annotating all the given data, the cost for data annotation could be extremely high, such as in medical area (Gorriz et al.) 2017]] Konyushkova et al.) 2017). Due to the limited labeling budget, it may be more feasible annotate a portion of the given data. This is also aligned with the prerequisite of semi-supervised learning. Then one can ask: which data deserved for being annotated?
to the supervised way to learn model. Nevertheless,
Active learning was proposed to solve this question. One of the main ideas, namely uncertainty- based method, aims to select a portion of the most uncertain or informative data from an unlabeled pool for annotation. The newly annotated data are then used to train task model. Note that un- certainty estimation with deep neural networks (DNNs) remains challenging, due to the potential over-confidence of deep models. That is, the primitive softmax outpu[" as a score does not nec- essarily reflect a reliable uncertainty or confidence for the prediction. Several methods have been proposed to address this challenge. For example, MC-Dropout (Gal & Ghahramani] 2016) imple- ments a Single-Training Multi-Infereeee algorithm by aggregating multiple confidences sampled by Monte Carlo dropout. The algorithm has been shown to be equivalent to the approximate Bayesian inference. Another typical idea is QBC in short, which requires training multiple auxiliary models committee. Besides, many other efforts suggest to use specially designed auxiliary modules or training fashions, such as varia- tional auto-encoder (Sinha et al. 2019), adversarial learning (Ducoffe & Precioso 2018 Mayer & Timofte 2020) and Graph Convolutional Network (Caramalau et al. 2021).
(Freund et al. 1997 Gorriz et al., 2017).
In this work, we study the problem of deep active learning from different viewpoint, inspired by a relevant topic of noise stability. Noise stability, in terms of the output stability w.r.t. the input
CConidering classification problems

---

Under review as a conference paper at ICLR 2022
noise, is well studied in literature. For example, the authors in (Bishop 1995) propose to train model with perturbed input samples as a regularizer to improve the model performance. A recent theoretical work (Arora et al. 2018) shows that, the stability of each layer's computation to noise injected at lower layers acts as a good indicator of the generalization bounds for DNNs. Behind these achievements is intuitive implication that, model (or more exactly, its parameters) robust to perturbed input examples tends to be easy to recognize unseen examples. For the active learning problem, we give an analogous intuition by exchanging the role of the input and parameter that, an example robust to the perturbed current parameters tends to be easy to be recognized by a future model. In other words, examples less robust to the parameter perturbation would be regarded as having higher uncertainty.
Specifically, we introduce simple algorithm of uncertainty estimation by measuring how far does the output deviate from the original value, when imposing small noise on the parameters. The induced distance is used as a measurement of uncertainty used to select unlabeled examples in active learning. We provide theoretical analysis under the condition that the noise magnitude small. By imposing standard multivariate Gaussian noise to the parameter, we prove that the proposed noise stability is equivalent to the parameter-outply Jacobian norm under the first order Taylor assumption. Furthermore, we show that, our algorithm has a close connection with variance reduction (Cohn et al., 1996). That is, selecting unlabeled samples with low noise stability would yield the same effect of reducing the prediction variance of existing training samples.
Our method is easy to implement and free of customized auxiliary models. Therefore, it can be exploited in various tasks, such as image classification and semantic segmentation, leading to its task-agnostic nature. We conduct extensive experiments evaluate our method various datasets including Cifarl0 (Krizhevsky et al. (209). Cifar100 (Krizhevsky et al.] 2009). SVHN (Netzer et al. 2011, Caltech101 (Fei-Fei al., (2066), Cityscapes (Cordts et al. (0016). and cryo-ET (Chen al. The performance of our method significantly exceeds that of the state-of-the-an
2017
The contributions of this work are summarized as follows.
1. We propose novel effective method of noise stability select unlabeled data for active learning. The proposed method is free of any auxiliary models or special training fashions.
2. We provide a theoretical analysis to show that, selecting unlabeled data with higher noise stability is equivalent selecting that with higher Jacobian norm w.r.t. the parameter. We also establish a connection between noise stability and variance reduction for the existing training set.
3. We conduct extensive experiments benchmark datasets of image classification, semantic segmentation, and 3D cryo-ET subtomogram classification. The results demonstrate the effectiveness of the proposed method.
ACTIVE LEARNING WITH NOISE STABILITY
2.1 PROBLEM DEFINITION
Here we formulate the active learning problem. Given pool of unlabeled data (Xt: J. and labeled pool (Xt, Y2) which initially empty. Active learning aims to select portion of data from (Xy) depending labeling budget (e.g. select 2500 samples out of 50000
time). The selected data
XNy is then annotated by human oracles (or equivalent), and added to the labeled pool. That is, {Xu, Y2) (Xz, Yz} {XN, ).), where YN is the new annotation for XN- The unlabeled pool is then updated by removing the selected data: {Xe} } + {Xe) {Xx]- {XL, Yz} is then used train task model f(:; 0) in supervised fashion by minimizing an empirical loss L (e.g. cross- entropy loss for classification), where ((; () denotes a neural network f(.) parameterized with 0.
Note that in this paper.
2.2 UNCERTAINTY ESTIMATION WITH NOISE STABILITY
We use the noise stability to estimate uncertainty for unlabeled data. Specifically, for an input sample x, we explore to what degree will the model's output deviate from the original observation (a:; f), when adding random noise to the model parameter 0. Let {AN denote the added noise,
2

---

Under review as a conference paper at ICLR 2022
Algorithm 1:
Active learning with noise stability as uncertainty estimation.
Input : T: random initialized task model, Ut unlabeled pool of training data, labeled pool of training data, C: number of cycles in active learning;
Output: Final learned parameter of begin
forit: to C do train T with L. obtaining the current parameter 0;
for A < 1 to do sample A create perturbed model T' parameterized with 0 + E20;
I²I); oN(
for every æ in 24 do
calculate (x) [T(x)) (x)))
8
compute every R(x) by averaging all (xx):
10
select samples in U/ with top N largest R(xx) as {. and obtain their labels {Yn):
update C with = C NN, update U with 2 (XN]:
11
12
13
return 0:
where & controls the magnitude of the noise. If 2 conforms to the standard multivariate Gaussian distributior (0,, I)), the criterion to quantify the degree of noise instability can be formulated as
R(x)) 10~N(0.021)
(a 0) (ax 0 <~
(1)
where |1-1| is the 2 norm of vector. In active learning, we select samples with largest R(x) for an- notation. approximate the expectation, we perform Monto-carlo sampling from the multivariate Guassian distribution to generate multiple A0(), Then we get the approximated noise stability by
R(x) Σ [f(x f(x; 0 5(5) 11
The noise magnitude should be small relative that of the original parameter (e.g. g(i) 10 [ll|| |
in order to avoid catastrophic perturbation to the clean model. Note that, we use a single (1) for the entire A i.e. each element in ( is equally re-scaled by g()) As for the sampling number, find the setting K = 5 or 10 works well in practice. A complete procedure of our method is presented in Algorithm
THEORETICAL UNDERSTANDINGS
In this section, we provide theoretical understandings about our simple method for uncertainty esti- mation. Our main conclusions are summarized as follows.
When the noise magnitude Ç is sufficiently small, selecting data according to R(x) in Eq 1 is equivalent to selecting data by the Frobenius norm parameter-output Jacobian w.r.t. J.
Based on the above conclusion, selecting new samples with higher R(x) are expected to contribute more in reducing the prediction variance of existing training samples.
3.1 NOISE STABILITY AS JACOBIAN NORM
Let (æ: 0) be differentiable (w.rtt. 0 at the point 0 given æ as the input. When the imposed noise (A0 in Eq (1 has sufficiently small magnitude, we use the first-order Taylor expansion to estimate f( 0 + (AO) as
f(x; 0 E10) f(æ; 0) Jo 0)
"ee Appendix A.1 for preliminaries about the standard multivariate Gaussian distribution

---

Under review as a conference paper at ICLR 2022
where (z; 0) is the Jacobian matrix of f with respect to the parameter @ as Jo( 8 (i.j) (æ;; ()) By substituting the Taylor approximation into Eq (T). the uncertainty estimation can be reformulated as
I R(x) 3e~V(..021) |J 0)52012
(4)
Since the elements of A are independent with each other, it's easy to derive that the expected uncertainty in Eq (4) is in proportion with the Frobenius norm Jacobian as
R(z) = E o² [o;
(5)
In fact, Eq 5 stands for any appropriate noise distribution with the above mentioned independence prerequisite and zero mean. We put detailed proof of Eq 5 in Appendix A.2
3.2 CONNECTIONS WITH PREDICTION VARIANCE REDUCTION
We next show an interesting connection between uncertainty estimation with noise stability and existing theoretical literature of variance reduction. For convenience, we assume that ğ f(: 0) is a real-valued function parameterized with that results in ÿ R - IR, and the results are easy generalize vector-valued functions. Given a labeled training set L 9: we aim at minimizing the Mean Squared Error as
S
(6)
1=1
Then, we define by [z(x)) the Jacobian w.r.t. the parameter @ at æ, i.e. Oj(z)/00. Let Aga de- note the Fisher Information Matrix that Ag² 5 52055/000 Note that we omit the notation of in J and A to without ambiguity. According to the well-established theory of variance estima- tion (Thisted 1988), the output variance at the input point x can represented as (z) (a( 0)* A07A5511 Jo 0)-
With the normality and local linearity assumptions, the authors in (MacKay 1992 Cohn 1996 further quantify the change of output variance. Specifically, when a new sample æ is labeled and added to the training set, the expected output variance at
az(x)) R oz(x)) 52 02
(8)
where (z, @) defined as
(g a) Jo ax; A A (a;
Therefore, by adding new sample i. the reduction of output variance xz) is:
(z)))
Ag)(2)
S2 + oz(
|| (z; 0) T Jo 0)
(10)
S Jol 0 Jo
|Jo(;; 07
3)
(1) 0112
52
u || (a; 0)
where we use u()) with the unit length to denote the direction of ( as u()) Ja():0)
[
Eq (10b implies that, for any direction u the optimal choice for maximizing A is the ones with the largest [JJo (;; I|? Though the objective is also affected by the direction u @)), searching for an optimal direction is usually inefficient due to the characteristic of almost orthogonality of high-dimensional independent vectors (Vershynin) 2018).
Active learning with the objective of maximizing variance reduction has been successfully validated for shallow models such as mixture of Gaussians and kernel regression (Cohn et al. 1996). However, computing the inversion of A for each new sample is too cumbersome for deep neural networks. Our work establishes a strong connection between variance reduction and noise stability, and renders the algorithm implementable easily.
4

---

Under review as a conference paper at ICLR 2022
Performance comparison Citar10
Parforaance Comparison Citar100
Mean Accuracy (%)
Mesn Accuracy
69
88
53
86
82
DBC
SAKA
Sandom
VMAL
1PAAL
ConnGet
- GCM
40
10
15
20
30
35
% of Labeled Data
% of Labeled Dala
Performance Comparison SVHN
Performannc Comparison Caltech
95.5
Mean Accuracy (86)
Mean Accuracy (96)
96.0
96.5
80
94.0
63.5
Karetuni
LIAAL WAAL
Tunsam
ILAAL
MCCCrppu
93.0
65
10
15
20
10
10
15
20
30
35
Labeled
of Labeled
Figure I: Classification performance due to different active learning methods on Cifar10 (top left), Ci (top right), SVHN (bottom left), and Caltech101 (bottom right).
EXPERIMENTAL RESULTS
We compare our method with the state-of-the-art active learning methods. Furthermore, since ran- dom selection is the most straightforward strategy for data selection, we also include it as baseline in this paper. We first validate our method on classical image classification and semantic segmen- tation tasks. Then we investigate its performance on simulated cryo-ET data (see Appendix [A.4.I). demonstrating its competency for domain tasks. For all the tasks, the labeling budget varies from active learning cycles.
10% 40% corresponding will be selected for annotation. For the first cycle, we randomly select and annotate 10% of the data from the unlabeled pool and use the selected data as initial training data for all the compared methods. Each reported result is based three trials to obtain reliable evaluation. We conduct all the experiments using Pytorch (Paszke et al. 2017) and the source code will be publicly available.
each cycle, 5% of all the training data
4.1 IMAGE CLASSIFICATION
Datasets. We adopt four benchmark classification datasets for evaluations, including Cifar10 (Krzhevsky et al., 2009 C0far100 (Krzhevsky al. 2009). SVHN (Netzer al. 2011). and Cal- tech101 (Fei-Fei al. (006). Each Cifar dataset contains 50000 training and 10000 testing samples, uniformly distributed across 10 and 100 classes, respectively. SVHN includes 73257 training and 26032 testing samples, and do not use the additional training data in SVHN following the stan- dard settings. There are 8677 images of higher resolution (c.g. 300 200 pixels) in the Caltech dataset. We follow the common practice to resize the images 256 x 256, and center-crop the images within a 224 224 region. Both SVHN and Caltech101 are typical imbalanced datasets. In the latter, for example, the number of samples in class varies from 40 to 800.
Model and Training We use ResNet-18 He et al. (2016) as the task model all the classification experiments. Specifically, follow the practice in Caramalau al. (2021): Zhang al. 2020 to
5

---

Under review as a conference paper at ICLR 2022
adopt special version of ResNet-18 et al. (2066) as the task model to make the model compat- ible with the smaller image resolution (i.e. 32 x 32) in 10far10/100 and SVHN. For Caltech we use the original ResNet-18. Since active learning algorithm is generally independent of network architectures, we use the same task model for all the compared methods in each experiment. For instance, 10, the original CoreSet (Sener & Savarese (2018) and VAAL|Sinha al.) (2019) employ VGG-16 as the task model which will be replaced by ResNet-18 in our reproduction.
In each cycle of the Cifar and SVHN experiments, we use a batch size of 128 samples train the task model with an initial learning rate of 0.1. The training continues for 200 epochs and we decay the learning rate with 0.1 at the 160 epoch. In Caltech 101, the training will continue for 50 epochs and we choose 0.01 as the initial learning rate, which will be decayed by 0.1 at the 40th epoch. Due to the higher image resolution, a smaller batch size of 64 is used to avoid GPU memory issues. In order to make the evaluations not impacted by different optimization routines, we employ the same optimizer (i.e. SGD (Bottou 2010)) for all the experiments of image classification. A momentum of 0.9 and a weight decay rate of 5 x 10 are used with the optimizer.
Observations. We compare our method with the state-of-the-art baselines and illustrate the results in Figure [ The baseline methods include random selection, MC-Dropout (Gal & Ghahramani
2016). QBC (Kuo et al. 2018). CoreSet (Sener & Savarese 2018). VAAL (Sinha et al., 2019).
LL4AL (Yoo & Kweon 2019), SRAAL (Zhang et al. 2020) and GCN (Caramalau et al., 1021
as suggested in their original papers. For the proposed noise stability, we default hyper- parameters in all experiments: 0² 1, K 10 and = = 10-3
For all these algorithms, we use the recommended
|@]|| to ensure the relative noise magnitude constrained. Aell
As can be seen in Figure I the proposed method (NoiseStabillit) consistently outperforms the base- lines (in terms of accuracy) in each active learning cycle. In partieular, for harder datasets such as Cifar10, Cifar100 and Caltech, our method achieves remarkably higher final accuracy against the others. For the easier task SVHN, our method yields more rapid and stable increase of accuracy the first several cycles, more competitive than the baselines. In addition to the general comparison, we present a detailed analysis below to demonstrate more merits of our method.
In the later cycles (e.g. 35% and 40%), our method usually exceeds the others by larger margin (e.g. in Cifar10 and Caltech101). This indicates that when the task model re- members more samples (i.e. the trained model is more likely to behave over-confident), estimating uncertainty by our noise stability is much more accurate than other competitors.
The superior performance in the Caltech101 experiment suggests that method is more suitable for imbalanced datasets than the existing baselines. We conjecture this is due to the simplicity of our method, since training helper models (Kuo al.) 2018 Sinha et al.) 2019 Zhang et al. 2020 Caramalau et al. (02T) with imbalanced data could be adverse to these helpers themselves, thus leading to an inferior performance of the task model.
Lastly, we observe that in the Cifar10 experiment, the performance of our method (94.25%) exceeds that (93.6%) of the same model which is trained on the entire dataset without considering active learning. This counter-intuitive observation indicates that training only with a subset of hard samples may make DNNs generalize better than training with an entire set.
4.2 SEMANTIC SEGMENTATION
Semantic segmentation is more complex than image classification due to the requirement of high classification accuracy at pixel level. As consequence, in several baseline methods, such as VAAL Sinha et al. (2019), SRAAL Zhang et al. (2020) and GCN Caramalau et al. (202T), the auxiliary models need careful design in order to achieve an satisfactory performance. This increases the burden of deploying Al algorithms in practice. In contrast, method is free of extra design and can be easily adopted for this problem, demonstrating its task-agnostic advantage.
Dataset. We the Cityscapes Cordts et al. (016)) large-scale street scene dataset. It includes images taken under various weather conditions in different seasons from 50 cities, making the task on more challenging. We consider the standard training and validation data of this set. We crop

---

Under review as a conference paper at ICLR 2022
Performance Comparison on Cityscapes
60
Mean Accuracy (%)
58
56
54
MC Dropout
VAAL
OBC
SRAAL
Coreše:
NosuStabilty
50
10
15
20
30
Labeled Data
Figure 2: Classification performance due to different active learning methods on the semantic seg- mentation task Cityscapes.
the images to a dimension of 688 x 688, and there are categories (classes) total for each pixel. It is worth noting that the cropping size also depends on the task model, discussed in next section.
Model and Training. As a common practice Zhang et al. (2020), we choose a dilated residual network (DRN-D-22) (Yu et al. 2017) the task model. It includes 8 layer modules and each module is composed 2017) for the network details.
or BasicBlock of layers. One can refer to the original paper (Yu et al.
Similar to the classification evaluations, we train the model for 7 cycles and 50 epochs for each cycle. Following the suggestion in (Yu et al. 2017),
the Adam optimizer (Kingma & Ba 2014). We choose a learning rate of x 10- remained unchanged in the entire training. For the proposed noise stability, we K 3 and keep the remaining classification experiments.
the same specified in the
Observations. We compare our method with MC-Dropout (Gal & Ghahramani 2016), QBC (Kuo) et al. 2018), CoreSet (Sener & Savarese 2018). VAAL (Sinha et al. 2019), SRAAL (Zhang et al. 2020 this experiment. We illustrate the results in Figure 2 We observe that the improvement of the proposed method is solid. Given Cityscapes is highy imbalanced, this experiment once again demonstrates the superiority of our method on imbalanced datasets. More importantly, from classifi- cation to segmentation, our method can easily adapt to the new task by simply sampling and adding noise to the output (without being affected by the change of the output format).
5 DISCUSSIONS
In this section, we provide further analysis on the effect of the proposed method. First in Section 5. we discuss the choice of hyper-parameter used in noise stability. Then we evaluate our algorithm with fewer cycles and larger selection size Section 5.2 Morcover, in Appendix [A.7] we provide an empirical validation for the theoretical hypothesis that selecting samples by noise stability is conducive to reducing the output variance for the training set.
5.1 SENSITIVITY TO HYPER-PARAMETERS
We first investigate how our algorithm is sensitive to the hyper- parameters (i.e. the noise magnitude, the noise distribution and the sampling times). We conduct experiments on Cifar10 and Cifar 100 for this analysis.
Noise magnitude. Specifically, noise with too large magnitudes tends to destroy the learned knowledge in DNNs. For
The criterion of noise stability Eq is sensitive to the noise magnitude.
simplicity, we denote by X the relative (to the clean parameter) magnitude, i.e. & 1B00T We
evaluate the choices of (10-5 10-4, 10-3, 10-2, 10-1, 1} for A. Results in Figure 3 (left) show that

---

Under review as a conference paper at ICLR 2022
Actuvyy of Wieeetiii morhisss
Accusvy of offewwr surping firve
Accuvey dffiww sein divns
Mepntcoree 10
Gausdan Unf
Ladkace
1055
Te-1
1e-3
7e-7
160
30
50
20
Samp
Figure 3: Evaluations of different typper-parameters Cifar100: noise magnitude A (left), sampling times K' (middle) and noise distribution (right). In the left and middle plots, we use dots with different colors denote different cycles, e.g. the blue dots at the bottom refer cycle I and the pink dots at the top refer to cycle 7.
using À between 10-4 and 10 10-2 is likely to yield good performance. From the various experiments of this work, we observe that default choice of 10-3 would be suitable in general.
Sampling times. Accurate approximations such as that for Eq ( usually require a considerable sampling times. However, we surprisingly find that the noise stability scheme used in our algorithm is very insensitive to the sampling times K. We evaluate observe very similar results, shown in Figure 3 (middle). We conjecture this might be because the uncertainty estimation in active learning does not require an accurate approximation of the exact value, even nor the exact order. The main factor that impacts the final results is only the selected subset of data.
with choices of (1,3, 10, 30, 50} and
Noise distribution. Note that our method is not limited the multivariate Gaussian distribution. Any distribution with all the elements independent of each other satisfies the theoretical properties. Here we evaluate two alternative noise distributions, which are the Uniform distribution A0; U(-1,1) 1) and the zero-mean Laplace distribution A0; ~ Laplace(0, 6) with = 1. Results in Figure (right) show that both of them achieve similar good performance as the normal distribution.
5.2 PERFORMANCE ON LARGER SELECTION SIZE
remains a long-term open problem that whether data uncertainty and diversity are empirically consistent or in a trade-off relationship. Instead of directly addressing this dilemma, we design an experiment to investigate how our algorithm performs with an increased selection size") As the training pool is enlarged by adding large amount of samples, both uncertainty and diversity tend to be crucial to further training, e.g. highly imbalanced (lack of diversity) data is definitely harmful to learning. For efficiency, we perform a single cycle of sample selection after cycle 1. We different algorithms to select unlabeled samples with the labeling budgets of 10%, 15% 20%, 25%, 35% and 40% respectively. Note that is diffeerent from the standard experiments presented Figure [ since the sample selection here is one-off rather than iterative. Results in Figure 4 demonstrate that our method still consistently outperforms the SOTA baseline GCN Caramalau et al. 2021).
RELATED WORK
Active learning has been long-term open research topic in the community. Most of the methods can be categorized into three types including uncertainty- based, diversity-based and synthesis-bass methods. In this section we provide a brief review of these typical studies.
Uncertainty-based. Traditional methods solve the problem mainly via estimating uncertainty for unlabeled data. Some typical works (Joshi et al. |009 Lewis & Catleti 1994, Lewis & Gale
1994
Roth & Small 2006 Tong & Koller 2001 Vijayanarasimhan Grauman| [014 Li Guo
2014)
have been proposed for shallow or statistical models. Nevertheless, as the rapid development of deep learning, new challenge has emerged. That is, how to estimate data uncertainty with deep neural
*Seeectoon size this section refers the number of the unlaheled samples selected in cycle.
8

---

Under review as a conference paper at ICLR 2022
Performance Comparison Crar10
Parforaanc Comparison Citar100
Mean Mean Accuracy (16
Mean Accuracy (%5
60
66
53
04
mon-itobbie
40
10
15
20
25
30
10
15
20
30
35
40
of Labeled Data
% of Labeled Data
Figure 4: Evaluation of using larger selection size, denoted by non-iterative as only performs an one-off selection. Results with the standard setting (denoted as iterative), copied from Figure I are also shown for comparison. Experiments are conducted on Cifar10 (left) and C0 (right).
networks (DNNs). Several methods (Sinha et al.) 2019; Yoo & Kweon 2019 Zhang et al. 2020
Mackowiak et al. 2018) have been proposed to address this challenge with the aid of specialized DNNs and training fashions, such as VAE (variational auto-encoder) (Kingma & Wellingl 2013) and adversarial learning. [Sener & Savarese (2018); Kuo et al. (2018) attempt to combine a classical algorithm (e.g. K-center or 0-1 Knapsack problem) with DNNs select the most uncertain data for annotation. Note that our method belongs to this category.
In terms of methodology, the most similar existing work ours is MC-Dropout (Gal & Ghahramani 2016). However, the motivation is quite different in that we adopt noise stability to estimate uncer- lainty, whereas MC-Dropout resorts Bayesian inference approximate the uncertainty (measured in the output entropy). Another essential difference lies in that, noise stability only needs a local sam- pling that generates parameter noise around specific solution, whereas Bayesian inference requires global sampling over the entire hypothesis space. As consequence, the necessary sampling times of MC-Dropout (e.g. ~ 100 or ~ 1000) orders magnitude larger than noise stability. Our work is also relevant to recent studies on linearised Laplace approximation of the BNN posterior (Immer ct al. 2021 Daxberger et al. 2021). We put a more detailed discussion in Appendix A.8
Diversity-based. In addition to the uncertainty-basee methods, several works attempt select unlabeled data that can diversify the labeled pool. This category of methods are also known as issribution-based (Bilgic& & Getoor 2009 Elhamifar et al.] [013 Freytag et al. [2014) Hasan & Roy-
Mac Aodha et al. [2014 Nguuen & Smeulders 20044 Roy & McCallum) 2000 [000| traditionally treated as essentially different from the uncertainty-basee methods, it is worth noting that a recent
Chowdhury] 2013
Yang al. 2015 Settls al. (2008.. Although the diversity-based methods
study (Loquercio al. 2020) argues that data diversity correlated data uncertainty.
Synthesis-based. Another typical category of methods aim to diversify the labeled pool by syn- thesizing new data that can benefit the task model training. Typical methods (Zhu & Bento 20177 Mayer & Timofte| 2020) resort to generative models, such as GAN (Good- fellow et al., 2014) VAE (Kingma & Welling 2013) synthesize diverse data samples train the task model.
Mahapatra et al., 2018)
CONCLUSION
This paper proposes a simple method that utilizes noise stability for uncertainty estimation in active learning. For an input sample, noise stability measures how stable can the output be when ran- dom perturbation is added on the model parameters. Our theoretical analysis prove that the noise stability approximates Jacobian norm w.r.t. the parameters assuming the noise magnitude is small. Theoretical and empirical analysis also demonstrate that, selecting those samples mostly unstable to parameter noise has the effect of reducing the prediction variance of the training samples. Experi- ments on image classification, semantic segmentation, and 3D cryo-ET subtomogram classification demonstrate the efficacy of the proposed method.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Sanjeev Arora, R. Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via a compression approach. ArXiv, hww/1882.052966 2018.
Mustafa Bilgic and Lise Getoor. Link-based active learning. In NIPS Workshop on Analyzing Networks and Learning with Graphs, 2009.
Chris M Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7(1):108-11 116, 1995.
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of the COMPSIAT, pp. 177-186. Springer, 2010.
Razvan Caramalau, Binod Bhattarai, and Tae- Kyen Kim. Sequential graph convolutional network for active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern June 2021.
Recognition (CVPR), pp.
Muyuan Chen, Wei Dai, Stella Y Sun, Darius Jonasch, Cynthia Y He, Michael F Schmid, Wah Chiu, and Steven Ludtke. Convolutional neural networks for automated annotation cellular cryo-electron tomograms. Nature Methods, 14(10):983, 2017.
Kashyap Chitta, Jose M Alvarez, Elmar Haussmann, and Clement Farabet. Training data subset search with ensemble active learning. arXiv preprint arXiv: 12737, 2019.
D. A Cohn, Z Ghahramani, and M. Jordan. Active learning with statistical models. The Journal of artificial intelligence research, 4: 129-1444, 1996. ISSN 1076-9757.
David Cohn. Neural network exploration using optimal experiment design. Neural networks, (6):1071-1083, 1996. ISSN 0893-6080.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Erik Daxberger, Eric Nalisnick, James Allingham, Javier Antorán, and José Miguel Hernández- Lobato. Bayesian deep learning via subnetwork inference. In International Conference Ma- chine Learning, pp.
PMLR, 2021.
Dean De Cock. Ames, iowa: Alternative to the boston housing data as an end of semester regression project. Journal of Statistics Education, 19(3), 2011.
Jun Deng, Wei Dong, Richard Socher, Li-Jia Li, Kuntai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. CVPR, pp. 248-255, 2009.
Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: a margin based approach. arXiv Preprint 100.00180..09841,, 2018
Ehsan Elhamifar, Guillermo Sapiro. Allen Yang, and S Shankar Sasrty. convex optimization framework for active learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 209-216, 2013.
Li Fei-Fei, Rob Fergus, and Pietro Perona. One-shot learning of object categories. IEEE TPAMI, 28 (4):594-611, 2006.
Yoav Freund, H. Sebastian Seung, Eli Shamir, and Naftali Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3):133-168, 1997.
Alexander Freytag, Erik Rodner, and Joachim Denzler. Selecting influential examples: Active learn- ing with expected model output changes. In Proceedings of the European Conference Com- puter Vision, pp. 562-577. 2014.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, 2016.
10

---

Under review as a conference paper at ICLR 2022
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
Mare Gorriz, Axel Carlier, Emmanuel Faure, and Xavier Giro-i Nieto. Cost-effective active learning for melanoma segmentation. arXiv Preprint
2017.
I Gubins, G van der Shot, RC Veltkamp, F Foerster, X Du, X Zeng, et al. SHREC' 19 track: Classi- fication in cryo-electron tomograms. In Eurographics Workshop on 3D Object Retrieval, 2019.
Mahmudul Hasan and Amit K Roy-Chowaae noy-Chowddury. Context aware active learning of activity recog- nition models. In Proceedings of the IEEE International Conference Computer Vision, pp- 4543-4551,2011
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: / //2.5745, 2011.
Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, and Mohammad Emtiyaz Khan. Scalable marginal likelihood estimation for model selection in deep learning. arXiv preprint arXiv:2104.04975. 2021.
Ajay J Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recogni- tion, pp. 2372-2372 2009.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv Preprint arXiv: 1412.6940,,,00, 2014.
Diederik P Kingma and Max Welling. arXiv:1312.61 14, 2013.
Auto-encoding variational bayes.
arXiv Preprint
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in neural information processing systems, 32:7026-7037, 2019.
Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua. Learning active learning from data. In Proceedings of the Advances in Neural Information Processing Systems, pp. 4225-4235, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.
2009.
Weicheng Kuo, Christian Häne, Esther Yuh, Pratik Mukherjee, and Jitendra Malik. Cost-sensitive active learning for intracranial hemorrhage detection. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018.
David D Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Machine learning proceedings, pp. 148-156. Elsevier, 1994.
David D Lewis and William A Gale. A sequential algorithm for training text classifiers, In SIGIR, 1994.
Xin Li and Yuhong Guo. Multi-level adaptive active learning for scene classification. In Proceedings of the European Conference on Computer Vision, pp. 234-249, 2014.
Antonio Loquercio, Mattia Segu, and Davide Searamuzza. A general framework for uncertainty estimation in deep learning. IEFE Robotics and Automation Letters, 5(2):3153-311 2020.
Oisin Mac Aodha, Neill DF Campbell, Jan Kautz, and Gabriel J Brostow. Hierarchical subquery evaluation for active learning on graph. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 564-571, 2014.

---

Under review as a conference paper at ICLR 2022
David J. C MMcKaay Information- based objective functions for active data selection. Neural com- putation, 4(4):590-60)) 604, 1992. ISSN 0899-7667.
Radek Mackowiak, Philip Lenz, Omair Ghori, Ferran Diego, Oliver Lange, and Carsten Rother. Cereals-cost-effective region-based active learning for semantic segmentation. arXiv Preprint arXiv:/810.09726, 2018.
Dwarikanath Mahapatra, Behzad Bozorgtabar, Jean- Sean-Philippp Thiran, and Mauricio Reyes. Efficient active learning for image classification and segmentation using sample selection and conditional generative adversarial network. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018.
Christoph Mayer and Radu Timofte. Adversarial sampling for active learning. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 3071-3079, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011.
Hieu Nguyen and Aroold Smeulders. Active learning using pre-clustering. In International Con- ference Machine Learning, 2004.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Dan Roth and Kevin Small. Margin- based active learning for structured output spaces. In Proceed- ings of the European Conference on Machine Learning, pp. 413-424, 2006.
Nicholas Roy and Andrew McCallum. Toward optimal active learning through monte carlo estima- tion of error reduction. International Conference on Machine Learning, 2001.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018.
Burr Settles, Mark Craven, and Soumya Ray. Multiple-instance active learning. In Proceedings of the Advances in Neural Information Processing Systems, pp. 1289 1289-1296, 2008.
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5972-5981, 2019,
Ronald (Ronald Aaron) Thisted. Elements of statistical computing. Chapman and Hall, New York, 1988. ISBN 0412013711.
Simon Tong and Daphne Koller. Support vector machine active learning with applications text classification. Journal of Machine Learning Research, 2(Nov):45-66 66, 2001.
Roman Vershynin. High-ddmensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
Sudheendra Vijayanarasimhan and Kristen Grauman. Large-scale live active learning: Training object detectors with crawled data and crowds. International Journal of Computer Vision, 108 (1-2):97-114, 2014.
Yi Yang. Zhigang Ma, Feiping Nie. Xiaojun Chang, and Alexander G Hauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 13-, 2015.
Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 93-102, 2019.
Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. Dilated residual networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 472-480, 2017.
12

---

Under review as a conference paper at ICLR 2022
Beichen Zhang, Liang Li, Shijie Yang, Shuhui Wang, Zueng-Jun Zha, and Qingming Huang. State- relabeling adversarial active learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3756-8765. 2020.
Jia-Jie Zhu and José Bento. arXiv:1702.07956. 2017.
Generative adversarial active learning.
arXiv Preprint
A APPENDIX
A.1 STANDARD MULTIVARIATE GUASSIAN DISTRIBUTION
(M, E), is The univariate Gaussian distribution is often referred to N(u o²), where μ denotes the expecta- tion of the distribution and a is the standard deviation. NN o²) can be described by its probability
(x-w)
density function p(z) _
20
Multivariate Gaussian distribution
generalized form for the vector variable, where μ is the mean vector and Σ the covariance ma- trix. In this work, we shall focus on the standard case, in which is the zero vector 0 and Σ is diagonal matrix with all diagonal elements being o², i.c.
I Note that if æ NO I all the samplings { are independent of each other, and each conforms to the standard univariate Gaussian distribution as 24 ~N(00
A.2 PROOF OF EQ (5)
We first assume [-: 0) is real-valued function and the Jacobian is vector. If A0 Rd conforms to the standard multivariate Gaussian distribution that AO ~ (0, II),, each its elements are independent with each other. That is, for any i,j S d, the following true:
40,00 o'o.j.
(11)
20~
where 6új 1;=j. Without ambiguity, here use J as shorthand for Jo(x;; 0). The notation of Gaussian distribution under the expectation is also omitted for simplicity.
Proof. Based on Eq III we can expand the inner product between J and in Eq as
d
R(x) = =
Σ
J,
20
= =² E
LA + E
Σ
J,A0,J,00;
A0
i,j=1
ifj
d
(12)
J E
A0? + Σ
J.J,
E
40
(=1
2
Ji02
When [-: 0) is vector-valued, we can straightforwardly put each row of the Jacobian matrix into Eq 12 and finally get
R(x)= =(x)===0?11113.
(13)
Note that, the conclusion is true for any other zero-mean distribution long it satisfies the independence prerequisite and has a definite variance. The multivariate Gaussian distribution is used only as typical example.
13

---

Under review as a conference paper at ICLR 2022
A.3 MORE COMPARISONS WITH DEEP BAYESIAN ACTIVE LEARNING
In this section, we compare our method with more deep Bayesian active learning methods on MNIST and Cifar10. The deep Bayesian method is one of the most important schemes for uncertainty estimation. We evaluate MC-Dropout Gal & Ghahramani (2016), BALD Houlsby et al. (2011), BatchBALDD Kirsch et al. 2019) and Checkpoint Ensemble Chitta et al. (2019). For all the dropout based methods (MC-Dropout, BALD and BatchBAL.D), which can be regarded as a kind of implicit ensemble, we set the number of Monte-Carie sampling to 50. For Checkpoint Ensemble, we save the checkpoints for the last 10 epochs as the ensemble models. BALD (mutual information) is used as the acquisition function for the Ensemble method. For our method, we still sample noise for 10 times.
Model and Training, For Cifarl0, we use ResNet-18 with exactly the same settings as described in Section 4.1. For MNIST, we employ small convolutional network composed of two 5x5 convo- lutional layers, followed by two fully connected layers. We use the Adam optimizer with a learning rate of 0.001. For each AL cycle, train the model for 50 epochs the labeled pool. The initial labeled pool consists of 50 samples, and another 50 unlabeled samples are selected for annotation at each AL cycle.
Observations. We present the results in Figure 5 We observe that the basic deep Bayesian methods, such as MC-Dropout and BALD, perform better than Random selection on both datasets. The ad- vanced Bayesian method BatchBALD does not show superiority, compared with BALD on MNIST. On Cifar10 with a selection size of 2500, BatchBALD almost degenerates Random selection, and we conjecture this is due to the inaccurate joint entropy estimation. Though using a smaller sampling number, the Ensemble and our NoiseStability methods perform the best. Similar to our method, for MC-Dropout, BALD and BatchBALD, further increasing the number of Monte-Carlo samplings (e.g. an increment to 100) cannot bring any performance gain on the two datasets.
Performance Comparison
Performance Comparison on
95
94
(Mean Accuracy (%0
Mean Accuracy (%6
90
80-
86
Random
SALDAE 50i
Ranzon
BALD b1
Veloe by:k-11
BaschRALEae- 500
Kenedaabl Pyyk-100
100
1.50
250
350
400
450
19
20
Number
Data
Number of Labeled Data
Figure 5: Classification performance due to different Bayesian active learning methods on MNIST
(left) and Cifarl0 (right).
A.4 EXPERIMENTS ON ADDITIONAL SETTINGS
Here we present more experimental results to evaluate our algorithm.
A.4.1 1 EXPERIMENTS ON MORE DATA DOMAINS
In this part, we evaluate AL methods on the Cryo-ET Challenges. Cryo-Electron Tomography (cryo- ET) is an important imaging technique to study biological macromolecules and cells, used to pro- duce their high-resolution 3D tomograms of cellular landscapes and structures. Active learning is of great interest for such kind of tasks since the annotation cost is extremely high. We evaluate active learning algorithm on cryo-ET subtomogram classification (Chen et al. 2017), demonstrating the capability of our method in various domains.
Dataset. We compare our noise stability against the baseline methods on one simulated cryo-ET dataset consisting of subtomogram images contaminated by noise with SNR (signal-noise-ratio)
14

---

Under review as a conference paper at ICLR 2022
of 0.03. Given different conformational states, all the samples in this dataset are categorized into 50 classes, namely 50c-snr003. There are 24000 training and 1000 test samples, uniformly distributed across all the classes. One can refer to Gubins et al. (2019) for how to generate the data.
Model and Training. The dimension of the subtomogram data (i.c. 32 x 32 x 32) is different with that of RGB images, making the classification task more challenging. To adapt to this particular data dimension, we employ a customized ResNet-11 with 3D convolutional layers as the task model. In addition to the task model, in the baseline methods one also needs customize the auxiliary models (e.g. VAE in VAAL (Sinha et al., 2019) and SRAAL (Zhang et al. 2020), and the loss prediction module in LL4AL (Yoo & Kweon 2019)). whereas our method free of this extra implementation.
The SGD optimizer (Bottou 2010) is employed to train the task model. The training will continue for 100 epochs and the initial learning rate is set 0.1, and we decay it by 0.1 the 80th epoch. We choose momentum of 0.9 and weight decay rate of 5 x 10-4, respectively. For the proposed noise stability, we set K and remain the syper-parameters the same as specified in the classification experiments.
Performance Comparison cryo-ET
Mean Accuracy (%)
50
30
20
Random MC-Dropout
LEGAL
VAAL
NozeStabilit
10
10
15
20
25
30
35
40
Labeled Data
Figure 6: Classification performance due to different active learning methods on the cryo-ET task: 50c-snr003.
Observations. We present the comparisons of the different methods in Figure 6 At each cycle, our method demonstrates solid improvement over the others. Moreover, the later cycles, our method outperforms the others by significant margin, demonstrating its consistent superiority. Moreover, due to the noise in the data, some other methods show reduced accuracy as cycle increases (c.g. CoreSet (Sener & Savarese 2018), whereas our method keeps climbing towards a higher accuracy.
A.4.2 EXPERIMENTS ON REGRESSION PROBLEMS
Here we show that our method can also be applied regression problems. We evaluate several active learning algorithms on Ames Housing dataset De Cock (201T) to predict house prices.
Dataset. The Ames Housing dataset De Cock (011 describes the sales of individual residential properties in Ames, Iowa from 2006 to 2010. The data set includes 1461 labeled training examples. There are 60 explanatory variables involved in assessing home values. The Ames Housing dataset considered an expanded version of the classical Boston Housing dataset, which has only 506 labeled samples and 14 attributes.
Model and Training. We use 50% of the data for training set and the remaining for testing. We follow common practices to transform the categorical features into numerical ones with one-hot encoding. A four-layer MLP employed to extract features of the input vectors, followed by another fully connected layer as the predictor. To adapt the regression task, we re-implennen re-implement MC-Dropout to compute the output variance among multiple predictions. Methods such BALD Houlsby et al. (011) and BatchBALD Kirsch et al. (2019), which rely on the output entropy are not suitable for regression tasks.
The Adam optimizer is employed to train the task model. The training will continue for 300 epochs and the learning rate is set 0.001. For the proposed noise stability, we K 10 and remain the
15

---

Under review as a conference paper at ICLR 2022
hyper-parametesd the same specified in the classification experiments. For MC-Dropout, we set the number of Monte- Monte-Carlo sampling to 50 as usually recommended.
Observations. We start the training with 20 labeled samples and iteratively acquire 20 unlabeled samples for annotation. As illustrated in Figure 7 our NoiseStability yields the lowest mean absolute selected data in earlier stages reduces the error quickly than
error in most places. Moreover, the others, demonstrating the advantage of our method when there less labeled data in training. As the training gets into later stages, our method also shows a superiority than most of the others, indicating its stability when more labeled data is involved in training.
Regression Performance Comparison house
19000
Random
GCN
NctreStabiity
Mean Absolute Error
17000
15000
14000
150
175
Number of Labeled Data
Figure 7: Regression performance due to different active learning methods on the Ames Housing dataset. The mean absolute error compared. Lower is better.
A.4.3 EXPERIMENTS WITH SMALL SELECTION SIZF.
To further validate the effectiveness of our method, we consider smaller selection size 10 and respectively on MNIST. Though such very small selection sizes are not frequently used in practice, they can be treated as a popular indicator evaluate the performance of an acquisition function. Moreover, under the extreme case in which the selection size is I, results can reveal whether an AL algorithm will successfully select the most informative representative samples at each AL cycle, without concerning the problem of batch acquisition.
We use the same settings as in the MNIST experiments described in Appendix A.3 As demon- strated in Figure all the algorithms perform consistently when different selection sizes are used, iee algorithms perform well with larger selection sizes also have good performance with smaller selection sizes. Under smaller selection sizes, all the state-of-the-art methods outperform Random selection by large margin. Morcover, our NoiseStability method yields the highest accuracy.
A.4.4 EXPERIMENTS LARGE SCALE DATASETS
Our method can be easily applied large scale datasets with large selection size. Here, we compare our method with Random selection and MC-Dropout on such a large dataset, namely Ima- geNet Deng et al. (2009).
The ImageNet dataset Deng et al. 2009 consists of total of 14 million images with more than 20K categories. We use the popular subset II. 1SSR22012 which includes around 1.3M images distributed across IK categories. We randomly select 15% of the total training samples to build the initial labeled pool and iteratively select 5% of the total training samples for annotation. Note that the data selection is conducted without replacement, we do in all the other experiments.
Dataset.
Model and Training. We employ the standard ResNet-18 18 model for this task. SGD is used as the optimizer with an initial learning rate of 0.1, momentum of 0.9, weight decay of 0.0005 and batch size of 256. We train the model for 90 epochs following the practice in Pytorch official code. The learning rate decays by factor of 0.1 every 30 epochs.
16

---

Under review as a conference paper at ICLR 2022
Performance Compartsan on MNISST ISelection Size 10)
Performance Comparison on MNIST ISelection Size
Mean Accuracy (15)
Mean Accuracy (%S
85
75-
100
150
200
250
300
350
400
450
50
100
150
200
250
300
350
400
450
Number Labeled Data
Number Labeled Data
Figure 8: Classification performance due to different Bayesian active learning methods on MNIST with selection size of 10 (left) and (right), respectively.
Observations. As illustrated in Figure MC-Dropout and Noise Stability are consistently superior to the naive Random selection. Our method significantly outperforms the two baselines at the later AL cycles.
Performance Comparison on ImageNet
Mean Accuracy (%)
68
66
62
NolseStablity
MC Dropout
25
35
40
45
% of Labeled Data
Figure 9: Classification performance due to different active learning methods on the ImageNet dataset. Top-1 Accuracy is reported.
A.4.5 EXPERIMENTS MORE ARCHITECTURES
Since the proposed method directly manipulates the task model, conduct additional experiments on another architecture in order to validate the architecture-agossti nature of the proposed method. Specifically, we choose MobileNet-v2, compact yet effective DNN architecture with fewer param- eters. We evaluate the AL algorithms on Cifarl0 and Cifar using exactly the same settings as the experiments in which ResNet-18 used. As shown in Figure 10 our method still yields very competitive performance. In Cifar 100, method is on par with CoreSet, but it outperforms the others by a decent margin. Note that in the regression experiment in Appendix (A..2 our method is also shown effective MLP architectures. These results demonstrate that method is indepen- dent of model architectures.
A.5 ADDITIONAL ABLATION STUDY
Here we conduct an ablation study to analyze the two important designs in our method. One the adaptive noise magnitude. The other is the position inject noise.
17

---

Under review as a conference paper at ICLR 2022
MobileNet-v2 Perforaacc Comparison Citar10
MoblichNet-v2 Performance Comparison on Cira1100
02
Mean Accuracy (15)
Mean Accuracy (%)
81-
53
04-
50
Sarelum
48
Kandar
10
15
20
30
10
15
20
30
35
Labeled Data
% of Labeled Data
Figure 10: Classification performance due different active learning methods on Cifar (left) and Cifar100 (right), using MobileNet-v2 as the backbone.
Comparison adaptive and constant noise magnitude
Comparison of different noise injected positions
92
82
Mean Accuracy (56
Mean Accuracy
26
02
12
coredant Led conndant 2e2
layer
% Labeled Data
% of Labeled Data
Figure 11: Ablation study of our NoiseStability method on Cifar10. We present the effect of adaptive noise magnitude (Ieft) and the position to inject noise (right).
A.5.1 ADAPTIVE NOISE MAGNITUDE
The noise magnitude should be proper, i.c. very large magnitude will destroy the network and very small one makes the model less capable to distinguish uncertain samples. We adopt an adaptive strategy control the noise magnitude. In order validate its effectiveness, with several constant noise magnitudes including le-1, le-2, le-3 and le-4. As shown in Figure [I] (left), our method performs much worse if the constant magnitude is too small or too large. When proper constant is used, i.c. Ie-2 or le-3, the performance is comparable to the adaptive strategy. However, selecting a reasonable constant noise magnitude for each specific model sounds ad-hoc, since model parameters w.r.t. different architectures usually have different value distributions. More importantly, our experimental results on ResNet-18, MobileNet-v2 and ML.P have demonstrated that the adaptive noise magnitude works well across various architectures, demonstrating the rationale of using the adaptive strategy.
evaluate
method
A.5.2 POSITION APPLYING NOISE
Here we investigate how the position of applying noise affects the performance. We take ResNet- 18 as the task model that includes four layer modules, each of which is composed of a sequence of BasicBlocks. In this experiment, we inject noise each layer module respectively. As shown in Figure 11 (right), changing the position of noise injection does not affect the performance much.
A.6 RUNNING TIME COMPARISON
To demonstrate the time efficiency, we report the running time the acquisition function used in each AL. baseline in Table We conduct the experiments Cifar10, with selection size
18

---

Under review as a conference paper at ICLR 2022
of 2500 and 250, respectively. The size of the corresponding unlabelled pool is 10000 and 20000, respectively. The rumning time is measured on a single Nvidia Tesla V100 GPU. For fair comparison, set the number of dropout sampling to 10 for all the Bayesian methods. As can be seen, NoiseStability is relatively efficient among these AL baselines. Moreover, the time complexity of method increases approximately linearly the size of the unlabelled pool is increasing, but is almost independent of the selection size.
Unlabeled Pool 10000
Unlabeled Pool 20000
Method
250 Selected
2500 Selected
250 Selected
2500 Selected
MC-Dropout
1.01
1.02
1.92
2.25
CoreSet
2.19
49.26
6.15
54.72
LL4AL
0.13
0.28
0.30
0,48
VAAL
225.96
402.32
270.18
462.24
GCN
0.36
1.10
0.99
1.34
BALD
1.53
2.10
3.39
4.08
BatchBALD 7.29
77.17
18.17
115.66
NoiseStability
2.23
2.28
4.81
5.04
Table 1: Data selection time (in minutes) for the first cycle on Cifar10 with two selection sizes 2500
and 250. All the results are measured on Nvidia Tesla V100 GPU with an Intel(R) Xeon(R) Gold 5117 CPU.
A.7 EMPIRICAL VALIDATION OF VARIANCE REDUCTION
The theoretical analysis in Section 3.2 reveals the connection between NoiseStability and vari- ance reduction. Here we validate this connection, by quantifying the variance reduction for the training samples used in cycle 1, using the model trained in the subsequent cycle. As the original form of output variance is intractable for DNNs (Thisted 1988). we choose MC-Dropout (Gal & Ghabramani 2016) as the criterion for variance estimation evaluate our method and the others. Specifically, we approximate the uncertainty with 1) the confidence represented by Bayesian pos- terior probability for the true label and 2) the prediction entropy. As illustrated in Table 2 while training with more samples always reduce the output variance, selecting samples with our NoiseS- tability exhibits the best effect.
Table 2: Evaluation of variance reduction for different active learning methods at cycle 2 on Ci- far10. The confidence of the true label (higher is better) and Entropy (lower is better) are used to approximate the variance. We use the same random seed in the all experiments. The Confidence and Entropy cycle 1 for training samples are 0.64 and 0.69, respectively.
Criterion Confidence 0.77
Random
CoreSet
VAAL
GCN
NoiseStability
0.79
0.74
0.78
0.80
Entropy 0.45
0.44
0.51
0.44
0.42
A.8 DISCUSSION ABOUT MORE RELATED WORKS
Studies in active learning with linearised Laplace approximation of the BNN posterior (Immer et al. 2021: Daxberger et al. 202T) are very relevant to our method. Here we briefly discussing their connections and differences. From a high level, our criterion is the minimum mean-squared eттor estimation (MMSE) while theirs are Bayesian estimation, corresponding to the frequentists' and Bayesian perspective statistics. They are not incompatible, yet they are inextricably linked. For instance, under the jointly Gaussian conditions (such as the Gaussian priors assumed in those papers) the MAP and MMSE are known to be equivalent; They face similar computational chal- lenges as MMSE is the conditional mean that are difficult to calculate the MAP. Our approach to approximate full conditional mean by the linear MMSE, which only pertains to the mean and covariances. This is principled approach that is also used in practice in other areas such as the Kalman filter. In contrast, the Laplace approximation (c.g. (3) in Immer et al. (202T) accurate only when 0 is close the optimal value 0* (where the first order term is negligible) and thus the
19

---

Under review as a conference paper at ICLR 2022
theory only provides local guarantees. It is our perspective that the condition is restrictive and does not apply to practical deep learning methods. The resulting algorithms are also different: [Immer et al. (202I) is based the determinant of the Hessian matrix; the method in Daxberger et al. (2021) involves the diagonal of inverse Hessian. According to our findings, the predictive variance reduction conditioning on any direction is always monotonic decreasing with the gradient norm and servers as the cornerstone of our approach.
20


Reviews:
[{"review_id": "IQC--4QY0-2", "reviewer": "Reviewer_9WDB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "+:\n\n+ The method is simple and does not involve auxiliary models.\n+ Experiments are performed on several tasks and datasets\n+ The paper provides some theoretical connection to variance reduction (they are straightforward though).\n\n\n-:\n- The clarity of the presentation can be improved in several places. For example:\nIt seems that in the current presentation, the assumption is that epsilon (noise magnitude) is the same for all parameters. It is not clear if this is implemented. Also, is this a valid assumption in terms of noise being relatively “small\"? Is the noise added to all parameters? One or more baselines are missing from some of the results figures without any explanation.\n\n- Since the proposed method directly works on the task model as opposed to some of the baselines, it would be helpful to see more models/architectures tested for each task.\n\n- The proposed method does not seem specific to image data, so evaluating it on tasks in other domains would be beneficial to validate its applicability across different domains.\n", "summary_of_the_paper": "A method for batch active learning for DNNs is proposed in the paper. The main idea is to consider robustness to parameter perturbation as an uncertainty measure. In other words, prediction deviation when adding a small noise to the parameters is used as a score for selecting data to be labeled in the active learning loop. The authors provide some theoretical analysis for the method to connect it to variance reduction. Experiments on several image classification and semantic segmentation datasets show that the method performs better than several SotA baselines.", "summary_of_the_review": "Overall, the paper proposes a simple method for active learning, provides some theoretical connection with existing literature, and shows its effectiveness where it improves SotA. While there is room for improving the paper, I tend towards acceptance. \n\n--Update--\nI think the authors have addressed most of the concerns and comments, and I remain positive about the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Hmh9oZZcdUK", "reviewer": "Reviewer_pooj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The idea is simple and easy to implement. \n- Results seem to generalize to different datasets.\n\nWeaknesses:\n- Links to calibration are confusing and not empirically supported.\n- Results are not convincing:\na) Large scale is needed (Imagenet for conclusive results)\nb) Ensembles are not compared (and MC Dropout results are somehow questionable). See below\nc) Compute cost is not provided. ", "summary_of_the_paper": "This paper proposes a single-model active learning method for classification and segmentation. The key idea is to add noise to the parameters in order to generate multiple outputs and be able to estimate the uncertainty of a sample. The paper presents results for classification and segmentation in different datasets.", "summary_of_the_review": "The idea behind the paper is interesting. Instead of using MC Dropout of Ensemble methods trained with different seeds, use a single model and do multiple forward passes adding a perturbation to the parameters for computing the output distribution needed to estimate uncertainty.\n\nAs this is basically a multi-model approach (using variations of a single model), would be interesting to see a comparison with ensembles. In the end, ensembles are the best approximation to Bayesian Uncertainty estimation.  \nFor MC Dropout, I wonder about the parameters. I do believe the results are taken from one of the references, but details there are needed. MC Dropout easily outperforms single models like Entropy when the number of forward passes is large enough. How many models are used here? Would be good to see, at least 50-100 forward passes. It is actually surprising that MC Dropout performs on par or worse than random on CIFAR. \n\nSame with the proposed method. How does the method vary as a function of the number of forward passes? How does that compare to the same ensembles? A similar comparison was done (for classification) in: Chitta et al. Training Data Subset Search with Ensemble Active Learning arXiv 2020. The number of forward passes needed is larger compared to models trained with different seeds. \n\nThe paper talks about uncertainty calibration but there is no empirical evidence. That would be very interesting to see. \n\nI would also be interested in understanding how the noise is applied to the layers and if noise in all layers is needed. That would have an impact in the inference time needed for computing uncertainty. An ablation study here would be interesting. \n\nSame for cityscapes. I am missing Random as comparison. \n\n\nFor all the experiments, I would suggest adding Entropy as a baseline for comparison. Entropy is straightforward and provides very solid results. \n\n\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Cza6gCZn3Xp", "reviewer": "Reviewer_owNu", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n* **Relevance**: The active learning problem is an important one in today's age of 'big data' and deep learning. Deep neural networks are particularly data-hungry and we wish to be able to train them with as few data points as possible. However, as noted by the authors, deep neural networks are also pathologically overconfident in their predictions. Thus, predictive entropy (for example) cannot be reliably used as a proxy for the informativeness of unlabelled examples. Therefore, methods for AL with deep neural networks are of high relevance and importance to the field. \n\n* **Simplicity**: The proposed method is simple to understand and implement. Many state-of-the-art AL methods rely on complicated Bayesian approximations and are difficult to understand/implement. A simple method that works well is a good method!\n\n## Weaknesses\n\n* **Clarity**: I found the paper hard to understand in many places and there were many grammatical errors. Overall, I think a lot more effort needs to be put into making the ideas presented easy to understand and follow. I'll provide some specific examples here, however, this is a general comment for the paper. I encourage the authors to spend more time thinking about how best to present their ideas. I also encourage them to use a tool like Grammarly which will flag problematic sentences. \n    1.  \"This is also aligned with the pre-requisite of semi-supervised learning.\" – What is the pre-requisite of semi-supervised learning? What is the point of this sentence? \n    2. \"... namely uncertainty based method, aims to select a portion of the most uncertain or representative data ...\" – uncertain is not the same as representative so it is not clear exactly what is meant by this sentence. \n    3. What is the \"interlayer cushion\"? This seems like a pretty specific term so it should be explained. \n    4. What does \"tends to be easy to recognize future examples\" mean? Is this to say that the model is good at determining which examples should be labelled?\n   5. \"Specifically, we introduce a simple algorithm of uncertainty estimation by computing the distance between outputs of the clear and perturbed **input** ...\" (emphasis my own) – this sentence makes it sound like the inputs are also perturbed, when in fact it is only the model parameters that are perturbed. \n    6. \"Our method is easy to implement and free of customized auxiliary models. Therefore, ...\" – I think a bit more context is needed for what kinds of auxiliary models other AL methods require, in order for this statement to make sense to the reader. \n    7. \"pool of unlabelled data\" – I believe the standard term is \"pool set\".\n    8. \"labelled pool\" – this is just the train set.\n    9. I don't think the set update equations are needed to understand the moving of labelled examples from the pool set to the train set. If anything the equations make this seem more complicated than it actually is.\n    10. In section 3.2 the authors switch notation for the Jacobian of the output w.r.t the parameters. This doesn't seem necessary to me and makes things more difficult to follow. \n    11. \"Note we omit the notation of \\theta in J and A to avoid ambiguity\" – presumably the authors mean that the omission is okay because there is no ambiguity? \n    12. Where does equation 7 come from? Some elaboration would be helpful.\n    13. In equation 10, in the last line the numerator contains a J_\\hat{y}(x)^T. Should this be u(x)?\n\n\n* **Experimental evaluation**: while the experimental evaluation provides a good start (the ablations provided are especially appreciated), it is not of the standard required for an ICLR paper, in my opinion. I have concerns with:\n    1. **The narrow range of underlying tasks.** The tasks considered in the paper are all image classification tasks of various kinds. Regression tasks and non-computer vision tasks should be added. \n    2. **Missing baselines.** In particular, batch-aware acquisition strategies such as BatchBALD (Kirsch et al.) and sparse subset approximation (Pinsler et al.). The BALD acquisition function  (Houlsby et al.) is also missing and would likely improve the strength of the Dropout baseline.\n    3. **Experiments with *smaller* step sizes.** The chosen step sizes (e.g. 5% of MNIST = 3000 images) are very large. Most active learning works present results for step sizes of 1, 10, 50, etc. (see the experiments in the references below). With such a large step size a random (or near random) acquisition can easily perform well because there are already enough examples to train a decent model. With smaller step sizes the choice of acquired examples becomes much more important for good model performance. \n    4. The adaptive noise magnitude \\eta. An ablation, showing the performance when using a constant value for the noise magnitude (e.g. a value that does not depend on \\Delta\\theta^{(i)}, should be added to verify that this is an important part of the proposed method. \n    5. No runtime comparisons. In the text there are brief discussions of the runtime for the proposed method and some of the baselines, however there are no empirical results to go with the discussion. Runtime comparisons should ideally be provided for all baselines and the proposed method.\n\n* **Connections to other work.** The proposed method seems very closely related to doing active learning with a linearised Laplace approximation of the BNN posterior (see Immer et al. and Daxberger et al. for good descriptions of the linearised Laplace approximation). However this is not discussed at all. I suspect that this link could provided a more concrete link to the predictive variance (the provided connection being somewhat weak in my opinion.) \n\n\n## References\n\n### Active Learning \n\nAndreas Kirsch, Joost van Amersfoort, Yarin Gal:\nBatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning. NeurIPS 2019: 7024-7035\n\nRobert Pinsler, Jonathan Gordon, Eric T. Nalisnick, José Miguel Hernández-Lobato:\nBayesian Batch Active Learning as Sparse Subset Approximation. NeurIPS 2019: 6356-6367\n\nNeil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745, 2011\n\n### Linearised Laplace\n\nAlexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, Mohammad Emtiyaz Khan:\nScalable Marginal Likelihood Estimation for Model Selection in Deep Learning. ICML 2021: 4563-4573\n\nErik A. Daxberger, Eric T. Nalisnick, James Urquhart Allingham, Javier Antorán, José Miguel Hernández-Lobato:\nBayesian Deep Learning via Subnetwork Inference. ICML 2021: 2510-2521\n", "summary_of_the_paper": "This paper provides a new method for solving the active learning problem, i.e. choosing from a set of unlabeled examples that should be labelled and added to the training set. The proposed method makes use of 'noise stability' of inputs–specifically if the model output corresponding to an input changes to a large extent when small amounts of noise are added to the model parameters, then this input should be labelled. The authors compare their method to a range of existing AL methods and draw theoretical connections to predictive variance reduction.", "summary_of_the_review": "While the proposed method is simple to understand and solves an important problem, I do not think the paper is of the standards required for ICLR primarily due to the weak experimental evaluation and lack of clarity in the writing. \n\n++++ Updated review ++++\n\nThe authors have provided many clarifications and additional experimental results which do strengthen the paper. As a result, I have increased my score from 3 to 5. Unfortunately, the experimental evaluation is still a little weak which prevents me from increasing my score further. Similarly, I am not sure that the updated presentation is quite clear enough.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission considers a new acquisition function for active learning. The method considers the sensitivity of the prediction for a given datapoint with respect to parameter perturbations. Points with the largest variance under these perturbations are selected for labelling.  The method is simple and the empirical results are reasonable. Some weaknesses are the clarity of writing, and somewhat limited experimental comparisons.

The discussion was useful and helped improve the clarity. Additional experiments also helped improve the paper, although some reviewers still felt the experimental comparisons were lacking, including using entropy as a baseline acquisition function. Despite these improved scores, the overall average score remains below threshold I'm afraid.

I feel this is a useful paper, but perhaps needs a little more polishing in the writing and some additional experiments. As such, it just falls short of the acceptance threshold.
Decision: Reject