A Simple Yet Powerful Deep Active Learning With Snapshots Ensembles
Authors: Seohyeon Jung
Sanghyun Kim
Juho Lee
Venue: ICLR 2023 poster (2023)
Keywords: Active learning
Snapshot ensemble
Uncertainty estimation

Abstract: Given an unlabeled pool of data and the experts who can label them, active learning aims to build an agent that can effectively acquire data to be queried to the experts, maximizing the gain in performance when trained with them. While there are several principles for active learning, a prevailing approach is to estimate uncertainties of predictions for unlabeled samples and use them to define acquisition functions. Active learning with the uncertainty principle works well for deep learning, especially for large-scale image classification tasks with deep neural networks. Still, it is often overlooked how the uncertainty of predictions is estimated, despite the common findings on the difficulty of accurately estimating uncertainties of deep neural networks. In this paper, we highlight the effectiveness of snapshot ensembles for deep active learning. Compared to the previous approaches based on Monte-Carlo dropout or deep ensembles, we show that a simple acquisition strategy based on uncertainties estimated from parameter snapshots gathered from a single optimization path significantly improves the quality of the acquired samples. Based on this observation, we further propose an efficient active learning algorithm that maintains a single learning trajectory throughout the entire active learning episodes, unlike the existing algorithms training models from scratch for every active learning episode. Through the extensive empirical comparison, we demonstrate the effectiveness of snapshot ensembles for deep active learning.

Published as a conference paper at ICLR 2023
A SIMPLE YET POWERFUL DEEP ACTIVE LEARNING WITH SNAPSHOTS ENSEMBLES
Seohyeon Jung" Sanghyun Kim
Juho Lee
Kim Jaechul Graduate School of Al
Korea Advanced Institute Science and Technology (KAIST)
Daejeon, Republic of Korea
{heon2203, nannul Ina, jubolee)jkaistt.cckkk
ABSTRACT
Given an unlabeled pool of data and the experts who can label them, active learn- ing aims to build an agent that can effectively acquire data to be queried to the experts, maximizing the gain in performance when trained with them. While there are several principles for active learning, prevailing approach is to es- timate uncertainties of predictions for unlabeled samples and use them to de- fine acquisition functions. Active learning with the uncertainty principle works well for deep learning, especially for large-scale image classification tasks with deep neural networks. Still, it is often overlooked how the uncertainty of pre- dictions is estimated, despite the common findings on the difficulty of accu- rately estimating uncertainties of deep neural networks. In this paper, we high- light the effectiveness of snapshot ensembles for deep active learning. Com- pared to the previous approaches based on Monte-Carlo dropout deep en- sembles, we show that a simple acquisition strategy based on uncertainties es- timated from parameter snapshots gathered from single optimization path sig- nificantly improves the quality the acquired samples. Based on this observa- tion, we further propose an efficient active learning algorithm that maintains single learning trajectory throughout the entire active learning episodes, unlike the existing algorithms training models from seratch for every active learning episode. Through the extensive empirical comparison, we demonstrate the ef- fectiveness of snapshot ensembles for deep active learning. Our code is available at: wwtptttt//// httpbb..tttttt/gt//gg// com/ nannulnnnlllnnnmmnnm lnailllnnn/ee nnnpoonns/s/ssssttoto
INTRODUCTION
The progress of deep learning is largely driven by data, and we often work with well-curated and labeled benchmark data for model developments. However, in practice, such nicely labeled data are rarely available. Many the data accessible to practitioners are unlabeled, and more importantly, labeling such data incurs costs due to human resources involved in the labeling process. Active Learning (AL) may reduce the gap between the idcal and real-world scenarios by selecting the in- formative samples from the unlabeled pool of data, after being labeled and trained with them, model can maximally improve the performance.
The main ingredient of an algorithm is the acquisition function which ranks the samples in an unlabeled pool with respect to their utility for improvement. While there are several possible design principles (Ren et al., 2021), 1), in this paper, mainly focus on the acquisition functions based on the uncertainty of the predictions. Intuitively speaking, given a model trained with the data acquired so far, an unlabeled example exhibiting high predictive uncertainty with respect to the model would be a "confusing" sample which would substantially improve the model if being trained with the label acquired from experts. A popular approach in this line is Bayesian Active Learning by Disagreement (BALD) (Houlsby et al., 201 1), where a committee of multiple models predicts an unlabeled sample, and the degree of disagreement measured as a ranking factor. Here, the multiple models are
Equal contribution

---

Published as a conference paper at ICLR 2023
usually constructed in Bayesian fashion, and their disagreement reflects the model uncertainty about the prediction. BALD is demonstrated to scale well for modern deep neural networks for high-dimensional and large-scale data (Gal et al., 2017).
Similar to BALD, many AL algorithms based on uncertainty employ a committee of models to es- timate the uncertainty of predictions. The problem is, for deep neural networks trained with high- dimensional data, it is often frustratingly difficult to accurately estimate the uncertainty. To address this, Gal al. (2017) proposed use Monte-Carlo DropOut (MCDO) (Gal and Ghahramani, 2017), an instance of variational approximation to the posteriors and predictive uncertainty, while Rakesh and Jain (2021) suggested using more generic spike-and -ndksaadd-llab variational posteriors (Louizos al., 2017). Nevertheless, variational approximations tend to underestimate the posterior variances (Blei et al., 2017; Folgoe et al., 2021), so the uncertainty- -based acquisition functions computed from them may be suboptimal. Alternatively, one can employ Deep Ensemble (DF) (Lakshminarayanan et al., 2017), where a single model is trained multiple times with the same data but with different random seeds for initialization and mini-batching, Despite being simple to implement, DE works surprisingly well, surpassing most of the Bayesian Neural Networks (BNN) alternatives terms of accuracy and predictive uncertainty (Fort et al., 2021; Ovadia et al., 2019). To this end, Beluch et al. (2018) highlighted the effectiivness of DE as a way to estimate uncertainty for acquisition functions and demonstrated excellent performance.
A drawback of DE is that it is computationally expensive, as it requires multiple models to be trained and maintained for inference. As an alternative, Snapshot Ensemble (SE) (Huang al., 2017; Garipov et al., 2018) proposes to collect multiple model snapshots (checkpoints) within a single learning trajectory, rather than collecting them at the end of the multiple learning trajectories in DE. Compared to DE, SE enables the construction of decent set of models without having to go through multiple training runs while not losing too much accuracy.
Inspired by the advantage of SE, we study its use in the context of AL. Specifically, we estimate the uncertainties from SE and use them evaluate the uncertainty-baaee extensive empirical comparisons, we demonstrate that the AL based on SE significantly outperforms existing approaches, even comparable to or better than the one with DE. This result is somewhat surprising since it is often reported that SE is less accurate than DE (Ashukha et al., 2020). Moreover, based on this observation, we propose a novel AL algorithm that can substantially reduce the number of training steps required until the final acquisition. Typically, an AL algorithm alternates between acquiring labels based on a model and re-training the model with the newly acquired labels. Here, for every re-training step, the old models Instead, we suggest maintaining a model on a single learning trajectory throughout the entire AI. procedure and gathering snapshots from the trajectory to compute acquisition functions. We show that this can significantly reduce the number of training steps without sacrificing too much accuracy. In summary, our contributions are as follows:
acquisition functions. Through
discarded, and a new model is trained from scratch.
We propose to use SE for the uncertainty- -based acquisition functions for AL and demon- strate its effectiveness through various empirical evaluations.
We propose novel AL algorithm where a single learning trajectory is maintained and used to compute acquisition functions throughout the entire AL procedure. We demonstrate that our algorithm could achieve decent accuracy with much fewer training steps.
2 BACKGROUND
2.1 SETTINGS AND BASIC ACTIVE LEARNING ALGORITIIM
In this paper, we mainly discuss K way classification problem, where the goal to learn classifier f(; 0), parameterized by 0, taking an input x E Rd to produce a K K-dimensional probability vector, that is, fæx; 0) E [0, 1] such that ΣK= [x: 0) 1. To learn 0. need labeled dataset consisting of pairs of an input & and corresponding label y E K)....,K, but in AI, we are given only an unlabeled dataset 1/ {a. without labels.
An AL algorithm is defined with a classifier model f(::0) and acquisition function Rd R measuring how useful an unlabeled example x is to the classifier f(::0). Given f and a, an AL algorithm alternates between acquiring the labels for chosen unlabeled samples and training the
2

---

Published as a conference paper at ICLR 2023
classifier with the labeled samples. A single iteration of acquiring samples and training the classifier called an episode. In the first episode, m samples are randomly chosen from 2, and the labels are acquired for them to constitute initial training set Dunin The classifier is then trained with Pirain to obtain 0,, or in case of the ensemble-based AL, a set of parameters [0{)) The labeled samples 2, with the parameters {0 1 from the
removed from U. For all subsequent episodes previous episode, the samples remaining in 24 are ranked with the values of the acquisition function, and the top of them selected to be labeled. The newly labeled samples then appended to the labeled training set Drain. and the classifier is trained from scratch with the extended Dinin to The algorithm terminates when it reaches the predetermined number of episodes T, and the goal of AL is to maximize the accuracy of the classifier after the final episode.
obtain { Js=1-
2.2 ACQUISITION FUNCTIONS
Here, we review some popular choices for the acquisition functions, especially the ones based on predictive uncertainties. We consider the following acquisition functions which estimate the un- certainty of the predictions via set of parameters (0(*))s -==1- This of parameters defines committee of models {f(; 0 )
Maximum Entropy (ME). ME measures the predictive entropy of given example æ which can be approximated as,
H [ ac, Dinin]
Σ
ply kla,, Dimin) log ply k/a, Dirain)
k=1
8 log $ Sx
Σ Sk (z; (((
5
Larger entropy means that the model is more uncertain about the prediction.
Variation Ratio (VR). (Freeman, 1965) measures how certain model is about its prediction for æ or how many of the committee agree with the prediction, and is calculated as fm S, where fm the frequency of a mode prediction over committee members. Similarly, Least Confident (LC) sampling chooses the least confident sample 1 1--mmaaa max; p(y k/æ,, Durain) and Margin (MAR) sampling chooses examples with the smallest difference between the largest and the second largest probabilities.
Bayesian Active Learning by Disagreement (BALD). BALD (Houlsby et al., 2011) measures the mutual information between the label and the parameter given an input æ and the training data Durin It can also be interpreted as measuring disagreement among the predictions of the committee members. BALD maximized when each of the committee members is certain about their own pre- dictions (small H [y] ar, g, but the predictions disagree with each other, so the averaged prediction becomes uncertain (high [/|xr, Preain).
[y, 0 0|0, Duain = H(y x, Durinl Eoj Dain H[ylæ, 0]
K
1 fo (z; log S
Σ fa (z; 0(:))
k=1
(2; ((0)) log fa (2; o(+)).
2.3 ESTIMATING UNCERTAINTY
In Bayesian AL, the model parameter is treated as random variable with prior p(0) and the posterior (D)iniin) is approximated via a set of samples (0(*) There are several ways to approximate it.
3

---

Published as a conference paper at ICLR 2023
Variational approximations. For variational approximation, an easy-to-handle variational dis- tribution g()) is introduced, learned to minimize ( and used as a proxy for
(i That is, once obtained approximate distribution q(0), we draw (()) ( Lid. g(0). A popular choice for g(0) is mean-field Gaussian distribution (Blundell et al., 2015). In AL lit- erature, MCDO is widely used after (Gal et al., 2017), where the dropout (Srivastava et al., 2014) is applied to the model f and the randomness due to the dropout is interpreted as approximate posterior (0). While relatively simple to implement, the variational approximations are known to underestimate the posterior variances.
Deep ensembles. DE (Lakshminarayanan et al., 2017) trains f(: multiple times with the same Ja=1 Deaiin but with different random initializations to obtain { DE is simple to implement, and yet its performance is remarkable, achieving state-of-the-art across various applications. The power of DE is mainly from its property to pick parameters from multiple modes (Fort et al., 2021), so the committee constructed from them yields a diverse set of predictions. Even if is not explicitly assuming the prior for p(0), DE can roughly be interpreted as an approximate Bayesian inference method (Wilson and Izmailov, 2021; D' Angelo and Fortuin, 2021), the parameters (0(+)) ===1 can be interpreted as posterior samples approximating the uncertainty of the models.
Snapshot ensembles. DE is expensive, both for training and inference since it has to keep multiple models with different parameters. SE (Huang et al., 2017; Garipov et al., 2018) is an idea to reduce the training cost of DE, where the multiple parameters (0(e) } J==1 are gathered within a single training run rather than collected from multiple training runs. To obtain diverse parameters, the learning rate of the training run is carefully chosen to encourage the optimization path to explore a wide area of the loss surface, and the parameter "snapshots" are periodically captured during the training run. The SE usually underperforms DE with the same number of parameters gathered, but it can collect the parameters much faster since requires only single training run.
METHODS
3.1 ACTIVE LEARNING WITH SNAPSHOT ENSEMBLES
We first present an AL algorithm based on SE that simple and efficient. In each episode, we store parameter snapshots at regular intervals during the classifier training stage, which are then used to compute the acquisition function at the end of the episode. This approach incurs no additional computation cost for training, unlike AL based on DE. In the final episode, we have several options to choose from when we train the classifier using the acquired data D following a single learning trajectory and picking the parameter at the last step point estimate, applying DE obtain an ensembled model. Algorithm 1 summarizes our SE-based AL algorithm, where the final classifier is obtained with vanilla Stochastic Gradient Descent (SGD), but DE can be applied instead. In Section 5, we demonstrate that this simple modification with SE, albeit no increase in the training and inference costs, significantly improves the performance, even outperforming the AL with DE.
3.2 ACTIVE LEARNING WITH SNAPSHOT ENSEMBLES AND FINE-TUNING
Algorithm trains classifiers for intermediate episodes and discards them after computing the ac- quisition function. However, considering that the acquisition function computed from a single learn- ing trajectory works well, we can improve efficiency by keeping single trajectory throughout all episodes and using the resulting parameter of the previous episode as initialization for the next episode. We call this strategy SE. + FT. There are two things to note here. First, since we continu- ously fine-tune single model, we need fewer training steps for each episode than the vanilla AL.. Second, although the intermediate classifiers may be less accurate than those trained from scratch, as expected, this is less important since what really matters is the accuracy after the final episode. We argue that the intermediate episodes are to acquire samples quickly for the final training, so the accuracy of the classifier during that process is not important.
Also, our AL procedure with fine-tuning is reminiscent of continual learning, in the sense that single model is continually being trained for multiple episodes having different data. To this end, employ two commonly used tricks in continual learning literature.
4

---

Published as a conference paper at ICLR 2023
Algorithm 1: AL with SE
Algorithm 2: AL with SE Fine-tuning (FT)
Input
Unlabeled dataset M, number of episodes T, number of (acquisitions m,
Input Input for Algorithm regularization
parameter X.
snapshots S, SGD steps N) per episode, acquisition function a, snapshot threshold steps Nimes, objective
Ouput: classifier f(: 0.))
Randomly draw m samples from Lt remove them from U. and set them as Diin.
function J. learning rate schedule v/- Ouput: classifier f(:: 0.).
Randomly initialize Oo.
for .,..., T, do
Randomly draw m samples from 24 remove them from U. and set them as Dirain for t = 1, 1..... T'd do Randomly initialize 0o.
Only at final episode!
if then | Randomly initialize Oo. end
Set + < 23
Set e + +.
for I.....:. do
for j 1,..., N do
Draw mini-batch B from Denin
Draw mini-batch B from Duain-
0, + 00 - m()) VoJ -)
0, O5-1
if } Nines A
All All 00.1
mod
N Mines: N-N |) =)=0 then
if j 2 Naues A
- θ U }
mod
Nares N--Nas J) then
end
θ + u {{,}.
end
end
if
< T then
end if
Compute axx; θ) for all a E U.
then
Pick top m samples, remove them from U. and append them Dinin.
Compute a(x 9) all U. Pick top m samples, remove them from U4 and append them to Dunin // seuse in the zext episode.
else
Set 0, ON
end
0o 4 ON.
else
end
Set 0. +- ON-
end
end
Replay buffer. In principle, for the purpose of fine-tuning, may use only the newly acquired data from the previous episode for fine-tuning. However, this would cause catastrophic forgetting (McCloskey and Cohen, 1989), so the acquisition function based on it may be biased towards cently acquired data. In order to prevent this, we adopt the idea of using replay buffer (Jung et al., 2016; Rolnick et al., 2019; Aljundi et al., 2019), where we draw some portion of the data from the newly acquired data and the remaining portion from the past data. We empirically find that this significantly improves the stability of the acquisition functions.
Regularization. Similar to the replay buffer, regularize the fine-tuning procedure to avoid deviating too much from the previous parameter (Kirkpatrick et al., 2017). That is, optimize the parameter 0 with the fz-regularizer | 0ull where Ou is the starting point of the fine-tuning (the parameters passed from the previous episodes). We also find that this regularization improves the quality of the acquisition functions, leading better classification accuracy in the final episode.
Algorithm summarizes the AL. with fine-tuning. The parts that are different from the AI without fine-tuning marked as blue.
4 RELATED WORKS
Active Learning. Based on how an unlabeled example is fed into an AL agent, AL can broadly be categorized into membership query synthesis, where the agent even generates examples in the sample space, stream-based selective sampling, where the agent decides whether given input is helpful if labeled in online settings, and pool-based AL, where the agent can access large unla- beled pool (Settles, 2009). Pool-based AL be further divided into uncertainty-base diversity-bant based approach, and hybird approach (Ren et al., 2021). Geifman and El- Yaniv (2019) first introduced neural architecture search into AI., claiming that the peremeeraeteri model could lead
approach,
5

---

Published as a conference paper at ICLR 2023
to overfitting especially in the earlier episodes, and therefore, uncertainty estimates could be inac- curate. Similarly, Munjal et al. (2022) argued that the optimal hyperparameters may vary with the size of labeled examples and used Bayesian hyperparameter optimization (AutoML) every episode.
Ensemble. Ensembles of neural networks have shown competitive performance improvement and are widely used in machine learning and deep learning. addition, it shows improvements es- timates for predictive uncertainty. DE (Lakshminarayaman et al., 2017) is one of the well bell-behaved methods estimating the uncertainty of deep neural networks. This method works with several clas- sifiers trained with the same dataset and architecture but with different seeds for random number generator. However, the biggest limitation of ensemble is its computational costs because requires training multiple models (Izmailov al., 2018).
Active Learning with Ensemble. The uncertainty of an example for neural networks can be esti- mated by using an ensemble of neural networks in the context AL. Gal et al. (2017) used BALD (Houlsby al., 2011 1) on MCDO (Gal and Ghahramani, 2017) and later extended it to batch settings where it considers overlaps among data points to be acquired (Kirsch et al., 2019). However, due the expensive cost of training multiple models, most research ensemble-based AL had been restricted to the traditional ML algorithms (Melville and Mooney, 2004; Körner and Wrobel, 2006). Beluch al. (2018) compares the performance of various acquisition functions and uncertainty esti- mation methods on large-scale image classification tasks, proving that DE consistently outperforms other uncertainty-buue based methods such as MCDO(Gal et al., 2017) or single model. In the same manner, Bayesian neural networks show much robustness and reliability compared to DE and MCDO in the context of AI. with continual learning (Rakesh and Jain, 2021).
5 EXPERIMENTS
In this section, through an extensive empirical comparison on three image classification benchmarks (CIFAR-10. CIFAR-100 (Krizhevsky al., 2009), and Tiny ImageNet (Le and Yang, 2015)), we would like to demonstrate the followings:
Measuring uncertainty via SE for AL is effective, comparable, or even better than the one based on DE. across various choices of uncertainty- acquisition functions.
AL with FT can significantly reduce the training cost without sacrificing too much accuracy.
The reason why SE is effective for AI is that it can build a committee of models yielding diverse predictions. Interestingly, for the purpose of acquiring samples of better qualities, the diversity in predictions of the models is more important than the accuracy of the models.
We compare the AL algorithms with three variables acquisition functions (VR, ME, BALD, MAR), algorithms to measure uncertainty (DE, SE, MCDO), and how to train the classifier in the final episode (single model via a vanilla SGD, DE). We report the results with ResNet-18 (He et al., 2016). Please refer to Appendix B for more details, such as experimental protocols or hyperparameter settings. The test accuracy results on CIFAR-10, CIFAR-100, and Tiny ImageNet are summarized in Table 1, Table 2, and Table 3, respectively, according the proportion of labeled examples. Due to the limited resources, we report the results with four and three acquisition functions for CIFAR-10 and CIFAR- 100, respectively, and the results with only for Tiny ImageNet.
5.1 ANALYSIS OF THE MAIN RESULTS
Effectiveness SE for AL. Table confirms that the measuring uncertainty with SE acquires the samples leading to the best classification accuracies, regardless of the choice of acquisition functions or how we train the final classifier. This is somewhat remarkable, considering that the runtimes of SE-based methods are not significantly longer than those for the random baselines, while DE requires significantly longer runtimes due to its requirement to train and test with multiple models. is also noteworthy that the choice of uncertainty estimation method was much more crucial than the choice of acquisition functions. A similar trend is evident for CIFAR- 100 and Tiny ImageNet, where sE generally produces the best classification accuracies. We set S 5 for SE and DE and S = 25 for

---

Published as a conference paper at ICLR 2023
Table 1: Test accuracy on CIFAR-I0 according to the ratio of labeled examples
single model at the final episode
DE at the final episode
Acq Fa
Uncertainty
10%
15%
20%
25%
10%
15%
20%
25%
Runtime
SE 19.1000..46
79.60=00.5
84.22.0.49
87.39.00226
75.01
83.38
87.22
89.75
15.3 hr
FT 69.72
79.36.10:3
84,24 25
87.06220:38
74.94
83.11
87.14
89.26
2.3 hr
VR
DE 66.43++0.7
77.44=0.7
82.48=0.88
86.34
73.23
81.77
85.96
88,95
60.5 hr
MCDO
67.73
76.9820-45
81.8720.52
86.05
73.12
81.52
85,70
89.10
13.3 hr
SE 70.80
79.63+0..7
84.3220.55
87.43
76.37
83.19
86.86
89,25
15.3 hr
SE +
70.28+0.5a
78.00
83.36+0.24 24
86.64:00226
75.27
81.57
85.99
88.85
2.3 hr
BALD
DE
68. 16
78.
10.0.50
82,43 16
86.12
74.07
82.23
85,67
88,81
60.5 hr
MCDO
69.28:10.4
77.32
82.85
86.15
74.03
81.17
86.41
88.52
13.3 hr
SE
68. 39..9
79.11
84, 19 av
87.37
74.79
82,99
86.82
89 69.65
15.3 hr
SE
70.25:0.81
79.68 18
84.5620. 19
87.52:
76.00
83.47
87.33
89.69
2.3 hr
ME
DE 65.77=1.59
77.31
82.5420.26 26
86.24
72.30
82.08
85,97
88,97
60.5 hr
MCDO
67. 67.96
77.67=0.67
82.70so.68 68
86,79+0.26
74.43
81.89
85,94
89.31
13.3 hr
SE 70.89
84.03 79.12
87.49=0.20
76.34
82.80
86.82
89,53
15.3 hr
SE + FT 73.26.:0.36
11.09
85.43..0.17
87.87
77.95
83,93
87.88
89.70
2.3 hr
MAR
70.48::0:5
79.79
83.48
87.03:20.39
76.49
83,47
86,78
89.51
60.5 hr
MCDO
71 3M
79.2520.44
82.75 39
85.19
75.77
82.15
84,98
87,32
13,3 hr
Random
69.88:00.25
78.0820.37
82.4420.55
84,5820008
74.85
81,32
84,95
86,92
12.1 hr
Table 2: Test accuracy on CIFAR- 100 according to the ratio of labeled examples
A single model at the final cpisode
DE at the final opisode
Acg Fa
Uncertuinty
18%
22%
26%
30%
18%
22%
26%
30%
Runtime
SE
58. 38.58.9.19
62.75...40
65.77 27
68.022:0.24
64.02
68.27
70.87
73.37
4.7 hr
SE
59.04:0.27
62.3552015
65.21 15
67.27:20:2
64.06
69.53
71.71
hr
VR
57.22:0.3
61.7320.10
64.7920.31
67.41
63.35
67.65
70.55
72.71
18.5 hr
MCDO
58.30+0.25
62,49
65.00g0.17
67.70.a.15
63,95
68.18
70.39
72,91
4,5 hr
SB 57.13
61.10=0.42
63.85=0.30
66.05
61.86
65.51
68.39
70.63
4.7 hr
SE + FT
57.
37.82.0.36
61 61.46.0.22
63.45
65.29
62.55
66.21
68.30
69.96
1.1 hr
BALD
58.25
62.26=0.32
65.40::.221
67.40=0.22
63,28
67.04
70.27
72.18
18.5 hr
MCDO
58.03
02.28
65.06.0.22 24
67.2320.0
62.84
67.07
69.66
71.81
4,5 hr
SE 57.21
61.84
64.61 00
67.08
62,73
67.73
70.36
72,24
4.7 hr
SE + FT
58.92:0.25
65.04:0.22 62.41
67.08
64.11
67.53
69.91
71.43
1.1 hr
ME
56.40:0.14
61.37:2048
63.95 92
67.03-20-15
62.77
69.83
72.65
18.5 hr
MCDO
56.90=0.as
61.28+0.55
64.89 22
67.05
62.39
67.10
70.27
72.57
4.5 hr
Random
57.55
61.8220.13
64.4620.37
66.0629.27
62.47
66.10
68.93
70.55
3.7 hr
MCDO. The reported means and standard deviations were averaged over five trials and one trial for retraining a single model and DE, respectively.
Effectiveness of fine-tuning. Tables 1 to 3 compare the AL with SE + FT (Algorithm 2) to base- lines. Somewhat surprisingly, SE+ + FT achieved comparable or even better test accuracies with much shorter runtimes. Fig. shows the progression of test accuracies during the episodes of the SE + FT procedure. The intermediate models, which are only used for acquisitions, exhibit lower accura- cies, as expected, since they are fine-tuned and trained with fewer examples for a lower number of epochs. However, at the end of the final episode, where a new classifier is trained from scratch, the test accuracy catches up with the vanilla AL, indicating that even if the accuracies of the intermediate classifiers are inferior, the acquired samples are good enough to obtain a decent classifier at the final episode. We set S = 5 for SE FT, and the reported means and standard deviations were averaged over five trials and one trial for re-training a single model and DE, respectively.
AL with pretrained models. As one can see from Table 3, the performance of the methods on Tiny ImageNet is generally poor, presumably due to the complexity of the dataset. To this end, we em- ployed models pretrained on ImageNet (Deng et al., 2009) and examined whether the AL algorithms pretrained models. We evaluated ResNet-50 and Vision Transformer (Dosovit-
could fine-tune skiy et al., 2021) backbones and compared SE-based and ba-based AL with VR acquisition. For both backbones, SE outperformed DE significantly, especially with small number of acquisitions. We
7

---

Published as a conference paper at ICLR 2023
Table 3: Test accuracy on Tiny ImageNet according the ratio of labeled examples
Acq Fo
Uncertainty
single model at the final episode
DE at the final episode
ResNet-18 from scratch
14%
15%
16%
17%
14%
15%
16%
17%
Runtime
SE
31.28:10.16
22.38
33.37:00.24
36.23
37.49
38.58
39.79
14.0 hr
FT
30.86.0.a1
31.77
32.9620 as
33.74:0031
36.32
37.13
38.19
39.01
2.3 hr
VR
DE
30.05+00.21
31.20+0.48
31.63+0.25 28
32.81 +0.65
35.83
37.17
37.97
39.34
26,1 hr
MCDO
29. 29.48:0030
31.4020.80
32.35 ss
33.32:2008
35.12
37.33
38.16
38.94
14.8 hr
Random
28. 25.45
28.99
29.04.0.12
29.19
32.17
33.23
32,94
32,90
8.5 hr
ResNet-50 pretrained
3%
5%
7%
10%
3%
5%
7%
10%
Runtime
SE
60.89
66.46+0.30
69.71=0.42
72.11
62.72
68.59
71.01
73.85
3.2 hr
VR
DE
59 88
68.21 20.4
28
70.44=00.6
62.19
67.14
70.18
73.23
5.8 hr
Random
61.82 19.34
64.11 22:222
68.
68.7620.07
20.75
63.40
67.59
69.64
72.09
1.8 hr
ViI-Base pretrained
3%
5%
7%
10%
3%
5%
7%
10%
Runtime
SE 77.12
80.68
15.31
86.39..0114
80.51
85,81
86.95
88.73
7.1 hr
VR
DE
69.43
23.20
1..8
86.17:0026
77.01
75.92
85,86
88.06
14.9 hr
Random
11.43
75.9812.59
82.20
84.74.00.2
78.73
84,50
86.83
86,87
4.9 hr
CIFAR10
CIFAR10D
Tiny ImageNet
Test Accuracy (%)
40
80-
35
55-
30
n0-
25
45-
50-
20
15
35
500 2500 5000 7530 10000 32500 35000
5k
9k:
10k
13x
10c
12k
14kc
1.54
labeled examples
# llablled examples
# labeled examples
Vanilla AL with SE
SE+FT
Final episode. Train from scratch after.
Results of vanilla AL (gray) and fine-tuning (FT) at specific episodes (the rest of the colors). CIFAR-10, CIFAR-I00 and Tiny ImageNet all used R8 architecture. Note that before the final episode, the intermediate models show low accuracies.
Figure 1:
set S = 5 for SE and S = 3 for DE, and the reported means and standard deviations were averaged over three trials and one trial for re-training a single model and DE, respectively.
5.2 ANALYSIS OF THE UNCERTAINTY ESTIMATION
In this section, we analyze SE in more detail to see why it is more effective than DE for AL. We conjecture that SE builds a committee of models whose predictions are more diverse than the ones from other methods within a single trajectory, and this is a key to its success in AL. To verify this, we measure average KL.-divergences and pair-wise disagreement values (Melville and Mooney, 2005) of the predictions computed from SE, DE, and MCDO. The disagreement between two class probabilities f(; ())) and f(::0()) on an example is calculated as
(x)( (argmax, (z( Farg max, fa (x:00))))
(3)
As summarized in Fig. 2 (left), SE generally exhibits much higher KL-divergences and disagreement values among their predictions compared to DE and MCDO. As reported in the literature (Fort et al., 2021), DE shows higher disagreements than MCDO but still much less than SE. We believe that this is mainly due to the nature of AL, where we usually work with a relatively small amount of data and less number of training steps than in typical supervised learning settings. DE parameters are collected at the end of each training run, so the models are likely to reach the local optimum. On the other hand, SE collects the parameter snapshots during a single training run, so some of them may not converge to the local optimum. This degrades the classification accuracy, but we point out in Section 5.1, for the purpose of acquisition, the diversity in predictions within a single trajectory is more important than the accuracy of the individual models.
Fig. (right) depicts the correlation between the VR values and the predicted probabilities of the ground truth class, along with distributions of VR scores. The test error rate for each bin shown,
8

---

Published as a conference paper at ICLR 2023
-0.4
12-
14
0.6
Cor Uoe VR & Ore
-0.2
14
Aug www.sprcccooeoo
-0.3
0.8
-0.4
0.6
0.3
0.5
-0.6
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.1
oramples
6000
1000
5000
30000
15000
1000
5000
10000
15000
3000
15.5%
43.2%
57 3hh
46.76
# train exsmples
train examples
MC30
Variation ratio Test wrroor t4 SE
Variation ralio Test
DE
Figure 2: Left panel: (Left) average KL-Divergence and disagreement = between the predic- tions with different uncertainty estimation methods. (Right) Spearman's 's rank correlations E and Pearson's ss correlations E between VR and a predicted probability of the ground truth class Pirue- Right panel: boxplots (up) and histograms (down) of samples binned with the VR values.
X
X
examples with lowest VR scores
examples with highest VR scores
Figure 3: Probability maps for the upper right corner images where the blue means dog class and red means cat class. The black small crosses indicate snapshots obtained from SE in the parameter space, while red crosses indicate instances which the model committee predicted incorrectly.
and the number of correctly classified examples is visualized as light gray areas. This can be inter- preted unnormalized reliability diagram (Murphy and Winkler, 1977). For instance, for the bin with VR 0.8 (all disagree), SE exhibits a test error 72.1 while DE shows only 46.7% However, for the bin with VR = 0.0 (consensus), respectively. As examples with high VR scores are initially selected to be labeled, DE is more likely query examples that the model is already familiar with but has misclassified confusing.
and DE display
rates of 6. 1% and 15.5%,
To provide a qualitative comparison, Fig. 3 shows the map of class prediction probabilities for the committee constructed from samples with low VR values, whereas they exhibit disagreement on those with high VR values, as expected. This observation underscores the ability of to capture parameter snapshots within the same minima while generating diverse predictions on challenging examples with slight changes in the parameter space. Such a phenomenon is closely related to recent advancements in mode connectivity (Garipov et al., 2018 8) and fast ensembling methods (Huang et al., 2017; Izmailov et al., 2018; Maddox et al., 2019), which reveal that wandering around the wider optima leads to diverse yet more reliable predictions. Based on this empirical evidence, at discovering samples with high uncertainties which are predicted to be difficult for the classifier.
the parameter space. The commitiee members tend to agree on
conclude that SE is effective
CONCLUSION
In this paper, we demonstrate that estimating uncertainties of the predictions using works effi- ciently for uncertainty-based AL. Through extensive experiments with real-world image classifica- tion benchmarks, we empirically confirmed that the AL. with outperforms AL with or MCDO for various choices of acquisition functions. We further presented novel AL algorithm based on fine-tuning, where we keep a single model and continuously fine-tune it instead re-initializing the models at the beginning of every episode. The resulting algorithm could achieve comparable classification accuracies given the same number of acquired samples compared to the baselines with much smaller training steps. We provide detailed analysis of the effectiveness of SE for AL and show that SE builds model committees that yield diverse predictions that are useful for acquiring informative samples.

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We used the Pytorch (Paszke et al., 2019) library in our experiments and algorithms which are described in Algorithm and Algorithm 2. In addition, all experimental details and configurations on hyperparameters are recorded in Appendix B. We will provide an open-source implementation of AL environments and our code of and SE + algorithms.
ACKNOWLEDGEMENT
This work was partly supported by Institute of Information & communications Technology Plan- ning & Evaluation (ITTP) grant funded by the Korea government (MSIT) (No.2022-0-00184 Development and Study of AI Technologies to Inexpensively Conform to Evolving Policy on Ethics, and No.2019-0-00075. Artificial Intelligence Graduate School Program (KAIST)), and National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF- 2021M3E5D90025000
REFERENCES
R. Aljundi, M. Lin, B. Goujaud, and Y. Bengio. Gradient based sample selection for online continual learning. Advances neural information processing systems, 32, 2019.
A. Ashukha, A. Lyzhov, D. Molchanov, and D. Vetrov. Pitfalls of in-domain uncertainty estima- tion and ensembling in deep learning. In International Conference Learning Representations (ICLR), 2020.
W. H. Beluch, T. Genewein, A. Nürnberger, and J. M. Köhler. The power of ensembles for active learning in image classification. In Proceedings of the IEEE conference on computer vision and pattemn recognition, pages 9368-99689 06777777, 2018. 2, 6, 14, 17
D. M. Blei, A. Kucukelbir, and D. McAuliffe. Variational inference: a review for statisticians. Journal of the American Statistical Association, 112:859-877 2017.
C. Blundell, Cornebise, K. Kavukcuoglu, and D. Wierstra. Weight uncertainty in neural networks. In Proceedings of The 32nd International Conference on Machine Learning (ICML 2015), 2015.
F. D' Angelo and V. Fortuin. Replulsive deep ensembles are Bayesian. In Advances in Neural Information Processing Systems (NeurIPS 2021), 2021. 4
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), 2009. 7
Dosovitskiy, L. Beyer, A. Kolsenikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly. J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: transformers image recognition scale. In International Conference on Learning Representations (ICLR), 2021. 7
S. Fort, H. Hu, and Lakshminarayanan. Deep ensembles: loss landscape perspective. arXiv preprint arXiv:2106. 11642, 2021. 2, 2.4,
L. C. Freeman. Elementary applied statisties: for students in behavioral science. John Wiley and Sons, 1965.
Y. Gal and Z. Ghahramani. Dropout as a Bayesian approximation: representing model uncertainty in deep learning. Proceedings of The 33rd International Conference on Machine Learning (ICML 2016), 2017. 2,6
Y. Gal, R. Islam, and Z. Ghabramani. Deep bayesian active learning with image data. Interna- fional Conference Machine Learning, pages 1 1183-1192. PMLR, 2017. 2, 4,
10

---

Published as a conference paper at ICLR 2023
T. Garipov, P. Izmailov, D. Podoprikhin, D. Vetrov, and A. G. Wilson. Loss surfaces, mode connec- tivity, and fast ensembling of DNNs. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 2018. 2, 4,9
Y. Geifman and R. El-Yaniv. Deep active learning with neural architecture search. Advances in Neural Information Processing Systems, 32, 2019. 5
K. He, X. Zhang. S. Ren, and Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016. 6
N. Houlsby, F. Huzár, Z. Ghahramani, and M. Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5744 2011. 1, 6
G. Huang, Y. Li, G. Pleiss, Z. Liu, J. E. Hopcroft, and K. Q. Weinberger. Snapshot ensembles: train
1, get M for free. In International Conference on Learning Representations (ICLR). 2017. 4, 9
Izmailov, D. Podoprikhin, Garipov, D. Vetrov, and A. G. Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv: / 1803.05007, 2018. 6, 9, 13
H. Jung, Ju. M. Jung, and J. Kim. Less-forgetting learning in deep neural networks. arXiv preprint arXiv: /607.00/ 022 2016. 5
Kirkpatrick, R. Pascanu. N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, Quan, T. Ramalho, A. Trabska-Barwinskee et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, I (14(13):551-3322 2017.
A. Kirsch, J. Van Amersfoort, and Y. Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in neural information processing systems, 32, 2019. 6
C. Körner and S. Wrobel. Multi-class ensemble-based active learning. In European conference machine learning, pages 687-694. Springer, 2006.
A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. 6
B. Lakshminarayanam, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty esti- mation using deep ensembles. In Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017. 2, 4, 6
Y. Le and X. Yang. Tiny imagenet visual recognition challenge. Technical report, Stanford Univer- sity, 2015. 6
L. Le Folgoc, V. Baltatzis, S. Desai, A. Devaraj. S. Ellis, O. E. M. Manzanera, A. Nair, H. Qiu, Schnabel, and B. Glocker. dropout bayesian? arXiv preprint arXiv:2110.04286, 2021.
C. Louizos, K. Ullrich, and M. Welling. Bayesian compression for deep learning. In Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017.
W. J. Maddox, P. Izmailov, T. Garipov, D. P. Vetrov, and A. G. Wilson. A simple baseline for bayesian uncertainty deep learning. Advances in Neural Information Processing Systems, 32, 2019. 9
M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989. 5
P. Melville and R. J. Mooney, Diverse ensembles for active learning. In Proceedings of the twenty- first international conference on Machine learning, page 74, 2004.
Melville and R. J. Mooney. Creating diversity in ensembles using artificial data. Information Fusion, ((1):99-111, 2005. 8
P. Munjal, N. Hayat, M. Hayat, J. Sourati, and S. Khan. Towards robust and reproducible active learning using neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 223-232, 2022. 6, 16

---

Published as a conference paper at ICLR 2023
A. H. Murphy and R. L.. Winkler. Reliability of subjective probability forecasts of precipitation and temperature. Journal of the Royal Statistical Society: Series (Applied Statistics), 26( (6(1):41-47) 1977. 9
Y. Ovadia, E. Fertig, Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and J. Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 2019.
A. Paszke, S. Gross, Massa, A. Lerer, J. Bradbury, Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style. nigh-performance deep learning library, Advances in neural information processing systems, 32, 2019. 10, 17
V. Rakesh and Jain. Efficacy of bayesian neural networks in active learning. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2601-26011 2021. 2,
P. Ren, Y. Xiao, X. Chang, P.-Y. Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang. A survey of deep active learning. ACM Computing Surveys (CSUR), 54(9):1-40, 2021. 1,5
D. Rolnick, Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019. 5
B. Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009. 5
L. N. Smith and N. Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for cations, volume 1 11006, pages 369-386. SPIE, 2019. 17
operations appli-
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56); 1929-1958. 2014. 4
R. Wightman. Pytorch image models. 2019. 17
httpps///ggtaabb.cmm///i..ggggg
pytorch-image
A. G. Wilson and P. Izmailov. Deep ensembles as approximate Bayesian inference. https://
cims. . nyu. edu/ andrewgw dewpeneembleeseeeeeeeeeeeeeeeeee 2021. 4
D. Yoo and I. Kweon. Learning loss for active learning. Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 93-102, 2019. 17
12

---

Published as a conference paper at ICLR 2023
A
ADDITIONAL RESULTS AND DISCUSSIONS FOR SE
A.1
VGG-16
Table 4: Comparison between SE and FT with VGG-16 on CIFAR-10
A single model at the final episode
DE the final episode
10%
15%
20%
25%
10%
15%
20%
25%
SE
59.79.2.36
68.18
68.88
72.
1..
66.97
74.65
77.60
81.10
SE + FT
57.7813.31
65.22.2.21
68.58 10.80
72.50
68.43
74.84
78.20
81.45
We also used VGG-16 architecture for fine tuning experiments in Table Somewhat surprisingly, SE FT has caught with SE.
A.2
HYPERPARAMETERS FOR SE
Table 5: Test accuracy on different hyperparameters for SE
# snapshots S
10%
15%
20%
25%
30 %
5
7..7
81.55
86.17:0.05
88.34410.39
90.09110.4
10
11.91
81.05++0.26
85.65+00.3
88.6040.08
90.06:t0.20
20
70,49
81.27+0.62
0..04
88.68+0.11
90.45+00.50
50
71.29 10.76
80.36
85.30
88.64
1...4
90.52 10.11
SE learning rate Y)
10%
15%
20%
25%
30%
0.0001
69.66
80.25
84.95
87 87.42.10.48
88.8511..3
0.001
71.33:00.87
19.97
85.11
88.3340.21
$9.77=+000
0.005
74.76 1.15
81.34
85.73+0.12
88.87:0.05
90.34
0.01
:4.49
81.95
85.52+0.16
88.43+00.0
90.35=t0.22
starting point Nitess
10%
15%
20%
25%
30%
100
78.29
82.96:20133
84,89+0.13
85.6640.04
86.08+00.1
125
78.02
82.74
80.33
86.65.10.08
87.43.10.39
150
1..55
81.95
85.52
88.43 10.40
90.35 10.20
175
14.65=1143
82.09+0.44
85.66+00116
88. 14.3.77
89.75++0.6
Random
74.74.10.05
80.64
83.91
86.00
87.53.10.24
Table summarizes various settings of SE hyperparameters according to the proportion of labeled examples. Here, we only compare test accuracies of a single model trained with vanilla SGD on CIFAR-10 10 dataset with VR acquisition function. We additionally applied Stochastic Weight Aver- aging (SWA) (lzmailov et al., 2018) when training from scratch at the end to be less sensitive to hyperparameter settings and effectively compare the quality of queried examples.
Number of snapshots S. One may increase the number of snapshots collected to better acquire examples be labeled since SE does incur additional training costs. However, collecting more snapshots linearly increases the inference time for AL due to multiple forward passes. Overall, there no significant difference performance gains from the increase the number of snapshots. Increasing the number of snapshots results in lower performance than random at the beginning, but performance tends to increase as episodes continue. Based on these findings, one might come up with a strategy that uses a small number of snapshots at first and then increases the number of snapshots in latter episodes. All results used SE with VR acquisition function and were averaged over two trials including the ones with random acquisition. We used the same hyperparameter settings described in Appendix B. Here, the total epochs N and SE staring epoch N Nirrss are fixed to 200 and 150, respectively. The jump between snapshots differs accordingly.
13

---

Published as a conference paper at ICLR 2023
SE learning rate. In Algorithm 1 and Algorithm 2, learning rate is adjusted by the learning rate scheduler 77.,, and we used a high constant learning rate while collecting snapshots except for CIFAR-100 due to the instability of training. We maintain the same learning rate for CIFAR- 100. The high constant learning rate used when collecting snapshots contributes to diverse predictions and, consequently, the success of SE in the AT. context. The learning rate during training all fixed to 0.001. When the SE learning rate value was too small (1) 0.0001), the performance was inferior. However, too high learning rate may cause model diverge and move different mode values or meaningless areas in the weight space, which surely degrades the acquisition quality. We also include the previous result with CIFAR-100 dataset in Table 9, where we used 17 0.005 during SE. In our preliminary experiments, the VR acquisition function showed robustness despite decrease the accuracy of the model's predictions because it uses the count of members who agree rather than predicted probabilities, whereas other acquisition functions did not perform well. Here, the number of snapshots S and SE starting epoch Nikres are fixed to and 150, respectively. The total number of epochs N was also fixed as 200.
The starting point of SE Nitres We tried four different starting points of SE, or burn-in time, Nitres- This experiment shows how important to collect snapshots after the model sufficiently converges and why previous methods have failed. For example, Beluch et al. (2018) collected snap- shots from the beginning of the training (like at 40, 80, 120, 160, and 200 epoch). We showed that when snapshots were obtained before the model sufficiently converged (Nutres 100, 125), it per- formed even worse than the random acquisition. Similarly, when the jump between snapshots was too small (in case of =175 and therefore jump=5), the performance dropped in later episodes. Here, the number of snapshots S and SE learning rate were fixed to 5 and 0.01, respectively. The total number of epochs N was also fixed as 200.
A.3 SE WITH FINE- TUNING HYPERPARAMETEN,
SE FT algorithms governed mainly by the two hyperparameters: regularization hyperparameter and replay buffer size.
Table 6: Test accuracy on different regularization hyperparameter À for FT
lambda A
10%
15%
20%
25%
30%
0.0
77.66=0.10
82.45=0.22
86.01 #0.43
87.70=00.36
89.05=0.75
0.001
76.8610.40
82.83
86.27
18.12.0.19
88.4 88.45...11
0.005
76.9910.10
82.70
86.24
87.7990.06
89.12 19.12
0.01
77.09=0.02
82.86=0.13
85.78=0.59
87.79=0.10
89.56=+0.22
0.02
73.91
82.74-0.72
86.30=0.255
87.60=0.74
82.1448.42
Regularization hyperparamter >. The regularization hyperparamter À has a role of controlling the balance between maintaining a single trajectory from the previous episode and adapting the model to newly acquired samples. is crucial to find appropriate ||0 On I| Table 6 shows test accuracies at 10, 15, 20, 25, and 30 episode (with 5K, 7.5K, 10K, 12.5K, and 15K labeled examples, respectively) depending on different values on CIFAR-10 10 dataset. Here, the replay buffer size was fixed as 2,500. Although the acquisition quality is improved with regularization, there was no substantially outstanding value throughout the entire episodes. Therefore, in the experiments in Section 5, the À value was fixed 0.01.
values for fz-regularizer
Table 7: Test accuracy at episode 30 according to the replay buffer size on CIFAR-10
Replay buffer size
500
1,000
1,500
2,000
2,500
3,000
Ace (%)
87.79
88.26
88.57
89.30
89.39
88.91
Replay buffer size. In the case of the number of data used for fine tuning process, some of the data labeled in the previous episodes were added the newly acquired data. For CIFAR-10,
14

---

Published as a conference paper at ICLR 2023
used the budget size for 500. Moreover, add additional 2,000 randomly sampled examples from the previous labeled data. For both CIFAR-100 and Tiny ImageNet, we add 1,000 random sampled from labeled data to 1,000 newly acquired data. This has significantly reduced the training cost. Table 7 shows test accuracies of the final model (trained from scratch) with 15,000 labeled examples according to the size of replay buffer in each episode.
A.4 VISUALIZATIONS
5405
645
484
5.00
0
6375
5:500
0
0.345
Figure 4: Loss surface on set of (left) and DE with same initialization (right) for the visu- alization purpose in the parameter space when trained with the first 2,000 examples of CIFAR-10 dataset. Contours represent test accuracy, and red points denote weights gathered AL.
1.00
0.50
0.25
0.0
0.4
0.6
0.8
0.0
0.7
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
37.7%
55.7%
71.5%
81.8%
23.2%
47.0%
$1.4%
15.3%
48.2%
57.3%
65.6%
46.7%
0
examples
examples
examples
0.75
0.25
0.00
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
3000
1000 train examples
2000 examples
examples
SE
Figure 5: More boxplots and histograms for DE (up) and (down) plotted in the same manner as Fig. 2 (right), showing the tendency as the number of labeled examples increases.
Fig. 4 shows test accuracy map in the parameter space, and red crosses indicate parameters collected for ensemble. Note that two contours have different color scales. With high learning rate, SE picks snapshots that are weak individually but strong together around the wider optimum. Each member of DE falls into their own narrow optima.
15

---

Published as a conference paper at ICLR 2023
Similar Fig. 2 (right), Fig. 5 shows the correlation between the VR values and the predicted probabilities of the ground truth class, along with a histogram of VR values. In the histogram, the percentages denote the test error for each bin, and light gray areas depict the number of examples which the committee predicted incorrectly. Fig. 5 clearly illustrates that DE tends to exhibit overcon- fident results. Even when the number of acquired data increases, the degree of overconfidence by DE remains severe. However, VR scores calculated with SE showed much better correspondence with the actual error rate. Here, we trained the model the first {1000, 2000, 4000) CIFAR-IO examples in the train set.
B EXPERIMENTAL DETAILS
B. BASELINE DESIGN
When conducting our experiments, we placed great importance achieving robustness, repro- ducibility, and generalizability. Our experiments on CIFAR-I0 and CIFAR-100 datasets yielded average test accuracy scores of 90.15% and 68.02%, respectively, over five trials. We achieved these results using the VR acquisition function and acquiring 30% of the labels (equivalent to 15,000 ex- amples) under the settings outlined in Appendix B.3. In contrast, recent survey by Munjal et al. (2022) reported baseline results of 90.87% and 59.36% test accuracy with 40% random samples.
In order to facilitate reproducibility of results, provide not only the code and configurations, but also the indices of the queried examples. Reported runtimes are based on an Ubuntu 20.04 server with AMD Ryzen-4 5900X CPU and 64GB RAM, well as an NVIDIA RTX-3090 GPU with 24GB VRAM. For faster training on the Tiny-ImageNet dataset, we additionally employed FP16 (Automatic Mixed Precision).
B.2 ACQUISITION QUALITY
To ensure fair and objective comparison of acquisition quality, we devised an experiment in which the data selected by three different methods, namely SE, DE, and MCDO, were used to re-train both single model and ensembled model. By comparing the effectiveness of the selected data in learning, we aimed to provide a comprehensive evaluation the performance of each method. In cases where multiple experiments were conducted, carefully examined the results ensure that they exhibited a similar trend across experiments. We randomly selected one experiment and reported the re-trained results obtained from that experiment due the limited resources.
B.3 HYPERPARAMETERS
Table 8: Summary of hyperparameters
Datasei
Archilecture
Optimiier
Base Ir
Momentum
Weight decay
Schoduler
SE lr
Max spoch
SE epochs
# *naapshots
CIFAR-10
ResNet-18
SGD
0.001
0.9
0.01
ONECYCLE
0.01
200
50
VGG-16
SGD
0.001
0.01
ONECYCLE
0.01
100
100
CIFAR-100
ResNet-18
SGD
0.001
0.085
ONECYCLE
0.01
50
Tiny ImageNet
ResNet-18
SGD
0.001
0.9
0.0001
Steps
0.01
100
50
50
SGD
0,001
0,0001
ONECYCLE
0.05
50
25
ViT
Coustant
We used standard SGD optimizer with the following hyperparameters for both CIFAR- 10 and CIFAR-100 100 datasets: a base learning rate of 0.001, momentum of 0.9, and weight decay of 0.01. The mini-batch size was set to 64 for CIFAR-10 and 128 for CIFAR- 100. During SE, we raised the learning rate to 0.01 for CIFAR-10 or dropped it to 0.0001 for 100AR-100. In our preliminary experiments, we also tried increasing the learning rate to 0.005 during SE for CIFAR-I00, 100, but the results (shown in Table 9) were not good as those obtained with the final settings. Our results showed that SE with VR outperformed DE and MCDO, but other acquisition functions did not perform well with SE. Using learning rate lower than the base learning rate can help collect snapshots yielding decent predictions the training itself is unstable. In contrast, a learning rate higher than the base learning worked well in other hyperparameter settings and datasets.
16

---

Published as a conference paper at ICLR 2023
Table 9: Previous results on CIFAR-100
single model at the final episode
DE at the final episode
Acq Fa
Uncertainty
16%
18%
20%
22%
16%
18%
20%
22%
Runtime
SE
30
61.02:0.11
63.02
64.14
64.26
66.21
68.27
69.24
7.2 hr
FT
59.5610.58
61.17
62.88
64.81
64.35
66.23
67.68
69.72
0.8 hr
VR
DE
18..20
60.44
62.08 29
63 63.4620.355
63.78
65.99
67.99
69.22
28.7 hr
MCDO
58.81 20.35
60.6220.26
62.38 26
63.5020.17
63.23
65.09
66.80
67.69
6.2 hr
Random
58.93-00.00
60.58
61.95
62. 22.80
63.63
65.09
66.68
67.82
5.7 hr
To speed up convergence and reduce the effort of finding optimal hyperparameters, we used One Cycle learning rate scheduler (ONECYCLE) proposed by Smith and Topin (2019), setting max_1r to 0.01, for both datasets. For augmentations, we normalized images with the mean and variance of all images in the train set and applied random horizontal flip to both datasets. Additionally, random cropping was applied to CIFAR- 100. We trained the models for total of 200 epochs, and for SE, we collected five snapshots with an additional 50 epochs (10-epoch interval).
Table 10: Performance gain with ONECYCLE scheduler with 10,000 labeled examples on CIFAR- 10. The differences from random acquisition are in parentheses.
Acq Fn
w/ ONECYCLE
w/o ONECYCLE
VR
86.0% (A 1)9%p)
84.1% (A 2.9%p)
1.9%p
BALD
85.4% (A 1.3%p)
83.4% (A 2.2%p)
2.0%pp
ME
85.2% (A 1.1%p)
84.1% (A 2.9%p)
1.1%p
Random
84.1 %
81.2%
1.9%p
We also evaluated the effect of using ONECYCLE on the performance of various acquisition functions for CIFAR-10 10 with ResNet- 18 in Table 10. Without ONECYCLE, the performance gain compared to random acquisition of all acquisition functions decreased, with the differences ranging from 2.2%p to 2.9%p. However, when using ONECYCLF, the performance of all acquisition functions improved, but the differences from random acquisition decreased to a range of 1.1%p to 1.9%p. Despite the fact that the differences from random sampling were lower with ONECYCLE than without ONECYCLE, we chose to report the results with ONECYCLE as they are more relevant for practical applications with limited labeled examples. The reported accuracies are averaged over five trials.
For Tiny-ImageNet dataset with R8sNet-18, we also used a SGD optimizer with momentum of 0.9 and weight decay of 0.0001. We adjusted the learning rate to 0.1 for the first half of the training epochs, 0.01 until the 75% the training epochs, and 0.001 for the rest. During SE, the learning rate is increased to 0.05. For augmentations, we used random crops and random horizontal flips, which are the de-facto standard random augmentation strategies when reporting in academia. To reduce the computational overhead in ensemble-based acquisitions due multiple forward passes of large unlabeled pool U. we first randomly draw Q unlabeled examples from the pool and measured the scores, following Beluch et al. (2018) and Yoo and Kweon (2019). We set Q 10,000. The total number of training epochs is 100, and for SE we collected five snapshots with additional 50 epochs (10 epoch interval).
For transfer learning experiments on Tiny ImageNet, we used pretrained weights for ResNet-50 from Torchvision (Paszke et al., 2019) and those for ViT-hase-224 from PyTorch Image Models (Wight- man, 2019) and replaced the final linear classification head. Instead of using the original image size, an image was scaled up to 224 x 224 resolution to match the model trained with ImageNet dataset. The total number of training epochs is 50, and for SE we collected 5 snapshots with additional 25 epochs (5 epoch interval). We here also used a SGD optimizer with momentum of 0.9 and weight decay of 0.0001. For ResNet-50, did not use a learning rate scheduler (constant learning rate). We also set Q to 10,000. Please see Table 8 for summary of hyperparameters.
used ONECYCLE (the same above) and for ViT-
17

---

Published as a conference paper at ICLR 2023
B.4 MODEL STRUCTURES
For all experiments above, the structure of ResNet-18 model is slightly modified in order to fit 32 x 32 and 64 x 64 images following the standard protocol follows:
The kernel size of the first convolution layer (convl) has changed to 3, and the stride is changed to 1.
The max pooling layer is disabled.
For MCDO, a dropout layer with dropout rate p = 0.5 added front of the final linear classifier layer, since the original implementation has no dropout layers.
Similarly, the structure of VGG-16 model is slightly modified as follows:
The batch normalization layer added next every convolution layer.
The average pooling layer is disabled.
No additional dropout layer is attached to the model since its classifier already has two dropout layers with dropout rate p 0.5. We turned on the dropout layer when querying with MCDO.
These additional changes including MCDO layers had no effects on all reported figures in Section since we reported acquisition qualities by re- re-training models with the same structure with the exam- ples queried by each method.
18


Reviews:
[{"review_id": "QqQwqhh-UB", "reviewer": "Reviewer_qtq1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed a simple idea of using the snapshot ensembles instead of deep ensembles for active learning for uncertainty based methods. The paper compared their method with traditional DE and MCDO on three different dataset (CIFAR and Tiny ImageNet) with three differenty acquisition functions and showed that the SE + FT method achieved both a high accuracy as well as a low computation requirement.", "summary_of_the_review": "I think this is a very interesting paper that made a simple idea that other people have tried actually working. The experiments are sound and the clarity is good, overall a good paper.", "strength_and_weaknesses": "Strength:\n\nThe idea is simple. Combining SE to AL is not a new idea as I believe a lot of AL practitioners (like myself) training multiple models (DE) would think of possibly training less, like using FT. The problem is majority of us stop short when we see the fine-tuned accuracy is much lower than retraining from scratch, not realizing that the final accuracy is what we want....\n\nJust like the title, this simple idea is powerful as demonstrated in the experiments. The paper is well-written and easy to understand. The details are sufficient for the readers to follow and reproduce the results.\n\nAlso, just want to point out that Figure 1 is crucial for helping the reader as it conveys an important point that before the final episode, the model is at low accuracy. I couldn't understand why other fine-tuning efforts failed until I see Figure 1.\n\nWeakness / Suggestions:\nThe final analysis part is a bit weak (possibly due to the limitation of article length and the desire to squeeze things into 9 pages). I would suggest prioritizing Fig 2 and put Fig 3 (and more analyses like this) into the appendix.\nThe table 1 and 2 might require some caption to explain the 10%, 15%, 20% meaning. \nbest-performing results should be highlighted in Tables\nMinor typo on page 4: \"are to acquire acquire samples\"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Q34APMcXB9", "reviewer": "Reviewer_WKSq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to use snapshot ensemble (SE) to replace deep ensemble (DE) for modeling uncertainty in active learning. SE takes predictions from different epochs of training of a single model and thus is more efficient than DE. The authors compare SE with DE and MC-dropout on several basic AL acquisition functions including Variation Ratio, Max-entropy and BALD, and show that SE is comparable. They also use SE with continual learning where the model is not trained from scratch but is continuously finetuned after each acquisition. Such training strategy achieves comparative result with vanilla AL when using VR as acquisition.", "summary_of_the_review": "The paper proposes a simple modification to ensemble-based AL by relacing DE with SE and shows some preliminary results using basic AL strategies. Although the result on VR is promising, more thorough comparison with recent AL techniques is needed to prove the significance of the result. ", "strength_and_weaknesses": "Strength\n\n- The authors propose a simple modification of ensemble-based AL learning by using Snapshot ensemble and continual learning, which achieves comparable or better result compared to deep ensemble while reducing the computational cost, when used with simple uncertainty-based acquisition functions. \n- The finding that snapshot ensemble produces more diverse predictions than deep ensemble is interesting. \n\nWeakness\n\n- The novelty of the work is limited, as both the snapshot ensemble and continual active learning have been introduced previously, and there is no novel acquisition function or AL algorithm. \n- The authors only consider very basic AL methods (VR, Entropy, BALD) which are relatively old and has shown to be inferior than recent SOTAs. Moreover, the empirical experiments are incomplete, with full comparison among MC-dropout, DE, SE and continual-SE only available for VR, making the conclusion on superiority of SE unconvincing. \n- The authors use a batched AL setting; however, the chosen acquisition functions are not suitable for batched AL as it just greedily takes the top m examples (in fact BALD can be worse than random in batched setting). It would be more compelling if result on algorithms designed for Batched AL is shown.\n- The claim that prediction diversity is more important than model performance could be misleading, as overly diverse predictions could result in under-confidence, making choices of samples almost random. The right amount of uncertainty should be calibrated and informative of actual model uncertainty. No theoretical analysis is provided on why SE always produce better diversity. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4-sQmCTwzi", "reviewer": "Reviewer_s1VV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a more time efficient active learning method based upon epistemic uncertainty by incorporating snapshot ensembles rather than using the traditional method of deep ensembles. The experimental results demonstrate comparable recognition performance with a significantly lower runtime.", "summary_of_the_review": "The work appears novel and the experiments demonstrate the effectiveness of the method. The results seem somewhat incomplete at the SE+FT seems like the significant results that could be explored more. ", "strength_and_weaknesses": "The strength of the paper is that is well written, presents a reasonable somewhat novel approach for active learning, and the experimental results do show the potential of the approach.  The weakness of the paper is that both the deep ensemble and snapshot ensemble approaches are controlled by many various hyperparameters (e.g. number of samples, burn in time N_thresh, jumps between snapshots J, etc.).  These particular hyperparameters do not seemed to be discussed in the context of any hyperparameter search method in the paper or the appendices.  Choosing these hyperparameters judiciously seems important to appreciate the reduction in runtime. Also, the SE with fine tuning seems to be a significant advance over SE, but it is not fully evaluated in the experimental results.  It seems that the authors ran out of time to run the experiments, and the results are incomplete. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1dEr50fB8HT", "reviewer": "Reviewer_qYVA", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper combines two ideas: snapshot ensembles and sampling-based (uncertainty and mutual information) active learning. Snapshot ensembles are shown to empirically perform better than standard (deep) ensembles and Monte Carlo dropout, when used in conjunction with uncertainty sampling (and BALD). Furthermore, warm-starting (instead of training from scratch) is shown to work well empirically for data collection with active learning as long as the model is evaluated with a model trained from scratch.", "summary_of_the_review": "While \"only\" combining existing ideas, this paper experimentally finds two very practically useful observations that I think the research community will appreciate.", "strength_and_weaknesses": "Strengths:\n - The combination of techniques used in the experiments dramatically improves runtime with equal or better runtime.\n - The observation that warm-starting works for data selection (though not for the final model) is very interesting and important.\n - This paper's combination of recent ideas in parameter sampling with old ideas in active learning is very timely.\n\n\nWeaknesses:\n - It seems that some rows are missing for the experiments. For example, for CIFAR10, BALD with SE+FT, ME with MCDO, and ME with SE+FT are missing. Furthermore, it appears that MCDO and SE+FT are missing for the pre-trained models.\n - I'm curious why margin uncertainty sampling (best vs second best or BvSB) is not used. In the literature, this method usually outperforms entropy uncertrainty sampling.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster