A S IMPLE YET P OWERFUL D EEP A CTIVE L EARNING WITH S NAPSHOTS E NSEMBLES
Anonymous authors
Paper under double-blind review
A BSTRACT
Given an unlabeled pool of data and the experts who can label them, active learning aims to build an agent that can effectively acquire data to be queried to the experts, maximizing the gain in performance when trained with them.
While there are several principles for active learning, a prevailing approach is to estimate uncertainties of predictions for unlabeled samples and use them to define acquisition functions.
Active learning with the uncertainty principle works well for deep learning, especially for large-scale image classification tasks with deep neural networks.
Still, it is often overlooked how the uncertainty of predictions is estimated, despite the common findings on the difficulty of accurately estimating uncertainties of deep neural networks.
In this paper, we highlight the effectiveness of snapshot ensembles for deep active learning.
Compared to the previous approaches based on Monte-Carlo dropout or deep ensembles, we show that a simple acquisition strategy based on uncertainties estimated from parameter snapshots gathered from a single optimization path significantly improves the quality of the acquired samples.
Based on this observation, we further propose an efficient active learning algorithm that maintains a single learning trajectory throughout the entire active learning episodes, unlike the existing algorithms training models from scratch for every active learning episode.
Through the extensive empirical comparison, we demonstrate the effectiveness of snapshot ensembles for deep active learning.
I NTRODUCTION
The progress of deep learning is largely driven by data, and we often work with well-curated and labeled benchmark data for model developments.
However, in practice, such nicely labeled data are rarely available.
Many of the data accessible to practitioners are unlabeled, and more importantly, labeling such data incurs costs due to human resources involved in the labeling process.
Active
Learning ( AL ) may reduce the gap between the ideal and real-world scenarios by selecting the informative samples from the unlabeled pool of data, so after being labeled and trained with them, a model can maximally improve the performance.
The main ingredient of an AL algorithm is the acquisition function which ranks the samples in an unlabeled pool with respect to their utility for improvement.
While there are several possible design principles (Ren et al., 2021), in this paper, we mainly focus on the acquisition functions based on the uncertainty of the predictions.
Intuitively speaking, given a model trained with the data acquired so far, an unlabeled example exhibiting high predictive uncertainty with respect to the model would be a “confusing” sample which would substantially improve the model if being trained with the label acquired from experts.
A popular approach in this line is Bayesian Active Learning by Disagreement ( BALD ) (Houlsby et al., 2011), where a committee of multiple models predicts an unlabeled sample, and the degree of disagreement is measured as a ranking factor.
Here, the multiple models are usually constructed in a Bayesian fashion, and their disagreement reflects the epistemic (model) uncertainty about the prediction.
BALD is demonstrated to scale well for modern deep neural networks for high-dimensional and large-scale data (Gal et al., 2017).
Similar to BALD , many AL algorithms based on uncertainty employ a committee of models to estimate the epistemic uncertainty of the predictions.
The problem is, for deep neural networks trained with high-dimensional data, it is often frustratingly hard to accurately estimate the epistemic uncertainty.
Gal et al.
(2017) propose to use Monte-Carlo DropOut ( MCDO ) (Gal and Ghahramani, 2017), an instance of variational approximation to the posteriors and predictive, and similarly, Rakesh and
Jain (2021) suggest using more generic spike-and-slab variational posteriors (Louizos et al., 2017).
However, variational approximations tend to underestimate the posterior variances (Blei et al., 2017;
Le Folgoc et al., 2021), so the uncertainty-based acquisition functions computed from them may be suboptimal.
Alternatively, one can employ Deep Ensemble ( DE ) (Lakshminarayanan et al., 2017), where a single model is trained multiple times with the same data but with different random seeds for initialization and mini-batching.
While being simple to implement, DE works surprisingly well, surpassing most of the Bayesian Neural Networks ( BNN ) alternatives both in terms of accuracy and predictive uncertainty (Fort et al., 2021; Ovadia et al., 2019).
To this end, Beluch et al.
(2018) highlighted the effectiveness of DE as a way to estimate uncertainty for acquisition functions and demonstrated excellent performance.
A downside of DE is that it is expensive as it requires multiple models to be trained and maintained for inference.
As an alternative, a Snapshot Ensemble ( SE ) (Huang et al., 2017; Garipov et al., 2018) proposes to collect multiple model snapshots (checkpoints) within a single learning trajectory, rather than collecting them at the end of the multiple learning trajectories as in DE .
Compared to DE , with
SE , one can quickly construct a decent set of models without having to go over multiple training runs while not losing too much accuracy.
Inspired by the advantage of the SE , we study the use of the snapshot ensemble to AL .
Specifically, we estimate the epistemic uncertainties from SE and use them to evaluate the acquisition functions.
Through an extensive empirical comparison, we show that the AL based on SE significantly outperforms the existing approaches, even comparable to or better than the one with DE ; this is somewhat surprising since it is often reported that SE is less accurate than DE (Ashukha et al., 2020).
Moreover, based on this observation, we propose a novel AL algorithm that can greatly reduce the number of training steps until the final acquisition.
Typically, an AL algorithm alternates between acquiring labels based on a model and re-training the model with the newly acquired labels.
Here, for every re-training step, the old models are discarded, and a new model is trained from scratch.
Instead, we suggest maintaining a model on a single learning trajectory throughout the entire AL procedure and gathering snapshots from the trajectory to compute acquisition functions.
We show that this can significantly reduce the number of training steps without sacrificing too much accuracy.
We summarize our contributions as follows:
• We propose to use SE for the uncertainty-based acquisition functions for AL and demonstrate its effectiveness through various
empirical
evaluations.
• We propose a novel AL algorithm where a single learning trajectory is maintained and used to compute acquisition functions throughout the entire AL procedure.
We demonstrate that our algorithm could achieve decent accuracy with much fewer training steps.
B ACKGROUND 2. S ETTINGS AND BASIC ACTIVE LEARNING ALGORITHM
In this paper, we mainly discuss K -way classification problem, where the goal is to learn a classifier f ( · ; θ ) , parameterized by θ , taking an input x ∈ R d to produce a K -dimensional probability vector, that is, f ( x ; θ ) ∈ [0 , 1] K such that (cid:80)
Kk =
1 f k ( x ; θ )
= 1 .
To learn θ , we need a labeled dataset consisting of pairs of an input x and corresponding label y ∈ { 1 , . .
. , K } , but in AL , we are given only an unlabeled dataset U
= { x i } ni =1 without labels.
An AL algorithm is defined with a classifier model f
( · ; θ ) and an acquisition function a : R d → R measuring how useful an unlabeled example x is to the classifier f ( · ; θ ) .
Given f and a , an AL algorithm alternates between acquiring the labels for chosen unlabeled samples and training the classifier with the labeled samples.
A single iteration of acquiring samples and training the classifier is called an episode .
In the first episode, m samples are randomly chosen from U , and the labels are acquired for them to constitute an initial training set D train .
The classifier is then trained with D train to obtain θ 1 , or in case of the ensemble-based AL , a set of parameters { θ ( s ) 1 } Ss =1 .
The labeled samples are removed from U .
For all subsequent episodes t ≥ 2 , with the parameters { θ ( s ) t − 1 } Ss =1 from the previous episode, the samples remaining in U are ranked with the values of the acquisition function, and the top m of them are selected to be labeled.
The newly labeled m samples are then appended to the labeled training set D train , and the classifier is trained from scratch with the extended D train to obtain { θ ( s ) t } Ss =1 .
The algorithm terminates when it reaches the predetermined number of episodes
T , and the goal of AL is to maximize the accuracy of the classifier after the final episode.
2. A CQUISITION FUNCTIONS
Here, we review some popular choices for the acquisition functions, especially the ones based on predictive uncertainties.
We consider the following acquisition functions which estimate the uncertainty of the predictions via a set of parameters { θ ( s ) }
Ss =1 .
This set of parameters defines a committee of models { f ( · ; θ ( s ) ) }
Ss =1 .
Maximum Entropy ( ME ).
ME measures the predictive entropy of a given example x which can be approximated as,
[Equation]
Larger entropy means that the model is more uncertain about the prediction.
Variation Ratio ( VR ).
VR (Freeman, 1965) measures how certain the model is about its prediction for x , or how many of the committee agree with the prediction, and is calculated as 1 − f m /S , where f m is the frequency of a mode prediction over S committee members.
Similarly, Least Confident ( LC ) sampling chooses the least confident sample as 1 − max k p ( y = k | x , D train ) and Margin-based
Sampling ( MAR ) chooses samples with the smallest difference between the most probable and the second probable probabilities.
Bayesian Active Learning by Disagreement ( BALD ).
BALD (Houlsby et al., 2011) measures the mutual information between the label and the parameter given an input x
and the training data D train .
It measures epistemic uncertainty and can be interpreted as measuring disagreement among the predictions of the committee members.
BALD is maximized when each of the committee members is certain about their own predictions (small entropy H
[ y | x , θ ( s ) ] ), but the predictions disagree with each other, so the averaged prediction becomes uncertain (high entropy H [ y | x , D train ] ).
[Equation]
2. E STIMATING EPISTEMIC UNCERTAINTY .
In Bayesian AL , the model parameter θ is treated as a random variable with prior p ( θ ) and the posterior p ( θ |D train ) is approximated via a set of samples { θ ( s ) } Ss =1 .
There are several ways to approximate it.
Variational approximations.
For variational approximation, an easy-to-handle variational distribution q ( θ ) is introduced, learned to minimize D KL [ q ( θ )
∥ p ( θ |D train )] , and used as a proxy for p ( θ |D train ) .
That is, once obtained approximate distribution q ( θ ) , we draw θ (1) , . .
, θ ( S ) i .
i .
d . ∼ q ( θ ) .
A popular choice for q ( θ ) is mean-field Gaussian distribution (Blundell et al., 2015).
In AL literature, MCDO is widely used after (Gal et al., 2017), where the dropout (Srivastava et al., 2014) is applied to the model f
and the randomness due to the dropout is interpreted as an approximate posterior q ( θ ) .
While relatively simple to implement, the variational approximations are known to underestimate the posterior variances.
Deep ensembles.
DE (Lakshminarayanan et al., 2017) trains f ( · ; θ ) multiple times with the same
D train but with different random initializations to obtain { θ ( s ) } Ss =1 .
DE is simple to implement, and yet its performance is remarkable, achieving state-of-the-art across various applications.
The power of DE is mainly from its property to pick parameters from multiple modes (Fort et al., 2021), so the committee constructed from them yields a diverse set of predictions.
Even if it is not explicitly assuming the prior for p ( θ ) , DE can roughly be interpreted as an approximate Bayesian inference method (Wilson and Izmailov, 2021; D’Angelo and Fortuin, 2021), so the parameters { θ ( s ) } Ss =1 can be interpreted as posterior samples approximating the epistemic uncertainty of the model.
Snapshot ensembles.
DE is expensive, both for training and inference since it has to keep multiple models with different parameters.
SE (Huang et al., 2017; Garipov et al., 2018) is an idea to reduce the training cost of DE , where the multiple parameters { θ ( s ) } Ss =1 are gathered within a single training run rather than collected from multiple training runs.
To obtain diverse parameters, the learning rate of the training run is carefully chosen to encourage the optimization path to explore a wide area of the loss surface, and the parameter “snapshots” are periodically captured during the training run.
The SE usually underperforms DE with the same number of parameters gathered, but it can collect the parameters much faster since it requires only a single training run.
M ETHODS 3. A CTIVE LEARNING WITH SNAPSHOT ENSEMBLES
We first present an AL algorithm based on SE .
The idea is simple; during the classifier training stage of each episode, after a predefined number of steps, we periodically store the parameter snapshots with a high constant learning rate and use them to compute the acquisition function at the end of the episode.
Unlike the AL based on DE , SE does not incur any additional computation cost for training.
In the final episode, we have several options to choose from when we train the classifier using the acquired data D train .
As in vanilla AL , we may follow a single learning trajectory and pick the parameter at the last step as a point estimate.
Alternatively, we can apply DE to get an ensembled model.
Algorithm 1 summarizes the AL algorithm with acquisition functions based on SE .
Note that we are obtaining the final classifier with the vanilla Stochastic Gradient Descent ( SGD ) in Algorithm 1, but we may apply DE instead.
In Section 5, we show that this simple modification with SE , albeit no increase in the training and inference costs, significantly improves the performance, even outperforming the AL with DE .
3. A CTIVE LEARNING WITH SNAPSHOT ENSEMBLES AND FINE - TUNING
As described in Algorithm 1, the classifiers being trained for the intermediate episodes are used to compute the acquisition function and then thrown away.
However, considering that the acquisition function computed from a single learning trajectory works well, we may just keep a single learning trajectory throughout all episodes to collect snapshots on it.
That is, whenever an episode terminates, instead of throwing away the classifier parameter, use it as an initialization for the next episode.
We call this strategy Fine-tuning ( FT ) since the classifier training of each episode can be considered as fine-tuning from the result of the previous episode.
There are two things to note here.
First, since we are continuously fine-tuning a single model, we need much fewer training steps for each episode than the vanilla AL .
Second, when evaluated at the intermediate episodes, the classifier may be less accurate than the classifiers being trained from scratch, as in vanilla AL .
This is somewhat expected, but after all, what really matters is the accuracy of the classifier after the final episode .
We argue that the intermediate episodes are to acquire samples quickly for the final training, so the accuracy of the classifier during that process is not important.
Also, our AL procedure with fine-tuning is reminiscent of continual learning, in the sense that a single model is continually being trained for multiple episodes having different data.
To this end, we employ two commonly used tricks in continual learning literature.
Replay buffer.
In principle, for the purpose of fine-tuning, we may use only the newly acquired data from the previous episode for fine-tuning.
However, this would cause catastrophic forgetting
[Table]
Randomly draw m
U , remove them from and set
D train .
for t
. , T do
Randomly initialize θ 0 .
Set Θ ← ∅ .
for j = 1 , . . .
, N do Draw a mini-batch B from D train .
θ j ← θ j − 1 − η ( j ) ∇ θ J ( B , θ j − 1 ) .
if j ≥ N thres ∧ mod ( j − N thres , ⌊ N − N thres S ⌋ )
=
then Θ ← Θ ∪ { θ j } .
end endif t < T then
Compute a ( x , Θ ) for all x ∈ U .
Pick top m samples, remove them from U , and append them to D train .
else Set θ ∗ ← θ N .
end end them
Randomly initialize do if t
T then Randomly initialize θ end
Set Θ ← ∅ .
for j
, N do Draw a mini-batch B from D train .
θ j ← θ j − 1 − η ( j ) ∇ θ ( J ( B , θ j − 1 )+ λ 1 { t> 1 } ∥
θ
j − 1 − θ
∥ 2 ) .
if j ≥ N thres ∧ mod ( j − N thres , ⌊ N − N thres S ⌋ )
=
then Θ ← Θ ∪ { θ j } .
end endif t < T then
Compute
Θ all
U .
Pick
(McCloskey and Cohen, 1989), so the acquisition function based on it may be biased towards recently acquired data.
In order to prevent this, we adopt the idea of using a replay buffer (Jung et al., 2016; Rolnick et al., 2019; Aljundi et al., 2019), where we draw some portion of the data from the newly acquired data and the remaining portion from the past data.
We empirically find that this significantly improves the stability of the acquisition functions.
Regularization.
Similar to the replay buffer, we regularize the fine-tuning procedure to avoid deviating too much from the previous parameter.
That is, optimize the parameter θ with the ℓ 2regularizer ∥ θ − θ
∥ 2 where θ 0 is the starting point of the fine-tuning (the parameters passed from the previous episodes).
We also find that this regularization improves the quality of the acquisition functions, leading to better classification accuracy in the final episode.
Algorithm 2 summarizes the AL with fine-tuning.
The parts that are different from the AL without fine-tuning are marked as blue.
R ELATED WORKS
Active Learning.
Based on how an unlabeled example is fed into an AL agent, AL can broadly be categorized into membership query synthesis, where the agent even generates examples in the sample space, stream-based selective sampling, where the agent decides whether a given input is helpful if labeled in online settings, and pool-based AL , where the agent can access a large unlabeled pool (Settles, 2009).
Pool-based AL can be further divided into uncertainty-based approach, diversitybased approach, and hybird approach (Ren et al., 2021).
Geifman and El-Yaniv (2019) first introduced neural architecture search into AL , claiming that the over-parameterized model could lead to overfitting especially in the earlier episodes with fewer labeled examples, and therefore, uncertainty estimates could be inaccurate.
Similarly, Munjal et al.
(2022) argued that the optimal hyperparame-
[Table] [Table]
Ensemble.
Ensembles of neural networks have shown competitive performance improvement and are widely used in machine learning and deep learning.
In addition, it shows improvements in estimates for predictive uncertainty.
DE (Lakshminarayanan et al., 2017) is one of the well-behaved methods estimating the uncertainty of deep neural networks.
This method works with several classifiers trained with the same dataset and architecture but with different seeds for a random number generator.
However, the biggest limitation of ensemble is its computational costs because it requires training multiple models (Izmailov et al., 2018).
Active Learning with Ensemble.
The uncertainty of an example for neural networks can be estimated by using an ensemble of neural networks in the context of AL .
Gal et al.
(2017) used BALD (Houlsby et al., 2011) on MCDO (Gal and Ghahramani, 2017) and later extended it to batch settings where it considers overlaps among data points to be acquired (Kirsch et al., 2019).
However, due to the expensive cost of training multiple models, most research on ensemble-based AL had been restricted to the traditional ML algorithms (Melville and Mooney, 2004; K¨orner and Wrobel, 2006).
Beluch et al.
(2018) compares the performance of various acquisition functions and uncertainty estimation methods on large-scale image classification tasks, proving that DE consistently outperforms other uncertainty-based methods such as MCDO (Gal et al., 2017) or a single model.
In the same manner, Bayesian neural networks show much robustness and reliability than MCDO in the context of AL with continual learning (Rakesh and Jain, 2021).
E XPERIMENTS
In this section, through an extensive empirical comparison on three image classification benchmarks (CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009), and Tiny ImageNet (Le and Yang, 2015)), we would like to demonstrate the followings.
• Measuring epistemic uncertainty via SE for AL is effective, comparable, or even better than the one based on DE across various choices of uncertainty-based acquisition functions.
• AL with SE + FT can significantly reduce the training cost without sacrificing too much accuracy.
• The reason why SE is effective for AL is that it can build a committee of models yielding diverse predictions.
Interestingly, for the purpose of acquiring samples of better qualities, the diversity in predictions of the models is more important than the accuracy of the models
We compare the AL algorithms with three variables - acquisition functions ( VR , ME , BALD ), algorithms to measure uncertainty ( DE , SE , MCDO ), and how to train the classifier in the final episode (single model via a vanilla SGD , DE ).
We report the results with ResNet-18 (He et al., 2016).
Please refer to Appendix B for more details, such as experimental protocols or hyperparameter settings.
The test accuracy results on CIFAR-10, CIFAR-100, and Tiny ImageNet are summarized in Table 1,
Table 2, and Table 3, respectively, according to the proportion of labeled examples.
Due to the limited resources, we report the results with three acquisition functions for CIFAR-10 and the results with only VR for CIFAR-100 and Tiny ImageNet.
5. A NALYSIS OF THE MAIN RESULTS
Effectiveness of SE for AL .
Table 1 confirms that the measuring uncertainty with SE acquires the samples leading to the best classification accuracies, regardless of the choice of acquisition functions or how we train the final classifier.
This is somewhat remarkable, considering that the runtimes of SE -based methods are not significantly longer than those for the random baselines, while DE requires much longer runtimes due to its requirement to train and test with multiple models.
It is also notable that the choice of uncertainty estimation method was much more important than the choice of acquisition functions.
A similar trend is observed for CIFAR-100 and Tiny ImageNet; overall, SE leads to the best classification accuracies.
We set S = 5 for SE and DE and S = 25 for
MCDO .
We repeated the experiment five times, and the reported means and standard deviations were averaged over five trials and one trial for retraining a single model and DE , respectively.
[Table] [Table]
Effectiveness of fine-tuning.
Tables 1 to 3 compare the AL with SE + FT (Algorithm 2) to the baselines with VR as acquisition functions.
Somewhat surprisingly, SE + FT achieved comparable or even better test accuracies given much shorter runtimes.
Fig.
1 shows the progression of test accuracies during the episodes of SE + FT procedure.
Before the final episode, the intermediate models (used only for the acquisitions) exhibit lower accuracies.
This is somewhat expected as they are fine-tuned and trained with fewer examples for less number of epochs.
At the end of the final episode, where a new classifier is trained from scratch, the test accuracy catches up with the ordinary AL , indicating that even if the accuracies of the intermediate classifiers are inferior, the samples acquired with them are good enough to obtain the decent classifier at the final episode.
We set S = 5 for SE + FT , and the reported means and standard deviations were averaged over five trials and one trial for retraining a single model and DE , respectively.
AL with pretrained models.
As one can see from Table 3, the performances of the methods on
Tiny ImageNet are generally bad, presumably due to the complexity of the dataset.
To this end, we employed models pretrained from ImageNet (Deng et al., 2009) and tested whether the AL algorithms could fine-tune on pretrained models.
We tested ResNet-50 and vision transformer (Dosovitskiy et al., 2021) backbones and compared SE -based and DE -based AL with VR acquisition.
For both backbones, SE significantly outperformed DE , especially with the small number of acquisitions.
Here, we set S = 5 for SE and S = 3 for DE , and the reported means and standard deviations were averaged over three trials and one trial for retraining a single model and DE , respectively.
5. A NALYSIS OF THE UNCERTAINTY ESTIMATION .
In this section, we analyze SE in more detail to see why it is more effective than DE for AL .
We conjecture that SE builds a committee of models whose predictions are more diverse than the ones
[Table]
[Figure]
CIFAR-10, CIFAR-100 and Tiny ImageNet
ResNet
Note before the show low from other methods within a single trajectory, and this is a key to its success in AL .
To verify this, we measure average KL-divergences and pair-wise disagreement values (Melville and Mooney, 2005) of the predictions computed from SE , DE , and MCDO .
The disagreement between two class probabilities f ( · ; θ ( i ) ) and f ( · ; θ ( j ) ) on an example x is calculated as
[Equation]
As summarized in Fig.
2 (left), SE generally exhibits much higher KL-divergences and disagreement values among their predictions compared to DE and MCDO .
As reported in the literature (Fort et al., 2021), DE shows higher disagreements than MCDO , but still much less than SE .
We believe that this is mainly due to the nature of AL where we usually work with a relatively small amount of data and less number of training steps than in typical supervised learning settings.
DE parameters are collected at the end of each training run, so the models are likely to reach the local optimum.
On the other hand, SE collects the parameter snapshots during a single training run, so some of them may not converge to the local optimum.
This degrades the classification accuracy, but as we have pointed out several times, for the purpose of acquisition, the diversity in predictions within a single trajectory is more important than the accuracy of the individual models.
Fig.
2 (right) shows the correlation between the VR values and the predicted probabilities of the ground truth class, along with a histogram of VR values.
The percentages denote the test error for each bin, with the number of correct examples visualized as light gray areas over the histograms.
You can consider them as an unnormalized reliability diagram (Murphy and Winkler, 1977).
For example, in the bin with VR = 0 .
8 (all disagree), SE shows 72.1% test error, while DE shows 46.7%.
However, in the bin with VR = 0 . 0 (consensus), SE and DE shows 6.1% and 15.5% error rate,
[Figure]
R
[Figure]
p t r u e
[Figure] [Figure]
respectively.
Expected calibration error ( ECE ) (Naeini et al., 2015) based on VR scores (considered as hard-voting) is 0.025 for SE and 0.166 for DE .
Since unlabeled examples are labeled in the order of high VR scores, DE is more likely to choose redundant examples that the model is already aware of but misunderstood as confusing ones.
As a qualitative comparison, in Fig.
we show the map of class prediction probabilities for the committee constructed from SE .
For the samples with low VR values, the committee members tend to agree with each other, but for those with high VR values, the members disagree with each other as expected.
This illustrates that SE manages to collect parameter snapshots within the same minima, while exhibiting diverse predictions on hard examples by slight changes in parameters.
This observation is closely related to recent progress in mode connectivity (Garipov et al., 2018) and fast ensembling methods (Huang et al., 2017; Izmailov et al., 2018; Maddox et al., 2019) in that we get diverse, yet more reliable predictions by wandering around the wider optima.
With these empirical evidences, we can conclude that SE finds more samples with larger VR values that are actually predicted to be confusing samples for the classifier.
C ONCLUSION
In this paper, we demonstrate that estimating epistemic uncertainties of the predictions using SE works efficiently for uncertainty-based AL .
Through extensive experiments with real-world image classification benchmarks, we empirically confirmed that the AL with SE outperforms AL with DE or
MCDO , for various choices of acquisition functions.
We further presented a novel AL algorithm based on fine-tuning, where we keep a single model and continuously fine-tune it instead of re-initializing the models at the beginning of every episode.
The resulting algorithm could achieve comparable classification accuracies given the same number of samples acquired compared to the baselines with a much less number of training steps.
We provide a detailed analysis of the effectiveness of SE for AL and show that SE builds model committees that yield diverse predictions that are useful for acquiring informative samples.
R EPRODUCIBILITY STATEMENT
We used the Pytorch (Paszke et al., 2019) library in our experiments and algorithms which are described in Algorithm 1 and Algorithm 2.
In addition, all experimental details and configurations on hyperparameters are recorded in Appendix B. We will provide an open-source implementation of AL environments and our code of SE and SE + FT algorithms
A A DDITIONAL R ESULTS AND D ISCUSSIONS FOR SE A. VGG-
We also used VGG-16 architecture for fine tuning experiments in Table 4.
Somewhat surprisingly,
SE + FT has caught up with SE .
A. H YPERPARAMETERS FOR SE
Table 5 summarizes various settings of SE hyperparameters.
Number of snapshots S .
One may increase the number of snapshots collected to better acquire examples to be labeled since SE does not incur additional training costs.
However, collecting more snapshots linearly increases the inference time for AL due to multiple forward passes.
Tableshows the test accuracy results on different number of snapshots.
Overall, there is no significant difference in performance gains from the increase in the number of snapshots.
Increasing the number of snapshots results in lower performance than random at the beginning, but performance tends to increase as episodes continue.
Based on these findings, one might come up with a strategy that uses a small number of snapshots at first and then increases the number of snapshots in latter episodes.
All results use SE with VR acquisition function and are averaged over two trials including ones with random acquisition.
We used as the same hyperparameter settings as ones described in Appendix B.
Here, the total epochs N and SE staring epoch N thres are fixed to 200 and 150, respectively.
The jump between snapshots differs accordingly.
SE learning rate.
In Algorithm 1 and Algorithm 2, a learning rate is adjusted by the learning rate scheduler η ( . ) , and we used a high constant learning rate while collecting snapshots.
The high constant learning rate used when collecting snapshots contributes to the success of SE in the
[Table] [Table]
AL context.
For simple implementation
[Table]
we multiply specific value to initial learning rate.
Tableshows test accuracy on different high constant learning rate while collecting snapshots.
When the SE learning rate value is too small, it is difficult to improve the performance because sufficient diverse predictions are not obtained.
In the case of high constant learning rate, it contributes to yielding diverse predictions which lead to performance gain.
However, too high learning rate may cause a model to diverge and move to different mode values or meaningless areas, which surely degrades the acquisition quality.
The number of snapshots S and starting epoch SE are fixed to 5 and 150, respectively.
The staring point of SE N thres .
We have four choices for the starting epoch of SE , or burn-in time, N thres .
This experiment shows how important it is to SE after the model converges and why previous methods failed.
According to Beluch et al.
(2018), they collected snapshots when the epoch is same with the total number of epoch / number of snapshots which is exactly same with the original snapshot ensemble.
Like these, if the snapshot ensemble is obtained before the model sufficiently converges, the performance may be good in the early stages of AL, as shown in starting pointand 125 in Table 5, but the performance degrades as it goes.
The number of snapshots S and SE learning rate are fixed to 5 and 0.01, respectively.
A. SE WITH F INE - TUNING H YPERPARAMETERS
SE + FT algorithms are governed mainly by the two hyperparameters: regularization hyperparameter and replay buffer size.
Regularization hyperparamter λ .
The regularization hyperparamter λ has a role of controlling the balance between maintaining a single trajectory from the previous episode and adapting the model to newly acquired samples.
It is crucial to find appropriate λ values for ℓ 2 -regularizer ∥ θ − θ
∥ 2 .
Table 6 shows test accuracies at 10, 15, 20, 25, and 30 episode (with 5K, 7.5K, 10K, 12.5K, and 15K labeled examples, respectively) depending on different λ values.
Here, the replay buffer size was fixed as 2,500.
Although the acquisition quality is improved with regularization, there was no substantially outstanding λ value throughout the entire episodes.
Therefore, in the experiments in Section 5, the λ value was fixed as 0.01.
Replay buffer size .
In the case of the number of data used for fine tuning process, some of the data labeled in the previous episodes were added to the newly acquired data.
For CIFAR-10, we used budget size for 500.
Moreover, we add additional 2,000 randomly sampled data from previous labeled data.
For both CIFAR-100 and Tiny ImageNet, we add 1,000 random sampled from labeled data to 1,000 newly acquired data.
In Table 7, there are the test accuracy according to the number of training data when it trained from scratch at episode 30 (15,000 labeled examples).
[Figure]
A. F URTHER A NALYSIS AND D ISCUSSIONS
Fig.
4 shows test accuracy map in the parameter space, and red crosses indicate parameters collected for ensemble.
Note that two contours have different color scales.
With a high learning rate, SE picks snapshots that are weak individually but strong together around the wider optimum.
Each member of DE falls into their own narrow optima.
Same as Fig.
2 (right), Fig.
5 shows the correlation between the VR values and the predicted probabilities of the ground truth class, along with a histogram of VR values.
In the histogram, the percentages are the test error for each bin and light gray areas are the number of correct examples.
Fig.
5 clearly shows that DE shows overconfident results.
It can be seen that as the number of acquired data increases, the degree of overconfident that DE showed becomes relatively more severe than that of
SE.
B E XPERIMENTAL D ETAILS B. B ASELINE D ESIGN
When we designed the experiments, our priorities were the robustness, reproducibility, and generalizability.
For CIFAR-10 and CIFAR-100, we scored 90.15% and 68.20% test accuracy (averaged over five trials), respectively, with VR acquisition function until acquiring 30% of the labels with the settings mentioned above, while Munjal et al.
(2022) reported a baseline with 90.87% and 59.36% test accuracy with 40% random samples.
We will not only release code and configurations, but also disclose the indices of examples queried by acquisition functions and strategies on each dataset for reproducibility.
Reported runtimes are based on an Ubuntu 20.04 server with AMD Ryzen-4 5900X CPU with 64GB RAM, NVIDIA
RTX-3090 GPU with 24GB VRAM.
We additionally used FP16 (Automatic Mixed Precision) for
Tiny-ImageNet for faster training.
B. A CQUISITION Q UALITY
For fair comparisons, we designed an experiment to measure the acquisition quality.
The data selected by SE , DE , and MCDO were retrained in the single model and the ensemble model, respectively, to compare how effective the selected data showed in learning.
The experimental details are the same as Appendix B.3.
B. H YPERPARAMETERS
We used a mini-batch of size 64 and a standard SGD optimizer with a learning rate of 0.001, momentum of 0.9, and weight decay of 0.01 for both CIFAR-10 and CIFAR-100 dataset.
During
SE , a learning rate was raised to 0.01 and 0.005 for CIFAR-10 and CIFAR-100, respectively.
For faster convergence and reducing efforts in finding optimal hyperparameters, One Cycle learning rate scheduler ( ONECYCLE ) proposed by Smith and Topin (2019) with max lr set to 0.01 has been adopted for CIFAR-10 and CIFAR-100 datasets.
For augmentations, we normalized images with the mean and the variance of all images in the train set and applied random horizontal flip for both datasets.
For CIFAR-100, random cropping was also applied.
The total number of training epochs is 200, and for SE we collected five snapshots with additional 50 epochs (10 epoch interval).
Without ONECYCLE , the performance dropped from 84.1% to 81.2% for Random, 86.0% ( ∆ 1.9%p) to 84.1% ( ∆ 2.9%p) for VR , 85.4% ( ∆ 1.3%p) to 83.4% ( ∆ 2.2%p) for BALD , 85.2% ( ∆ 1.1%p) to 84.1% ( ∆ 2.9%p) for ME at episode 20 (10,000 labeled examples) on CIFAR-10 with ResNet-18, respectively (averaged over five trials.
In parentheses are the differences between random sampling and the acquisition function).
Although the differences are almost twice as ones without ONECYCLE , we decided to report the figures with the higher performance in that it is more suitable to real-world applications with less number of labeled examples.
For Tiny-ImageNet dataset with ResNet-18, we also used a SGD optimizer with momentum of 0.9 and weight decay of 0.0001.
We adjusted the learning rate to 0.1 for the first half of the training epochs, 0.01 until the 75% of the training epochs, and 0.001 for the rest.
During SE , the learning rate is increased to 0.05.
For augmentations, we used random crops and random horizontal flips, which are the de-facto standard random augmentation strategies when reporting in academia.
To reduce the computational overhead in ensemble-based acquisitions due to multiple forward passes of a large unlabeled pool U , we first randomly draw Q unlabeled examples from the pool and measured the scores, following Beluch et al.
(2018) and Yoo and Kweon (2019).
We set Q to 10,000.
The total number of training epochs is 100, and for SE we collected 5 snapshots with additional 50 epochs ( epoch interval).
For transfer learning experiments on Tiny ImageNet, we used pretrained weights for ResNet-50 from
Torchvision (Paszke et al., 2019) and those for ViT-base-224 from PyTorch Image Models (Wightman, 2019) and replaced the final linear classification head.
Instead of using the original image size, an image was scaled up to 224 × 224 resolution to match the model trained with ImageNet dataset.
The total number of training epochs is 50, and for SE we collected 5 snapshots with additionalepochs (5 epoch interval).
We here also used a SGD optimizer with momentum of 0.9 and weight
[Table]
decay of 0.0001.
For ResNet-50, we used ONECYCLE (the same as above) and for ViT-base, we did not use a learning rate scheduler (constant learning rate).
We also set Q to 10,000.
Please see
Table 8 for a summary of hyperparameters.
B. M ODEL STRUCTURES
For all experiments above, the structure of ResNet-18 model is slightly modified in order to fit 32 × 32 and 64 × 64 images following the standard protocol as follows:
• The kernel size of the first convolution layer (conv1) has changed to 3, and the stride is changed to 1.
• The max pooling layer is disabled.
• For MCDO , a dropout layer with dropout rate p = 0 .
5 is added in front of the final linear classifier layer, since the original implementation has no dropout layers.
Similarly, the structure of VGG-16 model is slightly modified as follows:
• The batch normalization layer is added next to every convolution layer.
• The average pooling layer is disabled.
• No additional dropout layer is attached to the model since its classifier
already has two dropout layers with dropout rate p = 0 .
We turned on the dropout layer when querying with MCDO .
These additional changes including MCDO layers had no effects on all reported figures in Sectionsince we reported acquisition qualities by retraining the same model with the queried examples.