SAAL: Sharpness-Aware Active Learning
Authors: Yoon-Yeong Kim
JoonHo Jang
Byeonghu Na
Yeongmin Kim
Kyungwoo Song
Wanmo Kang
Il-chul Moon
Venue: Submitted to ICLR 2023 (2023)
Keywords: active learning
loss sharpness
SAM

Abstract: While modern deep neural networks play significant roles in many research areas, they are also prone to overfitting problems under limited data instances. Particularly, this overfitting, or generalization issue, could be a problem in the framework of active learning because it selects a few data instances for learning over time. To consider the generalization, this paper introduces the first active learning method to incorporate the sharpness of loss space in the design of the acquisition function, inspired by sharpness-aware minimization (SAM). SAM intends to maximally perturb the training dataset, so the optimization can be led to a flat minima, which is known to have better generalization ability. Specifically, our active learning, Sharpness-Aware Active Learning (SAAL), constructs its acquisition function by selecting unlabeled instances whose perturbed loss becomes maximum. Over the adaptation of SAM into SAAL, we design a pseudo labeling mechanism to look forward to the perturbed loss w.r.t. the ground-truth label. Furthermore, we present a theoretic analysis between SAAL and recent active learning methods, so the recent works could be reduced to SAAL under a specific condition. We conduct experiments on various benchmark datasets for vision-based tasks in image classification and object detection. The experimental results confirm that SAAL outperforms the baselines by selecting instances that have the potentially maximal perturbation on the loss.

Under review as a conference paper at ICLR 2023
SAAL: SHARPNESS-AWARE ACTIVE LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
While modern deep neural networks play significant roles in many research areas, they also prone overfitting problems under limited data instances. Partic- ularly, this overfitting, or generalization issue, could be a problem in the frame- work of active learning because it selects few data instances for learning over time. To consider the generalization, this paper introduces the first active learning method to incorporate the sharpness of loss space in the design of the acquisition function, inspired by sharpness- sharpness-aware aware minimization (SAM). SAM intends to max- imally perturb the training dataset, so the optimization can be led to a flat minima, which is known to have better generalization ability. Specifically, our active learn- ing, Sharpness-Aware Active Learning (SAAL., constructs its acquisition function by selecting unlabeled instances whose perturbed loss becomes maximum. Over the adaptation of SAM into SAAL, we design a pseudo labeling mechanism to look forward to the perturbed loss w.r.t. the ground-truth label. Furthermore, we present a theoretic analysis between SAAL and recent active learning methods, so the recent works could be reduced to SAAL under specific condition. We con- duct experiments on various benchmark datasets for vision-based tasks in image classification and object detection. The experimental results confirm that SAAL outperforms the baselines by selecting instances that have the potentially maximal perturbation on the loss.
INTRODUCTION
Recently, deep learning is widely utilized in many research areas, such computer vision, natural language processing, recommender systems, etc., but its success deeply depends on the large-scale labeled dataset for training the deep neural networks. The importance of the dataset is related to the generalization issue in deep learning, which refers that the model learned with the training dataset suffers from the degradation of performance when the unseen test dataset is encountered for deploy- ment. This degradation results from the neural networks that are prone to overfitting under the lack of the training dataset (Keskar al. 2016 Neyshabur et 2017 Kawaguchi al. 2017
The dependency on the dataset also invokes an adaptive data selection by acquisition functions, or active learning, which aims at the efficient use of the limited budget for annotations from oracle Cohn et al., 1996; Tong 2001; Settles] 2009).. Recently. various methods for active learning have been proposed; but the model trained with small number of data from the adaptive selection is often difficult to be generalized (Dasgupta & Hsu 2008). Although there exist some prior works that deal with the generalization issue in active learning: those methods solve the problem by either proposing a new risk function (Farquhar et al.) (020) or adopting a new classifier network (Wan) et al. (02T), rather than by inventing a new acquisition function that considers the generalization.
In this paper, we propose a new active learning algorithm, named Sharpness-Aware Active Learning (SAAL), that connects active learning and generalization ability to construct the acquisition function. Specifically, we inspired by Sharpness- Aware Minimization, or SAM (Foret et al. 2020), which minimizes the maximally perturbed loss of training dataset, leading to minimizing the loss sharpness as well the task loss, itself. Such optimization leads to a flat minima of the loss landscape, which is shown to have a strong correlation with the generalization performance (Jiang et al. 2019). Hence, SAAL adopts the maximally perturbed loss the acquisition score.
When calculating the acquisition score for SAAL, we cannot observe the labels for the unlabeled instances, it is infeasible compute the perturbed loss. To overcome this challenge, utilize

---

Under review as a conference paper at ICLR 2023
pseudo labels predicted by the current model, and we theoretically show that proposed pseudo labeling conservatively estimates the maximally perturbed loss w.r.. ground-truth label. Also, we theoretically derive the upper bound of the acquisition score of SAAL, which includes the loss, the norm of gradients, and the first eigenvalue of loss Hessian. Among the three terms of the up- per bound, the loss and gradient terms widely used metrics for active learning, which captures the model change by acquiring the instance (Yoo & Kweon 2019; Ash et al. [2000] (Sttles et al.) 2007). Meanwhile, the first eigenvalue, which is newly considered by SAAL, is connected to the loss sharpness (Keskar et al. 2017). Therefore, the selected instances by SAAL might contribute to the generalization of the model.
We summarize our contributions in three points. First, we propose Sharpness-Aware Active Learn- ing (SAAL), which considers the loss sharpness for constructing the acquisition function. The loss sharpness is related to the generalization of model, so selecting instances with a high value of loss sharpness might lead model with a better generalization performance. Second, we theoretically derive the upper bound of the acquisition score of SAAL and show the connection with the recent active learning methods. Specifically, we find that the upper bound also contains the first eigenvalue of loss Hessian, which is related to the generalization ability. Third, we empirically show that SAAL outperforms the baselines in various vision-base tasks on the benchmark dataset.
2
PRELIMINARIES
2.1 NOTATIONS
Throughout this paper, we assume a classification problem and we represent our current deep learn- ing model parameterized by 0 as for Rd -, IR s1; where d is the dimension of data instance, X, and y/ is the set of candidate classes that x can have. There are two datasets: a dataset with labels, Lr, and the other unlabeled dataset, Zv. We denote the acquisition function of active learning as faeg: Rd + R. where facy receives data instance as input, and calculates the informativeness, or the acquisition score, of the instance as output. The loss of data instance, z, w.r.t. the given label y is represented as 1(z, y; f) = (cn(a (c v), where (-)) is a softmax function. The total loss of dataset, S, is represented as (s) N
Σ Dimi 10 y43 0). where S = { y: 1,..., N). Lastly, we define the pseudo label, = argmax jea (ff()))); and we denote the ground-truth label as y.
2.2 ACTIVE LEARNING
There are several active learning scenarios that differ by the setting of data accessibility; which include membership-quer synthesis (Angluin) 1988 2004), stream-based active learning (Atlas) et al. 1989; Cohn al. 1994), and bose-based active learning (Lewis & Gale, 1994). In this paper, we focus on pool- based active learning, where the unlabeled data instances are provided as a large set of data pool, and the active learner sequentially selects the informative instances by certain criterion.
Pool-based active learning is categorized by the definition of informativeness, which includes the uncertainty, diversity, and hybrid-based methods. Uncertainty-bast acquisition function, faog. to calculate the uncertainty of each unlabeled instance with regard the current deep learning model, and an oracle provides the ground ground-truth truth label of the selected unlabeled instances with the highest uncertainty. Since the acquisition score is usually calculated for an unla- beled instance, Ia LV, w.r.t. the current model, fo. it is expanded as Sacy(Tui So), resulting in the selection rule as the below.
active learning adopts the
Vg argmax
Σ
Saca fo)
C. F.E.'
Entropy, which is denoted as fay (x fo) HI (x = -E, log, (x)(x.)) or variation ratio, which denoted as /Var for calculating uncertainty (Shannon, 1948; Freeman 1965). These days, additional networks used to approximate the uncertainty of each instance. Learning Loss for Active Learning (LL4AL)
max, o (o(( )), are the most widely used methods
Yoo & K Kweon 2019) trains the loss prediction module, fLPM, which takes the hidden feature maps
2

---

Under review as a conference paper at ICLR 2023
as input and predicts the expected loss as output. Then, LL4AL constructs the acquisition functions fI.1.41 (Ix) = fLPM (5g (E.)lk=......))) where fg is the k-thh hidden feature map. Variational Adversarial Active Learning (VAAL) (Sinha et al.) (219)) trains discriminator, fdis. which takes data instance as input and discriminates whether the instance belongs to the labeled dataset or the unlabeled dataset. Then, VAAL calculates the probability of 2., belonging the unlabeled dataset, active learning, such as Coreset approach (Sener & Savarese 2018), selects instances that represent the whole distribution of unlabeled instances, by solving mixed integer programming. Recently, to make use of both uncertainty and diversity methods, Hybrid-base active learning proposed to select the uncertain instances in diverse way. In BADGE (Ash et al., 2020), the acquísition function is calculated as the gradient embedding of In w.r.t. the parameter of the last fully connected layer, Oonts that is (BADGE (Ir,) 20... laog
Lv, as the acquisition score, i.c., fugA (r.) = (z.). Diversity
0 ( Mus 0); where Du is the pseudo label of J... Then, this embedding becomes an input to the k-means++ seeding algorithm (Arthur & Vassilvitskii 2006).
Recently, pool-based active learning has been developed to deal with certain problematic scenarios. Such development includes using random round-robin sampling to efficiently apply active learn- ing in large-batch setting (Citovsky et al.) (02T); actively selecting test samples to query (Kossen) et al. 2021>; improving conventional active learning algorithm for high dimensional observational data JJsssnn et al.) 2021); etc. Compared to those researches, our proposed SAAL deals with a conventional scenario and focuses on how the proposed acquisition function selects the informative instances.
2.3 SHARPNESS- -WARE MINIMIZATION (SAM)
As an independent research direction from active learning. there is an increasing investigation on the flatness or sharpness of loss response surfaces, and their corresponding optimization. The flat min- ima of loss landscape analyzed in various research areas and is confirmed to have deep connection to generalization of neural networks. A recent broad study on the various measures of generaliza- tion has confirmed that sharpness- based measure has the strongest correlation to the generalization Jiang et al.) 2019). Hence, the flat minima is utilized in various research areas where generalization is important, such as domain generalization (Cha et al., 202T), adversarial robustness (Stutz et al, 2021. 1. or domain adversarial training (Rangwani et al.) 2022).
Having said that, Sharpness- Aware Minimization (SAM) is an optimizer for training the deep neural network (Foret et al., 2020) to weigh the importance of flat minima. Denoting the loss the dataset S w.r.t. the current parameter 0 as (0). the optimization objective of SAM is to minimize the maximally perturbed loss with the regularization the parameter, below.
min max (s + e) + >1001 Ell
(2)
Here, y is a hyperparameter that controls the magnitude of the effect of regularization, is the perturbation to the parameter, and defines the size of the perturbation.
The maximally perturbed loss can be decomposed ma Ls(A + (max & Ls + <) (0)) + (O), which is interpreted as the sharpness term (first term of the RHS) and the classification loss term (second term of the RHS). Hence, SAM minimizes the sharpness of the loss as well as the classification loss value, itself: that is SAM aims at seeking the flat minima among the local minimas. This optimization is a max- min problem, which solves max Sp Ls(0 + e) first than solves minę max]eel Sp (0 + e). The inner maximization problem is solved by finding argmax|| (0 + e). By deriving Taylor expansion of Ls + c) w.r.t. 0 around O, and by introducing a dual norm problem, the e is approximated as the follow, with 1 + 1 1.
(oLs
p sign (Q))((0))
After solving the inner maximization using e", the minimization problem is solved by obtaining the gradient, while excluding the Hessian term, the below.
Vo max Ls(0 + t) z VoLs(0))o+>- Sel
(4)
3

---

Under review as a conference paper at ICLR 2023
3 METHOD
3.1 MOTIVATION
According to SAM (Foret et al. 2020), the loss of the population dataset, 9, is upper bounded by the maximally perturbed loss of the training dataset, L. From the perspective of active learning, the training dataset is decomposed into the labeled dataset, LL, and the unlabeled dataset, Ly i.e.,
T = L.U U Lv- Hence, the upper bound can be decomposed as the below, with n = 12
12t and
12 Tu T
(0) max L (0 5 TL max Lxz (0 + TU [e
Ley (0 + L SAAL Sp
(5)
sellsp
Since the population loss, Lg (0), is never accessible; instead access the rightmost upper bound denoted in Eq. 5 which represented as LSAAL, and train our network to minimize the upper bound. Among the two terms of L the first term, TL mmx L (f+c), will be minimized we use SAM optimizer. Then, the remaining second term, Ty |ell 1. (0 + +e), becomes the key component for our optimization in the sharpness-aware active learning scenario. During the active learning iterations, we suppose that we select unlabeled instances, Ty E Lv. with high values of maximally perturbed loss. In other words, the remaining Ly consists of unlabeled instances whose maximally perturbed loss value small. Then, it leads to minimizing two terms of L SAAAL simultaneously; which contributes minimizing the generalization error, Lg)())
query the label of such unlabeled instances, that
Comparison to Semi-Supe Learning Our proposed active learning algorithm is not the only way for decreasing the loss of unlabeled dataset, Zy; because traditional semi-supervised learning (SSL) is another way that utilizes Lz (0) while training the model. However, it should be noted that SSL does not guarantee to minimize the rightmost upper bound, L.AAL, SSL mini- mizes the average of unlabeled dataset loss instead of the maximum perturbed loss. Hence, it is hard to guarantee that SSL will contribute to minimizing the generalization error without prior knowl- edge on label distribution (Ben-David et al. 2008). We can categorize the SSL approach three ways (Berthelot et al. [2019] 2005), which are consistency regularization (Laine & Aila 2016)) Sajjadi al. 2016), entropy minimization (Cireşan et al. 2010 Lee et al.) 2013), and traditional regularization, such weight decay (Zhang al. 2018 8a b). First, consistency regularization and entropy minimization completely depend on the pscudo-label, and an incorrect pscudo-label might increase the generalization error. Second, the worst-case pseudo-label. In other words, SSL, training the model with an incorrect pseudo-label, might fail to model the maximum perturbed loss. Third, the minimization of maximum perturbed loss is an independent approach to the previous semi-supervised learning methods, such as traditional regular- ization well consistency and entropy minimization. They are potentially compatible with our active learning algorithm.
hardest instances might have incorrect
3.2 SHARPNESS- AWARE ACTIVE LEARNING
Motivated by the Sharpness- Aware Minimization (SAM), our active learning algorithm selects in- stances with high perturbed loss under some perturbation on the model parameters, 9. Hence, our acquisition function is as follows:
SAAL (Iui fo) = max (za, Dui 0 + e),
(6)
||
where is the cross-entropy loss function for the model, and 0 is the current parameter of the model. Algorithm|] describes the overall process of our Sharpness-Aware Active Learning. Since quisition function is calculated for the unlabeled instances, there comes problem when calculating the maximally perturbed loss function, which requires label. Hence, we use a pseudo label, t.. for the loss calculation. To provide the validity of utilizing pseudo label, we first provide Theorem 3.1 which explains the relation between the maximally perturbed losses which are calculated with pseudo label and with ground- -oound-truth label, respectively.
Theorem 3.1. (Proof in Appendix (.2.I] For a data instance x, let & be the pseudo label predicted by the network Jo and y be the ground-trut truth label. Then, the maximally perturbed loss calculated with
4

---

Under review as a conference paper at ICLR 2023
Algorithm 1 Sharpness-Aware Active Learning
Input: Labeled dataset -D. Unlabeled dataset 2 Classifier fo
1: Initially train fo by the cross-entropy loss of 22
2: for 2,,.,.... .. do
active learning
3: Randomly sample 27 2-pool C C29
4: for Thu E Et do
5: Calculate f
(Tu; fo) as Eq.
6: end for
7:
Select 2g argmax x.g 2goot Ex.Egg aa0 (SAAL(INs ScniSol S0)
8:
Query the label of Lg to oracle
9:
Update the labeled dataset, 27" 1 27 U Us
10:
Update the unlabeled dataset, 27 2+1 L7 \
Is
11: Train fo by the cross-entropy loss of 2.++1
12:
end for
(I, () is lower bound of the maximally perturbed loss calculated with (x, I): with a non non-negative margin, dg, the below:
max 1x, 0 c) S max [x,, D; 8z.
Sp
1ellSS
Next, Proposition 3.2 shows that the inequality has zero margin under a mild condition.
Proposition 3.2. (Proof in Appendix A.2.2) For data instance and the corresponding pseudo label ÿ. let ê be the maximal perturbation over the parameters w.r.. the loss 1(x G; + c). If the perturbed network, fo+z, keeps the predicted label the same as the label predicted from the original network, fo: then the maximally perturbed loss calculated with (r, 9) is a lower bound of the maximally perturbed loss calculated with (x. V). as the below:
max 1xx, ÿ; 0 +c) max 1(x, T: c). Hellsp
(8)
[|e] Sp
Theorem 3.1 and Proposition 3.2 provide that the selected instances by acquisition score of SAAL with pscudo label would also have high scores w.r.t. the ground-truth label. It indicates that we conservatively estimate the maximally perturbed loss for the acquisition score.
3.3 CONNECTION TO RECENT ACTIVE LEARNING ALGORITHMS
Here, we theoretically derive the upper bound of the acquisition of SAAL, and show the connection to the recent active learning algorithms as well as the generalization ability. To begin with, we provide Theorem 3.3 below.
The acquisition function, SSAAL, of Eq. 6 is upper bounded by 1(0) + pl Ve((0)]l 3p"²> + max ((p²v²); where 1(0) abbreviates the loss of data pair, (1c, y), and dı is the first eigenvalue the loss Hessian.
Theorem 3.3. (Proof in Appendix A.2.3)
Theorem 3.3 derives the upper bound of the acquisition score of SAAL, which consists of the task loss, the gradient norm, and the first eigenvalue of loss Hessian. Since we are selecting instances which have high value of ISAAL, the selection refers that we are also selecting instances which have high values of the loss, 1(0), and the magnitude of the gradient embedding, ([)]l2) which are connected L.LAII YYoo & Kweon 2019) and BADGE (Ash et al., (020), respectively. Fur- thermore, SAAL considers the first eigenvalue of the loss Hessian w.r.t. the current model, denoted as A1. The importance of the first eigenvalue for generalization is widely studied, that is the first eigenvalue is used as the indicator of the sharpness of the loss surface (Keskar al. 2017 Zhuang et al., 2022 Kaur et al.] 2022). Hence, the selected instances by SAAL might contribute to the generalization of the model.
Figure shows that there exists a positive correlation between our acquisition score, f 4SAAI, and the three terms of upper bound. It should be noted that the upper bound is not our optimization
5

---

Under review as a conference paper at ICLR 2023
objective, because use the acquisition score for ranking the instances with top values of the score. Hence, the correlation between the acquisition score and the upper bound is of our interest. Having said that, by selecting the instances with the high acquisition score of SAAL, fa444, we are selecting instances that have high values of the loss, gradient norm, and the first eigenvalue. Also, Figure [h]shows the value of the three terms of upper bound. Interestingly, as the acquisition iterations proceed, not only the loss and the gradient value, but the first eigenvalue gets smaller. The change of the value of the first eigenvalue is more noticeable in Figure Ic] which plots the value of λ without the scaling term of p². This indicates that SAAL leads the model to a flat minima, which results in better generalization performance.
I" Ogencelte 2
Pearson Correlation
1 R
Gradeet, 2l T.00I
Digarrauee 1or
R 8
Velure
c2-
Less, K00)
Gradient, AlRs/15Hli
1at Eigenvaluo, fo
66
Acquisition Iterat on (x10)
Acquisition Iteration (x1.0
Acquisition teration (x10)
(a) Correlation between faci AT. and upper bound terms
(b) Magnitude of the upper bound terms
(c) Detailed view of
the first eigenvalue, À
Figure I: Correlation and magnitude of upper bound terms
EXPERIMENTS AND RESULTS
We examined the performance of SAAL in two vision-based tasks; which are image classification and object detection. Specifically, for image classification, we quantitatively compared the test accuracies among various active learning algorithms; and qualitatively analyzed the performance of SAAL by examining the upper bound of population loss. Also, we analyze the effect of p, which defines the size of the perturbation,
4.1 IMAGE CLASSIFICATION
Experiment Setting We conduct our experiment on Fashion-MNIST (Fashion) (iaa et al., 2017]. SVHN (Netzer et al. 2011), , CIFAR-10, and CIFAR- (Krizhevsky et al. 2009) dataset. We adopt the 18 (He et al. 2016) network for our classifier, and we train the network for 50 epochs after each acquisition step, using Adam optimizer (Kingma & Ba 2015) with a learn- ing rate of le-3; and SAM optimizer (Foret et al. 2020) with a learning rate of 1e-3 for Fashion, SVHN, CIFAR-10 10 and le-1 for CIFAR-100. To simulate an experiment scenario with bad general- ization cases,
followed the settings of the prior works (Kim et al. 2021), which assumes very low amount of allowed budget. For Fashion, SVHN, and CIFAR- 10, we construct the initial labeled dataset with 20 instances, which are random but balanced; and we select 10 additional instances with the highest acquisition score among the randomly selected 2,000 unlabeled instances. For CIFAR- 100, the initial labeled dataset consists of 1,000 instances, and we select 100 additional instances for 100 repeated iterations. We repeat the acquisition for 100 iterations, and we report the results with three repeated trials. Here, SAAL introduces the perturbation size, p. of the perturbation, c, in Eq. 6 and we set the value of p as 0.05 for all the datasets.
Baselines We compared the performance of SAAL with Random, Entropy (Shannon, 1948),
Coreset (Sener & Savarese 2018). Learning Loss for Active Learning (LL4AL) (ooo & Kweon 2019, Variational Adversarial Active Learning (VAAL) (Sinha et al., 2019). and BADGE (Ash) al. 2020). Since most relevant algorithm, BADGE, adopts k-means+ + seeding algorithm to introduce diversity on the acquisition; we also provide an experimental result with diversity follow- ing the same practice from BADGE. Specifically, after calculating our acquisition function using Eq. 6 we implement k-means++ seeding algorithm with the acquisition score as input.
Quantitative Analysis Table T indicates that SAAL outperforms the baselines in seven out of eight combinations of experiments. The advantage of SAAL becomes obvious when we use the Adam optimizer, rather than the SAM optimizer. We conjecture that this gain for Adam optimizer

---

Under review as a conference paper at ICLR 2023
originates from Eq. 5 which motivates SAAL in modeling the expected flat local minima after acquisitions. Recall that our inaccessible goal, L (0). is upper bounded by TL maxij< L,2 (6+ <) + TV max]jelsss T. (0 + <), as we discussed in Section 3.1 When using Adam optimizer, the first term, ma Le, (0 + c), in the upper bound is weakly optimized compared using SAM optimizer, which we will present qualitative analyses in the next section; because SAM optimizer directly minimizes s Læz (0 + e). Hence, the importance of the second term in the upper bound, [ellsp 2x (9 E). becomes more significant for Adam optimizer.
We also provide the test accuracy along the acquisition iterations in Figure of Appendix A.1 The figures show that SAAL achieves higher accuracy more quickly than baselines in most cases, see Figure oa
6d
or ogg
Table 1: Comparison of test accuracy (%) using Adam optimizer and SAM optimizer. The best performance is indicated as boldface. The results are replicated by three times.
Adam optimizer
SAM optimizer
Method
Fashion
SVHN
CIFAR-10
CIFAR-100 100
Fashion
SVHN
CIFAR-10
CIFAR- 100
Random
81.2 + 0.5
72.4 + 0.9
50.7 + 1.5
43.3 0.3
83.7 + 0.3
78.1 +.1
$2.6 288
44.0 + 0.7
Entropy
81,5 + 1.4
73.1 1.
51.9 +
0.7
84.1 + 0.2
77.5 3.2
54.6 0.4
44.1 # 1.0
Coreset
83.8 0.7
5.8
51.7
0.5
84.4 0.6
78.9 1: 1.3
1.
47.6 1.4
LI 0
83.5 + 1.8
75.1
0.47
0.3
83.2 1.4
72.2 22.2000 0.2
1.12+1.1
35.7 001
VAAL
83.4
1.
44.8 5.9
84.1
77.1
45.5 5..1
BADGE
85.4 0.6
74.9 + 1.1
52.3
45.7 0.6
86.2 + 0.2
78.8 + 0.9
56.8 1.99
47.4 + 0.7
SAAL
85.8 0.8
76.8 + 0.7
54.4
47.6
86.3 + 0.5
78.8 + 1.0
57.0 1.1
48.4 + 0,9
Next, we compare the time complexity of SAAL and baselines. We used the CIFAR-IO dataset and measured the time for single iteration of acquisition and training.
Time loq-scale
Figure 2howss the time in seconds with a log scale. The
results of Random acquisition show that the SAM opti- mizer takes twice longer time than the Adam optimizer, because takes two steps of gradient calculation. How- ever, the gap between Adam and SAM becomes mere when using other active learning algorithms. indicating that the time for calculating acquisition score is the largest bottleneck. SAAL calculates the perturbation, c, for every single unlabeled instance, instead of batch-wise calcula- tion; so takes longer than most the other baselines. The time complexity of SAAL can be reduced if we adopt the improved SAM models (Du et al., 202T 2022) that have been proposed for an efficient calculation.
Randam
Entropy
Coreser
lddl
BADGE
Soal
Figure 2: Comparison of time complex- ity, in log-scale
Qualitative Analysis Figure supportss the conjecture for the advantage of SAAL by anticipating the flat local minima in the acquisition process. Figure measures the maximally perturbed loss for the labeled dataset, Li; the unlabeled dataset, Lu: and the total dataset, 22 U Zv. We compare the results between the models trained with either SAM optimizer Adam optimizer. Since it is computationally hard to calculate the corresponding perturbation for every single unlabeled instance, Fr E Lv. we uniformly sample 2,000 unlabeled instances from Lo at each iteration; and we report the averaged results for three independently repeated trials.
Figure Baj shows the maximally perturbed loss of Lv when using SAM optimizer. If we compare the result of SAAL with the results of basclines, SAAL shows the lowest value of the maximally perturbed loss, because SAAL selected the instances with high values of perturbed loss; and because SAAL removes such instances by passing those instances to the labeled dataset. Figure 3b shows the maximally perturbed loss of Hy, when using SAM optimizer. This loss also indicates the flatness of the model; the lower value of the maximally perturbed loss of Lú, indicates that the model does not change the result even the parameter is changed in a small range, which refers to the flat model (Keskar al. 2016 Neyshabur et al.) 2017 Kawaguchi et al. 2017). Hence, SAAL results in flat network compared to the basclines. We conjecture that the flat model attained by SAAL is explained
For I.I4AL, failed to reproduce the performance when using SAM optimizer.

---

Under review as a conference paper at ICLR 2023
by the look-ahead concept (Roy & McCallum 2001 Konyushkova et al. 2017: Kim et al. 2021 If we are planning to minimize max <| L
22(2 (6 c by SAM optimizer, SAAL looks ahead the
high values of the |ax]] T (0 + <) from unlabeled instances, and SAAL actively selects such unlabeled instances to flatten the future response surface.
Finally, Figure 3e shows the maximally perturbed loss the total dataset, which is equivalent to the upper bound in Eq. 5 As confirmed in the figure, SAAL achieves the lowest upper bound, which indicates that the model trained with SAAL is more likely to achieve lower population loss, which is ultimate goal of minimization objective. When comparing the results of SAM (Figure
3a Figure 3cb and the results of Adam (Figure 3d Figure 3t). the gap between SAAL and other
baselines becomes clearer in Adam optimizer.
Maximary Perturbed Lons
bnbray Camor
Fertureed
WAl
Masomally perturted
BADUL
Moximall
aagu Iteraatio
Acquiittion Ber Beration
Acquisiiion Banation
(a) Maximally perturbed loss of
(b) Maximally perturbed loss of LL with SAM
(c) Maximally perturbed loss of
F6 with SAM
B2 U Zo with SAM
Moxlmaty Parurbod La5s
Retrepy
Maxime Pertur Lotes
Perturbeeg Lass
Fetmey
Will
LL4AL
BiOCE
SAAL
Beration
Acquistion
Acquistiion
(d) Maximally perturbed loss of Lo with Adam
(e) Maximally perturbed loss of (f) Maximally perturbed loss of Lr with Adam
Lt U Ey with Adam
Figure 3: Maximally perturbed loss of the labeled dataset, unlabeled dataset, and total dataset during the active learning iterations. (a) - (c) are the results the model trained by SAM optimizer. (d) (f) are the results of the model trained by Adam optimizer.
Sensitivity Analysis on p SAAL introduces a hyperparameter, P. which represents the size of the perturbation, e. Hence, we conduct the sensitivity analysis on o with the CIFAR-I0 dataset, and we set the candidate values for p as 0.01, 0.05, and 0.10.
First, we examined the validity of Theorem 3.1 by investigating if the network with the max- imally perturbed parameters keeps the predicted label same the original network. Fig- ure [4a shows the proportion of unlabeled data instances whose predicted labels are remained the same by the perturbed network during the active learning iterations; that is (p) = [ool Exe 2y angmax, fo)+(") where 1 is the indicator function that returns 1 if the condi- tion A is satisfied. When the size, p. of the perturbation, e, is zero, that is if we do not perturb the network; then the inequality of Eq. is satisfied for all instances, by the definition of the pseudo la- bel, ÿ. As we increase the value of p, some instances fail to keep the predicted label as the same as ÿ, because the parameter of the model changes drastically so that the model loses the prediction ability that it has learned so far. Also, examined the validity of Proposition 3.2 by investigating the value of the margin, dz, for the unlabeled data instances in Figure 4b) It should be noted that 8, is not our hyperparameter, but dependent variable subject change by p. We only investigate 55 to reveal the characteristics of p, not for the hyperparameter optimizations. To show how the value of the margin, 8,, affects the inequality, we measure the relative value of the margin, ò.. compared to the maxi- 8
mally perturbed loss, max/<]]ssp [(x T: 0 e); that is r(p, 5.) Le ExeRi maxii- (x.0:0++)) From the analyses of Figure 4a and 4b we adopted p = 0.05, because this value 1) keeps the pre- dicted label of data instance from the original network with high probability and 2) keeps the value of the margin relatively small compared to the max perturbed loss w.r.t. the ground -ruth label; while p 0.05 is confirmed to perturb the parameters of the network effectively (Foret al. 2020).
8

---

Under review as a conference paper at ICLR 2023
The proper selection of p also affects the test accuracy, shown in Figure If we select p with too small value, that is p= 0.01, the parameter of the model not perturbed enough to measure the sharpness, SAAL cannot catch the informative instances. If we select p with too large value, that is = 0.10, the maximally perturhed loss 1) does not satisfy Proposition 3.2 as confirmed in
Figure 4a and have too large value of margin, confirmed in Figure 4b Meanwhile, proper value of 0.05 for the perturbation, e, shows the best performance.
D-G.11
D-C.05
Accuracy 0%)
0-0.00
p.0.06
0-0.10
Acquisition Iteration (x10)
Acquisition Iteration (x10)
p-0.01
P-0.05
P-0.10
(a) Proportion of unlabeled instances satisfying the assumption
(b) Averaged value of margin, ò2. in Theorm for the unlabeled dataset; with varying
(c) Test accuracy; with varying P
in Proposition 3.2 with varying
Figure 4: Sensitivity analysis on p
4.2 OBJECT DETECTION
To show the effectiveness of SAAL complex task, we conduct an object detection task. Object detection returns the locations of semantic objects and the corresponding labels for a given input image, Hence, the loss for training detection model consists of the bounding box re- gression loss and the classification loss. We experiment with PASCAL VOC 2007 and 2012 dataset (Everingham et al. 2010), which contains 5,011 images and 4,952 im- ages with 20 object classes, respectively. We adopt Single Shot Multibox Detector (SSD) (Liu et al., 2016) the detection model. To apply SAAL for object detection, perturb the parameters maximize the classification loss; and use the summation of the perturbed loss from every corresponding detection box in the image, æ, as the acquisition score for x. Afterward, we select the images with the highest scores. We construct the initial labeled dataset with 1,000 randomly selected images, and se- lect additional 1,000 instances at every acquisition itera- tions, so that attain 10,000 final instances with nine repeated acquisitions. We train the model for 300 epochs with batch size of 32. Figure 3 reports the mean av- erage precision (mAP) for three repeated trials of SAAL and baselines. As shown in the figure, SAAL achieves high performance at the earlier iterations and shows the highest mAP of 0.7541 at the last iteration; while BADGE, Entropy, and Random show 0.7493, 0.7518, and 0.7403, respectively.
Random
Entropy BADGE
SAAL
mAp
010
055
26
Number Labeled Instances
Figure 5: mAP of object detection task with PASCAL VOC 2007+2012
CONCLUSION AND FUTURE WORKS
We propose a new active learning method named Sharpness-Aware Active Learning, SAAL. SAAL considers the loss sharpness of data instances, which is strongly related to the generalization performance of deep learning. Furthermore, we derive the upper bound of SAAL acquisition score and find the connection to the recent active learning methods; as well as the connection to the first eigenvalue of loss Hessian which is widely used as the indicator of loss sharpness. By various experiments with benchmark datasets, SAAL shows better performance than baselines. As a future work, will improve the time complexity of SAAL by adopting recent SAM models.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Dana Angluin. Queries and concept learning. Machine learning, 2(4): 342, 1988.
Dana Angluin. Queries revisited. Theoretical Computer Science, 313(2):175-1944 2004.
David Arthur and Sergei Vassilvitskii. k-means+4: The advantages of careful seeding. Technical report, Stanford, 2006.
Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In ICI.R, 2020.
Les Atlas, David Cohn, and Richard Ladner. Training connectionist networks with queries and selective sampling. Advances in neural information processing systems, 2, 1989.
Shai Ben-David, Tyler Lu, and Dávid Pál. Does unlabeled data provably help? worst-case analysis of the sample complexity of semi- cemi-supervised learning. In COLT, pp. 33 44, 2008.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural informa- tion processing systems, 32, 2019.
Junbum Cha. Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Infor- mation Processing Systems,
2021.
Dan Claudiu Cireşan, Ueli Meier, Luca Maria Gambardella, and Jürgen Schmidhuber. Deep, big, simple neural nets for handwritten digit recognition. Neural computation, 22(12):3207-3220, 2010.
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Ros- tamizadeh, and Sanjiv Kumar. Batch active learning at scale. Advances in Neural Information Processing Systems, 34:11993-1194 1944, 2021.
David Cohn, Les Atlas, and Richard Ladner. Improving generaliztinn with active learning. Machine learning, 15(2):201-221, 1994.
David A Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models. Journal of artificial intelligence research, 4:129-145, 1996.
Sanjoy Dasgupta and Daniel Hsu. Hierarchical sampling for active learning. In Proceedings of the 25th international conference on Machine learning, pp. 208 215, 2008.
Jiawei Du, Hanshu Yan, Jiashi Feng, Joey Tianyi Zhou, Liangl Zhen, Rick Siow Mong Goh, and Vincent Tan. Efficient sharpness-awan aware minimization for improved training of neural networks. In International Conference on Learning Representations, 2021.
Jiawei Du, Daquan Zhou, Jiashi Feng, Vincent YF Tan, and Joey Tianyi Zhou. Sharpness-aware training for free. arXiv preprint arXiv:2205.14083 2022.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2): 303-338, 2010.
Sebastian Farquhar, Yarin Gal, and Tom Rainforth. On statistical bias in active learning: How and when to fix it. In International Conference on Learning Representations, 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness- minimiza- tion for efficiently improving generalization. In International Conference on Learning Represen- fations, 2020.
L.C. Freeman. Elementary applied statistics: for students behavioral science. Wiley, 1965. URL https::/bboee goog le. wtt.................oooooo krr/bookks7.
10

---

Under review as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Andrew Jesson, Panagiotis Tigas, Joost van Amersfoort, Andreas Kirsch, Uri Shalit, and Yarin Gal. Causal-bald: Deep bayesian active learning of outcomes to infer treatment-effects from observational data. Advances in Neural Information Processing Systems, 14:30465-30477 2021.
Yiding Jiang. Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantas- tic generalization measures and where to find them. In International Conference on Learning Representations, 2019.
Simran Kaur, Jeremy Cohen, and Zachary C Lipton. On the maximum hessian eigenvalue and generalization. arXiv preprint 2rX.v:220. 2022.
Kenji Kawaguchi, Leslie Pack Kaelbling, and Yoshua Bengio. Generalization in deep learning. arXiv preprint arXiv: 1710.5567, 2017.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv: /609. 04836, 2016.
Nitish Shirish Keskar, Jorge Nocedal, Ping Tak Peter Tang, Dheevatsa Mudigere, and Mikhail Smelyanskiy. On large-batch training for deep learning: Generalization gap and sharp minima. In 5th International Conference on Learning Representations, ICLR 2017, 2017.
Yoon- Boon-Yeon Kim, Kyungwoo Song, JoonHo Jang, and Il-Chul Moon. Lada: Look-ahead data
acquisition via augmentation for deep active learning. Advances in Neural Information Processing Systems, (44:22199-22999 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015. URL http:
//arxiv. abs 11111411120 6980
Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua. Learning active learning from data, arXiv preprint arXiv:1703.03366 2017.
Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-efficient model evaluation. In International Conference Machine Learning, pp. 5753-5763. PMLR, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. arXiv preprint 1rXiv:6600.0200 2016.
Dong-Hyun Lee al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges representation learning, ICML, volume 3, pp. 896, 2013.
David D Lewis and William A Gale. sequential algorithm for training text classifiers. In SIGIR '94, pp. 3-12. Springer, 1994.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pp. 21-37. Springer, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring general- ization in deep learning. Advances neural information processing systems, 30, 2017.

---

Under review as a conference paper at ICLR 2023
Harsh Rangwani, Sumukh K Aithal, Mayank Mishra, Arihant Jain, and Venkatesh Babu Radhakr- ishnan. A closer look at smoothness domain adversarial training. In International Conference on Machine Learning, pp. 18378-18399, PMLR, 2022.
Nicholas Roy and Andrew McCallum. Toward optimal active learning through monte carlo estima- tion of error reduction. ICML, Williamstown, 2:441-448, 2001.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29, 2016.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018.
Burr Settles. Active learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.
Burr Settles, Mark Craven, and Soumya Ray. Multiple-instance active learning. Advances in neural information processing systems, 20, 2007.
Claude E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27(3):379-423, 1948.
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5972-5981, 2019,
David Stutz, Matthias Hein, and Bernt Schiele. Relating adversarially robust generalization to flat minima. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 7807-7817, 2021.
Simon Tong. Active learning: theory and applications, volume 1. Stanford University USA, 2001.
Fang Wan, Tianning Yuan, Mengying Fu, Xiangyang Ji, Qingming Huang, and Qixiang Ye. Nearest neighbor elassifier embedded network for active learning. In Proceedings the AAAI Conference on Artificial Intelligence, volume 35, pp. 10041-10043 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint urXiv: 1708.07777 2017.
Donggeun Yoo and In So Kweon. Learning loss for active learning. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 93-102, 2019.
Guodong Zhang, Chaoqi Wang, Bowen Xu, and Roger Grosse. Three mechanisms of weight decay regularization. arXiv preprint 181iv 2018a.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018b.
Xiaojin Jerry Zhu. Semi-supervised learning literature survey. 2005.
Juntang Zhuang, Boqing Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha Dvomek, Sekhar Tatikonda, James Duncan, and Ting Liu. Surrogate gap minimization improves sharpness-aware training, arXiv preprint arXiv:2203. 2000000, 2022.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 TEST ACCURACY
We provide the learning curve of SAAL and baselines along the acquisition iterations.
rarton
Acqulaiioon Beration
Acquiatton coration
Acquiaiiann aliun
(a) Test accuracy on Fashion, with Adam
(b) Test accuracy on SVHN, with Adam
(c) Test accuracy on CIFAR-10, with Adam
(d) Test accuracy on CIFAR-100, with Adam
Tec: Accurazy
revey
Acqu Arquishion Beracion
Acquisition neration
Acquisitoon heration
Acg istiin fteration
(e) Test accuracy on Fashion, with SAM
(1) Test accuracy on SVHN, with SAM
(g) Test accuracy on CIFAR-IO, with SAM
(h) Test accuracy on CIFAR- 100, with SAM
Figure 6: Test accuracy along the acquisition iteration; with Adam and SAM optimizers
A.2 PROOF DETAILS
A.2.1 PROOF THEBOREM 3.1
Theorem A.1. For data instance let ÿ be the pseudo label predicted by the network fo and y be the ground-truth label. Then, the maximally perturbed loss calculated with (z, () is a lower bound of the maximally perturbed loss calculated with (xr, W): with non-negafive margin, 8z. as the below:
1(x, 0:0 + + max 1(x, Ú:0 c) + 8z. Se
|cc|S
Proof.
The cross-entropy loss, 1(x,01 y: 0), is represented with the logit vector fo(x)) E R as the below:
exp(fo(x)g)
1(x, v: In Σ, exp(fo(z)g)
In (Ix(exx(fo(x))))
Σ exp(ff(x)))
= Σ exp(Jo(x))) So(x)
Then, the maximally perturbed loss of a data pair (x, y) is represented as the below:
max 1(x,, y: +) max (In Σ exp((fot((z)) fo+e(x)g)- <
[|
Since the pseudo label, e. satisfies 9 argmax fo(x), by the definition, it holds that fo(x); > fo(x) for all V. Let argmax Se [55, Dz + e). Define the margin, dz. 8,-
13

---

Proposition A.2. For a data instance x and the corresponding pseudo label ÿ, let be the maximal perturbation over the parameters w.r.t. the loss (x, ú: c). If the perturbed network, fo+é, keeps the predicted label as the same as the label predicted from the original network, fo: then the Under review as a conference paper at ICLR 2023
maxx (f0) - fo+e(2)))l) 11 where |-] maxf=; 0}. Then, the following holds.
max Se
(z, yz: 0 +
In E exp(fo+c(x))) fo+c(z)s
I In Σ exp(fo+e(x))) fo+e(z) + 6z
In Σ
exp exp(ff+c(x))) fotc(x)) de
max ((x, y: 0 +) + S
A.2.2 PROOF PROPOSITION
maximally perturbed loss calculated with (a,, D) is lower bound of the maximally perturbed loss calculated with (I, W) as the below:
max lx, 9r 0 + < max 1(x, D: 0 + sellsp
[[ Sp
Proof. Since the perturbed network, fo+e, keeps the predicted label as the same as the label pre- dicted from the original network, fo: it holds that argmax fo+=(x) argmax fo(x) and accordingly fx+)(x), S fo+e(x); for all Hence, max, (fo+e()); fo+e(1)g) S 0. Thus, by the definition of the margin in Theorem 3.1 o becomes zero.
A.2.3 PROOF OF THEOREM 3.3
Theorem A.3. The acquisition function, JSAAL, of Eq. 6 upper bounded by [(0) + (ol 12 d2>11 + max]jos şı Opp²v²); where I(A) abbreviates the loss of a data pair, (x. y), and x the first eigenvalue of the loss Hessian.
Proof. Recall that our acquisition function is fSAAL = ma (x Du: 0 + c). Since we limit the size of the perturbation as ||e|| I P, we can write maxile Isp (a Un; 0 + = max [ool|s (xx, Vui 0 pu) = max]ells [x Un; + pu). Then, by Taylor expansion of (I u. Du: po) w.r.t. the below holds, where we abbreviate (xx. Dus 0) as I(0).
pu with ||a||| < 1, and
FSAAL (Zui fe)
max 1(0 e) max 1(0 + pv)
SP
1
max (1(0) + (pv) vo((0) + / (pu) (0((0)))))+((p)))))))
[|v|
1((8) max (Qu()))o(( + (pu)"VV(((0)))))) +(()))))) |
< ((0) + max (po) T vol(0)
max |ells1
( + max ((vv) 1e
|elll1
1(0)
Vo100)) +
max (((pu)*)
|
Loss
Gradhent Norm
Eigenvalue
14


Reviews:
[{"review_id": "Hs3r44NDFD", "reviewer": "Reviewer_Xi2g", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes to consider the generalization ability into the acquisition process of new labeled instances in active learning. The work selects instances with a high value of loss sharpness on pseudo-labels, which can be used for improving the generalization ability of the model based on the finding of Sharpness-Aware Minimization (SAM). Besides, the work theoretically gives the connection between the acquisition score and other scores, which are related to the generalization ability. In addition, the work shows consistent improvement over multiple datasets.", "summary_of_the_review": "This work proposes to consider the generalization ability into the acquisition process of new labeled instances in active learning. The work selects instances with a high value of loss sharpness on pseudo-labels, which can be used for improving the generalization ability of the model based on the finding of Sharpness-Aware Minimization (SAM). Besides, the work theoretically gives the connection between the acquisition score and other scores, which are related to the generalization ability. In addition, the work shows consistent improvement over multiple datasets. However, the proposed method seems extend the SAM to active learning setting.", "strength_and_weaknesses": "\nStrength:\na. The approach is simple. It is interesting to see that selecting instances with a high value of loss sharpness improves the performance.\nb. The results look promising.\nc. The manuscript is easy to read\n\nWeaknesses\na.\tThe motivation may overlook the relationship of the selected instances and unselected ones. The work simply reduces the second term in Eq.5 by removing the unlabeled instances with highest loss sharpness. The challenges to compute sharpness should be further explained.\n\nb.\tThe method to estimate the loss sharpness of the unlabeled instances may be easy. And a wide lower bound in Theorem 3.1 may be not enough to guarantee the effect of the estimation.\nc.\tIn Experiment Setting, the work follows the settings of the prior works to a very low amount of allowed budget. But the setting is not practical in real world. It will be better if the work can provide the performance on larger budget.\nd.\tIn Baselines of 4.1, the work states it adopts k-means++ seeding algorithm as prior work. The ablation study on the effect of k-means++ seeding algorithm is needed.\ne.\tIn Quantitative Analysis, the work analyzes the time complexity. The description of the equipment is needed.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8q1A1mFPyG2", "reviewer": "Reviewer_3NDR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The study investigates the active learning problem with sharpness information in loss space. It proposes a method named sharpness-aware active learning to select unlabeled instances potentially having maximal perturbation loss. The method has a theoretical guarantee on the generalization error of active learning. Experiment results show the effectiveness of the proposed method. ", "summary_of_the_review": "The work proposes a new method for active learning by adopting the sharpness information in loss space. The method is interesting with a design of minmax optimization. Experiments show the effectiveness of the method. However, the method does not outperform the existing methods, and the SAM optimizer affects more in the improvements. Also the proposed method is time consuming. Overall, the reviewer believes that the work is boarderline and needs further polishing.", "strength_and_weaknesses": "Strengths:\na. The proposed method takes full advantage of sharpness information of loss. \nb. Generalization error of active learning is upper bounded by the method.\nc. Experiments verify the performance of the method.\n\nWeakness:\na. More datasets should be considered in the experiments, especially for large-scale ones (e.g., Webvision, ImageNet).\nb. In Table 1, the proposed method does not beat the previous studies with a much margin. It seems that SAM optimizer plays a more important role in the improvement.\nc. The time complexity of the proposed method is much higher than that of the most of existing methods. Is it possible to accelerate the method without degradation?\nd. Is there a clear cut of value for \\rho to satisfy Proposition 3.2?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6J8RmLMdVk", "reviewer": "Reviewer_tmqo", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This manuscript describes a new active learning method considering generalization, which is very practical. The key insight is to incorporate the sharpness of loss space when designing the active acquisition function and select unlabeled samples with the maximal perturbed loss for labeling. The experimental results and theoretical evidence confirm the effectiveness of the proposed approach.", "summary_of_the_review": "Generally, I find this paper to touch on an interesting problem with a novel sharpness-aware minimization based active learning for generalization. Yet, it still requires clarification and some solid empirical support before warranting acceptance of this paper.\n\n\n", "strength_and_weaknesses": "Strength:\n1. The paper is well written, the motivations for choices in the method are clear and the idea is simple yet effective, making full use of sharpness-aware minimization.\n2. The presentation of background in Section 2 is thorough and complete.\n3. The empirical evaluation is thorough and conducted on diverse vision-based tasks, including image classification and object detection.\n\nWeakness:\n\n[Motivation] The motivation is reasonable for me to consider the generalization ability in active learning. It might be better to give more examples to help understand why we need to select samples whose perturbed loss is maximum. For example, why should we apply SAM to active learning? Does SAM inherently have generalization properties?\n\n[Relevant works] The work aims to connect active learning and generalization ability. Recent works about active domain adaptation also concern generalization. Several relevant references are missing [a, b, c, d]. The similarities and dissimilarities should be discussed.\n\n[Experiment]\n- For more clear understanding, could you provide some qualitative and quantitative results for the loss landscape?\n- I would like to see why this work is not applied to the semantic segmentation problem and compared to works in that.\n\nRefs:\n[a] Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings.\n\n[b] Active Learning for Domain Adaptation: An Energy-Based Approach.\n\n[c] Discrepancy-Based Active Learning for Domain Adaptation.\n\n[d] Active Adversarial Domain Adaptation.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject