Hybrid Active Learning with Uncertainty-Weighted Embeddings
Authors: Yinan He
Lile Cai
Jingyi Liao
Chuan-Sheng Foo
Venue: Accepted by TMLR ()

Abstract: We introduce a hybrid active learning method that simultaneously considers uncertainty and diversity for sample selection. Our method consists of two key steps: computing a novel uncertainty-weighted embedding, then applying distance-based sampling for sample selection. Our proposed uncertainty-weighted embedding is computed by weighting a sample's feature representation by an uncertainty measure. We show how this embedding generalizes the gradient embedding of BADGE so it can be used with arbitrary loss functions and be computed more efficiently, especially for dense prediction tasks and network architectures with large numbers of parameters in the final layer. We extensively evaluate the proposed hybrid active learning method on image classification, semantic segmentation and object detection tasks, and demonstrate that it achieves state-of-the-art performance.

Jingyi Liao Published in Transactions on Machine Learning Research (06/2024)
Hybrid Active Learning with Uncertainty-Weighhee Embed- dings
Yinan He Nanyang Technologioal University, Singapore
yinan.he. 688 @amallgmmllmmmommm com
Lile Cai*
coill@12r.a a-star.sstaaa..... eatar.edu.50 edu.sg
Institute for Infocomm Research (PR), A A,TAR, Singapore
liao hho__inngy@@h.... aeteettt..sssrrrrr aastaa.ssaaraaee edu.s) sg
Institute for Infocomm Research (F)), A ASTAR, Singapore
Chuan-Sheng Foo
foo chuan sheng a a-ttar..aa.aa.....o.. edu.sg
Institute for Infocomm Research RJ),, A A Singapore
Reviewed OpenReview: https: https://openrevveau openreview. net/ forum? id= j0761bb5
Abstract
We introduce a hybrid active learning method that simultancously considers uncertainty and diversity for sample selection. Our method consists of two key steps: computing novel uncertainty-weighnee embedding, then applying distance-based sampling for sample selec- tion. Our proposed uncertainty-woighteed embedding is computed by weighting sample's feature representation by an uncertainty measure. We show how this embedding generalizes the gradient embedding of BADGE so it can be used with arbitrary loss functions and be computed more efficiently, especially for dense prediction tasks and network architectures with large numbers of parameters in the final layer. We extensively evaluate the proposed hybrid active learning method on image classification, semantic segmentation and object detection tasks, and demonstrate that it achieves state-of-thes performance.
1 Introduction
There has been recent resurgence of interest in using active learning (AL.) to address the annotation burden for training deep neural networks. By selecting only an informative subset of samples to label for model training, AL methods
significantly reduce the cost of data annotation while maintaining satisfactory performance (Ren et al. 2021). Depending on the criterion used to select samples, AL methods can be broadly categorized into uncortainty-based, diversity-basee
and hybrid methods. Uncortainty-based methods (Wang et al. 2016 Gal et al., 2017 Beluch et al. 2018 Yoo Kweon 2019 Liu et al.) 2021
select samples that the model is most uncertain about, while diversity- methods (Sener & Savarese al.) 2021 select subset of samples that are representative of the training data distribution. Employing uncertainty- based sampling alone result in redundancy in selected samples, as similar samples are likely to have similarly high uncertainly values. On the other hand, diversity- based sampling may waste the annotation budget on distinctive yet easy samples. Hybrid approaches aim to overcome these drawbacks by considering both uncertainty and diversity for selecting samples.
2017 Sinha et al. 2019 Caramalau
Various methods have been proposed for hybrid AL. The first approach combine uncertainty and diversity sampling to apply them sequentially, e.g., apply uncortainty sampling select the top-k most uncertain samples (where is larger than the budget size) and then enforce diversity sampling to select subset that
"Joint first authors
Work done during internship with 12R

---

Published in Transactions on Machine Learning Research (06/2024)
satisfies the budget (Yang et al. 2017 A second approach is to sum up uncertainty and diversity terms
in the acquisition function (Yang et al. 2015 Liu & Ferrari 2017 Paul et al. 2017 Cai et al. 2021b). A
third approach is to perform diversity- based sampling on uncertainty-aware features Ash et al. 2019 Kim
ct al. 2021). In particular, the BADGE (Ash et al. 2019) method has been shown to achieve state-of oh-thr--rt
performance on image classification lasks QJi el. al. 2023 BADGE applies diversity sampling on gradient embeddings - the first-order derivative of the loss function with respect to the parameters of the last layer. Gradient embeddings of uncertain samples have large magnitude; this correlation belween uncertainty and embedding magnitude allows BADGE to select a batch of both uncertain and diverse samples.
While BADGE performs well on image classification, three challenges arisc when applying BADGE to other types of tasks. First, the computation of gradient embedding can be expensive for dense prediction tasks like segmentation and for network architectures that have a large number of parameters in the last layer. Second, the gradient embedding only derived for cross-entropy loss in the BADGE paper (Ash et al., 2019); the form of gradient embedding with other loss functions can be complicated and hard to interpret. Finally, computing gradient embeddings on unlabelled data requires pseudo-labels as proxies for true labels. While predicted labels have been shown to be effective proxies for classification tasks, this is not the case for regression tasks- directly using predicted values proxies for the ground truth results in embeddings with zero magnitudes (details in Section 3.1
In this work, we propose a simple yet effective embedding method to address these challenges. Our method is motivated by the observation that the gradient embedding of a neural network model can decomposed into two terms. The first term can be interpreted as an uncertainty measure of the current model on the sample, while the second term is the feature representation extracted from the penultimate layer of the network. We propose to keep this general two-term structure while relaxing the constraint of computing the embedding as the derivative of the loss. We achieve this by instead using an embedding that weights the feature representation by an arbitrary uncertainty term. The proposed uncertainty uncertainty--eightted embedding enables diversity sampling in an uncetaiint--wwar fcature space like gradient embedding, without requiring a specific form of loss function. As a result, our proposed embbdddig can used with any learning lask while also being much faster to compute than BADGE embeddings. We integrate our proposed uncertainty- weighted embedding within distance-based sampling framework to perform hybrid AL, and demonstrate the effectiveness of the proposed AL method wide range of tasks and datasets.
Our contributions can be summarized below:
We propose novel uncertaiiny--weigheedd embedding for hybrid AL. We show that the proposed uncertainty-weightt embedding can be viewed as generalized version of gradient embedding that does not rely on the derivative of the loss function.
We integrate the uncertainty-weighteed embedding within distance-based sampling framework to perform hybrid AL. We extensively evaluate this hybrid AL method on three tasks: image classifica- tion, semantic segmentation and object detection, and demonstrate that it obtains state-of-the-art results on all three tasks.
2 Related Work
2.1 Deep Active Learning
As a promising technique to alleviate the data annotation burden in deep lcarning, AL has recently attracted significant attention from the research community. Depending on the criterion used to select unlabelled data, AL methods can be categorized into uncertainty-based, diversity-based and hybrid methods.
Uncertainty-based methods Methods falling into this group select samples based on uncertainty mea- sure. Some commonly used uncortainty measures include entropy (Shannon 1948). BvSB (Best versus Second Best) (Joshi et al. 2009) and LC (Least Confidence) (Wang et al., (2066). BALD (Gal al.) (0017) selects samples that. maximize the mutual information (i.e., reduction in uncertainty) between predictions
2

---

Published in Transactions on Machine Learning Research (06/2024)
and model posterior. Instead of pre-defining the uncertainty measure, Yoo & Kweon 2019 propose to em-
ploy a loss prediction module to learn the uncertainty. Beluch et al. (2018) show that deep ensembles lead to more calibrated uncertainties and better AL performance. ISAL (Liu et al. 2021] estimates a sample's inffuence on model parameters via influence function and selects samples that can provide the most positive influence.
Diversity-based methods Such methods aim to select a subset of samples that are representative of the training data distribution. CoreSet (Sener & Savarese 2017) employs K-Center-Greedy algorithm to select a core set such that the largest distance between data point and its nearest neighbor in the core set is minimized. VAAL (Sinha et al., 2019) learns latent space in an adversarial manner and selects samples that are most different from labelled ones. Caramalau et al. (2021) build graph convolutional network the features extracted from the target model to learn a better feature space for active selection.
Hybrid methods The various methods of combining uncertainty and diversity sampling in AL can be grouped into three categorics. The first is to apply the two sampling strategies sequentially. Yang et al. (2017) first select lop-k samples with the largest uncertainty scores and then perform diversity sampling on the samples to remove redundancy. Random sampling can be applied before uncertainty sampling to improve the diversity of the selected samples (Beluch et al. 20188] Yoo & Kweon 2019). The second approach is to combine uncertainty and diversity terms by summation in acquisition function. USDM (Yang) et al.] 2015) formulates the batch mode AT. a quadratic programming problem, with the unary term optimized for uncertainty and the pairwise term accounted for diversity. [aul et al. (2017) build graph to reprosent the unlabelled pool and employ submodular objective function to select samples that minimize the joint entropy of the nodes of the graph. The third approach is to perform diversity- sampling on umcertainty-aware features. BADGE (Ash et al. 2019) represents each sample by its gradient embedding and applies K-Means++ seeding algorithm to select samples with large magnitude. TA-VAAL (Kim et al. 2021) conditions the discriminator of VAAL on the learned loss to obtain an uncertainty-avee aware diversity score for sample selection. Our method falls into the third category of hybrid AL. Compared to BADGE, our method does not rely on specific form of loss function, and has significantly reduced computation costs.
2.2 Active Learning for Visual Tasks
AL for image classification AL for image classification has been extensively studied in the literat ure and most methods can be classified into the aforementioned three categories, namely uncertainty- based methods
Gal et al. 2017 Beluch al. 2018
Yoo & Kweon 2019 Liu et al. 2021), diversity- based methods (Sener
& Savarese 2017
Sinha et al.)
2019 Caramalau et al. 2021) and hybrid methods (Ash et all 2019 Ji et al.
(023) conducts comprehensive benchmarking on 7 state-of-the-art AL. methods for image classification,
and shows that BADGE wins the overall ranking. We compare with BADGE in our experiments.
AL for semantic segmentation Semantic segmentation aims to assign class label to each pixel in an image. Based on the granularity of data selection for annotation, previous methods can be classified into imago-based methods (Yang et al.] 2017 Sinha et al.] 2019 and aegion-based methods (Mackowiak et al. 2018 Casanova et al. 2020 Cai et al. 202Tb). Yang et al. (2017) aggregate the pixel-level uncertainties and features to obtain an imaee-level score and feature vector and seloct images to label grcedily. VAAL (Sinha et al. 2019) learns a model to score each image based on its difference from labelled data. Region- based
methods divide image into regularly-shaped (Mackowiak et al. 2018 Casanova et al. 2020 Cai et al, 2021bb or irregularly-shaped (Cai et al. 2021a) regions for selection. We demonstrate our method with both image-level and region-level selection.
AL for object detection Object detection network consists of a backbonc feature extractor, class prediction branch and a location prediction branch. Depending on whether the method relies on some specific architecture design, methods can be divided into architecture--specffc (Roy et al. 2018 Yuan et al.,
2021) and architocture-agnostic YYoo & Kweon) 2019 Agarwal et al., 2020 Wu et al.] 2022) approaches. Roy et al. (2018) measures the uncertainty of a prediction by the disagreement of convolution layers that
predict for the same object in one-stage object detector. MI-AOD (Yuan et. al. 2021 construct detector
3

---

Published in Transactions on Machine Learning Research (06/2024)
with two class prediction heads and one multiple instance learning head, and measure uncertainty by the discrepancy of the two classifiers. The learning loss-based approach LLAT. (Yoo & Kweon 2019: has been employed for object detection by learning the total loss of the detection model. CDAL (Agarwal et al. 2020) exploits the contextual information captured in the predicted probability distribution to enforce contextual diversity selected samples. Pansoovproto (Wu el al. 2022) considers bounding box level similarity to generate imago-lovel uncertainty scores and reject redundant samples. CALD (Yu al., 2022) measures the uncertainty of sample by its prediction consistency between original and augmented views. Our method is architecture-agnostic and can be readily applied to both one-stage and two-stage object detectors.
2.3 Uncertainty Weighting for Active Learning
The idea of using uncortainty to weight feature representation has been explored for active domain adaptation (Prabhu et al.) 2021] h and semi-supervised active learning (Buchert et al. 2022). Prabhu et al. (2021) proposed to perform K-Means clustering on the feature space, with the center of each cluster being updated by senertainty-weighhae averaging. Bucherrt et al. (2022) proposed consistency-
embedding space
for active learning where the feature of each image is weighted by the prediction consistency of the image under various augmentations. Unlike these works which are only applicable to image classification lasks, our proposed hybrid AL framework is general and can be readily applied to wide range of tasks.
3 Method
We consider pool-based batch-mode active learning, where batch of samples is selected from a pool of unlabelled samples in each cycle. Our method, described in Algorithm [ starts with a randomly selected batch and proceeds iteratively until the annotation budget is met. Each iteration involves two main com- ponents: embedding extraction and sample selection. We first provide recap on BADGE with discussions its strengths and weaknesses. We then describe each component of our method in detail, followed by illustrations how it be applied to three different visual tasks, namely, image classification, semantic segmentation and object detection.
Algorithm 1 Active Learning with Uncertainty-Weightee Embeddings
Require: Budgets { b1, b2, .... br]; Dataset D
1: Randomly select bı samples to form the initial labelled pool L and train the initial model by
2: Initialize the unlabeled set 2 = 2
3: for f 2,
do
4: B, =
5: For all samples, compute mucertainty-weigtter embeddings based on Eq using model h:-1
6: For each sample E it compute its minimum distance to C: D(i) minjec []wwe(() wwee())l
7:
while [8x]] <. by do
8:
arg max:gu D(1)
9:
B, <-8
For each sample E 44 update minimum distance L D(1) min(D(1), [[wmee)) uwe(s)])
end while
Annotate the samples in B.
13:
L+ C-LUB, B.
14:
u+u B,
15: Train model h2 on
16:
end for
17:
return model hr
3.1 Recap of BADGE
BADGE selects a batch of diverse and uncertain samples by applying K-Means++ seeding algorithm on gradient embeddings. The gradient embedding of each sample is computed as the first-order derivative of the loss function with respect parameters the final layer. As ground truth label unknown for unlabelled data, BADGE uses the model's current prediction as a proxy of the true label. Let z denote a sample and h denote laylyer neural network parameterized by (01,..., 0z). With softmax activation and
4

---

Published in Transactions on Machine Learning Research (06/2024)
cross-entropy loss, the gradient embedding is computed as:
ge(x:) = ((p: - 100= - 1)) fa; -1.1-1))
(1)
where p; is the predicted probability for class i, f arg [[[K] Ps is the pscudo label for æ, f(x; f(2; 112-11 is the outpul of the network's penultimate layer, and K is the number of classes. The gradient embedding is essentially scaling the feature presentation f(z; by the predicted probabilities. When the model is certain for a sample, Pg would be relatively large and [Ige())]] would be small. The correlation between uncertainty and gradient embedding magnitude allows BADGE to select a batch of both uncertain and diverse samples by K-Means++ seeding algorithm.
BADGE has been shown perform well on image classification task. However, when applying BADGE to other types of tasks, there are three challenges. First, the computation of gradient embedding can be expensive for dense prediction tasks like segmentation, as the derivative needs to be evaluated on every pixel; it is also expensive for network architectures that have large number of parameters in the last layer, e.g., for RetinaNet (more details in Section 5.1). Second, it is common lo employ loss function that is customized for data distribution in some tasks, e.g., focal loss for object detection and dice loss for medical image segmentation. The form of gradient embodding under such loss functions complicated and hard to interpret. Third, for regression task the pseudo label can not be obtained by simply discretizing the prediction classification task does. Let denote the predicted value and y be the ground truth. Tt can be derived that the gradient embedding of L2 loss takes the form 2 - y)0 Directly using as proxy for results in embedding of zero magnitude for all samples.
3.2 Uncertainty- Embedding (UWE)
We note that the gradient embedding of neural network can be decomposed into two terms using the chain rule:
de 0z
ge(x) Oz 00,*
(2)
where = 0g fxx; 01.2-1) is the last layer output before applying activation function. The second term 0s is essentially the fcature representation extracted from the penultimate layer and is independent of loss 3e,
function:
0z a02- f(x; f(x;011-11) (1:001111)
a0,
00,
The first term Øt depends on the form of loss function and activation function. For the two cases discussed 85 above, dz can be derived below:
de
1. Cross-entropy loss with softmax activation:
0= (p. - 1 1yy- K
2. Le loss with identity activation:
at = 2(0 - v),
0=
where y and ÿ is the ground truth and prediction, respectively. We observe that in both cases the magnitude of as can be interpreted as an uncertainty measure of current model on the sample. In the first case, the magnitude of (p+ - (yy = i = would be large if model is uncertain on the prediction (as Py is small); the sccond case, we approximate true value y by the average of an ensemble of models, the magnitude of to relax
()-)) is a measure of dispersion and would be large when uncertainty is high. This motivates the constraint of computing the embedding as the derivative of loss function, and to generate an embedding by directly weighting the feature by uncertainty. Specifically, we define function u(x) to measure the uncertainty of current model on sample x, and a feature extractor f(x) by taking the intermediate output of the network. With the uncertainty measure and feature representation, the proposed uncertainty - embedding is defined as:
uwe(x)) u(x)) f(x)-
(4)
5

---

Published in Transactions on Machine Learning Research (06/2024)
The benefits of the proposed embedding two-fold. First, it shares the same property of gradient embed- ding that the magnitude is positively correlated to sample's uncertainly. As illustrated in Fig. 1 weighting the feature by uncertainty imposes positive correlation between the resulting UWE's magnitude and uncer- tainty. This property facilitates diversity sampling in an uncortainty-awalt feature space by distanco-basod selection (detailed in Section 3.3 Second, it does not rely on a specific form of loss function. Both the uncertainty measure u(x) and feature extractor f(x) can take any off-tho-shelf method instead of being tied to specific form or layer. This makes our method flexible and versatile for different types of tasks.
2.0
2.0
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
0
10 20 30 40 50
0
5 10 15 20 25 30 [[ge(x)]
||uwe(x]]l
(a)
(b)
Figure 1: Weighting the feature by uncertainty imposes positive correlation between the magnitude of UWE and uncertainty. (a) Correlation between the magnitude of UWE and uncertainty (Pearson correlation coefficient م = 0.98). (b) Correlation between the magnitude of gradient embedding and uncertainty (p 0.90). Density plots are drawn on CIFAR-10 using the model trained in the first batch for uncertainty (measured by entropy) and feature extraction.
3.3 Distance-baset Sampling
With uncerrlantyyweiihheed embeddings, selecting samples of large magnitude is more likely to obtain samples of large uncertainty. However, directly selecting the top-k largest ones can result in redundancy as similar samples will have similarly high valuos. We thus propose to employ distance- selection. Distance-basel sampling selects samples based on its distance to previously selected ones. As illustrated in Figs. [a and 2b samples of larger magnitude also have larger Fuclidean distance between them, which are more likely be selected; while similar samples or samples of small magnitude have small Euclidean distance between them, which are unlikely to be repeatedly selected. Therefore, by applying distance-based sampling on uncertainly-aware feature space, we are able to obtain a set of both uncertain and diverse samples.
sampling to perform
There are two popular distance- based sampling methods, CCenter-Greedy algorithm (KCG) used in CoreSet (Sener & Savarese 2017). and K- K-Means++ seeding algorithm (KM++) (Arthur & Vassilvitskii 2006 used in BADGE (Ash al. 2019). Both mothods docide the next sample to select based on its distance its nearest neighbor in previously selected ones. The only difference that KCG deterministically selects the sample with the largest distance in each iteration, while KM++ selects samples probabilistically with probability proportional to the squared distance to the selected samples. The probabilistic nature of KM-+ makes it prone to select samples from high density region, which corresponds to region of small magnitude in the UWE space (as illustrated in Fig. [] samples of low uncertainty have small magnitude and thus concentrate around the origin (high density), while samples of high uncertainty have large magnitude and would spread out in space far from the origin (low density)). Figure 2c visualizes the samples selected by KCG and KM++ KCG is able to select samples of larger uncertainty and diversity. We thus adopt KCG the selection method (see Sec. A.1 for more discussion on selection methods).
3.4 Application to Visual Tasks
Image classification Deep models for multi-class image classification employ softmax activation function on logits and output a probability distribution vector. Sample uncertainty can be measured by entropy. Let denote a sample, p(y illr) the predicted probability for class and K the number of classes, entropy is
6

---

Published in Transactions on Machine Learning Research (06/2024)
20
20
PCA 2nd Dimension
15
10
5
10
-5
2
-10
0
155
KM-
0 10 20 30 40
50
-10
0
10
||uwe(x)]]
PCA 1st Dimension
(a)
(b)
(c)
Figure Distance- based sampling on uncertainty--wiihheed embeddings. (a) Illustration of the selection order for samples by KCG. Samples of small magnitude (in orange) have smaller distance between them and will not be repeatedly selected (for budget 3)- (b) Correlation between ||wwe(z))]| and d(x,, (p = 0.88). d(x, Unn) represents the distance of sample to its nearest neighbor Inn in the dataset. (c) Visualization of the samples selected by KCG and KM++ on the density plot of the UWE space. Plots are drawn on CLFAR-10 using the model trained in the first batch for uncertainty (measured by entropy) and feature extraction, and the visualized samples in (c) are those selected for the second batch.
defined as:
Ment (x) - Σ
-yy- - i|æ) -lgg i|z))
(5)
kE(K]
For feature representation, we take the output of the penultimate layer following CoreSet (Sener & Savarese 2017) and BADGE Ash et al. 2019
Semantic segmentation Semantic segmentation is essentially classification at pixel level. Deep segmen- tation models output a probability distribution over the label space for each pixel the input image. To apply our method for semantic segmentation, we first obtain the pixel-level uncertainty and feature vector in similar way image classification, and then use mcan aggregation to obtain the image-level uncertainty score and feature vector:
u(x; 01.1) = mean(u(p: (1.)),
(6)
((1;0)11-1) = mean(f(p; (1
(7)
where x denotes a sample (an image or patch) and pixel within x. The aggregated uncertainty and feature are then multiplied to form the embedding using Eq. 4
Object detection Given an input image, modern object detectors output a set of detections, each asso- ciated with a confidence score, a class label and four bounding box coordinates (Ren et al. 2015 Lin et al. 2017b). Previous work (Brust et al.) 2018) suggested that the BvSB (Best versus Second Best) metric serves a8 a better uncertainty measure object delection task. As our method is not tied to specific type of loss function or uncertainty measure, this flexibility allows us to select a suitable uncertainty measure for a given task based on prior knowledge. We follow (Brust et al. 2018 measure the uncortainty of cach box by BvSB, which is computed as:
Wboub (x) ply = c+
(8)
ply (x)))
where d and d is the class label for the sccond most confident and most confident prediction. The box- level UWE can be obtained in a similar way to image classification. To obtain imago-level UWEs, we adopt class-wise aggregation, which is more effective than global aggregation due the instance-level processing nature of object detection (Agarwal et al. 2020 Wu et al. 2022). Specifically, each class is represented by the UWE of the box with the largest uncertainty within that class:
(w) u(d:) I(d.), i 1, K
(9)

---

Published in Transactions on Machine Learning Research (06/2024)
where d, arg (..)) u(d), Dz(x) is the set of detections with predicted class label i for image x, and K is the number of classes. The embedding of each class is then concatenated to form the final representation for the image.
4 Experiments
In this section, we report benchmarking results on three different tasks, including image classification, se- mantic segmentation and object detection. All benchmarking methods start from the same initial randomly selected batch (batch 1). We report the mean and standard deviation of independent runs with different random seeds. To more clearly compare the overall performance under different. settings, we summarize the benchmarking results using the pair-wise penalty matrix (PPM) proposed by Ji et al. (2023) More specifically, two AL methods i and j are compared after each batch. If method i outperforms method j, a penalty score of (where is the number of batches) is added to the cell of the PPM at (i,j); likewise, if method j outperforms method i, the penalty score is added to the cell at (j. i). The higher the value at (i, the stronger the method i dominates method j. The column-wise average PPM (marked as Q) indicates the overall performance of each method. The method with the lowest value performs the best. The PPMs over various settings (e.g., different datasets, network architectures) can be combined to form a single one by element-wise averaging.
4.1 Image Classification
Datasets We evaluate method on CIFAR-10 Krizhevsky al. 2009 and CIFAR-100 datasets KKiizhvskyy et al. 2009), each consisting of 50,000 training images and 10,000 testing images of resolu- tion 32 x 32 x 3. We use the training set as the initial unlabelled pool and report the model performance on the testing set.
Implementation details We conduct our experiments on ResNet-18 (He el al. 2016 The hyperparam- eters for training on both CIFAR-10 and CIFAR-100 are set as follows: batch size 256, total epochs = 100, initial learning rate = 2e-2 which is decayed by 0.5 after epoch 60 and 80. The fully-supervised baseline is 94.13% accuracy for CIFAR-10 and 75.36% accuracy for CIFAR100.
Benchmarking results We benchmark with Uncertainly (measured by entropy), CoreSet. (Sner & Savarese 2017), BADGE (Ash et al. 2019). BALD (Gal et al. 2017), LLAL (Yoo & Kweon 2019), VAAL (Sinha et al. 2019). ISAL (Liu et al. 2021 and TA-VAAL (Kim et al. 2021). The results are presented in Fig. 3 We observe that UWE and BADGE perform comparably and outperform the rest. However, embedding our method is much more efficient. than BADGE's gradient embedding (see Section 5.1 for computational complexity analysis). Single criterion-based method, e.g., Entropy, obtains strong performance CIFAR-10 and CIFAR-100. This is probably due to the fact that CIFAR-10 and CIFAR-100 are hand-crafted datasets and contains diverse samples by construction. However, single criterion-based method cannot perform well when the dataset characteristic changes, as we shall see in the following experiments on semantic segmentation.
the computation of uncertainty-
4.2 Semantic Segmentation
Datasets
We evaluate our method on Cityscapes dataset (Cordts et al. 2016). The dataset contains 19 object classes and the image resolution is 1024 x 2048. We use the training set (2,975 images) as the initial unlabelled pool and report the performance on the validation set (500 images). We experiment with both image-level and region-level selection. For region-level AT., each image is divided into patches of size 512 512, and each patch is considered as sample for selection and annotation.
Implementation details We conduct our experiments based on the open-source MMSegmentation (Con-
tributors 2020) framework. The model architecture is DeepLabV3+ with ResNet-18 backbone. The training hyperparameters set below: batch size 8, iterations 80000, initial learning rate 1e-2, which is
8

---

Published in Transactions on Machine Learning Research (06/2024)
BADGE
CoreSet
Entropy
Ramdom
TA-MAL
BALD
ISAL
LLAL
UWE
0.80 0.30 0.20 0.70
0.90
90
70
BALD 8.30 0.00 0.00 0.
0.00
0.10 0.20
0.20
65
100.000.0000
0.20
0.50
80
Entropy 10.00000000 0.300.33000
0.300.60
60
Accuracy
70
Accuracy
93% Sull-SSueeviieed
ISAL 0.10 0.000.00
0.20
0.00
0.50
0.60
0.70
0.20
0.66
Random
55
Uncertainty
Unceriaiity
60
Σ- BADGE
Conseet
50
CoreSett
Random 00 0.160.00
0.000
0.00
0.00
0.10
0.10
BALD
45
1-
BALD
50
LEAL
TAWAAL a 00 0.20 0.00
0.00
0.10
0.00
0.00
TA-MAL
40
TAVAAL
00
ISAL
ISAL
0.2
40
UWE (Ours)
35
UWE 5. 0.80
0.50
0.30 0.50 0.60
0. 0.70
0.70
0.00
WHN
4k
8k
16k
8k
12k
20k
28k
00 0.87
0.12 0.
08
0.12
32 0 0.45 0.40
Number of Annotated Images
Number of Annotated Images
(b)
Figure 3: Active learning results of image classification. (a) and (b).
CIFAR-10; (b) CIFAR-100; (c) PPM over (a)
decayed using the "poly' policy (Ir= (Initiol MinLR) (1- Curiter power + MinnLR where MinLR 0.0001 and power 0.9). The fully supervised baseline is 76.30% mloU.
Maxiter
Benchmarking results We benchmark with Uncertainty (measured by entropy), CoreSet (Sener & Savarese 2017), BADGE (Ash et al.) 2019 and CDAL (Agarwal et al. 2020). The results are presented in Fig. 4 We observe that Uncertainty outperforms CoreSet and CDAL (both are diversity-based sampling methods) in image-level AL, but the order is reversed region-level AL. Region-level AL needs to han- dle a candidate pool with more redundant samples (as neighboring regions may contain similar content) than image-level AL. Single selection criterion, e.g-, uncertainty alone, is unable to accommodate dalasets with different degrees of sample redundancy. Our method is able to outperform competing methods in both cases, demonstrating the advantage of the proposed hybrid sampling method in handling datasets of different characteristics.
BADGE
CoreSet
Entropy
Randemn
COAL
UWE
BADGE
0.00
0.31
0.00
0.00
0.83
0.00
75
0.8
70
70
CoreSet
0.33
0.50
0.00
0.17
1.00
0.00
65
0.6
mloU(%)
65
mlou(%)
Entropy
60
Fully-Supevvised
60
Fully-Sueervised
Random
0.00
0.1.7
0.00
0.00
0.00
0.00
0.4
Random
Random
55
55
BADGE
T
BADGE
COAL
[Ours)
(Ours)
50
0.36
100200 400
0.5k 1k
2k
Number of Annotated Images
Number of Annotated Regions
(a)
(b)
(c)
Figure 4: Active learning results of semantic segmentation. selection; (c) PPM over (a) and (b).
(a) Image-level selection; (b) Region-level

---

Published in Transactions on Machine Learning Research (06/2024)
4.3 Object Detection
Datasets
We evaluate method PASCAL VOC0712 dataset. |Everingham et. al. 2010 There are 20 object classes and the median image shape is 500 375. Following the setting of previous works (Yoo & Kweon 2019 Wu et al. 2022), we combine trainval '7 (5,011 images) and trainval' 12 (11,540 images) to make & super-sel trainval 0012 (16,551 images) and use it as the initial unlabelled pool. The performance is reported on test'00 10 (4,952 images).
Implementation details Our experiments based on the open-source MMDetection (Chen et al. 2019
framework. We experiment with onc-stage detector RetinaNet (Lin et al. 2017b) well as two-stage detector Faster R-CNN RRe et al. 2015). Both detectors use fealure pyramid nel works (Lin et al. 2017a on top of ResNet-50 (He et al. 2016) as feature extractor. The hyperparameters are set as follows: batch size = 2, total epochs 26, initial learning rate = le-3 for RetinaNet and 5e-3 for Faster R-CNN, which is decayed by 0.1 after 20 epochs. When training with the full trainval P0712, we obtained a mAP of 79.87% for RetinaNet and 78.73% for Faster R-CNN.
Benchmarking results We benchmark with Uncertainty (measured by BvSB), CoreSet Sener & Savarese 2017), BADGE (Ash et al. 2019), CDAL (Agarwal et al. 2020), LLAL YYo & Kweon 2019) EnnsDivproto (Wu et al., 2022) and CALD Yu al. 2022). The results are presented in Fig. We observe that not every method perform well with the former, but the performance suffers with the latter. UWE is able to consistently per- form the best on both detectors. EnnssDvvrooto also achieves strong performance on both detectors, but the method is more complicated, involving additional steps for entropy-based non- maximum suppression and pseudo-label based minority class sampling on top of diversity-basel sampling on class prototypes, while our method is simple and efficient (up to 4x faster than EnmsDivproto in selection), as it only involves diversity-based sampling on class-wing aggregated features.
perform well on both Faster R-CNN and RetinaNet, e.g., CALD and BADGE
BADGE
CoreSet
EnmiDDiv
Random
Uncert
CALD
COAL
LLAL
UWE
1.0
BADGE
0.00
0.17
0.00
0.50
0.37
1k 2k
4k
8k
1k
2k
4k
8k
80
CALD
0.37
75
0.8
CDAL
0.00
70
CoreSet
0.6
70
60
0.00
0.00
mAP(%)
96% Fully- Sully-Superiieed
MAP(%)
96% Fully-Superiied
LLAL
0.00
0.00
0.67
0.00
0.00
Uncertainty
Uncortainty
65
CoreSit
CDA)
50
CoreSet
CDAL
Random
0.17 0.00
0.00
0.83
0.00
0.33
0.00
0.00
0.00
0.4
LLAL
LLAL
Uncert
0.17 0.17
0.00
1.00
0.00
0.67
0.33
0.00
0.00
BADGE
EnmsDivpprtt
CALD
40
BADGE
EnmsDDipprtt
CALD
UWE 0.50 0.17
0.83
1.00
0.50
1.00
1.00
0.17 0.00
0.2
60
UWE (Ours)
UWE (Ours)
Number of Annotated lmages
Number of Annotated Images
0.04
0.20
0.05
0.0
(a)
(b)
(c)
Figure 5: Active learning results of object detection. (a) Faster R-CNN; (b) RetinaNet; (c) PPM over (a)
and (b).
4.4 Ablation Studies
We report ablation studies on the design components of our method in Table Here, U represents uncer- tainty sampling that selects the top-k most uncertain samples, F represents diversity sampling using KCG without uncortainty weighting, í.e., setting u(x) in Eq. 4 to 1, and U+F refers the proposed method. The consistent performance gain of U+F over demonstrates the advantage performing diversity sampling the proposed uncertainty-weighted features.
10

---

Published in Transactions on Machine Learning Research (06/2024)
Table 1: Ablation studies on the design choices of our method. represents uncertainty sampling that selects the top-k most uncertain samples, F represents diversity sampling using KCG without uncertainty weighting, and U+F is the proposed method.
U F Batch 2
Batch 3
Batch
Image classification on CIFAR10 with ResNet-18
46.96 (0.29)
60.08 (0.08)
76.02 (0.20)
48.61 (0.71)
60.46 (0.14)
74.35 (0.41)
49.45 (0.41)
61.84 (0.37)
76.37 (0.99)
Semantic segmentation on Cityscapes with DeepLabV3+
61,63 (0.85)
66.88 (0.07)
71.77 (0.59)
61.82 (1.04)
68.19 (1.25)
71.75 (0.57)
62.78 (0.98)
68.42 (0.13)
72.87 (0.71)
Object detection on VOC0712 with Faster R-CNN
67.53 (0.40)
71.61 (0.23)
75.51 (0.23)
67.20 (0.29)
71.75 (0.38)
75.83 (0.51)
68.09 (0.10)
72.51 (0.37)
76.38 (0.32)
5
Further Analysis
5.1 Computational Complexity Analysis
We analyze the computational complexity of our method to demonstrate its efficiency over BADGE. Let faim denote the dimension of the feature appresnntation, and ne the number of classes. For image classification and segmentation, the dimension of UWE is the same as faim, while the dimension of the gradient embedding used in BADGE is n. fam. The time complexity of KCG and KM++ is O(ns nh -adim), where nh is the number of selected samples, ny is the total number of unlabelled samples, and Addi is the input sample dimension. As such, our method is more efficient than BADGE by factor of ne- For object detection, the dimension of UWE is ne faom due to class-wise aggregation. The dimension of gradient embedding varies according to detector architectures and is typically much higher as it not only depends on the number of classes, but also other network parameters. For example, the dimension of gradient embedding for RetinaNet is 122,na R. fdim; where na is the number of anchors and is the filter size for the prediction head. Table 2 compares the embedding dimension, embedding extraction time and sample selection time on BADGE vs. UWE. Our method reduces the storage and time cost significantly compared to BADGE, especially when the number of classes is large (i.e., on CIFAR-100) and for dense prediction task (í.e., on Cityscapes).
Table 2:
The dimension of the embedding and rumning time (in seconds) for embedding extraction and sample selection. The time cost measured for selecting the second batch.
Dimension
Extraction
Selection
Image classification CIFAR-10 with ResNet-18
BADGE
5120
18.84 (1.30)
129.95 (1.23)
UWE
512
4.78 (0.05)
22.55 (0.28)
Image classification on CIFAR 100 with ResNett-18
BADGE
51200
126.87 (12.20)
2223.94 (0.18)
UWE
512
4.98 (0.03)
42.16 (0.08)
Semantic segmentation Cityscapes with DeepLabV34
BADGE
2432
1593.17 (618.16)
34.18 (5.27)
UWE
128
997.26 (12.13)
23.22 (0.34)
Object detection on VOC0712 with RetinaNet
BADGE UWE
414720
1651.34 (21.22)
200.66 (30.87)
5120
1181.62 (166.13)
82.33 (2.62)
11

---

Entropy Published in Transactions on Machine Learning Research (06/2024)
5.2 Adjusting Trade-off between Uncertainty and Diversity Sampling
In Section 4.2 we observe that uncertainty sampling outperforms diversity sampling in image-level AL, but in region-level AL the order is reversed. Intuitively, when region size is large, candidate samples are already diverse and uncertainty sampling that focuses on sclecting difficult samples would outperform. On the other hand, when region size is small, there are more redundant samples, and thus a selection strategy that encourages diversity sampling would perforrmm This suggests that the trade-off between uncertainty and diversity sampling needs be adjusted for different region sizes. This can be achieved by applying a power to the uncertainty term in Eq. + i.e., computing the embedding as uwe(x) = u(x)) f(z). Increasing would up-weight the contribution of uncertainty and select more uncertain samples, while decreasing т would down-weight the role of uncertainty and select more diverse samples. In Section 4.2 we show that T T= works well for 1024 x 2048 and 512 x 512. If the region size is further reduced, a smaller T should be more suitable. To verify this, we experiment with smaller region sizes using reduced т. Results are reported in Table 3. When region size 256 x 256, we observe some marginal advantage of UWE (T = 0.1) over UWE The advantage is more significant when the region size is further reduced to 128 128. We observe that CoreSet significantly outperforms Entropy this region size, confirming the advantage of diversily sampling for datasets with more redundant samples. UWE with 0.1 further improve upon CoreSet, demonstrating that a hybrid approach that considers uncertainty in diversity sampling is still beneficial.
Table 3: Effect of applying power т to the uncertainty term in computing the embedding.
Batch 2
Batch 3
Batch 4
Region size 256 256
Random
60.35 (0.17)
64.75 (0.38)
68.70 (0.59)
Entropy
67.18 (0.52)
71.41 (0.13)
72.67 (0.68)
CoreSct
66.38 (0.08)
71.19 (0.36)
72.86 (0.31)
BADGE
65.10 (0.21)
69.87 (0.14)
70.03 (1.15)
UWE (T
67.21 (0.46)
71.21 (0.80)
73.72 (0.24)
UWF (T 0.1)
67.23 (0.21)
71.65 (0.67)
73.71 (0.18)
Region size 128 128
Random
62.56 (0.09)
67.38 (0.29)
69.05 (1.70)
70.09 (0.33)
69.99 (0.43)
71.38 (0.09)
CoreSet RADGE
70.41 (0.34)
72.49 (0.54)
74.28 (0.47)
68.50 (0.00)
71.91 (0.00)
71.78 (0.00)
UWE (r
69.80 (0.26)
72.10 (0.59)
73.40 (0.13)
UWE (r 0.1)
70.75 (0.49)
73.15 (0.24)
74.40 (0.37)
5.3 Effect of Uncertainty Measures
The u(x) term in Eq. 44 can be an arbitrary measure of uncertainly. We experiment with three commonly used measures: Entropy, BvSB and LC. Figure 6 shows the results. While the performance of different uncertainty measures varies across different datasets and tasks, our method is able to consistently improve upon the performance of uncertainty sampling. This demonstrates the effectiveness of UWE as a general method to perform hybrid AL.
5.4 Correlation between Gradient Embedding and UWE
We inspect the correlation between the magnitude of gradient embedding and UWE to shed more insight into the proposed two-term decomposition of gradient embedding. Mathematically, the magnitude of gradient embedding under cross-entropy loss and softmax activation function can be computed as:
l1ge(z, 00ll
Σ
p 2
+1 11122p,) 22,)
(10)
where pa is the probability for the pseudo label. The magnitude of UWE can be computed as:
[]wee(>))] u(x) [I]()]]
(11)
12

---

Published in Transactions on Machine Learning Research (06/2024)
75
Accuracy
70
70-
mAP(%)
60
(Bv58)
UWE (BvSB)
Uncertainty (BvS8)
65
Uncertainty (BvSB)
UWE [Ent]
T
UWE (Ent)
Uncertaitty (End)
Uncertainty (Ent)
40
Uncertainly (UC)
UWE (LC)
4k
8k
16k
60
Uncertainty (LC)
1k 2k
4k
8k
Number of Annotated Images
Number Annotated Images
(a)
(b)
Figure 6: Performance of UWE with different uncertainty measures. (a) Image classification on CIFAR-10
with ResNet-18; (b) Object detection on VOC0712 with Faster R-CNN.
It is clear that [gge(x, 0)ll and |]]ee[=)]] share the same second term [If()))] (the feature representation). while the correlation between the first terms depends on the choice of uncertainty measure and the joint distribution p(r, We compute the Pearson correlation cocfficient between llge(æz, 0)ll and [[uwe(x) |]- Pearson correlation coefficient measures the linear correlation between two sets of data, with 1/-1 indicating Deeitive/negative
perfect positive 'egative correlation, and 0 indicating no linear correlation. Results are reported in Table 4 We observe that the magnitude of BADGE's gradient embedding and UWE is highly correlated, achieving correlation coefficient above 0.9. We further inspect how well BADGE and UWE in approximating real gradient, i.e., gradient embedding computed with ground truth label. Results in Table 5 show that UWE obtains similar, or sometimes higher correlation coefficients than BADGE, suggesting that UWE performs comparably to BADGE with regard to approximating the magnitude of real gradient.
Table 4: Correlation between the magnitude of BADGE's gradient embedding and UWE We report the mean and standard deviation of runs.
Batch
Batch
Batch 3
Batch
Batch
Ratch 6
CIFAR-10
0.94 (0.00)
0.94 (0.00)
0.95 (0.00)
0.95 (0.00)
0.95 (0.00)
0.95 (0.00)
CIFAR-100
0.93 (0.00)
0.93 (0.00)
0.94 (0.01)
0.93 (0.00)
0.93 (0.00)
0.94 (0.00)
Table 5: Correlation between the magnitude of real gradient (using ground truth label) and the magnitude of BADGE's gradient embedding (using psendo label) and UWE, respectively. We report the mean and standard deviation of 3 runs.
Batch
Batch
Batch
Batch
Batch 5
Batch
Image classification on CIFAR-10 10 with ResNet-118
BADGE 0.11 (0.00)
0.19 (0.02)
0.28 (0.00)
0.37 (0.00)
0.44 (0.01)
0.51 (0.00)
UWE
0.12 (0.00)
0.22 (0.02)
0.32 (0.01)
0.42 (0.00)
0.48 (0.01)
0.55 (0.00)
Image classification on CIFAR-100 with ResNet-18
BADGE 0.46 (0.00)
0.47 (0.03)
0.55 (0.00)
0.59 (0.04)
0.61 (0.01)
0.67 (0.02)
UWE 0.47 (0.00)
0.48 (0.03)
0.56 (0.00)
0.59 (0.04)
0.61 (0.01)
0.65 (0.03)
6 Conclusions
In this work, we proposed a novel uncortainty-wcighted embedding for hybrid AL. Our proposod uncertainty- weighted embedding generalizes the gradient embedding form of BADGE by allowing arbitrary uncertainty measures to be used, thereby greatly reducing the computational complexity of computing the embedding,
13

---

Published in Transactions on Machine Learning Research (06/2024)
and allowing it to be used with arbitrary loss functions. Our hybrid AL method then utilizes this embedding as basis for distance-based sampling to select examples for labelling. We evaluate our method under a wide range of tasks and settings, e.g., image classification, image-level and region-level AL for semantic segmentation, one-stage and two-stage detectors for object detection. Experimental results show that our method achieves state-of-the-arl performance, and demonstrate the generality of our approach.
Acknowledgements
The authors would like to thank the anonymous TMLR reviewers for their constructive leedback during the review process. This work is supported by the Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funds (Grant No. A20H6b0151) and Career Development Fund (Grant No. C210812052). The computational work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (https:/ htttps//wwww.r www.ssc.ss).....................
References
Sharat. Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. Contextual diversity for active learning. In European Conference on Computer Vision, pp. 137-153. Springer, 2020.
David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seeding. Technical report, Stanford, 2006.
Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv: 1Xii:1906.00 190.00990,,,7,,, 2019.
William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensembles for active learning in image classification. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 9368 9377, 2018.
Comens-Alexandee Brust, Christoph Käding, and Joachim Denzler. Active learning for deep object detection. arXiv preprint (rXii:180900987: 2018.
Felix Buchert, Nassir Navab, and Seong Tae Kim. Exploiting diversity of unlabeled data for label-efficient semi-supervised active learning. In 2022 26th International Conference on Pattern Recognition (ICPR), pp. 2063-2000000 2069. IEEE, 2022.
Lile Cai, Xun Xu, Jun Hao Liew, and Chuan Sheng Foo. Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognétion, pp.
2021a.
Lile Cai, Xun Xu, Lining Zhang, and Chuan-Sheng Foo. Exploring spatial diversity for region-basod active learning. IEEE Transactions on Image Processing, 30:8702 8712, 2021b.
Razvan Caramalau, Binod Bhattarai, and Tae-Kyun Kim. Sequential graph convolutional network for active learning. In Proceedings of the IEFE/CVF conference computer vision and pattern recognition, pp. 2021.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégon, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervieed vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8650-96600 2021.
Arantxa Casanova, Pedro O Pinheiro, Negar Rostamzadeh, and Christopher J Pal. Reinforced active learning for image segmentation. arXio preprint arXim:2002.06: 06588, 2020.
Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qjjie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1000 1906000066 2019.
14

---

Published in Transactions on Machine Learning Research (06/2024)
MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and bench- 2020.
mark. http:/ittuub.oooppeeeeeeeeeeeeee http://ww.....ooooooooooaaaaaaaa
Marius
Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understand- ing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213 3223, 2016.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1 11929, 2020.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88())3003,338, 2010.
Yarin Gal, Riashat, Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. International Conference on Machine Learning, pp. 1183 1192. PMLR, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770- 708, 2016.
Yilin Ji, Daniel Kaestner, Oliver Wirth, and Christian Wressnegger. Randomness is the root of all evil: More reliable evaluation of deep active learning. In Proceedings of the IEEE/CVF Winter Conference on Applicalions of Compuler Vision, pp. 3943 033522 2023.
Ajay J Joshi, Fatih Porikli, and Nikolaos Papamikolopoulos. Multi-class active learning for image classifica- tion. In 2009 seee conference on computer vision and pattern recognition, pp.
TEFF, 2009.
Kwanyoung Kim, Dongwon Park, Kwang In Kim, and Se Young Chun. Task- variational adversarial active learning. In Proceedings of the IEEE/CT E/VF Conference Computer Vision and Pottern Recognition, pp. 8166 8175, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Tsung-Yi Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 2117-2125, 2017a
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017b.
Buyu Liu and Vittorio Ferrari. Active learning for human pose estimation. In Proceedings of the IEEE International Conference on Computer Vision, pp.
2017.
Zhuoming Liu, Hao Ding, Huaping Zhong, Weijia Li, Jifeng Dai, and Conghui He. Influence selection for active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9274-9283, 2021.
Radek Mackowiak, Philip Lenz, Omair Ghori, Ferran Diego, Oliver Lange, and Carsten Rother. Cereals- cost-cffective region-bass active learning for semantic segmentation. arXiv preprint orXiw:1810.09996, 2018.
Sujoy Paul, Jawadul H Bappy, and Amit K Roy- Kly-Chhoddhrry Non-uniform subset selection for active learning in structured data. In Proceedings of the IEEE conference. on computer vision and pattern recognilion, pp. 6846-6855, 2017.
Viraj Prablu, Arjun Chandrasckaran, Kate Sacnko, and Judy Hoffman. Active domain adaptation via clustering mucertainty-weighee embeddings. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8505-8800 05155854,, 2021.
15

---

Published in Transactions on Machine Learning Research (06/2024)
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po- Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM compuling surveys (CSUR), 54(9):1-40, 2021.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural informalion processing systems, 28, 2015.
Soumya Roy, Asim Unmesh, and Vinay P Namboodiri. Deep active learning for object detection. In BMVC, pp. 91, 2018.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint Xiv: 1708: 0088, 2017.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27 (3):379-423, 1948.
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 5972-5981, 2019.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advonces in neural information processing systems, 29, 2016.
Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, and Liang Lin. Cost-effective active learning for deep image classification. IEEE Transactions on Circuits and Systems for Video Technology, 27(12))2591-2600 2016.
Jiaxi Wu, Jiaxin Chen, and Di Huang. Entropy-based active learning for object detection with progres- sive diversity constraint. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9397 9406, 2022.
Lin Yang. Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. Suggestive annotation: A deep active learning framework for biomedical image segmentation. In International conference on medical image computing and computer-assio assisted intervention, pp. 399-407. Springer, 2017.
Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, and Alexander G Hauplmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 113:113- 127, 2015.
Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE/CVF conference computer vision and paltern recognition, pp. 93 102, 2019.
Weiping Yu, Sijic Zhu, Taojiannan Yang, and Chen Chen. Consistency-base active learning for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 3951-3960, 2022.
Tianning Yuan, Fang Wan, Mengying Fu, Jianzhuang Liu, Songcen Xu, Xiangyang Ji, and Qixiang Ye. Multiple instance active learning for object detection. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 5330-5339, 2021.
A Appendix
A.1 Effect of Selection Methods
We compare the uncertainty and diversity of samples selected by KCG and KM++ on both UWE and gradient embedding. Diversity is measured by the average pairwise distance of samples within a batch. Results are presented in Fig. 7 KCG selects samples of larger uncertainty, embedding magnitude and diversity on both embeddings. We also follow BADGE measure diversity by the log determinant of the selected samples' Gram matrix Table G. We observe similar results to those reported in Fig. 7 í.c., KCG is able to select. more diverse batch than KM++. We further experiment with replacing KM++ used in
16

---

Published in Transactions on Machine Learning Research (06/2024)
BADGE by KCG. The results presented in Fig. g demonstrate that KCG performs better than KM++ gradient embeddings as well. One of the weaknesses of KCG is that it may select outliers that are far away from each other (Sener & Savarese 2017). However, public benchmarking datasets are usually carefully crafted and may not suffer from the outlier problem. As a result, the deterministic nature of KCG allows it to select more diverse and uncertain samples and performs better than KM++.
1.4
27.5
Avg |Juwe (ell In Batch
25.0
1.2
25
lluwert) -ww
22.5
Avg ubx) in
1.0
20.0
20
17.5
15.0
15
0.6
KM++
12.5
KM++
KCG
KCG
KCG
10.0
0.4
16k
16k
Number Annotated Images
Number Annotated Images
Number of Annotated Images
(a)
(b)
(c)
1.1
25.0
in Batch
1.0
Avg [[ge())) In Batch
16
22.5
GeUiil in
0.9
20.0
0.8
12
17.5
0.7
100
15.0
Avg
0.6
8
12.5
KM++
KM++
KM+
10.0
KCG
KCG
KCG
1K2K 4k
8k
16k
1K2k
4k
8K
16k
1K2K
4k
8k
16k
Number Annotated Images
Number of Annotated Images
Number of Annotated Images
(d)
(e)
(f)
Figure Comparing uncertainty and diversity of samples selected by KCG and KM+++ on CIFAR-10.
Top row: results umcertainty-weigheee embeddings; bottom results gradient embeddings. (a)(d): Comparing selected samples' uncertainty; (b) Comparing selected samples' embedding magnitude; (f)(f): Comparing selected samples' diversity (measured by average pairwise feature distance within batch).
Table 6: The log determinant of Gram matrix of samples selected by KCG KM++ CIFAR-10. We compute the Gram matrix on both the selected samples' UWFs and their original (unweighted) features. KCG obtains larger determinant values in both cases, indicating the selected samples are more diverse.
Log determinant Gram matrix of UWEs in batch
Batch (Budget)
(500)
3 (1000)
(2000)
KM I
813.86 (5,04)
-11789.8 39 (70.67)
-37898.37 (7.50)
KCG
1353.61 (4.02)
-10780.49 (18.76)
-36313.16 (70.82)
Log determinant of Gram matrix of (unweighted) features in batch
KM++ KCG 1088.14 (0.31)
1034.34 (0.38)
-1704.00 (3.20)
-7499.03 (0.48)
-1643.87 (3.35)
-7470.63 (25.94)
A.2 Effect of Aggregation for Object Detection
We use Mar aggregation to generate class-wise UWE (Eq. 9 for object detection. Alternatively, the class-wise UWE can be obtained by averaging the box-level UWE within each class, i.e., uwe;(x) Eaep,e, f(d) where D.(x) is the set of detections with predicted class label i for image Table 7
D.(x)
17

---

Published in Transactions on Machine Learning Research (06/2024)
70
65
mlo
mAP(%)
60
55
GE (KCG)
(KM++)
UWE (KCG)
(OK++ ++++)
[KM++)
UWE (KCG)
100
800
(KM++)
Number of Annotated Images
1k
2k
4k
8K
Number Annotated Images
(b)
Figure 8: Replacing KM++ BADGE with KCG. KCG performs better than KM++ on both gradient embeddings and eecertainty-weighteee embeddings. (a) Semantic segmentation on Cityscapes; (b) Object detection VOC0712 with Faster R-CNN.
compares the performance of Max us. Meon aggregation. We observe that Maz performs better or compa- rably to Mean. The reason may be that Mar generate more distinctive embedding cach class is represented by its most uncertain object instead of its "average" object, and more distinctive embeddings facilitate the selection of more informative samples by distance-based sampling.
Table 7: Comparing Mar us. Mean aggregation for generating class-wise UWE for object detection.
Batch 2
Batch 3
Batch 4
Object detection on VOC0712 with Faster R-CNN
Mean
67.18 (0.66)
72.03 (0.39)
75.69 (0.19)
Mar
68.09 (0.09)
72.50 (0.37)
76.37 (0.31)
Object detection VOC0712 with RetinaNet
Mean
64.08 (0.38)
72.79 (0.09)
77.62 (0.20)
Mar
64.61 (1.10)
72.58 (0.12)
77.59 (0.10)
A.3 BADGE for Object Detection
To evaluate BADGE's effectiveness for object detection, we experiment with variants of BADGE. Modern object detectors like RetinaNet (Lin et al. 2017b) and Faster R-CNN (Ren et al.] 2015 typically consists of two prediction branches, i.c., classification (CLS) and regression (REG). Two-stage detector like Faster R-CNN has another prediction branch for region proposal (RPN, which itself consists of classification and regression head). We experiments with three variants of BADGE, i.e., BADGE (CLS), BADGE (CLS+REG) and BADGE (CLS+REG+RPG), where the last layer paramelers of the corresponding branches are taken into consideration when computing the gradient embedding. We observe that BADGE (CLS) performs the best, while using the gradient embedding from the regression branch or region proposal network harms the performance. These results suggest that BADGE is not guaranteed to work well for complicated tasks like object detection.
18

---

Published in Transactions on Machine Learning Research (06/2024)
75.0
72.5
70.0
60
mAP(N)
mAP(35)
67.5
50-
62.5
BADGE (CLS)
40
(CLS+REG)
BADGE (CLS+REG+RPN)
BADGE (CLS+REG)
60.0
UWE (Durs)
UWE (Ours)
Number Annotated Images
Number Arnotated Images
(a)
(b)
Figure 9: BADGE for object detection with variants of gradient embeddinggs. (a) Results on VOC0712 with Faster R-CNN; (b) Results on VOC0712 with RetinaNet.
A.4 Uncertainty and Diversity of Selected Samples
To validate our method is indeed able to select set of both uncertain and diverse samples, we compare the mean uncertainty and diversily (measured by average pairwise feature distance within a batch) of samples selected by different methods in Fig. [0] We observe that UWE obtains the second best regarding to uncer- tainty, just behind Uncertainty sampling, which selects samples of top-k uncertainty. However, Uncertainty sampling does not consider diversity, thus ranking bottom regarding to diversity for image classification. Our method, being a hybrid method, is still able to achieve good diversity for image classification and ranks first for object detection.
A.5 Results on mini-lmagener
We present the results on mini-Imagenot (Vinyals et al.) 2016 in Fig. 11 Mini-Imagenct contains 48,000 images for training and 12,000 images for lesting, evenly distributed across 100 classes. The image resolution is 84 84 3. We use the training set as the initial unlabelled pool and the testing set to evaluate model performance. The network architecture is Vision Transformers (ViT-Base) (Dosovitskiy et al. 20200), which is initialized with ImageNet pre-trained weights (self-supervised pre-training with DINO (Carrn et al. 2021 At each AI, cycle, the model is trained for 20 epochs by ADAM optimizer with learning rale 5e-5 and batch size 64. The fully-supervised baseline achieves 80.60% in testing accuracy. We use BvSB as uncertainty measure for UWE. We observe that UWE performs slightly better than BADGE, and outperform others significantly.
A.6 Qualitative Results
We provide detailed detection results of models trained with samples selected by different active learning methods in Fig. 12 Our method is able to obtain more accurate detection results, with fewer false positives and false negatives (e.g., potted plant in Fig. 12)
(e.g., dog/cat cat in Fig. 12 (b)
A.7 Numerical Results
We provide the raw data for Figs.3, 4 and 5 in Table 5 and 10 respectively. Our method (UWE) achieves best or second best performance in most cases.
19

---

Published in Transactions on Machine Learning Research (06/2024)
Batch (Budget)
1 (500)
(500)
(1000)
4 (2000)
5 (4000)
(8000)
ResNet-18 on CIFAR-10
Random
41.3 (0.49)
49.97 (1.16)
60.31 (0.11)
72.78 (0.17)
82.42 (0.66)
88.21 (0.04)
Uncertainty
41.3 (0.49)
46.96 (0.29)
60.08 (0.09)
76.02 (0.20)
85.96 (0.19)
91.60 (0.07)
CoreSet
41.3 (0.49)
48.61 (0.71)
60.46 (0.14)
74.35 (0.41)
84.11 (0.28)
90.11 (0.05)
BADGE
41.3 (0.49)
48.35 (0.51)
61.56 (1.02)
75.75 (0.52)
85.64 (0.42)
91.40 (0.22)
BALD
41.3 (0.49)
50.09 (0.65)
59.68 (0.27)
73.63 (0.31)
82.99 (0.58)
88.58 (0.25)
LLAL
41.3 (0.49)
48.73 (0.68)
59.89 (0.23)
74.37 (0.23)
84.77 (0.43)
90.50 (0.12)
VAAL
41.3 (0.49)
48.26 (0.72)
59.09 (0.13)
72.62 (0.58)
82.99 (0.32)
88.39 (0.10)
TA-VAAL
41.3 (0.49)
48.53 (1.23)
59.45 (1.21)
72.90 (1.27)
82.63 (0.96)
88.71 (0.37)
ISAL
41.3 (0.49)
49.31 (0.90)
60.59 (1.12)
75.15
(1.25)
85.26 (0.17)
90.55 (0.25)
UWE (Ours)
41.3 (0.49)
49.46 (0.41)
61.84 (0.37)
76.38
(1.00)
85.85 (0.51)
91.46 (0.08)
Batch (Budget)
(5000)
2 (1000)
(2000)
(4000)
(8000)
(8000)
ResNet-18 on CIFAR-100
Random
33.83 (0.08)
37.87 (0.28)
44.79 (0.71)
53.77
(0.30)
63.28
(0.05)
68.68 (0.46)
Uncertainty
33.83 (0.08)
36.98 (0.39)
45.99
(0.14)
55.66 (0.58)
66.23 (0.03)
71.41 (0.11)
CoreSet
33.83 (0.08)
38.54 (0.59)
45.45 (1.57)
55.52 (0.35)
65.63 (0.10)
70.62 (0.20)
BADGE
33.83 (0.08)
38.20 (0.25)
45.98 (0.38)
55.69
(0.07)
66.51 (0.03)
70.99 (0.03)
BALD
33.83 (0.08)
38.10 (0.06)
44.93 (0.16)
54.02 (0.27)
63.39 (0.13)
67.53 (0.09)
LLAL
33.83 (0.08)
37.54 (0.28)
43.09 (1.05)
52.79 (0.03)
65.09 (0.41)
69.97 (0.16)
VAAL
33.83 (0.08)
38.73 (0.30)
44.84 (0.36)
53.45 (0.14)
63.35 (0.47)
68.74 (0.15)
TA-VAAL
33.83 (0.08)
38.07 (0.39)
44.37 (0.46)
53.50 (0.58)
63.96 (0.17)
68.77 (0.24)
ISAL
33.83 (0.08)
39.17 (0.36)
45.92 (0.98)
55.57 (0.55)
65.86 (0.28)
70.46 (0.36)
UWE (Ours)
33.83 (0.08)
38.49 (0.55)
45.28 (0.08)
55.59 (0.80)
67.11 (0.48)
72.09 (0.12)
Table 8: Benchmarking results for image classification. Best results are bolded, and the second best results are underlined.
Batch (Budget)
1 (100)
(100)
3 (200)
4 (400)
D on Cityscapes (Imago-level AL)
Random
52.54 (1.05)
59.24 (0.36)
65.44 (0.22)
70.83 (0.02)
Uncertainty
52.54 (1.05)
63.55 (0.48)
70.09 (0.35)
73.80 (0.03)
CoreSct
52.54 (1.05)
62.06 (0.14)
69.24 (0.97)
73.89 (0.09)
CDAL
52.54 (1.05)
58.00 (0.94)
65.11 (0.54)
71.81 (0.52)
BADGE
52.54 (1.05)
61.28 (1.43)
67.87 (0.71)
72.77 (0.26)
UWE (Ours)
52.54 (1.05)
63.54 (1.06)
69.83 (0.49)
74.09
(0.03)
Batch (Budget)
1 (500)
(500)
(1000)
(2000)
DeepLabv3+ on Cityscapes (Region-level AL with region size 512 x 512)
Random
50.71 (1.19)
56.94 (0.67)
63.73 (1.23)
69.00 (0.38)
Uncertainty
50.71 (1.19)
61.63 (0.85)
66.88 (0.07)
71.77 (0.59)
CoreSet
50.71 (1.19)
61.82 (1.04)
68.19 (1.25)
71.75 (0.57)
CDAL
50.71 (1.19)
61.01 (0.11)
68.03 (0.77)
72.75 (1.09)
BADGE
50.71 (1.19)
61.60 (0.83)
67.07 (0.16)
71.24 (0.04)
UWE (Ours)
50.71 (1.19)
62.78 (0.98)
68.42 (0.13)
72.87 (0.71)
Table 9: Benchmarking results for semantic sogmentation. Best results are bolded, and the second bost results are underlined.
20

---

Published in Transactions on Machine Learning Research (06/2024)
Bardom Uncertainty
Bandom
unsertality
CoreSet
ConeSet
144
BALD
LAA
LLAL
VAAL
26
VAAL
1.2
VALLLLL
TA
1.0
uscora
0.4
0.2
16
Baach
Batch
Randimm
0.75
Uncertainty
CDAL
GALD
0.70
EnmsDivproto
UWE (Ours)
0.65
Diversity
0.60
88
Random
0.55
CoreSet
CDML
0.50
BADGE
Enmisiipprat
(Durs)
0.45
Batch
Batch
(c)
(d)
Figure 10: Comparing uncertainty and diversity of samples selected by different AL methods. (a)(b): Image
classification on CIFAR-10 with ResNet-18; (d): Object detection on VOC0712 with RetinaNet.
21

---

Published in Transactions on Machine Learning Research (06/2024)
77.5
75.0
72.5
Accuracy (%)
70.0
67.5
95% Fully-Supervised
65.0
Random
Uncertainty
62.5
Coreset
BADGE
ISAL
60.0
UWE (Ours)
1k
4k
8k
Number of Annotated Images
(a)
Figure 11: Active learning results on mini-Imagenet.
Batch (Budget)
(1000)
2 (1000)
(2000)
4 (4000)
Faster R-CNN on VOC0712
Random
59.79 (0.23)
66.22 (0.32)
70.78 (0.30)
75.14 (0.40)
Uncertainty
59.79 (0.23)
67.53 (0.40)
71.61 (0.23)
75.51 (0.23)
CoreSet
59.79 (0.23)
64.64 (0.62)
69.11 (0.32)
73.39 (0.44)
CDAL
59.79 (0.23)
66.95 (0.36)
71.35 (0.38)
75.31 (0.07)
LLAL
59.79 (0.23)
66.23 (0.21)
70.42 (0.23)
74.83 (0.31)
BADGE
59.79 (0.23)
66.98 (0.09)
71.63 (0.11)
75.70 (0.25)
Einsoivproto
59.79 (0.23)
67.44 (0.25)
71.94 (0.22)
75.64 (0.69)
CALD
59.79 (0.23)
67.75 (0.43)
72.81 (0.40)
76.15 (0.33)
UWE (Ours)
59.79 (0.23)
68.10 (0.10)
72.51 (0.37)
76.38 (0.32)
RetinaNet on VOC0712
Random
38.86 (6.32)
61.75 (0.63)
70.64 (0.67)
76.14 (0.38)
Uncertainty
38.86 (6.32)
62.61 (1.21)
71.12 (0.96)
76.93 (0.40)
CoreSet
38.86 (6.32)
59.02 (0.70)
68.37 (0.86)
75.21 (0.45)
CDAL
38.86 (6.32)
61.68 (2.63)
71.32 (0.90)
76.69 (0.31)
LLAL
38.86 (6.32)
58.63 (0.64)
69.10 (0.46)
74.98 (0.23)
BADGE
38.86 (6.32)
58.65 (4.18)
68.07 (3.23)
75.55 (0.41)
EnmsDivproth
38.86 (6.32)
63.29 (0.83)
71.87 (0.62)
77.09 (0.11)
CALD
38.86 (6.32)
57.50 (0.20)
68.64 (0.50)
76.18 (0.45)
UWE (Ours)
38.86 (6.32)
64.62 (1.10)
72.59 (0.12)
77.60 (0.10)
Table 10: Benchmarking results for object detection. Best results are bolded, and the second best results
are underlined.
22

---

Published in Transactions on Machine Learning Research (06/2024)
Random
Uncertainty
CALD
EnmsDivproto
BADGE
UWE (Ours)
(a)
(b)
(c)
(d)
Figure 12: Visualization of detection results of RetinaNet on VOC0712. Blue boxes denote ground truth and orange boxes denote detection results. Model is trained with 2k images selected by various active learning methods. Our method is able to obtain detection results of higher quality, with fewer false positives (e.g., dog/cat in (b)) and false negatives (e.g., potted plant in (d)).
23


Reviews:
[{"review_id": "URNdc7Cxrx", "reviewer": "Reviewer_dMNC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Relevant topic (active learning), with the proposal of a complete method addressing it.\n- Results on three distinct and relevant computer vision problems (image classification, semantic segmentation, object detection), with comparison with many techniques.\n\n\nWeaknesses:\n- The paper is not always easy to follow and read, the overall writing quality is fairly low, and the ideas are not presented very clearly.\n- The working principles are relatively ad hoc, based on some observations on equations that are simplified with no clear justifications.\n- It is argued that one advantage of the approach is the computational speedup in training such a model. These gains are, however, relatively modest (linear) in my opinion and come from simplifying the gradients by heuristics that are not themselves well justified and evaluated. For classification for example, the simplification mostly comes from the fact that BADGE will multiply the output by the number of classes in the dataset. This justification appears weak to me, and the gains are factors of the number of classes, not in order of magnitudes.\n- The results are not easy to interpret and the advantage of the proposed approach over the others is not very clear. I am in fact doubtful of the gains of the proposed UWE over the BADGE approach, as UWE appears to be an approximation version of BADGE, using custom uncertainty measure vs real gradients, and making a sequential greedy selection of samples to label vs using k-means++.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper is proposing a method for active learning based on the embedding values weighted by the uncertainty associated with them. Using such a modeling, which is related to the gradient-based approach proposed in BADGE, an iterative selection method is used to select the samples to label in the active learning process. This selection is based on the selection of the sample the furthest away from the other samples in the uncertainty-weighted embedding, in order to maximize diversity. That makes a specific uncertainty-diversity tradeoff in a given active learning routine, with results reported for image classification, semantic segmentation, and object detection.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "First, I found the overall proposal not super clear nor well supported, explanations are relatively loose over the proposal and I am not convinced by the reasoning. For example, Sec. 3.2 presents the proposed UWE method, showing that we can use $\\frac{\\partial \\ell}{\\partial z}$ as an uncertainty measure. Then, it states “This motivates us to relax the constraint of computing the embedding as the derivative of loss function, and to generate an embedding by directly weighting the feature by uncertainty.” And then, the uncertainty function $u(x)$ is used as a replacement for $\\frac{\\partial \\ell}{\\partial z}$. So far, that makes sense, but it appears simply algebraic manipulations. What I got from that in the later experiment presentation is that from these manipulations, arbitrary uncertainty functions $u(x)$ are defined for each case tested, in replacement of the gradient $\\frac{\\partial \\ell}{\\partial z}$. I mean, why doing so, and what’s the issue in using the gradient, which is more exact? All this appears quite ad hoc and not very well supported, moreover given that we already have the analytical version available easily, as the gradients.\n\nAlso, the other part of the proposal is to make a greedy sequential selection of the instances to label from the current batch – see Algo. 1. This is a simple heuristic that may work reasonably well, but still may get stuck with a suboptimal set of instances given the myopic sequential selection done, where interactions between the instances selected is not taken into account globally, but rather after each iteration. Compared to this, the k-means++ algorithm used in BADGE appears to me better suited and less prone to provide suboptimal selection.\n\nIn Sec. 5.1, arguments on the computational advantages of UWE vs BADGE are made, which seems to be the main point supporting the approach. However, I am not sure about the claims made there, as everything seems to come from the fact that the uncertainty function approximates the gradient in some way, without being furthermore justified when they are presented (in section 3.4). So, basically, UWE is more efficient, computationally speaking, than BADGE as it relies on some heuristics that have been proposed with little justifications, whose heuristics are more efficient than computing the full gradients at hand. This is somehow expected, but the real question is whether the heuristics are **good** approximation of the real gradients. It might be so if we look at the results, but this is not studied in a direct and systematic way in the paper.\n\nComments on presentation:\n- The bibliographic references are not well formatted in the document, with a lot of repetition of author names, having those written in the text and repeated from the citation. That’s almost everywhere in the paper, I would have expected the authors to be much more careful on that, a proofreading of the paper before the submission would have allowed the authors to catch that. Moreover, depending on the context, references should put the names and date in parentheses (e.g., (Smith et al., 2023)) and not names inline in the text (e.g., Smith et al. (2023)).\n- Captions are not providing sufficient description of the figures and table to allow proper interpretation of those, in particular for figures 3 to 5, where panels (c) are not clear to me even after looking at the text.", "broader_impact_concerns": "There are no ethical implications in this paper that would require a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wTWcd37amU", "reviewer": "Reviewer_PkSp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The simplicity and effectiveness of uncertainty-weighted embedding (UWE) extend beyond image classification to encompass image-level and region-level active learning tasks such as semantic segmentation and object detection.\n\n2. In comparison to BADGE, uncertainty-weighted embedding offers advantages in terms of memory and computational efficiency.\n\nWeaknesses:\n\n1. The novelty of the paper appears limited. While the authors introduce uncertainty-weighted embedding (UWE), the overall methodology incorporates elements from existing approaches. \n\n2. The algorithm relies heavily on heuristics, lacking thorough justification for its components. A more comprehensive discussion, including theoretical support, could bolster the paper's strength.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores the active learning problem, focusing on the iterative selection of informative unlabeled data points for labeling. The current state-of-the-art (SOTA) algorithm, BADGE, employs gradient embedding to identify uncertain samples, primarily tailored for classification tasks relying on cross-entropy (CE) loss. However, this paper extends the concept of gradient embedding to accommodate various loss functions. To achieve this, the authors decompose the gradient vector into an uncertainty measure and the feature extractor, inspired by the original BADGE embedding, resulting in what is termed uncertainty-weighted embedding (UWE). Subsequently, leveraging UWEs, the authors propose a distance-based sampling approach.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Despite achieving state-of-the-art performance across various vision-related tasks with the introduction of UWE, the novelty of this paper remains somewhat limited. The authors should provide a detailed exploration of potential areas for increased novelty or the development of novel methodologies to enhance the paper's contribution.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "t0LgFlLgo7", "reviewer": "Reviewer_uw77", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Originality. UWE is very innovative. The author uses the chain rule to decompose gradient embedding into two terms: the first term depends on the form of the loss function and activation function; the second term is the feature representation extracted from the penultimate layer and is independent of the loss function. Therefore, the authors define a function u (x) to measure the uncertainty of the current model on sample x, and define a feature extractor f (x) by obtaining the intermediate output of the network, which is independent of loss functions. This decomposition method is novel and practical. \n2. Quality. The paper conducts comprehensive experiments. Firstly, the authors conduct a correlation experiment between uwe (x), ge (x), and uncertainty, demonstrating a strong positive correlation between uwe (x) and uncertainty. Secondly, the author conduct experiments on three common visual tasks, showing the superior performance of UWE compared to other methods. Finally, the authors conduct ablation experiments to demonstrate the effectiveness of both parts of the UWE, and quantitatively compare the computational complexity with BADGE.\n3. Clarity. The writing is clear. The authors start with the decomposition of gradient embedding and decompose it into a feature representation related to the loss function and activation function, as well as a feature representation unrelated to loss. Based on this decomposition, uncertainty-weighted embedding is proposed, which consists of two parts: u (x), which is positively correlated with uncertainty, and feature extractor f (x). The motivation of ideas is very intuitive, and the writing is clear.\n4. Significance. UWE has good significance for the Active Learning community. This paper proposes a method with fewer restrictions, more flexibility, and better performance compared to BADGE, which can be migrated to many other tasks.\n\nWeaknesses:\n1. It is difficult to distinguish these methods in (a) and (b) of Figures 3, 4, 5, and 6. Especially, the lines corresponding to the better-performing methods almost overlap. It would be better if there were other high-contrast display methods available.\n2. For image classification, it would be more convincing to add the experiment on ImageNet.\n3. The authors simply summarize the results of different tasks, but a deeper analysis is missing but required, such as why the proposed method works and why it works better than existing methods.\n4. Is it possible to give some visualizations, such as the detailed detection results, to better demonstrate the effectiveness of the proposed method?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper mainly focuses on Active Learning. The authors mainly address three issues in the BADGE method: 1) the High computational complexity of gradient embedding in dense prediction tasks, 2) restrictions on the loss function of prediction labels, and 3) the inability to handle the regression problem well. A Hybrid method is proposed in the paper, which considers both uncertainty and diversity in sample selection, known as UWE. UWE mainly consists of two steps: 1) computing a novel uncertainty-weighted embedding and 2) applying distance-based sampling for sample selection. UWE has achieved state-of-the-art in three main visual tasks: image classification, semantic segmentation, and object detection. UWE is simple, not limited by loss functions or tasks, and has superior performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weaknesses above.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an uncertainty-based strategy for active learning. The final recommendations from the three reviewers are at odds: two weak accepts and one weak reject. By inspecting all reviews and the revised paper, the AE thought that this submission can be accepted contingent on minor revisions. The merits of the method lie in the high efficiency, the extensibility to different tasks, the capability to handle object detection and semantic segmentation tasks with improved performances, and the insights into the decomposition of the gradient-based criterion. However, the paper shall undergo a minor revision, better addressing the comments from Reviewer #dMNC. In particular, the rationale behind the decomposition of gradient-based criterion, i.e. into an uncertainty measure and a feature extractor, shall be elaborated further; Some empirical evidence shall be provided that supports each part of the decomposition.
Decision: Accept (poster)