Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models
Authors: Ziting Wen
Oscar Pizarro
Stefan B. Williams
Venue: Accepted by TMLR ()

Abstract: Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs. However, this combination introduces significant computational costs, particularly with the growing scale of pre-trained models. Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs. Yet, this approach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings. This paper demonstrates that not all sample selection differences result in performance degradation. Furthermore, we show that suitable training methods can mitigate the decline of active learning performance caused by certain selection discrepancies. Building upon detailed analysis, we propose a novel method, aligned selection via proxy, which improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method. Extensive experiments validate that our method improves the total cost of efficient active learning while maintaining computational efficiency.

Published in Transactions on Machine Learning Research (11/2024)
Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models
Ziting Wen
muen4889@uniissddeey....
Australian Centre for Robotics, University of Sydney
Oscar Pizarro
oscur-pizarrookntruu.
Department of Marine Technology, Norwegian University of Science and Technology Australian Centre for Robotics, University of Sydney
Stefan Williams
stefan, httff//.williommmssssst eda.
Australian Centre for Robotics, University of Sydney
Reviewed OpenReview: pttps://openreviiie net/ forum? id= PNcg.MMJJdd
Abstract
Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs. However, this combination introduces significant computational costs, particularly with the growing scale of pro-trained models. Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs. Yet, this ap- proach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings. This paper demonstrates that not all sample selection dif- ferences result in performance degradation. Furthermore, we show that suitable training methods can mitigate the decline of active learning performance caused by certain selec- tion discrepancies. Building upon detailed analysis, propose novel method, aligned selection via proxy, which improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method. Extensive experiments vali- date that our mothod improves the total cost of efficient active lcarning while maintaining computational efficiency. The code available at htttp:: //gith////tubb/// com/ ZiTinng//asst asvp
1 Introduction
Training effective deep neural networks typically requires large-scale data (Deng et al., 2009 He et al. 2016 Liu et al. 2021). However, the high cost of acquiring data, especially annotated data, poses significant challenge for practitioners (Budd et al. 2021 Norouzzadeh et al. 2021). Active learning and the pre-training fine-tuning paradigm are widely adopted strratggiss to address this challenge. Active learning reduces the demand for extensive annotation by iteratively selecting and labeling the most informative samples (Ren) ct al. 2021 Additionally, the pre-training fine-tuning method leverages large-scale unsupervised pre-training to create & powerful foundational model, enabling exceptional performance on downstream tasks with only limited amount of labeled data (Chen et al., 2020a Grill et al. 2020 Caron et al. 2021: He al. 2022 To further minimize labels' demands, researchers have turned their altention to active line-tuning, which fine-tunes the pre- tre-training model with labeled samples selected by active learning strategies {Xie et al. 2023
Chan et al.] 2021 Bengar et al. 2021 However, as the scale of pre-training models continues to grow, the sample selection time of active learning, which often is overlooked in traditional active learning, becomes a challenge (Coleman et al. 2019). As illustrated in fig. larger pre-training models lead to higher accuracy, but also significantly increase the time required for active learning sample selection.
To address this issue, recent research has introduced an efficient active learning framework known as Selection via Proxy based pre-trained features (SVPp) (Zhang et. al. 2024). SVPp begins by forwarding the entire

---

Published in Transactions on Machine Learning Research (11/2024)
79-
Random Sampling, ResNet:50
200-
Standard AL
-67
Margin Sampling, ResNet-50
SVPp AL
78-
Random Sampling, ResNet-50 (2x)
Year cae setp
tearcue
ASVP AL (ours)
Sampling Time (his)
Accumcy (96)
77
Margin Sampling, ResNet-50
Training Cost (5)
150
-50
76-
100-
-33
75
74
73
0
0
0
50 100 150
200
250
8600
8700
8800
8900
9000
9100
Sample Selection Time (hrs)
Labeling Cost ($)
Figure 1: Comparing sample selection time and accuracy in active learning (margin sampling) with different scale models. The active learning included a total of 16 it- erations, selecting total of 400k samples the ImageNet datasct. All models are pre-trained using BYOL-EMAN
Comparison of the labeling and training costs across random sampling, the standard active learning pipeline, the efficient active learning method (SVPp), and our method (ASVP), using margin sampling to achieve the same accuracy as randomly selecting 400k samples on the ImageNet datasct. A ResNet-50 model was employed, with training costs calculated based AWS EC2 P3 instances (following Zhang et al., 2024), and annotation costs esti- mated using AWS Mechanical Turk' with triple reviews.
Figure 2:
(Cai el al..
2021
dataset through a pre-trained model once, recording pre-trained features for all samples. Subsequently, in multiple iterations of active learning, a simple MLP classifier is trained using the pre-computed features for sample selection. After the sample selection, the entire model is fine-tuned once. However, we observe that, in comparison to the standard active learning method (fine-tuning the whole pre-trained model in each active learning iteration), SVPp may compromise the effectiveness of active learning, resulting in additional annotation costs. As depicted in fig. 2 we noticed that while SVPp significantly reduces active learning time (training costs), its docline in active learning performance lcads to a notable increase in annotation costs. Practitioners therefore have to face the dilemmna of weighing computational efficiency against overall cost (the sum of labeling cost and the training cost).
In light of this challenge, in this paper, we analyze the factors contributing to the decline in active learning performance when using SVPp in sec. Beyond the intuition-aligned reason for SVPp active learning performance decline: fine-tuned features being more capable of distinguishing sample categories than pre- trained features, leading to the selection of redundant samples (i.e., samples the fine-tuned model can already predict correctly), our analysis reveals an intriguing discovery: when pre-trained features are of comparable quality to fine-tuned features, not all sample selection differences result. a decline in SVPp performance. Morcover, some performance drops are due to samples selected by SVPp causing substantial modifications to the pre-trained model during fine- fine-tuning.
Based on these insights, propose aligned selection via proxy strategy, ASVP, to enhance the per- formance of efficient active learning in sec. 5 First, we align the pro-computed features used by the proxy model with the one used in standard active learning. Specifically, we update the pre-computed features based on the fine-tur tune model when the pre-trained features' ability to distinguish sample categories significantly lags behind the finc-tuned features. In addition, we switch the training method between tincar-probing then fine-tuning (LP-FT) (Kumar et al. 2022) and fine-tune fine-tuning (FT) to mitigate the impact of samples selected by the proxy model on the model's performance.
Extensive experimental validation shows that our method improves the effectiveness of efficient active learn- ing, achieving similar or better total cost savings compared to those of standard active learning, while still maintaining computational efficiency.
2

---

Published in Transactions on Machine Learning Research (11/2024)
The contributions of this paper can be summarized as follows: (1) This paper empirically analyzes which sample selection differences contribute to the performance drop when employing SVPp. (2) We propose a novel efficient active learning approach, aligned selection via proxy (ASVP), based on the detailed analysis. It improves the performance of efficient active learning while incurring a marginal amount of computation time. (3) We introduce novel and practical evaluation metric for efficient active learning: the sample saving ratio. This metric directly quantífies the savings annotation achieved by employing active learning strategies compared with the random baseline. It also facilitates the estimation of overall savings, including savings in both computational and labeling costs, thereby assisting practitioners in making a decision whether to use efficient active learning strategies.
2 Related Work
Active learning is a technique that minimizes the number of labels needed for model training by selectively requesting annotations for samples. Numerous active learning strategies have been proposed, primarily in-
cluding uncertainty sampling (Lewis & Catlett 1994 Scheffer et al. 2001 Gal et al.] 2017 Kirsch et al.
2019 Woo 2022), diversity sampling (feature space coverage) (Sener & Savarese, 2018 Mahmood et al.
2022 their combinations (Yang et al.) 2017 Ash et al. 2020 Shui et al. 2020), and some learning-basee
approaches (inha et al., 2019 Yoo & Kweon 2019 Tran et al. 2019 With the advancement of large-scale
unsupervised pro-training and fine-tuning, an increasing number of researchers have turned their attention to active fine-tuning, which fine-tuning the pre-trained model with samples selected by active learning strate- gies. Given the success of the pre-training fine-tuning mode, most works of active fine-tuning have focused developing active lcarning strategics that can work with extremely limited annotations (Hacohen et al., 2022 Yehuda et al.) 2022 Wen et al., 2023 Furthermore, researchers have validated the performance of ex- isting active learning methods when applied to pre-training fine-tuning mode, remonstrating that traditional margin sampling exhibits strong performance (Zhang et al. 2024 Emam et al. 2021).
Efficient active learning. Most active learning methods operate in a multi-round mode, iteratively training models and selecting samples to label. This mode needs significant. training time and costs. In the context of active fine-tuning, this phenomenon becomes more pronounced, as larger models are better equipped to effectively leverage data in unsupervised pre-training (Chen el al. 2020b Oquab el al. 2023
Increasing the active learning batch size, which refers to the number of samples selected per active learning iteration, is one solution to enhance computational efficiency (Citovsky et al.) 2021 Zhang et al. 2022) However, it weakens the performance of active learning, and there remaais the considerable computal ional cost associated with training the entire network multiple times. Therefore, recent research has explored single-shot active lcarning based on pro-trained features (Xie al., (023). However, as the number of labels increases, the effectiveness of this method gradually diminishes in comparison to other standard active learning strategies.
Another solution to boost active learning efficiency is to use less computationally intensive models or training methods as proxy tasks for sample selection, known as Selection via Proxy (SVP) (Coleman et al. 2019).
Typical proxy tasks include reducing model complexity, such as using models like ResNet-8 or Res Net-Net12 14, early stopping, or training linear classifiers or MLPs based on pre-trained features (Zhang et al., 2024). However, these proxy tasks often have an impact on active learning performance and may result in savings in computation costs not outweighing the increase in annotation costs. As result, practitioners are often faced with tradc-off between computational efficiency and the overall cost.
3 Preliminary
3.1 Selection via Proxy based on Pre-computed Features
We briefly review the Selection via Proxy framework based on pre-computed features, SVPp, as shown in fig. 3 We denote the labclod set as L, the unlabeled set as U. The pro-trained neural network backbone is defined as ((::8p) : X + Rd, where 8 represents weights of the pre-trained model, X is the data space and Rd is feature space. And we define the predictor as h(;00)) IRd Re, where 0 is weights of the predictor
3

---

Published in Transactions on Machine Learning Research (11/2024)
and R is the output space. SVPp follows threc-step process. First, all samples are forward-passes through [(:0p) and the pre-trained features are saved. Subsequently, the proxy model is trained, and samples are selected iteratively based on the active learning strategy. In this process, the predictor h(=;0p serves as the proxy model, utilizing pre-computed features as its input. The final step fine-tuning the pro-trained model h(((;;;)) (x) based on the obtained labels.
(1) Record features
(2) Data selection
Classifier
Active learning sampling
Encoder
proxy
model training
Oracle labeling
Labeled Set (features)
33) Final model
Classifier
training
Random init.
Labeled
Encoder
Set (img.)
Pre-trained init.
Figure 3: The Selection via Proxy based on pre-trained features (SVPp) Framework. Stage 1: pre-computing features. Stage 2: sample selection based on the proxy model (a simple classifier) with pre-computed features as input. Stage Fine-tuning the pre-trained model using labeled samples.
3.2 LogME- PED
LogME-PED is efficient metric for selecting pre-trained models, which outputs score reflecting the performance of fine-tuned models. LogME uses evidence metric to assess the compatibility between a given pre-trained feature and the labels of the training samples (You et al. 2021), as shown in equation [ where F denoles the pre-trained feat ures of the labeled set and w denotes the parameters of the linear classifier. For specific calculation methods, refer to (You et al. 2021)
p(ulF) p(w))(pluuu F)dw
(1)
PED a physically inspired model, that simulates feature learning dynamics through multiple iterations (Li al. 2023). The approach draws an analogy between the optimization objective during fine-tuning and the concept of potential energy in physics. In fine-tuning, adjusting the model parameters forces the features of different classes to be distinguished, akin to the movement of objects influenced by forces in physics. Following this intuition, PED models the features of different classes separate small balls, where the mean and standard deviation of each class's features correspond to the position and radius of the ball, respectively. The force between balls is proportional to their overlapping length. In this physical model, the positions and forces the objects are updated iteratively until convergence. Finally, the updated features are regarded as the features after fine-tuning the pre-trained model. Then, LogME computed based on these updated features assess the performance of the fine-tuned model.
4 Observation and Analysis
Since practitioners utilizing proxy models for elficient active learning aim to reduce sample selection time while minimizing the loss of active learning gains (between standard active learning based on fine-tuned models and SVPp), the exact overlap between samples selected by the proxy model and those selected by the fine-tuned model is not their primary concern. Therefore, in this section, we first investigate which differences in sample selection lead to the degradation of SVPp active learning performance.
4

---

Published in Transactions on Machine Learning Research (11/2024)
4.1 Impact of Sample Selection Differences on SVPp Performance
We categorize the samples selected by both the proxy model and the fine-tuned model into different regions based on whether the models can correctly predict the samples and the active learning metrics (such margin (Scheffer et al. 2001), entropy (Lewis & Catlett 1994), etc.). As shown in fig. the x and y axes represent whether the proxy model and the fine-tuned model can correctly predict the samples, respectively, The closer a sample is to the origin, the more likely it is to be selected in active learning. For uncertainty- based active learning strategies (e.g., margin, entropy, confidence (Wang & Shang, 2014)) and strategies combining uncertainly and diversily (e.g., BADGE (Ash et al.) (020)), a smaller distance from the origin indicates higher uncertainty, such as a smaller margin predicted by the model. For distance- based active learning strategies (e.g., coresct (Sener & Savarcse 2018)), smaller distance from the origin indicates greater distance between the sample and the labeled set. Most elfective active learning strategies for fine- tuning pre-trained models are uncertainty-based or combine uncertainty with diversity (where the latter extends the former by selecting diverse samples from the uncertainty candidates) (Zhang et al. 2024 Emam ct al. 2021); this paper will use uncertainty-ban based active learning as an example for analysis.
Standard active learning based on the fine-tuned model selects samples from regions O, A, and B, while efficient active learning based on the proxy model selects samples from regions O, C, and D. In other words, compared to the fine-tuned model, the proxy model misses samples from regions A1, A2, B1, and B2 while additionally selecting samples from regions C and D.
Fine-tuned Model
Proxy Model
A1
A2
x
B1
B2
Model Predict correctly
x Model Predict incorrectly
Figure 4: Sample Selection Discrepancy: Proxy vs. Fine-tuned Models. For instance, in uncertainty-basee active learning strategies, the two axes represent the confidence of predictions for the proxy model and the fine-tuned model, with the positive half-axis indicating correct predictions and the negative half-axis indicating incorrect predictions. AL strategy selects samples with the lowest prediction confidence, meaning the proxy model chooses samples from regions O, C, and D, while the fine-tuned model selects samples from regions O, A, and B. Depending on whether the proxy model correctly predicts the samples, regions A and B are further divided into subregions A1, A2, B1, and B2. Both the proxy model and the fine-tuned model confidently predict the remaining white region, hence they do not select samples from it.
Empirical Evidence We conducted replacement experiments to demonstrate the impact of missing sam- ples from regions A1, A2, B1, and B2 on SVPp active learning performance. In this section, we used one of the state-of-the-art strategies, margin sampling, with fine-tuned pre-trained models. A total of 400k samples were selected over 16 rounds, with the first 10k samples selected randomly. The experiments were conducted on ImageNet, with detailed settings provided in sec. 6.1
Specifically, during each round of sample selection, we replaced the tail samples selected by the proxy model (i.e., those with the largest margins) with samples from regions A1, A2, B1, and B2. As shown in fig. we observed that not all differences in sample selection caused performance degradation SVPp active learning. Replacing samples with those from regions A2 and B2 improved SVPp active learning performance, replacing samples from region A1 had effect, while replacing samples from region B1 decreased performance. Based
5

---

Published in Transactions on Machine Learning Research (11/2024)
2.5
Active Learning Gain (96)
2.0
1.5
1.0
0.5
Standard (wlo replace)
Replace with A2
0.0
Proxy (w/o replace)
Replace with B1
Replace with
Replace with B2
50k 100k
150k Number Labels
200k 250k
300k
350k
400k
Figure 5: The impact of different regions on SVPp active learning performance. Replacing samples selected by the proxy model with those from regions A1, A2, B1, B2. The active learning gain refers to the difference in accuracy between active learning and the random baseline.
on these observations, to improve the performance of SVPp active learning, we should focus on the sample selection differences from regions A2 and B2.
To investigate why samples from regions A1 and B1 fail improve SVPp performance, we analyzed the difficulty of samples across different regions. Recent studies suggest that hard-to-learn samples are likely to undermine active learning performance (Karamcheti et al 2021). A1 and B1 regions, compared to A2 and B2, likely contain higher proportion of difficult samples. This is because the fine-tuned model's predictions on these samples show high uncertainty, while the proxy model's predictions on them are wrong, To validate this hypothesis, we utilized Dataset Maps (Swayamdipta et al. 2020) to assess sample difficulty, Dataset Maps depict sample learnability by analyzing training dynamics from models trained on the entire dataset, using metrics such as average confidence for the correct class and prediction accuracy across training epochs to quantify difficulty. Following Karamcheti et al. (2021), we categorized sample difficulty based mean confidence levels: very hard (<0.25), hard (>=0.25 and <0.5), medium (>=0.5 and <0.75), and easy (>=0.75). As illustrated in fig. 6 regions Al and B1 indeed contain higher proportion of very hard samples.
Very Hard
Hard
Medium
Easy
(a) A1
(b) A2
(c) B1
(d) R2
Figure 6: Difficulty distribution of samples from different regions.
4.2 Role of Region A2 and B2 in Improving SVPp Performance
Those missed samples (Region A2, 12) that improve SVPp performance share common characteristic: the proxy model can confidently predict them correctly, while the fine-tuned model shows high uncertainty in its predictions. This indicates that these samples are well-distinguished using pre-trained features, whereas they are harder to differentiate in the fine-tuned features. Including these samples in the next round of active learning will help the fino-tuned model distinguish them. question arises: are the pre-trained feature manifold and the fine-tuned feature manifold (training with samples from regions A2 and B2) similar?

---

Published in Transactions on Machine Learning Research (11/2024)
Empirical Evidence To answer this question, conducted backbone exchange experiment to compare the similarity between the fine-tuned and pre-trained features. We put the linear classifier of the fine-tuned model on top of the pre-trained backbone and evaluated its accuracy loss on the test set. The greater the accuracy drop after exchanging the backbone, the more dissimilar the fine-tuned features are from the pre-trained features.
We created five sets of training samples by replacing 0% (w/o replacing), 5%, 10%, 15%, and 20% of the samples selected by the proxy model with the samples from regions A2 and B2 that had the smallest margins. After training on these sets, the accuracy loss is evaluated by putting the fine-tuned model's linear classifter on top of the pre-trained backbone. As shown in fig. 7 the accuracy drop decreased as more samples from A2 and B2 were included in the train set, indicating that incorporating these samples helps maintain the feature manifold similar to the are-trained model.
2.5
Proxy (wio replace) Replace 20%
Replace 10%
-6
Replace 5%
Active Leariin Gain (9)
2.0
-8
Replace 15%
Exchangeetaciioon
1.5
1.0
0.5
-16
Standard
Proxy (FT)
-18
Standard (LP-FT)
Proxy ILP-FT)
50k
100k
150k 200k
250k 300k 350k 400k
100k
150k 200k 250k 300k Number of Labels
350k 400k
Number of Labels
Figure 7: The inffuence of samples from regions A2 and B2 on the similarity between fine-tuned and pre- trained leatures. Exchange-backkone accuracy loss refers the accuracy difference between the fine- tuned model's linear classifier placed the pre- trained backbone and the fine-tuned model.
Figure 8: Comparison of active learning performance based on the fine-tuned model (standard active learn- ing) and proxy model when employing different train- ing methods. The active learning gain refers to the difference in accuracy between active learning and the random baseline (using fine-tuning..
Based on this observation, another intriguing question arises: can directly maintaining the similarity between fine-tuned and pre-trained features mitigate the performance drop caused by the proxy model missing samples from regions A2 and B2?
Empirical Evidence To explore this, we employed two training methods: fine-tuning and LP-FT (Linear Probing and then Fine- Tine-Tuning,, to compare the performance differences between SVPp (sample selection via proxy model and evaluation via fine-tuning LP-FT) and standard active learning (both sample selection and evaluation via fine-tuning LP-FT)), In LP-FT, linear probing is followed by fine-tuning to preserve the similarity between the fine-tuned features and the pre-trained features.
As shown in fig. [ after applying LP-FT, the active learning performance based on the proxy model was almost identical to that based on the LP-FT model. This suggests that samples from regions A2 and B2 enhance active learning performance by preserving the similarity between the fine-tuned and pre-trained feature manifold. Therefore, directly preserving this similarity compensates for the decrease in active learning performance caused by the proxy model's omission of samples from regions A2 and B2.
4.3 Discussions
The previous section discussed that not all differences in sample selection between SVPp and standard active lcarning result in performance differences (only those from regions A2 and B2 do). Morcover, when using LP-FT to train the model, the sample selection differences from regions A2 and B2 do not cause decline in SVPp active learning performance. Next, we discuss the limitations of the above analysis.

---

Published in Transactions on Machine Learning Research (11/2024)
Firstly, given the training samples, we always aim to choose the training method that achieves the highest accuracy. Therefore, it is only meaningful to use LP-FT to compensate for the proxy model missing samples from regions A2 and B2 if LP-FT can achieve accuracy that is at least as good as fine-tuning, Secondly, as the number of selected samples increases, the performance gap between the fine-tuned model and the proxy model may widen (Cai et al.) 2021). This could reduce the active learning gain of SVPp, thus affecting the previous conclusion about the impact of missed regions (A1, A2, B2, B2) by the proxy model on active learning performance. Therefore, we further analyzed the proportion of samples from regions C and D, well as their impact active learning gain.
Reglon
Region
60
Active Leaming Gain
Proportion (86
50
40
-1
30
Standard (w/o replace) Proxy tw/o replace))
only C
only D
-2
50k
150k 200k 250k 300k 350k 400k Number Labels
50k
100k
150k Number of Labels
200k
250k 300k 350k 400k
(a) Proportion of Region C and D
(b) Active Learning Gain
Figure 9: Observations on region C and D: (a) the proportion of region C and D, (b) comparison of active learning gain when selecting only samples from region C or D. The active learning gain refers to the difference in accuracy between active learning and the random bascline.
As shown in fig. 9a with more samples selected during active learning, the proportion of samples from region C selected by the proxy model increases, while the proportion from region D decreases. This aligns with intuition: as the number of annotations increases, the features of the fine-tuned model become better at distinguishing between sample categorics than those of the proxy model, incrcasing correctly predicted samples (samples from region C). Additionally, as shown fig. 9h selecting samples from region results in an active learning gain of less than 0, meaning it performs worse than random selection. These two observations imply that as more samples are selected during active learning, the proxy model selects more samples from region C, causing its active learning performance to decline. As the proxy model's active learning performance declines, it may lead to all missed regions (A1, A2, B1, B2) contributing to performance differences between SVPp and standard active learning.
In summary, when LP-FT training achieves performance comparable to fine-tuning and the proportion of samples from region C is small, using LP-FT can prevent the decline in active learning gains caused by SvPp. The next section discusses how to adjust proxy model-based active learning to build an efficient active learning framework when these conditions not met.
5 Method
This section first addresses the issue of the increasing proportion of region samples, which makes L.P FT insufficient to prevent the decline in SVPp active learning performance. Region C refers to samples that the fine-tuned model can predict correctly with high confidence, while the proxy model struggles to differentiate them (high uncertainty). This indicates that the fine-tuned model's features are more suitable for distinguishing these samples. To address this, we recommend fine-tuning the pre-trained model as the number of region C samples increases. Updating the pre-computed features enhances the proxy model's discriminative capacity, thereby reducing the proportion of region C samples. Additionally, when updating pre-computed features, we switch the model training method from LP-FT to fine-tuning. Continuing with LP-FT when fine-tuned features outperform pro-trained features can lead suboptimal results by hindering necessary modifications to the pre-trained model.
8

---

Published in Transactions on Machine Learning Research (11/2024)
The ASVP framework is divided into three stages: pre-computation of features, data selection, and final model training. The data selection stage of our method is illustrated in lig. 10 During the data selection stage, the proxy model uses pre-computed features (pre-computed features refer to pre-trained features when feature updating is not triggered) as imput and performs multiple rounds of active learning (alternating between sample selection and proxy model training). At each iteration, we check if an update to the pre- computed features needed. If the annotation budget exhausted without triggering a feature update, LP-FT is used to train the final model. If an update is detected, fine-tuning is performed to generate new pre-computed features for the following rounds of sample selection. After the sample selection, the final model is trained using fine-tuning.
Classifier
No
proxy
model training
Active leaming sampling
Oracle labeling
Update feature
dataset
Labeled Set (features)
Yes
Classifier
Encoder
Encoder
Labeled Set (img.)
Figure 10: The Data Selection Pipeline in our approach, Aligned Selection via Proxy (ASVP). After acquiring new labels from the oracle, we assess the necessity of updating the pre-computed features. If required, the pre-trained model is fine-tuned, and the pre-computed features are updatod.
Next, explain when it becomes necessary to update the pro-computed features. As discussed in sec. the number of annotations increases, the quality of fine-tuned features improves gradually, resulting in a higher proportion of samples selected by the proxy model from the redundant region. Also, including the samples from the redundant region rarely improves the performance of the fine-tuned model. Hence, we assess the ratio redundant region samples by evaluating the changes the performance of the finc-tuned model after adding more annolated samples. To assess the changes the fine-tuned model's performance, we employ LogME-PED (You et al. 2021 Li al.) (023)) an indicator.
Specifically, in every active learning iteration, achieved within a single iteration, we infer that the fine-tuned features will maintain similarity to the pre- trained features. So, there is no need to calculate the LogME metric and update the pre-computed features. However, if the PED component converges with more than one iteration, we proceed to compute the LogME metric. When the difference between the current LogME score and that from the previous active learning iteration below a predefined threshold, we deduce that the fixed pre-trained features pose a significant obstacle to the performance improvement of the proxy model, prompting us fine-tune the model and update the pre-computed features.
first compute the PED component. If convergence is
6 Results
Our method is validated four datasets: ImageNet-1k (Deng al. 2009). CIFAR-10 (Krizhevsky et al, 2009), CIFAR-100 (Krizhevsky et al. 2009) and Oxford-IlIT Pet dataset Parkhi et al. 2012) with five typical active learning strategics. The experiment setup is clarified in sec. 6.1 We propose the new efficient active learning evaluation metric in sec. 6.2 The results are shown in sec. 6.3 and sec. 6.4 The ablation study is shown in sec. 66.] The detailed study the impact of the position updating features is shown in soc. [6.6] Additionally, given the efficiency of our framework, decreasing the amount of sample selected each active learning iteration alleviate the redundant problem of batch active learning strategy is feasible. The results shown in appendix

---

Published in Transactions on Machine Learning Research (11/2024)
6.1 Experiment Setup
Active learning strategies. To evaluate the compatibility of our proposed ASVP with existing active learning strategies, four typical active learning strategies are included: (1) Margin: uncertainty-basel sam-
pling, one of the SOTA in the context of fine-tuning pre-trained models (Schelfer et al. 2001], (2) Confidence:
imcertainty-basee sampling (Wang& & Shang, 2014), (3) Coreset: diversity-based sampling (Sener & Savarese,
2018), (4) BADGE: combination of uncertainty and diversity (Ash et al. 2020). (5) ActiveFT[all), feature
space coverage with fcatures from fine-tuned model (Xie al. 2023).
Baseline. (1) Standard active learning: training the whole model with fine-tuning (FT) linear-probing then fine-tuning (LP-FTT) (Kumar et al. 2022) in each active iteration, (2) SVPp (Zhang et al., 2024): selecting samples via MLP proxy model based on pre-trained features and training the final model with FT, (3) (prr (Xie et al. 2023]: a single-shot active learning strategy that selects all sample in single iteration based on pre-trained features.
Implementations. We utilized the ResNet-50 model (He et al.) 2016 pre-trained on ImageNet using the BYOL-EMAN (Cai et al., 2021) method in all our experiments. The training model training approaches, fine-tune (FT) and LP-FT, are provided in appendix [A] For the baseline method (SVPp) and our method (ASVP), we employed 2-layer MLPs as proxy models, where its architecture is Linear + BatchNorm + ReLU + Linear. The hidden layer width matched the input feature dimensions. We performed multiple active learning iterations until the model performance approached that of training on the entire dataset (upper bound). Specifically, for the ImageNet, CIFAR-10, CIFAR-100, and Pets datasets, we conducted 16, 18, 22, and 10 active learning iterations, respectively. The query step was set to 200 samples per iteration for the Pets dataset. For ImageNet, CIFAR-10, and CIFAR-100 we used varying query steps: for ImageNet, we queried 10,000 samples per itcration until reaching 100,000 samples, after which we queried 50,000 samples per iteralion. For CIFAR-10, queried 200 samples per iteration until reaching 2,000 samples, then increased to 500 samples per iteration. For CIFAR-100, we queried 400 samples per iteration until reaching 6,000 labeled samples, then switched to 2,000 samples per iteration. The difference in LogME-PED scores between consecutive active learning iterations set as for the threshold of updating pre-computed features.
of the two
6.2 Metric
Efficient active learning methods often exhibit a lower accuracy compared to standard active learning ap- proaches. Understanding the additional annotation costs needed to compensate for the accuracy gap caused by efficient active learning allows practitioners to weigh the computational cost savings against increased labeling expenses, aiding their decision to utilize these methods.
To this end, we propose a new evaluation metric: equivalent saving amount. Let N, represent the number of samples selected by the active learning strategy, and let A be the accuracy achieved by model trained with these N₁ labeled samples. We utilize interpolation to estimate the number of samples, N2, required to achieve accuracy A for random baseline. We refer to N2 the equivalent number of non-AL labels in the subsequent discussions. The number of saved samples is N₂ - Nı and sample saving ratio is Na-N, Subsequently, we compute the average sample saving ratio across all iterations of active learning to assess the performance of the active learning strategy. Furthermore, the overall cost of active learning, including annolation and training costs, equals N₁ x Pt + Ciy, where Pt denotes the unit price of annotation and Car represents the training cost of active learning.
N₂
6.3 Sample Saving Ratio and Overall Cost
The average sample savings ratios are presented in table 1 with each experimental setting repeated three times to report both the average and standard deviation. The table also illustrates the total cost required for active learning to reach the accuracy achieved by the random baseline at the last active learning iteration (i.c., the accuracy achieved when randomly selecting 400k, 6000, 20000, and 2000 samples from ImageNet, CIFAR-10, CIFAR-100, and Pets datasets, respectively). The training cost is assessed based on AWS EC2 P3 instances (Zhang et al. 2024), while the annotation cost is evaluated using AWS Mechanical Turk,
10

---

Published in Transactions on Machine Learning Research (11/2024)
with three annotations per sample to ensure labeling quality. Additionally, Figures and 12 show the accuracy and corresponding label savings achieved with margin sampling, while results using other active learming strategies are provided in Appendix D Figure 13 shows the sampling time and total cost required by standard active learning, SVPp, ASVP (ours), and the single-shot active learning strategy, ActiveFT (pre), to achieve the accuracy attained by the random baseline at the final active learning ileration.
Table I: Average sample savings ratios, with each configuratio repeated three times to calculate the average and standard deviation. The training method indicates how the final model is trained, and the selection method indicates the model used for sample selection. The cost refers to the total cost required for active learning achieve the accuracy achieved by the random baseline in the last active learning iteration. The cost-saving ratio refers to the average ratio of the cost of active learning to the cost. of the random baseline. The best results are shown in red and the second-best results are in blue.
ImageNet
CIFAR-10
CIFAR-100
Pets
Cost
Selection Method
Training Method
Avg. Sample Saving Racio
Cost
Avg. Sample
Cost (8)
Avg. Sample Saving Ratio
Cost
Avg. Sample Saving Ratio
Cost
Saving Ratio
Strategy
[8)
Saving Ratio
($)
Standard
FT
Random
0%
14400
0%
216
0%
720
0%
72
100%
Margin
30.30:11000
8732
00.23:0.51
131
14.5142.
16
578
51.6722.
49
68%
BADGE
21 11.3310.33 38
10018
36.18:10.44
128
13.73.12.07
580
33.0113..
47
69%
Standanl
FT
Coufidence
-3.42.0.77
9619
24.99.11.37
1.37
142
4.23.12.34
567
27.24.13.22
50
70%
Coreset
12.44422.22
192
-13.004-2.81
-4005+22.31
(ctiveFT(al))
12.39:+3.26
186
3.03++.117
798
-5.20+-2.22
97
111%
Margin
32.2210.27
9884
37.39.1 1.228
139
12.31118.3
509
6.97.12.55
73
76%
BADGE
24.1240.54
11901
29.54-40.70
1.36
19.81 +-1128
528
6.744-3.84
68
78%
Standard
LP-FT
Confidence 0.11+1.51
10125
20.10-4-.00
138
13.494+2.00
510
9.58-+33441
73
77%
Coreset
11.00++:1.09
241
-16.76+1.78
769
-36. 933--.5333333
110
124%
(ct)veFT(al)
-2.13:20.11 11
249
6,38+1.35
753
-29,33311..58
122%
Siuglesshot
FT
ActiveFT(pze)
1.40++1.50
235
-0.21+1.41
822
-11 11..22+11220 1.26
118
129%
LP-FT
5.02+:4.02
238
8. 8.92+22.44
742
338.10+2200
235
180%
Margin
25.16:11117
9091
19 86:22.553 53
184
3.51
65-1
23. 184-1.09
50
77%
BADGE
16.17118.4
11374
19.35:13.66
156
0.51100.4
657
-7.78::2
85
90%
SVPp
Coufidence
-8.6612.88
9678
1.53.1882
77.93.1.85
2.40.17.25
66
Coresct
54.78 8.888
1355
56.53.13.12
903
-38.63+-1 11.64
87
291% %
(ativeFTTal))
3.79++4.31
192
-0.50++1.42 142
713
-11 63-+6-+-6600 85
319
210%
Margin
34.38.1.577
8690
$2.451:113
127
12.821140
580
21 11.9811..50 50
47
67%
BADGE
25.57
10305
12.87.0.84
137
10.1111.13
577
6.10.15.59
55
73%
FT/LP-FT
Confidence
5.8840.66
9479
25.67422.88
167
2.75+1117
14.81+3.177
50
Coreset
11.1
192
16.11611.76
767
91
107%
ActiveFT(sl)
26.67 26.67
135
6,81+1177
775
-7. 7..4111881
47
82%
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
Upper Bound (Full dataset)
70
Matter
Rambe of
Ramber
(a) ImageNet
(b) CIFAR-10
(c) CIFAR-100
(d) Pets
Figure 11: The accuracy of the standard method, SVPp, and our method ASVP on (a) ImaagNee, (b)
CIFAR-10, (C) CIFAR-100 and (d) Pets using margin sampling.
Our method consistently outperforms the efficient active learning method, SVPp, in terms of sample saving ratio and overall cost reduction. It surpasses the single-shot active learning strategy, ActiveFT(pree) when combined with most active lcarning strategics. In most cases, our approach offers similar or greater overall cost savings compared to standard active learning. This alleviates practitioners' concerns about overall costs when using efficient active learning. Additionally, in standard active learning methods, the lack of a clear criterion for choosing the proper training method may lead suboptimal results. Specifically, the standard active learning with FT outperforms the standard active learning with LP-FT on CIFAR-10 and Pets, while on ImageNet and CIFAR-100, the standard (LP--FT) method demonstrates better performance.
11

---

Published in Transactions on Machine Learning Research (11/2024)
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
1300
150
5000
Sunber
(a) ImageNet
(b) CIFAR-10
(c) CIFAR-100
(d) Pets
Figure 12: The equivalent number of non-active learning label amounts comparison of the standard method,
SVPp, and our method ASVP on (a) ImageNet, (b) CIFAR-10, (C) CIFAR-100 and (d) Pets using margin sampling.
Random
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
ActiveFT (pre)
120
1480
209
13000
1000
Simpl Selecin Time
3332
180
(a) ImageNet
(b) CIFAR-10
(c) CIFAR-100
(d) Pets
Figure 13: Computation efficiency and overall cost comparison the standard active learning method, SVPp, and our method ASVP on (a) ImageNet, (b) CIFAR-10, (C) CIFAR-100 and (d) Pets using margin sampling.
In contrast, our approach, by switching training methods, ensures relatively favorable performance in practical applications.
Compared to the single-shot active learning method (ptiveFT(pre). our approach is compatible with various existing active learning strategies and exhibits lower dependence on the quality of pre-trained features. The reliance of ActiveFT(pre) on the quality of pre-trained features leads to suboptimal results. This is particularly evident in datasets like Pets, where the pre-trained feature quality is low.
6.4 Computational Time
We define the average training time for the fine-tuned and the proxy models in each active learning ileration as full and Thrpprozy, respectively. The time for processing all unlabeled samples through the whole pre- trained model and the proxy model to compute active learning metrics and select samples is denoted as Ty ful and Trrrozy respectively. The time spent forwarding all samples through the model to record features is represented by Trre- The number of active learning iteration is given by Nat- The total active learning time for standard active learning, Full is Nat x (Trnful + fatt.. The total time for SVPp, Ts is For ASVP, the initial pre-computed features are obtained from a pre-trained
Nat x (Ter.prozy proxy) model. When updating these pre-computed features is necessary, the model is fine-tuned on labeled data, and the pre-computed features are refreshed accordingly. Let Npre denote the number of times the ASVP features are pre-computed, then, the time required for pre-computing features is Npre xpre (Npre- 1) x total time required by ASVP, denoted as Ts.aspp is Nai (Ttr.poozy + ( +N xTpre + ppre 12) fr,
firffull The
The computation time of standard active learning, SVPp, and ASVP on ImageNet is presented in table 2 The results on other datasets are shown in appendix B The time for ASVP mainly arises from updating pre-computed features, as the computation time for the proxy model, an MLP, significantly lower than that of the finc-tuned model. This means that increasing the number of active learning itcrations N., has a marginal impact on ASVP's total time. Therefore, this opens up the potential improve the diversity of selected samples by reducing the number of samples chosen in each active learning iteration.
12

---

Published in Transactions on Machine Learning Research (11/2024)
Table 2: Computation time comparison of various active learning sample selection methods on ImageNet using V100 GPU.
T Ppre(hrg Nat x Ty(rr))
Nat (hr(hrs)
T,(rr))
Standard SVPp
16.36
80.97
97.33
0.92
0.10
1.48 0.46
ASVP
8.60
0.10
0.46
9.16
6.5 Ablation Study
Our ablation study was conducted on CIFAR-10, CIFAR-100, and Pets, using 10, 15, and 10 active learning iterations, respectively, with each iteration querying 200, 400, and 200 samples. All other settings follow those described in sec. 66.1 The results with margin sampling are shown in Table while additional ablation provided in Appendix C Across all three datasets, updating
studies with other active learning strategies pre-computed features consistently led to a noticeable performance boost. Meanwhile, switching training methods improved performance on all datasets except for the Pets dataset, suggesting that our method may not preciscly determine the optimal point for switching training methods. However, considering the practical scenario where we cannot. pre-determine whether using FT or LP-FFT for training is better, our approach still aids in selecting preferable training method.
Table 3: Ablation Study, Comparison of the average sample saving ratio of only updating pre-computed features (training the final model using fine-tuning), solely switching training methods (using pre-trained features), and the complete ASVP approach (both updating pre-computed features and switching training methods) under different datascts with margin sampling.
Updale Pre-computed Features
Switch Training Methods
CIFAR-10
CIFAR-100
Pets
No
No
23.63+4.01
1.36+1.45
23.48++4.09
No
Yes
24.96112.7
7.6911.56
18.45:3.89
Yes
No
31.4212.65
6.9812.03
27.01:1.551
Yes
Yes
35.03+1.899
13.31++1886
21.98++0.50
6.6 Influence from Position of Updating Features
We examined the impact of the position of updating pre-computed features CIFAR-10. The average savings ratio is presented in table 4 The choice of updating pre-computed features does indeed affect the final performance of ASVP. Nevertheless, across the six different positions explored in the experiments, ASVP consistently outperforms the method relying solely on pre-training features (SVPp). Furthermore, our proposed method for determining the update location based on LogME--PED proves to be effective. On CIFAR-10, this method selects the update of pre-computed features at 600 labels, ranking as the second-best among the six experimental positions, as shown table The positions of updating pre-computed features estimated through LogMF-PPD are outlined in appendix
Table 4: The influence of the position of updating pre-computed feature on average sample savings ratios on CIFAR-10. The best results are shown in red and the second-best results are in blue.
Change at
Avg. Sample Saving Ratio
Change at Avg. Sample Saving Ratio
w/o
23.03
200
33.1312.32
800
36.03+1.44
400
32.13+4.60
1000
32.75 1.991 1.91
600
35.03+1.80
1200
31.33+1.87
13

---

Published in Transactions on Machine Learning Research (11/2024)
7 Conclusion
Active learning on pre-trained models offers label efficiency but requires significant computational time, whereas current elficient active learning methods lend to compromise a notable fraction of active learning performance, resulting in increased overall costs. To address this challenge, this paper attributes the reduced performance of SVPp to (1) the absence of samples necessary to preserve finc-tuned features similar to the pre-trained features and (2) an increased selection of redundant samples due to fine-tuned features becoming better than pre-trained features as the annotation increases. Building upon this analysis, we propose simple and effective method, ASVP, improve the effectiveness (in terms of accuracy) of efficient active learning with comparable marginally increased computational time. Additionally, we introduce the sample savings ratio as metric to assess the effectiveness of efficient active learning, providing a straight measure for labeling cost savings. Experiments show that our proposed ASVP saves similar or greater total costs in most cases while maintaining computational efficiency.
8 Limitations and Future Directions
Due to limited computational resources, this paper evaluates efficient active learning methods' labeling and training costs using ResNet-50 model with specific pre-training method, BYOL-EMAN (Grill et al. 2020;
Cai et al.
2021 Although ResNet-50 serves as practical choice for our experiments and provides a reliable
baseline. larger models achieve higher accuracy and are often more helpful in practical applications (Zhai
ct al. 2022 Existing research shows that larger models exhibit better label efficiency, requiring fewer labeled samples to reach the same accuracy as smaller models (Chen et al. 2020b). This implics that in active learning, using larger models may reduce labeling costs while increasing training costs. Therefore, the total cost savings achieved by both standard and efficient active learning methods are closely tied model size. Evaluating the impact of model size total cost savings, and exploring efficient active learning methods that can effectively reduce sample selection time and total cost (labeling and training costs) in the context of larger models, are important directions for future rescarch.
Furthermore, when transferring to downstream datasets (used for active learning), diffcrent pre-trained models exhibit varying performance gaps between fine-tuning and linear probing. A smaller gap suggests higher-quality pre-trained features, making casier for proxy models based on these features to select samples similar to those chosen by fine-tuned models. This means that proxy model-based active learning may compromise less on performance compared to standard active learning. Therefore, exploring how to select suitable pre-trained models is another interesting direction for future research.
References
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alckh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations, 2020.
Javad Zolfaghari Bengar, Joost van de Weijer, Bartlomiej Twardowski, and Bogdan Raducanu. Reduc- ing label effort: Self-supervised meets active learning. In Proceedings of the IEEE/CVF International Conference Compuler Vision, pp. 1631-1633 2021.
Samuel Budd, Emma C Robinson, and Bernhard Kainz. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis, 71:102062, 2021.
Zhaowei Cai, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Zhuowen Tu, and Stefano Soatto. Exponential moving average normalization for supevvipe and semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 194 203, 2021.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650 (6600096600000 2021.
14

---

Published in Transactions on Machine Learning Research (11/2024)
Yao- Chan, Mingchen Li, and Samet Oymak. On the marginal benefit of active learning: Does self- supervision eat its cake? Tn ICASSP 2021-2021 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3455 3459. LEEE, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self- supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243 22255, 2020b.
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, and Sanjiv Kumar. Batch active learning at scale. Advances in Neural Information Processing Systems, 04:11993-11144, 2021.
Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. Tn International Conference on Learning Representations, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pallern recognition, pp. 248 255. Ieee, 2009.
Zeyad Ali Sami Emam, Mong-Min Chu, Ping- Yeh Chiang, Wojciech Czaja, Richard Leapman, Micah Gold- blum, and Tom Goldstein. Active learning at the imagenet scale. arXiv preprint arXiv:2111.128 11.12800, 2021,
Yarin Gal, Riashat, Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. Tn International Conference on Machine Learning, pp. 1183-1192. PMLR, 2017.
Jcan-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to sulf-super learning. Advances in Neural Information Processing Systems, 03:21271-21284 2020.
Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets. In International Conference on Machine Learning, pp. 8175-8195. PMLR, 2022.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual lcarning for image recognition. In Proceedings of the IEEE conference on compuler vision and pattern recognition, pp. 770 778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVT conference on compuler vision and pallern recognition, pp.
2022.
Siddharth Karamcheti, Ranjay Krishna, Li Fei-Fei, and Christopher D Manning. Mind your outliers! inves- tigating the negative impact of outlicrs on active learning for visual question answering. In Proceedings of the 59th Anreual Meeting of the Association for Compulational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp- 7265-7281, 2021.
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in neural information processing systems, 32, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto, 2009.
Ananya Kumar, Aditi Raghunathan, Robbic Jones, Tengyu Ma, and Percy Liang. Finc-tuning can dis- tort pretrained features and underperform out-of-distribution. In International Conference on Learning Representations, 2022.
15

---

Published in Transactions on Machine Learning Research (11/2024)
David D Lewis and Jason Catlett. Heterogencous uncertainty sampling for supervised learning. In Machine learning proceedings 1994, pp. 1181556. Flsevier, 1994.
Xiaotong Li, Zixuan Hu, Yixiao Ge, Ying Shan, and Ling- Yu Duan. Exploring model transferability through the lens of potential energy. In Proceadings of the IEEE/CVE International Conference on Compuler Vision, pp. 5429-5438, 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012-10022, 2021.
Rafid Mahmood, Sanja Fidler, and Mare T Law. Low-budget active learning via wasserstein distance: An integer programming approach. In International Conference on Learning Representations, 2022.
Mohammad Sadegh Norouzzadch, Dan Morris, Sara Becry, Neel Joshi, Nebojsa Jojic, and Jeff Clunc. A deep active learning system for species identification and counting in camera. trap images. Melhods in ecology and evolution, 12(1):150-161, 2021.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin Fl-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07199, 2023.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3498-3508 IEEE, 2012.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po- Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1-40,, 2021.
Tobias Scheffer, Christian Decomain, and Stefan Wrobel. Active hidden markov models for information extraction. In International symposium intelligent data analysis, pp. 309 318. Springer, 2001.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018.
Changjian Shui, Fan Zhou, Christian Gagné, and Boyu Wang. Deep active learning: Unified and principled method for query and training. In International Conference Artificial Intelligence and Statistics, pp. 1308-1318. PMLR, 2020.
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5972-550 5981, 2019.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannanch Hajishirzi, Noah Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamies. Tn Proceedings of the 2020 Conference Empérical Methods in Natural Language Processing (EMNLP), pp. 9275-9293, 2020.
Toan Tran, Thanh-Toae Do, Ian Reid, and Gustavo Carneiro. Bayesian generative active deep learning. In International Conference on Machine Learning, pp. 6295 6304. PMLR, 2019.
Dan Wang and Yi Shang. A new active labeling method for deep learning. In 2014 International joint conference neural networks (IJCNN), pp. 112-119. IEEE, 2014.
Ziting Wen, Oscar Pizarro, and Stefan Williams. Ntkepl: Active learning on top of self-supervised model by estimating true coverage. arXio preprint Xrx:2:3306. 04099, 2023.
Jac Oh Woo. Active learning bayesian neural networks with balanced entropy learning principle. In The Eleventh International Conference on Learning Representations, 2022.
Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, and Wei Zhan. Active finetuning: Exploiting annotation budget the pretraining-fiette paradigm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23715-23724, 2023.
16

---

Published in Transactions on Machine Learning Research (11/2024)
Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. Suggestive annotation: A deep active learning framework for biomedical image segmentation. In International conference medical image computing and computer-
comptter-assisted assisted intervention, pp. 399 407. Springer, 2017.
Ofer Yehuda, Avihu Dekel, Guy Hacohen, and Daphna Weinshall. Active learning through covering lens. In Advances in Neural Information Processing Systems, 2022.
Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 93 102, 2019.
Kaichao You, Yong Liu, Jianmin Wang, and Mingsheng Long. Logme: Practical assessment of pre-trained models for transfer learning. In International Conference on Machine Learning, pp. 12133 12143. PMLR, 2021.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 12104-12113, 2022.
Jifan Zhang, Yifang Chen, Gregory Canal, Arnay Mohanty Das, Gantavya Bhatt, Stephen Mussmann, Yinglun Zhu, Jeff Bilmes, Simon Shaolci Du, Kevin Jamicson, et al. Labelbench: A comprehensive framework for benchmarking adaptive label-efficient learning. Journal of Dala-centric Machine Learning Research, 2024.
Renyu Zhang, Aly A Khan, Robert I. Grossman, and Yuxin Chen. Scalable batch-mode deep bayesian active learning via equivalence class annealing. In The Eleventh International Conference on Learning Representations, 2022.
A Hyperparameters
Finc-tuning, linear probing then fino-tuning (LP-FT), and proxy model training all utilized the SGD with momentum optimizer, with a momentun value of 0.9. For fine-tuning and LP-FT, the batch size in the training stage is 128 and the batch size in the sample selection stage of the active learning is 512.
The hyper-parameters of fine-tuning are shown in table 5 For ImageNet, we adopted the finc-tuning pa- rameters following the prior work (Cai et al. 2021 Setting the learning rales separately for the classifier head and the backbone. For CIFAR-10, CIFAR- 100, and Pets, we followed prior work (Cai et al., 2021) set the learning rate of the classifier as 0.1 and search other paper-pprameters. The syper-parameter are searched from (0.01, 0.05, 0.1 0.3) and the weight
detailed in table The learning rate of the backbone decay from (0, 0.0001). Additionally, to maintain consistency in training iterations between fine-tuning and LP-FT, we set the number of fine-tuning epochs as the sum of linear probing and fine-tuning epochs in LP-FT training.
Table 5: The hyper hyper-paramenn parameters of the fine-tuning.
Dataset
Learning Rate
Training Epochs
Weight
Classifier
Backbone
Decay
ImageNet
0.1
0.01
50
0.0001
CIFAR-10
0.1
0.1
130
0
CIFAR-100
0.1
0.05
150
Pets
0.1
0.1
140
For the LP-FT hyper-parameters, we conducted a search within the following ranges: (0.01, 0.1) for the classiffer's learning rate, (0.01, 0.05, 0.1, 0.3) for the backbone's learning rate, (0.05,0.1.0.5) for learning rate in lincar-probing (L.) stage, and (0, 0.0001) for weight decay. Fine-tuning (FT) training epochs were set at 120. For ImageNet, linear probing (LP) training epochs were searched from (5, 10), while for other datasets, LP training epochs were explored from (5, 10, 20, 30). Given the numerous hyper-parameters in I.-FFT, we
17

---

Published in Transactions on Machine Learning Research (11/2024)
sequentially search the backbone learning rate, classifier learning rate, weight decay, and LP training epochs. where the weight decay is 0 and the learning rate in the
The final hyperr-arameters are shown in table 0
LP stage is 0.5 across all datasets.
Table 6: The pyper-paramctors of LP-FT.
Dataset
Learning Rate
Training Epochs
Classifier
Backbone
LP
FT
ImageNNtt
0.01
0.01
5
45
CIFAR-10
0.1
0.01
10
120
CTFAR-100
0.1
0.01
30
120
Pets
0.1
0.1
20
120
The proxy model was trained 50 epochs with learning rate of 0.01 and the weight decay of 0. During the training stage, the batch size is 2048 and in the active learning stage, the batch size is 16384.
B Computation Time
The time for standard active learning, SVPp and ASVP V100 GPU is presented in table 7 The active learning selup, outlined in sec. 6.1 performs total of 10 active learning iterations CIFAR-10 and Pets, where 200 samples are selected for each iteration. Regarding CIFAR-100, it spans 15 iterations, selecting 400 samples per itcration.
Table 7: Sample selection time of standard active learning methods, SVPp, and our proposed method ASVP on CIFAR-10, CIFAR-100 and Pets. Experiments conducted on V100 GPU,
CIFAR-10 (hrs)
CIFAR-100 (hrs)
Pets (hrs)
Standard 6.94
20.96
1.50
SVPp ASVP
0.22
0.13
0.03
0.85
1.10
0.05
C
Ablation Study
The results of ablation experiments with various active learning strategics are shown in table g table g (a TQ Specifically, feature alignment refers to whether the ASVP method updates pre-computed features using fine-tuned model features, while training alignment indicates the selection of the final model train- ing method (FT LP-FT) based LogME-PED scores. In all ablation experiments, feature alignment consistently demonstrates notable improvements. Training alignment, except for the Pets dataset, exhibits enhancements.
Table 8: Ablation Study: Comparing the average sample saving ratio using the active learning strategy BADGE.
Feature alignment
Training aligument
CIFAR-10
CIFAR-100
Pets
No
No
14.9645.88
-2.1140.85
-7.7842.42
No
Yes
18.3
3.81+00.4
-15.61+1.93
Yes
No
30.34412.06
2.25+12.25
13.93+5.21
Yes
Yes 33.67+0.88
8.16+1.98
6.145.59
Postion of Updating Pre-computed Features
The ASVP method uses the difference in LogME-PED scores between consecutive active learning iterations to decide if updating pre-computed features is required. The absolute differences in LogME-PPDD scores
18

---

Published in Transactions on Machine Learning Research (11/2024)
Table 9: Ablation Study: Comparing the average sample saving ratio using the active learning strategy confidence.
Feature alignment
Training alignment
CIFAR-10
CIFAR-100
Pets
No
No
-11.54+2.3
-14.43#2.49
2.447.25
No
Yes
-9.61=1.43
-7.86+22.16
-0.83:17.34
Yes
No
21.9943.32
-7.19=1.69
18.05-22.2 14.81+3.11
Yes
Yes
23.92+5.16
-0.62+1.43
Table 10: Ablation Study: Comparing the average sample saving ratio using the active learning strategy ActiveFT(al).
Feature alignment
Training aligmment
CIFAR-10
CIFAR-100
Pets
No
No
-.72:#5.78
0.25++0922
-11.63
No
Yes
-1.89=6.11
5.66:t0.97
-14.76=66.74
Yes
No
24.62+1.31
5.86++2.24
-4.61+4.97
Yes
Yes
27.45:11.42
11.28:2.01
-7.7444.81
are illustrated in fig. 14a detailed in table
fig.
14b
fig.
14c
When the threshold defined 1, the updating positions are
Table 11: Positions of updating pre-computed features. The number the PED convergence iterations is
used determine updating pre-computed features required. The positions are estimated by the absoluto difference LogME-PED scores between consecutive active learning iterations, where difference smaller than 1 indicates update.
Updating Features
Position
ImageNet CIFAR-10
Yes
150k
Yes
600 and 2000
CIFAR-100
Yes
2000 and 6000
Pets
Yes
800
3.0
Booghe PED diterrre
2.5
2.0
1.0
0.
0.0
550
1050
1250
1752
2050
1070
2030
3000
4000
1002
1007
505
1003
1250
1750
3002
Ammunt
Lucel
Amrrunt
(a) CIFAR-10
(b) CIFAR-100
(c) Pets
Figure 14: The absolute differences in LogME-PED scores between consecutive active learning iterations on (a) CIFAR-10, (b) CIFAR-100 and (c) Pets.
E Results of other Active Learning Strategies
The accuracy and the equivalent non-active learning label amount all active learning strategies on Ima-
geNet, CIFAR-10, CIFAR-100 and Pets are shown the fig. fig. 22 The equivalent non-active lcarning label amount refers to the number of samples selected by random baseline to achieve the same accuracy active learning.
19

---

Published in Transactions on Machine Learning Research (11/2024)
In fig. 23 fig. 26
illustrate the overall costs (the labeling cost and the active learning training cost) and computational efficiency achieved by the standard active learning method, SVPp, and our proposed ASVP across different datasets. Compared to the existing efficient active learning method, SVPp, our method significantly improves the overall cost while maintaining the computational efficiency. ASVP achieves comparable overall costs while significantly reducing active learning computation time compared the standard active learning method. This effectively addresses the trade-off practitioners encounter between computational efficiency and total costs, offering confidence in adopting efficient active learning methods.
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
Upper Bound (Full dataset)
70
70
80
13
33
50
50
50
120k
2008 Number Labels
2018
2006 4d0k
100k 150k 250k:
352K
1021 153k 2006
3001
402x
Number Labels
Number Lebee
(a) Margin
(b) BADGE
(c) Confidence
Figure 15: The accuracy the standard method, SVPp, and our method ASVP on the ImageNet using (a) Margin, (b) BADGE and (c) Confidence sampling.
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
8924
800
Exer Menull
250h Number oflatels
Murber
Number
(a) Margin
(b) BADGE
(c) Confidence
Figure 16:
The equivalent number of non-active learning label amounts comparison of the standard method,
SVPp, and our method ASVP on ImageNet using (a) Margin, (b) BADGE and (c) Confidence sampling. The equivalent non-active learning label amount refers to the number of samples selected by a random baseline achieve the same accuracy active learning.
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
Upper Bound (Full dataset)
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveF'T(al)
(e) Coreset
Figure 17: The accuracy of the standard method, SVPp, and method ASVP the CIFAR-10 using (a) Margin, (b) BADGE, (c) Confidence, (d) ActiveFT(al) and (e) Coreset sampling.
F
Improvement from Small Batchsize
The training costs associated with deep neural networks often lead practitioners to increase the number of samples selected per active learning iteration to ensure computational feasibility. Unfortunately, this
20

---

Published in Transactions on Machine Learning Research (11/2024)
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveFT(al)
(e) Coreset
Figure 18: The equivalent number of non-active learning label amounts comparison of the standard method, SVPp, and our method ASVP on the CIFAR-10 using (a) Margin, (b) BADGE, (c) Confidence, (d)
(al) and (e) Coreset sampling. The equivalent non-active learning label amount refers to the number of samples selected by random baseline to achieve the same accuracy as active learning.
Standard
Standard (LP-FT)
SVPp
ASVP
Upper Bound (Full dataset)
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveFT((al)
(0) Coreset
Figure 19: The accuracy of the standard method, SVPp, and our method ASVP on the CIFAR-100 using (a) Margin, (b) BADGE, (c) Confidence, (d) ActiveFTT(al) and (e) Coreset sampling.
Standard (FT)
Standard (LP-FT)
SVPP
ASVP
Sunter
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveFT(al)
(e) Coreset
Figure 20: The equivalent number of non-active learning label amounts comparison of the standard method, SVPp, and our method ASVP the CIFAR-100 using (a) Margin, (b) BADGE, (c) Confidence, (d)
(ct) and (e) Coreset sampling. The equivalent non-active learning label amount refers to the
number of samples selected by random baseline to achieve the same accuracy as active learning.
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
Upper Bound (Full dataset)
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveFT(al))
(e) Coreset
Figure 21: The accuracy of the standard method, SVPp, and our method ASVP on the Pets using (a)
Margin, (b) BADGE, (c) Confidence, (d) ActiveFT(al) and (c) Coresct sampling.
stratogy tends to result in selecting some redundant samples. Given the efficiency of our method, we can alleviate this problem. We conducted experiments on ImageNet, randomly selecting an initial pool of 10,000 samples. Subsequently, we performed 9, 45, 180, and 900 active learning iterations using margin sampling, selecting a total of 100,000 samples. The accuracy of the final model and the total sample selection time for both FT and LP-FT training are presented in table [2] The results show general improvement in active learning performance with increase in the number of iterations. However, beyond 180 iterations, the
21

---

Published in Transactions on Machine Learning Research (11/2024)
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
100
300
(a) Margin
(b) BADGE
(c) Confidence
(d) ActiveFT(al)
(e) Coreset
Figure 22: The equivalent number of non-active learning label amounts comparison of the standard method, SVPp, and our method ASVP on the Pets using (a) Margin, (b) BADGE, (c) Confidence, (d) Activel"T(al) and (e) Coreset sampling. The equivalent non-active learning label amount refers to the number of samples selected by a random baseline to achieve the same accuracy as active learning.
Random
Standard (FT)
Standard (LP-FT)
SVPp 19380
ASVP
15000
15806
14000
14000
14000
12000
12000
12000
21000
12800
11080
10000
10000
10080
Somple Selction Tine I15)
Sandic Seloctio Time hi5)
Sample Selccton Time Ihs!
(a) Margin
(b) BADGE
(c) Confidence
Figure Computation efficiency and overall cost comparison of the standard active learning method, SVPp, and method ASVP on ImageNet. using (a) Margin, (b) BADGE and (c) Confidence sampling.
Random
Standard (FT) Standard (LP-FT)
SVPp
ASVP
ActiveFT (pre)
tember Tow
(a) Margin
(b) BADGE
(c) Confidence
(d) Coreset
(e) ActiveFT (al)
Figure 24:
Computation efficiency and overall cost comparison of the standard active learning method,
SVPp, and our method ASVP on CIFAR-10 using (a) Margin, (b) BADGE, (c) Confidence, (d) Coreset
and (e) ActiveFT (al) sampling.
Random
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
ActiveFT (pre)
(a) Margin
(b) BADGE
(c) Confidence
(d) Coresel
(e) ActiveFT (al)
Figure 25:
Computation efficiency and overall cost comparison the standard active learning method,
SVPp, and our method ASVP on CIFAR-100 using (a) Margin, (b) BADGE and (c) Confidence sampling.
performance improvement. diminishes, indicating a limit to the benefits of increasing the number of active learning iterations.
Additionally, the accuracy of SVPp (final model is fine-tuned) with more active learning iterations remains lower than that of the standard active learning pipeline. However, when using I.P-FT for final model training
22

---

Published in Transactions on Machine Learning Research (11/2024)
Random
Standard (FT)
Standard (LP-FT)
SVPp
ASVP
ActiveFT (pre)
110
(a) Margin
(b) BADGE
(c) Confidence
(d) Coreset
(e) ActiveFT (al)
Figure 26:
Computation efficiency and overall cost comparison of the standard active learning method, SVPp, and our method ASVP on Pets using (a) Margin, (b) BADGE and (c) Confidence sampling.
and increasing the iteration count to 45 or more, ASVP surpassed the standard active learning method in terms of accuracy.
Table 12: The inffuence of the number active learning iterations on average sample savings ratios and sampling time on ImageNet.
# AL
Training Method
Sampling Time (hrs)
iterations
FT
LP-FT
16.64=2.06
30.22+1.885
1.1
17.66+1.37
31.20++0.40
1.5
180
19.43-1088
32.00:0.446
3.7
900
18.21=0.67
31.6440.24
15.8
G Replacement Experiment on CIFAR-10
We conducted replacement experiments on CIFAR-10 to demonstrate the impact of replacing missing samples from regions A1, A2, B1, and B2 on SVPp active learning performance. As shown in Figure 27 similar to the ImageNet replacement experiments, replacing samples from regions A1 and B1 results in little change in active learning performance, whereas replacing samples from regions A2 and B2 improves performance.
We also used Dataset Maps to illustrate the difficulty distribution of samples from different regions. The Dataset Map calculates the average and standard deviation of the model's confidence for the correct class (need true labels) using training dynamics from the entire dataset. Due to the simplicity of CIFAR-10, the Dataset Map shows very high mean confidence for almost all samples. Therefore, the difficulty based on mean confidence, as used Section 4.1, would categorize nearly all samples as "easy".
Instead, we classified sample difficulty based on the standard deviation of the model's confidence the correct class. Specifically, we defined the difficulty levels as follows: very hard (std. >= 0.15), hard (std. >= 0.1 and std. < 0.15), medium (std. >= 0.05 and std. 0.10), and easy (std. < 0.05). The resulting distribution of different difficulty levels across regions is shown in Figure 28 where the proportion of each difficulty level within the entire dataset is displayed in the last column.
23

---

Published in Transactions on Machine Learning Research (11/2024)
Active Learning Gain (96
Standard taie replace)
Seplace with A2
(wic reptace!
Replace
Replace
Ruplace
1000
2000 3000 4000
5000
6000
Number of Labels
Figure 27: The impact of different regions on SVPp active learning performance. Replacing samples selected by the proxy model with those from regions A1, A2, B1, B2. The active learning gain refers to the difference in accuracy between active learning and the random baseline CIFAR-10.
Very Hard
Hard
Medium
Easy
(a) A1
(b) A2
(c) B1
(d) B2
Figure 28: Difficulty distribution of samples from different regions on CIFAR-10.
24


Reviews:
[{"review_id": "NryyLluUzX", "reviewer": "Reviewer_KTcw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper addresses an important problem in learning.\n2. The proposed method upgrades the second stage in SVPp where SVPp's data selection uses only a simple classifier to select data which is hard to sample a good distribution of examples. The methodology is sound.\n\nWeaknesses:\n1. Overestimation of contributions: the diversity of sample distributions are important in active learning, which is known in the field. also, the metric for evaluating active learning does not seem novel. It is equivalent to sample size @ certain accuracy which is a common way to evaluate learning methods.\n2. The paper is a bit hard to follow and confusing. \n3. Based on (1), I think the paper's contribution is quite limited and doesn't really worth publishing a long paper. Could be interesting workshop paper though.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper's contributions are three folded:\n1. It empirically analyzes the sample selection differences that contribute to the performance drop when employing Selection\nvia Proxy based on pre-trained features  (SVPp).\n2. It proposes a novel efficient active learning approach, Aligned Selection via Proxy (ASVP), which improves the performance of efficient active learning while incurring a marginal amount of computational time.\n3. It introduces an evaluation metric for efficient active learning, the Sample Saving Ratio, which directly quantifies the savings in annotation achieved by employing active learning strategies compared to the random baseline.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I suggest the authors restructure the paper's organization. The current one reads a bit confusing as the major contributions are not emphasized. The observations can be shortened and some of them are experimental results (should be moved to experiment sections).\n\n2. emphasize why the observations and the proposed evaluation metric is novel.", "broader_impact_concerns": "No specific impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dxFplDoXaY", "reviewer": "Reviewer_MQfq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper tackles an important practical issue: how to frugally improve the fine-tuning of a model. The proposed approach appears to be novel, original, sound, and potentially impactful. It also shed light on the root causes of the limitations of existing approaches. \n\nThe paper's main limitation is that it only analyses active learning in its early stages (that is, on the first N examples, as seen, for instance, in Fig 11) when the model's performance is still far from SOTA. For all 4 datasets in Fig 11, SOTA is far superior to the performance of AL, which raises three questions: \n(1) how many more examples are required to reach SOTA? \n(2) why should we care which approach is better at reaching a (sub-)mediocre performance? \n(3) most importantly, will the paper's insights & findings hold once enough examples are labeled to reach SOTA(-adjacent) performance? This is closely related to the seminal findings of [Banko & Brill, 2001] on what area of a learning curve does performance really matter.\n\n[Banko & Brill, 2001] Banko, Michele, and Eric Brill. \"Scaling to very very large corpora for natural language disambiguation.\" Proceedings of the 39th annual meeting of the Association for Computational Linguistics. 2001.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel approach to proxy-based active learning that balances the trade-offs computational cost savings and model performance optimization. It shows that (1) only some types of sample selection result in the model's  performance degradation, and (2) suitable training methods can mitigate the decline of active learning performance. The proposed novel method, Aligned Selection Via Proxy (ASVP), improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Fig 1 has pedagogical value, but the paper would greatly benefit from using at least one real-world motivating example in which SOTA is reached with realistic (a) annotation time/cost, and (b) model re-training time/cost. In many practical applications, getting an example's label is not instantaneous (as in CIFAR or ImageNet), but it may take significant time (eg, 3-7 days for 2K examples). In such scenarios, the time required to re-train the model may be dominated by the examples' labeling, so only the cost of retraining is a factor. Ideally, you should use a version of Fig 1 for such a realistic scenario.\n\n2.  The example in Fig 2 seems to be poorly chosen, as the savings of Standard AL appear to be larger than for SVP AL (i.e., $854 = $961 - $107  vs   $805 = $808 - $3). Again, a variation of Fig 2 based on a real-world application would be greatly motivating.\n\n3. Why did you change the setup between Figs 1 & 2 (i.e., 10x 200 vs 10x 10K)? Ideally, they should be two illustrations of the same experiment. \n\n4. Section 4.1 is a good start, but should go deeper. Fig 6 is interesting, but it raises several questions: \n(1) in itself, querying more difficult examples is NOT a bad thing, especially if the concept is \"highly-learnable;\" how many of these difficult ImageNet examples are part of the ~8% that SOTA still makes mistakes on? if you can identify-and-ignore such examples, you may have a (another) novel approach to AL.  \n(2) on a related note, could you please add the counter-part of Fig 6 for the other three datasets? are the findings holding on easier domains, like CIFAR-10, where nearly-perfect learning is possible? if yes, why is it bad to identify early these difficult examples? intuitively, it should actually help you converge faster.\n(3) given that the overall proportion of very hard examples is comparable to the SOTA error rate on ImageNet, how would your zone A/B/C/D analysis change if you could simply identify-and-NeverQuery these difficult examples? Add a figue in which you compare ASVP vs this \"oracle-fueled\" strategy of NOT querying very hard examples? If \"the oracle-base AL\" is better, whichis a simpler strategy that focusing on A2/B2, it may be even worth creating an \"online version/approximation\" of Dataset Maps that would allow you to identify/ignore such examples.\n\n5. Figs 7, 8, 9, 11: extend X axis until (near-)SOTA performance\n\n6. In Table 1:\n   - why is ASVP the top-performer on Cost but NOT on Avg Sample Saving Ratio?\n   - how meaningful are the actual cost savings? again, a real-world example with real annotation costs would greatly help\n   - is a cost saving ratio of 1% worth achieving/pursuing (ie, 74% vs 75%)? again, a real-world example would greatly help\n   - how meaningfull are the ASVP cost savings, given that they are dominated by ImageNet, where the accuracy in Fig 11.a ends at 68% vs SOTA's 92%? Will these savings hold if you label another 200K/400K examples for ImageNet?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PzZPcLVUjC", "reviewer": "Reviewer_ytc5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n 1. The paper tackles an important problem, and provides a nuanced take.  \n\n 2. The writing style mixes explanation for design choices well with experiments.  \n\n 3. The paper seems to provide an honest take of pros and cons of the techniques\n\n\nWeaknesses:\n 1. The improvements of the proposed method seem small, and sporadic.  \n\n 2. The distinction between samples of type A vs B (and A1 vs. B1 and A2 vs B2) was confusing.  This seems to be whether the fine-tuned model has a positive or negative sign?  Or maybe was correct or incorrect (I am guessing from Figure 4).  This should be more clear in text.  \n  Moreover, if it is based on signs, then how does this make sense -- the sign choice should be semantic, and the algorithm should run the same if the signs are flipped?  Did you consider this (if it has to do with signs)?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes and investigates a more nuanced approach to fine-tuning large pre-trained models.  Fine-tuning models helps performance, but can be expensive on large models.  Actively selecting training points for fine tuning helps, but takes time to select the best points.  This work shows that sometimes the pre-trained model can be leveraged to actively select points quickly, and other times its representation of the margin is off, and one needs to iteratively update the model to get a better sense of margin, and get improved active learning.  \n\nThe paper explains the ideas fairly well, and provides a reasonable set of experiments.  It shows the gains are modest, but still might be enough to make a difference.  Regardless of the degree of benefit, it looks like there may be some, and the overall study seems of interest to the ML community.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address Weakness 2", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers think that the paper is of interest to TMLR audience. Regarding the supporting evidence, the reviewers are mostly satisfied with the responses, but they still think the current revision needed more work. Specifically they requested the following changes in the final version: 

- Revise Figure 2 (so that the training costs of SVP_p is clear)
- Add explanations in the paper on why experiments are only evaluated on smaller models rather than SoTA large models
- For Fig 11, add explanations on why only two of the four axis were extended, and what is the expected performance upper bound if we use the full dataset

@authors: as we enter the final revision stage, please feel free to follow up if any clarification on the requests are needed.
Decision: Accept (poster)