AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Authors: Yifeng Wang
Xueying Zhan
Siyu Huang
Venue: ICML 2025 poster (2025)
Keywords: Active Learning
Differentiable Bi-level Optimization
Primary area: general_machine_learning->online_learning_active_learning_and_bandits
TL;DR: To our knowledge, we present the first automatic AL query strategy search method that can be trained in a differientiable way.

Abstract: As deep learning continues to evolve, the need for data efficiency becomes increasingly important. Considering labeling large datasets is both time-consuming and expensive, active learning (AL) provides a promising solution to this challenge by iteratively selecting the most informative subsets of examples to train deep neural networks, thereby reducing the labeling cost. However, the effectiveness of different AL algorithms can vary significantly across data scenarios, and determining which AL algorithm best fits a given task remains a challenging problem. This work presents the first differentiable AL strategy search method, named AutoAL, which is designed on top of existing AL sampling strategies. AutoAL consists of two neural nets, named SearchNet and FitNet, which are optimized concurrently under a differentiable bi-level optimization framework. For any given task, SearchNet and FitNet are iteratively co-optimized using the labeled data, learning how well a set of candidate AL algorithms perform on that task. With the optimal AL strategies identified, SearchNet selects a small subset from the unlabeled pool for querying their annotations, enabling efficient training of the task model. Experimental results demonstrate that AutoAL consistently achieves superior accuracy compared to all candidate AL algorithms and other selective AL approaches, showcasing its potential for adapting and integrating multiple existing AL methods across diverse tasks and domains.

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Yifeng Wang' Xueying Zhan?
Siyu Huang
Abstract
1. Introduction
With the development of deep learning (DL) techniques, large volumes of high-quality labeled data have become increasingly crucial as the model training processes usually data hungry (Ren et al., 2021; Zhan et al., 2022), Active learning (AL) addresses this challenge by iteratively selecting the most informative unlabeled samples for anno- tation, thereby boosting model efficiency (Xie et al., 2021; Wang et al., 2023). Deep active learning strategies are typically divided into two categories: uncertainty-base and representiivanesddivvvstyyyy & Savarese, 2017; Zhu & Bento, 2017; Ash et al., 2019). Uncertainty- methods focus on querying samples the model is most uncertain about, while diversity- -asse strate- gies aim to select a diverse subset of samples that effectively represent the entire dataset (Citovsky et al., 2021). Both strategies have limitations, especially in batch selection and varying dataset complexities, leading to the inconsistency of the strategy efficiency (Ren et al., 2021). Hybrid strate- gies, which balance uncertainty and diversity (Zhan et al., 2021a;b), offer partial solutions. However, these strategies rely on strategy choices and subjective experiment settings, which can be ineffective in real-life applications.
As deep learning continues evolve, the need for data efficiency becomes increasingly impor- tant. Considering labeling large datasets is both time-consuming and expensive, active learning (AL) provides a promising solution to this chal- lenge by iteratively selecting the most informa- tive subsets of examples to train deep neural net- works, thereby reducing the labeling cost. How- ever, the effectiveness of different AL algorithms can vary significantly across data scenarios, and determining which AL algorithm best fits a given task remains a challenging problem. This work presents the first differentiable AL strategy search method, named AutoAL, which designed on top of existing AL sampling strategies. AutoAL consists of two neural nets, named SearchNet and FitNet, which are optimized concurrently un- der a differentiable bi-level optimization frame- work. For any given task. SearchNet and Fit- Net are iteratively co-optimized using the la- beled data, learning how well a set of candi- date AL algorithms perform on that task. With the optimal AL strategies identified, SearchNet selects small subset from the unlabeled pool for querying their annotations, enabling efficient training of the task model. Experimental results demonstrate that AutoAL consistently achieves superior accuracy compared to all candidate AL algorithms and other selective AL approaches, showcasing its potential for adapting and inte- grating multiple existing AL methods across di- verse tasks and domains. Code is available at: https://githu..com// http://www.....oaaaaaaaaa//////a
approaches (Sener
Previous works (Baram et al., 2004; Hsu & Lin, 2015; Pang et al., 2018) have explored selecting among different AL strategies to achieve optimal selections without human ef- fort. SelectAL (Hacohen & Weinshall, 2024) introduces method for dynamically choosing AL strategies for dif- ferent deep learning tasks and budgets by estimating the relative budget size of the problem. involves evaluating the impact of removing small subsets of data points from the unlabeled pool predict how different AL strategies would perform. However, this strategy depends on approximating the reduction in generalization error on small subsets, which would not fully capture the complexities of real-world tasks. Zhang et al. (2024) proposes to select the optimal batch of AL strategy from hundreds of candidates, aiming to maxi- mize future cumulative rewards based on noisy past reward observations of each candidate algorithm. However, both the computational cost and the lack of differentiability make the optimization of these works inefficient.
'Department of Electrical and Computer Engincering, Camegic Mellon University. Pittsburgh. USA 'Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University, Pittsburgh. USA VVisual Computing Division, School of Com- puting, Clemson University, Clemson, USA. Correspondence to: Xucying Zhan aaneyynnzzeaddee.mmm....
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
To address these challenges, we propose AutoAL, an au- tomated active learning framework that leverages differen-

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
tiable query strategy search. AutoAL offers two key advan- tages: 1) it enables automatic and efficient updates during optimization, allowing the model to adapt seamlessly; 2) it is inherently data-driven, leveraging the underlying data distribution to guide the AL. strategy selection. However, integrating multiple AL strategies into the candidate pool makes achieving differentiability difficult. To overcome this issue, AutoAL leverages a bi-level optimization frame- work that smoothly integrates and optimizes different AL strategies. We design two neural networks: SearchNet and FitNet. FitNet is trained on the labeled dataset to adap- tively yield the informativeness of each unlabeled sample. Guided by FitNet's task loss, SearchNet is efficiently op- timized to accurately select the best AL strategy. Instead of searching over a discrete set of candidate AL strategies, we relax the search space to continuous domain, allowing SearchNet to be optimized via gradient descent based FitNet's output. The gradient-based optimization, which is more data-efficient than traditional black-box searches, enables AutoAL to achieve state-of-the-an performance in strategy selection with significantly reduced computational costs. Our key contributions are as follows:
diversity-based and uncertainty-based. The diversity- -based methods select the hatch of samples that can best represent the entire dataset distribution. CoreSet (Sener & Savarese, 2017) selects a batch of representative points from a core set, which is a subsample of the dataset that effectively serves as a proxy for the entire set. In contrast, uncertainty- based methods focus on selecting the samples with high uncertainty, which is due to the data generation or the model- ing/learning process. Bayesian Active Learning by Disagree- ments (BALD) (Gal et al., 2017) chooses data points that are expected to maximize the mutual information between pre- dictions and model posterior. Meta- veta-Query Net (Park et al., 2022) trains an MLP that receives one open-set score and one AL. score as input and outputs a balanced meta-score for sample selection in the open-set dilemma. Yoo & Kweon (2019) incorporates loss prediction strategy by appending compact parametric module designed to forecast the loss of unlabeled inputs relative to the target model. This is achieved by minimizing the discrepancy between the pre- dicted loss and the actual target loss. It requires subjective judgment and strategic selection of active learning methods to be effective on specific datasets in real-world settings.
We introduce a novel AutoAL framework for differen- tiable AL query strategy search. To the best of our knowl- edge, AutoAL, is the first automatic query strategy search algorithm that can be trained in differentiable way, achieving highly efficient updates of query strategy in a data-driven manner.
Adaptive Sample Selection in AL. Neither diversity- based nor uncertainty-base methods are perfect in all data scenarios. Diversity- based methods tend to perform better when the dataset contains rich category content and large batch size, while uncertainty-aware methods typically per- form better in opposite settings (Zhan al., 2021b; Citovsky et al., 2021). Several past works have explored adaptive selection in active learning to solve this problem. A com- mon approach is to choose the best AL strategy from a set of candidate methods and use it to query unlabeled data, such as (Hacohen & Weinshall, 2024) and (Zhang et al., 2024) introduced in Section Active Learning By Learn- ing (ALBL) (Hsu & Lin, 2015) adaptively selects from set of existing algorithms based their estimated contribu- tion to learning performance. It frames each candidate as a bandit machine, converting the problem into a multi--rmed bandit scenario. However, none of these methods leverage the differentiable framework for automatic AL selection. Compared to the methods mentioned above, AutoAL first relaxes the search space be continuous, thus can opti- mize automatically via gradient descent. This framework enables Au to seamlessly integrate multiple existing AL strategies and rapidly adapt to select the optimal strat- egy based on the labeled pool. The simple gradient- based optimization is much more data-efficient than traditional black-box searches. Therefore, AutoAL. can significantly reduce computational costs compared these methods.
AutoAL is built upon existing AL strategies (i.e., uncertainty-based and diversity-based). By incorporat- ing most AL strategies into its search space, AutoAL leverages the advantages of these strategies by finding the most effective ones for specific AL settings, objectives, or data distributions.
We evaluate AutoAL across various AL tasks. Given the importance of AL in domain-specific tasks, particularly where data quality and imbalance are concerns, we con- ducte extensive experiments on both nature image datasets and medical datasets. The results demonstrate that Au- toAL consistently outperforms all candidate strategies and other state-of-the the AL methods across these datasets.
2. Related Work
Active Learning. Active Learning has been studied for decades to improve the model performance using small sets of labeled data, significantly reducing the annotation cost (Cohn et al., 1996; Settles, 2009; Zhan et al., 2021b). Most AL research focuses on pool-based AL, which identi- fies and selects the most informative samples from a large unlabeled pool iteratively. Pool-based AL sampling strate- gies are generally categorized into two primary branches:
Bilevel Optimization. Bilevel optimization is a hierarchi- cal mathematical framework where the feasible region of one optimization task is constrained by the solution set of
2

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Loss Detach
Samples
Half
FitNet
AutoAL
Queue
After Training
Batch B+1
Fitteet [Cycle c-2)
Loss
4p
Labeled Pool
Half Queue
Batch
SearchNet [Cycle Cycle c)
Weights
M
Sigmold Relaxation
0.002
0.044
00.55
Score
0
0.02
0
Probabilistic Query
0.00
о 0.73
0.99
AL Score
B Bi-Level Optimization
Update
AL Candidates
Annotator
After Training
Top-b Samples
Labeled Pool
AutoAL
Unlabeled Pool
Selection
Figure 1. Overall framework of differentiable query strategy search for automated active learning (AutoAI.). AutoAL leverages labeled pool to train the FitNet and SearchNet in a bi-level optimization mode. Data samples with the largest search score are then selected from the unlabeled pool.
another optimization task (Liu et al., 2021). has gained sig- nificant attention due its nested problem structure, which allows the two tasks to optimize jointly. Bilevel optimization adapts to many DL applications (Chen et al., 2022), such as hyperparameter optimization (Chen et al., 2019; MacKay et al., 2019), meta-knowledge extraction (Finn et al., 2017), neural architecture search (Liu et al., 2018; Hu et al., 2020), and active learning (Sener& & Savarese, 2017). For instance, Sener & Savarese (2017) formulates AL bi-level coreset selection problem and designs 2-approximation method to select subset of data that represents the entire dataset. In this work, we novelly formulate the Al query strategy search as a bi-level optimization problem, i.e., iteratively updating FitNet and using its task loss to guide the optimization of SearchNet. We further reformulate it as differentiable learning paradigm to enable more efficient selection of examples across various objectives and data distributions.
iteration, we select a batch of the most informative data Q' arg max a(x, (2) with batch size from U based the basic learned model Ω and the acquisition function (xr, 22), and query their labels from oracle/annotator. With these samples, the expected loss function f of the basic learned model can be minimized. L and U are then updated.
The performance of existing AL methods relies on the choice of query strategies, which should be carefully adapted to different tasks applications. To achieve ro- bust and adaptive AL performance, this work novelly creates an automated AL strategy selection framework. AutoAL in- tegrates two neural nets, FitNet Qi and SearchNet Ωs upon each basic learned model f2 to select the best AL strategy be- fore selecting Q. To facilitate automatic selection, AutoAL incorporates them into bi-level optimization framework and relaxes the search space enable differentiable up- dates. We discuss more details of AutoAL framework in the following sections.
3. Methodology
3.2. Automated Active Learning
3.1. Problem Setting
We propose Automated Active Learning (AutoAL), which aims to adaptively deliver the optimal query strategy for each sample in given unlabeled data pool. Specifically, AutoAL consists two neural networks: FitNet 2p and SearchNet f2g. Ug selects the optimal AL strategy from set A, which contains candidate AL sampling strate- gies [AA.)e([]] (e.g., BALD, Maximum Entropy, etc). ΩF models the data distribution within the unlabeled dataset and guides the training of SearchNet Qs. Since annotations
Pool-based AI focuses on selecting the most informative data iteratively from a large pool of unlabeled independent and identically distributed (i.i..)) ). data samples until a fixed budget is exhausted the expected model performance has been reached. Specifically, in AL processes, assuming that we have an initial seed labeled set L [( M and large unlabeled data pool U {æ, N. where M & N, y: E {0, 1} is the class label of z; for binary classification and y1 E {1, ..... p} for multi-class classification. In each
3

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
for the unlabeled data cannot be accessed directly by the model 52, AutoAL requires no data from the unlabeled pool U, using only the labeled dataset L for training.
S(rry) which is a combination of all the AL search scores Sk(ay,, we adopt Gaussian Mixture approach (Reynolds et al., 2009), which has been proved efficient in AL (Zhao et al., 2020):
In AutoAL, each AL iteration consists of C cycles. In each cycle c, the labeled data L is randomly split into two equal- sized subsets: training and validation sets. FitNet Qp com- putes the cross-entropy loss C (Zhang & Sabuneu, 2018) for data. Meanwhile, SearchNet Ui generates scores for the samples, which are used to rank them. This raises key question: If only the training set is available, without access to the unlabeled pool, can f2g still effectively select the optimal samples?
p(S) = Ex Amml
Th N pag Ex),
(2)
M {#Ks juk En} = arg max Pi II ((S()))),
(3)
where TT is the weight of the k-th gaussian component and A is the probability density function. Then we generate samples according to the Gaussian Mixture model and get the t-th maximum value, where is the ratio of batch size b to the total pool size M N. Wx.5 is the probability of each query strategy for each sample predicted from the neural network f2g.
Our key design is intuitive: SearchNet S2y treats the training batch as the labeled pool and the validation batch as the un- labeled pool. This allows Ωg to simulate the process of AL selection, without direct access to the actual unlabeled data pool. The output of f2g is shown in Figure 1, sample-wise score that aggregates the candidate AL strategy selection. The output of fgg is the informative loss of each sample. The optimization goal of fig is to select data with higher losses, as they are expected to provide more informative up- dates to the neural network. For S2p, the training objective is to minimize the loss of the selected samples, particularly those prioritized by S2g. AutoAL is formulated as a bi-level optimization problem, as the optimization of Qp is depen- dent only on itself, but the optimization of Ωg depends on the optimal 2F..
Ssample E
K k=1 tx. N(S juR: En),
(4)
Sx(2,, (5) Ur (Saamlt)) * Wn.
(5)
A sample with a higher score S indicates that it has been selected by more candidate AL. strategies, and therefore should be given higher priority for labeling.
Differentiable acquisition function optimization. we limit the individual query score (x,) to discrete values (0, 1). the bi-level optimization objective of Eq. 1 becomes non-differentiable and still difficult to optimize. To en- able differentiable optimization, we relax the categorical selection of strategies into a continuous space, we apply a Sigmoid function over all possible strategies.:
22'- = arg max Us
Σ
M/2
Ls Cs(( (Rs,
M/2 Σ 15 j=1 CF Cr((1,, U,). 22))
s.t. 26- arg argmin
OF
LF and Ls represent the losses of S2r and Ugg, respectively. The neural nets Q2F and Qg are optimized jointly as out- lined in Eq. 1. At the lower level of the nested optimization framework, 52p is optimized to approach the distribution of the dataset. At the upper level, 22g is optimized to out- put the informativeness of each data, based on the optimal distribution returned from 12p-
S(I,) = Σ
A
! SA Us),
(6)
KEK 1+ expl
where the strategy mixing weights for each sample j is pa- rameterized by a vector ( of dimension |A|]. X is scaling vector. S(x,) is the final differenniable learning objective function, which can be optimized with back nock-propagatio. Therefore, Op and Qg can be efficiently optimized under Eq.
3.3. Differentiable Query Strategy Optimization
Although Eq. shows an effective framework for auto- matically deriving the optimal query strategy, solving the bi-level optimization problem is inefficient in practice. To address this, derive a probabilistic query strategy coupled with differentiable optimization framework, improving the efficiency of the optimization process.
3.4. Learning and Algorithm of AutoAL
With differentiable query strategy optimization in Sec. 3.3, we reformulate the bi-level optimization problem in Sec. 3.2 as efficient optimization objectives as follows:
Probabilistic query strategy. During training, for each la- beled data we query a score Sk(z,) from each candidate AL sampling strategy Ai, where Sk(2,) E 0, indicates whether the sample selected. To model the overall score
2n+ Σ
1
CF
Saetack (1/c (x4)) 1/)+ + B),
B
j'=1+(nt)BB
(7)
4

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
(n+1)B S(zz) Edetach(2z XCre B), 11++nB
the baselines by fully automating the training within the dif- ferentiable bi-level optimization framework. Additionally, we conduct ablation studies to analyze the contribution of each module and examine the effects of the candidate AL strategies, including different numbers of candidates.
Ls
Σ
(8) As described in section 3.2, in cycle 2-2, S2p is optimized to get the data distribution. The final loss is calculated according to Eq. 7. In cycle c-1, AutoAL optimizes fg to get the data sample with highest loss. The loss function from S2p is detached to avoid updating itself, and the final loss is calculated according to Eq. 8. Here we use negative loss to achieve gradient ascent. For n E {o. L AutoAL additionally integrates a loss prediction module according to (Yoo& & Kweon, 2019) to help update Ug as
4.1. Experiment Settings
Datasets and Baselines. We conduct AL experiments on seven datasets:
10 and Cifar-100 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), TinyImageNet (Le & Yang. 2015) in the nature image domain, and OrganCM- NIST, PathMNIST, and TissueMNIST from MedMNIST database (Yang al., 2023) in the medical image do- main. The details of datasets are presented in Appendix Table 1. We compare our method against multiple AL sampling strategies, including Maximum Entropy Sam- pling (Shannon, 1948), Margin Sampling (Netzer et al., 2011), Least Confidence (Wang & Shang. 2014), KMeans Sampling (Ahmed et al., 2020), Bayesian Active Learn- ing by Disagreements (BALD) (Gal et al., 2017), Variation Ratios (VarRatio) (Freeman, 1965) and Mean Standard Deviation (MeanSTD) (Kampffmeyer et al., 2016). We also consider the statc-of-the-aar deep AL methods as base- lines, including Batch Active learning by Diverse Gradient Embeddings (BADGE) (Ash et al., 2019), Loss Prediction Active Learning (LPI.) (Yoo & Kweon, 2019), Variational Adversarial Active Learning (VAAL) (Sinha al., 2019), Core-set Selection (Coreset) (Sener & Savarese, 2017), En- semble Variance Ratio Learning (ENSvarR) (Beluch et al., 2018), 8), Deep Deterministic Uncertainty (DDU) (Mukhoti et al., 2023). We also include ALBL (Hsu & Lin, 2015) to compare AutoAL with existing selective AL strategies.
LI,
1 exp (0.5 Ja - t. B)) 0.5
(9)
Cre represents regularization loss that limits the number of selected samples, as shown in Eq. 9. Here, a is the number of selected samples, t is the ratio of the query batch size b in each AL iteration to the total pool size M + N, and À is a scaling vector. The final learning objective is to select the most informative samples while effectively leveraging the underlying data distribution.
The AutoAL algorithm is illustrated in Algorithm 1. In each AL iteration, the labeled set is first used to train the FitNet and SearchNet as described in section 3.2. Then the optimal SearchNet O'g is applied to the unlabeled pool to select a new batch of data, where Di is used to generate score S for each sample 2;, and Q" arg max Sax,) with top-b highest scoring samples
selected and labeled.
The labeled set L and the unlabeled set U are updated as L= L + Q* and U = U Q", respectively. The updated labeled set is then used for task model training.
Algorithm 1 AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Input: K candidate algorithms A = (Ax) labeled pool = {(x,, y; Af and unlabled pool U = JM total number of AL rounds R, batch size b, task model T.
Implementation Details. For 2p and Ugg we build the backbone using ResNet-18 (He et al., 2016). We also em- ploy ResNet-18 18 as the classification model on all baselines and AutoAL for fair comparison. Since AutoAL is built upon existing AL strategies and focuses selecting the op- timal strategy, we integrate seven AL methods into AutoAL: Maximum Entropy, Margin Sampling, Least Confidence, KMeans, BALD, VarRatio, and MeanSTD. For the opti- mization of Us and loss prediction module, we use SGD optimizer (Ruder, 2016). For the optimization of Q2p, we use Adam optimizer (Kingma, 2014). Both use 0.005 as the learning rate. While training. FitNet will first update for 200 epochs using the validation queue, then Rr, 2g and the loss prediction module will update iteratively with a total of 400 epochs. All experiments are repeated three times with different randomly selected initial labeled pools, reporting mean and standard deviation.
Output: task model T
1: Initialize T
2: for c=1,...,RR do
3: Optimize 2ir,, 22' according to Eq. 1;
4: Calculate S(z:) by Eq. 6 for all E N:
5: Select the top b samples with the highest score S(xy);
6: Query label y: for all i b. Update L and U:
7: Train task model T by using L;
8: end for
9:
return T
4. Experiments
We empirically evaluate the performance of AutoAL on a wide range of datasets with both nature and medical images. These experiments demonstrate that AutoAL outperforms
5

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
EntropySSmpling
MarginSampling
BALD
ALBL
Coresct
ENSvarR
KmeansSampling
MD
BadgeSanpling
DDU
VAAL
AutoAL
LeaseConfidence
VarRatio
LPL
(a) Cifar-10
Cifar-100
(c) SVHN
TinyemageNet
60.0
50.0
94.0
85.0
Mcan Accuracy (%)
Mcan Accuracy (%)
35.0
Mcan Mcan Accuracy (36 (36)
45.0
80.0
50.0
Mcan Accuracy
92.0
40,0
75.0
45.0
90.0
33.0
70.0
40.0
88.0
65.0
86.0
15.0
400
%/ labeled data
%/ of labeled data
1/6 labeled data
% labeled data
OrgNSC
(1)
PathMNIST
(g)
MNIST
$8.0
90.0
38.0
Mean Accurncy (%6
56.0
Mean Accurccy (%))
Mean Acauracy
64.0
$4.0
84.0
83.0
82.0
$1.0
80.0
30.0
40 5.0 10.0 12.0 14.0 15.0 18.0 20.0
20
3.0
4.0
5.0
R.0 9.5
10.0
4.0 5.0 8.0 1000 12.0 14.0 18.0 2000
of labeled data
5/ of labeled datn
%/ of labeled data
Figure Overall performance on seven benchmark datasets: natural image datasets (top) and medical image datasets (bottom).
4.2. Evaluation of Active Learning Performance
3.
The labeled data vs. accuracy curves of our method are relatively smooth compared to those of other methods. The baseline methods sometimes show significant accu- racy drops during certain active learning rounds. This mainly dues harmful data selection (Koh & Liang, 2017) overfitting problem. However, by selecting the optimal strategy thus selecting the informative data, Au- toAL can alleviate the problem and make the curve much more smooth than baseline strategies.
Fig. 2 shows the overall performance comparison of differ- ent AL methods, where AutoAL consistently outperforms the baselines across all datasets. Additionally, we have the following observations:
1. Our method consistently outperforms the other ap- proaches terms of rounds, a crucial attribute for a successful active learning method. This adaptability is particularly valuable in real-world applications, where the labeling budget may vary significantly across differ- ent tasks. For instance, one might only have the resources to annotate 10% of the data, rather than 20% or 40%.
4. Our method shows smaller standard deviations, indicat- ing that it is robust across repeated experiments. This robustness is crucial for AL applications, especially in real-world setting, where the AL process is typically conducted only once.
Our method demonstrates robust performance not only on easier datasets but also on more challenging ones, such as Cifar-100, OrganCMNIST and TinylmageNet. Cifar-100 and TinylmageNet has significantly more classes than other datasets, while OrganCMNIST has smaller data pool, with only 12, 975 images. These challenges make active learning more difficult, yet our method's 's superior performance across these datasets highlights its robustness and generalizability.
5. Different AL strategies produce varying results across datasets. For instance, Margin Sampling underperforms on the SVHN dataset, while KMeans Sampling and VAAL yield the worst performance on the OrganCM- NIST and Cifar-10 datasets. This variability highlights the significance of our approach, which aims identify and select the optimal strategy for each dataset.
6

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
ResNet Backbone
Loss Prediction Module
LesNet+Loss Prediction
Cifar-100
(b) SVHN
(c) Organ CMNIST
60.0
Mean Accuracy (%)
Mean Accuracy (%)
55.0
93.0
Mean Accuracy
88.0
92.0
91.0
86.0
45.0
90.0
84.0
40.0
89.0
88.0
82.0
35.0
87.0
80.0
8.0 12.0 16.0 20.0 24.0 28.0 32.0 36.0 40.0 44.0
4.0
8.0 10.0 12.0 14.0 16.0
18.0 20.0
4.0
6.0
12.0 14.0 16.0 18.0 20.0 % of labeled data
% of labeled data
% of labeled data
Figure Ablation study on three components AutoAL.. 'ReNNet Backbone': without the loss prediction module, updating only the ResNet. 'Loss Prediction Module': without updating the ResNet backbone, optimizing solely with the loss prediction module. LResNettLoss Prediction': the full AutoAL pipeline.
candidate
3 candidates
candidates
candidates
(a) Cifar-100
(b) SVHN
(c) Organ CMNIST
60.0
Mean MeanAccure Accuracy (%)
55.00
Mean Accuracy (%)
94.0
Mean Accuracy (%)
90.0
92.0
88.0
50.0
86.0
45.0
90.0
84.0
82.0
35.0
86.0
80.0
12.0 16.0 20.0 24.0 28.0 32.0 36.0 40.0 8.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0 18.0 20.0
% of labeled data
% of labeled data
% of labeled data
Figure 4. Ablation study on size the candidate pool AutoAL strategy selection.
4.3. Ablation Studies
with network optimization, the main focus shifts from select- ing the best AL strategy to merely minimizing the sample loss. This misalignment will result in incorrect AL strategy weight estimation (See Fig. 1).
We perform the ablation study on two key components of AutoAL: the SearchNet architecture and the AL candidate strategies. Experiments are conducted on three datasets: Cifar- 100, SVHN, and OrganCMNIST, which are chosen for their mix of natural and medical images, as well as the increased difficulty they present for active learning applica- tions.
Size of Active Learning Candidate Pool. Since AutoAL is built on multiple AL strategies, study how the size of the AL candidate pool impacts performance. We start from the Maximum Entropy as the sole candidate for the initial baseline, then expand the candidate pool to three strategies by adding Margin Sampling and Least Confidence. For the pool of five candidates, we further include KMeans and BALD, As shown Fig. for all three datasets, the single candidate variation performs the worst but still outperforms the Entropy Sampling baseline in Fig. 2. This is because AutoAL does not directly query data samples with maxi- mum entropy but queries the data with the highest score from SearchNet, which has been optimized with the help of the loss prediction module and FitNet. For Cifar- 100 and
SearchNet Architecture. Our SearchNet consists of two main components: a ResNet-18 backbone and loss pre- diction module for sample loss prediction (Yoo & Kweon, 2019). In this ablation study, disable one component at time and record the results. Fig. 3 illustrates the per- formance, revealing the effectiveness of both ResNet- 18 backbone and the loss prediction module. However, when only the loss prediction module is used to optimize the net- work, performance drops significantly. This aligns with our intuition, as although the loss prediction module can assist

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
(a) Cifar-100
OrganCMNIIST
0.8
EntropySampling
EntropySSmpling
0.7
MarginSampling
MarginSampling
LeastConfidence
LeassConfidence
0.5
KmeansSampling
KmeansSampling
BALD
BALD
0.3
VarRatio
VarRatio
0.1
McanSTD
MeanSTD
2
3
4
5
7
8
4
5
6
AL Rounds
AL Rounds
Figure 5. strategy scores across different AL rounds on CIFAR-I00 OrganCMNIST datasets.
OrganCMNIST, AutoAI. can perform well when there are at least three candidates in the selection pool. However, for
cess progresses and more diverse data become available, the model attempls to develop a better understanding the underlying structure of the target data. Uncertainty-bast -based measures are more effective now because the major chal- lenge switches to refining the decision boundaries. The transition from diversity to uncertainty-basse strategies is consistent with the model's evolving needs. In early rounds, the model requires diversity to build a broad understanding of the data, but once that foundation is established, its fo- cus shifts toward uncertainty. which helps fine-tune model recisiin-maaiin ability. This adaptive approach ensures the model targets the most diverse subsets each stage of learning process, from broad exploration in the early rounds to fine-graine exploitation in the later rounds. By automat- ically adjusting strategy priorities, AutoAL improves the overall accuracy and shows stronger robustness.
SVHN, more AL candidates result better performance. This indicates that the upper bound of candidate pool size varies across different datasets. In certain datasets, incorpo- rating more active learning strategies yields better results, but this is not the case for other datasets. Additionally, pools with more candidates exhibit lower standard devi- ations. This demonstrates AutoAL's strong capability to select the best AL strategy. can consistently choose the optimal one and exclude the less effective options.
4.4. AL Selection Strategy
Here, we examine which AL strategy is prioritized by Au- toAL in different active learning rounds. For each AL strat- egy candidate,
calculate the AL score of each image according to Eq. 6. Notice that we don't sum all the scores according to the dimension of each active learning, but compute the average value of all images over round, and normalize the results for visualization. As illustrated in Fig. 5, in both experiments, KMaans dominate the sample selection during the initial rounds, such as the first round, but be de- de-priori in subsequent rounds. This indicates that diversity- based measures, which select samples that represent diverse regions of the data distribution, are more critical in the early stages of active learning. In the early rounds, when only a small percentage of the data has been labeled, the model's understanding of the entire data distri- bution is limited. As a result, selecting samples that cover broad spectrum of the dataset becomes crucial provide the model with more comprehensive view of the data.
5. Conclusion
In this paper, have introduced AutoAL, the first auto- mated search framework for deep active learning. AutoAL employs two neural networks, SearchNet and FitNet, inte- grated into a bilevel optimization framework to enable joint optimizations. To efficiently solve the bi-level optimization problem, have proposed a probabilistic query strategy that relaxes the search space from discrete continuous, enabling differentiable and data-driven learning paradigm for AutoAL. This framework not only accelerates the search process but also ensures that AutoAL can generalize across a wide range of tasks and datasets. Furthermore, AutoAL. flexible, allowing for easy integration of most of the existing active learning strategies into the candidate pool, making adaptable evolving active learning techniques. Our ex- tensive empirical evaluation, conducted across both natural and medical image datasets, has demonstrated the robust- ness and generalizability of AutoAL. The results show that
On Cifar- 100 dataset, both Least Confidence and MeanSTD, two uncertainty-basee measures, demonstrate improved per- formance in the final rounds. AutoAL assigns them higher priority to enhance overall performance. As the AL pro-
8

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
AutoAL consistently outperforms baselines, highlighting its capability to optimize sample selection and enhance model performance. In future, we plan to apply AutoAL to boarder machine learning tasks such as structured prediction and consider more sophisticated AL method combinations.
Citovsky, G., DeSalvo, G., Gentile, C., Karydas, L., Ra- jagopalan, A., Rostamizadeh, A., and Kumar, S. Batch active learning at scale. Advances in Neural Information 2021.
Processing Systems, 34: 1
Cohn, D. A., Ghahramani, Z., and Jordan, M. I. Active learning with statistical models. Journal of artificial intelligence research, 4: 4:129-145, 1996.
Acknowledgements
The authors acknowledge research support from Clemson University with generous allotment of computation time on the Palmetto cluster.
Finn, C., Abbeel, P., and Levine, Model-agnostic meta- learning for fast adaptation of deep networks. In Interna- tional conference machine learning, pp. 1126-1135. PMLR, 2017.
Impact Statement
Freeman, L. C. Elementary applied statistics: for students in behavioral science. (No Title,, 1965.
This paper presents work whose goal advance the field of Machine Learning by introducing a novel automated search framework for deep active learning. Our work has many potential societal consequences, including reducing the annotation cost of training deep learning models and improving the efficiency of data annotation, etc. While our proposed method has the potential to promote decision- making in various downstream tasks (e-g., natural and medi- cal image data selection), we acknowledge that our AutoAL has been carefully designed to mitigate potential biases and ensure fair selection.
Gal, Y., Islam, R., and Ghahramani, Z. Deep bayesian active learning with image data, In International conference on machine learning, pp. 1183-1192. PMLR, 2017.
Hacohen, G. and Weinshall, D. How to select which active learning strategy is best suited for your specific problem and budget. Advances in Neural Information Processing Systems, 36, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual lear- ing for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
References
Ahmed, M. Seraj, R., and Islam, S. M. S. The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 988):1295, 2020.
Hsu, W.-N. and Lin, H.-T. Active learning by learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 29, 2015.
Ash, J. T., Zhang, C., Krishnamurthy, A., Langford, J., and Agarwal, A. Deep batch active learning by di- verse, uncertain gradient lower bounds. arXiv preprint arXiv: 1906009906 2019.
Hu, Y., Wu, X., and He, R. Tf-nas: Rethinking three search differentiable neural ar-
freedoms of latency- chitecture search. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XV 16, pp. 123-139. Springer, 2020.
Baram, Y., Yaniv, R. E., and Luz, Online choice of active learning algorithms. Journal of Machine Learning 2004.
Research, 5(Mar):255
Kampffmeyer, M., Salberg, A.-B., and Jenssen, R. Se- mantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion workshops, pp. 1-9, 2016.
Beluch, W. H., Genewein, T., Nürnberger, A., and Köhler, J. M. The power of ensembles for active learning in image classification. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9368-9377, 2018.
Chen, C., Chen, X., Ma, C., Liu, Z., and Liu, X. Gradient- based bi-level optimization for deep learning: A survey. arXiv preprint orXiv:2207.1 1719, 2022.
Kingma, D. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.66900. 2014.
Koh, P. W. and Liang, P. Understanding black-box predic- tions via influence functions. In International conference on machine learning, pp. 1885-1894. PMLR, 2017.
Chen, Y., Chen, B., He, X., Gao, C., Li, Y., Lou, J.-G., and Wang, Y. Aopt: Learn to regularize recommender models in finer levels. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 978-986. 986, 2019.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
9

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Le, Y. and Yang. X. Tiny imagenet visual recognition chal- lenge. CS 23IN, 7(7):3, 2015.
Shannon, C. E. A mathematical theory of communication.
The Bell system technical journal, 27(3):379-44 -23,, 1948.
Liu, H., Simonyan, K., and Yang, Y. Darts: Differentiable architecture search, arXiv preprint arXiv:/806.0903 09055, 2018.
Sinha, S., Ebrahimi, S., and Darrell, T. Variational adver-
sarial active learning. In Proceedings of the IEEE/CVF international conference on computer vision, pp. pp. 5972- 1981,20199
Liu, R., Gao, J., Zhang, J., Meng, D., and Lin, Z. Investigat- ing bi-level optimization for learning and vision from a unified perspective: survey and beyond. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 44 (12):10045-10067, 2021.
Wang. and Shang, Y. A new active labeling method for deep learning. In 2014 International joint conference on neural networks (IJCNN) pp- 1 12-119. IEEE, 2014.
Wang. Y., Tu, 7.,, Xiang, Y., Zhou, S., Chen, X., Li, B., and Zhang, Rapid image labeling via neuro-symbolic learning. In Proceedings of the 29th ACM SIGKDD Con- ference Knowledge Discovery and Data Mining, pp. 2467-2477, 2023.
MacKay, M., Vicol, P., Lorraine, J., Duvenaud, D., and Grosse, R. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. arXiv preprint arXiv:1903 00088,3888, 2019.
Xie, Y.. Tomizuka, M.. and Zhan, W. Towards gen- eral and efficient active learning. arXiv preprint arXiv:21 2021.
Mukhoti, J., Kirsch, A., van Amersfoort, J., Torr, P. H., and Gal, Y. Deep deterministic uncertainty: A new simple baseline. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24384- 24394, 2023.
Yang. J., Shi, R., Wei, D., Liu, Z., Zhao, L.,, Ke, B., Pfister,
H., and Ni, B. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng. A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 4. Granada, 2011.
Yoo, D. and Kweon, I. S. Learning loss for active learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 93-102, 2019.
Pang, K., Dong, M. Wu, Y., and Hospedales, T. M. Dy- namic ensemble active learning: A non-stationary bandit with expert advice. In 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2269-2276. IEFF, 2018.
Zhan, X., Li, Q., and Chan, A. B. multiple-criteriy based active learning with fixed-size determinantal point pro- cesses. arXiv preprint arXiv:2107.01662. 2021a.
Zhan, X., Liu, H., Li, Q., and Chan, A. B. comparative survey: Benchmarking for pool-based active learning. In IJCAI, pp. 4679-4686, 2021b.
Park, D., Shin, Y., Bang. J., Lee, Y., Song, H., and Lee, J.-G. Meta-query-net Resolving purityiinformativensss dilemma in open-set active learning. Advances in Neural Information Processing Systems, 35:31416-3142,, 2022.
Zhan, X., Wang, Q. Huang, K.-h., Xiong, H., Dou, D., and Chan, A. B. A comparative survey of deep active learning. arXiv preprint 2rXhv:2203.13450 2022.
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., and Wang, x. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1-40, 2021.
Zhang, J.,, Shao, S., Verma, S., and Nowak, R. Algorithm se- lection for deep active learning with imbalanced datasets. Advances in Neural Information Processing Systems, 36, 2024.
Reynolds, D. A. et al. Gaussian mixture models. Encyclo- pedia of biometrics, 741(659-663) 2009.
Zhang, Z. and Sabuncu, M. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018.
Ruder, S. An overview of gradient descent optimization algorithms. arXiv preprint arXiv: 1609.04477 2016.
Zhao, J., Sun, S., Wang, H., and Cao, Z. Promoting active learning with mixtures of gaussian processes. Knowledge- Based Systems, 188: 105044, 2020.
Sener, O. and Savarese, S. Active learning for convolutional neural networks: A core-set approach. arXiv preprint 1eXiv:1708.0040 2017.
Zhu, J.J.. and Bento, J. Generative adversarial active learn- ing. arXiv preprint 1722v: 2017.
Settles, B. Active learning literature survey. 2009.
10

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
A. Appendix
Table 1. A summarization of datasets used the experiments. The Imbalance Ratio (IR) is the ratio of the number of images in the majority class the numher of images in the minority class. IR represents balance dataset.
Dataßet
Training Size
Test Size
Class Numbers
Imbalance Ratio
Cifar-10
50,000
10,000
1.0
Nature Images
Cifar-100
50,000
10,000
100
1.0
SVHN
73,257
26,032
10
3.0
TinyInageNett
100,000
10,000
200
1.0
OrganCMNIST
12,975
8,216
11
5.0
Medical Images
PuthNNST
89,996
7,180
1.6
TisSueNNIST
165,466
47,280
8
9.1
A.1. Complexity Analysis
Table average runtime of entire active learning (AL) process (including training and querying), where runtime of Entropy- Sampling is the unit time. We separate total time cost of AutoAL into three main parts: candidate ALs querying AL score for images (AutoAI AllrAAAeCCommeellSSaa Score), SearchNet and FitNet updating AutoAA--Search) and querying samples and training classification model
after AutoAI. search (AutoAll-TranReeNNeet)
CIFAR-I0
CIFAR-100
SVHIN
TinyImageNet
OrganCMNIST
PathMNIST
TissueMNIST
SampopyS
1.00
1.00
1.00
1.00
1.00
1.00
1.00
MarginSampling
0.99
1.02
5.97
0.99
2.01
1.01
1.84
LeastConfidence
0.85
1.01
1.03
0.96
1.85
0.97
1.81
KMeansSampling
1,20
1.24
1.59
1.33
0.93
2.37
MeanSTD
1.46
1.18
0.89
0.23
2.07
0,85
2.35
VarRatio
1.40
1.17
0.87
0.87
1.87
0.78
2.30
BALD
1,48
1.18
0.90
0.92
2.04
1.02
1.86
BadgeSampling
1.83
2.34
1.21
1.49
2.52
LPL.
2.03
0.87
0.98
0.37
1.87
1.35
1.77
ALBL
1.08
0.57
1.01
0.82
2.10
0.88
1.42
DDU
1.48
1.87
2.13
1.62
1.32
2.32
1.68
Coreset
1.68
1.42
1.07
1.34
1.25
1.47
2.05
VAAL
1.24
1.27
7.02
2.05
5.28
4.36
7.68
ENSvarR
4.21
4.52
6.58
12.27
8.33
7.63
10.46
AutALL-CoppteALSSom
2.34
2.03
2,80
3.40
3.79
3.24
1.74
AutoAL-Search
0.12
0.15
0.18
0.20
0.17
0.09
AutoAL-TrainResNet
1.57
1.36
1.88
2.28
2.55
2.18
1.15
AutoAL-Total
4.03
3.50
6.54
5.59
2.95
In this subsection, we perform additional analysis experiments showcase AutoAL's efficiency and generalizability. We mainly the average running time verify the results, and all the experiments was done Nvidia A100 GPU. We use the running time of EntropySampling as benchmark and calculate other ALs running costs relative to it. We find:
1. Our AutoAL demonstrates its ability to generalize to large datasets, and the time cost compatible to some of the baselines such as ENSvarR.
2. We separate the total time cost of method into three main parts, and the AL strategy sampling will cost the most time and the update of SearchNet and FitNet will nearly cost none. This means AutoAL will time cost rely heavily on the candidate pool. When the candidate ALs are fast enough, AutoAL will be in low cost and complexity.
3. We conducted the experiment on changing the candidate pool size, and we found that when there are candidates, the time cost just 1.3x compared to EntropySampling, and our method with three candidates can also gain a high

---

AutoAL: Automated Active Learning with Differentiable Query Strategy Search
improvement on labeling accuracy. This will give real-world users flexibility in choosing the number of candidate ALs considering both accuracy and complexity.
12


Reviews:
[{"review_id": "mhSnotHJLx", "reviewer": "Reviewer_pp3d", "summary": "This paper proposes a new method for active selection that leverages existing AL algorithms as constituent agents. It consists of two neural networks, fitnet and searchnet, each trained using the pool of data that has already been labeled. searchnet is fit to select the best ament a set of pre-chosen active learning algorithms, and fitnet, which is used to judge the usefulness of each unlabeled samples and helps guide searchnet. The two are trained in a bilevel optimization framework that’s partly enabled by some additional machinery that makes the learning objective for both models differentiable. The fully trained models are used to guide data selection.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and time. We appreciate that you confirm our contribution and appreciate our work.\n\nFor your questions, we add more experiment results for your reference:\n**Q1:** Experiments are only done on image data and with resnets. I wonder how contingent the performance of the approach is on the convolution inductive bias—it would be useful to see some experiments with more naive models, like MLPs. Along these lines, I’ve seen other papers show results for different acquisition batch sizes, which I feel would also give more clarity to how the approach performs.\n**Q2:** Experiments only show the earliest parts of learning curves. The results look very promising, but it would be nice to see asymptotic performance.\n\n**A1 and A2:** To answer your questions, we conduct new experiments to show the results, especially on CIFAR10 dataset, as it's commonly used by other related works [1,2]. Speicially, we changed the acquisition batch size from 1000 (original setting of our paper) to 5000 to show the difference. Then we compare the results between our proposed AutoAL with two baselines, KMeansSampling [3] and LPL [4], which are the worst and best performed baseline in our original CIFAR 10 experiments. The results show that our method can still outperform the baselines. Also using a basic learning is a normal setting which has been used in published works [1,4], but we plan to add the experiments for MLPs in the future.\n\n\n| Labeled dataset | KMeansSampling | LPL        | AutoAL     |\n| --------------- | -------------- | ---------- | ---------- |\n| 10000           | 74.3 ± 0.1     | 72.6 ± 0.2 | 74.2± 0.1  |\n| 15000           | 77.2 ± 0.1     | 78.7 ± 0.1 | 78.8 ± 0.1 |\n| 20000           | 80.4 ± 0.0     | 82.2 ± 0.1 | 82.7 ± 0.0 |\n| 25000           | 82.3 ± 0.1     | 85.0 ± 0.2 | 85.3 ± 0.1 |\n| 30000           | 84.2 ± 0.1     | 87.8 ± 0.1 | 88.4 ± 0.1 |\n| 35000           | 85.6 ± 0.2     | 88.6 ± 0.2 | 89.3 ± 0.1 |\n| 40000           | 86.1 ± 0.1     | 89.8 ± 0.1 | 90.6 ± 0.0 |\n\n\n**Q3:** This is a weak point, but calling the method AutoAL seems very general, as other techniques, such as ALBL, also do what I think I’d classify superficially as “automated active learning” in the same way. I'd consider switching to something more descriptive of this technique in particular.\n\n**A3:** Thanks for your suggestions. We plan to change the name AutoAL to DDAL, representing **D**ifferentiable **D**eep **A**ctive **L**earning in the final version.\n\n**References:**\n[1] Sener, Ozan, and Silvio Savarese. \"Active learning for convolutional neural networks: A core-set approach.\" arXiv preprint arXiv:1708.00489 (2017).\n\n[2] Hacohen, Guy, and Daphna Weinshall. \"How to select which active learning strategy is best suited for your specific problem and budget.\" Advances in Neural Information Processing Systems 36 (2023): 13395-13407.\n\n[3] Ahmed, Mohiuddin, Raihan Seraj, and Syed Mohammed Shamsul Islam. \"The k-means algorithm: A comprehensive survey and performance evaluation.\" Electronics 9.8 (2020): 1295.\n\n[4] Yoo, Donggeun, and In So Kweon. \"Learning loss for active learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.", "claims_and_evidence": "The authors claim that this is an effective approach to data selection, and they provide experiments on image data demonstrating that this seems to be the case.", "methods_and_evaluation_criteria": "The criteria make sense for the problem at hand, but it would have been nice to have seen more variance in the model architecture (they just used resnets), batch size, and data type (currently just images).", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "The experimental design is typical for pool-based active learning, consisting of labeled data, unlabeled data, and a sequential learning algorithm. The agent selects unlabeled points it believes are most productive for learning, they're labeled by an oracle, placed in the labeled pool, and the classifier is updated. The goal is to have the highest performing model possible given a fixed labeling budget.", "relation_to_broader_scientific_literature": "The key contribution is in a straightforward and effective way to do unlabeled data selection in a fashion that aggregates existing method. Such an approach can be expensive, as the constituent algorithms can be expensive on their own, but I believe there is a growing need for effective data selection strategies.", "essential_references_not_discussed": "Active learning is a very large field, and people have been thinking about it for a long time, but I think limiting the related work to the few mentioned contemporary papers to be acceptable. I wouldn't say anything truly essential has been omitted.", "other_strengths_and_weaknesses": "The provided results seem compelling, but here are a few notes that I feel would make the paper stronger:\n\n- Experiments are only done on image data and with resnets. I wonder how contingent the performance of the approach is on the convolution inductive bias—it would be useful to see some experiments with more naive models, like MLPs. Along these lines, I’ve seen other papers show results for different acquisition batch sizes, which I feel would also give more clarity to how the approach performs. \n\n- Experiments only show the earliest parts of learning curves. The results look very promising, but it would be nice to see asymptotic performance.", "other_comments_or_suggestions": "This is a weak point, but calling the method AutoAL seems very general, as other techniques, such as ALBL, also do what I think I’d classify superficially as “automated active learning” in the same way. I'd consider switching to something more descriptive of this technique in particular.", "questions_for_authors": "As mentioned above, questions are around the robustness of the approach to other data types, batch sizes, and model architectures.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zMzdTy5aXS", "reviewer": "Reviewer_khE5", "summary": "The proposed AutoAL is an automatic query strategy search algorithm that utilizes bi-level optimization framework to select optimal AL strategies built upon existing uncertainty and diversity-based approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "hank you for your valuable feedback. We appreciate that you confirm our method development, promising performance, and fluent paper writing.\n\nFor your concerns, we make the following comments to clarify our points:\n\n**Essential References Not Discussed:** We thank for your efforts in finding these related materials for us. We plan to add them to the related works. Especially for [1], it contains a lot of datasets, which worth trying.\n\nWe plan to add one sentence to related works part, in *adaptive  sample selection in AL* section, [1] frame the algorithm selection task as \"learning how to actively learn\" and use deep Q-learning algorithm to select which strategy is best suited.\n\n\n**Q1:** The proposed AutoAL does not seem novel. It utilizes the same uncertainty and diversity-based criterion to select batches of samples.\n\n**A1:** Our contribution creates a differentiable bridge connecting \"search\" and \"fit\" models in Active Learning, outperforming manual and non-differentiable approaches. While AutoAL builds on existing uncertainty and diversity-based methods, it solves two key problems:\n\n1) Single-criterion AL methods accumulate bias over iterations. AutoAL incorporates multiple methods to reduce this bias.\n2) Criteria that work well in current settings may perform bad in future rounds. For example, BADGE [3] performs poorly with low budgets but excels with increased resources. AutoAL selects the optimal criterion each round to address this issue.\n\nThis integration reduces cumulative bias, with experiments confirming AutoAL outperforms two-stage selection methods like BADGE [3].\n\n\n**Q2:** The average run time of the total AutoAL is significantly higher than most of the baselines used as part of the experiments, perhaps because of the involvement of deep neural nets, SearchNet and FitNet co-optimization in a bi-level optimization structure, whereas the accuracy is relatively 1%-3% better than most of the baselines.\n\n**A2:** Thanks for pointing out the average runtime issue. Please refer to Appendix A.1, where we divided the total runtime into three main parts. The co-optimization runtime is relatively small compared to the time that candidate ALs spend querying scores for images. We believe this cost is acceptable. Our tests used 7 candidate ALs, which isn't always necessary (as shown in Figure 4 for OrganCMNIST). Using fewer candidates would significantly reduce runtime. We plan to integrate AL methods with lower computational complexity to further reduce AutoAL's total runtime.\n\n**Q3:** It is not clear how AutoAL could be used to add more AL frameworks to extend its applications to other domains such as image segmentation, object detection etc.\n\n**A3:** Thanks for your feedback. We plan to test AutoAL on various applications, as our method is task-agnostic. For tasks like image segmentation, whenever candidate AL methods [4,5] can score instances, SearchNet and FitNet will train on these scores. Since most published works [2,6] also only focus on image classification datasets, we believe our results demonstrate AutoAL's superiority in most scenarios.\n\n**Q4:** It would be interesting to see how vision transformer models would work based on this framework.\n\n**A4:** Thanks for your feedback. We want to clarify that using a basic learning is a normal setting which has been used in published works [1,4], however, we plan to add the experiments for transformers in the future to see the performance of AutoAL [6,7].\n\n**Reference:**\n[1] Desreumaux, Louis, and Vincent Lemaire. \"Learning active learning at the crossroads? evaluation and discussion.\" arXiv preprint arXiv:2012.09631 (2020).\n\n[2] Hacohen, Guy, and Daphna Weinshall. \"How to select which active learning strategy is best suited for your specific problem and budget.\" Advances in Neural Information Processing Systems 36 (2023): 13395-13407.\n\n[3] Ash, Jordan T., et al. \"Deep batch active learning by diverse, uncertain gradient lower bounds.\" arXiv preprint arXiv:1906.03671 (2019).\n\n[4] Li, Jun, José M. Bioucas-Dias, and Antonio Plaza. \"Hyperspectral image segmentation using a new Bayesian approach with active learning.\" IEEE Transactions on Geoscience and Remote Sensing 49.10 (2011): 3947-3960.\n\n[5] Yang, Lin, et al. \"Suggestive annotation: A deep active learning framework for biomedical image segmentation.\" Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III 20. Springer International Publishing, 2017.\n\n[6] Sener, Ozan, and Silvio Savarese. \"Active learning for convolutional neural networks: A core-set approach.\" arXiv preprint arXiv:1708.00489 (2017).\n\n[7] Yoo, Donggeun, and In So Kweon. \"Learning loss for active learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.", "claims_and_evidence": "Yes, most of the claims are supported by clear and convincing evidence. The computational overhead is not very convincing considering the performance the proposed method delivers.", "methods_and_evaluation_criteria": "Yes, several baselines are explored with a range of well-known datasets.", "theoretical_claims": "The theoretical claims look fine to me. The differentiable query strategy optimization, the bi-level optimization problem setup look reasonable.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "AutoML is basically an automatic query search algorithm which may help in selecting the optimal AL strategies for various applications.", "essential_references_not_discussed": "1.\tDesreumaux, L., & Lemaire, V. (2020). Learning active learning at the crossroads? Evaluation and discussion. arXiv preprint arXiv:2012.09631.\n2.\tFang, M., Li, Y., & Cohn, T. (2017). Learning how to active learn: A deep reinforcement learning approach. arXiv preprint arXiv:1708.02383.\n3.\tMakili, L. E., Vega Sánchez, J. A., & Dormido-Canto, S.  Active learning using conformal predictors: Application to image classification.\n\nThe approaches in these papers are somewhat similar to the proposed method. These papers could have been used as part of the baselines or references.", "other_strengths_and_weaknesses": "Strengths – \n\nThe proposed framework AutoAL designed on top of uncertainty and diversity based AL approaches, utilizes two neural networks optimized concurrently under bi-level optimization framework to select optimal AL strategies to solve the traditional active learning problem. Overall, the performance looks promising and the paper is easy to follow.\n\nWeaknesses - \n\nThe proposed AutoAL does not seem novel . It utilizes the same uncertainty and diversity-based criterion to select batches of samples. \n\nThe average run time of the total AutoAL is significantly higher than most of the baselines used as part of the experiments, perhaps because of the involvement of deep neural nets, SearchNet and FitNet co-optimization in a bi-level optimization structure, whereas the accuracy is relatively 1%-3% better than most of the baselines.\n\nIt is not clear how AutoAL could be used to add more AL frameworks to extend its applications to other domains such as image segmentation, object detection etc.", "other_comments_or_suggestions": "It would be interesting to see how vision transformer models would work based on this framework.", "questions_for_authors": "Please answer the concerns in the weaknesses section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BAVr4jo2RY", "reviewer": "Reviewer_xZxW", "summary": "This paper addresses the challenge of active learning (AL) by proposing AutoAL, a automated active learning framework. The authors highlight that optimal AL strategies vary across different datasets and problem settings. To address this, AutoAL first extracts scores from multiple acquisition functions and then employs a bi-level optimization approach to identify the most effective acquisition strategy dynamically. The model consists of FitNet and SearchNet, which are trained in a differentiable framework. Specifically, the labeled dataset is partitioned into a pseudo-validation set and a training set, enabling FitNet to fit the training set while SearchNet determines the most informative unlabeled samples for annotation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback.  For your questions, we make the following comments to clarify our points:\n\n**Q1:** **Sampling Bias in Actively Selected Data**\n\n**A1:** For our initial seed dataset setting, it is i.i.d., ensuring an unbiased starting point. Traditional single-criterion AL strategies, especially those based solely on uncertainty [1], are more susceptible to accumulating such bias over iterations. In contrast, AutoAL integrates multiple AL strategies, each contributing different perspectives (uncertainty, diversity, etc.). This fusion, combining multiple AL candidates while maintaining a differentiable framework between search and fit nets, inherently reduces bias, prevents overfitting to any individual AL sampling strategy, and adapts dynamically as the AL process evolves.\n\n**Q2: Baseline implementation Issue**\n\n**A2:** Thanks for the question. We observed this problem too. To clarify, all baseline implementations come from public GitHub repositories, not from our own. Our framework builds upon the open-source deepAL [1]. The accuracy degradation appears in many works, including Figure 1 in [3,4], Figure 3(b) in [5], Figure 3 and 4 in [6], and deepAL [1]. We attribute this to two factors:\n1) The amount of data causing overfitting in the classification model.\n2) Medical datasets often contain redundancy and confusing information. These medical images have many-to-one relationships, as one patient may correspond to several pathology images. Some patients may have multiple conditions (e.g., X-rays showing posterior spinal fixators used for spine repair). These features can influence predictions. Proper diagnosis requires both local and global features, making bias problems more severe in these cases.\n\n**Q3: Writing Issues**\n**Q3.1:** Notion Issues.\n\n**A3.1:** We are sorry that there are many notations in our method parts. We will make a table in the appendix to define the important notions.\n\nWe tried to describe the input for SearchNet and Fitnet in Figure 1. The initialed labeled pool is divided to two queues. The first queue is used to train FitNet. The second queue is used to train both FitNet and SearchNet. Thanks for pointing out, we will polish the writing in our final version.\n\n**Q3.2:** Notions used before defined.\n\n**A3.2:** Our writing logic was to first interpret the whole framework of AutoAL, then the detailed parts such as the loss functions in Equation (1).\n\n**Q3.3:** Figure 1.\n\n**A3.3:** AutoAL's FitNet is first trained on the first queue of labeled dataset. After the parameter of FitNet is adapted to the second training step, FitNet and SearchNet are jointly trained with the second data queue. We will add more explanation to the description in Figure 1.\n\n**Q4: Runtime Complexity**\n\n**A4:** Please first refer to the rebuttal to **Reviewer khE5, A2**. Follow the part 5.1 of pulished work [7], we made a similar complexity analysis. As described in our appendix A.1. let $N_{search}$ denotes the updating time of SearchNet and FitNet in each AL round per batch, $N_{train}$ denotes the sample querying and classification model training in each AL round per batch. We can define the AL score querying time for each candidate AL per batch as $Q_{i}$. When there are $M$ AL algorithms, in each batch, the computation complexity is O(T$\\sum_{b=1}^{B_{max}}(N_{search}+N_{train}+\\sum_{i=1}^{M}Q_{i})$) in the worst case, where T is the total number of rounds and $B_{max}$ is the number of batches depending on the stop criterion of AL. In the future, we plan to do parallel processing for the candidate AL score calculation, which will further improve the upper bound to O(T$\\sum_{b=1}^{B_{max}}(N_{search}+N_{train}+Q_{imax}$))\n\n**Q5: Essential References Not Discussed**\n\n**A5:** Thanks for finding these. About the low budget and high budget setting, we have added new experiments to show the results. Please refer to the reply to **Reviewer bNSN.**\n\nWe hope you can reconsider the final rating.\n\n**Reference:**\n[1] Sharma, Manali, and Mustafa Bilgic. \"Evidence-based uncertainty sampling for active learning.\" Data Mining and Knowledge Discovery 31 (2017): 164-202.\n\n[2] Zhan, Xueying, et al. \"A comparative survey of deep active learning.\" arXiv preprint arXiv:2203.13450 (2022).\n\n[3] Gal, Yarin, Riashat Islam, and Zoubin Ghahramani. \"Deep bayesian active learning with image data.\" International conference on machine learning. PMLR, 2017.\n\n[4] Geifman, Yonatan, and Ran El-Yaniv. \"Deep active learning over the long tail.\" arXiv preprint arXiv:1711.00941 (2017).\n\n[5] Kim, Seong Tae, Farrukh Mushtaq, and Nassir Navab. \"Confident coreset for active learning in medical image analysis.\" arXiv preprint arXiv:2004.02200 (2020).\n\n[6] Mishal, Inbal, and Daphna Weinshall. \"DCoM: Active Learning for All Learners.\" arXiv preprint arXiv:2407.01804 (2024).\n\n[7] Zhang, Jifan, et al. \"Algorithm selection for deep active learning with imbalanced datasets.\" Advances in Neural Information Processing Systems 36 (2023): 9614-9647.", "claims_and_evidence": "1. First Differentiable AL Strategy Selection\n    - To the best of my knowledge, AutoAL is the first differentiable active learning selection method. The use of bi-level optimization for acquisition function selection has been explored in other fields, such as few-shot learning, but it has not been explicitly applied in AL before.\n    - The paper provides empirical evidence across multiple datasets showing that AutoAL consistently outperforms prior AL methods.\n2. Assumption on Sampling Bias in Labeled Data\n    - AutoAL assumes that dividing the labeled set into two subsets can sufficiently approximate the data distribution of the unlabeled pool (Line 173 - 178).\n    - However, active learning is inherently biased, as sampling bias exists in the labeled data [Farquhar’21]. The assumption that the sampled labeled data follows the original data distribution does not hold, which could affect the performance of AutoAL. (See discussion under \"Methods And Evaluation Criteria.\")\n3. Baseline Implementation Concerns\n    - The result shows performance gains across most datasets, but there are concerns about whether the baselines were correctly implemented.\n    - Some prior AL studies indicate that performance may degrade as AL rounds progress, which is observed in Figure 2, particularly for PathMNIST.\n\n\n[Farquhar’21] Farquhar, S., Gal, Y., & Rainforth, T. (2021). On statistical bias in active learning: How and when to fix it. ICML21", "methods_and_evaluation_criteria": "1. Sampling Bias in Actively Selected Data\n    - One major issue with AutoAL is that actively sampled data does not follow the same distribution as the unlabeled pool.\n    - Farquhar et al. (2021) demonstrated that actively sampled data has inherent statistical biases, meaning FitNet and SearchNet do not necessarily generalize well to the full data distribution.\n    - Moreover, since the labeled dataset is halved for training FitNet and SearchNet, this could further change the optimal AL strategy, leading to unexpected selection behaviors.\n2. Other Methodological Concerns\n    - Apart from the issue mentioned above, the design of AutoAL's framework and its evaluation metrics appear reasonable.\n    \n\n[Farquhar’21] Farquhar, S., Gal, Y., & Rainforth, T. (2021). On statistical bias in active learning: How and when to fix it. ICML21", "theoretical_claims": "This work does not introduce new theoretical claims.", "experimental_designs_or_analyses": "1. Potential Issues in Baseline Comparisons\n    - In Figure 2, several baseline methods show performance degradation in later rounds, which is unexpected.\n    - This issue is particularly severe in PathMNIST, where adding more labeled data results in worse accuracy.\n    - Given that the experiments were conducted with three trials and reported variance, such trends contradict intuition.\n    - There is a concern that some baselines may not have been properly implemented, which could unfairly favor AutoAL.\n2. Ablation Studies Provide Useful Insights\n    - Figure 4 (Candidate set size ablation) and Figure 5 (AL strategy score visualization) provide useful explanations regarding how AutoAL selects acquisition strategies dynamically.\n    - The ablation design is clear and informative.", "relation_to_broader_scientific_literature": "1. AutoAL aligns with prior studies that show the best AL strategy depends on budget and dataset properties.\n2. Bi-level optimization has been applied in few-shot learning and hyperparameter tuning, but this is the first work to introduce differentiability into AL strategy search.", "essential_references_not_discussed": "Some recent active learning methods [Mahmood’21, Kim’23, Yehuda’22] are missing. Considering the problem setting [Hacohen’22] needs to be included.\n\n[Mahmood’21] Mahmood, R., Fidler, S., & Law, M. T. (2021). Low budget active learning via Wasserstein distance: An integer programming approach. ICLR 2022.\n\n[Yehuda’22] Yehuda, O., Dekel, A., Hacohen, G., & Weinshall, D. (2022). Active learning through a covering lens. NeurIPS 2022.\n\n[Kim’23] Kim, Y. Y., Cho, Y., Jang, J., Na, B., Kim, Y., Song, K., ... & Moon, I. C. (2023, July). SAAL: Sharpness-aware active learning. In ICML 2023.\n\n[Hacohen’22] Hacohen, G., Dekel, A., & Weinshall, D. (2022). Active learning on a budget: Opposite strategies suit high and low budgets. In ICML 2022.", "other_strengths_and_weaknesses": "1. Clarity Issues in Writing and Notation\n    - The notation is difficult to follow, and the mathematical expressions are not clearly defined.\n    - For example, the inputs and outputs of SearchNet and FitNet (Lines 165 - 178) are not explicitly defined, making the method difficult to understand.\n    - Some notations are used before being defined (e.g., Equation (1) loss terms are not introduced until Equations (7) and (8)). Some notation in Equation (5) is not defined.\n    - Figure 1 is difficult to interpret, as it reuses model illustration with \"After Training\" arrow without clear explanations.\n2. High Computational Cost\n    - AutoAL requires training two networks and running R inner-loop updates.\n    - According to Table 2, AutoAL is up to 7x more expensive than entropy-based sampling.\n    - The paper does not analyze the algorithmic complexity, making it difficult to assess scalability to large-scale datasets like ImageNet.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PHSJN8aRwB", "reviewer": "Reviewer_bNSN", "summary": "The paper proposes an algorithm selection strategy for active learning. The proposed method utilizes the existing labeled dataset and trains differentiable policies for data selection. Experiments are conducted on numerous datasets showing the effectiveness of their algorithm.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We appreciate that you confirm our contribution to the community on solving the strategy selection problem and our method: using differentiable bi-level framework.\n\nFor your concerns, we add more experiments to show the results as below:\n**Q1:** The author should compare against (Hacohen & Weinshall) and (Zhang et al.), and in their own games. (Hacohen & Weinshall) is proposed for different computation budget settings, and (Zhang et al.) is proposed for imbalance. \n\n**A1:** We thank for your insightful comments. We do find we should compare with their works so that it can further show the effectiveness of our proposed AutoAL. To compare with [1], we conducted the experiements on two datasets, CIFAR10 and CIFAR100, which are used in both of our experiments and their experiements. In their settings, the CIFAR10 dataset only has two classes and CIFAR100 with 10 classes respectively. The results are shown as follows:\n\nFor CIFAR-10:\n| AL Labeled datasets | TAILOR     | AutoAL     |\n| ------------------- | ---------- | ---------- |\n| 2000                | 80.2 ± 0.2 | 80.2 ± 0.1 |\n| 3000                | 84.5 ± 0.1 | 85.3 ± 0.2 |\n| 4000                | 87.5 ± 0.2 | 89.4 ± 0.1 |\n| 5000                | 89.4 ± 0.2 | 91.7 ± 0.1 |\n| 6000                | 91.9 ± 0.2 | 92.8 ± 0.2 |\n| 7000                | 93.3 ± 0.1 | 93.5 ± 0.1 |\n| 8000                | 95.1 ± 0.1 | 94.8 ± 0.2 |\n| 9000                | 96.5 ± 0.2 | 96.6 ± 0.1 |\n\nFor CIFAR-100:\n| AL Labeled datasets | TAILOR     | AutoAL     |\n| ------------------- | ---------- | ---------- |\n| 4000                | 37.8 ± 0.3 | 38.2 ± 0.2 |\n| 6000                | 49.8 ± 0.1 | 50.4 ± 0.1 |\n| 8000                | 58.2 ± 0.1 | 62.3 ± 0.2 |\n| 10000                | 65.8 ± 0.2 | 68.7 ± 0.1 |\n| 12000                | 70.9 ± 0.2 | 72.3 ± 0.2 |\n| 14000                | 75.5 ± 0.3 | 78.4 ± 0.2 |\n| 16000                | 80.0 ± 0.2 | 80.2 ± 0.1 |\n| 18000                | 83.2 ± 0.1 | 83.4 ± 0.3 |\n\nWe found that AutoAL can outperform TAILOR in nearly all AL iterations, both in CIFAR-10 and CIFAR-100, this result is consistent with the results shown before because for SVHN and other medical datsets, although they are imbalanced, AutoAL can still outperform the baselines.\n\nFor [2], unfortunately they didn't open-source their code, but we found a toolbox from [3] which officially implemented Probcover [4] and TypiClust [3], two baselines used in [2]. From the experiment results in [2], we observed that Probcover consistently performs best in low budget settings while Badge [5] excels in high budget settings. Therefore, we selected Probcover, TypiClust, and Badge as our baseline methods, focusing our tests on CIFAR-10.\nAs shown in our results, AutoAL performs well in medium and high budget settings but underperforms in low budget scenarios. We attribute this to deep neural networks typically requiring larger datasets for proper training—when the budget is low, SearchNet and FitNet cannot be fully optimized. However, traditional AL settings rarely use extremely low budgets, and AutoAL demonstrates its effectiveness when the budget is adequate. We acknowledge that AutoAL is designed for general active learning scenarios, not specifically for low-budget settings.\n\n| Budget (L+A) | 500+500    | 7k+1k      | 20k+5k     |\n| ------------ | ---------- | ---------- | ---------- |\n| ProbCover    | 50.3 ± 0.4 | 79.8 ± 0.2 | 87.5 ± 0.2 |\n| TypiClust    | 49.8 ± 0.3 | 80.0 ± 0.2 | 87.2 ± 0.1 |\n| Badge        | 50.2 ± 0.4 | 79.9 ± 0.1 | 88.1 ± 0.3 |\n| AutoAL       | 46.8 ± 0.3 | 80.3 ± 0.1 | 89.3 ± 0.1 |\n\n**Essential References Not Discussed:** Thanks for providing the related works. In our initial version, we mainly consider the algorithm selection works with many candidate ALs, but not focused on providing a new AL strategy. However, we believe the works you provided is great and we plan to adds them into our final version.\n\nWe hope these results can address your problems, and reconsider our contributions. We appreciate if you can consider raising your score.\n\n**References:**\n[1] Zhang, Jifan, et al. \"Algorithm selection for deep active learning with imbalanced datasets.\" Advances in Neural Information Processing Systems 36 (2023): 9614-9647.\n\n[2] Hacohen, Guy, and Daphna Weinshall. \"How to select which active learning strategy is best suited for your specific problem and budget.\" Advances in Neural Information Processing Systems 36 (2023): 13395-13407.\n\n[3] Hacohen, Guy, Avihu Dekel, and Daphna Weinshall. \"Active learning on a budget: Opposite strategies suit high and low budgets.\" arXiv preprint arXiv:2202.02794 (2022).\n\n[4] Yehuda, Ofer, et al. \"Active learning through a covering lens.\" Advances in Neural Information Processing Systems 35 (2022): 22354-22367.\n\n[5] Ash, Jordan T., et al. \"Deep batch active learning by diverse, uncertain gradient lower bounds.\" arXiv preprint arXiv:1906.03671 (2019).", "claims_and_evidence": "I did find the claim in this paper to be unsupported by evidence. This is mostly centered around the comparison against existing algorithm selection algorithms. The authors mention\n\"[Compared to Hacohen & Weinshall and Zhang et al.], both the computational cost and the lack of differentiability make the optimization of these works inefficient.\"\n\nI am very confused by this comment as\n1. The proposed bilevel optimization approach seems to be much more computational expensive than both of these existing works. Both of the existing works use simple statistics in choosing algorithms, while this paper uses parameterized neural network classes, which is much more computational costly.\n2. The author mention the lack of differentiability make existing methods inefficient, but never compare to any of them in experiments. There is also no analysis other than this single sentence.", "methods_and_evaluation_criteria": "If the authors are trying to make an argument that their methods is better, the author should compare against (Hacohen & Weinshall) and (Zhang et al.), and in their own games. (Hacohen & Weinshall) is proposed for different computation budget settings, and (Zhang et al.) is proposed for imbalance. There is currently no experiment.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "See sections above.", "relation_to_broader_scientific_literature": "This paper studies the algorithm selection work for active learning.", "essential_references_not_discussed": "I think the paper could benefit from discussing related differentiable policy learning work in active learning. They should also discuss how their parameterization scheme is similar/different from these work.\n\n[1] https://proceedings.neurips.cc/paper_files/paper/2017/file/8ca8da41fe1ebc8d3ca31dc14f5fc56c-Paper.pdf\n\n[2] https://arxiv.org/abs/1909.03585\n\n[3] https://arxiv.org/abs/2010.15382\n\n[4] https://arxiv.org/abs/2108.07670\n\n[5] https://www.jmlr.org/papers/v23/21-0387.html", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "I would recommend the authors position their paper for general active learning scenarios. The two existing literature only study certain scenarios of deep active learning (low-high budget and imbalance). However, I still think it is necessary to compare against these algorithms under their proposed scenarios. If the author's algorithm indeed performs better than those papers in their proposed scenarios, this would be a very influential work. Even if the algorithm does not perform as well, I think the algorithm is still a solid contribution to the community. However, at the same time, the authors also need to note the shortcomings of their approach in such cases.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LcJbUjxxdz", "reviewer": "Reviewer_u8CR", "summary": "This paper addresses the active learning problem. Given the existence of numerous active learning methods for selecting the most informative samples, this work proposes an end-to-end framework that integrates multiple approaches. The framework consists of SearchNet and FitNet:\n\nSearchNet assigns a score to each sample, ranks them, and selects those with the highest loss.\nFitNet models the data distribution within the unlabeled dataset and guides the training of SearchNet.\nHowever, the main contribution requires further clarification. Instead of selecting a single strategy for the dataset, the framework appears to consider all active learning strategies based on the loss function (Equation 6) and Algorithm 1 (Page 5). If all strategies are incorporated, it is not surprising that the proposed method, AutoAL, achieves the best performance in Figure 2. This suggests that the score definition in Equation 6 is well-designed, but contradicts the claim that the framework selects a single strategy for the dataset.\n## update after rebuttal\n\nAdditionally, it is unclear whether FitNet is truly necessary. For instance, Line 157 states that FitNet is trained on unlabeled data, whereas Figure 1 indicates that FitNet is trained and fine-tuned using labeled pool data. This inconsistency raises questions about its necessity. Moreover, the ablation study primarily focuses on SearchNet, which appears to play a more significant role in performance and seems to dominate the results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank for your feedback and the confirmation of our proposed AutoAL! We agree that AutoAL is an end-to-end framework could effectively select the best strategy for a given dataset or task. We also thank for your valuable question, so we want to clarify the followings:\n\n**Q1:** Is there any optimization strategy used to compute the scores fast, as shown in Table A.1 (Line 565)?\n\n**A1:** This is due to our proposed differential bi-level optimization strategy. Actually all the time cost for AutoAL-Search only contains the training time of the two neural networks. We do find AutoAL may need extra time compared with other baselines, however, the main cost is due to the sample query process of the candidate ALs.\n\n**Q2:** Is FitNet truly necessary? Why do the ablation studies not evaluate its impact?\n\n**A2:** Thanks for pointing out this. Because FitNet is the same net as the final classification net, and is used in AutoAL to yield the informativeness of each unlabeled sample, so we thought it's unnecessary to do ablation on it in our original version. However, we added this experiment on CIFAR100 for you to refer to the results:\n\n\n| AL Labeled datasets | AutoAL w/o FitNet | AutoAL     |\n| ------------------- | ----------------- | ---------- |\n| 4000                | 33.1 ± 0.2        | 33.2 ± 0.1 |\n| 6000                | 36.8 ± 0.2        | 39.4 ± 0.1 |\n| 8000                | 40.6 ± 0.1        | 44.1 ± 0.1 |\n| 10000               | 41.5 ± 0.2        | 47.2 ± 0.0 |\n| 12000               | 45.8 ± 0.3        | 50.4 ± 0.1 |\n| 14000               | 47.9 ± 0.2        | 52.5 ± 0.0 |\n| 16000               | 50.1 ± 0.1        | 54.9 ± 0.2 |\n| 18000               | 49.8 ± 0.3        | 56.1 ± 0.1 |\n| 20000               | 53.2 ± 0.1        | 57.0 ± 0.1 |\n\n\n\n**Q3:** Why is FitNet both trained and fine-tuned twice? What is the influence of loss Ls and Lf on the final selection?\n\n**A3:** In our settings, FitNet is first trained on the labeled dataset to yield the informativeness of unlabeled sample, then co-optimized with the SearchNet. For $L_s$, it guides the update of SearchNet, to better decide which AL candidate performs the best in the settings now. For $L_f$, it will guide the FitNet to better draw the distribution of unlabeled dataset.\n\n\n**Q4:** The experiments focus on classification problems—can the framework also be applied to regression tasks?\n\n**A4:** Yes, this is definitely possible. Such as in object detection tasks, as long as the candidate AL strategy can output the most possible coordinates of the target object, AutoAL can learn from it and decide which one is the best coordinate. However, in this paper, we just demonstrate the superior of AutoAL in classification task. We plan to do more tasks as a future work.\n\n**Essential References Not Discussed:** Thanks fo providing the related works. In our initial version, we mainly consider the algorithm selection works with many candidate ALs, but not focused on providing a new AL strategy. However, we believe the works you provided is great and we plan to adds them into our final version.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "This work proposes a new method that incorporates all active learning algorithms for sample selection.", "theoretical_claims": "no theory", "experimental_designs_or_analyses": "The experimental design is well-structured. It compares the proposed method with various active learning strategies and includes an ablation study on SearchNet and the active learning candidate strategies. Additionally, experiments are conducted on multiple datasets for each evaluation.", "relation_to_broader_scientific_literature": "It is particularly interesting for researchers focused on active learning.", "essential_references_not_discussed": "The work in [1] explores the use of active learning strategies in deep learning networks for question answering. It specifically incorporates uncertainty-based active learning strategies into the training process of question answering models.\n\n[1] Uncertainty-Based Active Learning for Reading Comprehension. Jing Wang, Jie Shen, Xiaofei Ma, and Andrew Arnold. Transactions on Machine Learning Research 2022.", "other_strengths_and_weaknesses": "If an end-to-end framework could effectively select the best strategy for a given dataset or task, it would be highly interesting. This work is a pioneering effort in that direction.\n\nHowever, further clarification is needed to demonstrate the necessity of FitNet, as SearchNet alone appears to be sufficient.\n\nAdditionally, according to the table, the time cost of the proposed method is relatively low, which is surprising.", "other_comments_or_suggestions": "no", "questions_for_authors": "1. Is there any optimization strategy used to compute the scores fast, as shown in Table A.1 (Line 565)?\n\n2. Is FitNet truly necessary? Why do the ablation studies not evaluate its impact?\n\n3. Why is FitNet both trained and fine-tuned twice?  What is the influence of loss Ls and Lf on the final selection?\n\n4. The experiments focus on classification problems—can the framework also be applied to regression tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers appreciated the intuitively motivated method to select among active learning strategies and the strong experimental results. However, given that the method lacks theory (e.g., the labeled set is not representative of the data distribution), the method isn't fully contextualized within similar active-learning-policy-learning (a.k.a. "learning to active learn"), and this research area is very well-trodden, I only recommend a weak accept.
Decision: Accept (poster)