Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
Authors: Andreas Kirsch
Sebastian Farquhar
Parmida Atighehchian
Andrew Jesson
Frédéric Branchaud-Charron
Yarin Gal
Venue: Accepted by TMLR ()

Abstract: We examine a simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning. Unlike acquiring the top-K points from the pool set, score- or rank-based sampling takes into account that acquisition scores change as new data are acquired. This simple strategy for adapting standard single-sample acquisition strategies can even perform just as well as compute-intensive state-of-the-art batch acquisition functions, like BatchBALD or BADGE while using orders of magnitude less compute. In addition to providing a practical option for machine learning practitioners, the surprising success of the proposed method in a wide range of experimental settings raises a difficult question for the field: when are these expensive batch acquisition methods pulling their weight?

Published in Transactions on Machine Learning Research (09/2023)
Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
Andreas Kirsch"
andross.kirsch@es.o..coo
OATML, Department of Computer Science, University of Oxford'
Sebastian Farquhar'
sebustiin..arrpaannnrrs.........
OATML, Deparbment of Computer Science, University of Oxford
Parmida Atighehchian
parmi.atg@gmail.com
ServiceNows
Andrew Jesson
fnddre..jesoon@essoo.com
OATML, Department of Computer Science, University of Ouford'
Frédéric Branchaud-Charron NowviceNow!
Yarin Gal
yarin.gal@ccsoor.cccu
OATML, Department of Computer Science, University Onford
Reviewed on OpenReview: https: openreview. net/ net/foruue forum? id- dcuccuuQNNBB
Abstract
We examine simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning. Unlike acquiring the top-K points from the pool set, score- or rank-based sampling takes into account that acquisition scores change as new data are acquired. This simple strategy for adapting standard single-sample acquisition strategies can even perform just as well as compute-intensive state-of-the-art batch acquisition functions, like BatchBALD or BADGE while using orders of magnitude less compute. In addition to providing practical option for machine learning practitioners, the surprising success of the proposed method in wide range of experimental settings raises difficult question for the field: when are these expensive batch acquisition methods pulling their weight?
1 Introduction
Active learning is widely used strategy for efficient learning in settings where unlabelled data are plentiful, but labels are expensive (Atlas et al., 1989; Settles, 2010). For example, labels for medical image data may require highly trained annotators, and when labels are the results of scientific experiments, each one can require months of work. Active learning uses information about unlabelled data and the current state of the model to acquire labels for those samples that are most likely to be informative.
While many acquisition schemes are designed to acquire labels one time (Houlsby al., 2011; Gal al., 2017), recent work has highlighted the importance of batch acquisition (Kirsch et al., 2019; Ash et al., 2020). Acquiring in batch lets us parallelise labelling. For example, in parallel or run more than one experiment at once. Batch acquisition also saves compute as single-point selection also incurs the cost of retraining the model for every new data point.
could hire hundreds of annotators to work
"Joint first authors Work done while there.

---

Published in Transactions on Machine Learning Research (09/2023)
Table 1: Acquisition runtime (in seconds, trials, + s.d.). The stochastic acquisition methods are as fast as top-K, here with BALD scores, and orders of magnitude faster than BADGE or BatchBALD. Synthetic pool set with M 10, 000 pool points with 10 classes. BatchBALD & BALD: 20 parameter samples. In superscript, mean accuracy over acquisition steps from Figure 2 on Repeated-MNIST with 4 repetitions (5
trials).
K
Top-K (BALD) 80%
Stochastic (PowerBALD) 90%
BatchBALD 90%
BADGE 86%
10
0.2 0.0
0.2 0.0
566.0 = 17.4
9.2 - 0.3
100
0.2 + 0.0
0.2 + 0.0
5, 36366.6 95.4
82.1 + 2.5
500
0.2 + 0.0
0.2 + 0.0
29, 984.1 598.7
409.3 + 3.7
Unfortunately, existing performant batch acquisition schemes are computationally expensive (Table 1).
Intuitively, this is because batch acquisition schemes face combinatorial complexity when accounting for the interactions between possible acquisition points. Recent works (Pinsler et al., 2019; Ash et al., 2020; 2021) trade off principled motivation with various approximations to remain tractable. A commonly used, though extreme, heuristic is to take the top-K highest scoring points from acquisition scheme designed to select a single point ("top-K acquisition").
This paper examines a simple baseline for batch active learning that competitive with methods that cost orders of magnitude more across a wide range of experimental contexts. We notice that single-acquisition score methods such as BALD (Houlsby et al., 2011) do not take correlations between the samples into account and thus only act as noisy proxy for future acquisition scores we motivate in Figure 1. This leads to stochastically acquire points following distribution determined by the single-acquisitioon scores instead of acquiring the top-K samples. In sequential decision making, stochastic multi-armed bandits and reinforeement learning, variants of this are also known a Boltzmann exploration (Cesa-Bianchi et al., 2017). We examine other related methods in $4. In deep active learning this simple approach remains under-explored, yet it can match the performance of earlier more complex methods for batch acquisition (e.g., BatchBALD, Kirsch et al. (2019); see in $5) despite being very simple. Indeed, this acquisition scheme has a time complexity of only O(M log K') in the pool size M and acquisition size K, just. like top-K acquisition.
We show empirically that the presented stochastic strategy performs as well as better than top- acquisition with almost identical computational cost on several commonly used acquisition scores, empirically making it a strietly-better batch strategy. Strikingly, the empirical comparisons bet ween this stochastic strategy and the evaluated more complex mothods cast doubt on whether they function as well as claimed. Concretely, in this paper we:
examine a family of three computationally cheap stochastic batch acquisition strategies (softmax, power and soft-rank) the latter two which have not been explored and compared to in detail before;
demonstrate that these strategies are preferable to the commonly used top-K acquisition heuristic; and
identify the failure of two existing batch acquisition strategies to outperform this vastly cheaper and more heuristic strategy.
Outline. In $2, we present active learning notation and commonly used acquisition functions. We propose stochastic extensions in $3, relate them to previous work in $4, and validate them empirically in $5 on various datasets, showing that these extensions are competitive with some much more complex active learning approaches despite being orders of magnitude computationally cheaper. Finally, we further validate some of the underlying theoretical motivation in 56 and discuss limitations in $7.
2

---

Notation. Following Farquhar et al. (2021), we formulate active learning over indices instead over data points. This simplifies the notation. The large, initially fully unlabelled, pool set containing M input points is Published in Transactions on Machine Learning Research (09/2023)
2 Background & Problem Setting
The stochastie approach we examine applies to batch acquisition for active learning in pool-based setting (Settles, 2010) where have access to large unlabelled pool set, but can only label a small subset of the points. The challenge of active learning is to what already know to pick which points to label in the most efficient way. Generally, we want to avoid labelling points similar to those already labelled. In the pool-based active learning setting, we are given a large pool of unlabeled data points and can request labels for only a small subset, the acquisition batch. To formalize this, we first introduce some notation.
D = {2; Jizpool,
(1)
where ZPool = (.....,AM M is the initial full index set. We initialise a training dataset with No randomly selected points from Drool by acquiring their labels, Vi+
Durain {12 3:)) heeuin
(2)
where Zuaan is the index set of Dirain, initially containing No indices between I and M. A model of the predictive distribution, p(y) z), can then be trained on Dirain
2.1 Active Learning
At each acquisition step, we select additional points for which to acquire labels. Although many methods acquire one point at a time (Houlsby et al., 2011; Gal et al., 2017), one can alternatively acquire a whole batch of K examples. An acquisition function a takes Iuain and IPool and returns K indices from I be added to Ztrain We then label those K datapoints and add them to Itrain while making them unavailable from the pool set. That is,
Irrann + Itraln U Ua(Itrain (
(3)
Tpool Ipool
Itrain
(4)
A common way to construct the acquisition function is to define some scoring function, and then select the psint(s) that score the highest.
Probabilistic Model. We assume classification with inputs X, labels Y, and a discriminative classifier ply =)) In the case of Bayesian models, we further assume a subjective probability distribution over the parameters, p(w), and we have p(y x)) (p(w)p((p w,w)]
Two commonly used acquisition scoring functions are Bayesian Active Learning by Disagreement (BALD) (Houlsby et al., 2011) and predictive entropy (Gal et al., 2017):
BALD. Bayesian Active Learning by Disagreement (Houlsby et al., 2011) computes the expected information gain between the predictive distribution and the parameter distribution p(w Diraai) for a Bayesian model. For cach candidate pool index, i, with mutual information, I, and entropy, H, the score
SBALD Itrain) I[;; Ω X = Iy;; Dirain]
H X 3i, Dtrain -E P [[ X x,,
(5)
Entropy. The (predictive) entropy (Gal et al., 2017) does not require Bayesian models, unlike BALD, and performs worse for data with high observation noise Mukhoti et al. (2021). is identical to the first term of the BALD score
Sentropy (i; Itrain) = H|| X x;; Diraiu]
(6)
3

---

Published in Transactions on Machine Learning Research (09/2023)
2.2 Acquisition Functions
These scoring functions were introduced for single-point acquisition:
ag (Iurain) arg max s(i; sii; Itrain.
(7)
iEToool
For deep learning in particular, single-point acquisition is computationally expensive due to retraining the model for every acquired sample. Moreover, it also means that labelling can only happen sequentially instead of in bulk. Thus, single-point acquisition functions were expanded to multi-point acquisition via acquisition batches in batch active learning. The most naive batch acquisition function selects the highest K scoring points
batch (Irrinn K)
Σ Itrain) IC MI|K TE!
arg max
Maximizing this sum is equivalent to taking the top-k scoring points, which camnol account. for the interactions between points in acquisition batch because individual points are scored independently, For example, if the most informative point is duplicated in the pool set, all instances will be acquired, which is likely wasteful when we assume no label noise (see also Figure 1 in Kirsch et al. (2019)).
2.3 Batch Acquisition Functions
Some acquisition functions are explicitly designed for batch acquisition (Kirsch et al., 2019; Pinsler et al., 2019; Ash et al., 2020). They try to account for the interaction between points, which can improve performance relative to simply selecting the top-K scoring points. However, existing methods can be computationally expensive. For example, BatchBALD rarely scales to acquisition sizes of more than 5 10 points due to its long runtime (Kirsch al., 2019), as we evidence in Table
ACS-FW. Pinsler et al. (2019) propose "Active Bayesion CoreSets with Frank- Wolfe optimization" which successively minimizes the distance between the (expected) loss the pool set and the loss on polential training set using a greedy Frank-Wore optimization in a Hilbert space. Either posterior posterior-weighted Fisher inner-product or a posterior-weightee loss inner-product between two samples are used. For non-lincar modols, random projections of the gradients speed up the Fisher inner-product.
BADGE. Ash et al. (2020) propose "Batch Active learning by Diverse Grodient Embeddings": motivates its batch selection approach using k-Determinantal Point Process (Kulesza & Taskar, 2011) based on the (inner product) similarity matrix of the scores (gradients of the log loss) using hard pseudo-labels (the highest probability class according to the model's prediction) for each pool sample. See also Kirsch & Gal (2022) for more detailed analysis. In practice, they use the intialization step of k- k-MEANS++ with Euclidian distances between the scores to select an acquisition batch. BADGE is also computationally expensive as we elaborate in Section 4.
BatchBALD. Kirsch et al. (2019) extend BALD to batch acquisition using the mutual information between the parameter distribution and the joint distribution of the predictions of multiple point in acquistion batch: this mutual information is the expected information gain for full acquistion batch:
BaathhBALD(i1.... ix; Itrnin = I[Y,.... ..., Yx: 2 X1 2i,, ..., Xx Σig Diraing.
Kirsch et al. (2019) greedily construct an acquisition batch by iteratively selecting the next unlabelled pool point that maximizes the joint score with the already selected points. This is 1/e-optimal as the expected information gain is submodular (Krause & Golovin, 2014). They note that their approach is computationally expensive, and they only consider acquisition batches of up to size 10.
3 Method
Sclecting the top-K points at acquisition stop t amounts to the assumption that the informativeness of these points is independent of each other. This leads to the pathology that the most informative pool point is duplicated in the pool set, each instance would be selected (up to the acquisition size). This is clearly wrong.
4

---

Published in Transactions on Machine Learning Research (09/2023)
Table Summary of stochastic acquisition variants. Perturbing the scores s; themselves with E; Gumbel(0; (-1) i.i.d. yields softmax distribution. Log-scores result in power distribution, with sumptions that reasonable for active learning. Using the score-ranking, r finally is robustifying assumption. 3 is included for completeness; we use B == 1 in our experiments-xxcoe for the ablation in $6.1.
Perturbation
Distribution
Probability mass
8; +
Softmax Power Soft-rank
x exp 3s;
log + 6j -log ry + Cj
s r;
Accuracy
0.6
0.7
0.8
0.9
Rank Correlation So
1.0
50
Min Training Set Size
100
0.5
150
PowerBALD
BADGE
200
SoftmaxBALD
Uniform
50 100
150 Acquisition Step n
200
250
SoftrankBALD
BALD
BatchBALD
300
Figure 1: Acquisition scores at individual acquisition step t only loose prozy for later scores at 0). Specifically, the Spearman rank- correlation between acquisition scores on the zerot- th and n'th time-step falls with While top- K acquisition incorrectly implicitly assumes the rank- correlation remains 1, stochastic acquisitions do not. Using Monte-Carlo Dropout BNN trained on MNIST initial 20 points and 73% initial accuracy; score ranks computed over test set.
Figure 2: Performance on Repeated-MNISST MNIST with 4 repetitions (5 trials). Up and to the right is better (7). PowerBALD outperforms (top-K) BALD and BADGE and is on par with BatchBALD. This is despite being orders of magnitude faster. Acquisition size: 10, except for BatchBALD (5). See Figure 10 in the appendix for an ablation study of BADGE's acquisition size, and Figure 11 for comparison of all BALD variants at acquisition size 5.
(here: 0).
Top-K Acquisition Pathologies. Another way to that this is wrong is to think step by step, that is to split the batch acquisition into multiple steps of size 1: We select the top pool sample by acquisition score and retrain the model once for each possible class label for this point. We then compute the averaged acquisition scores on the pool set given each of these models weighted by the original model's probability of each class label. We select the top pool sample using this new (averaged) score, and repeat the process, exponentially branching out as necessary. This equivalent to the joint acquisition batch selection in BatchBALD (Kirsch et al., 2019):
I[Y Yk X1 T,, Xx Tix: Dirain
(10)
K
E
Ep(mn ... wiy (Kg Damin T[Y;; X, = Fiy, X1 = 8ig Y1 Biy,... X1 = Ii,-1; Yi Viy-1 train
J=1
K
TV; = Dirain 1=1
Σ
Σ J=1
(BAL Itrain)
(11)
using the chain rule of the mutual information. We see that the informativeness of the samples will usually not be independent each other. Of course, the acquisition scores for models trained with these additional points will be quite different from the first set scores. After all, the purpose of active learning to add the most informative points-te sints-those that will update the model the most. In contrast, selecting top-K batch using the same scores implicitly assumes that the score ranking will not change due to other points.
5

---

Published in Transactions on Machine Learning Research (09/2023)
Digh
Degh1
Deght
0.7
Digh
Dight
Digit
0.6
Digit
Dight
Digit
Digit
0.1
0.0
Figure 3: BALD scores for 1000 randomly-chosen points from the MNIST dataset (handuritten digits). The points are colour-coded by digit label and sorted by score. The model used for scoring has been trained to 90% accuracy first. If we were to pick the top scoring points (e.g. scores above 0.6), most of them would 8s, even though we can assume that after acquiring the first couple of them, the model would consider them less informative than other available data. Points are slightly jittered on the x-axis by digit label to avoid overlaps.
We provide empirical confirmation in Figure 1 that, in fact, the ranking acquisition scores at step t and + K is decreasingly correlated as K grows when we retrain the model for each acquired point. Figure 3 also illustrates this on MNIST, Moreover, we will see in $6, this effect is the strongest for the most informative points.
Instead, we investigate the use of stochastic sampling as an alternative to top-K acquisition, which implicitly acknowledges the uncertainty within the batch acquisition step using simple noise process model governing how scores change. We motivate and investigate the theory behind this in $6, but given how simple the examined methods are, this theory only obstructs their simplicity. Specifically, we examine three simple stochastic extensions of single-sample scoring functions s(i; Ztrain) that make slightly different assumptions. These methods are compatible with conventional active learning frameworks that typically take the top- highest scoring samples. For example, it is straightforward to adapt entropy, BALD, and other scoring functions for use with these extensions.
Gumbel Noise. These stochastic acquisition extensions assume that future scores differ from the current score by a perturbation. We model the noise distribution of this perturbation as the addition of Gumbel- distributed noise & ~ Gumbel(O; 1), which is frequently used to model the distribution of extrema. At the same time, the choice of Gumbel distribution for the noise also one of mathematical convenience, in the spirit of ftraightforward baseline: the maximum of sets of many other standard distributions, such as the Gaussian distribution, is not analytically tractable. On the other hand, taking the highest-scoring points from distribution perturbed with Gumbel noise is equivalent to sampling from softmax distribution' without replacement. We investigate the choice of Gumbel noise further in $6.
This follows from the Gumbel-Mar trick (Gumbel, 1954; Maddison et al., 2014) and, more specifically, the Gumbel-Top-K trick (Kool et al., 2019). We provide a short proof in appendix BB.2. Expanding on Maddison et al. (2014):
Proposition 3.1. For scores si, i E (I,...,u}, and k and 3 > 0, if we draw e GumbellO; 8-1) independently, then arg togt {s; + e. is an (ordered) sample without replacement from the categorical distribution Categorical(<xp(3- s.) 2,//E, exp(0 i [1, nI).
3 2 0 is a 'coldness' parameter. In the spirit of providing a simple and surprisingly effective basclinc without by perrpaaraeeerss, we fix 8 == for our main experiments. However, to undersland its effect, we examine
Also known Roltzmann GGibbs distribution.

---

Published in Transactions on Machine Learning Research (09/2023)
ablations of B in $6.1. Overall, for 3 00, this distribution will converge towards top-K acquisition; whereas for 3 -> 0, it will converge towards uniform acquisition.
We apply the perturbation to three quantities in the three sampling schemes: the scores themselves, the log scores, and the rank of the scores. Perturbing the log scores assumes that scores are non-negative and uninformative points should be avoided. Perturbing the ranks can be seen as a robustifying assumption that requires the relative scores to be reliable but allows the absolute scores to be unreliable. Table 2 summarizes the three stochastic acquisition variants with their associated sampling distributions, which make slightly different assumptions about the acquisition scores:
Soft-Rank Acquisition. This first variant only relies on the rank order of the scores and makes no assumplions on whether the acquisition scores are meaningful beyond that. It thus uses the least amount of information from the acquisition scores. It only requires the relative score order to be useful and ignores the absolute score values. If the absolute scores provide useful information, we would expect this method to perform worse than the variants below, which make use of the score values. As we will see, this is indeed sometimes the case.
Ranking the scores si;; sii; (train) with descending ranks (riliez such that s(r;; Ztrain) 2 s(r;; Itrin) for T. S ry and smallest rank being I, we sample index i with probability Psoftrank (i) x ry with coldness B. This is invariant to the actual scores. We can draw G Gumbel(O; 3-1) and create perturbed 'rank'
softrank (i; I == - logr; +
(12)
Following Proposition 3.1, taking the top-K K points from softrank is equivalent to sampling without replacement from the rank distribution Psoftrank()) (i).
Softmax Acquisition. The next simplest variant uses the actual scores instead of the ranks. Again, it perturbs the scores by a Gumbel-distrriuteed random variable e, ~ (0
soottmax (train) s(t; Itrain) +
(13)
However, this makes no assumptions about the semantics of the absolute values of the scores: the softmax function is invariant to constants shifts. Hence, the sampling distribution will only depend on the relative scores and not their absolute value.
Power Acquisition. For many scoring functions, the scores are non-negative, and a score close zero means that the sample is not informative in the sense that we do not expect will improve the model we do not want to sample it. This is the case with commonly used score functions such as BALD and entropy. BALD measures the expected information gain. When it is zero for sample, we do not expect anything to be gained from acquiring a label for that sample. Similarly, entropy is upper-bounding BALD, and the same considcration applics. This assumption also holds for other scoring functions such as the standard deviation and variation ratios; see Appendix B.1. To take this into account, the last variant models the future log scores as perturbations of the current log score with Gumbel-distributed noise
power (i; Itrain log s(i; Iurain) + Gg-
(14)
By Proposition 3.1, this is equivalent to sampling from a power distribution
-9
Prower (i)
s(i,
(15)
This may be seen by noting that exp(3 @ log s(i; (train) s(i; Itraiin Importantly, as scores + 0, the (perturbed) log scores + -00 and will have probability mass + 0 assigned. This variant takes the absolute scores into account and avoids data points with score 0.
Given the above considerations, when using BALD, entropy, and other appropriate scoring functions, power acquisition is the most sensible. Thus, we expect it to work best. Indood, we find this be the case in the toy experiment on Sapeated-MNIST) (Kirsch et al., 2019) depicted in Figure 2. However, even soft-rank acquisition often works well in practice compared to top-K acquisition; also appendix $D for a more
7

---

Published in Transactions on Machine Learning Research (09/2023)
in-depth comparison. In the rest of the main paper, we mostly focus on power acquisition. We include results for all methods in &C. In Appendix G, we present the simple implementation ion of the stochastic acquisition variants we use in our experiments.
4 Related Work
Researchers in active learning (Atlas et al., 1989; Settles, 2010) have identified the importance of batch acquisition as well as the failures of lop- acquisition using straightlorvard extensions of single-sample methods in a range of settings including support vector machines (Campbell al., 2000; Schohn & Cohn, 2000; Brinker, 2003; Hoi et al., 2006; Guo & Schuurmans, 2007), GMMs (Azimi et al., 2012), and neural networks (Sener & Savarese, 2018; Kirsch et al., 2019; Ash et al., 2020; Baykal et al., 2021).
Many of these methods aim to introduce structured diversity to batch acquisition that accounts for the interaction of the points acquired in the learning process. In most cases, the computational complexity scales poorly with the acquisition size (K) or pool size (M), for example because of the estimation of joint mutual information (Kirsch et al., 2019); the (KK)) complexity of using a k-means++ initialisation scheme (Ash et al., 2020), which approximates k-DPP-bases Frank-Wolfe optimization (Pinsler et. al., 2019); or the O(M log M) complexity of methods based on K-centre coresets (Sener& & Savarese, 2018) (although heuristics and continuous relaxations can improve this somewhat).. In contrast, we examine simple and efficient stochastic strategies for adapting well-known single-sample acquisition functions to the batch setting, The proposed stochastic strategies are based on observing that acquisition scores would change as new points are added to the acquisition batch and modelling this difference for additional batch samples in the most naive way, using Gumbel noise. The presented stochastic extensions have the same complexity O(M log K) as naive top- batch acquisition, yet outperform it, and they can perform on par with above more complex methods.
batch active learning (Bıyık et al., 2019), or
For multi-armed bandits, Soft Acquisition is also known as Boltzmann exploration (Cesa-Biande et al., 2017). It has also been shown that adding noise to the scores, specifically via Thompson sampling, is effective for choosing informative batches (Kalkanli & Özgür, 2021). Similarly, in reinforcement learning, stochastic prioritisation has been employed as prioritized replay (Schaul et al., 2016) which may be effective for reasons analogous to those motivaling the approach examined in this work.
While stochastic sampling has not been extensively explored for acquisition in deep active learning, most recently it has been used as an auxiliary step in diversity-based active learning methods that rely on clustering as main mechanism (Ash et al., 2020; Citovsky el al., 2021). Kirsch et al. (2019) empirically find that additional noise in the acquisition scores seems to benefit batch acquisition but do not investigate further. Frodlund et al. (2010) suggest modeling single-point acquisition as sampling from a "query density" modulated by the (unknown) sample density p(z) and analyze a binary classification toy problem. Farquhar et al. (2021) propose stochastic acquisition as part of de-biasing actively learned estimators.
Most relevant to this work, and building on Fredlund et al. (2010) and Farquhar et al. (2021), Zhan et al. (2022) propose stochastic acquisition scheme that is asymptotically optimal. They normalize the acquisition scores via the softmax function to obtain a query density function for unlabeled samples and draw an acquisition batch from it, similar to SoftmaxEnntropy Their method aims to achieve asymptotic optimality for active learning processes by mitigating the impact of bias. In contrast, in this work, we examine multiple stochastic acquisition strategies based on score- based or rank- based distributions and apply these strategies to several single-sample acquisition functions, such as BALD and entropy (and standard deviation, variation ratios, see Figure 12); and we focus on active learning in (Bayesian) deep learning setting. As such our empirical results and additional proposed strategies can be seen as complementary to their work.
Thus, while stochastic sampling is generally well-known within acquisition functions, to our knowledge, this work is the first to investigale simple stochastic sampling methods entirely as alternatives to naive top-K acquisition in (Bayesian) deep active learning and to compare them to more complex approaches in various settings.
2A workshop version was presented at ICML. 2021, and the first submission of this work was concurrent Zhan et (2022).
8

---

Published in Transactions on Machine Learning Research (09/2023)
5 Experiments
In this section, we empirically verify that the presented stochastic acquisition methods (a) outperform top- acquisition and (b) are competitive with specially designed batch acquisition schemes like BADGE (Ash et al., 2020) and BatchBALD (Kirsch et al., 2019); and are vastly cheaper than these more complicated methods.
To demonstrate the scriousness of the possible weakness of recent batch acquisition methods, we use a range of datasets. These experiments show that the performance of the stochastic extensions is not dependent on the specific characteristics of any particular dataset. Our experiments include computer vision, natural language processing (NLP), and causal inference (in $6.1). We show that stochastic acquisition helps avoid selecting redundant samples Repeated-MNIST (Kirsch et al., 2019), examine performance in active learning for computer vision on EMNIST (Cohen et al., 2017), MTO-TCD (Tuo et al., 2018), Synbols (Lacoste et al., 2020), and CLINC-150 (Larson et al., 2019) for intent classification in NLP. MIO-TCD is especially close to real-world datasets in size and quality. In appendix $C.5, we further investigate edges cases using the Synbols dataset under different types of biases and noise, and in Appendix C.7, we also separately examine stochastic batch acquisition using last-layer MFVI models on CTFAR-10 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), Repealed-MNISST Pashion-MNIST (Xiao et al., 2017) and compare to ACS-FW (Pinsler al., 2019).
In the main paper, consider BALD as scoring function. We examine predictive entropy on many of these datasets in the appendix. Additionally, other scoring functions are examined on Repeated-MNIST in appendix SC.2.1. Overall, we observe similar results as for BALD.
For the sake of legible figures, we mainly focus on power acquisition in this section, as it fits BALD and entropy best: the scores are non-negative, and zero scores imply uninformative samples. We show that all three methods (power, softmax, softrank) can perform similarly in appendix $D.
We are not always able to compare to BADGE and BatchBALD because of computational limitations of those methods. BatchBALD is computationally infeasible for large acquisition sizes (2 10) because of time constraints, cf. Table 1. When possible, we use BatchBALD with acquisition size 5 as baseline. Note that this gives BatchBAL.D an advantage as it is known to perform better with smaller acquisition sizes (Kirsch et al., 2019). Similarly, BADGE ran out of memory for large dataset sizes, such as EMNIST 'ByMerge' with 814,255 examples, independently of the acquisition size.
Figures interpolate linearly between available points, and show 95% confidence intervals.
Experimental Setup& & Compute. We document the experimental setup and model architectures in detail in appendix SC.1. Our experiments used about 25,000 compute hours on Titan RTX GPUs.
Runtime Mcasurements. We emphasize that the stochastic acquisition strategies are much more com- putationally efficient compared to specialised batch-acquisition approaches like BADGE and BatchBALD. Runtimes, shown in Table 1, are essentially identical for top-K and the stochastic versions. Both are orders of magnitude faster than BADGE and BatchBALD even for small batches. Unlike those methods, stochastic acquisition scales linearly in pool size and logarithmically in acquisition size. Runtime numbers do not include the cost of retraining models (identical in each case). The runtimes for top- K and stochastic acquisition appear constant over K because the execution time is dominated by fixed-cost memory operations. The synthetic dataset used for benchmarking has 4,096 features, 10 classes, and 10,000 pool points.
Repeated-MNIST. Repeatedd-MNIST (Kirsch et al., 2019) duplicates MNIST specified number of times and adds Gaussian noise to prevent perfect duplicates. Rodundant data are incredibly common in industrial applications but are usually removed from standard benchmark datasets. The controlled redundancies in the dataset allow to showcase pathologies in batch acquisiiion methods. We use an acquisition size of 10 and 4 dataset repetitions.
Figure 2 shows that PowerBALD outperforms top-K BALD. While much cheaper computationally, ef. Table 1, PowerBALE also outperforms BADGE and even performs on par with BatchBALD. We use an acquisition size of 10 for all methods, except for BatchBALD, for which we use an acquisition size of 5, as explained above. Note that BatchBALD performs better for smaller acquisition sizes while BADGE (counerintuitively) can perform better for larger ones; see Figure 10 in the appendix for an ablation. BatchBALDD BALD, and
9

---

Published in Transactions on Machine Learning Research (09/2023)
Accuracy
Accuracy
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.2
0.4
0.6
Min Training Set Size
Min Training Set Size
50
50
100
100
150
PowerBALD
150
SoftmaxBALD
PowerBALD
200
SoftrankBALD
200
SoftmaxBALD
BatchBALD 5
SoftrankBALD
250
BADGE
250
BatchBALD
300
300
(a) EMNIST 'Balanced' (5 trials)
(b) EMNIST 'Bygerge' (5 trials)
Accuracy
Accuracy
0.96
0.97
0.98
0.99
0.80
0.85
0.90
0.95
Min Training Set Size
Min Training Set Size
2500
2500
5000
5000
7500
PowerBALD
7500
PowerBALD
SoftmaxBALD
SoftmaxBALD
10000
SoftrankBALD
10000
SoftrankBALD
BADGE
BADGE
12500
BALD
12500
BALD
Uniform
Uniform
15000
15000
(c) MIO-TCD (3 trials)
(d) Synbols with minority groups (10 trials)
Figure 4: Performance on various datasels. BatchBALD took infeasibly long on these datasets & acquisition
sizes. (a) EMNIST 'Balanced': On 132k samples, PoWerBALD (acq. size 10) outperforms BatchBALD (acq- size 5) and BADGE (aeq. size 10). (b) EMNIST 'ByMerge': On 814k samples, PowerBALD (acq. size 10) outperforms BatchBALD (acq. size 5). BADGE (not shown) OO,'ed, and BatchBALD took > 12 days for 115 acquisitions. (c) MIO-TCD: PoWerBALD performs better than BALD and on par with BADGE (all acq.
size 100). (d) Synbols with minorily groups: PowerBALD performs on par with BADGE (all acq. size 100).
the stochastic variants all become equivalent for acquisition size when points are acquired individually, which performs best Kirsch et al. (2019).
Computer Vision: EMNIST. EMNIST (Cohen et al., 2017) contains handwritten digits and letters and comes with several splits: cxamine the 'Balanced' split with 131,600 samples in Figure 4a² and the 'ByMerge' split with 814,255 samples in Figure 4b. Both have 47 classes. We use an acquisition size of 5 for BatchBALDD, and 10 otherwise.
We see that the stochastic methods outperform BatchBALD on it and both BADGE and BatchBALD on 'Balanced' (Figure 4a). They do not have any issues with the huge pool set in 'ByMerge' (Figure 4b). In the appendix, Figures 27 and show results for all three stochastic extensions, and Figure 17 shows an ablation of different acquisition sizes for BADGE. For 'ByMerge', BADGE ran out of memory on our machines, and BatchBALD took more than 12 days for 115 acquisitions when we halted execution.
Computer Vision: MIO-TCD. The Miovision Traffic Camera Datasct (MIO-TCD) (Luo et al., 2018) is a vehicle classification and localisation dataset with 648,959 images designed to exhibit realistic data characteristics like class imbalance, duplicate data, compression artefacts, varying resolution (between 100 and 2,000 pixels), and uninformative examples; see Figure 9 in the appendix. As depicted in Figure 4c, PowerBALD performs better than BALD and essentially matches BADGE despite being much cheaper compute. We use an acquisition size of 100 for all methods.
Thhis result exactly reproduces BatchBAL.Dss trajectory in Figure from Kirsch al. (2019).
10

---

Published in Transactions on Machine Learning Research (09/2023)
Computer Vision: Synbols. Synbols (Lacoste et al., 2020) is character dataset generator which can demonstrate the behaviour of batch active learning under various edge cases (Lacoste et al., 2020; Branchaud- Charron et al., 2021). In Figure 4d, we evaluate PowerBALD on a dataset with minority character types and colours. PowerBALD outperforms BALD and matches BADGE. Further details as well as an examination of the 'spurious correlation' and "missing synbols' edge cases (Lacoste et al., 2020; Branchaud- Chhrroonnn et al., 2021) can be found in appendix $C.5.
Natural Language Processing: CLINC-150. We perform intent classification on CLINC-150 (Larson et al., 2019), which contains 150 intent classes plus an out-of-scope class. This setting captures data seen in production for chatbots. We fine-tune a pretrained DistiBBERT model from HaggingFace (Wolf et al., 2020) on CLINC-150 for 5 epochs with Adam as optimiser. In appendix $C.6, we see that PowerEntropy shows strong performance: it performs better than Entropy and almost on par with BADGE. This demonstrates that our technique is domain independent and can be easily reused for other tasks.
MFVI Last-Layer Comparison with ACS-FW. In Appendix C.7, we provide a comparison of the proposed stochastic acquisition functions with BALD and ACS-FW (Pinsler et al., 2019) in a Bayesian last-layer setting using variational inference with mean-field Gaussian approximation (instead of Monte-Carlo dropout). We find that for smaller acquisition sizes, the stochastic acquisition functions also outperform BALD on Fashion-MNIST and Repeated-MNIST. For larger acquisition sizes on CIFAR-10 and SVHN, this is not the case, however: no single method seems to perform better than the others. However, we find that ACW-FW overall takes much longer to run than stochastic and top- acquisition (Figure 25). We leave a more thorough investigation of this setting to future work.
In Summary. We have verified that stochastic acquisition functions outperform top- batch acquisition in several settings and perform on par with more complex methods such BADGE or BatchBALD. Moreover, we refer the reader to Jesson et al. (2021), Murray et al. (2021), Tigas et al. (2022), Holmes et al. (2022), Malik et al. (2023), and Rubashevskii et al. (2023) for additional works that use the proposed stochastic acquisition functions in this paper and provide further empirical validation.
6 Further Investigations
In this section, we examine and validate assumptions about the underlying score dynamics by examining the scores across acquisitions. We further hypothesise about when top-K acquisition is the most detrimental to active learning.
Why Gumbel Noise? Intuitively, to select the k-th point in the acquisition batch, we want to take into account how much additional information (increase in acquisition scores) the still-to-be-seleette additional K K- points will provide. As such want to model the maximum over all possible additional candidate points that are still to be selected to complete the acquisition batch. Empirically, acquisition scores are similar to a truncated exponential distribution ('80/20' rule) as visualized in Figures 3 and 7c. Note that this is a rough approximation wwe do not claim that the distribution of acquisition scores really truncated exponential. sum of i.i.d. exponential variables follows the Erlang distribution (which is a special case of the Gamma distribution) and has an exponential tail. We know that the maximum of set of i.i.d. random variables that follow an exponential distribution or distribution which has an exponential tail is known to be well approximated by Gumbel distribution in the sample limit (Gumbel, 1954)*, and thus the maximum over sums of such random variables also follows a Gumbel distribution. Concretely, we can use the following result from Garg et al. (2023):
Theorem 6.1 (Extreme Value Theorem (EVT) (Mood, 1950; Fisher & Tippett, 1928)). For i..dd ran- dom variables XI,...,,,, Xn ~ fx, with exponential tails, lim max; (X.) follows the Gumbel (GEV-1) -1) distribution. Furthermore, 9 is maz-stable, i.e. if X2 ~ 9, then max; (X.) ~ G holds.
However, it is unlikely that the increase in acquisition scores can be modeled well i.i.d exponential random variables with the same rate, but the Gumbel approximation also seems to hold empirically for the hypocxponential distribution which is a sum of exponential distributions with different rates. We do not have a proof for this, but we present a numerical simulation in appendix SH.
4 4See also the following Math StackExchange thread.
11

---

Published in Transactions on Machine Learning Research (09/2023)
Overall, this motivates to use Gumbel distribution simple model for the increase in acquisition scores the still-to-be-selected additional K - k points will provide under the modelling assumption that the increase in acquisition scores at each step is exponentially distributed with different rates at each step. Zhan et al. (2022) provide a different analysis for the use of Gumbel noise in the context of active learning as we relate in $4.
Acquisition Asymptotics of Bayesian Models. For well-specified and well-defined Bayesian parametric models, the posterior distribution of the model parameters converges the true parameters the mumber of data points increases (Van der Vaart, 2000).
For such models and assuming that the predictions are independent given the model parameters, the total correlation between the predictions decreases as the number of training points increases, as the posterior distribution of the model parameters becomes more concentrated around the true parameters:
TC[V,,.... YK 01,-.. FK. Derain) 0 as |Dussia oc.
(16)
This can be proved by noting that in the finite data limit, the posterior parameter distribution converges to the true model parameters, and the marginal distribution then factorizes. This means that the predictions become more independent as the number of training points increases and fully independent in the infinite data limit.
The total correlation is defined as:
TCI Ykl T1...., TK. Dirain =
Σ H[Y; xi, Dirainj - H[Y,
YK Il,..., TK, D
(17)
'Batch Entropy'
top-K Eutropy
We can also write the total correlation difference between top-K K BALD and BatchBALD:
TC|. Yx ,1,...,,KK Dirain
Σ
TY x4, Dirain I[,, YR -1,---, TK, D
(18)
BatchBALD
top-K K BALD
As the total correlation converges to 0, the top-K BALD term (first term) becomes equal to the BatchBALD term (the second term on the right side), and the same happens for top- K entropy and 'BatchEntropy', which we could similarly define.
Thus, for well-specified and well-defindd Bayesian parametric models, the top-K acquisition functions will eventually become equivalent to the BatchBALD and 'BatchEntropy'' acquisition functions as the number of training points increases. This tells us that top- acquisition is the most detrimental to active learning in the earlier stages of learning, when the total correlation between the predictions is still high. This is consistent with our empirical results below ('Inereasing Top-K Analysis').
At the same time, as the number of training points increases and the model parameters concentrale, the expected information gain (BALD) also decreases. The mutual information with deterministic variable is always 0, and thus:
12 x, Dirain + 0 as
Ditrann + cc.
(19)
This asymptotic behavior is a trivial but important result, as tells us that the expected information gain (BALD) will eventually become uninformative the number of training points increases and better than random acquisition, and the important question is: when? Given that we only have noisy estimators, this determines until when active learning is of use compared to random acquisition.
Many different active learning methods that are considered non-Bayesian nevertheless approximate the expected information ion gain or the expected predictive information (Kirsch & Gal, 2022; Smith et al., 2023), which is an expected total correlation. Hence, the considerations apply to those methods, too.
Finally, we observe that estimators such as BatchBALD, which utilize Monte-Care samples for parameter approximations, are inherently limited by the logarithm of the total number M of these samples, log M.
12

---

Published in Transactions on Machine Learning Research (09/2023)
Rank Correlation Spãeas
1.00
1.00
0.95
Top
1%
0.75
Bottom
10%
Accuracy
0.90
0.50
100%
Rank Correlation
0.50
0.85
0.25
0.25
0.80
0.00
0.00
0.75
-0.25
-0.25
0
50
100
150
200
0
20
40
60
80
100
120
0
50
100
150
200
Acquisition Step
Acquisitinn Delta Step n
Training Set Size
(a) t=
(b) 100
Figure 5: Rank correlations for BALD scores on MNIST between the Figure 6: Top-K acquisition hurts initial scores and later scores of the top- or bottom-scoring 1%, 10% and less later in training (BALD on 100% of lest points (smoothed with a size-10 Parzen window). Rank- MNIST). At E {20, 100} (blue), orders decorrelate faster for the most informative samples and in the we keep acquiring samples using the early stages of training. The top scorers' ranks anti-correlate after BALD scores from those two steps. roughly 40 (100) acquisitions unlike the bottom ones. Later in training, At 20 (orange), the model per- the acquisition scores stay more strongly correlated. This suggests the forms well for = 20 acquisitions; at acquisition size could be increased later in training.
t = 120 (green), for ss 50; see $6.
This constraint implics that their informativeness can diminish rapidly. Concretely, consider an empirical imtito [[: built using Monte-Carlo samples wi,-. wk-- This is analogous to computing the exact mutual information [[=:[]] with the 'empirical'' random variable Q which uniformly samples from Wkk Given that the discrete mutual information restricted by the entropy of its terms, we have:
[[: = T[: s H[K] = log M.
For instance, BatchBALD employs a greedy approach to select the 1-th acquisition sample in its batch. It does this by maximizing the empirical i[Y; x, Y,-,, Vi, x. Derain] for the subsequent candidate samples denoted by x. We can represent this relationship as:
K Σ
log M 2 iY 0 21 ......,KKKKK Dirrin
1 Zg........ Y1, x1 Dirain].
From the above equation, K grows, the estimated f|Y:; IK, YK-I, Y1, Dirain] approaches zero since it is restricted by log M. For scenario with M = 100 parameter samples (leading to log M = 2), BatchBALD might rapidly lose its informativeness after just two acquisitions in a classification scenario involving 10 categorics. This situation arises if the pool set contains a minimum of two highly diverse, that is uncorrelated, data points with maximum disagreement.
Rank Correlations Across Acquisitions. In Section 3, we made the following assumptions: (1) the acquisition scores 8t step are proxy for scores Sy at step t' t; (2) the larger t' -t is, the worse proxy 8 is for 8/: (3) this effect is the largest for the most informative points.
We demonstrate these empirically by examining the Spearman rank correlation between scores during acquisition. Specifically, we train a model for n steps using BALD as single-point acquisition function. We compare the rank order at each step to the starting rank order at step l. To denoise the rankings across n, we smooth the rank corrclations with a Parzen window of size 10 and to reduce the effect of noise to the rank order, we round all scores to decimal places. This especially removes unimportant rank changes for points with low scores around 0.
Figure 1 shows that acquisition scores become less correlated as more points are acquired. Figure 5a shows this in more detail for the top and bottom 1%, 10% or 100% of scorers of the test set across acquisitions starting at step f = for a model initialised with 20 points. The top-10% scoring points (solid green) quickly become uncorrclated across acquisitions and even become anti-correlated. In contrast, the points overall (solid blue) correlate well over time (although they have a much weaker training signal on average). This result supports all three of our hypotheses.
13

---

Published in Transactions on Machine Learning Research (09/202))
1.0
1,0
12
Step
NUROC tn
1.0
Step 100
0.8
S:
0.8
0.8
BAL
0.6
0.6
Top
Bottom
0.4
0.4
02
50
100
150
200
20
40
60
80
100
120
0.0
Acquisition Della Step n
Acquisition
Step
Sorted Test Samples
(a) t=
(b) = 100
(c) Scores at Step 20 and 100.
Figure AUROCs for BALD scores on MNIST between the initial scores and later scores of the top- or bottom-scoring 1%, 10% and 50% of test points (smoothed with size-10 Parzen window). AUROC between original points as 'ground truth' and later scores predictors. This is equivalent to the probability that the acquisition score at n for a point in t = 0's top or bottom 1%, etc. is larger than points outside. This tells us how likely other points outside the batch have higher acquisition scores. This ignores the ranking of points otherwise. (a, b) Points in the top quantiles are superseded by other points in the top quantiles in the later acquisitions to a large degree. This is much more pronounced early in the training than later. The bottom quantiles are more stable. (c) The overall score distributions steps O, 100 are visualized and the relevant top and bottom quantiles are marked.
At the same time, we see that as training progresses, and we converge towards the best model, the order of scores becomes more stable across acquisitions. In Figure 5b the model begins with 120 points (1 = 100), rather than 20 (t = 0). Here, the most informative points are less likely to change their rank-evt even the top-1% ranks do not become anti-correlated, only decorrelated. Thus, we hypothesise that further in training. we might be able to choose larger K.
We do not display the correlations for bottom 1% of scorers as their scores are close to 0 throughout training and thus noisy and uninformative, see also Figure 7e for this.
Overall, this analysis can be confounded by noisy samples and swaps in rank order between samples with similar scores that are not meaningful in the sense that they would not influence batch acquisition.
Thus, provide a different analysis, we also consider the more direct question in Figure of how likely other samples have higher acquisition scores + n than the top samples from for different quantiles (1%, 10%, 50%) of the test set. As a sanity check, we also examine the bottom quantiles. This is equivalent to computing the AUROC between the original points as 'ground truth' and later scores predictors: for acquisition step n, the AUROC is p(S) t//p/////tto p3 $ Sf. s/.bnttom/top 1- 1-%% with S s.ftop/bottom p% {st+n, : $t,i E top, bootomm p% of {s, Specifically, we set up binary classification with the top or bottom 1%, 10% 50% of the test set as positive and the rest negative. This again helps quantify how much the scores meaningfully change across acquisition steps. These results match the previous ones and provide another validation for the mentioned assumptions.
Increasing Top-K Analysis. Another way to investigate the effect of top- selection is to freeze the acquisition scores during training and then continue single-point 'active learning' as those were the correct. scores. Comparing this the performance of regular active learning with updated single-point scores allows to examine how well earlier scores perform as proxies for later scores. We perform this toy experiment MNIST, showing that freezing scores early on greatly harms performance while doing it later has only a small effect (Figure 6). For frozen scores at training set size of 20 (73% accuracy, t t= 0), the accuracy matches single-acquisition BALD to a training set size of roughly 40 (dashed orange lines) before diverging to a lower level. But when freezing the scores a more accurate model, at a training set size of 120 labels (93% accuracy, = 100), selecting the next fifty points according to those frozen scores performs indistinguishably from by-seb acquisition (dashed green lines). This result shows that top-K acquisition hurts less later in training but can negatively affect performance the beginning of training.
14

---

Published in Transactions on Machine Learning Research (09/2023)
0.80
0.85
Accuracy
0.90
0.95
0.2
0.3
0.4
Accuracy
0.5
0.6
2.5
0-4 Top-K
--1
Min Training Set Size
50
ETEHE
2.0
100
100
1.5
Min Itaniin
150
SoftmsdBALD c
1.0
200
PowerBALD d==
200
BAKD
PowerBALD
250
100
200
300
400
300
300
Training Set Size
(a) Repeated-MNIISTx
(b) EMNIST 'ByMerge'
(c) IHDP
Figure 8: Liffect of changing 3. (a) Repeated-MNISTzd MNISTT4 (5 trials): PowerBALD outperforms BatchBALD
for B 8. (b) EMNIST 'ByMerge' (5 trials): SoftmaxBALD for = 4 performs best. (c) IHDP (400 trials): At high temperature (B 0.1), CausalBALD with power acquisition is like random acquisition. As
the temperature decreasos, the performance improves (lower V(PEHE), surpassing top-K acquisition. Both experiments use an acquisition size of 10.
These observations lead us to ask whether we could dynamically change the acquisition size: with smaller acquisition batches at the beginning and larger ones towards the end of active learning. We leave the exploration of this for future work.
6.1 Ablation: Changing 3
So far, we have set 8 in the spirit of examining a simple bascline without additional hyperparameters. The results above show that this already works well and matches the performance of much more expensive methods, raising questions about their value. In addition, however, tuming 3 may be able to further improve performance. In the following, we show that other values of 3 can yield even higher performance on Repeated- MNIST, EMNIST, and when estimating causal treatment effects; we provide many additional results in Appendix F.
In Figure Sa, we see that for PowerBALD the best-performing value, 3 = 8,even outperforms BatchBALD. Similarly, tuning B can improve the performance of other strategies well we see for SoftmaxBALD EMNIST, where performs better than B = 8 and ,3 = showing that the best 3 depends the datasct.
Repeated-MNIST & EMNIST.
Causal Treatment Effects: Infant Health Development Programme, Active learning for Conditional Average Treatment Effect (CATE) estimation Heckman et al. (1997; 1998); Hahn (1998); Abrevaya et al. (2015) on data from the Infant Health and Development Program (IHDP) estimates the causal effect of treatments on an infant's health from observational data. Statistical estimands of the CATE are obtainable from observational data under certain assumptions. Jesson et al. (2021) show how use active learning acquire data for label-efficient estimation. Among other subtlcties, this prioritiscs the data for which matched untreatee pairs are available.
We follow the experiments of Jesson et al. (2021) on both synthetic data and the semi-syuthetic IHDP dataset (Hill, 2011), a commonly used benchmark for causal effects estimation. In Figure 8e we show that power acquisition performs significantly better than both top-K and uniform acquisition, using an acquisition size of 10 in all cases with further. We provide additional results on semi-synthetic data appendix $F.3. Note that methods such as BADGE and BatchBALD are not well-defined for causal-effect estimation, while stochastic batch acquisition remains applicable and is effective when fine-tuning B.
Performance on these tasks is measured using the expected Precision in Estimation of Heterogeneous Effecc (PEHE) (Hill, 2011) such that VEPEHE = /E[(F(X) - ((((X)))r(X))))) [()))J] (Shalit et al., 2017) where 7 is the estimated CATE and F is CATE (i.e. a form of RMSE).
Limitations. Although we highlight the possibility for future work to adapt 8 to specific datasets or score functions, our aim is not to offer a practical recipe for this to practitioners. Our focus is on showing how even the simplest form of stochastic acquisition already raises questions for some recent more complex methods.
15

---

Published in Transactions on Machine Learning Research (09/2023)
7 Discussion & Conclusion
Our experiments demonstrate that the stochastic sampling approach we have examined is orders of magnitude laster than sophisticated batch-acquisition strategies like BADGE and BatchBALD while retaining comparable performance in settings across computer vision, NLP, and causal inference. Compared to the flawed top- batch acquisition heuristic, it is never worse: we see no reason to continue using top-K acquisition.
Importantly, our work raises serious questions about these current methods. If they fail to outperform such a simple baseline in wide range of settings, do they model the interaction between points sufficiently well? If so, are the scores themselves unreliable? We call on future work in batch active learning to at least demonstrate that it can outperform simple stochastic batch acquisition strategies.
At the same time, this opens doors for improved methods. Although we only put forward a naive model due its computational and mathematical simplicity, future work can explore more sophisticated modelling of the predicted score changes that take the current model and dataset into account. In its simplest form, this might mean choosing the hyperparameter of the acquisition distribution based on the dataset and adapting it online. Our experiments also highlight that the acquisition size could be adapted dynamically, with larger batch sizes acceptable in later acquisition steps.
Acknowledgements
The authors would like to thank their anonymous TMLR reviewers for their kind, constructive and helpful feedback during the review process, which has significantly improved this work. We would also like to thank Freddie Bickford Smith and Tom Rainforth as well as the members of OATML in general for their feedback at various stages of the project. SF is supported by the EPSRC via the Centre for Doctoral Training in Cybersecurity at the University of Oxford as well as Christ Church, University of Oxford. AK is supported by the UK EPSRC CDT in Autonomous Intelligent Machines and Systems (grant reference EP EP (PP/138897/)
References
Jason Abrevaya, Yu-Chin Hsu, and Robert P Lieli. Estimating conditional average treatment effects. Journal of Business & Economic Statistics, 33(4):485-505, 2015.
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurth: John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview net, 2020. URL https://openreviiet net seet/toruuntdderyyi/BBB33
Jordan T. Ash, Surbhi Goel, Akshay Krishnamurhh, and Sham M. Kakade. Gone fishing: Neural active learning with fisher embeddings. In Advances in Neural Information Processing Systems 34: Annual Conference on Neurol Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021.
Parmida. Atighehchian, Frédéric Branchaud- Charrro,, and Alexandre Lacoste. Bayesian active learning for production, a systematic study and a reusable library. arXie preprint arXiv:2006.099) 2020.
Les E. Atlas, David A. Cohn, and Richard E. Ladner. Training connectionist networks with queries and selective sampling. In Advances in Neural Information Processing Systems 2, [NIPS Conference, Denwer, Colorado, USA, November 27-30, 1989], 1989.
Javad Azimi, Alan Fern, Xiaoli Zhang Fern, Glencora Borradaile, and Brent Heeringa. Batch active learning via coordinated matching. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26- - -Jlly 1, 2012. icmlec / Omnipress, 2012. URL http:://icmm.cc//22//ppaaa/ cttpl1//
pdf.
Cenk Baykal, Lucas Liebenwein, Dan Feldman, and Daniela Rus. Low-regret active learning. orXiv preprint arXiv:2104.02822 2021.
16

---

Published in Transactions on Machine Learning Research (09/2023)
Sarah Bird, Miro Dudik, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna. Wallach, and Kathleen Walker. Fairlearn: A toolkit for assessing and improving fairness in AT. Technical Report SUR-TR--220-33 Microsoft, 2020.
Erdem Biyik, Kenneth Wang, Nima Anari, and Dorsa Sadigh. Batch active learning using determinantal point processes. arXiv preprint arXi::1906.0799 2019.
Frédéric Branchaud-Charro, Parmida Atighehchian, Pau Rodríguez, Grace Abuhamad, and Alexandre Lacoste. Can active learning preemptively mitigate fairness issucs? ICLR Workshop on Responsable AT, 2021.
Klaus Brinker. Incorporating diversity in active learning with support vector machines. In Tom Fawcett. and Nina Mishra (eds.), Machine Learning, Proceedings of the Twenlieth International Conference (ICML 2003), August 21-24, 2003, Washington, DC, USA, pp. 59-66. AAAI Press, 2003. URL 00m.m0000-0000pp--111111 php.
aaai.
Colin Campbell, Nello Cristianini, and Alexander Smola. Query learning with large margin classifiers, In Pat Langley (ed.), Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29 July 2, 2000, pp. 111-118. Morgan Kaufmann, 2000.
Nicolò Cesa-Bianchi, Claudio Gentile, Gábor Lugosi, and Gergely Neu. Boltzmann exploration done right. Advances in newral information processing systems, 30, 2017.
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadch, and Sanjiv Kumar. Batch active learning at scale. In Advonces in Neural Information Processing Systems 34: Annuul Conference on Newral Information Processing Systems 2021, NewrIPS 2021, December 6- 6-14, 2021, virtual, 2021.
Gregory Cohen, Saced Afshar, Jonathan Tapson, and André van Schaik. EMNIST: an extension of MNIST to handwritten letters. arXiv preprint arXic:1702.0555 05373, 2017.
Sebastian Farquhar, Yarin Gal, and Tom Rainforth. On statistical bias in active learning: How and when to fix it In Ith International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, Muy 3-7, 2021, 2021.
Ronald Aylmer Fisher and Leonard Henry Caleb Tippett. Limiting forms of the frequency distribution of the largest smallest member of sample. In Mathemalical proceedings of the Cambridge philosophical society. Cambridge University Press, 1928.
Richard Fredlund, Richard M. Everson, and Jonathan E. Fieldsend. A bayesian framework for active learning. In International Joint Conference on Neural Networks, IJCNN 2010, Barcelona, Spain, 18-23 July, 2010, 2010.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NS W,, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1183 1192. PMTR, 2017.
Divyansh Garg, Joey Hejna, Matthieu Geist, and Stefano Ermon. Extreme q-learning: Maxent rl without entropy. arXiv preprint XXXi:2301.02328, 2023.
Emil Julius Gumbel. Slatistical theory of extreme values and some practical applicalions: a series of lectures, volume 33. US Government Printing Office, 1954.
Yuhong Guo and Dale Schuurmans. Discriminative batch mode active learning, In John C. Platt, Daphne Koller, Yoram Singer, and Sam T. Roweis (eds.), Advances in Newral Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007, pp. 593 600. Curran Associates, Inc., 2007. URL https: htt. Abstract. html.
17

---

Published in Transactions on Machine Learning Research (09/2023)
Jinyong Hahn. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Eeonometrion, pp. 315-331, 1998.
James J Heckman, Hidchíko Ichimura, and Petra E Todd. Matching econometric evaluation estimator: Evidence from evaluating job training programme. The review of economic studies, 64(4):605-654, 1997.
James Heckman, Hidehiko Ichimura, and Petra Todd. Matching as an econometric evaluation estimator.
The review of economic studies, 65(2):261-29) 1998.
Jennifer Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217 2400 2011.
Steven CH Hoi, Rong Jin, Jianke Zhu, and Michael R Lyu. Batch mode active learning and its application to medical image classification. Proceedings of the 23rd international conference on Machine learning, pp- 417-424, 2006.
Geoff Holmes, Eibe Frank, Dale Fletcher, and Corey Sterling. Efficiently correcting machine learning: considering the role of example ordering in human-in-the-loop training image classification models. IUI 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 25, 2022, 2022.
Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classificat tion and preference learning. Xixiv preprint ar arXiv:111..57/55 2011.
Andrew Jesson, Panagiotis Tigas, Joost van Amersfoort, Andreas Kirsch, Uri Shalit, and Yarin Gal. Causal- bald: Deep bayesian active learning of outcomes to infer treatment-effects from observational data. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6- 6-14, 2021, virtual, 2021.
Cem Kalkanli and Ayfer Özgür. Batched thompson sampling. Advances in Newral Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021.
Andreas Kirsch and Yarin Gal. Unifying approaches in active learning and active sampling via fisher information and tntormation-theoretiic quantities. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https: http:////pppppeeeee..eee///aaaar net//forum?AddUVVAAAAAAM
Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' AlchééBBuc, Fmiil B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Anruual Conference Newral Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancowerr, BC, Canada, pp. 7024-7035, 2019.
Wouter Kool, Herke Hoof, and Max Welling. Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replacement. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 44999-5000 PMLR, 2019.
Andreas Krause and Daniel Golovin. Submodular function maximization. Tractability, 3:71-104, 2014.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alex Kulesza and Ben Taskar. k-dpps: Fixed-size determinantal point processos. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 July 2, 2011, 1, 2011.
18

---

Published in Transactions on Machine Learning Research (09/2023)
Alexandre Lacoste, Pau Rodriguez López, Frederic Tranchaud-Charros Parmida Atighehchian, Massimo Caccia, Issam Hadj Laradji, Alexandre Drouin, Matt Craddock, Lauuent Charlin, and David Vázquez. Synbols: Probing learning algorithms with synthetic datasets. In Hugo Larochelle, Marc' Aurelio Ran- zato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL
htt.:/insnnnuii..ce/eeeeeeee ccppaper/2000 /
0169cf885f882ef1779855888 25w.555d1533dddcffcc Abstract. hhmll..
Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lec, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzanio, Lingjaa Tang, and Jason Mars. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, 2019.
Zhiming Luo, Frederic Saanchaud-Charron, Carl Lemaire, Janusz Konrad, Shaozi Li, Akshaya Mishra, Andrew Achkar, Justin A. Eichel, and Pierre-Marc Jodoin. MIO-TCD: A new benchmark dataset for vehicle classification and localization. IEEE Trans. Image Process., 2018.
Chris J. Maddison, Daniel Tarlow, and Tom Minka. A sampling. In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (eds.), Advances in Neural Informalion Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pp. 3086-3094, 2014.
Shreshth A Malik, Salem Lahlou, Andrew Jesson, Moksh Jain, Nikolay Malkin, Tristan Deleu, Yoshua Bengio, and Yarin Gal. Batchgfn: Generative flow networks for batch active lcarning. arXiv preprint arXiv:2306.1500 2023.
Alexander McFarlane Mood. Introduction to the theory of statistics. 1950.
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty: simple baseline. arXiv e-prints, pp. arXiv-2102 2021.
Chelsea Murray, James Urquhart Allingham, Javier Antorán, and José Miguel Hernández-Lobato. Depth uncertainty networks for active learning. arXiv preprint arXiv:2112.0677 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Jerzy Neyman. edited and translated by dorota dabrowska and terrence p. speed (1990). on the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science, 5(4): 465 472, 1923.
Robert Pinsler, Jonathan Gordon, Eric T. Nalisnick, and José Miguel Hernández-Lobato Bayesian batch active learning as sparse subset approximation. In Hanna M. Wallach, Hugo Larochelle, Alina Boygelzimer, Florence d' d'Alché-Bue, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Informalion Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancowver, BC, Canada, pp. 6356-6367, 2019. URL https://proceeding wwwpoddiinnnnn.uuipsp....e.. cc/ Abstract. html.
Aleksandr Rubashevskii, Daria Kotova, and Maxim Panov. Scalable batch acquisition for deep bayesian active learning. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pp. 739-747. SIAM, 2023.
Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688, 1974.
Donald Rubin. Randomization analysis of experimental data: The fisher randomization test comment.
Journal of the American Statistical Association, ((((33)))591-8))) 1980.
19

---

Published in Transactions on Machine Learning Research (09/2023)
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learning Represenlations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016. URL http://arxiv.ors 0ts/15111005552
Grog Schohn and David Cohn. Less is morc: Active learning with support vector machinos. In Pat Langley (ed.), Proceedings of the Sevenleenth International Conference on Machine Learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29 July 2, 2000, pp. 839 846.446. Morgan Kaufmann, 2000.
Jasjeet S Sekhon. The neyman-rubin model of causal inference and estimation via matching methods. The Oxford handbook of political methodology, 2:1-32, 2008.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In 6th International Conference on Learning Representations, ICLR 2018, Vancoweer, BC, Canada, April
May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview:
Burr Settles. Active Learning Literature Survey. Machine Learning, 2010.
Uri Shalit, Fredrik D. Johansson, and David A. Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, 1CML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3076-3085. PMLR, 2017. URL http://proocedung..aa mml. Tates//77//haaitti1
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition.
In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: //arxiv. httpxii.orrgabbs/tee 1400. 1556.
Freddic Bickford Smith, Andreas Kirsch, Sebastian Farquhar, Yarin Gal, Adam Foster, and Tom Rainforth. inniiction-oriented bayesian active learning, arXiv preprint arXiv:2304.08151, 2023.
Panagiotis Tigas, Yashas Amnadani, Andrew Jesson, Bernhard Schölkopf, Yarin Gal, and Stefan Bauer Inter- ventions, where and how? experimental design for causal models at scale. orXiv preprint arXiv:2203.0200 02011 2022.
Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. Improving deterministic uncertainty estimation in deep learning for classification and regression. arxiv preprint arXiv:2102.11400 11409, 2021.
Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
Sahil Verma and Julia Rubin. Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness, Fair Wairaae@@ICE 2018, Gothenburg, Sweden, May 29, 2018, 2018
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Authony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2020 Demos, Online, November 16-20, 2020, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms, 2017.
Xueying Zhan, Yaowei Wang, and Antoni B. Chan. Asymptotic optimality for active learning processes. In Uncertainty in Artificial Intelligence, Proceedings of the hhirty-Fighth Conference Uncertainty in Artificial Intelligence, UAI 2022, 1-5 August 2022, Eindhoven, The Netherlands, 2022.
20

---

Published in Transactions on Machine Learning Research (09/2023)
A
Ethical impact
We do not foresce any ethical risks related to this work. Insofar as the sampling methods we have examined reduce computational costs, applications might benefit from reduced resource consumption. The methods appear to be as good or better than alternatives on evaluations examining the ability to learn from data with under-represented groups and on ovaluations that measure the difference between performance for the most- and least-represented groups, which may aid algorithmic fairness (see C.5).
B
Method
B.1
Other scoring functions
Following Gal et al. (2017), we also examine using variation ratios (least confidence) and standard deviation scoring functions.
Variation Ratio. Also known as least confidence, the variation-ratios is the complement of the least-confident class prediction:
Svariation-ratios (i; Itrain) -max x.)).
(20)
This scoring function is non-negative and a score means that the sample is uninformative: a score of means that the respective prediction is one-hot, which means that the expected information gain is also 0 can be easily verified. Thus, variation ratios matches the intuitions behind power acquisition.
Standard Deviation. The standard deviation score function measures the sum of the class probability deviations and closely related to the BALD scores:
Setd-dev Itrain
Σ Varp(a)lp(y) X
2,, w).
(21)
This scoring function is also non-negative, and no variance for the predictions implies zero expected information gain and thus an uninformative sample. Thus, the standard deviation should also perform well with power acquisition.
B.2 Proof of Proposition 3.1
First, remind the reader that random variable G' is Gumbel distributed G Gumbel(J;; 3) when its cumulative distribution function follows p(G S g) = exp(-ccp(-"-7)
Furthermore, the Gumbel distribution is closed under translation and positive scaling: Lemma B.1. Let G Gumbellu; 8) be a Gumbel distributed random variable, then:
aG d Gumbel(d a9;
(22)
Proof. We have p(aG + d x) Gp S z-4 Thus, we have:
R-d
11
p(aG d x) exp( exp( 8
(23)
I - (d + ap)
expl hop(-epp exp(
(24)
a3
SaG + d Gumbel(d aq
(25)
We can then easily prove Proposition 3.1 using Theorem from Kool al. (2019), which we present it here slightly reformulated to fit. notation:
21

---

Published in Transactions on Machine Learning Research (09/2023)
(b) Duplicates.
(c) Class confusion.
(d) Compression artefacts.
(e) Low-resolution sample.
(a) Good sample.
Figure 9: MIO-TCD TCD Dataset is designed to include common artifacts from production data. The size and quality of the images vary greatly between crops; from high-quality cameras on sumny days to low-quality cameras at night. (a) shows an example clean samples that can be clearly assigned to a class. (d) and (e) show the different categories of noise. (b) shows an example of duplicates that exist the dataset. (c) good example where the assigned class is subject to interpretation: motorcycle or bicycle? (d) is a sample with heavy compression artefacts and (e) example of samples with low resolution which again is considered hard example to learn for the model.
Lemma B.2. For n, let li...., arg tops {st + ei}i with e; Gumbel(O; 1), i.i.d. Then li...., 18 is (ordered) sample without replacement from the Categorical for a realization i it holds that
i E [[.......]) distribution, e.g. E,en exps;
exps;
cxp St;
P P = 19
II
Dieny exp
where Nj- = N. (ii.....,711) is the domain (without replacement) for the j-th sampled element.
Now, it casy to prove the proposition:
Proposition 3.1. For scores 8i, i E nI, and k and 3 > 0, if we draw Gumbell 0: 8-1) independently, then arg tops {&t eils is (ordered) sample without replacement from the categorical distribution Categorical(exp(0 s exp(O (I....,n)).
Proof. As E; Gumbel(O; B)1), define ęę := 38g Gumbel(0; 1). Further let = 9st. Applying Lemma. B.2 on sy and é; arg top {s! + ch yields (ordered) samples without replacement from the categorical distribution nl)...,m)). However, multiplication by 3 does not change the resulting indices
Categorical Σ, exp(888))
ex)(
of arg tops:
arg top (8/ + c): = arg top (s; 4h.
(26)
concluding the proof.
22

---

Published in Transactions on Machine Learning Research (09/2023)
C Experiments
C.1 Experimental setup & compute
Frameworks. We use PyTorch. Repeated-MNIST and EMNIST experiments use PyTTrrh Ignite. Synbols and MIO-TCD experiments use the BaaL library: https://github. ceatiihuu.com/baaaaaa.a com/baal- aoaal-rr/baaal baal (Atighehchian et al., 2020). Predictive parity is calculated using FairLearn (Bird et al., 2020). The CansalBALD ex- periments use https://github. comparing to ACS-FW (Pinsler al., 2019) use the original authors' implementation with added support for stochastic batch acquisitions: https://github. http.///////./////aa/aeeeeeeeeee
(Jesson et al., 2021). The experiments
wwwresiiaancrreess/reelleee wowesets/reeesseeeeeeeeee cestocchbatchhh _a______c_______________ The Repeatedd-MNIST experiments were run using https://giite com/ httpaiiieeeeeeeeeeeeeeeee///ttte haapcchhhhbchhhhaae acq, and the results are also available on WandB (https://wandes
Compute. Results shown in Table were run inside Docker containers with 8 CPUs (2.2Ghz) and 32 Gb of RAM. Other experiments were run on similar machines with Tilan RTX GPUs. The Repeated-MNTD and EMNIST experiments take about 5000 GPU hours. The MIO, Synbols and CLINC-150 experiments take about 19000 GPU hours. The CausalBALD experiments take about 1000 GPU hours.
Dataset Licenses. Repeated-MNISS is based on MNIST which is made available under the terms of the Creative Commons Attribution-Share Alike 3.0 license. The EMNIST dataset is made available as CCO 1.0 Universal Public Domain Dedication. Synbols is a dataset generator. MIO-TCD is made available under the terms of the Creative Commons CaabbuttoooNonCCConnracaaaa---- CLINC-150 is made available under the terms of Creative Commons Attribution 3.0 Unported License.
4.0 International License.
C.1.1 Runtime measurements
The synthetic dalasel used for benchmarking has 4,096 features, 10 classes, and 10,000 pool points. VGG-16 models (Simonyan & Zisserman, 2015) were used to sample predictions and latent embeddings.
C.1.2 Srpeated-MNIST
The Repeated-MINIS) dataset also constructed following Kirsch et al. (2019) with duplicated examples from MNIST with isotropic Gaussian noise added to the input images (standard deviation 0.1).
We use the same setup as Kirsch et al. (2019): a LeNet-5-like architecture with ReLU activations instead of tanh and added dropout. The model obtains 99% test accuracy when trained on the full MNIST dataset. Specifically, the model is made up of two blocks a convolution, dropout, max-pooling, ReLU with 32 and 64 channels and 5x5 kernel size, respectively. As classifier head, a two-layer MLP with 128 hidden units (and 10 output units) is used that includes dropout. between the layers. We use a dropout probability of 0.5 everywhere. The model trained with early stopping using the Adam optimiser and learning rate of 0.001. We sample predictions using 100 MC-Dropout samples for BALD. Weights are reinitialized after each acquisition step.
C.1.3 EMNIST
We follow the setup from (Kirsch et al., 2019) with 20 MC dropout samples. We use a similar model as for Repeated-MNIST but with three blocks instead of two. Specifically, we use 32, 64, and 128 channels and 3x3 kernel size. This is followed by 2x2 max pooling layer before the classifier head. The classifier head is a two-layer MLP but with 512 hidden units instcad of 128. Again, we use dropout probability 0.5 everywhere.
C.1.4 Synbols& & MIO-TCD
The full list of hyperparameters for the Synbols and MIO-TCD experiments presented in Table Our experiments are built using the BaaL library (Atighehchian et al., 2020). We compute the predictive parity using FairLearn (Bird al., 2020). We use VGG-16 model (Simonyan & Zisserman, 2015) trained for 10 epochs using Monte Carlo dropout for acquisition (Gal et al., 2017) with 20 dropout samples.
23

---

Published in Transactions on Machine Learning Research (09/2023)
Table 3:
used Section 5 and C.5
Hyperparameter
Value
Learning rate
0.001
Optimiscr
SGD
Weight decay
0
Momentum
0.9
Loss function
Crossentropy
Training duration
10
Batch size
32
Dropout p
0.5
MC iterations
20
Query size
100
Initial set
500
In Figure 9, we show a set of images with common problems that can be found in MIO-TCD.
C.1.5 CLINC-150
We fine-tune a pretrained DisúiBBRRT model from HuggingFace (Wolf et al., 2020) on CLINC-150 for 5 epochs with Adam as optimiser. Estimating epistemic uncertainty in transformer models is an open research question, and hence, we do not report results using BALD and focus on entropy instead.
C.1.6 CausalBALD
Using the Neyman-Rubin framework (Neyman, 1923; Rubin, 1974; Sekhon, 2008), the CATE is formulated in terms of the potential outcomes, Yt. of treatment levels t E {0, 1}. Given observable covariates, X, the CATE is defined as the expected difference between the potential outcomes at the measured value X = x: T(x) = E[Y, Yo X = x]. This causal quantity fundamentally unidentifiable from observational data without further assumptions because it not possible observe both Y, and Y, for given unit. However, under the assumptions of consistency, non-interference, iguoreability, and positivity, the CATE is identifiable the statistical quantity F(x) E|Y T = 1,X = E T = 0, = x] (Rubin, 1980).
Jesson et al. (2021) define BALD acquisition functions for active lcarning CATE functions from obscrvational data when the cost of acquiring an outcome, y, for a given covariate and treatment pair, (x,1), 1), is high. Because we do not have labels for Y, and Yn for each (x, pair the dataset, their acquisition function focusses on acquiring data points (x, t) for which data or has already been acquired a previous step. We follow their experiments on their synthetic dataset with limited positivity and the semi-synthetic THDP dataset (Hill, 2011). Details of the experimental setup are given in (Jesson et al., 2021), we use their provided code, and implement the power acquisition function.
likely that a matched pair (x, - t) exists in the pool
The settings for causal inference experiments are identical to those used in Jesson et al. (2021), using the THDP dataset (Hill, 2011). Like them, we use a Deterministic Uncertainty Estimation Model (van Amersfoort et al., 2021), which initialised with 100 data points and acquire 10 data points per acquisition batch for 38 steps. The dataset has 471 pool points and 201 point validation set.
C.1.7 ACS-FW
We compare to the experiments in the paper 'Bayesian Batch Active Learning as Sparse Subset Approximation' (Pinsler el al., 2019) which introduces the ACS-FW algorithm for batch acquisition. For its deep learning experiments, the paper uses a ResNet feature extractor, followed by Bayesian multi-class classification model on the final layer. Since exact inference intractable, variational inference used with factorized Gaussian posterior approximation. As prior a zero-mean Gaussian is used, and Monte- Carlo samples are drawn to approximate the predictive distribution.
24

---

Published in Transactions on Machine Learning Research (09/202))
C.2 Repeated- RepSated-MNIST
hopet Foctor
Reputtiin Factur
2.6
0.7
0.8
0.6
0.5 0.7
0.8
2.6
0.7
5.8
3.9
50
50
53
Acquetionn Functinn
BADCE
BADCE
BADCE
200
200
203
Figure 10: Repeated-MNIST MNIST x4 (5 trials): acquisition size ablation for BADGE. Acquisition size 40 performs best out of {10, 20, 40) for repetition factor 1, acquisition 20 performs best for repetition factor 2, and all three perform similarly for repetition factor 4.
BADGE Ablation. In Figure 10, we see that BADGE performs best with acquisition size on Repeated- MNISTx4 overall. BADGE 40 and BADGE 20 have the highest final accuracy, cf. BADGE 10 while BADGE 20 performs better than BADGE 40 for small training set sizes.
Accuracy
0,6
0.7
0.8
0.9
Min Training Set Size
100
PowerBALD
SoftmaxBALD
SoftrankBALD
250
BatchBALD
BALD
Figure 11: Repealed-MMISTT 24 (5 brials): acquisition size To compare BatchBALD, BALD and the stochastic acquisition strategies more fairly, also show the performance of acquisition size 5. This is in line with Figure? 2.
Acquisition Size 5. In Figure 11, we see that the stochastic acquisition strategics perform better than BALD and well as BatchBALD for acquisition size well. Thus, we see no qualitative difference the results in Figure! 2.
25

---

Published in Transactions on Machine Learning Research (09/2023)
C.2.1 Other scoring functions
Accuracy
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
50
Min Training Set Size
100
150
200
PowerBALD
BALD
PowerStdDee
StdDev
PowerVariationRatios
VariationRatios
250
PowerEntropy
Entropy
Uniform
300
Figure 12:
RepIated-MNIST 24 (5 trials): Performance for other scoring functions. Entropy, std dev,
variation ratios behave like BALD when applying stochastic sampling scheme (Power from $3).
In Figure 12 shows the performance of other scoring functions than BALD on RepS x4.
C.2.2 Redundancy ablation
Repetition Fector
Repelition Factor
Repei Factor Arcuracy
Accuraty 0.75
Accuracy 0.75
0.55
0.60
0.55
0.70
0.80
0.85
0.90
0.95
0.55
0.60
0.85
0.70
2.80
2,85
0.90
0.95
0.55
0.60
0.65
0.70
0.75
0.50
0.55
0.90
0.95
100
100
100
200
200
200
230
POWNTRALD
Figure 13: Repeated-MNNL (5 trials): Performance ablation for different repetition counts.
In Figure 13, see the same behaviour in ablation for different repetition sizes of Repeated-MNNIS
26

---

Published in Transactions on Machine Learning Research (09/2023)
C.3 MIO-TCD
Accuracy
Accuracy
0.95
0.96
0.97
0.98
0.99
0.95
0.96
0.97
0.98
0.99
2000
2000
Min Training Set Size
Min Training Set Size
4000
4000
6000
8000
8000
12000
BALD Uniform
12000
PowerEntropy
BADGE
Entropy
Uniform
(a) BALD
(b) Entropy
Figure 14: MTO-TCD (5 trials).
In Figure 14, we see that power acquisition performs on with BADGE with both BALD and entropy as underlying score functions.
C.4 EMNIST
Accuracy
Accuracy
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.2
0.4
0.6
0
Training Set Size
Min Training Set Size
50
50
100
100
150
PowerBALD
150
SoftmaxBALD
PowerBALD
200
SoftrankBALD
200
SoftmaxBALD ALD
SoftrankBALD
250
BADGE
250
BatchBALD 5
300
300
Figure 15: EMNIST (Balanced) (5 trials): Per- formance with BALD.
Figure 16: EMNIST (ByMerge) (5 trials): Per- formance with BALD.
In Figure 15 and 16, that. PowerBALD outperforms BALD, BatchBALD, and BADGE.
27

---

Published in Transactions on Machine Learning Research (09/2023)
Accuracy
0.0
0.1
0.2
0.3
0.4
0.5
0
Min Training Size
50
Acquisition Function
100
BADGE 10
BADGE 20
150
BADGE 40
200
250
300
Figure 17: EMNTST (Balanced) (5 trials): acquisition size ablation for BADGE.
BADGE Ablation. In Figure 17, we see that BADGE performs similarly with all three acquisition sizes. Acquisition size 10 is the smoothest.
C.5 Edge cases in Synbols
We use Synbols (Lacoste et al., 2020) to demonstrate the behaviour of batch active learning in artificially constructed edge cases. Synbols is character dataset generalor for classification where user can specily the type and proportion of bias and insert artefacts, backgrounds, masking shapes, and so on. We selected three datascts with strong biases supplied by Lacoste et al. (2020); Branchaud-Charmn et al. (2021) to evaluate stochastic batch acquisition methods. We use an acquisition size of 100. The experimental settings are described Appendix C.I.
For these tasks, performance evaluation includes 'predictive parity', also known 'accuracy differencc', which is the maximum difference in accuracy between subgroups-wiit which are, in this case, different coloured characters. This measure is used most widely in domain adaptation and ethics (Verma & Rubin, 2018). We want to maximise the accuracy while minimising the predictive parity.
0.65
0.70
0.75
Accuracy
0.80
0.85
0.90
0.95
1.00
0.25
PowerBALD
BALD
BADGE
Uniform
Predictive Parity
0.20
2000
Min Training Set Size
4000
0.15
0.10
0.05
10000
0.00
12000
PowerBALD
BALD
14000
BADGE
Uniform
0
2000
4000
10000 12000 14000 Training Set Size
6000
8000
(a) Accuracy
(b) Predictive parity (Down and left is better.)
Figure 18: Performance on Synbols Spurious Correlations (3 trials) with BALD. Stochastic acquisition matches BADGE and BALD's predictive parity and performance, which is reassuring stochastic acquisition functions might be affected by spurious correlations.
28

---

Published in Transactions on Machine Learning Research (09/2023)
Spurious Correlations. This dataset includes spurious correlations between character colour and class. As shown in Branchaud-Charro et al. (2021), active learning is especially strong here characters that do not. follow the correlation will be informative and thus selected.
We compare the predictive parity between methods in Fig. 18b. We do not see any significant difference between the stochastic batch acquisition method and BADGE or BALD. This encouraging, as stochastic approaches might select more examples following the spurious correlation and thus have higher predictive parity, but this is not the case.
0.75
0.80
0.85
Accuracy
0.90
0.95
1.00
0.07
BALD
0.06
BADGE
Uniform
Predictive Parity
2000
0.05
Min Training Set Size
4000
0.04
6000
0.03
8000
0.02
10000
0.01
12000
PowerBALD
BALD
14000
BADGE
Uniform
2000
4000
Training Set Size
6000 8000 10000 12000 14000
(a) Accuracy
(b) Predictive parity
Figure 19: Symbols Minority Groups (3 trials): Performance on BALD. PowerBALD outperforms BALD and matches BADGE for both accuracy and predictive parity.
Minority Groups. This dataset includes a subgroup of the data that is under-represented; specifically, most characters are red while few are blue. As Saanchand-Charee al. (2021) shows, active lcarning can improve the accuracy for these groups
Our stochastic approach lets batch acquisition better capture under-represented subgroups. In Figure 19a, PowerBALD has an accuracy almost identical to that of BADGE, despite being much cheaper, and outperforms BALD. At the same time, we see in Figure 19b that PowerBALD has lower predictive parity than BALD, demonstrating a fairer predictive distribution given the unbalanced datasct.
Accuracy
Accuracy
0.955
0.960 0.965 0.970 0.975 0.980 0.985
0.955
0.960 0.965 0.970 0.975 0.980 0.985
2000
2000
Min Training Set Size
Min Training Set Size
4000
4000
6000
6000
8000
8000
10000
10000
12000
PowerBALD
BALD
12000
PowerEntropy
BADGE
14000
BADGE
Uniform
14000
Entropy
Uniform
Figure 20: BALD
Figure 21: Entropy
Figure 22: Performance on Synbols Missing Characters (3 trials). In this dataset with high alcatoric uncertainty, PowerBALDD matches BADGE, and BALD performance. PowerEntropy significantly outperforms Entropy which confounds aleatoric and epistemic uncertainty.
29

---

Published in Transactions on Machine Learning Research (09/2023)
Missing Synbols. This dataset has high aleatoric uncertainty (input noise). Some images are missing information required to make high-probability predictions- these images have shapes randomly occluding the character so even perfect model would remain uncertain. Lacoste et al. (2020) demonstrated that entropy ineffective on this data as it cannot distinguish between alcatoric and epistemic uncertainty (imput noise and model uncertainly), while BALD can do so. As consequence, entropy will unfortunately prefer samples with occluded characters, resulting degraded active learning performance. For predictive entropy, stochastic acquisition largely corrects the failure of entropy acquisition to account for missing data (Figure 22) although PowerEntropy still underperforms BADGE here. For BALD, show in Figure 20 in the appendix that, as before, the stochastic batch acquisition method performs par with BADGE and marginally better than BALD.
C.6 CLINC-150
Accuracy
0.60
0.65
0.70
0.75
0.80
0.85
1000
2000
Min Training Set Size
3000
4000
5000
PowerEntropy
6000
BADGE
Entropy
7000
Uniform
8000
Figure 23: Performance on CLINC- 150 (10 trials). PowerEntropy performs much better than entropy, which only performs marginally better than uniform, and almost on par with BADGE.
In Figure 23, we see that PowerEntropy performs much better than entropy which only performs marginally better than the uniform baseline. PowerEntropy also performs better than BADGE at low training set sizes, but BADGE performs better in the sccond half. Between 2300 and 4000 samples, BADGE and Enwrrpn perform the same. We use an acquisition size of 100, starting from an initial training set of 1510 points (10 points per intent class).
30

---

Published in Transactions on Machine Learning Research (09/2023)
C.7 ACS-FW
0.90
0.80
Accuracy
Accuracy
0.85
0.75
BOWD
BALD
PowerBALD
BALD
ACS-FW
Uniform
ACS-FW
Uniform
0.70
12000
16000
4000
6000
Training Set Size
Training Set Size
(a) CIFAR-10 10
(b) SVHN
0.9
0.7
0.8
Accuracy
0.7
Accuracy
0.8
0.8
0.5
0.5
PowerBALD
BALD
PowerBALD
BALD
0.4
ACS-FW
Uniform
0.4
ACS-FW
Unifomm
100
200
100
200
Training Set Size
Training Set Size
(c) Seppeteed-NNSST
(d) Mashion-MNIST
Figure 24: MFVI Last-Layer Performance (9 trials).
2500
1000
PowerBALD
BALD Uniform
PowerBALD
BALD Uniform
2000
ACS-FW
800
ACS-FW
Waltime
1500
Wallime
600
500
200
6000
10000
12000
16000
1000
3000
6000
Training Set Size
Training Set Size
(a) CIFAR-10 10
(b) SVHN
PowerBALD
BALD
1200
PowerBALD
BALD Unifomm
ACS-FW
Uniform
ACS-FW
600
Waltime
Walliime
400
600
400
200
200
50
100
200
100
150
200
Training Set Size
Training Set Size
(c) Mopeateed-NNIST
(d) Fashion-MNIST
Figure 25: MFVI Last- Last-Layer Walltime (3 trials).
31

---

Published in Transactions on Machine Learning Research (09/2023)
We run comparisons for several datasets using last-layer mcan-field variational inference: CIFAR-10 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), Repeated-MNIST. (Kirsch et al., 2019), and Fashion- MNIST (Xiao et al., 2017). We use 5000 initial training samples with an acquisition size of 4000 for CIFAR-10, 1000 initial training samples with an acquisition size 2000 for SVHN, 20 initial training samples with an acquisition size of 100 for Repealed-MNISST, and 20 initial training samples with an acquisition size of 25 for Fashion-MNIST. The same hyperparameters as in Pinsler al. (2019) are used.
In Figure 24,
that PowerBALD performs best on Fashion-MNIST and Repeated-MNNIST with smaller acquisition sizes, but ACS-FW and BALD perform similarly on CIFAR-10 10 and SVHN at larger acquisition sizes. This is in line with Section 6, where we have seen in Figures 5 and 7 that the top 1% of the samples are more sensitive than the top 10%: larger acquisition sizes can be less sensitive to the issues of top-K. The performance of ACS-FW contrasts to the results in Pinsler et al. (2019) where ACS-FW performs better than BALD on CTFAR-10 and SVHN. In Appendix E.3, we provide ablations with different acquisition sizes and initial training set sizes for these four datascts. Overall, beyond what we have noted above, these results are not conclusive, however, and the performance curves are very close to each other. On the other hand, the wall time curves in Figure 25 show that ACS-FW is significantly slower than PowerBALD, This is congruent with the motivation that stochastic sampling methods are fast yet competitive and don't 't underperform compared to top-K acquisition, thus being a good substitute for top-K K acquisition.
D Comparing Power, Softmax and Soft-Rank
D.1
Empirical Evidence
Repotttion
Ropetition Factor
etttion
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.30
0.90
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.85
0.55
0.60
0.65
0.70
6.75
0.80
0.85
0.90
0.85
Min Tralingg
1.00
100
100
200
202
200
280
250
250
PoVeRRALD KattouaRALD
(a) BALD
Repetition Facior
Repal Aion Fector
Repailiion Factor -
Accuracy
Accuracy
Accuracy
0.55
0.60
0.55
0.70
0.75 0.80
0.85
0.90
0.95
0.55
0.60
0.85
0.70
0.75 0.80
0.85
0
0.95
0.55
0.50
0.65
0.70
0.75 0.30 0.55
0.90
0.95
1.00
1.0
100
200
200
200
We
250
(b) Entropy
Figure 26: Repealed-MNIST (5 trials): Performance with all three stochaslic stralegies.
Repeated-MNIIST. In Figure 26, power acquisiiiinn performs best overall, followed by soft-rank and then softmax.
32

---

Published in Transactions on Machine Learning Research (09/2023)
Accuracy
Accuracy
Min Training Set Size
Min Training Set Size
100
100
150
150
200
200
PowerBALD
BatchBALD
250
PowerBALD
BitehBAIDD 5
250
SoftmaxBALD
BALD
SoftrankBALD
SoftmarBALD
BADSE
BALD
SoftrankBALD
300
300
Figure 27: EMNIST (Balanced) (5 trials): Per- formance with all three stochastic strategies with BALD. PowerBALD performs best.
Figure 28: EMNIST (ByMerge) (5 trials): Per- formance with all three stochastic strategies with BALD. PowerBALD performs best.
EMNIST. In Figure 27 and 28, we see that PowerBALD performs best, but Softmax- and SoftrankBALD also outperform other methods. BADGE did not on EMNIST (ByMerge) due to out-of-memory issues and BatchBALD took very long EMNIST (ByMerge) has more than 800,000 samples.
Accuracy
Accuracy
0.95
0.96
0.97
0.98
0.95
0.96
0.97
0.98
0.99
2000
2000
Min Training Set Size
Min Training Set Size
4000
4000
6000
6000
8000
8000
10000
10000
12000
PowerBALD
SoftrankBALD
12000
PowerEntropy
SoftrankEntropy
14000
SoftmaxBALD
BALD
14000
SoftmaxEntropy
Entropy
(a) BALD
(b) Entropy
Figure 29: MIO-TCD (3 trials): Performance with all three stochastic strategies.
MIO-TCD. In Figure 29, we that all three stochastic acquisition methods perform about equally well.
33

---

Published in Transactions on Machine Learning Research (09/2023)
Ostaset Rspresented Groups
Dataset Mizsing Synbols
Sputious Conretatoon
Accuracy
0.80
0.85
3.50
0.45
1,00
0.955
0,960
0,955
D:470 3.975 0,980 0.905
0.65
0.70
0.80
0.85
0.95
1.00
2300
5000
5000
5000
7500
7500
7500
12500
12500
12500
PowerBALD SoftmaxBALD
SoftrankBALD
15000
(a) BALD
Dataset Undern-Sepreeettee Groups
Datsset Missing Synbols
Dataset Spurious Conelstions
Arcurecy
Accurary
Arrunacy
0.85
0.980
0.65
2500
2500
2500
5000
Traninn
7500
7500
10000
10000
10000
SoftrankEntropy
SoftmaxEntropy
15000
15000
15000
(b) Entropy
Figure 30: Synbols edge cases (3 trials): Performance with all three stochastic stralegies.
Synbols. In Figure 30, power acquisition seems to perform better overall mainly due to the performance in Synbols Missing Characters.
Accuracy
0.60
0.65
0.70
0.75
0.80
0.85
1000
Min Training Set Size
2000
3000
4000
5000
PowerEntropy
6000
SoftmaxEntropy
7000
softrankEntropy
8000
Figure 31: CLINC-150 (10 trials): Performance with all three stochastic strategies.
CLINC-150. In Figure 31, all three stochastic methods perform similarly.
D.2 Investigation
To further examine the three stochastic acquisition variants, we plot their score distributions, extracted from the same MNIST toy example, in Figure 32. Power and softmax acquisition distributions are similar for B = 8 (power, softmax) and B =4 (softmax). This might explain why active learning with these & shows similar accuracy trajectories.
We find that power and softmax acquisition are quite insensitive to and thus selecting و might generally work quite well.
34

---

Published in Transactions on Machine Learning Research (09/2023)
0.0010
B
10-5
0.0008
0.25
1.0
10-11
Probabilty
0.0006
2.0
10 10-17
4.0
8.0
0.0004
10 -3
Distribution Type
Power
0.0002
10-29
Softmax
Softrank
0.0000
10-35
Figure 32: Score distribution for power and softmar acquisition of BALD scores on MNIST for varying Coldness 3 at t 0. Linear and log plot over samples sorted by their BALD score. At = both softmax and power acquisition have essentially the same distribution for high scoring points (closely followed by the power distribution for 3 4). This might explain why the coldness ablation shows that these / to have very similar AL trajectorics on MNIST. Yet, while softmax and power acquisition scem transfer to RMNIST, this is not the case for softrank which is much more sensitive to B. At the same time, power acquisition avoids low-scoring points more than softmax acquisition.
E Effect of changing the aquisition sizes
E.1
Repeated-MMN -NNITT
Npstinn Spar- (amuiiin
100
120
300
Musdtii Nactle Aqqustoe 52x-1
Aepattio (Aquaiio
Aepttiinlaaee e1 AAraatii
Sertaaefeete Ropradie
Ayediin (ogohke Oee
Appliastaie ad tor
Figure 33: Pepented-MNTSS Acquisition size ablation grouped by acquisition size. Stochastic acquisition strategies (except for Softrank at acquisition size 5) always perform better than top-K BALD and also better than BADGE. Softrank is the most sensitive to acquisition size changes because it is independent of the scores.
35

---

Published in Transactions on Machine Learning Research (09/2023)
Acqubition
Acqubition PowerBALD Factor
Acpu Punddinn
5.6
0.7
0.9
0.6
0.0
50
1.00
130
130
1.50
150
130
200
230
230
250
250
250
Acqusilian Funcian SoftmaxBALD Repeliiio facior 1
Acquiatton Funcian SoftmacBALD Repetiion Factor 2
AcqusRion Funcion SoftmonBALD Resettion Factor 4
5.6
0.7
5.8
0.9
0.5
077
0.
0.9
0.6
0.7
0.0
0.8
50
50
50
150
150
150
Nin
250
230
250
Acqusiion Fundtin = SoftraniBALD| | Nosetttin Foctor -1
AcoMstlnn Fundinn
SoftrunkB4LD DI |Resciiion Factor 22
Acquisition Fusction = SoftramBBALD |Ropetkk on Foctar 44
3.5
3.6
0.7
0.9
0.6
0.8
50
50
50
Acceestion Siee
1.00
100
130
Tounto
10
1.50
150
150
200
230
230
300
330
330
Acqusiion function BADGE RRppett on Factor
Acaus Son Function
BADGE Reperttoon Factor 2
Acqus tion Functinn BADGE| Repetiton Factor
07
0.9
0.8
Tran Q1e
1.00
100
130
200
200
230
250
300
330
330
Acqulatioo Fundton BALD Repection Puctor 1
Acqubiton Fandionn - BALD hapettion Pactor 2
Acqu on Fanction
BBLDD Ropeth on factor 4
Acculacy
Accurayy
Accuracy
5.8
07
09
0.6
0.8
50
50
1.00
130
130
155
150
130
200
230
230
250
230
250
Figure 34: Repeated- - Acquisition size ablation grouped by acquisition function. PowerBALD and SolunaxBALD are less sensitive to acquisition size changes while SoltrankBALD is more sensitive to acquisition size changes. Overall, all three are much less sensitive to acquisition size changes than top- BALD.
36

---

Published in Transactions on Machine Learning Research (09/2023)
E.2 EMNIST
2.00|
Acqusstion slion Function PowerBALD
2.2 Acqus tin Funciion - BALD
p- 1.0 Furdian SoftmurBADD
0.5
0.1
0.
0.3
0.4
2.5
0.0
0.3
0.2
0.3
04
0.5
0.3
5.1
0.2
6.3
0.4
5.5
Nn Taanire San
50
50
50
150
150
150
300
330
330
Aequectionn
1.0
96 Takins S48 31m
Anquiition
200
100
120
150
250
250
250
PowcIBALD
A Acqusstion functonn
Acquiaiion Furciaan SoftmaxBALD
Acarany
Accuraty
0.2
0.2
0.4
3.0
0.0
0.3
8.
Nn Tintannng. Se: 52e
500
50
50
100
130
200
230
230
Figure 35: EMNIST 'Balanced': Acquisition size ablation. For (top-K) BALD, 3 makes no difference, so 8 = 1 is used placeholder. The other two subplots are intentionally left out as they are very similar to this one.
37

---

Published in Transactions on Machine Learning Research (09/2023)
0.0
0.5
3.6
07
0.5
0.6
0.5
0.6
50
50
150
150
150
250
250
250
F= [8|Actu Funttion
PowerBALD
A- 4.8 (Aquustion Furction BALD
4.4.0 Acquetiion Furction SofteadBALD
2.4
0.5
255
0.7
0.4
055
0.
03
0.5
0.5
0.7
50
50
Arqulitioon Stze
100
100
150
10
He Pa0000
200
250
230
250
A
(cquuctton
p
0.0
0.1
02
0.4
0.5
3.6
077
0.0
3.3
0.2
0.4
855
0.6
07
2.0
0.1
0.
0.4
0.5
0.6
0.7
500
50
50
150
150
230
220
300
320
320
Figure 36: EMNIST 'ByMerge': Acquisition size ablation. For (top-K) BALD, 3 makes no difference, so
B == 1 is used as placeholder. The other two subplots are intentionally left out as they are very similar this
one.
38

---

Published in Transactions on Machine Learning Research (09/2023)
E.3 FCS-FW
Acquasion 75 Intial Training Sat
1030
Acquastion Sice 75 Inisal Trairing Sat Size 5030
0.85
0.60
0.5
2002
4000
9000
2000
10000
12000
14000
15000
5000
50000
10002
12000
14000
19000
Acquétion Size 125 Intial Traiiing Bet Size - 1000
Acquiation Size 125 Intia Traiiing Bet Size 5000
0.05
0.6
0.70
2002
4000
6900
1000
10000
12000
14030
16000
6200
1030
10002
12000
14000
10000
Acqubition Size = 250 Intial Traniin Set = 1000
Acquiation Size = 250] Intial Traniin Size 5000
0.8
0.85
0.75
0.70
500 Intial Training
Acqpistion 550 Intisl
0.85
0.5
PowerBALD
0.75
0.70
0.5
4000
0000
2000
10000
12000
18500
6000
5020
10002
10300
14000
Acquialion Siz 1030 Ins Truniing Sut Size 1003
Acquisiton Szz 1030 Truning Sui Size 5003
0.60
0.7
0.6
0.70
2032
1000
6000
2002
10000
12000
1403)
10000
0200
103)
10003
12000
54000
10000
Acspultton Son + 2000] Inll: Traning Swi Siow 1003
Aqqukition Sow - 2000 Intbe Thining Swi Sine 5003
0.6
0.75
0.6
2302
4000
1900
1000
10000
12000
14030
10000
5200
1033
10002
12300
14000
19300
Acquiiiiun Size 44000] Intl: Triniing Swt Stow 1003
Asquielton Size 4000 Intix Towning Swi Siow 5003
0.8
0.75
0.5
Trakring Sizs
Training Set Sizs
Figure 37: MFVI Last-Layer Classification CIFAR-10: Acquisition size ablation.
39

---

Published in Transactions on Machine Learning Research (09/2023)
Arquaation Trwniig
80000
19000
10000
10000
5030
2000
4000
6000
8000
10003
12000
56000
16300
8000
8000
10000
12000
14000
15000
Acquistton Size 125 Intial Training Set Bize 1000
Acquisition Size 125 (nitaal Traininng Get Size 0000
20000
20000
18000
10000
5000
2000
4002
5000
8000
10002
12000
14200
19332
5000
10000
12000
14000
10000
Acquialion Bize 253 Iniial Training Sei Siau 1000
Acquiaitin Size 250 Iritial Training But Size 9003
10000
1030
8000
8000
6000
2030
Thaining
8000
4000
Walime
3000
3000
2000
2000
1920
2000
4000
6000
8000
10000
12000
56000
16000
9000
8000
10000
12000
14000
19000
Acquistion Size 1000 Intial Training Set Site 1000
Acquistion Size 1000 Intial Training Set
5000
3000
1900
Waltme
2000
2000
3000
1930
2000
4002
5000
8000
10002
12000
14200
19332
6000
10000
12000
14000
15000
Acquisiiion Size 2000 Intial Training Bet Size 1000
Acquis lian Bize 2000 Intaal Trainnn Bet Siza 5000
2000
2000
Waline
1600
1500
1000
500
2530
2000
2000
1500
Wallme
1900
3000
1930
600
2000
4002
6000
8000
10003
12000
56000
16300
5000
8000
10000
12000
14000
15000
Tmiining Set
Training
Figure 38: MFVI Last- Layer Classification on CIFAR-101 10: Acquisition size ablation (Walltime).
40

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisition Size 75| Initial Training Set Size 1000
Acquisition Size 75 Initial Training Set Size 3000
0.90
0.95
0.08
0.80
0.08
0.75
0.84
1000
2300
3000
4030
5000
6000
7000
3000
3500
4000
4500
5000
6600
6030
6600
7000
Acquisition Size
125 Initial Training Set Size 1000
Acquisition Size 125 Initial Training Set Size
3000
0.92
0.90
Acauraty
0.86
0.88
0.80
0.75
0.84
1000
2300
3000
4030
5000
6000
7000
3000
3500
4000
4500
6000
6600
5030
6000
7000
Acquiiilio Size 250 Initial Training Set Size 1000
Acquisition Size 250 Initial Training Set Size
3000
0.90
Accuracy
0.90
0.80
0.86
0.84
PowerBALD
1000
2000
3000
4000
5000
6000
7000
3060
3500
4000
4500
5000
5500
8030
6600
7000
SoftmaxBALD
SoftrankBALD
Acquisition Size 500 Initial Training Set Size 1000
Acquisttio Size 500 Initial Trinning Set Size
3000
ACS-FW
0.92
BALD
0.30
Uniform
0.90
0.85
0.88
0.80
0.86
1000
2000
3000
4030
5000
6000
7000
3000
3500
4000
4500
6000
6600
6030
6600
7000
Acquisttin Size 1000 Initial Training Set Size = 1000
Acquisitinn Size
1000 | Initial Training Set Size
3000
0.92
0.90
Acauraty
0.85
0.88
0.84
1000
2000
9000
4000
5000
8000
7000
3000
3500
4000
4500
5000
$500
6000
6500
7000
Acquisition Size 2000 Initial Training Set Size
1000
Acquisition Size 2000 | Initial Training Sat Size
3000
0.85
0.80
0.75
0.70
1000
2000
3000
4000
5000
6000
7000
3000
3500
4000
4500
5000
5600
8030
6600
7000
Training
Training
Figure 39: MFVI Last- Layer Classification on SVHN; Acquisition size ablation.
41

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisition Size = 75| Initial Training Set Size 1000
Acquisition Size = Initial Training Set Size 3000
10000
8080
4000
4000
2000
2000
1000
2000
9000
4000
5000
6000
7000
3000
3500
4000
4500
5000
5500
6000
6500
7000
Acquiattion Size
125 Initial Training Set Size 1000
Acquisilion Size 125 Inttial Training Set Size
3000
Waltine
4000
1000
1000
2000
3000
4000
5000
6000
7000
3000
3500
4000
4500
5000
5500
6000
6500
7000
Acquisition Size 250 Initial Training Set Size
1000
Acquisilion Size
250 Initial Training Set Size
3000
3000
2000
1600
1000
500
PowerBALD
2000
3000
4000
5000
6000
7000
3600
4000
4500
5500
6000
6500
7000
SoftrankBALD
Acquisition Size 600 Initial Training Set Size 1000
Acquisition Size 500 Initial Training Set Size
3000
BALD
1500
800
Wrltime
800
590
200
0
3000
6000
7000
4500
8500
Acquistion Size = 1000 Initial Training Set Size 1000
Acquisitin Size
1000 Inidal Training Set Size = 3000
1250
800
Waltime
600
750
460
250
200
0
3000
7000
4500
6500
7000
Acquistian Size = 2000 Initial Training Set Size 1000
Acquisition Size 2000 Inidill Triining Set Size = 3000
1000
800
600
400
400
200
Training Set Size
Training Set Size
Figure 40: MFVT Last-Layer Classification on SVHN: Acquisition size ablation (Walltime).
42

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisitin Size Initial Training Set Size
20
Acquisiiion Size 25 Initial Training Set Size 100
0.8
0.90
0.88
0.8
0.82
0.4
0.80
150
200
275
Acquistion Size 50 Initial Training Set Size 20
Acquisition Size 50 Initial Training Set Size = 100
0.9
0.950
0.926
Accuraey
0.7
0.800
SoftmaxBALD
0.875
BofDrankBALD
0.850
ACS-FW
BALD
Uniform
Acquisitin Size Initial Training Set Size
Acquisition Size
100 Inisal Training Set Size 100
0.925
0.800
0.875
0.850
0.5
0.825
0.800
25
75
100
125
160
176
200
225
100
125
150
175
200
225
250
276
300
Training Set Size
Training Set Size
Figure 41: MFVI Last-Layer Classification on Repeated-MNISST: Acquisition size ablation.
43

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisition Size 25| Initial Training Set Size = 20
Acquisition Size 25 Initial Training Set Size 100
1500
1500
1000
1000
0
225
275
Acquisition Size 50 Initial Training Set Size
20
Acquisition Size
50 Initial Training Set Size 100
1000
800
600
SoftmaxBALD
SoftrankBALD
400
BALD
200
200
Unitorm
125
150
225
100
175
200
275
Acquis tion Size
Initial Training Sat Size 20
Acquisition Size
100 Initiaal Training Set
1200
600
1030
800
600
600
400
200
25
50
75
100
125
150
175
200
225
100
125
150
175
200
225
250
275
300
Training Set Size
Training Set Size
Figure 42: MFVI Last- Layer Classification on Repeated- MNISS: Acquisition size ablation (Walttime).
44

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisiiin Size Initial Training Set Size
20
Acquisiiion Size 25 Initial Training Set Size
100
Acouracy
0.72
0.70
0.68
150
200
275
Acquistion Size 50 Initial Training Set Size 20
Acquisition Size 50 Initial Training Set Size = 100
0.76
0.8
0.74
SoftmaxBALD
0.72
BofDrankBALD
0.5
0.70
BALD
ACS-FW
Uniform
Acquisitinn Size
Initial Training Set Size
Acquisition Size
100 Inisal Training Set Size 100
0.76
0.72
0.68
25
75
100
125
160
176
200
225
100
125
150
175
200
225
250
276
300
Training Set Size
Training Set Size
Figure 43: MFVI Last- Layer Classification on Moshion-MNIST: Acquisition size ablation.
45

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisitionn Size 25| Initial Training Set Size = 20
Acquisition Size 25 Initial Training Set Size 100
1200
1500
Waltime
800
1030
600
400
500
0
225
275
Acquisition Size 50 Initial Training Set Size
20
Acqulsition Size
50 Initial Training Set Size 100
1200
Waltime
800
600
SoftmaxBALD
600
SoftrankBALD
400
600
BALD
Unitorm
125
150
225
100
175
200
275
Acquis tion Size Initial Training Sat Size 20
Acquisition Size
100 Initiaa Training Set
800
500
400
400
200
200
100
25
50
75
100
125
150
175
200
225
100
125
150
175
200
225
250
275
300
Training Set Size
Training Set Size
Figure 44: MFVI Last-Layer Classification on Fashion MNIST: Acquisition size ablation (Walltime).
46

---

Published in Transactions on Machine Learning Research (09/2023)
F Effect of changing& G
F.1 Sypeated-MNIST
100
250
100
250
200
250
100
350
200
200
250
250
100
340
200
200
-00
200
200
400
200
200
200
305
350
100
300
127
305
fandle Ealto a hoedsstttee
Aupicin fastire SetneM (Peuliis falired
122
100
Figure 45: SespettedMMNSTT 3 ablation for *BALD.
47

---

Published in Transactions on Machine Learning Research (09/2023)
F.1.1 MIO-TCD and Synbols
335
C37
C39
079 035
1.0 135 0.0
8.5 100
0.55
6.00
045
8.50
0.05
1.0
0555 E.00 855 100 0.05 (583 631
2333
200
2300
2300
3000
1300
1900
35493
3989
3593
1900
Coneatore
SottroeRLD undertaceerr Grau
SoftrWORALD Maary
3.5
197
125
9.0
1.0
855 100
0.1
035
0.9
0355 555 kon 0383
2.31
7601
2503
2503
2032
3.00
3.80
5300
3991
1991
1990
1259
1259
1250
SofmentaeL 105-70 Acunyy
SefrendAAL Sayta Simbiee
SofrentaiD mw/eereeepeen Greze
SeframbOALD Nating Syrool
Ses
2433
5.99
3000
530
3200
1300
3250
32503
1250
Figure 46: MIO-TCD and Synbols: 3 ablation for *BALD.
ED 855 100
0.5
Pumerar-e CARD MAR
045
Poe Maragho Kil
2014
2533
250
2309
1.00
5009
3000
1300
1900
1200
1259
1230
JrEnasdraupy POTD
Subradrr US (edeedapeeenn Grai
suftraadstigyc Mar RYO0A
3.00
3000
2.39
250
750
10"
1250
1250
1250
1500
Seftranceetryy ME Haiing Sirtee
2530
7500
730
7500
3230
1259
12309
3003
3500
1909
Figure 47: MIO-TCD and Synbols: B ablation for *Entropy.
48

---

Published in Transactions on Machine Learning Research (09/202))
F.2 EMNIST
Acquisition Size
Acquisition Function
PowerBALD
Acquisition Size Acquisition Function SoftmaxBALD
0.0
0.1
0,2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Min Training Set Size
50
50
100
100
150
150
250
250
300
300
Acquisition Size
20 Acquisition Function PowerBALD
Acquisition Size 20 Acquisition Function SoftmaXBALD
0.1
0.4
0.5
0.7
0.2
0.5
Min Training Set Szz
150
150
200
200
250
250
300
300
Acquisition Size
40 Acquisition Function PowerBALD
Acquisition Size 40 Acquisition Function SoftmaxBALD
Accuracy
Accuracy
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Min Training Set Size
50
100
100
150
150
200
200
250
250
300
Figure 48: EMNIST 'Balanced'; B ablation for Softmax- and PowerBALD.
49

---

Published in Transactions on Machine Learning Research (09/2023)
Acquisition Size
10 Acquisition Function
PowerBALD
Acquisition Size
10 Acquisition Function SoftmaxBALD
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Min Traiiing Set Size
50
100
100
150
150
200
200
300
300
Acquisition Size
Acquisition Function PowerBALD
Acquisition Size Acquisition Function SoftmaxBALD
0.1
0.4
0.7
0.5
Min Training Set Size
50
100
200
200
250
250
300
300
Acquisition Size 40 Acqulsition Function PowerBALD
Acquisition Size Acquisition Function SoftmaxBALD
Accuracy
Accuracy
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Min Training Set Size
100
100
150
150
200
200
250
250
300
300
Figure 49: EMNIST 'ByMerge': : B ablation for Softmax- and PowerBALD.
50

---

Published in Transactions on Machine Learning Research (09/2023)
F.3
CausalBALD: synthetic dataset
Uniform B 0.01
==1
B 100
8-4
Top-K
2.5
2.0
EEPEHE
100
200
300
Training
1.5
(b) Low Temperature Only
1.0
2.5
P-118L
2.0
1.5
0.5
1.0
50
100
150
200
250
300
0.5
Training Set Size
50
100
150
200
250
300
Training Set Size
(a) Overall Ablation (Subsel)
(c) High Temperalure Only
Figure 50: CausalBALD: Synthetic Dataset. (a) At a very high temperature (3 0.1), PowerBALD behaves very much like random acquisition, and as the temperature decreases the performance the acquisition function improves (lower V(PEHE). (b) Eventually, the performance reaches an inflection point (B 4.0) and any further decrease in temperature results in the acquisition strategy performing more like top-K. We see that under the optimal temperature, power acquisition significantly outperforms both random acquisition and top- K over wide range of temperature scttings.
We provide further 3 ablations for CausalBALD on the entirely synthetic dataset which is used by Jesson al. (2021). This demonstrates the ways in which interpolates between uniform and top-K K acquisition.
F.4 CLINC-150
PoworEntropy Accu/DCy
SoftmaxEntrrepy
Softrankt stropy
Accur-oy
Accurocx
2000
2000
2000
0.125
1.5
8.0
6000
6000
6000
Figure 51: Performance CLINC-150; 3 ablation for *Entropy.
G Simple Implementation of Stochastic Batch Acquisition
51

---

Published in Transactions on Machine Learning Research (09/2023)
def sampleessscores_..... torch. Tensor, aquisition_bacch.eo. batchssiaot int):
N len (scorees__])
aquisition_batiee batchann_aah_h__ee size - min(aquisition_ secpan_attc____eeeeeeeeeeee N)
indices www..a..cooiee..e..... choice lizeraquisitionnbaacc. batch_bbtcch_ttee replace=False)
return pudices.tolisse.
def get_ tor_xxfmaax toofttax_ammleeeeeeeee (scole N: terch. Tensor, beta: float, pquisition_batch.eee size: int): beta 0, obtain random sampl ing..
beta 0.0:
get_randon cet_rraddo__sapleeeeeeeeee samples N, aquisition batch_ btt.cccceeee-euuiiittioo..o size)
len (scores_M)
noised_scores_t scores_N scipy stats wttpy.ssttt.uuuoaa... r scale=1 beta, size=N, random_state=Nonee)
scipy.stata
return torch. torch.topk topk(moiee__scorre keaquisition_bataal baciion__atc_sizss
def get_pover_ saco_sss _ Tensor, *, beta: float, pquisition_batch_eee int):
return get_softmax,_e httttttaaaaaaeaaaeeaaaaaee (torch. log(scores__)) _I), beta-beta, aquisition batccbbattthhcceeeeeeeee siaeciiiizeaiiiiiiiinnn batch_size)
def www.rark.kkk..... cscoresss I:: torch. Tensor, beta: float, aquisition_ siiiiitionbbatcheeize: int):
N len (scores_M)
sorted indices_ torch argsort (scores_M, descending=frue)
ranks_ N torch. argsort (sorted indicea_M)
get_ppower___eeee ees_poxer_ammplea
ranke_l, N, bets-beta, htw.:aawccc.......uuiiittoo.o.ao
Listing 1: Code for stochastic batch acquisition. Colab here.
52

---

Published in Transactions on Machine Learning Research (09/2023)
H Hypoexponential Distribution Evaluation
import numpy as up
from scipy.stare stats import expon, gumbel_r import wetloottib.ppppp... as pit
Set the rate parameter Change here to get different scales of eaponentials lan 1/2
Generate 10 samples from the emponential distribution samples expon (vaaaaae=1//an,, size=5000)
sample le from samples using the previous samples lamdda of eaponentsal distributions new_samp samples expon. rva(scall=//samplee, size0(40),)0000)
that_are_you new_samples (exn(xxis=))
Fit a Gumbet distribution to the data params gumbel_r.fit (shat_are_you)
np. ppp.iispace (up. wwwnnuatttreeeeeee gumbel_cdf gumbel_r.cem _ cdf (x, *params)
waaaaxxaattarre_yooe 1000)
Plot both what yre_you and the Gumbel distribution in the same plot
alt.ppottie (whtt aatpatt_rre_ooo,,, up. linepace(O, len (vhat_are__ou), endpoint-Falee), label-"Simulted" ")poexponential.") Aet.ppoot (x, gumbel_edf, label="Gunble Fie")
legend))
show()
1.0 Simulated Hypoexponential
Gumble Fit
0.8
0.6
0.4
0.2
0.0
5
Listing 2: Code for fitting Gumbel distribution to the mean of samples from a hypoczponential distribution. Colab here.
53


Reviews:
[{"review_id": "DfvxjTw9mK", "reviewer": "Reviewer_uz6d", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. Challenging the conventional wisdom in batch active learning is both important and interesting. This paper has the potential to act as a stepping stone for the development of better theory and methods for batch active learning. \n\n2. The paper advocates for a simple but effective baseline which is underutilized in the literature and likely in practice too.\n\n3. The paper provides an interesting discussion of 3 different methods for constructing a distribution from which to sample from the pool set given scores. The power and soft-rank distributions are novel, as far as I am aware.\n\n## Weaknesses \n\n1. **Clarity:** the paper is unclear in several places; see the detailed list under \"Requested Changes\". The main issue in this regard is the connection between sampling from the acquisition distributions and the idea that scores at time $t+1$ are the scores at time $t$ with some perturbation applied. This idea is mentioned a few times in the paper but is not clearly described or obvious to me.\n\n2. **Evidence:** not all claims are well supported by evidence. I believe that most of these issues can be addressed by adding citations. However, one issue I am concerned about is that the paper only uses BatchBALD and BADGE as examples of batch acquisition strategies. The problem is that these are *extremely* computationally expensive (either in memory or compute time). However, there exist well-performing batch acquisition strategies that scale much more gracefully. In particular, ACS-FW (with random projections) from Pinsler et al. (2019) scales linearly with pool-set size. (The authors also mention in the related work that methods exist for improving the scaling of core-set-based methods). Without including comparisons to some batch acquisition methods with better scaling, and thereby providing a more comprehensive picture of such methods, I do not believe that the paper's (interesting and potentially important) question about whether or not batch acquisition methods are \"pulling their weight\" can be taken seriously. \n\n3. **Positioning:** the positioning of the paper is, in my opinion, somewhat strange and misleading. In particular, the idea of sampling from a distribution based on active learning scores in order to improve diversity in batch acquisition might not be well-known (and could definitely benefit from some advocacy) but has certainly been used before. It is a straightforward implementation of Boltzmann exploration to the active learning setting. Yet the paper presents this as a novel method, frequently referring to it as \"our method\", and the second claimed contribution of the paper is the demonstration that the sampling strategy improves on naive top-K acquisition. The specific implementations using the power and soft-rank distributions are novel as far as I know, but this paper is not clear enough in disentangling what is novel and what is not. (To be clear, I am not criticising this paper for lack of novelty. I am aware that this is not a criterion for acceptance in TMLR. Furthermore, I think there is plenty of novel/interesting content in the paper. However, the paper should be clear about its contributions and the sources of novelty.)\n\n\n\n\nRobert Pinsler, Jonathan Gordon, Eric T. Nalisnick, José Miguel Hernández-Lobato:\nBayesian Batch Active Learning as Sparse Subset Approximation. NeurIPS 2019: 6356-6367", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper shows that complex batch acquisition strategies commonly used in active learning provide comparable predictive performance to the simple baseline of stochastic sampling from a softmax (or other distribution) based on the scores of the pool set, despite the latter strategy providing a vastly reduced computational complexity. This result calls into question the conventional wisdom in the field – that sophisticated batch acquisition strategies are worth their additional computational cost. The paper also discusses 3 different methods for constructing the sampling distribution, each making different assumptions about the scores. The paper shows that the power distribution performs slightly better than the softmax and soft-rank distributions for commonly used active learning score functions. ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I have grouped my requested changes into \"Major\" and \"Minor\", which are the changes which are critical for securing my recommendation and those that would simply strengthen the paper, respectively. \n\n## Major changes\n\n1. Please clarify the following. While some of these are minor issues in isolation, they add up to make the paper significantly less clear than it could be.\n    1. What does figure 1 have to do with the statement \"... and it was assumed deep learning models hardly change after adding a single new point to the training set.\"? Is the large change in rank correlation at the initial acquisition steps being referenced here? \n    2. Some discussion should accompany equation 8 to highlight why this is a naive strategy. Figure 1 from the BatchBALD paper does a great job at this. A figure isn't necessary, but some discussion would be very helpful. \n    3. How does the $\\beta$ parameter of the softmax correspond to the \"expected rate at which the scores changed as more data is acquired\"?\n    4. After equation 9, it isn't obvious how taking the top-k samples corresponds to sampling from the soft-rank distribution. \n    5. Regarding power acquisition, what is meant by \"ideally\" in \"This assumption holds ideally for other scoring functions ...\"?\n    6. Regarding power acquisition, how does the notion of time (e.g., \"future log scores\"/\"current log scores\") relate to equation 11?\n    7. Similarly, in section 6, the assumptions regarding rank correlations across acquisitions were really not made clear to me.\n\n2. Please add at least a few experiments comparing to Pinsler et al., as discussed above. (Pinsler et al. should also be discussed in the related work). \n\n3. Please provide citations (or other demonstrations/evidence) for the following statements:\n    1. In section 2, regarding entropy scoring, \"... performs worse for data with high observation noise ...\".\n    2. In section 2, regarding acquisition functions, \"... and it was assumed deep learning models hardly change after adding a single new point to the training set.\" Who assumed this? \n    3. In section 3, \"... which is frequently used for modelling extrema.\".\n    4. In section 5, \"... BatchBALD, which was SotA for small batch sizes.\".\n    5. Similarly, in the caption for Figure 6, BatchBALD is mentioned as being SotA without a reference. \n\n4. In figures 2, 3a and 3b, why does PowerBALD use an acquisition size of 10? This seems like it might make the comparison between it and BatchBALD unfair. Can you provide results for PowerBALD with an acquisition size of 5? And/or an ablation for the acquisition size for PowerBALD like was done for BADGE in figure 14? In general, I also think that the choices of hyperparameters should be better motivated, and any sweeps done should be described.\n\n5. Similarly, why was the acquisition size for figures 3c and 3d raised to 100? My concern is that with a larger acquisition size, the impact of low diversity in the acquired examples can be mitigated. Would it be possible to see these results for an acquisition size of, say, 10?  \n\n6. It should be mentioned that for the CLINIC results, whose figure is somewhat hidden in the appendix, PowerEntropy does perform slightly worse than BADGE.\n\n7. This might be a big misunderstanding on my part, but in Figure 4, it is not clear to me that the \"Top\" and \"Bot\" lines should be so similar. In fact, I think this contradicts assumption (3) – aren't the bottom 1% or 10% supposed to be less informative, so shouldn't the change in correlation in these two cases be closer to, if not even worse than for 100%? Could you clarify this? This currently leaves me somewhat sceptical of the results of these further investigations. Perhaps providing the additional experiment (just on MNIST, for example) that K can indeed be increased would provide additional evidence here. \n\n8. An important missing sensitivity study is to use different BNN methods. Currently, the paper only uses MC Dropout for the BNN, so it is unclear whether these results generalize to other BNN approximations. In particular, I think that the linearised Laplace approximation (e.g., see Daxberger et al. (2021)) would be a great addition. Results for a single dataset would suffice.\n\nErik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, Philipp Hennig:\nLaplace Redux - Effortless Bayesian Deep Learning. NeurIPS 2021: 20089-20103\n \n## Minor changes\n\n1. After equation 2, I would suggest saying, \"... the index set of $\\mathcal{D}^\\text{train}$ **initially** containing ...\". \n\n2. The footnote on page 2 should be removed. \n\n3. In section 2, provide the expansion of the acronym for BALD as it helps provide intuition behind the method.\n\n4. In section 2, provide brief descriptions of BatchBALD and BADGE since these are the most important baselines in the experiments. \n\n5. Regarding the soft-rank acquisition, to be clear, there is only invariance insofar as the ranking us unchanged when the scores change. \n\n6. Since BADGE and BatchBald are (understandably) not included in every figure, it would be helpful to always include \"Uniform\", so that the improvements over Bald from the stochastic aquisiton have a consistent point of comparison. \n\n7. In the summary for section 5, the papers listed are not exactly \"new\", two being from 2021.\n\n8. It would be great to see the $\\beta$ ablation for the EMNIST dataset too.\n", "broader_impact_concerns": "No concerns. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nz3TW2yT3k", "reviewer": "Reviewer_TQqT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\nHow to properly extend acquisition functions to batch settings is an important problem in making active learning more practical, especially as model-training times grow larger for today’s large neural networks. \n\nMany of the methods previously published in this area are highly complex and computationally intensive.\nPapers like this one which show that a straightforward and cheap-to-implement/compute baseline outperforms large numbers of complicated publications serve as important reality-checks for the field. These are usually the papers with methods that end up actually being used in applications (eg. in industry/science).\n\nOverall the method is clearly presented and the writing is easy to understand.\n\nThe experiments cover many different datasets.\n\nWeaknesses:\n\nThe experiments do not compare against many other batch baseline methods. There have been many recent batch active learning methods published that could also be compared against. That said, I do find the existing experiments sufficient since they cover so many different datasets.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a simple way to extend any single-datapoint acquisition function for active learning to batch settings, where many labels are collected at once in a particular round before the model is re-trained.  The method is to simply sample from a distribution that upweights the datapoints with the highest acquisition values. This naturally encourages greater diversity than simply choosing the datapoints with K highest acquisition values, yet is not more computationally intensive. \n\nI find the proposed method highly practical and the empirical experiments on many image datasets and some text datasets convincing. Everything looks correctly done as far as I can tell.\n\nThe one thing I am not sure about is whether this or similar methods have been previously proposed, given how straightforward they are.  ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "If you want to present a more convincing case, consider including other recently-published batch active learning methods in your experimental comparisons. That said, I am already convinced that the proposed method is a good idea by the current experiments, especially given the method’s simplicity. \n\nAdding more active learning methods would help convince me that your method truly matches the state-of-the-art. For instance, methods which use stochastic acquisition with explicit encouragement of diversity such as:  choosing top acquisition values sampled from different clusters of the data, or using the acquisition values and data features to define a determinantal point process distribution.\n\nBatch Active Learning Using Determinantal Point Processes\nhttps://arxiv.org/abs/1906.07975\n\nRelated Work should mention other related ideas like “kriging believer”. That is another very simple approach to extend single datapoint acquisition values to batch settings, but does still involve a lot of model retraining to compute the batch scores.\n", "broader_impact_concerns": "I don't have any broader impact concerns about this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eaFF82nPcR", "reviewer": "Reviewer_NuWJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\nThe approach provides a simple and computationally cheap solution to the task of batch acquisition. It is well-motivated and evaluated on a large array of different data sets, where the approach can provide consistent results. The paper is well written.\n\nAs the authors acknowledge in the related work, similar stochastic approaches appear repeatedly within the active learning literature often hidden in experimental sections and appendencies as approximate heuristics that seem to help without being evaluated on their own. The contribution is therefore minor in the sense of coming up with the proposed idea. But given that, it is even more valuable as being a proper study that finally evaluates these approaches with surprisingly strong results. \n\nGiven these results presented in this work, the challenge to the community of providing a model that can improve upon this heuristic is indeed justified.\n\n\n## Weaknesses\n- The reference section is a mess. A lot of references simply have a year without further information on the venue, while others do. Provided venues vary a lot, e.g., NIPS, NeurIPS, Advances in Neural Information Processing Systems. Please reformat all references into a consistent style.\n\n### Typos\n- Fig 3: Plot sizes and font sizes vary a lot.\n- p8 last line: the ranks of the top-**1%** scoring...\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors consider the task of batch acquisition in active learning, i.e., acquiring multiple labels at a time, rather than single ones as is done in many popular approaches. Methods such as BatchBALD (Kirsch et al., 2019) or BADGE (Ash et al., 2020) are designed to model this task properly, while the simplest heuristic would be to take the top-K points at each labeling round as proposed by the acquisition function. The authors discuss and demonstrate the flaw in this naive heuristic (its assumption on the stability of the ranking between labeling rounds) and instead propose a simple sampling-based approach that performs as well as/better than methods designed for batch acquisition while remaining at the computation cost of top-K acquisition. The approach is tested extensively in a variety of experimental settings and ablations. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses.\n", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers found the contribution of this paper to be useful: a simple strategy can work as a convincingly effective baseline for batch mode active learning. The AE concurred and recommended acceptance. 

In addition to incorporating the reviewers' comments in the final version (if not yet), the AE also suggests the following revisions in the final version:

(Main suggestions)
- In the comparison plots, using the same K for all algorithms except for BatchBALD (and clarify the computational challenges & anticipated performance decline of running BachBALD with the same K).  

- Clarify the choices of K in all experiments for all algorithms

(Minor comments)
- Is it possible to clarify the motivation of the Gumbel perturbation on the acquisition scores in pages 5-6 more? If I understand correctly, the last paragraph of page 5 says that, for choosing the k-th point in the batch, ideally we would like to model the information gain of this k-th point (relative to the batches of examples queried so far and the k-1 examples already selected in this batch) assuming that the remaining K-k points are selected optimally. The paper then mentioned the maximum of a set of exponential random variables; however I had difficulty in understanding what are the iid exponential random variables in the context of batch active learning. It might be better to remove the discussion if this connection is not so precise..

- could you clarify why in Eq. (17), log M is greater than the mutual information expression? M depends on the number of samples, where as the mutual information expression does not?

- could you provide a mathematical expression of the AUROC evaluated in Figure 7?

- In Figure 9(b), why is it called a near-duplicate? Is it because it is very similar to Figure 9(a)?

- typo in B.2: Gumbel -> Gumble 
 
Decision: Accept (poster)