Gone Fishing: Neural Active Learning with Fisher Embeddings
Authors: Jordan T. Ash
Surbhi Goel
Akshay Krishnamurthy
Sham M. Kakade
Venue: NeurIPS 2021 Poster (2021)
Keywords: deep active learning
deep learning
active learning

Abstract: There is an increasing need for effective active learning algorithms that are compatible with deep neural networks. This paper motivates and revisits a classic, Fisher-based active selection objective, and proposes BAIT, a practical, tractable, and high-performing algorithm that makes it viable for use with neural models. BAIT draws inspiration from the theoretical analysis of maximum likelihood estimators (MLE) for parametric models. It selects batches of samples by optimizing a bound on the MLE error in terms of the Fisher information, which we show can be implemented efficiently at scale by exploiting linear-algebraic structure especially amenable to execution on modern hardware. Our experiments demonstrate that BAIT outperforms the previous state of the art on both classification and regression problems, and is flexible enough to be used with a variety of model architectures.

Gone Fishing: Neural Active Learning with Fisher Embeddings
Jordan T. Ash Microsoft Research NYC a jot.joordd@@microoofff com
Surbhi Goel Microsoft Research NYC
goel surbhi werbbii@iiioooo www.eiiootttcommmmmmmmmmmmmom..c com
Akshay Krishnamurthy Microsoft Research NYC akshaykr@mi crosoft.......ooooooooooooooo... com
Sham Kakade
Microsoft Research NYC
University of Washington
sham kwwade@miaiii.e.a. crwsoft..ooooooooooooooooooooo.e com
Abstract
There an increasing need for effective active learning algorithms that are com- patible with deep neural networks. This paper motivates and revisits a classic, Fisher-based active selection objective, and proposes BAIT, a practical, tractable, and high-performing algorithm that makes it viable for use with neural models. BAIT draws inspiration from the theoretical analysis of maximum likelihood esti- mators (MLE) for parametric models. It selects batches of samples by optimizing a bound on the MLE error in terms of the Fisher information, which we show can be implemented efficiently at scale by exploiting linear-algebrac structure especially amenable to execution on modern hardware. Our experiments demonstrate that BAIT outperforms the previous state of the problems, and is flexible enough to be used with a variety of model architectures.
both classification and regression
Introduction
The active learning paradigm considers sequential, supervised learning scenario in which unlabeled samples are abundant but label acquisition is costly. At each round of active learning, the agent fits its parameters using available labeled data before selecting batch of unlabeled samples be labeled and integrated into its training set. A well-chosen batch of samples is one that is maximally informative the learner, such that it can obtain the best hypothesis possible given a fixed labeling budget.
Active learning is well established as an area of machine learning research due to the ubiquity of impor- tant real world problems that fit the sample-abundant, label-expensive setting; commonly cited applica- tions range from medical diagnostics [1 2 to image labeling 3 Mitigating large sample complexity requirements is particularly relevant for deep neural networks, which have in recent years achieved impressive success on a wide array of tasks but often require considerable amounts of labeled data.
Shifting the focus of active learning to deep neural networks highlights several important problems. For one, most foundational active learning work assumes a convex setting, which clearly violated by massive nonlinear neural networks. Many of these approaches are computationally expensive, and it is not clear how to adapt them for real-world use 4 Further, because neural network training is
generally expensive, practical active learning algorithms must be able work in the batch regime, querying B samples at each round of active learning instead of a single point at a time [5]
Despite long history of active learning research, these constraints draw attention to need for practical, principled batch active learning algorithms for neural networks. Current of of-te-of-the-ar the-art methods, like Batch Active Learning by Diverse Gradient Embeddings (BADGE), perform robustly in experiments, but explanations for behavior are fairly limited 6 This drawback makes unclear how to scale some active learning algorithms into regimes that deviate somewhat from the setting for which they were designed-BB Bereed-BADGGR BADGE, for example, cannot run on regression problems, and as we show in this paper, performs poorly when used in conjunction with convex model.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

This article adopts probabilistic perspective of neural active learning. We view neural networks as specifying a conditional probability distribution pyy z, z.6) over label space Y given example X' where 0 are the network parameters. This perspective provides theoretical inspiration from the convex regime with which to examine and design neural active learning algorithms. From this viewpoint we motivate and revisit a classic, Fisher-based objective for idealized active selection. We argue that approximately minimizing this objective can be done tractably in the neural regime, despite their overparametrized structure and shifting internal representation. Accordingly, this work helps bridge the divide between algorithms that are performant but not well understood by theory, and those that theoretically transparent but not computationally tractable.
Experimentally, BAIT offers improved performance over baselines in deep classification problems, a trend that is robust to experimental conditions like batch size, model architecture, and dataset. Crucially, BATT is general purpose, and can be easily extended regression settings, where many other algorithms cannot. It further performs well on both regression and classification with convex models, a paradigm in which other algorithms often struggle.
In summary, this paper
puts neural active learning on firm probabilistic grounding, giving a new, rigorous perspective on the functionality of previously proposed algorithms.
provides in-depth empiries that elucidate differences between neural and convex regimes, and discusses simplifying assumptions that are sometimes reasonable in the neural case.
proposes a practical, unifying, high-performing active learning algorithm that leverages these insights in computationally tractable manner.
2 Related work
Active learning is very well-studied problem 77. There are two main sample selection approaches, diversity and uncertainty sampling, which are successful respectively for large and small batch sizes.
Diversity sampling strategies aim to select batches of data that best represent the space. In a deep learning context, these algorithms typically embed unlabeled samples using the neural network's penultimate layer and select a subset of samples that might act as proxy for the entire dataset [[ 12 proposed inducing batch diversity using a generative adversarial network formulation, selecting samples that are maximally indistinguishable from the pool of unlabeled examples.
There also rich body of work on batch active learning 13 177
These methods typically formulate batch selection as an optimization that minimizes an upper- upper-bound on some notion of model loss.
Efficiently adapting parameters to incrementally larger training set is not an issue in convex settings. Accordingly, with linear models, more common to use uncertainty sampling and a batch size of one. A frequently used approach is to query samples that lic closest to the current model's decision boundary, a quantity that's considered inversely proportional to uncertainty Some similar methods offer theoretical guarantees statistical consistency p 21. Other algorithms quantify uncertainty using the entropy of the predicted distribution over classes, or as the size of the expected gradient induced by observing the label corresponding with candidate sample [22]] The latter is known to be related to the Tootimallty criterion in experimental design, but is unable to account for batch diversity.
Similar approaches have been modified for use with neural networks as well. For example, exercise Dropout to sample weights to approximate the posterior distribution over labels, and use it to identify samples that reduce model uncertainty. Adversarial example generation has been used to approximate the distance between sample and the decision boundary 24 Model ensembling has also been used to approximate sample uncertainty, where the predictive variance across constituent models can be used to inform a sample selection strategy
23
2
There are a variety of algorithms that are meant combine uncertainty and diversity sampling 226 This trade-off is sometimes framed as its own optimization problem, for example using meta- learning approach that hybridizes both strategies [27]] 28 Among these active learning by learning, which uses a bandit approach to select which query rule to employ at any given round of active learning (28). BADGE, described in detail in Section 5.T also combines uncertainty and diversity
sampling, and is considered state-of-ther thr for deep neural networks.

---

3 Notation and Setup
We consider a standard setup for batch active learning with neural network models, where there is an instance space X, label space }/ and a distribution D over X' x y. We use Dx to denote the marginal distribution over the instance space and (r) to denote the conditional distribution over labels given example æ. For learning, we are given access to pool U { )
Dx of unlabeled examples and we have the ability request the label for any point E U. In the eth round of batch active learning, we select collection (#) ( ر C of examples (B is the batch size) and request the labels Vj Dy ((x)) for all examples in the batch. We use these labeled examples to update our neural network model and then we proceed to the next round.
In this setup, the ultimate objective is to achieve low loss on the data distribution D, that is we hope our learned parameters i nearly minimize Efz,,)--((x, 8), where é is some loss function like the cross entropy loss for classification. We always consider this objective in our experiments, but for algorithm development it is helpful to instead consider the fixed-design or transductive setting, where the goal is instead minimize (0) Ezzu x)))))
(=)))(( (2x, y; 8), essentially treating the unlabeled samples U as the entire distribution. Note that these two objectives can typically be related by generalization arguments.
Probabilistic Perspective
We consider neural networks as specifying a probability distribution p'y xr, O) over the label space y given an example vr where are the network parameters. Adopting this view, it is most natural to use the loss function 6(x, y; 0) = logp(s p(y æz, 0). choosing parameters that maximize the likelihood of observed labeled data. In classification problems, for example, we apply the softmax operation the output of the network and then evaluate the cross-entropy loss with the ground truth label. For regression problems, we use the square loss, which treats the neural network as specifying a Gaussian distribution for each x.
Bayesian linear regression. As a warm-up, is illustrative to consider an experimental design setting with Bayesian linear regression. We consider d-dimensional linear regression problem where we assume the parameter vector 0* has prior distribution N (N X -- and the conditional distribution (z) p(- xr 0*) = _N((0,, .x), o²) is Gaussian. For any set of labeled data {zg M) 3 the resulting maximum a posteriori (MAP) estimate is given by ridge regression with regularizer Ao?:
i argmin Σ (1),0) n,) + 200210012
(1)
In the experimental design setting, we have unlabeled data U = { and our goal is to select set S U of B points that the resulting MAP estimate has the lowest Bayes risk. Letting
= Σ N..;" denote the second moment matrix of the unlabeled data, the Bayes risk is
BayesRisk(S)) = E (0s T 0r]
(2)
where As is the MAP estimate after querying for labels on subset S, and the expectation is with respect the noise in the labels and the prior over 6*.
Lemma in the Appendix shows that for a subset 5 c U, letting As = Exes x. + + the Bayes risk in this setting is exactly:
BayesRisk(S) o² (r(A5' Σ).
Observe that the RHS does not depend on labels, implying that minimizing the RHS over subsets S feasible and optimal selection strategy under this criteria. This also verifies that multiple batches of active learning not required for Bayesian linear regression, although this observation does not carry forward to the neural setting. BAIT is designed to approximately minimize this objective.
3

---

Classical regime. An objective similar to B also emerges naturally in the analysis of maximum likelihood estimators (MLE) in the convex regime. Here, classical statistical theory posits that the model is well-specified, so that there is some parameter 0* such that Dyjx(()) pl- x, 0*) for each x E X. It is also common to impose regularity conditions including strong convexity of the loss function T [29. 30]. While these conditions certainly do not hold in the neural setting, BAIT builds on much of this classical technology. The key quantity is the Fisher information matrix [x; 0) == Eyr-pf-Iz.00 Iz.0) ,) V²((2z y; 0) which is known to determine the asymptotic distribution of the maximum likelihood estimator [29] In many probabilistic models, including linear and logistic regres- sion, the hessian of the loss function does not depend on the label y, which assume going forward.
For active learning in the classical setup, 44 give two-phase sampling scheme with provably near-optimal performance. In the first phase, the algorithm samples a batch of B points uniformly at random, requests their labels, and optimizes the tog-likelihoon to obtain an initial estimate 0j. In the idealized version of the second phase, a batch of B points is chosen to optimize
T[00001 ((I10s50)) WES
argmin tr ScU.ssie
To(Oi)
(4)
where (01) is the Fisher over all samples, Eney (2; 01). This combinatorial problem is intractable in general, so 4 instead solve semidefinite relaxation (SDP). They request labels on the obtained batch B of points and re-fit the model to obtain the final estimate f. For their setup, they prove the statistical properties of this two-phase estimator are near optimal.
Despite the theoretical properties, solving SDP is not feasible in high dimensions: instead provide a new, greedy algorithm for minimizing the objective that is usable in the neural regime. Several other works have also looked at this objective, either from purely theoretical perspective [4] (2 34 However, some of these do not ensure batch diversity, and none have been extended to the neural regime.
B1 or via relaxations
since in linear regression the Fisher information [a;; 0) This formulation essentially generalizes 3
the covariance matrix zx /o2: 2: the objectives in 3 and ( differ only in their use of the regularizer controlled by X. That is, essentially the same objective can be derived from two different perspectives, making a compelling object for active selection. As such, our starting point for the neural setting the ideal-bur wws--buut---ttte intractable optimization problem in 4
5 BAIT
Batch Active learning via Information maTrices (BAIT) is inspired by this theory, but adapted to the sequential, neural setting. To do this effectively, several key issues need to be addressed:
1. For neural models, the pointwise information matrix [a;; 0) is typically extremely large.
2. The internal representation learned by the network changes with each round of active learning, computation from previous rounds cannot be reused.
3. Solving the objective in Equation 4 suggested in more theoretical work, is computa- tionally infeasible 4
Outlined as Algorithm D BAIT addresses item in a somewhat standard way, by operating on the last layer of the network [6 [O). We consider last-layer Fisher matrices [(x; 8²) := E yy>pp(-1a0)) V2((x, y/ 8²)) for last-layer parameters g², Note that in the linear setting 8² = 8. Here, the top-layer representation starts to vell-appro a convex model, then the information geometry induced solely by these parameters can guide active sampling. Further, as we discuss shortly, this top-layer framework gives us more principled understanding of the empirical success of BADGE.
One more subtle issue (item 2) is the interplay between the changing representation as learning progresses. We address this with an iterative scheme, where the Fisher information matrix is continually recomputed as the algorithm changes its representation during the course of learning.
Rather than solving an SDP, BAIT approximates a solution to Equation 4 using a greedy approach, which we show can be made tractable in both classification and regression settings. At each step of

---

Algorithm BAIT
Require: Neural network f(x; f), unlabeled pool of examples U, initial number of examples Bo. number of iterations T, number of examples in a batch B.
1: Initialize S by drawing By labeled points from U & fit model on S: 0, - argming Es[((r, y; 0)]
2: fort = 1,,2... .... T: fforward greedy optimization) do
3: Compute I(e 2 EnEU [x; 0
4: Initialize Mo = XI
Σ
L I 0()
5: for 1.2.... 2B: do
6: T= = argmingett tr((M; + (x;0f)) --1))=1((001)
7: M1+1 + M, + (z;; 0f). S t
8:
end for
9:
1... B: bbackward greedy optimization) do for i = 2B, 2B - -, ....
= argmin (e (M: 1 (x; 0.-) ((()))
11:
M,-1 M₁ I [2: f.
S\
12:
end for
Train model on S: 0, argmin, Es[ y: 0).
14:
end for
15:
return Final model 0T+1-
the algorithm, the key computation lies in evaluating
f
argmin tr((M; [(x; ((x;6/))~'((()
(5)
zEU
where M, is the Fisher corresponding to samples that have been selected so far.
Unfortunately, the trace function is not submodular, and is thus not well suited for standard greedy optimization. To address this, during each iteration, where the goal is identify B points to query, sampling is done in two stages. For batch of B points, the first stage greedily oversamples, adding 2B samples to the initial batch. In the second stage, BAIT prunes B samples from the batch, better minimizing the objective described in 44) We find that this forward-backward strategy sometimes improves performance over the forward-only alternative (Figure I See Algorithm | for details. Choosing two as the oversampling factor of two done for computational reasons, trading- off between computational cost and batch quality. We did not see performance improvements for larger multipliers.
When evaluating the i-th sample to include in S, the minimization in Equation 5 is effi- ciently computed using a trace rotation and the Woodbury identity for low-rank inverse updates:
(M. V2V.. T(OF)
argmin
MNIST 10 mlp
argmintr tr (M, - -M; MM.VVa VV.A-'V M, 1(O5))
0.86
argminn ur (M,T((0')) ir (MT'V V.A V.A-'V,
V;J MT 1(0))) argmin ((0.) - ir (V."M.;"' M, 100) )M, \V.'V.AA)
Accurscy
0.84
Forward Greedy Backward Groedy [BAIT)
-argmaxt tr (v. M, I(OF)) (vo0)) VoA-
0.78
Forward Greedy
10
15
20
25
30
35
40
45
Active Learning Round
where A = T VT M. is an easily invertible k x k matrix. Here Vz is a dk & matrix of gradients, where each column is scaled by the square root of the corresponding prediction: V22 V. = [(x, 0*). This formulation keeps us from having compute and store all candidate (xx; 0²), drastically decreasing the algorithm's 's memory footprint. The trace rotation step, placing V₂ the leading term instead of M,- is essential, as it avoids computing new kd x kd matrix for each æ. As a practical matter, on all datasets we consider Section g this allows us to compute the trace contribution of all candidate samples simultancously on modern GPU.
Figure 1: A comparison between for- ward and horward-backwarr greedy ap- proaches for BAIT. Here we show a sim- ple active learning experiment using an MLP and MNIST data 355), and samples are acquired in batches of size 10 for 50 rounds. See Section X for more details.
5

---

Learned Embedding
Random Embedding ng
Gaussian
Orthonornna
1.0
Tracetffaler
Trace-iisher
Log Det
0.8
Log Dut
Bayes Riox
10.3
0.4
0.5-
0.5
BAlT (forward only)
04
0.4
0.4
Rank--nne Dee
Full-Rarkc
0.3
0.2
4
50
25
300
Number Labeled Samples
Number of Labeled Samdles
Batch
Batch
Figure 2: Linear classification on differ- representations of MNIST data. Left: Learned features, similar those from neural network. Right: A random, un- informed projection, simulating the raw
Figure 3: Bayesian linear regression simulations com- paring BAIT and determinantal maximization. In both cases the data have poorly conditioned covariance matri- ces with quadratic spectral decay. Determinantal maxi- mization exploits this in the Gaussian case, but not the orthonormal case. BAIT performs well in both settings.
features a convex model may have to use.
After the minimizer x is found, updating M, is done simply via the same Woodbury identity, M = M, M, V.A V M,¹ and the algorithm proceeds to identify the next sample.
Regression. In the regression setting we are able to further reduce the amount of required computation. Let x² denote the penultimate layer representation induced by f(x; 0). For linear models x² r. In Appendix Section A.3 we show that a k-output regression model trained to minimize squared error has pointwise Fisher Ixx;0") (xr))x (x")T @ E-1, wwhre Σ is the noise covariance of the estimator. Using this fact, the regression version of the Fisher objective is
Σ ((e EES
Io 10(00%) Atr (5
(Aaa(at)) (a0))
Σ (E WEU
(6)
The full derivation can be found in Appendix A.3.1 This observation greatly simplifies the mini- mization in Equation (5). allowing us to use only rank one matrices =²(x")T in place of the rank k matrices in the classification setting. The procedure is written explicitly in Appendix A.4
5.1 BADGE comparison
By comparison, BADGE, recently proposed, slate-of-the-art active learning classification algorithm. aims select batch of samples that are likely to induce large and diverse changes to the model ( This is done by representing each candidate sample E U as 9s = V = T. 8²), the d- dimensional last-layer gradient that would be obtained if the most likely label according to the model, D. were observed. BADGE selects a batch of samples that have large Gram determinant in this space.
The intuition behind BADGE is that sampling proportionally the Gram determinant of these hallucinated gradients trades-off between uncertainty and diversity; a batch of gradient embeddings that produce large Gram determinant will need to be both high magnitude (corresponding model uncertainty) and linearly independent (corresponding to batch diversity). It is worth noting that while BADGE sampling is motivated by determinantal point process (DPP) sampling, the actual BADGE algorithm only uses rough approximation to this procedure.
Still, from the perspective of BAIT, the "gradient embedding" used in BADGE is single column of the dk x l matrix Vz. but not scaled by vP:. These embeddings can correspondingly be thought of as rank-one approximations for I [2: 8). BAIT trades BADGE'S 's determinantal sampling for trace min- imization (A (A-optimality). This substitution is essential because the determinantal approach is unable to accommodate for T(0), as argmax, (e; 0) -0-01)0111 I(0)) argmax,
0)-1) for any I(O).
Thus, BAIT offers two main advantages over BADGE. First, it considers the entire rank-k pointwise Fisher, catching potentially useful information that's 's ignored by BADGE. Second, BAIT incorporates the Fisher over all samples / (0). a term we show to be essential for minimizing risk and bounding MLE error. Crucially, because BADGE identifies this vector as corresponding to the most likely label according to the model, and this only makes sense in classification settings, it is unable to handle regression problems, a regime to which BAIT naturally extends.
Comparing objectives. These observations make clear that BAIT is more general than BADGE, but it is not obvious which of the aforementioned algorithmic extensions is most important for boosting performance. Figure 2 directly compares those objectives for batch sample selection. Specifically, we run three variations: greedily maximizing the determinant of the rank-one BADGE

---

Learned Embedding
Random Embedding
Figure 4: The same plots as Figure but comparing BAIT baseline ac- tive learning algorithms. We include BAIT without I(0") for a clearer comparison. BAIT most drastically outperforms baselines on the unin- formed representation, where high- norm directions are not necessarily most discriminative.
0.8
0.8
Accuracy
BAIT
0.6-
BADGE
0.6
Coreset
Random
0.4
0.4
Confidence
BAIT (no fisher)
0.2
Number of Labeled Samples 1e3
Number Samples 1e3
gradient embeddings, greedily maximizing the determinant of the full-rank Fisher, and the BAIT approach, taking into consideration both the full-rank pointwise Fisher matrices and T(0²). The two determinantal algorithms are written formally in the Appendix as Algorithm 2 and Algorithm g and can be made efficient by taking advantage of Woodbury identities. Here BAIT uses only forward greedy optimization, rather than both forward and backward, to ensure fair comparison.
We study two simple projections of the MNIST dataset. In one, we fit a two-layer MLP on 50% of the training data, and embed the remaining 50% using the first layer. We perform active learning in this 228-dimensional space on the unseen 50% of examples, selecting 50 batches of size 10 in sequence.
We then conduct a similar experiment, but instead of using a learned representation, we use random (Gaussian with mean zero and unit variance) matrix to project samples into 128 dimensions, setting in which, unlike in the learned representation, the largest directions are not necessarily the most discriminative. Note that this representation allows us to control feature dimensionality but mimics the typical convex learning paradigm, where features are fixed, not conditioned on labels, and not controllable by the learner.
In both plots, the BAIT objective outperforms determinantal objectives. This effect is more drastic for the uninformed embedding, where the plot suggests that the full-rank pointwise Fisher is more useful than its low-rank counterpart for late-stage performance, and that T(O"), as used in BAIT, is especially beneficial for early stage performance.
Synthetic experiment. We conduct a small synthetic experiment to demonstrate the value of incorporating the Fisher information matrix into the acquisition strategy. In Figure 3 we plot the exact Bayes Risk (3) in the Bayesian linear regression setup described in Section 4 aas a function of the batch size 1 for both BAIT and the greedy determinant maximization strategy. Here we consider two distributions in d = 100 dimensions. In the left plot data are generated from Gaussian distribution with diagonal covariance with quadratic spectral decay Σn 1/62. On the right, the distribution is supported only on the standard basis, with probabilities that decay quadratically P: := P[r eq] x 1/12. Note that both distributions have identical and poorly conditioned covariance Σ (recall (3)
This allows us highlight the value of the Fisher matrix and how it leads robust performance across data distributions. Indeed, we see that in the Gaussian case, both the BAIT strategy (called "Trace+FFsher"" in the figure) and the determinental maximization strategy ("Log-det") perform almost identically. However, BAIT significantly outperforms the alternative in the orthonormal case. This occurs because the latter does not exploit the occurrence probabilities p; and in fact simply selects the coordinates in a cyclic fashion. On the other hand, the optimal strategy focuses effort on the high- probabbbbyyiity coordinates. which is exactly captured in the Fisher matrix
6 Experiments
In this section we detail extensive experiments that highlight the generality and performance of BAIT. We consider three settings: linear classification, deep classification, and regression. Throughout these sections, we compare BAIT to several recently proposed and classic active learning approaches.
Among these, we consider BADGE, CORESET, CONFIDENCE, and RANDOM sampling. BADGE, as mentioned earlier, is state-of-the-art approach that trades off between diversity and uncertainty by approximately sampling a batch of points that have high Gram determinant when represented as a
Note that in the orthonormal case, both greedy optimization algorithms are in fact optimal for their respective combinatorial problems.
7

---

MNIST, Batch Size 100, MLP
SVHN, Batch Size 10, ResNet
OML155, Batch Size 100, MLP
0.7
0.95
0.925
0.900
0.90
BADGE
0.775
Curesel
Random
0.800
Confidence
1.0
0.8
1.50
1.75
2.00
1ed
Figure 6: Three deep active learning experiments with different model architectures, datasets, and batch sizes. Left: An MNIST experiment, using a batch size of 100 and an MLP. Center: Active learning the SVHN dataset with an 18-layer ResNet and batch size of 10, smoothed for clarity (unsmoothed plot in the Appendix). Right: Active learning on the OpenML dataset 155 using an MLP and a batch size of 100. Here we zoom in on disriminative regions of learning curves.
gradient. CORESET represents items using the model's penultimate layer representation, then samples batch that describes the space well. CONFIDENCE sampling selects the points for which the model is least confident, measured by max (x; 0). RANDOM draws n points uniformly at random.
6.1 Linear Classification
Like BADGE and CORESET, BAIT caters to efficiency part by only considering the last layer of the network to select new batch. Despite this linear assumption, both CORESET and BAIT are unable to perform well outside of the neural regime.
This subsection revisits the simplified setting described in Section and and Figure 2 involving both informed and uniformed representations of MNIST. In the learned representation, performance differences between algorithms is relatively subdued, with BAIT, BADGE, and CONFIDENCE among the highest-performing agents. However, in the unstructured, random representation, the are stark differences in accuracy. While controlling for dimensionality, this representation mimics the convex case, where the model not able control how data are represented. Here, BAIT outperforms baseline approaches by a large margin (Figure 4
Among these comparisons, we include simplified version of BAIT, which omits the Fisher term I()), resulting in an objective that has been explored by [60] This approach performs on par with other baselines, suggesting that is the inclusion of [()) that allows BAIT succeed even in difficult, poorly structured feature spaces. This experiment further highlights a potential cause of the success of these baselines, the penultimate-layer representation will behave more like what's described here as a learned representation than a random representation. Still, the following subsection shows BAIT outperforming baselines deep classification.
Rand
Conf Coreset Badge
BAIT
6.2 Deep Classification
Rand
0.0
8.1
9.4
0.8
0.1
We now turn to our main experiments, active learning for classification with neural networks. This subsection provides extensive results for the above algorithms across
Conf
8.2
0.0
10.5
0.7
0.1
wide array of settings.
Coreset
2.7
4.4
0.0
0.3
0.0
We consider three datasets. Using an MLP, we perform active learning both MNIST data and OpenML dataset 155. We also use the SVHN dataset 37 of color digit images with both an MI. and an 18-layer ResNet. Last we explore the CIFAR-10 10 object dataset 38 with ResNet. All archtttctrr pairs three batch sizes-11 10, 100, and 1000. MLPs include a single hidden ReLU layer of 128 dimensions.
Badge
14.7
11.7
16.8
0.0
2.0
BAIT
16.1
13.8
19.1
5.4
0.0
experimented with at
8.3
7.6
11.2
1.4
0.4
Figure 5: A pairwise comparison plot. Element j roughly corresponds the number of times algorithm i outperforms algorithm j by a statistically significant degree. Columwise averages are given at the bottom, where lower number corre- sponds to biigher-pefformin algorithm.
All ResNets are trained with learning rate 0.01, and all other models (including linear models shown earlier) are trained with a learning rate of 0.0001. We parameters using the Adam variant of SGD, and use standard data augmentation for all CIFAR-10 experiments. Like other deep active learning work, avoid warm-starting and
8

---

Yearbook Data, Batch Size 100, ResNet
Rainfall, Batch Size Linear
Mean Mean Squared Error
0.6
0.4
Coreset
0.5
Coreset
Random
Random
0.4
0.2
0.2
2000
4000
6000
8000
10000
200
400
600
800
1000
Number Labeled Samples
Number of Labeled Samples
Figure 7: Two regression experiments with varying architectures. Left: Active regression using an 18-layer ResNet, predicting the year in which American yearbook photos taken. Right: A linear model used to predict rainfall from meteorological features.
MNIST, Batch Size 10, MLP
SVHN, Batch Size 100, MLP
MNIST, Batch Size Linear
0.12
BAIT
Moan Squarad Entrr
0.20
Cureset
0.08
0.08
0.04
0.06
0.05
0.0
0.2
0.4
0.5
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.5
0.8
1.0
Number of Labeled Samples
103
Number of Labeled Samples
164
Number of Labeled Samples
163
Figure 8: Three regression experiments with varying architectures. Left: Active regression using an MLP, MNIST data, and a batch size 10. Center: Active regression using SVHN data and a batch size of 100. Right: the same the leftmost plot, but using linear model instead of an MLP.
retrain model parameters from a random initialization after each query round [ Each learner is initialized with 100 randomly sampled labeled points, and each experiment is repeated five times with different random seeds. Shadowed regions in plots denote standard error. More empirical details
be found in Appendix Section C
Figure 5 zooms on the discriminative regions learning curves corresponding to three different settings. While the relative performance of baseline algorithms varies somewhat across scenarios, BAIT is consistently as good or better than the highest-performing approach. Full learning curves are presented in Appendix Section C.I
Due to the volume of settings investigated, we present aggregate results using the analysis approach of [6]. For each experiment, we note the round r of active learning for which random selection first obtains accuracy within 1% its final accuracy. We then checkpoint each algorithm exponential intervals up to r, that is, we log each labeling budget T, for which L Mo + 2 B S for batch size B and number of seed samples Mo. each L in given experiment, we compute the f-score,
VNg where N is the number of samples, between each pair of algorithms i +
t
N Σ
(d--e))
&
- -2)
c|
where el and e/ denote the 1-th accuracy respectively corresponding to algorithms i and labeling budget Lg. We then perform a two-sided test, where algorithm i is said to outperform algorithm j if t 2.776, and vice versa if < -2.776, marking a significant difference (p 0.05).
This formulation allows us to construct pairwise penalty matrix over all conducted experiments. The matrix has as many rows and columns as there considered algorithms (five); if algorithm i outperforms algorithm j for some experiment at some labeling budget, the corresponding element j of the matrix is incremented by !/z, where is the total number of labeling budgets considered for that experiment.
The resulting plot is given in Figure which aggregates results over all conducted experiments, and which suggests BAIT significantly outperforms baseline approaches. We also include columwise averages, which give holistic perspective algorithm performance.

---

We show more pairwise plots of this type in Appendix Section size and and architecture type. These figures all suggest BAIT is higher- gegher-performing than baseline approaches across environments.
C2
breaking up results by batch
6.3 Li Regression
Although deep learning is most commonly discussed within a classification framework, recent work has success- fully applied deep learning in regression settings as well,
Per Batch (seconds)
Compute Time
100
Bait
Coreset
with important scientific applications including areas like physical, biological, and chemical modeling [9 40 is therefore important to develop active learning algorithms that are flexible enough to be applied in these domains.
50
101
10
10a
Batch Size
active learning results using two different model architectures, two different batch sizes, and two different datasets. In the first, we train an 18-layer ResNet to predict the year in which photos from American yearbook were taken 4T. To do this successfully, the model must learn correlate trends in photography and fashion with time period. Here labels were Z-scored, so error is not measured in terms of year. In the second, we use meteorological features and linear model to predict the amount of rainfall in Austin, Texas 42] Note that there are few active learning algorithms made with regression in mind-the CONFIDENCE and BADGE algorithms omitted here, they rely on notion of uncertainty that requires classification environment.
Figure 9: In the regression setting, the rank-one reduction of greedy selection in BAIT makes the approach only slightly slower than Coreset.
In Figure [ we show a few regression active learning experiments that have been synthetically adapted from the classification setting. We treat SVHN and MNIST data as having k continuous outputs, regressing onto one-hot encodings of their labels.
Similar to the classificatton case, the relative performance of baseline approaches shuffles between environments. Batch active learning in regression is challenging, with simple random sampling often being surprisingly effective. Still, regardless of which baseline is highest performing, BAIT consistently performs as well better on both linear and non-lineat regression tasks. Further, because BAIT can be reduced to rank-one calculations (Equation 6). it is relatively efficient, and takes about as long to run as CORESET (Figure
Discussion
This article studies neural active learning from the theoretical perspective of maximum likelihood estimators, a viewpoint that sheds new light on the performance of previous approaches. We proposed BAIT, a generalized, nigh-performing. and tractable approach to neural active learning that makes use of this perspective, showing that a more classical approach is tractable and effective with modern neural architectures. We demonstrated that BAIT is successful in both convex and non-convex scenarios, and for both classification and regression settings.
is worth noting that, while tractable, the classification version of BAIT is more computationally inten- sive than Boughy-ruuhlly times slower to select a sample to include in a batch (in seconds, though total run times are largely dominated by retraining models after each batch acquisition (5]). This added computation is well justified in active scenarios for which the cost of label acquisition is high relative to the cost of computation. To trade-off between computation requirements and performance, one could estimate the Fisher using only the lowest-norm r < k columns of Vz, catching the more descrip- tive components of the Fisher. We leave the analysis of such an approach avenue for future work.
8 Acknowledgements
Sham Kakade acknowledges funding from the National Science Foundation under award #CCF- 1703574.
10

---

References
[1] Samuel Budd, Emma C Robinson, and Bernhard Kainz. A survey on active learning and suman-in-the-logy deep learning for medical image analysis. Medical Image Analysis, 2021.
[2] Asim Smailagic, Hae Young Noh, Pedro Costa, Devesh Walawalkar, Kartik Khandelwal, Mostafa Mirshekari, Jonathon Fagert, Adrián Galdrán, and Susu Xu. Medal: Deep active learning sampling method for medical image analysis. In International Conference on Machine Learning and Applications, 2018.
[3] Fabian Stark, Caner Hazırbas, Rudolph Triebel, and Daniel Cremers. Captcha recognition with active deep learning. In Workshop New Challenges in Neural Computation, 2015.
[4] Kamalika Chaudhuri, Sham Kakade, Praneeth Netrapalli, and Sujay Sanghavi. Convergence
rates of active learning for maximum likelihood estimation. In Advances in Neural Information Processing Systems, 2015.
[5] Jordan T Ash and Ryan P Adams. On warm-starting neural network training. Advances in Neural Information Processing Systems, 2020.
[6] Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agar- wal. Deep batch active learning by diverse, uncertain gradient lower bounds. International Conference Leurning Representations, 2020.
[7] Burr Settles. Active learning literature survey. University of Wisconsin, Madison, 2010.
[8] Sanjoy Dasgupta. Two faces active learning. Theoretical computer science, 2011.
[9] Steve Hanneke. Theory of disagreement-base active learning. Foundations and Trends in Machine Learning, 2014.
[10] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference Learning Representations, 2018.
[11] Yonatan Geifman and Ran El-Yaniv, Deep active learning over the long tail. 1.X0v:1711.00941, 2017.
[12] Daniel Gissin and Shai Shalev-Shwartz. Discriminative active learning. a7Xiv:190.006347. 2019.
[13] Yuhong Guo and Dale Schuurmans. Discriminative batch mode active learning. In Neural Information Processing Systems, 2008.
[14] Zheng Wang and Jieping Ye. mode active learning. Transactions on Knowledge Discovery from Data, 2015.
Querying discriminative and representative samples for batch
[15] Yuxin Chen and Andreas Krause. Near-optimal batch mode active learning and adaptive submodular optimization. In International Conference on Machine Learning, 2013.
[16] Kai Wei, Rishabh Iyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. In International Conference on Machine Learning, 2015.
[17] Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. In Advances in Neural Information Processing Systems, 2019.
[18] Simon Tong and Daphne Koller. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research, 2001.
[19] Greg Schohn and David Cohn. Less is more: Active learning with support vector machines, In International Conference on Machine Learning, 2000.
[20] Gokhan Tur, Dilek Hakkani-Tür, and Robert E Schapire. Combining active and sumi-supervised learning for spoken language understanding. Speech Communication, 2005.
[21] Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In International Conference Machine Learning, 2006.

---

[22] BurT Settles, Mark Craven, and Soumya Ray. Multiple Multiple-instance instance active learning. In Advances in Neural Information Processing Systems, 2008
[23] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, 2017.
[24] Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: a margin based approach. N.Kiv:880..0966 2018.
[25] William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensembles for active learning in image classification. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[26] Sheng-Jun Huang, Rong Jin, and Zhi-Hua Zhou. Active learning by querying informative and representative examples. In Neural Information Processing Systems, 2010.
[27] Yoram Baram, Ran El Yaniv, and Kobi Luz. Online choice of active learning algorithms. Journal of Machine Learning Research, 2004.
[28] Wei-Ning Hsu and Hsuan-Tien Lin. Active learning by learning. In AAAI Conference Artificial Intelligence, 2015.
[29] A. W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 2000.
[30] E. L. Lehmann and Casella. Theory of Point Estimation. Springer. 1998.
[31] Jamshid Sourati, Murat Akcakaya, Todd K Leen, Deniz Erdogmus, and Jennifer G Dy. Asymp- totic analysis of objectives based on Fisher information in active learning. The Journal of Machine Learning Research, 2017.
[32] Tong Zhang and F Oles. The value of unlabeled data for classification problems. In International Conference on Machine Learning, 2000.
[33] Steven CH Hoi, Rong Jin, Jianke Zhu, and Michael R Lyu. Batch mode active learning and its application to medical image classification. In International Conference on Machine Learning, 2006.
[34] Quanquan Gu, Tong Zhang, and Jiawei Han. Batch-mode active learning via error bound minimization. In Uncertainty and Artificial Intelligence, 2014.
[35] Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, al. Gradient- learning applied to document recognition. IEEE, 1998.
[36] Jamshid Sourati, Ali Gholipour, Jennifer G Dy, Sila Kurugol, and Simon K Warfield. Active deep learning with fisher information for patch-wise semantic segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pages 83-91. Springer, 2018.
[37] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[38] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
[39] Mohammed AlQuraishi Alphafold at casp13. Bioinformatics, 2019.
[40] Alex Beatson, Jordan Ash, Geoffrey Roeder, Tianju Xue, and Ryan P Adams. Learning com- posable energy surrogates for pde order reduction. Advances in Neural Information Processing Systems, 2020.
[41] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A Efros. A century of portraits: visual historical record of american high school yearbooks. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1-7, 2015.
[42] Kaggle. Historical temperature, precipitation, humidity, and windspeed for austin, texas. https: ://www kaggle. conn/gruben/austiineeeee
12

---

Checklist
be found in the experiments section and the 1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes] hyperparameters appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] In appendix
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes) In readme.txt
(b) Did you mention the license of the assets? [N/A] None is provided.
(c) Did you include any assets either in the supplemental material or URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "mnbeT_NahXz", "reviewer": "Reviewer_7jff", "summary": "The paper tackles active learning with deep learning models, focusing primarily on the problem that labels need to be requested in batches due to the large training cost. The authors propose to rely on the Fisher information and introduce a greedy approach to choose each set of points to be labelled efficiently.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Originality\nWhile the specific approach of the Fisher matrix is new and the paper mentions that it is revisiting a \"classic, Fisher-based objective\" (l42), it is lacking a proper discussion of the prior work of using such approaches, which have been explored before (e.g. back to MacKay, (1992) who relied on it for active learning with neural networks). \n\nSimilarly, the citations and general presentation require a lot of improvements. The main competitor, BADGE, is mentioned first in the introduction and repeatedly throughout the paper. Yet, it is never connected with a reference leaving the reader to guess its source (ref 35 is mentioned, but only independently). The same applies to a second baseline, coreset, which is never connected to reference 8. Similarly, at other locations, e.g. while one could, if need be, argue that MNIST/SVHN/Cifar10/ResNet/Adam have become common knowledge, it would still be cleaner to cite them. However, the same argument does not apply to something like \"OpenML dataset 155\" (l293), which I would consider to be nonstandard and unknown by the average reader.\n\n\n## Quality\nThe method is presented in a somewhat biased way. E.g. with repeated claims in the regression setting, both that other active learning methods are lacking in that direction, and even deep learning usually being only discussed for classification (l335); a consistent improvement, but small in absolute numbers, is sold as \"large margin\" (l282) and \"drastic\" (caption of Figure 4). \nApart from such outliers and the citation problems mentioned above, the presentation is well done. The method is explored on several data sets (synthetic and real), several architectures and batch size settings, with extensive further code being provided. \n\nThe comparisons are primarily against deterministic methods (Badge, Coresets), lacking a comparison against the growing literature of Bayesian Deep Learning. E.g. the Gal et al. (2017), Kirsch et al. (2019) (the latter of which actively targets batch active learning). What do the authors expect performance-wise compared to such approaches that actively include model uncertainty as well as predictive uncertainty, together with query rules adapted for Bayesian Neural Nets (e.g. (Batch)BALD)?\n\n\n## Clarity\nApart from the details mentioned in the two sections above, the paper is written clearly and well organized.\n\n## Significance\nThe significance of the proposed approach is difficult to judge. In relative performance, the proposed model shows consistent improvements upon the baselines (especially the extended experiments in the appendix). Yet, in absolute numbers, the improvements upon simple approaches such as randomly picking new points or a vague confidence metric remain small. This leaves the question open as to how much of a practical influence it will have. It would be interesting to see the performance of the proposed approach to a highly redundant setting where most of the unlabeled points are irrelevant, punishing random query strategies and requiring a clever query, such as the proposed, to select the few highly relevant points.\n\n\n\n## Further Questions/Minor comments/typos\n- l32 claims a lack of principled batch active learning algorithms. Yet, the paper cites both Sener et al. and Kirsch et al., which try to tackle exactly this task in a principled way.\n- Can the authors comment on the repeated claim of lack of approaches for active learning in the regression setting (e.g. l49-50)? A lot of Bayesian uncertainty-based methods are directly applicable to the regression setup. Similarly, Active Learning's closely related sister field, Bayesian Optimization, regularly tackles continuous and discrete parameter spaces. \n- l66 claims references [11-15] to be about batch AL but separate from deep learning. Yet [15] explicitly tackles the task of Bayesian Deep Learning.\n- Error: References [29], [30] are the same paper\n- $V_x$ appears out of thin air in the equation starting in l184 without a proper introduction (and in a typo misses one subscript after the first equality)\n- Error: The right plot in Figure 5 mentions OML155 in its title, while the caption claims it to be Cifar-10.\n- Section 6.3 claims deep learning to be most commonly discussed within a classification framework, but it is, of course, also applied to a lot of regression tasks. E.g. staying in the image world, the whole area of depth estimation etc. \n- Figure 1 shows a consistent improvement of the proposed forward/backward approach vs the strict forward, yet the text claims that this holds only \"sometimes\" (l180). Are there setups where it always performs worse or is it only due to random initializations that sometimes cause it to perform worse in some runs?\n\n\n\n___\n(MacKay, 1992: Information-based objective functions for active data selection)\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The main limitation discussed is the increased cost per sample selection. A societal impact is not discussed (chosen as not applicable in the checklist). \n\nThe societal impact follows the general trend of deep learning approaches here. Better active learning strategies allow for more efficient labeling, giving us less wasted human time as a direct societal benefit. To a second degree, it then depends on whether the labeled data is used for beneficial or harmful tasks.\n", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "cbyZWHNyYp-", "reviewer": "Reviewer_gqfA", "summary": "The paper introduces a new principled active learning algorithm called BAIT, which uses the Fisher information matrix to select informative samples. The approach uses a greedy approximation to solve the global batch acquisition problem. Compared to other approaches, it also works for regression.\n\nThe paper provides ablation and gives a new perspective on BADGE, another active learning algorithm.\n\nMultiple experiments both for classification and regression are provided.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality\n\nThe approach seems interesting. It focuses on using Fisher information matrices to select points that will minimize the overall loss on the pool set: $\\underset{S \\subset U,|S| \\leq B}{\\operatorname{argmin}} \\operatorname{tr}\\left(\\left(\\sum_{x \\in S} I\\left(x ; \\theta_{1}\\right)\\right)^{-1} I_{U}\\left(\\theta_{1}\\right)\\right)$.\n\nThe algorithm to select points is novel in particular. The objective is not submodular, so greedy selection does not have guarantees. Instead, the paper greedily selects more samples than required and then prunes the batch again. (I particularly like the forward (expansion) and backward (pruning) approach. Could this be repeated multiple times to improve the candidate batch?) However, there is no theoretical analysis available of this approach yet.\n\nFisher Information has been applied to Active Learning with DNNs previously, e.g. [\"Active Deep Learning with Fisher Information for Patch-wise Semantic Segmentation\", Sourati et al, 2018 (https://pubmed.ncbi.nlm.nih.gov/30450490/)](https://pubmed.ncbi.nlm.nih.gov/30450490/).\n\nThe main contribution lies in the novel algorithm (BAIT) that does not require semi-definite programming to find batches and in examining BAIT empirically in deep learning.\n\nMoreover, the paper provides a new perspective on BADGE, an earlier active learning algorithm. It shows that BADGE makes stronger approximations than BAIT. An ablation shows that BAIT outperforms BADGE. I like that BAIT does take into account the whole pool set (through the overall information matrix).\n\n### Quality\n\nOverall, the submission seems sound. The experiments in the main paper are convincing and necessary ablations are provided. The method is very well motivated and deduced. \n\nWhile Figure 6 in the paper convincingly shows that BAIT outperforms all other methods, the respective plots in the appendix are less clear than the ones in the main paper. In particular, it seems that the proposed method, BAIT, does not really outperform BADGE on SVHN and CIFAR-10 by much at all. It would be good to swap the experiments on MNIST with another higher-dimensional or more challenging dataset. Maybe CINIC-10 might provide more conclusive results (https://paperswithcode.com/dataset/cinic-10).\n\nResults are also presented for OpenML dataset 155. However, I do not know this dataset and it is not introduced. (That said, citations for MNIST, SVHN and CIFAR-10 are also missing.)\n\nLastly, the regression experiments use a rather contrived setup. Fitting one-hot encodings is a very specific objective. However, I cannot suggest anything better.\n\n### Clarity\n\nThe paper is very well written. The introduction and related work section are great. As noticed, a few more citations ought to be added. The introduction is also rather light on citations.\n\n### Significance\n\nThe results appear very significant assuming they perform better than previous methods, which I am not fully convinced of yet.\nThe proposed algorithm appears significant by itself as the idea of selecting and pruning alternatively is novel. However, given the shown results on CIFAR-10 and SVHN, it would not be clear whether this method is truly SOTA, and whether to recommend this method over BADGE as a baseline in the future.\n\nThe application to regression is significant and definitely an improvement over other methods (like BADGE) which only work with classification. However, the regression experiments are not as convincing.\n\n### Summary\n\nThe paper introduces a new AL method, which is well motivated and clearly communicated. This method is not limited to classification and also works for AL for regression.\n\nThe plots in the appendix are not as clear as I'd like, and I would like to verify that BAIT outperforms BADGE manually as well instead of trusting Figure 6.\n\n### Questions\n\n1. Could you provide experimental results on another higher-dimensional dataset? (E.g. CINIC-10)\n2. Could you smooth the plots (using a Gaussian window or similar) to make the SVHN and CIFAR-10 results more easily interpretable?\n3. Could you provide a less contrived regression experiment?\n4. How dependent is this method on the # of latent dimensions?\n\nI would be happy to increase my score if any or all of the above questions are answered.\n\n### Typos\n\n1. [29] and [30] in the bibliography refer to the same paper.\n2. (4): it is not clear what $I_U$ exactly is at first.\n3. l.184: what is $V_x$? It has not been introduced.\n4. Sec 5.1: Figure 2 is never directly referenced in the text.\n5. Figure 5's caption says CIFAR-10 while the plot says OML155.\n6. l. 316: what is N? Number of trials?\n\nDisclaimer: I did not check the proofs in the appendix in great detail.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is no mention of societal impact, which given what other papers have written is not a big loss, as this method does not have any specific new societal impact compared to existing AL methods.\n\nThe paper does not mention limitations per-se beyond the high computational load. I wonder how dependent this method is on the # of latent dimensions.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "A-WsS6YCOV", "reviewer": "Reviewer_GaQR", "summary": "The paper introduces an algorithm (BAIT) for batch active learning, designed for and tested on neural networks. It centres around an objective in eq 4, argmin_select trace(I_select^-1, I_all), with I as the Fisher information matrix of the final layer parameters of a neural network, an objective that has been previously derived/analysed in simpler settings. The contribution here is applying it to deep neural networks. It draws links with a current SOTA method, BADGE, and shows slight performance benefits over it.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I’m borderline on this paper. I do find the method interesting, and the empirical performance seems to be in line with the current SOTA method BADGE. However, I believe the presentation of the method needs improving, and there are various claims that I am uncomfortable with, including interpretation of results. Overall, I think the paper could be very strong following a sober revision.\n\n__Method Structure/Clarity__\n\nI found the motivating theory (section 4) of the method confusing. It begins by considering the case of Bayesian linear regression to derive eq 3, but then jumps to pulling out eq 4 from Chaudhuri paper, which it says the paper will optimise. What is the use of beginning with eq 3? The setting considered in the paper is neither Bayesian, nor linear, nor generally regression. I think it would be more worthwhile to discuss eq 4 and the Chaudhuri paper in fuller detail. E.g. it’s said this was shown to be ‘near optimal’ under their set up. What was this set up? What assumptions were required? How do these translate to the neural set up?\n\nSeveral times (including abstract) it’s stated that we’re optimising a bound on the MLE error. But nowhere in the technical sections did I see any bound. Under what conditions is it a valid bound (I'm assuming just the linear setting)?\n\nAs a smaller point, the paper gives the impression that viewing the output of a network as an estimated probability distribution is a novel perspective – I’d argue that this is the default position.\n\nIn general, I didn’t come away feeling like I have much intuition for why it’s a good idea to choose a set to minimise eq 4.\n\n\n__Claims and Wording__\n\nSeveral times it’s claimed that the method addresses problems of non-convexity in neural systems (including the abstract). But actually, the Fisher information matrix is only ever considered for the last layer parameters, with the majority of the network used as a feature extractor. This is ok as a methodology, but it feels like misleading advertising.\n\nThe paper claims to achieve SOTA results. This is a bit of a hard sell when one looks over the full learning curves in the appendix, where curves for BAIT vs BADGE for CIFAR10 and SVHN are pretty well indistinguishable. The zoomed regions in figure 5 seem rather cherry-picked. I’m unsure if very slight advantages justify BAIT’s increased computational demands (k times). As I reviewer, I don’t demand SOTA results if the method is interesting enough in itself, which I think is the case here, and would prefer the authors to have a more considered results discussion.\n\nThere does seem to be a performance improvement in the linear setting (‘random embeddings’), but this is of less interest given the paper’s title and main objective. In some ways it seems unfair to give these figures (2, 4) such prominent position.\n\n\n__Experiments__\n\nBeing able to apply the method in regression settings is a great advantage, but the experiments tested simply took the classification datasets (MNIST etc) already used and converted them to regression. This feels like an opportunity missed, as there there are plenty of interesting regression tasks, e.g. depth regression, counting problems, rainfall prediction...\n\nTo test the linear setting, data is 'randomly projecting data into d dimensions’ (rather than using features extracted by a neural net). I think the use of ‘random’ is a bit misleading here as it seems to have used some dimensionality reduction technique? I didn’t spot how this projection was done, but it seems important to describe it.\n\nThe algorithm works by selecting 2B examples, then discarding B of these. I’d be interested to see ablations/rationale for how this number was selected. Could the algorithm be improved with a larger number?\n\n\n__Minor__\n- Seems like section headings have been crushed up, eg line 262\n- Algorithm 1, perhaps move ‘forward greedy optimization’ to line 5?\n- Subscripts and superscripts missing on eqs on line 184\n- Think redundant /2 in line 498\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There’s a short paragraph at the end of the paper outlining the additional computation complexity of the method. Considering the addition of an extra page in this year’s conference, I think this should be expanded.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5-6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agreed that this is a good contribution to NeurIPS, *provided* that the additional experiments and clarifications provided by the authors during the discussion are included to the paper. 
Decision: Accept (Poster)