Gone Fishing: Neural Active Learning with Fisher Embeddings
Year: 2022

Abstract:
There is an increasing need for effective active learning algorithms that are compatible with deep neural networks. This paper motivates and revisits a classic, Fisher-based active selection objective, and proposes BAIT, a practical, tractable, and high-performing algorithm that makes it viable for use with neural models. BAIT draws inspiration from the theoretical analysis of maximum likelihood estimators (MLE) for parametric models. It selects batches of samples by optimizing a bound on the MLE error in terms of the Fisher information, which we show can be implemented efficiently at scale by exploiting linear-algebraic structure especially amenable to execution on modern hardware. Our experiments demonstrate that BAIT outperforms the previous state of the art on both classification and regression problems, and is flexible enough to be used with a variety of model architectures.

heading: 1 Introduction
text: The active learning paradigm considers a sequential, supervised learning scenario in which unlabeled samples are abundant but label acquisition is costly. At each round of active learning, the agent fits its parameters using available labeled data before selecting a batch of unlabeled samples to be labeled and integrated into its training set. A well-chosen batch of samples is one that is maximally informative to the learner, such that it can obtain the best hypothesis possible given a fixed labeling budget.
Active learning is well established as an area of machine learning research due to the ubiquity of important real world problems that fit the sample-abundant, label-expensive setting; commonly cited applications range from medical diagnostics [1, 2] to image labeling [3]. Mitigating large sample complexity requirements is particularly relevant for deep neural networks, which have in recent years achieved impressive success on a wide array of tasks but often require considerable amounts of labeled data.
Shifting the focus of active learning to deep neural networks highlights several important problems. For one, most foundational active learning work assumes a convex setting, which is clearly violated by massive nonlinear neural networks. Many of these approaches are computationally expensive, and it is not clear how to adapt them for real-world use [4]. Further, because neural network training is generally expensive, practical active learning algorithms must be able to work in the batch regime, querying B samples at each round of active learning instead of a single point at a time [5].
Despite a long history of active learning research, these constraints draw attention to a need for practical, principled batch active learning algorithms for neural networks. Current state-of-the-art methods, like Batch Active Learning by Diverse Gradient Embeddings (BADGE), perform robustly in experiments, but explanations for its behavior are fairly limited [6]. This drawback makes it unclear how to scale some active learning algorithms into regimes that deviate somewhat from the setting for which they were designed—BADGE, for example, cannot be run on regression problems, and as we show in this paper, performs poorly when used in conjunction with a convex model.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
This article adopts a probabilistic perspective of neural active learning. We view neural networks as specifying a conditional probability distribution p(y | x, ✓) over label space Y given example X , where ✓ are the network parameters. This perspective provides theoretical inspiration from the convex regime with which to examine and design neural active learning algorithms. From this viewpoint we motivate and revisit a classic, Fisher-based objective for idealized active selection. We argue that approximately minimizing this objective can be done tractably in the neural regime, despite their overparametrized structure and shifting internal representation. Accordingly, this work helps bridge the divide between algorithms that are performant but not well understood by theory, and those that are theoretically transparent but not computationally tractable.
Experimentally, BAIT offers improved performance over baselines in deep classification problems, a trend that is robust to experimental conditions like batch size, model architecture, and dataset. Crucially, BAIT is general purpose, and can be easily extended to regression settings, where many other algorithms cannot. It further performs well on both regression and classification with convex models, a paradigm in which other algorithms often struggle.
In summary, this paper
• puts neural active learning on firm probabilistic grounding, giving a new, rigorous perspective on the functionality of previously proposed algorithms.
• provides in-depth empirics that elucidate differences between neural and convex regimes, and discusses simplifying assumptions that are sometimes reasonable in the neural case.
• proposes a practical, unifying, high-performing active learning algorithm that leverages these insights in a computationally tractable manner.

heading: 2 Related work
text: Active learning is a very well-studied problem [7–9]. There are two main sample selection approaches, diversity and uncertainty sampling, which are successful respectively for large and small batch sizes.
Diversity sampling strategies aim to select batches of data that best represent the space. In a deep learning context, these algorithms typically embed unlabeled samples using the neural network’s penultimate layer and select a subset of samples that might act as a proxy for the entire dataset [10, 11]. [12] proposed inducing batch diversity using a generative adversarial network formulation, selecting samples that are maximally indistinguishable from the pool of unlabeled examples.
There is also a rich body of work on batch active learning [13–17]. These methods typically formulate batch selection as an optimization that minimizes an upper-bound on some notion of model loss.
Efficiently adapting parameters to an incrementally larger training set is not an issue in convex settings. Accordingly, with linear models, it is more common to use uncertainty sampling and a batch size of one. A frequently used approach is to query samples that lie closest to the current model’s decision boundary, a quantity that’s considered inversely proportional to uncertainty [18–20]. Some similar methods offer theoretical guarantees on statistical consistency [9, 21]. Other algorithms quantify uncertainty using the entropy of the predicted distribution over classes, or as the size of the expected gradient induced by observing the label corresponding with a candidate sample [22]. The latter is known to be related to the T -optimality criterion in experimental design, but is unable to account for batch diversity.
Similar approaches have been modified for use with neural networks as well. For example, [23] exercise Dropout to sample weights to approximate the posterior distribution over labels, and use it to identify samples that reduce model uncertainty. Adversarial example generation has been used to approximate the distance between a sample and the decision boundary [24]. Model ensembling has also been used to approximate sample uncertainty, where the predictive variance across constituent models can be used to inform a sample selection strategy [25].
There are a variety of algorithms that are meant to combine uncertainty and diversity sampling [26]. This trade-off is sometimes framed as its own optimization problem, for example using a metalearning approach that hybridizes both strategies [27, 28]. Among these is active learning by learning, which uses a bandit approach to select which query rule to employ at any given round of active learning [28]. BADGE, described in detail in Section 5.1, also combines uncertainty and diversity sampling, and is considered state-of-the-art for deep neural networks.

heading: 3 Notation and Setup
text: We consider a standard setup for batch active learning with neural network models, where there is an instance space X , label space Y , and a distribution D over X ⇥ Y . We use DX to denote the marginal distribution over the instance space and DY|X (x) to denote the conditional distribution over labels given example x. For learning, we are given access to a pool U = {xi}ni=1 ⇠ DX of unlabeled examples and we have the ability to request the label for any point x 2 U . In the tth round of batch active learning, we select a collection {x(t)j }Bj=1 ⇢ U of B examples (B is the batch size) and request the labels y(t)j ⇠ DY|X (x (t) j ) for all examples in the batch. We use these labeled examples to update our neural network model and then we proceed to the next round.
In this setup, the ultimate objective is to achieve low loss on the data distribution D, that is we hope our learned parameters b✓ nearly minimize E(x,y)⇠D`(x, y; b✓), where ` is some loss function like the cross entropy loss for classification. We always consider this objective in our experiments, but for algorithm development it is helpful to instead consider the fixed-design or transductive setting, where the goal is to instead minimize LU (✓) = Ex⇠UEy⇠DY|X (x)`(x, y; b✓), essentially treating the unlabeled samples U as the entire distribution. Note that these two objectives can typically be related by generalization arguments.

heading: 4 Probabilistic Perspective
text: We consider neural networks as specifying a probability distribution p(y | x, ✓) over the label space Y given an example x, where ✓ are the network parameters. Adopting this view, it is most natural to use the loss function `(x, y; ✓) = log p(y | x, ✓), choosing parameters that maximize the likelihood of observed labeled data. In classification problems, for example, we apply the softmax operation to the output of the network and then evaluate the cross-entropy loss with the ground truth label. For regression problems, we use the square loss, which treats the neural network as specifying a Gaussian distribution for each x.
Bayesian linear regression. As a warm-up, it is illustrative to consider an experimental design setting with Bayesian linear regression. We consider a d-dimensional linear regression problem where we assume the parameter vector ✓? has prior distribution N (0, 1I) and the conditional distribution DY|X (x) = p(· | x, ✓?) = N (h✓?, xi, 2) is Gaussian. For any set of labeled data {xj , yj}mj=1, the resulting maximum a posteriori (MAP) estimate is given by ridge regression with regularizer 2:
b✓ = argmin ✓
mX
j=1
(hxj , ✓i yj)2 + 2k✓k22 (1)
In the experimental design setting, we have unlabeled data U = {xi}ni=1 and our goal is to select a set S ⇢ U of B points so that the resulting MAP estimate has the lowest Bayes risk. Letting ⌃ =
1 n Pn i=1 xix > i denote the second moment matrix of the unlabeled data, the Bayes risk is
BayesRisk(S) = E h (b✓S ✓?)>⌃(b✓S ✓?) i , (2)
where b✓S is the MAP estimate after querying for labels on subset S, and the expectation is with respect to the noise in the labels and the prior over ✓?. Lemma 1 in the Appendix shows that for a subset S ⇢ U , letting ⇤S = P x2S xx > + 2I , the Bayes risk in this setting is exactly:
BayesRisk(S) = 2 tr(⇤ 1S ⌃). (3)
Observe that the RHS does not depend on labels, implying that minimizing the RHS over subsets S is feasible and optimal selection strategy under this criteria. This also verifies that multiple batches of active learning are not required for Bayesian linear regression, although this observation does not carry forward to the neural setting. BAIT is designed to approximately minimize this objective.
Classical regime. An objective similar to (3) also emerges naturally in the analysis of maximum likelihood estimators (MLE) in the convex regime. Here, classical statistical theory posits that the model is well-specified, so that there is some parameter ✓? such that DY|X (x) = p(· | x, ✓?) for each x 2 X . It is also common to impose regularity conditions including strong convexity of the loss function LU (✓) [29, 30]. While these conditions certainly do not hold in the neural setting, BAIT builds on much of this classical technology. The key quantity is the Fisher information matrix I(x; ✓) := Ey⇠p(·|x,✓)r2`(x, y; ✓) which is known to determine the asymptotic distribution of the maximum likelihood estimator [29]. In many probabilistic models, including linear and logistic regression, the hessian of the loss function does not depend on the label y, which we assume going forward.
For active learning in the classical setup, [4] give a two-phase sampling scheme with provably near-optimal performance. In the first phase, the algorithm samples a batch of B points uniformly at random, requests their labels, and optimizes the log-likelihood to obtain an initial estimate ✓1. In the idealized version of the second phase, a batch of B points is chosen to optimize
argmin S⇢U,|S|B tr
0 @ X
x2S I(x; ✓1)
! 1 IU (✓1)
1
A (4)
where IU (✓1) is the Fisher over all samples, P
x2U I(x; ✓1). This combinatorial problem is intractable in general, so [4] instead solve a semidefinite relaxation (SDP). They request labels on the obtained batch B of points and re-fit the model to obtain the final estimate b✓. For their setup, they prove the statistical properties of this two-phase estimator are near optimal.
Despite the theoretical properties, solving an SDP is not feasible in high dimensions; instead we provide a new, greedy algorithm for minimizing the objective that is usable in the neural regime. Several other works have also looked at this objective, either from a purely theoretical perspective [4, 31] or via relaxations [32–34]. However, some of these do not ensure batch diversity, and none have been extended to the neural regime.
This formulation essentially generalizes (3), since in linear regression the Fisher information I(x; ✓) is the covariance matrix xx>/ 2: the objectives in (3) and (4) differ only in their use of the regularizer controlled by . That is, essentially the same objective can be derived from two different perspectives, making it a compelling object for active selection. As such, our starting point for the neural setting is the ideal-but-intractable optimization problem in (4).

heading: 5 BAIT
text: Batch Active learning via Information maTrices (BAIT) is inspired by this theory, but adapted to the sequential, neural setting. To do this effectively, several key issues need to be addressed:
1. For neural models, the pointwise information matrix I(x; ✓) is typically extremely large.
2. The internal representation learned by the network changes with each round of active learning, so computation from previous rounds cannot be reused.
3. Solving the objective in Equation (4), as suggested in more theoretical work, is computationally infeasible [4].
Outlined as Algorithm 1, BAIT addresses item 1 in a somewhat standard way, by operating on the last layer of the network [6, 10]. We consider last-layer Fisher matrices I(x; ✓L) := Ey⇠p(·|x,b✓)r
2`(x, y; ✓L) for last-layer parameters ✓L. Note that in the linear setting ✓L = ✓. Here, if the top-layer representation starts to well-approximate a convex model, then the information geometry induced solely by these parameters can guide active sampling. Further, as we discuss shortly, this top-layer framework gives us a more principled understanding of the empirical success of BADGE.
One more subtle issue (item 2) is the interplay between the changing representation as learning progresses. We address this with an iterative scheme, where the Fisher information matrix is continually recomputed as the algorithm changes its representation during the course of learning.
Rather than solving an SDP, BAIT approximates a solution to Equation (4) using a greedy approach, which we show can be made tractable in both classification and regression settings. At each step of
Algorithm 1 BAIT Require: Neural network f(x; ✓), unlabeled pool of examples U , initial number of examples B0,
number of iterations T , number of examples in a batch B. 1: Initialize S by drawing B0 labeled points from U & fit model on S: ✓1 = argmin✓ ES [`(x, y; ✓)] 2: for t = 1, 2, . . . , T : {forward greedy optimization} do 3: Compute I(✓Lt ) = 1 |U | P x2U I(x; ✓ L t ) 4: Initialize M0 = I + 1|S| P x2S I(x; ✓ L t ) 5: for i = 1, 2, . . . , 2B: do 6: x̃ = argminx2U tr((Mi + I(x; ✓ L t ))
1I(✓Lt )) 7: Mi+1 Mi + I(x̃; ✓Lt ), S x̃ 8: end for 9: for i = 2B, 2B 1, ..., B: {backward greedy optimization} do
10: x̃ = argminx2S tr((Mi I(x; ✓Lt )) 1I(✓Lt )) 11: Mi 1 Mi I(x̃; ✓Lt ), S S \ x̃ 12: end for 13: Train model on S: ✓t = argmin✓ ES [`(x, y; ✓)]. 14: end for 15: return Final model ✓T+1.
the algorithm, the key computation lies in evaluating
x̃ = argmin x2U
tr((Mi + I(x; ✓ L t )) 1I(✓Lt )), (5)
where Mi is the Fisher corresponding to samples that have been selected so far.
Unfortunately, the trace function is not submodular, and is thus not well suited for standard greedy optimization. To address this, during each iteration, where the goal is identify B points to query, sampling is done in two stages. For a batch of B points, the first stage greedily oversamples, adding 2B samples to the initial batch. In the second stage, BAIT prunes B samples from the batch, better minimizing the objective described in (4). We find that this forward-backward strategy sometimes improves performance over the forward-only alternative (Figure 1). See Algorithm 1 for details. Choosing two as the oversampling factor of two is done for computational reasons, trading-off between computational cost and batch quality. We did not see performance improvements for larger multipliers.
When evaluating the i-th sample to include in S, the minimization in Equation (5) is efficiently computed using a trace rotation and the Woodbury identity for low-rank inverse updates:
Figure 1: A comparison between forward and forward-backward greedy approaches for BAIT. Here we show a simple active learning experiment using an MLP and MNIST data [35], and samples are acquired in batches of size 10 for 50 rounds. See Section 6 for more details.
argmin x tr
✓⇣ Mi + VxV > x ⌘ 1 I(✓Lt ) ◆
= argmin x
tr ⇣⇣
M 1i M 1 i VxA 1V >M 1i ⌘ I(✓Lt ) ⌘
= argmin x
tr ⇣ M 1i I(✓ L t ) ⌘ tr ⇣ M 1i VxA 1V >x M 1 i I(✓ L t ) ⌘
= argmin x
tr M 1i I(✓t)
tr ⇣ V >x M 1 i I(✓ L t )M 1 i VxA 1 ⌘
= argmax x
tr ⇣ V >x M 1 i I(✓ L t )M 1 i VxA 1 ⌘ ,
where A = I + V >x M 1 i Vx is an easily invertible k ⇥ k
matrix. Here Vx is a dk⇥k matrix of gradients, where each column is scaled by the square root of the corresponding prediction: VxV >x = I(x, ✓L). This formulation keeps us from having to compute and store all candidate I(x; ✓L), drastically decreasing the algorithm’s memory footprint. The trace rotation step, placing Vx as the leading term instead of M 1i is essential, as it avoids computing a new
kd⇥ kd matrix for each x. As a practical matter, on all datasets we consider in Section 6, this allows us to compute the trace contribution of all candidate samples simultaneously on a modern GPU.
After the minimizer x is found, updating M 1i is done simply via the same Woodbury identity, M 1i+1 = M 1 i M 1 i VxA 1V >x M 1 i , and the algorithm proceeds to identify the next sample.
Regression. In the regression setting we are able to further reduce the amount of required computation. Let xL denote the penultimate layer representation induced by f(x; ✓). For linear models xL = x. In Appendix Section A.3, we show that a k-output regression model trained to minimize squared error has pointwise Fisher I(x; ✓L) = (xL)(xL)> ⌦ b⌃ 1, where b⌃ is the noise covariance of the estimator. Using this fact, the regression version of the Fisher objective is
tr
X
x2S
I(x; ✓L) ! 1 IU (✓ L) ! = k tr
X
x2S
xL(xL)> ! 1 X
x2U
xL(xL)> !! . (6)
The full derivation can be found in Appendix A.3.1. This observation greatly simplifies the minimization in Equation (5), allowing us to use only rank one matrices xL(xL)> in place of the rank k matrices in the classification setting. The procedure is written explicitly in Appendix A.4.

heading: 5.1 BADGE comparison
text: By comparison, BADGE, a recently proposed, state-of-the-art active learning classification algorithm, aims to select a batch of samples that are likely to induce large and diverse changes to the model [6]. This is done by representing each candidate sample x 2 U as gx = r`(x, y = by; ✓L), the ddimensional last-layer gradient that would be obtained if the most likely label according to the model, by, were observed. BADGE selects a batch of samples that have large Gram determinant in this space. The intuition behind BADGE is that sampling proportionally to the Gram determinant of these hallucinated gradients trades-off between uncertainty and diversity; a batch of gradient embeddings that produce a large Gram determinant will need to be both high magnitude (corresponding to model uncertainty) and linearly independent (corresponding to batch diversity). It is worth noting that while BADGE sampling is motivated by determinantal point process (DPP) sampling, the actual BADGE algorithm only uses a rough approximation to this procedure.
Still, from the perspective of BAIT, the “gradient embedding” used in BADGE is a single column of the dk⇥k matrix Vx, but not scaled by p pi. These embeddings can correspondingly be thought of as rank-one approximations for I(x; ✓). BAIT trades BADGE’s determinantal sampling for a trace minimization (A-optimality). This substitution is essential because the determinantal approach is unable to accommodate for I(✓), as argmaxx det(I(x; ✓) 1I(✓)) = argmaxx det(I(x; ✓) 1) for any I(✓).
Thus, BAIT offers two main advantages over BADGE. First, it considers the entire rank-k pointwise Fisher, catching potentially useful information that’s ignored by BADGE. Second, BAIT incorporates the Fisher over all samples I(✓), a term we show to be essential for minimizing risk and bounding MLE error. Crucially, because BADGE identifies this vector as corresponding to the most likely label according to the model, and this only makes sense in classification settings, it is unable to handle regression problems, a regime to which BAIT naturally extends.
Comparing objectives. These observations make it clear that BAIT is more general than BADGE, but it is not obvious which of the aforementioned algorithmic extensions is most important for boosting performance. Figure 2 directly compares those objectives for batch sample selection. Specifically, we run three variations: greedily maximizing the determinant of the rank-one BADGE
gradient embeddings, greedily maximizing the determinant of the full-rank Fisher, and the BAIT approach, taking into consideration both the full-rank pointwise Fisher matrices and I(✓L). The two determinantal algorithms are written formally in the Appendix as Algorithm 2 and Algorithm 3, and can be made efficient by taking advantage of Woodbury identities. Here BAIT uses only forward greedy optimization, rather than both forward and backward, to ensure a fair comparison.
We study two simple projections of the MNIST dataset. In one, we fit a two-layer MLP on 50% of the training data, and embed the remaining 50% using the first layer. We perform active learning in this 128-dimensional space on the unseen 50% of examples, selecting 50 batches of size 10 in sequence.
We then conduct a similar experiment, but instead of using a learned representation, we use a random (Gaussian with mean zero and unit variance) matrix to project samples into 128 dimensions, a setting in which, unlike in the learned representation, the largest directions are not necessarily the most discriminative. Note that this representation allows us to control feature dimensionality but mimics the typical convex learning paradigm, where features are fixed, not conditioned on labels, and not controllable by the learner.
In both plots, the BAIT objective outperforms determinantal objectives. This effect is more drastic for the uninformed embedding, where the plot suggests that the full-rank pointwise Fisher is more useful than its low-rank counterpart for late-stage performance, and that I(✓L), as used in BAIT, is especially beneficial for early stage performance.
Synthetic experiment. We conduct a small synthetic experiment to demonstrate the value of incorporating the Fisher information matrix into the acquisition strategy. In Figure 3 we plot the exact Bayes Risk (3) in the Bayesian linear regression setup described in Section 4 as a function of the batch size B for both BAIT and the greedy determinant maximization strategy. Here we consider two distributions in d = 100 dimensions. In the left plot data are generated from a Gaussian distribution with diagonal covariance with quadratic spectral decay ⌃ii / 1/i2. On the right, the distribution is supported only on the standard basis, with probabilities that decay quadratically pi := P[x = ei] / 1/i2. Note that both distributions have identical and poorly conditioned covariance ⌃ (recall (3)).
This allows us to highlight the value of the Fisher matrix and how it leads to robust performance across data distributions. Indeed, we see that in the Gaussian case, both the BAIT strategy (called “Trace+Fisher” in the figure) and the determinental maximization strategy (“Log-det”) perform almost identically. However, BAIT significantly outperforms the alternative in the orthonormal case. This occurs because the latter does not exploit the occurrence probabilities pi and in fact simply selects the coordinates in a cyclic fashion. On the other hand, the optimal strategy focuses effort on the high-probability coordinates, which is exactly captured in the Fisher matrix.1

heading: 6 Experiments
text: In this section we detail extensive experiments that highlight the generality and performance of BAIT. We consider three settings: linear classification, deep classification, and regression. Throughout these sections, we compare BAIT to several recently proposed and classic active learning approaches.
Among these, we consider BADGE, CORESET, CONFIDENCE, and RANDOM sampling. BADGE, as mentioned earlier, is a state-of-the-art approach that trades off between diversity and uncertainty by approximately sampling a batch of points that have high Gram determinant when represented as a
1Note that in the orthonormal case, both greedy optimization algorithms are in fact optimal for their respective combinatorial problems.
gradient. CORESET represents items using the model’s penultimate layer representation, then samples a batch that describes the space well. CONFIDENCE sampling selects the n points for which the model is least confident, measured by max f(x; ✓). RANDOM draws n points uniformly at random.

heading: 6.1 Linear Classification
text: Like BADGE and CORESET, BAIT caters to efficiency in part by only considering the last layer of the network to select a new batch. Despite this linear assumption, both CORESET and BAIT are unable to perform well outside of the neural regime.
This subsection revisits the simplified setting described in Section 5.1 and Figure 2, involving both informed and uniformed representations of MNIST. In the learned representation, performance differences between algorithms is relatively subdued, with BAIT, BADGE, and CONFIDENCE among the highest-performing agents. However, in the unstructured, random representation, the are stark differences in accuracy. While controlling for dimensionality, this representation mimics the convex case, where the model is not able to control how data are represented. Here, BAIT outperforms baseline approaches by a large margin (Figure 4).
Among these comparisons, we include a simplified version of BAIT, which omits the Fisher term I(✓), resulting in an objective that has been explored by [36]. This approach performs on par with other baselines, suggesting that it is the inclusion of I(✓) that allows BAIT to succeed even in difficult, poorly structured feature spaces. This experiment further highlights a potential cause of the success of these baselines, as the penultimate-layer representation will behave more like what’s described here as a learned representation than a random representation. Still, the following subsection shows BAIT outperforming baselines in deep classification.
6.2 Deep Classification
We now turn to our main experiments, active learning for classification with neural networks. This subsection provides extensive results for the above algorithms across a wide array of settings.
We consider three datasets. Using an MLP, we perform active learning on both MNIST data and OpenML dataset 155. We also use the SVHN dataset [37] of color digit images with both an MLP and an 18-layer ResNet. Last we explore the CIFAR-10 object dataset [38] with a ResNet. All dataset-architecture pairs are experimented with at three batch sizes—10, 100, and 1000. MLPs include a single hidden ReLU layer of 128 dimensions.
All ResNets are trained with a learning rate of 0.01, and all other models (including linear models shown earlier) are trained with a learning rate of 0.0001. We fit parameters using the Adam variant of SGD, and use standard data augmentation for all CIFAR-10 experiments. Like other deep active learning work, we avoid warm-starting and
retrain model parameters from a random initialization after each query round [5]. Each learner is initialized with 100 randomly sampled labeled points, and each experiment is repeated five times with different random seeds. Shadowed regions in plots denote standard error. More empirical details can be found in Appendix Section C.
Figure 6 zooms in on the discriminative regions of learning curves corresponding to three different settings. While the relative performance of baseline algorithms varies somewhat across scenarios, BAIT is consistently as good or better than the highest-performing approach. Full learning curves are presented in Appendix Section C.1.
Due to the volume of settings investigated, we present aggregate results using the analysis approach of [6]. For each experiment, we note the round r of active learning for which random selection first obtains accuracy within 1% of its final accuracy. We then checkpoint each algorithm at exponential intervals up to r, that is, we log each labeling budget L for which Lk = M0 + 2kB  r, for batch size B and number of seed samples M0. At each L in a given experiment, we compute the t-score, t = p Nbµ b , where N is the number of samples, between each pair of algorithms i 6= j as
bµ = 1 N
NX
l=1
(eli elj), b = vuut 1 N 1 NX
l=1
(eli elj bµ)2,
where eli and elj denote the l-th accuracy respectively corresponding to algorithms i and j at labeling budget Lk. We then perform a two-sided t test, where algorithm i is said to outperform algorithm j if t > 2.776, and vice versa if t < 2.776, marking a significant difference (p < 0.05). This formulation allows us to construct a pairwise penalty matrix over all conducted experiments. The matrix has as many rows and columns as there are considered algorithms (five); if algorithm i outperforms algorithm j for some experiment at some labeling budget, the corresponding element i j of the matrix is incremented by 1/z, where z is the total number of labeling budgets considered for that experiment.
The resulting plot is given in Figure 5, which aggregates results over all conducted experiments, and which suggests BAIT significantly outperforms baseline approaches. We also include columwise averages, which give a holistic perspective on algorithm performance.
We show more pairwise plots of this type in Appendix Section C.2, breaking up results by batch size and and architecture type. These figures all suggest BAIT is higher-performing than baseline approaches across environments.
6.3 L2 Regression
Although deep learning is most commonly discussed within a classification framework, recent work has successfully applied deep learning in regression settings as well, with important scientific applications including areas like physical, biological, and chemical modeling [39, 40]. It is therefore important to develop active learning algorithms that are flexible enough to be applied in these domains.
Figure 7 presents active learning results using two different model architectures, two different batch sizes, and two different datasets. In the first, we train an 18-layer ResNet to predict the year in which photos from an American yearbook were taken [41]. To do this successfully, the
model must learn to correlate trends in photography and fashion with a time period. Here labels were Z-scored, so error is not measured in terms of year. In the second, we use meteorological features and a linear model to predict the amount of rainfall in Austin, Texas [42]. Note that there are few active learning algorithms made with regression in mind—the CONFIDENCE and BADGE algorithms are omitted here, as they rely on a notion of uncertainty that requires a classification environment.
In Figure 8, we show a few regression active learning experiments that have been synthetically adapted from the classification setting. We treat SVHN and MNIST data as having k continuous outputs, regressing onto one-hot encodings of their labels.
Similar to the classification case, the relative performance of baseline approaches shuffles between environments. Batch active learning in regression is challenging, with simple random sampling often being surprisingly effective. Still, regardless of which baseline is highest performing, BAIT consistently performs as well or better on both linear and non-linear regression tasks. Further, because BAIT can be reduced to rank-one calculations (Equation 6), it is relatively efficient, and takes about as long to run as CORESET (Figure 9).

heading: 7 Discussion
text: This article studies neural active learning from the theoretical perspective of maximum likelihood estimators, a viewpoint that sheds new light on the performance of previous approaches. We proposed BAIT, a generalized, high-performing, and tractable approach to neural active learning that makes use of this perspective, showing that a more classical approach is tractable and effective with modern neural architectures. We demonstrated that BAIT is successful in both convex and non-convex scenarios, and for both classification and regression settings.
It is worth noting that, while tractable, the classification version of BAIT is more computationally intensive than BADGE—roughly k times slower to select a sample to include in a batch (in seconds, though total run times are largely dominated by retraining models after each batch acquisition [5]). This added computation is well justified in active scenarios for which the cost of label acquisition is high relative to the cost of computation. To trade-off between computation requirements and performance, one could estimate the Fisher using only the lowest-norm r < k columns of Vx, catching the more descriptive components of the Fisher. We leave the analysis of such an approach as an avenue for future work.

heading: 8 Acknowledgements
text: Sham Kakade acknowledges funding from the National Science Foundation under award #CCF1703574.

Review summaries:
The paper introduces an algorithm (BAIT) for batch active learning, designed for and tested on neural networks. It centres around an objective in eq 4, argmin_select trace(I_select^-1, I_all), with I as the Fisher information matrix of the final layer parameters of a neural network, an objective that has been previously derived/analysed in simpler settings. The contribution here is applying it to deep neural networks. It draws links with a current SOTA method, BADGE, and shows slight performance benefits over it.

The paper introduces a new principled active learning algorithm called BAIT, which uses the Fisher information matrix to select informative samples. The approach uses a greedy approximation to solve the global batch acquisition problem. Compared to other approaches, it also works for regression.  The paper provides ablation and gives a new perspective on BADGE, another active learning algorithm.  Multiple experiments both for classification and regression are provided.