Active Learning in CNNs via Expected Improvement Maximization
Authors: Udai G. Nagpal
David A. Knowles
Venue:  (2021)
Keywords: active learning
batch-mode active learning
deep learning
convolutional neural networks
supervised learning
regression
classification
MC dropout
computer vision
computational biology

Abstract: Deep learning models such as Convolutional Neural Networks (CNNs) have demonstrated high levels of effectiveness in a variety of domains, including computer vision and more recently, computational biology. However, training effective models often requires assembling and/or labeling large datasets, which may be prohibitively time-consuming or costly. Pool-based active learning techniques have the potential to mitigate these issues, leveraging models trained on limited data to selectively query unlabeled data points from a pool in an attempt to expedite the learning process. Here we present "Dropout-based Expected IMprOvementS" (DEIMOS), a flexible and computationally-efficient approach to active learning that queries points that are expected to maximize the model's improvement across a representative sample of points. The proposed framework enables us to maintain a prediction covariance matrix capturing model uncertainty, and to dynamically update this matrix in order to generate diverse batches of points in the batch-mode setting. Our active learning results demonstrate that DEIMOS outperforms several existing baselines across multiple regression and classification tasks taken from computer vision and genomics.

Under review as a conference paper at ICLR 2021
ACTIVE LEARNING IN CNNS VIA EXPECTED IMPROVEMENT MAXIMIZATION
Anonymous authors
Paper under double-blind review
ABSTRACT
Deep learning models such as Convolutional Neural Networks (CNNs) have demonstrated high levels of effectiveness in a variety of domains, including com- puter vision and more recently, computational biology. However, training effec- tive models often requires assembling and/or labeling large datasets, which may be prohibitively cone-conuuming or costly. Pool-based active learning techniques have the potential to mitigate these issues, leveraging models trained on limited data to selectively query unlabeled data points from pool in an attempt to expe- dite the learning process. Here we present "Dropout-based Expected IMprOve- mentS" (DEIMOS), a flexible and computationalll-effiiii learning that queries points that are expected to maximize the model's improve- ment across a representative sample of points. The proposed framework enables to maintain prediction covariance matrix capturing model uncertainty, and to dynamically update this matrix in order to generate diverse batches of points in the batch-mode setting. Our active learning results demonstrate that DEIMOS outper- forms several existing baselines across multiple regression and classification tasks taken from computer vision and genomics.
approach to active
INTRODUCTION
Deep learning models (LeCun et al., 2015) have achieved remarkable performance on many chal- lenging prediction tasks, with applications spanning computer vision (Voulodimos al., 2018). computational biology (Angermueller et al. 2016), and natural language processing (Socher et al., 2012]. However, training effective deep learning models often requires a large dataset, and assem- bling such dataset may be difficult given limited resources and time.
Active learning (AL) addresses this issue by providing a framework in which training can begin with a small initial dataset and, based on an objective function known an acquisition function, choosing what data would be the most useful to have labelled (Settles) 2009). AL has successfully streamlined and economized data collection across many disciplines (Warmuth et al.] [003]] Tong & Koller 2001 Danziger et al., 2009 Tuia et al. 2009 Hoi et al.] [2006] [Thomsoon et al.] 1999). In particular, pool-based AI. selects points from given set of unlabeled pool points for labelling by an external oracle (e.g. human expert or biological experiment). The resulting labeled points are then added the training set, and can be leveraged improve the model and potentially query additional pool points (Settles 201T).
Until recently, few AL approaches have been formulated for deep neural networks such as CNNs due to their lack of efficient methods for computing predictive uncertainty. Most acquisition functions used in AL require reliable estimates of model uncertainty in order to make informed decisions about which data labels to request. However, recent developments have led to the possibility of computationally tractable predictive uncertainty estimation in deep neural networks. In particular, a framework for deep learning models has been developed viewing dropout (Srivastava et al., (014) as an approximation to Bayesian variational inference that enables efficient estimation predictive uncertainty (Gal & Ghahramani 2016).
Our approach, which we call "Dropout-based based Expected IMprOvementS" (DEIMOS), builds upon prior work aiming to make statistically optimal AL queries by selecting those points that minimize expected test error (Cohn et al.) [9966. Gorodetsky & Marzouk] 2016 Binois et al. 2019; Roy & McCallum 2001). We extend such approaches CNNs through a flexible and computationally

---

Under review as a conference paper at ICLR 2021
efficient algorithm that is primarily motivated by the regression setting, for which relatively few AL methods have been proposed, and extends to classification.
Many AL approaches query the single point in the pool that optimizes certain acquisition function. However, querying points one at a time necessitates model retraining after every acquisition, which can be campuutatoonall--xpennnii
and can lead time- consuming data collection (Chen & Krause, 2013). Simply greedily selecting a certain number of points with the best acquisition function values typically reduces performance due to querying similar points (Sener & Savarese 2018
Here we leverage uncertainty estimates provided by dropout in CNNs to create a dynamic repre- sentation of predictive uncertainty across large, representative sample of points. Importantly, we consider the full joint covariance rather than just point-wise variances. DEIMOS acquires the point that maximizes the expected reduction in predictive uncertainty across all points, which we show is equivalent to maximizing the expected improvement (EI). DEIMOS extends to batch-mode AL, where batches are assembled sequentially by dynamically updating a representation of predictive un- certainty such that each queried point is expected to result in a significant, non-redundant reduction in predictive uncertainty. We evaluate DEIMOS and find strong performance compared to existing benchmarks in several AL experiments spanning handwritten digit recognition, alternative splicing prediction, and face age prediction.
RELATED WORK
AL is often formulated using information theory (MacKay 1992b). Such approaches include query-
ing the maximally informative batch of points as measured using Fisher information in logistic re- gression (Hoi al.) 2006). and Bayesian AL by Disagreement (BALD) (Houlsby al. [01T)) which acquires the point that maximizes the mutual information between the unknown output and the model parameters.
Many AL algorithms have been developed based on uncertainty sampling, where the model queries points about which is most uncertain (Lewis & Catleti 1994 Juszezak Duin, 2003). AL via
uncertainty sampling has been applied to SVMs using margin- basgin-based uncertainty measures (Joshi jt al. 2009). AL has also been cast as an uncertainty sampling problem with explicit diversity maximization YYng et al. 2015) to avoid querying correlated points.
El has been used as an acquisition function in Bayesian optimization for hyperparameter tuning Eggensserrer et al. (003)). Other AL objectives similar to ours have been explored in (Cohn et al.) [996||Gorodetskk & Marzouk 2016 Binois et al. 2019 Roy & McCallum (00I), making statisti- cally optimal queries that minimize expected prediction error, which often reduces to querying the point that is expected to minimize the learner's 's variance integrated over possible inputs. DEIMOS extends EI and integrated variance approaches, which have traditionally been applied to Gaussian Processes, mixtures of Gaussians, and locally weighted regression, to deep neural networks.
Until recently, few AL. approaches have proven effective in deep learning models such as CNNs, largely due to difficulties in uncertainty estimation. Although Bayesian frameworks for neural networks (MacKay) [992a| Neal 1995) have been widely studied, these methods have not seen widespread adoption due to increased computational cost. However, theoretical advances have shown that dropout, common regularization technique, can be viewed as performing approximate variational inference and enables estimation of model and predictive uncertainty (Gal & Ghahra- mani 2016). Simple dropout- based AL objectives in CNNs have shown promising results com- puter vision classification applications (Gal et al.) 2017).
Several new algorithms show promising results for batch-mode AL on complex datasets. Batchbald (Kirsch et al.) 2019) extends BALD to batch-mode while avoiding redundancy by greedily con- structing query batch that maximizes the mutual information between the joint distribution over the unknown outputs and the model parameters. Batch-mode AL in CNN classification has also been formulated as a core-set selection problem (Sener & Savarese 2018) with data points represented (embedded) using the activations of the model's penultimate fully conneced layer. The queried batch of points then corresponds to the centers optimizing a robust (i.e. oullier-toleran) k-Center objective for these embeddings.
2

---

Under review as a conference paper at ICLR 2021
In spite of these recent advances, there no frameworks for batch-mode AL in CNNs that model the full joint (rather than point-wise) uncertainty and do not require a vector space embedding of the data. Additionally, few AL approaches for CNNs have been assessed (or even proposed) for regression as compared to classification.
3 ACTIVE LEARNING VIA EXPECTED IMPROVEMENT (EI) MAXIMIZATION
3.1 MOTIVATION
Our pool-based AL via El maximization framework, DEIMOS, aims to maximally reduce expected (squared) prediction eπror on a large, representative sample of points by querying as few points from the pool as possible. Let Dsamp = samp Ysamp) be an sufficiently large random sample from the training and pool points that it can be assumed representative of the dataset as a whole. Let D Dpool pool ypoat) denote the available pool of unlabelled data, (Inew, Ynew) E D candidate point for acquisition, 9 the model parameters with approximate posterior q()), and ÿı (0) the model prediction for an input vi,, Note that peol not known and Y samp is only partially known in AL as the pool consists of unlabeled points. For brevity, conditioning model input is omitted in subsequent equations (c.g. E, [0: (0) Inew. x..] will be written as Eg (0) Nnew)).
We seek to maximize El by acquiring the pool point that minimizes expected prediction error on Daamp- The expected prediction error on samp conditioned on some Tnew X pool is,
Σ
Equn (()) Jac=]
samp
(E Danp samp
1
Σ (EcD)E
Fnew] + (Th. (0) [new] F., Ev(lv))? + [(= F.n(vs))7] -
ssampl
(1) where E, and E., denote expectation over y(0) and observation noise, respectively, and Eg,, de- notes joint expectation over g(0) and observation noise. Here we see the terms contributing to the model's expected prediction error on D samp are (from left to right): 1) the trace of the pre- dictive variance matrix, 2) the sum of the predictive biases squared, and 3) the sum of the noise variances across all observations (a constant knew) (Cohn al.] 1996). For a purely supervised model, expected predictions remain the same unless the training set of input-output pairs is modi- fied. Therefore, expected model predictions are unchanged conditioned on any unlabeled point nnew: (0) Znew] E, [r))), and the predictive bias squared for any point (I4, y:) stays the same conditioned on Inow: (E, [ (0) Znew - [y. 22 = (Eg D (f)] - E, [2al)) Substituting accordingly in equation T. the pool point that would minimize expected prediction error on Dramp queried is:
arg min
tr (Var, (Fsamp (0) Knew)) ))
Tacn E X poct
samp
where Ysamp(() (0) denotes the model predictions for input Xamp and fixed parameters O and Var, denotes variance over the approximate posterior. Therefore, the neew that minimizes expected pre- diction error upon being queried is that expected to minimize average prediction variance (specif- ically the trace of the predictive covariance matrix). Under our assumptions, knowing Vnew is suf- ficient to calculate the predictive variance contribution expected prediction error conditioned on (Inew; Voew) ) even when Unew is unknown. Importantly, it follows that, even though Ypool is not known in AL, DEIMOS is still able identify and query the pool point Tnew E Xpool that minimizes expected prediction error across Dsamp after (Inow, Unew) is incorporated into the training set.
3.2
MC DROPOUT VARIANCE ESTIMATION
The proposed approach can be implemented with any uncertainty estimation method that enables calculation of the variance and covariance of model predictions. In our experiments, we use MC dropout (Gal & Ghahramani] 2016 to obtain estimates of model uncertainty in CNNs. Models are trained with dropout preceding all fulle-connected layers and dropout is used at test time generate T approximate samples from the posterior predictive distribution. The predictive mean is obtained as the average of these samples.
3

---

Under review as a conference paper at ICLR 2021
In order to estimate the predictive covariance matrix for all sample points, J dropout masks domly generated for all dropout layers in the neural network. Crucially, the same dropout masks used to make test-time predictions across all sample points to enable estimating correlation be- tween them. From the dropout sample covariance matrix Var, (Y samp.dropour capturing the sample variances and covariances of the J dropout predictions for each sample point, one can estimate the prediction covariance matrix (Gal & Ghahramani] 2016).
Var, (amp (0) - Var, samp (0))
(3)
Here (1=p)1²
2NA represents the model precision in regression tasks, where p is the dropout prob- ability, is a prior length-scale parameter, N the number training points, and À represents the weight decay (Gal & Ghahramani 2016). In the classification setting, 7 7- 0.
3.3 DEIMOS ACTIVE LEARNING IN CNN REGRESSION
DEIMOS queries the pool point Tnew that minimizes the expected prediction variance across Dsamp upon being queried and, by equation 2 minimizes the expected prediction error on Ysamp upon querying Vnew. Assuming that all it sap are jointly Gaussian, and considering candidate point for acquisition Nnew E Xpool:
Var, (Vrmmp (0) Fnew) Var, Var, (Ysum (0) CCova (msm (0), Bnew Var, (Unew (0)) Cov, (Yrump Pnen (0))² samp
where Cov, (Ysamp (0)) (0)) is a sammll x 1 column vector. Therefore, in order to maximize the expected model improvement and minimize the expected prediction error on Ysamps DEIMOS queries the point:
x. arg min tr( (Var, (Far,(Fsumgg (Fsump Samp(0) Fnew
Vee Kow Expst
arg max tr(Cov, "amppp (0), Unew (0) (Jnew (0)) - (Psump (0) Expst
Snew (0))²)
(5)
The El in regression is defined to be the quantity maximized in equation predictive variance across sample points upon querying a given pool point.
5 the total reduction in
3.4 DEIMOS ACTIVE LEARNING IN CNN CLASSIFICATION
In the classification setting, test-time dropout remains applicable as a means of drawing from the approximate model posterior. Here the DEIMOS approach for regression is extended to maximizing the expected reduction in uncertainty in predicted class probabilities classification. Assuming all predicted class probabilities across all sample points, denoted by Psamp. are jointly Gaussian:
Var, (Fsamp (0))zazw) = Var, (Psamp (0))
Cov, (pxmmp ()), Prew (0)) (Var, (Prew (0)) + 11) 1)) (isvv((samm(0)) (0) Pnew (0))
(6)
where is the number of classes, Var ( (0)) is a c|Ysumpl x Y samp matrix, Cov, (Pramp Prew (0)) is a c|msumpl matrix, Var, (Pnew (0)) is a X matrix, and << is smoothing parameter ensuring the invertibility of Var, (P
) We define El in classification the total reduction in variance of predicted class probabilities across all sample points that is expected upon querying a point, which is given in equation 6 by: (PaCon (0), Prew
(0)) +
T.- T)= Cov, Cov, (P<mmp (0), Pnew ()))T). All quantities in equation 6 are estimated using MC dropout equation 3 DEIMOS maximizes the EI by querying the point Znew E Xpoo that minimizes
tr( Var, [Psamp (()]))))) where Var, (Peamp (0) Inew) is given by equation
Treating the predicted class probabilities Psamp jointly Gaussian may seem poor approximation as probabilities are bounded. However, we find that DEIMOS performs better in the bounded prob- ability space than in the unbounded logit space, possibly because in the logit space unimportant differences between large positive (or negative) predictions that correspond to small differences in predicted probabilities are given undue importance.
4

---

Under review as a conference paper at ICLR 2021
Applying the joint Gaussian assumption to the class probabilities for all classes across all sample points provides an expression for the probability covariance matrix conditioned on each unlabeled candidate point. DEIMOS acquires the point that maximizes the expected reduction in predictive uncertainty across all classes and all points.
DEIMOS BATCH-MODE ACTIVE LEARNING
Algorithm DEIMOS batch-mode active learning in regression
Input: sampp Xuain fixed-mask dropout predictions Ý
samp.dropout batch size b, model precision
S = Xxampl Xbatch
Ø
V V
Vs Var, (Y, samp (0))
I Varg (Y dropod (0))
for i E (1,...,b) do
Xeand X samp
(Ximin U Xbatch)
ir VV:VVT (v²,')
In arg max, Xins
V V (v
X butch = X butch U Jan
end for return Xhatch
The joint Gaussian assumption, applied to real-valued outputs in regression or class probabilities in classification, enables estimation of the predictive covariance across the representative sample of points conditioned on unlabeled points, In the batch-mode setting, DEIMOS assembles each batch sequentially. After a point is added to the batch, having maximized the DEIMOS acquisi- tion function, the predictive covariance matrix is updated conditioned on the unlabeled point: the updated covariance matrix, in turn, used to calculate the acquisition function values, determine the subsequent point in the batch, and again update the predictive covariance matrix (Algorithm 1). The general procedure of Algorithm applies classification and regression, but in classification 0 and r- 1--1 is introduced, V is cS X cS matrix, V; is c$ X matrix, and Vi: is X matrix. Candidate points for acquisition are limited to the unlabeled points in Xsmmp as opposed to all of Xooo in Algorithm 1 and in our AL experiments in the interest of computational efficiency.
The DEIMOS approach to batch-mode AL provides a computationally efficient mechanism for batch selection. Assembling batches sequentially and updating the covariance matrix after cach point is added ensures that all acquired points result in relatively high reductions in predictive uncertainty and that no two points in the queried batch redundant in their reduction of predictive uncertainty across the sample points. For some Var, (Y, (m)p))) (0)). there can exist batches of b points that result in a greater variance reduction than the b points queried by the proposed greedy algorithm. However, simulation experiments for small batch sizes show that the variance reduction resulting from our greedy algorithm is typically very close to the optimal variance reduction (Appendix
EXPERIMENTS
First, we visualize MC dropout uncertainty estimates and the proposed acquisition function for synthetic data in a one-dimensional input space. Next, we evaluate DEIMOS empirically via exper- imental comparison with other acquisition functions and approaches.
We tested DEIMOS in CNNs across regression and classification tasks, and in single-point acqui- sition and batch-mode AL S Specifically, DEIMOS is compared to existing methods in alternative splicing prediction (Rosenberg et al., 2013) and face age prediction (susanqq) 2017) regression tasks. Classification experiments were run on MNIST (LeCun et al. 1998) binary clas- sification of handwritten and 9s, and on classification of all 10 digits. All AL results shown are averaged over three experiments, each beginning with new, randomly-selected training set. AL.
AL experiment details are detailed in Appendix D and hyperparameter tuning details are discussed in Appendix Code for all experiments is included in the supplementary material.
5

---

Under review as a conference paper at ICLR 2021
0.75
value
Training points
Normalized EI
0.90
Queriee
0.00
-0.
-0.50
model predictions New model presictions
training
-1.00
Original training points
-75
-5.0
-2.5
0.0
-10.0
-75
-5.0
-2.5
0.0
2.5
5.0
7.5
10.0
Input
Input
Figure 1: Left: DEIMOS acquisition function and batch query for simulated 1D data. Model pre- dictions 1 standard deviation shaded light blue. Points selected be queried in batch of size 5 are indicated on the x-axis (batch construction order: darker to lighter markers). Right: Model predictions after acquisition. The queried points from ((a) are labeled and added to the training and new model is retrained from scratch. substantially improved fit is obtained.
Table I: Number of single-point acquisitions required in order to achieve the 5' splicing test MSE listed the left column for each acquisition function.
Test MSE
DEIMOS
Max. variance acquisition
Random acquisition
0.11
69
92
80
0.10
109
318
180
0.095
174
492
306
0.09
369
708
387
performance is compared across algorithms using linear mixed model to account for dependence across acquístion iterations (Appendix C
5.1 VISUALIZING DEIMOS ACQUISITION IN ID NEURAL NETWORK REGRESSION
In order to visualize the DEIMOS acquisition function we simulated a 1D neural network regression task. Synthetic data with real-valued input and output is produced by a random dense neural network and then used to train dense neural network (Appendix D)
The DEIMOS acquisition function exhibits desirable properties an AL objective. It is high in [-2.5, 2.5]. where data is comparatively sparse, and [-10, -7). where the current model does not fully capture the underlying trend (Fig left). In the [2.5, 10j imput region where the model pre- dictions fairly accurate, EI is relatively low. DEIMOS batch-mode AL queries diverse set of points, concentrating on the [-3, 3] input region but also querying two points in the surrounding regions (Fig 1 left). In this example, DEIMOS batch-mode AL successfully improves model per- formance (Fig I right), querying points that are expected to reduce predictive uncertainty but are not redundant. Indeed, upon labeling of the requested points, model fit significantly improves in the [-10, -7] and 2-2,5, 2.5) input regions.
5.2 ACTIVE LEARNING EXPERIMENTS: CNN REGRESSION
We evaluated DEIMOS on prediction splice donor site 5' alternative splicing (Rosenberg et al., 2015) and face age prediction for images in the UTKFaae dataset (susanqq 2017). In each exper- iment, all acquisition functions start with the same initial training set, and, in each AL iteration, acquire specified number of points ("batch size") and the models retrained from scratch. The performance of DEIMOS is compared to that of random acquisition, and to acquiring the candidate pool point(s) with maximum MC dropout variance.

---

Under review as a conference paper at ICLR 2021
acquisition acquastion DEMOS
Maximum variance acquistion Random acquisitin
Hastmam acquistiio
0.16
DEIMOS
Randoms acquisiiion
Hean eroor test
0.4
RXs entr tcst set
RMS enror tarst
000
100
200
C29
800
Z50 sdo Po s000 1390 1900 1793 2000 Number asquire points 1130 initial training points)
256 500 750
1006 3259 1568 3750 2000
Number acquied points (13 Initial training points)
Number of acqulred points (150 inttaal training point)
Figure 2: Test performance across AL iterations with 1 standard deviation of average performance shaded. Left: 5' splicing (batch size 1) MSE; DEIMOS outperforms benchmarks (p = 2.1 x 10 278
for maximum variance acquisition and p = 6.9 x 10 10-34 34 for random acquisition) (Appendix
Center: Face age prediction RMSE (batch size 100); DEIMOS outperforms benchmarks (p
8 x 10 10 (Appendix C). Right: Face age prediction RMSE (batch size 200); DEIMOS outperforms
benchmarks (p- < 9 10 Appendix
Alternative RNA splicing prediction. CNN regression models are trained to predict the relative usage of one of at two alternative splice donor sites. One-hot encoded RNA sequences of 101 nucleotides for each set of splicing measurements are used as the model input.
Results are illustrated in Fig 2 (left). DEIMOS outperforms both benchmarks over the first several hundred iterations but random acquisition catches up to DEIMOS near the end of the experiment. DEIMOS achieves lower average MSE than both benchmarks for a given experiment and AL iter- ation (p < 10-33 Appendix C). Table 1] illlsstrttes the substantial data efficiency gains produced by DEIMOS: both alternative methods require over 65% more data than DEIMOS to achieve a test MSE of 0.10, and both require at least 75% more data than DEIMOS to achieve a test MSE of 0.095.
Maximum variance acquisition performs poorly throughout, indicating that acquiring uncertain points does not necessarily lead to effective AL. its own. The strong performance of DEIMOS compared to maximum variance acquisition demonstrates the benefit over traditional uncertainty sampling approaches gained by modeling correlations between predictions across data points and assessing El throughout the input space.
Face age prediction. AL experiments were run on the UTKKFace dataset (Fig 2 center and 2 right), with models trained to predict the age corresponding formed better than random and maximum variance acquisition in these experiments, with its root MSE (RMSE) being significantly lower than that of the other acquisition functions for given ex- periment and AL iteration. Notably, DEIMOS outperforms both benchmarks even for moderately large batch sizes (100, 200), providing empirical evidence for the effectiveness of the proposed batch-mode algorithm in querying informative, non-redundant points.
each 200x200) pixel face. DEIMOS per-
5.3 ACTIVE LEARNING EXPERIMENTS: CNN CLASSIFICATION
We evaluated DEIMOS on MNIST LeCun et al. 1998) for binary classification of 7s vs 9s and multiclass classification of all 10 digits. We compare to random acquisition, acquisition of the point(s) with maximum entropy (Gal et al. 2017). BAI.D/atchbald (Houlsby et al. 2011 Kirsch et al. 2019), and core-set AL via robust k-Center (Sener & Savarese 2018).
In binary classification of 7s and 9s, DEIMOS compares favorably to existing approaches. For single-point acquisitions, DEIMOS requires roughly one-fourth as much data as random acquisition to arrive at test accuracy thresholds of 0.97 and 0.975, and shows data efficiency improvements over all other benchmarked methods (Talle 2 Appendix B). For batch size 25 (Fig 3(a),, DEIMOS per- forms substantially better than random acquisition throughout the experiment and improves upon Batchbald and robust k-Center. The strong performance of DEIMOS is robust to the choice of rep- resentative sample size: there is statistically significant difference in DEIMOS AL performance
7

---

Under review as a conference paper at ICLR 2021
1.00
79
9
7
7
7
Accuracy on test
0:90
7
7
7 4
0.22
Accuracy
0:55
Maximum entrapy acquisition
9
7
Random acquisition
Maximum Entropy Acquisition
BetchBALD acquisition
Random Acquisition
0.75
Robust i- Center acquisition
BetchBALD Acquisition
7
DEIMOS
203 Number of acquired points (10 initial training points)
400
300
1000 acquired (30 training points)
(c)
(b)
entropy Random ecquistion BatchBALD acquisition
Entropy Acquisitio
Acculsttion
BatchBALD Acquistioo DEAMOS
DEINGS
0.503
Semoae Spr entt K. et sanven
6.03
dierpero
oxea
0:53
Random acquisttion
DENOS (2500 sample points)
DENOS (500 semple points)
6300
OE MOS (5000
100
20
200
830
1099
Number of acquirod points (LD intal traning points)
(f)
(d)
Figure 3: (a) MNIST 7 vs. 9 classification accuracy (=1 standard deviation shaded) across AL iterations for batch size 25. DEIMOS outperforms random and robust k- L-Center acquisition (p-values 4.4 x 10-11 and 1.9 10-11, respectively (Appendix C) (b) MNIST 0-9 classification accuracy (+1 standard deviation shaded) across AL iterations for batch size 25. DEIMOS outperforms random 10-3 10-3) acquisition (Appendix (c) Images 1-3 and 999-1000 (order: top to bottom) queried by DEIMOS in MNIST 7 vs. 9 classification with batch size 25; each column corresponds one experiment (order: left right). (d,e) Batch diversity, defined as average symmetric KL divergence of predicted class probabilities for queried points in each batch, across AL iterations MNIST vs. classification with batch size 25 and MNIST 0-9 classification with batch size 25, respectively. DEIMOS has higher batch diversity than Batchbald and maximum entropy acquisition (p 0.01) in almost AL iterations in both tasks (f) DEIMOS MNIST 7 classification accuracy across AL iterations (batch size 25) for several representative sample sizes |Xsampl. There is no statistically significant difference in DEIMOS AL performance across the X sampl shown (Appendix C
(p 10-3) and maximum entropy (p
8

---

Under review as a conference paper at ICLR 2021
Table 2: Top: Number of single-point acquisitions required in order to obtain the MNIST vs.
9 classification test accuracy listed in the left column for each acquisition function (Appendix B Bottom: p-values comparing benchmark accuracy to DEIMOS accuracy (Appendix
Test accuracy
DEIMOS
BALD
Max. entropy
Random
Robust k-Center
0.95
20
23
21
48
24
0,96
24
25
31
55
0.97
27
39
39
113
58
0.975
41
67
58
158
58
p-value
2.3 10-21
4.1 x 10-11
5.2 x 10-133 133
7.7 x 10-16
for representative set sizes of 500, 2500, and 5000 (Fig 3(D). Fig 3(c) illustrates the first three and last two images queried by DEIMOS in the batch size 25 experiments. Early on, DEIMOS queries class prototypes (e.g. first image queried in experiment 1) and probes the decision boundary (e.g. first and third images queried in experiment 2). When the training set is larger, DEIMOS focuses more on querying outliers (e.g. last two images requested in experiment 1) that may only clarify narrow subset of predictions.
In MNIST classification of all 10 digits (Fig 3()),, DEIMOS is comparable to state-of-the-art methods. For batch size 25, DEIMOS outperforms random and maximum entropy acquisition (p < 10-3). Batchbald slightly outperforms other methods early on in the experiments but DEIMOS is comparable to Batchbald most AL iterations. The strong performance of DEIMOS demonstrates the advantages of maximizing EI for all classes taking into account correlations between predicted class probabilities across points.
large sample of points and
The diversity of batches queried by DEIMOS fication experiments for batch size 25 (Fig 3(d), 3(e)). Batch diversity is measured by the average symmetric KL divergence in the predicted class probability distributions for pairs of points in the queried batch. Across both tasks, batches of points queried by DEIMOS had higher batch diver- sity (p 0.01) than those queried by Batchbald and maximum entropy acquisition in virtually all AL iterations. Random acquisition led to batches with the highest diversity, in large part due to the relatively high prevalence of pairs of points with very low predictive uncertainty that are pre- dicted by the model to belong to different classes (leading to high symmetric KL divergence) in randomly-assembled batches. Such points are less likely to queried by other AL methods as they are already predicted confidently and therefore are often uninformative. Figures 3(d) and 3(e) show the effectiveness of the DEIMÓS batch-mode AL algorithm in assembling heterogeneous batches of points.
analyzed in the MNIST vs. and 0-9 classi-
6 CONCLUSION
We formulate an active learning approach, DEIMOS, aimed at maximizing expected improvement by minimizing expected predictive variance. Our approach extends to the batch-mode setting when supplemented with an efficient greedy algorithm that sequentially assembles a diverse batch of queried points. The effectiveness of the proposed algorithm is studied across multiple classifica- tion and regression tasks, and it is found to outperform typical baselines in regression and to achieve performance on par with-and some cases better existiii methods in classification.
REFERENCES
Christof Angermueller, Tanel Pärnamaa. Leopold Parts, and Oliver Stegle. Deep learning for com- putational biology. Molecular Systems Biology, 12:1-16, 2016.
Mickaël Binois, Jiangeng Huang, Robert B. Gramacy, and Mike Ludkovski. Replication or explo- ration? sequential design for stochastic simulation experiments. Technometrics, 61:7-23, 2019.
Yuxin Chen and Andreas Krause. Near-optimal batch mode active learning and adaptive submodular optimization. In Proceedings Machine Learning Research, pp. 160-168, 2013.

---

Under review as a conference paper at ICLR 2021
David A. Cohn, Zoubin Ghahramani, and Michael I. Jordan. Active learning with statistical models. Journal of Artificial Intelligence Research, 4:129-145, 1996.
Samuel A. Danziger, Roberta Baronio, Lydia Ho, Linda Hall, Kirsty Salmon, G. Wesley Hatfield,
Peter Kaiser, and Richard H. Lathrop. Predicting positive p53 cancer rescue regions using most informative positive (mip) active learning. PLoS Computational Biology, 5, 2009.
Katharina Eggensperger, Matthias Feurer, Frank Hutter, James Bergstra, Jasper Snoek, Holger Hoos, and Kevin Leyton-Brown. Towards an empirical foundation for assessing bayesian optimiza- tion of hyperparameters. In Advances in Neural Information Processing Systems workshop Bayesian Optimization in Theory and Practice, 2013.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pp. 1050-1059, 2016.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning, pp. 1183-1192, 2017.
Alex Gorodetsky and Youssef Marzouk. Mercer kernels and integrated variance experimen- tal design: Connections between gaussian process regression and polynomial approximation. SIAM/ASA Journal on Uncertainty Quantification, 4, 2016.
Stephen C. H. Hoi, Rong Jin, Jianke Zhu, and Michael R. Lyu. Batch mode active learning and its application to medical image classification, In Proceedings of the 23rd International Conference on Machine Learning, pp. 417-424, 2006.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning, 2011.
A. J. Joshi, F. Porikli, and N. Papanikolopoulos. Multi-class active learning for image classification. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2372-2379, 2009.
Piotr Juszezak and Robert P.W. Duin. Uncertainty sampling methods for one-class classifiers. In Proceedings of the International Conference on Machine Learning, Workshop on Learning from Imbalanced Datasets, 2003.
Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acqui- sition for deep bayesian active learning. In Advances in Neural Information Processing Systems, pp. 7024-7035, 2019.
Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. Mnist database of handwritten digits. http://yann: lecun. haan/exdb/mmtt 1998.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521, 2015.
David D. Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Proceedings of the Eleventh International Conference on Machine Learning, pp. 148-156, 1994.
David MacKay. practical bayesian framework for backpropagation networks. Neural Computa- tion, 4:448 472, 1992a.
David MacKay. information-basse objective functions for active data selection. Neural Computa- tion, -059, 1992b.
Radford M. Neal. Bayesian Learning for Neural Networks. PhD thesis, University of Toronto, 1995.
Alexander B. Rosenberg, Rupali P. Patwardhan, Jay Shendure, and Georg Seelig. Learning the sequence determinants of alternative splicing from millions of random sequences. Cell, 163: 698-711, 2015.
Nicholas Roy and Andrew McCallum. Toward optimal active learning through monte carlo estima- tion of error reduction. In Proceedings of the Eighteenth International Conference on Machine Learning, pp. 441-448, 2001.
10

---

Under review as a conference paper at ICLR 2021
Ozan Sener and Silvio Savarese. Active learning for convolutional networks: A core-set approach. In Interaational Conference on Learning Representations, 2018.
Burr Settles, Active learning literature survey. Technical Report 1648, University of Wiscon- sin-Madison, 2009.
Burr Settles. From theories to queries: Active learning in practice. In JMLR Workshop and Confer- ence Proceedings, volume 16, pp. 1-18, 2011.
Richard Socher, Yoshua Bengio, and Christopher D Manning. Deep learning for nip (without magic). In Tutorial Abstracts of ACL, pp. 5, 2012.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958, 2014.
susangq. Utkface large scale face dataset.
watpb/oo//KKaaCCC 2017.
Cynthia A. Thompson, Mary Elaine CalifT, and Raymond J. Mooney, Active learning for natu- ral language parsing and information extraction. Proceedings of the Sixteenth International Conference Machine Learning, pp. 406 414, 1999.
Simon Tong and Daphne Koller. Support vector machine active learning with applications text classification. Journal of Machine Learning Research, 2, 2001.
Devis Tuia, Frédéric Ratle, Fabio Pacifici, Mikhail F. Kanevski. and William J. Emery. Active learning methods for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 47:2218 22221182222,,2 2009.
Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. Deep learning for computer vision: A review. Computational Intelligence and Neuroscience, 2018.
Manfred K. Warmuth, Jun Liao, Gunnar Rätsch, Michael Mathieson, Santosh Putta, and Christian Lemmen. Active learning with support vector machines in the drug discovery process. Journal of Chemical Information and Computer Sciences, 43:667-673, 2003.
Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, Alexander G. Hauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 11:1113-127 2015.

---

Under review as a conference paper at ICLR 2021
A OPTIMALITY OF DEIMOS BATCH-MODE ACQUISITION
The proposed greedy algorithm for batch-mode active learning is an efficient alternative to evaluat- ing the EI for all possible batches of unlabeled candidate points of specified size and selecting the batch that maximizes EI. Although the proposed algorithm is efficient, its greedy approach means that it does not always find the batch of points that maximizes El. An example of Var, where greedy batch-mode active learning may not maximize EI is provided below:
9
3
Var, (Y camp (0))
3
2
3
2
3 9
For the given Var, VYr,((Yam((0))) (0)), DEIMOS batch-mode acquisition with batch size 2 queries point and then either point or point (both options have equal expected improvement), resulting in a total variance reduction of approximately 16.9. However, the optimal trace reduction (i.e. expected improvement) of roughly 19.6 results from querying points 1 and 3. Therefore, in this example DEIMOS achieves only around 86% of the optimal trace reduction for matrix Var, (Ysapp())))))
Next we explore how sub-optimal the greedily built batches tend be for randomly generated vari- ance matrices. The empirical effectiveness of the proposed approach for sequential batch assembly compared to that of searching through all batches of points and acquiring the batch that maximally reduces predictive variance, We simulate i.i.d. standard normal predictions for specified number of pool points and dropout masks. Var, Y amm (0)) is calculated using the sample covariance between all points, Var, ( dropoud (f)). and adding r -- (equation 3 where -- 1 is set to 10% of the aver- age MC dropout sample variance across all points. Synthetic prediction variance matrices generated in this fashion are used compare the expected improvement from the greedy algorithm, given by the trace reduction in the matrix, to that resulting from acquiring the optimal batch of points.
The procedure detailed above is repeated to generate 200 instances of Var, Var, (Y.amp (f)) across batch sizes of 2, 5, and 10 and for varying numbers of i.i.d. normal variables designated as the number of model predictions per point (i.e. the number of dropout masks). For each Var, (Ysamp(()) (f)). the DEIMOS batch-mode algorithm was and the ratio of the resulting variance trace reduction to the optimal variance trace reduction, computed by searching through all possible batches of points, was computed. Although the scope of such experiments is limited by the exponential time complexity of iterating through all possible batches of points, the trace reduction ratios are typically quite close to 1 throughout our experiments (Figure A.1). Thus, while instances of Var, (Y samp (0)) where the greedy algorithm deviates significantly from the optimal trace reduction do exist, they appear to be relatively uncommon. These results provide some empirical support for the effectiveness of the proposed greedy algorithm achieving a variance reduction across predictions comparable to that resulting from querying the optimal batch of points under our assumptions.
(
B MNIST BINARY 7 VS. BATCH SIZE 1 RESULTS
We show that DEIMOS outperforms several existing active learning methods in MNIST binary clas- sification of 7 vs. 9. With single-point acquisitions and initial training sets consisting of 10 points, DEIMOS performs substantially better than random acquisition, and noticeably better than robust k-Center and BALD, throughout most of the active learning iterations (Figure B.T) Maximum en- tropy acquisition also performs well, achieving performance comparable to that of DEIMOS. Over- all, DEIMOS performs well compared to existing active learning methods for classification in the MNIST 7 vs. classification task.
C HYPOTHESIS TESTING FOR AL PERFORMANCE EVALUATION
We used hypothesis tests based on linear mixed models (LMMs) to compare the performance of dif- ferent AL algorithms in experiments. If two AL algorithms are equally effective, then predictive performance on test data will given as a function the number of acquisitions (the algorithm in use has no effect). However, if one active learning algorithm is more effective than another, predic-
12

---

Under review as a conference paper at ICLR 2021
150
Prequency
Frequency
Frequency
100
200
75-
75
30
30
25-
0.055
2.000
0.995
10000
0.004 0,900
0.908
Greedy Trace Reduction/Optimal Trace Reduction Greedy Reduction/Optimll Trace Reduction
Greedy Trace Reduction/Optimal Reduction
(a) Batch size 2, saml
200, (b) Batch size 2, Yaapll = 200, (c) Batch size 5, |Ysmpl
30. nor-
3 normally distributed samples per 50 normally distributed samples per mally distributed samples per point point
point
375
140-
140
150
Prequency
100
Frequency
100
60-
25
8970 0.955 6960 9985 0.990 0.945 1000 Greedy Trace Reductior/Optimal Trace Reduction
Gready Face Rediccioosysomal Roce Roduction 18-6+1
0.000 0.8ss n.90 n.ss 1.000
10. [Ysmpll 20, Greedy Trace Reduction/Optimal Trace Reduction
(e) Batch
(d) Batch size 5. |Ymmmll 50 normally distributed samples per point. All trace reduction ratios are normally distributed samples per point
30, 3 normally distributed samples per (f) Batch size 10, |Yampl 20,
between 0.99999 and 1.
point
Figure A.1: Histograms of the ratio of the variance trace reduction resulting from applying the proposed greedy algorithm to the variance trace reduction resulting from querying the optimal batch of points, under assumptions. Each trial uses different Var, (0)) initialized by calculating Var, ( (Y samp.drrppuu(()) (0)) + --1 from i..d. normally-distee distributed variables, where a specified number of normal samples corresponds to the model's MC dropout predictions for each point. All ratios of greedy trace reduction to optimal trace reduction are above 0.97 across all experiments.
13

---

Under review as a conference paper at ICLR 2021
0.95
Accuracy test set
0.90
Maximum entropy acquisition
Random acquisition
BALD acquisition
0.80
Robust k-Center acquisition
DEIMOS
50
75
100
150 175 200
Number of acquired points (10 initial training points)
Figure B.I: MNIST vs. 9 classification test accuracy (#: I standard deviation shaded) averaged over three active learning experiments as a function of number of acquired points, with batch size 1.
tive performance on test data will have contributions both from the algorithm in use and from the number acquisitions that have occurred.
Consequently, the null model for predictive performance during AL experiment is given by an LMM with random-effect terms corresponding to the number of elapsed acquisition iterations. The alternative model for predictive performance an LMM with fixed-effect term corresponding to the AL algorithm and random-effect terms corresponding to the iteration.
A likelihood ratio test is used throughout our experiments compare the null model and alternative model and determine if the fixed effect corresponding to the choice of AL algorithm is statistically significant.
D ACTIVE LEARNING EXPERIMENT DETAILS
Synthetic 1D neural network regression. 1D real-valued output is generated from input using a random neural network with zero-mean Gaussian weights and biases and fully-connected hidden layers of size [32, 32]; observed output is corrupted by zero-mean, i.i.d. Gaussian noise. A fully- connected neural network with hidden layers size [256, 256, 256), with = 0.2 and 0.0005, is fit to the synthetic data and used to evaluate EI.
Active learning experiments. Active learning experiments are run on individual Tesla P100 and Tesla V100 GPUs. Across all AL experiments, acquisition functions caleulated using 50 MC dropout predictions for each point. In the interest computational efficiency, only a specified num- ber of unlabeled points considered for acquisition in each iteration (as opposed considering all of pooot). In DEIMOS acquisition, the set of candidate points for acquisition in each AL iteration restricted the unlabeled points in X Xsang- X samp is randomly sampled from X X pool until the set of candidate points for acquisition Xeand Xsamp
Xurin reaches specified size.
Alternative splicing. Convolution performed on the sequence input by one-hot encoding the 101 101-nucleotide sequence as 2-dimensional array with four channels corresponding to the four possible nucleotides at each position. The dataset contains a total of 265,137 examples, of which 10% are used as the test data and the remaining 90% is subdivided into the training data and the pool (from which validation set consisting of 50 points is sampled). In the AL experiments (Fig left), all algorithms begin with training sets of size 75 and acquire 600 points from the pool through dingle-poin acquisitions. The number of unlabeled pool points considered for acquisition in each AL iteration is 5,000 for all algorithms tested.
14

---

Under review as a conference paper at ICLR 2021
UTKFaae face age prediction. Initial training sets consisted of 150 images, and 2,000 additional images were acquired over the course of the experiments. All 23,708 images in the dataset were used, with 10% held out test data and the remaining 90% consisting of the training data and the pool (from which a 50-image validation set is taken). During each acquisition iteration points are acquired from set of 5000 candidates randomly chosen from the pool set (for all algorithms tested).
MNIST handwritten digit classification. In the AL experiments, 10,000 images are used as test data while 60,000 images are used for the training data and the pool (from which a validation set of 10 images in binary classificationn300 images in 0-9 classification is sampled). In each AL iteration, points are queried from a randomly selected set of pool candidates; 2500 and 2000 candidates are considered per iteration in 7 vs. 9 and 0-9 classification, respectively. All algorithms begin with an initial training set of 10 images in binary classification experiments and 30 images in 0-9 clas- sification experiments. In experiments, initial training sets have equal representation from all classes.
E HYPERPARAMETER TUNING
Across all experiments, the dropout probability is calibrated such that the resulting model makes relatively high quality predictions and MC dropout uncertainty estimates the validation set. Qual- ity of MC dropout uncertainty estimates is assessed by Pearson R² between the observed squared error predicted variance the validation set.
L.2 regularization is applied in fully-connected layers with weight decay declining throughout the active learning experiments according À C/N, where C. is a weight decay hyperparameter tuned on the validation set and N is the size of the training set. C' is selected based on validation set accuracy resulting from model training on the initial training set for given number of epochs with the corresponding A. With this setup, the model precision in regression T is constant across active learning iterations (T, a constant in classification).
In regression experiments, the model precision hyperparameter r is set such that --1 is between 0.17 v and 0 where a² reg.o is the average dropout variance predicted by the initial model (excluding r 1) on the validation set.
class, In classification experiments, is set such that T is between 0.001 2001, and 0.010 2 where 8² d is the average dropout variance across classes predicted by the initial model on the validation set. The scaling factor multiplying odiass.v to determine -- should be much less than given that -- is merely smoothing parameter aimed at ensuring stable matrix inversion of Var, (Pacw (0)).
hyperparameter values used to generate results can be found in the corresponding code
"Code for all experiments is included the supplementary material.
15


Reviews:
[{"review_id": "VTPEarWmtRf", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper Summary\n\nThe paper considers the problem of active learning for training convolutional neural networks (CNN) in a sample-efficient manner. The proposed approach is built upon the existing idea of selecting points that maximally reduce expected mean squared error (MSE) on a large representative sample of points. MC-dropout is used for obtaining the estimates of model uncertainty. This idea is used for active learning in regression and classification problems with CNNs. A greedy method is proposed to select a batch of points by maximizing the acquisition function score sequentially obtained by updating the covariance matrix on previous points selected in the batch. Experiments are performed on two regression and one classification task. \n\n\nDetailed Comments\n\n- The paper tackles an important problem relevant for many practical applications. Although the proposed approach is based on an existing idea, the application to CNNs specifically seems novel. \n\n- It is a little worrying to see that random acquisition performs equally well with the proposed acquisition function (Figure 2 for instance). Please provide more discussion on this point and/or run one experiment for more than 3 (current) random runs to get a clear picture. Random acquisition is a simple to implement method and if the gains by the proposed acquisition function are not large enough, random acquisition will be a preferable option for a user. \n\n- There is limited description about how is the ''representative sample chosen'' in the experimental section. Please provide more details on this. \n\n- Greedy batch selection methods are known to select similar points losing the batch advantage. Please provide more details on the diversity of the points in batch selection.  \n\n- Why MC-dropout specifically is used for uncertainty estimation? The proposed approach should ideally be agnostic to the method for uncertainty estimation. Please provide more discussion on this choice. \n\n- In my opinion, although it is a minor point, it is better to show the visualization of EI acquisition function on a standard synthetic function like a sinusoidal function, Branin function etc. instead of a random neural network which might seem a little contrived.\n\n- The writing of the paper can be improved. For example, the usage of the term 'Expected improvement' is spread out across the paper without a proper technical description. Please try to provide a clear technical description of all the key concepts\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "QEOdILF63ia", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the active leaning problem by proposing a new acquisition function based on Expected improvement (EI). The paper shows that acquiring the point maximizing the expected reduction in predictive uncertainty across all points is equivalent to maximizing the expected improvement (EI).\n\n\nThe paper considers the setting for CNN classification and regression where the authors derive the variance estimations. The paper also extends to the batch mode to select multiple points simultaneously. The uncertainty is estimated using MC dropout.\n\nThe papers show that the improvement in the experiment is not significant comparing to the baselines.\n\n=============================\nMajor concern: although the paper claims to derive the EI-based acquisition function for active learning. The resulting derivation shows that the final acquisition function only depends on the uncertainty of the training data given the select point (x_new) while the relative contribution of the x_new toward the final performance E(y_train | x_new) has been vanished. Note that in the original form of the EI (Jones et al 1998), the acquisition function does not only depend on the uncertainty, but also the expected function value at the x_new.\n\nThe reviewer thinks that it may be better for this paper to position the contribution (abstract/introduction…) as the uncertainty based approach for active learning, then claim the minor/secondary contribution as showing the connection to the EI for the active learning setting. \n\nAt the current form, the reviewer thinks the paper is under the acceptance threshold.\n\n\nMinor concern:\n\nThe paper claims that they utilize full joint covariance rather than just point-wise variances used in previous work. However, the reviewer thinks this may be a wrong claim. In particular, the trace of variance considered in Eq 2 is equivalent to point-wise variance.\n\n\n\nOther comments:\n\nThe intuition in the resulting acquisition function is very much related to the Predictive Variance Reduction Search [1] – it is worth mentioning. Both approaches rely on:\n(1) uncertainty estimation of the some targets;\n(2) the objective function is defined using the sum of the uncertainty reduction – the trace of variance in Eq (2) and the original form in Eq (1) share this spirit. \n(3) similar in the batch setting where both approaches will sequentially select point to fill in a batch.\n\nThe difference is that [1] considers another problem in Bayes Opt while this paper considers the active learning task.\n\n\n\n[1]  Nguyen, V., Gupta, S., Rana, S., Thai, M., Li, C., & Venkatesh, S.  Efficient Bayesian Optimization for Uncertainty Reduction Over Perceived Optima Locations. In IEEE International Conference on Data Mining (ICDM) (pp. 1270-1275). 2019.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "KpLW8ytLb26", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for pool-based active learning in CNNs, selecting the next (batch of) data from an unlabeled pool to query their labels to expedite the learning process. The method computes the expected reduction in the predictive variance across a representative set of points and selects the next data point to be queried from the same set. In batch settings, the data points are sequentially selected in a batch (in a greedy way), with predictive variance representation updated after each selection. Experiments are performed on MNIST classification, and regression tasks of alternative splicing prediction, and face age prediction.\n\nPros:\n\nThe method is rather simple and (up to some extent) computationally efficient. The paper is generally well-written and puts the proposed method into context. The proposed method performs better than maximum variance acquisition and random acquisition in the regression experiments. For the classification tasks, its performance is comparable with maximum entropy, batch BALD and robust k-Center acquisition.\n\nConcerns:\n\nOverall, the novelty of the paper is limited. It employs dropout-based variational inference and applies the integrated variance minimization idea. The main contribution seems to be considering a joint normal distribution (whose level of validity is not completely clear/discussed). It is not clear if the proposed acquisition is the statistically optimal one that minimizes the expected MSE. It seems to me that requires taking into account the distribution of the (unobserved) label. Also, assuming all predicted class probabilities across all sample points as jointly Gaussian is strange as the probabilities are bounded and should sum to 1. In fact, the performance comparison in the classification setup is limited to one dataset and the results do not seem to confirm that the proposed method outperforms the baselines. \n\nFigure 2 seems to suggest that with increasing batch size, the performance gap shrinks. Have the authors tried a larger batch size or checked the sensitivity to it? \n\nAdditionally, how sensitive is the performance to the number of randomly sampled data points for the representative set?\n\nMy current rating is based on the aforementioned concerns.\n\nUpdate:\nI thank the authors for their response. I read the authors’ responses and the updated paper. As the authors have addressed some of my concerns and questions, I have adjusted my rating. However, I still have my concerns regarding the novelty and methodological contribution, and the classification setup. I also agree with other reviewers that the presentation can be improved.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach for active learning in CNNs. The method computes the expected reduction in the predictive variance across a representative set of points and selects the next data point to be queried from the same set. 

Pros:
- The method is rather simple and seems practical. 
- The paper is generally well-written.

Cons:
- The novelty of the paper is limited, as it essentially applies a known approach to CNNs.
- The performance gains presented in experiments seem rather mild, and may not justify using this method.
Decision: Reject