Title: Introducing Active Learning for CNN under the light of Variational Inference
Authors: Melanie Ducoffe
Frederic Precioso
Venue: ICLR 2017
Keywords: Deep learning
Supervised Learning
Optimization

Abstract: One main concern of the deep learning community is to increase the capacity of
representation of deep networks by increasing their depth. This requires to scale
up the size of the training database accordingly. Indeed a major intuition lies in
the fact that the depth of the network and the size of the training set are strongly
correlated. However recent works tend to show that deep learning may be handled
with smaller dataset as long as the training samples are carefully selected (let us
mention for instance curriculum learning). In this context we introduce a scalable
and efficient active learning method that can be applied to most neural networks,
especially Convolutional Neural Networks (CNN). To the best of our knowledge,
this paper is the first of its kind to design an active learning selection scheme based
on a variational inference for neural networks. We also deduced a formulation of
the posterior and prior distributions of the weights using statistical knowledge on
the Maximum Likelihood Estimator.
We describe our strategy to come up with our active learning criterion. We assess its
consistency by checking the accuracy obtained by successive active learning steps
on two benchmark datasets MNIST and USPS. We also demonstrate its scalability
towards increasing training set size.

TL;DR: Building automatically the labeled training set with active learning for CNN. The criterion is developed on a variational inference for NN and a kronecker approximation of Fisher matrices for CNN

Review discussion:
In the approximation of Q(\beta), which is crucial for the importance sampling of the data in the authors' framework of active learning, they assumed the current MLE estimator w_hat, is already a good approximation to the true parameter \theta_Y^*; how can this be justified? How is the sampling distribution initialized?
We thank warmly AnonReview4 for his feedbacks and interests!
The sampling distribution is initialized randomly, we also sampled the initial training set randomly for every runs.
Indeed the approximation of Q(\beta) is crucial for the importance sampling of the data.
Papers proposing active learning with second order information (1, 2, 3) assume that the current MLE estimator w hat, is already
a good approximation to the true parameter and we follow that guide of line.
references
(1) Zhang, T., & Oles, F. (2000). The value of unlabeled data for classifica-
tion problems. In Proceedings of the Seventeenth International Conference on
Machine Learning,(Langley, P., ed.) (pp. 1191-1198).
(2) Hoi, S. C., Jin, R., Zhu, J., & Lyu, M. R. (2006, June). Batch mode active
learning and its application to medical image classification. In Proceedings of
the 23rd international conference on Machine learning (pp. 417-424). ACM.
(3) Chaudhuri, K., Kakade, S. M., Netrapalli, P., & Sanghavi, S. (2015). Conver-
gence rates of active learning for maximum likelihood estimation. In Advances
in Neural Information Processing Systems (pp. 1090-1098).
In the Experiments of figure 2 what criterion did you use to decide to use 5 or 10 runs of experiments ? Does figure 2 correspond to only one active learning test or are the curves averages over several tests ?
We thank warmly AnonReview2 for his feedbacks and interests!
For a lack of time before the deadline, we run 5 experiments for each baseline
on MNIST while we run 10 experiments for USPS. We update the papers so the
baselines on MNIST are compared on a basis of 10 runs. Figures correspond
to the curves averages over several tests.
You mention existing work on the simple approach of uncertainty sampling. Did you try this as a baseline?
For a sake of concision, as the
initial submission is already two pages more that a standard submission should
be, we did not present further comparisons with simple approach. We updated
our paper with comparison with two naive baselines : uncertainty sampling and
curriculum sampling both based on the likelihood score of a sample (because
the true label of a sample is unknown, the likelihood is computed based on the
prediction of the current network). Uncertainty will select samples on which the
classifier is uncertain about, that means samples with the highest negative log
likelihood.
On the contrary, curriculum sampling selects the sample on which
the classifier is the more certain about. Figures 1 and 2 have been updated accordingly.
Section 5.2 studies the time complexity of your approach -- up to 30s to select the elements of one minibatch. How does this compare to the time required for using that minibatch to update the model by backprop?
The time required for using a minibatch to update the model by backprop is
much faster: the mean time for processing a minibatch (forward and backprop
pass) on USPS (8 samples) is 9.65475 ms with a Titan-X (GTX 980 M) with 8
GB RAM GPU memory. Because finding the optimal subset is a combinatorial
optimization problem and it is necessary to test every solution to check
whether one solution is optimal, the original problem is EXPTIME complete. We
propose a polynomial heuristic, which runs approximately in Nlog(N) given N
the number of unlabelled samples evaluated as potential queries.
In Figures 1 and 2, what do you mean by groundtruth?
We call groundtruth the average accuracy on 10 runs of the test error obtained
with the full annotated data shuffled randomly. Note that we do not optimize
the hyperparameters specifically for the size of the current annotated training
set.
Some of the approximations are quite complex; is your code available?
We thank warmly AnonReview3 for his feedbacks and interests !
Our code is now available on the github account mducoffe, it is developed with
Theano and Blocks. If the approximations mentionned are related to the KFAC
representation, the author’s version is supposed to be also published soon.
This paper introduces a mechanism for active learning with convolutional neural networks (CNNs). I would not go as far as the authors in calling these "deep", seeing that they seem to have only 2 hidden layers with only 20 filters each. The active learning criterion is a greedy selection scheme based on variational free energy and a series of approximations.
The paper is sometimes hard to read, due to (a) many grammatical errors and (b) sloppy notation in some places (e.g., on page 5, line 1, f is used but never introduced before).
Overall, I give an accepting score, but a weak one because of the grammatical errors. If the paper is accepted, these should be fixed for the final version, optimally by a native speaker.
The paper's topic is interesting, and the paper appears to succeed in its goal of showing a proof of concept for active learning in CNNs (if only on toy datasets). I'm surprised by the new results on uncertainty sampling and curriculum learning the authors added: why do these methods both break for USPS?
In particular, uncertainty sampling did very well (in fact, better than the authors' new method) on MNIST, but apparently horribly on USPS; some explanation for this would be useful.
I have one more question: why is it necessary to first sample a larger subset D \subset U, from which we select using active learning? Is this merely done for reasons of computational efficiency, or can it actually somehow improve results? (If so, it would be instrumental to see the worse results when this is not done.)
We are grateful to AnonReview3 for the interest and feedback. We have revised the paper to both improve the overall organization and stress the distinction between our contributions and previous works like the one from Graves which is now described in the related works. We have also revised the notations and mathematical formulations. The new version is now available on openreview.
We have validated our method on traditional datasets for the active learning community which are USPS and MNIST. Those are relatively small datasets compared to ImageNet, thereby not requiring networks as deep as for bigger datasets. The theory and the scalability of our method however holds for deeper networks.
The experiments made appear that curriculum learning is not a good active learning strategy for both tested datasets.
As for the uncertainty selection, it works really well on MNIST while it fails on USPS.
While MNIST is a pretty clean database, USPS contains more outliers and noisy samples rendering it more difficult in terms of accuracy even though both databases are designed to assess digit classification. As other works we mentioned in the related work section, we are led to explain uncertainty selection to select useless samples with the amount of outliers and noisy samples in USPS.
Another experiments demonstrating that sampling a larger subset D does not affect the accuracy is available in the apendix of the paper.
Quality:
The paper initiates a framework to incorporate active learning into the deep learning framework, mainly addressing challenges such as scalability that accompanies the training of a deep neural network.
However, I think the paper is not well polished; there are quite a lot of grammatical and typing errors.
Clarity:
The paper needs major improvements in terms of clarity. The motivations in the introduction, i.e., why it is difficult to do active learning in deep architectures, could be better explained, and tied to the explanation in Section 3 of the paper.
For example, the authors motivated the need of (mini)batch label queries, but never mention it again in Section 3, when they describe their main methodology.
The related work section, although appearing systematic and thorough, is a little detached from the main body of the paper (related work section should not be a survey of the literature, but help readers locate your work in the relevant literature, and highlight the pros and cons. In this perspective, maybe the authors could shorten some explanations over the related work that are not directly related, while spending more time on discussing/comparing with works that are most related to your current work, e.g., that of Graves '11.
Originality & Significance:
The authors proposed an active learning training framework. The idea is to treat the network parameter optimization problem as a Bayesian inference problem (which is proposed previously by Graves) and formulate the active learning problem as that of sampling the most informative data, where the informativeness is defined by the variational free energy, which depends on the Fisher information. To reconcile the computational burden of computing the inverse of Fisher Information matrix, the authors proposed techniques to approximate it (which seems to be novel)
I think that this paper initiates an interesting direction: one that adapts deep learning to label-expensive problems, via active learning.
But the paper needs to be improved in terms of presentation.
We are grateful to AnonReview4 for the interest and feedback. We have corrected the typos and grammatical errors and modified the structure of the text: we have trimmed the descriptions of unrelated active learning methods in the related work section and extended the discussion on Bayesian inference for active learning. We also included a detailed description of Graves' work on variational inference for neural networks. We have modified the paper to stress out the distinction between previous works (Graves) and our contributions, in particular to highlight our own proposition to get prior and posterior distributions of the weights from statistical assumption on the Maximum Likelihood Estimator. The new version is now available on openreview.
The paper proposes to perform active learning using pool selection of deep learning mini-batches using an approximation of the bayesian posterior. Several terms are in turn approximated.
The Maximum Likelihood Estimation (MLE) bayesian inference approach to active learning, the various approximations, and more generally the theoretical framework is very interesting but difficult to follow.
The paper is written in poor English and is sometimes a bit painful to read.
Alternative Active learning strategies and techniques do not need to be described with such detail.
On the other hand, the proposed approach has a lot of complex approximations which would benefit from a more detailed/structured presentation.
Another dataset would be a big plus (both datasets concern gray digits and USPS and are arguably somewhat similar).
We are grateful to AnonReview2 for the interest and feedback. We have paid a great attention to rewriting the paper, in particular straightening up the mathematical notations and restructuring the presentation of several sections. We specifically focused on introduction and related works so as to make clearly appear the novelty of our contribution. The new version is now available on openreview. As we focus on an active learning strategy, assessing the benefit of this very strategy on a dataset pre-requires to have the complete implementation of the training process on the full dataset for the right deep architecture. It will be very interesting to assess how the strategy performs on bigger datasets. Owing to the limited time frame, we could not get these results yet.

Meta-review:
role: PC
data: comment: The reviewers agree that the paper pursues an interesting direction to explore active example selection for CNN training, but have unanimously raised serious concerns with regards to overall presentation which needs further improvement (I still see spelling/grammatical errors/sloppy notation in the latest draft). Some sections in the paper are hard to follow. With regards to technical motivation, the link between depth and need for active example selection is alluded to, but not properly explained in the paper. The PCs think that this paper has too many areas in need of improvement to be accepted to the conference.

Decision: Reject