Paper:
In image classification tasks, the ability of deep convolutional neural networks (CNNs) to deal with complex image data has proved to be unrivalled. Deep CNNs, however, require large amounts of labeled training data to reach their full potential. In specialized domains such as healthcare, labeled data can be difficult and expensive to obtain. One way to alleviate this problem is to rely on active learning, a learning technique that aims to reduce the amount of labelled data needed for a specific task while still delivering satisfactory performance. We propose a new active learning strategy designed for deep neural networks. This method improves upon the current state-of-the-art deep Bayesian active learning method, which suffers from the mode collapse problem. We correct for this deficiency by making use of the expressive power and statistical properties of model ensembles. Our proposed method manages to capture superior data uncertainty, which translates into improved classification performance. We demonstrate empirically that our ensemble method yields faster convergence of CNNs trained on the MNIST and CIFAR-10 datasets.

Review:
The paper shows that Bayesian neural networks, trained with Dropout MC (Gal et al.) struggle to fully capture the posterior distribution of the weights.
This leads to over-confident predictions which is problematic particularly in an active learning scenario.
To prevent this behavior, the paper proposes to combine multiple Bayesian neural networks, independently trained with Dropout MC, to an ensemble.
The proposed method achieves better uncertainty estimates than a single Bayesian neural networks model and improves upon the baseline in an active learning setting for image classification.
The paper addresses active deep learning which is certainly an interesting research direction since in practice, labeled data is notoriously scarce. 
However, the paper contains only little novelty and does not provide sufficiently new scientific insights.
It is well known from the literature that combining multiply neural networks to an ensemble leads to better performance and uncertainty estimates.
For instance, Lakshminarayanan et al.[1] showed that Dropout MC can produce overconfident wrong prediction and, by simply averaging prediction over multiple models, one achieves better performance and confidence scores. Also, Huand et al. [2] showed that by taking different snapshots of the same network at different timesteps performance improves.
It would also be great if the paper could related to other existing work that uses Bayesian neural networks in an active learning setting such as Bayesian optimization [3, 4] or Bandits[5].
Another weakness of the paper is that the empirical evaluation is not sufficiently rigorous: 
1) Besides an comparison to the work by Lakshminarayanan et. al, I would also like to have seen a comparison to other existing Bayesian neural network approaches such as stochastic gradient Markov-Chain Monte-Carlo methods.
 2) To provide a better understanding of the paper, it would also be interesting to see how sensitive it is with respect to the ensemble size M. 
 
 3) Furthermore, for the experiments only one neural network architecture was considered and it remains an open question, how the presented results translate to other architectures. The same holds for the type of data, since the paper only shows results for image classification benchmarks.
 
 4) Figure 3: Are the results averaged over multiple independent runs? If so, how many runs did you perform and could you also report confidence intervals? Since all methods are close to each other, it is hard to estimate how significant the difference is.
 
[1] Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundel
NIPS 2017
[2] Gao Huang and Yixuan Li and Geoff Pleiss and Zhuang Liu and John E. Hopcroft and Kilian Q. Weinberger
    Snapshot Ensembles: Train 1, get {M} for free}
    ICLR 2017
[3] Bayesian Optimization with Robust Bayesian Neural Networks
    J. Springenberg and A. Klein and S.Falkner and F. Hutter
    NIPS 2016
 
[4] J. Snoek and O. Rippel and K. Swersky and R. Kiros and N. Satish and N. Sundaram and M. Patwary and Prabhat and R. Adams
    Scalable Bayesian Optimization Using Deep Neural Networks
    ICML 2015
[5] Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling
    Carlos Riquelme, George Tucker, Jasper Snoek
    ICLR 2018