Deep Ensemble Bayesian Active Learning : Adressing the Mode Collapse issue in Monte Carlo dropout via Ensembles

1 INTRODUCTION . The success of deep learning in the last decade has been attributed to more computational power , better algorithms and larger datasets . In object classification tasks , CNNs widely outperform alternative methods in benchmark datasets ( LeCun et al. , 2015 ) and have been used in medical imaging for critical situations such as skin cancer detection ( Haenssle et al. , 2018 ) , retinal disease detection ( De Fauw et al. , 2018 ) or even brain tumour survival prediction ( Lao et al. , 2017 ) . Although their performance is unrivalled , their success strongly depends on huge amounts of annotated data ( Bengio et al. , 2007 ; Krizhevsky et al. , 2012 ) . In specialized domains such as medicine or chemistry , expert labelled data is costly and time consuming to acquire ( Hoi et al. , 2006 ; Smith et al. , 2018 ) . Active Learning ( AL ) provides a theoretically sound framework ( Cohn et al. , 1996 ) that reduces the amount of labelled data needed for a specific task . Developed as an iterative process , AL progressively adds unlabelled data points to the training set using an acquisition function , ranking them in order of importance to maximize performance . Using Active Learning within a Deep Learning framework ( DAL ) has recently seen successful applications in text classification ( Zhang et al. , 2017 ; Shen et al. , 2017 ) , visual question answering ( Lin & Parikh , 2017 ) and image classification with CNNs ( Gal et al. , 2017 ; Sener & Savarese , 2017 ; Beluch et al. , 2018 ) . One key difference between DAL and classical AL is the sampling in batches , which is needed to keep computational costs low . As such , developing scalable DAL methods for CNNs presents challenging problems . Firstly , acquisitions functions do not scale well for high dimensional data or parameter spaces , due to the cost of estimating uncertainty measures , which is the main approach . Secondly , even with scalability not being an issue , one needs to obtain good uncertainty estimates in order to avoid having overconfident predictions . One of the most promising techniques is Deep Bayesian Active Learning ( DBAL ) ( Gal , 2016 ; Gal & Ghahramani , 2016 ) , which uses Monte-Carlo dropout ( MC-dropout ) as a Bayesian framework to obtain uncertainty estimates . However , as mentioned in Ducoffe & Precioso ( 2018 ) , uncertainty-based methods can be fooled by adversarial examples , where small perturbations in inputs can result in overconfident and surprising outputs . Another approach presented by Beluch et al . ( 2018 ) uses ensemble models to obtain better uncertainty estimates than DBAL methods , although there are no result on how it deals with adversarial perturbations . Whereas uncertainty-based methods aim to pick data points the model is most uncertain about , density-based approaches try to identify the samples that are most representative of the entire unlabelled set , albeit at a computational cost ( Sener & Savarese , 2017 ) . Hybrid methods aim to trade uncertainty for representativeness . Our belief is that overconfident predictions for DBAL methods are an outcome of the mode collapse phenomenon in variational inference methods ( Srivastava et al. , 2017 ) , and that by combining the expressive power of ensemble methods with MC-dropout we can obtain ” better ” uncertainties without trading representativeness . In this paper we provide evidence for the mode collapse phenomenon in the form of a highly imbalanced training set acquired during AL with MC-dropout , and show that ’ preferential ’ behaviour is not beneficial for the AL process . Furthermore , we link the mode collapse phenomenon to overconfident classifications . We compare the use of ensemble models to MC-Dropout for uncertainty estimation and give intuitive reasons why combining the two might perform better . We present Deep Ensemble Bayesian Active Learning ( DEBAL ) which confirms our intuition for experiments on MNIST and CIFAR-10 . In Section 2 we give an overview of current popular methods for DAL . In Section 3 , various acquisition functions are introduced and the mode collapse issue is empirically identified . Further on , the use of model ensembles is motivated before presenting our method DEBAL . The last part of section 3 is devoted to understanding the cause of the observed improvements in performance . 2 BACKGROUND . The area of active learning has been studied extensively before ( see Settles ( 2012 ) for a comprehensive review ) , but with the emergence of deep learning , it has seen widespread interest . As proved by Dasgupta ( 2005 ) there is no good universal AL strategy , researchers instead relying on heuristics tailored for their particular tasks . Uncertainty-based Methods . We identify uncertainty-based methods as being the main ones used by the image classification community . Deep Bayesian Active Learning ( Gal et al. , 2017 ) models a Gaussian prior over the CNNs weights and uses variational inference techniques to obtain a posterior distribution over the network ’ s predictions , using these samples as a measure of uncertainty and as input to the acquisition function of the AL process . In practice , posterior samples are obtained using Monte-Carlo dropout ( MC-dropout ) ( Srivastava et al. , 2014 ) , a computationally inexpensive and powerful stochastic regularization technique that performs well on real-world datasets ( Leibig et al. , 2017 ; Kendall et al. , 2015 ) and has been shown to be equivalent to performing variational inference ( Gal & Ghahramani , 2016 ) . However , these approximating methods suffer from mode collapse , as evidenced in Blei et al . ( 2017 ) . Another method , Cost-Effective Active Learning ( CEAL ) ( Wang et al. , 2016 ) , uses the entropy of the network ’ s outputs to quantify uncertainty , with additional pseudo-labelling . This can be seen as the deterministic counterpart of DBAL , that adds highly confident samples directly from predictions , without the query process . Käding et al . ( 2016 ) propose a method on the expected model output change principle . This method approximates the expected reduction in the model ’ s error to avoid selecting redundant queries , albeit at a computational cost . Lastly , as this work was being developed , we found the work of Beluch et al . ( 2018 ) , who propose to use deterministic ensemble models to obtain uncertainty approximations . Their method scores high both in terms of performance and robustness . Density-based Methods & Hybrid Methods . Sener & Savarese ( 2017 ) looked at the data selection process from a set theory approach ( core set ) and showed their heuristic-free method outperforms existing uncertainty-based ones . Their acquisition function uses the geometry in the data-space to select the most informative samples . The main idea is to try to find a diverse subset of the entire input data space that best represents it . Although achieving promising results , the core set approach is computationally expensive as it requires solving a mixed integer programming optimisation problem . Ducoffe & Precioso ( 2018 ) , on the other hand , rely on adversarial perturbation to select unlabeled samples . Their approach can be seen as margin based active learning , whereby distances to decision boundaries are approximated by distances to adversarial examples . To the best of our knowledge , the only hybrid method ( combining measures of both uncertainty and representativeness ) tested within a CNN-based DAL framework is the one proposed in Wang & Ye ( 2015 ) . Although originally not tested on CNNs , this method was shown to perform worse than the core set approach in Sener & Savarese ( 2017 ) . Deep Bayesian Active Learning . Given the set of inputs X = { x1 , .. , xn } and outputs Y = { y1 , .. , yn } belonging to classes c , one can define a probabilistic neural network by defining a model f ( x ; θ ) with a prior p ( θ ) over the parameter space θ , usually Gaussian , and a likelihood p ( y = c|x , θ ) which is usually given by softmax ( f ( x ; θ ) ) . The goal is to obtain the posterior distribution over θ : p ( θ|X , Y ) = p ( Y|X , θ ) p ( θ ) p ( Y|X ) ( 1 ) One can make predictions y∗ about new data points x∗ by taking a weighted average of the forecasts obtained using all possible values of the parameters θ , weighted by the posterior probability of each parameter : p ( y∗|x∗ , X , Y ) = ∫ p ( y∗|x , θ ) p ( θ|X , Y ) dθ = Eθ∼p ( θ|X , Y ) [ f ( x ; θ ) ] ( 2 ) The real difficulty arises when trying to compute these expectations , as has been previously covered in the literature ( Neal , 2012 ; Hinton & Van Camp , 1993 ; Barber & Bishop , 1998 ; Lawrence , 2001 ) . One way to circumvent this issue is to use Monte Carlo ( MC ) techniques ( Hoffman et al. , 2013 ; Paisley et al. , 2012 ; Kingma & Welling , 2013 ) , which approximate the exact expectations using averages over finite independent samples from the posterior predictive distribution ( Robert & Casella , 2013 ) . The MC-Dropout technique ( Srivastava et al. , 2014 ) will replace p ( θ|X , Y ) with the dropout distribution q̂ ( θ ) . This method scales well to high dimensional data , it is highly flexible to accommodate complex models and it is extremely applicable to existing neural network architectures , as well as easy to use . In DBAL ( Gal , 2016 ) , the authors incorporate Bayesian uncertainty via MC-dropout and use acquisition functions that originate from information theory to try and capture two types of uncertainty : epistemic and aleatoric ( Smith & Gal , 2018 ; Depeweg et al. , 2017 ) . Epistemic uncertainty is a consequence of insufficient learning of model parameters due to lack of data , leading to broad posteriors . On the other hand , aleatoric uncertainty arises due to the genuine stochasticity in the data ( noise ) and always leads to predictions with high uncertainty . We briefly describe the three main types of acquisition functions : • MaxEntropy ( Shannon , 2001 ) . The higher the entropy of the predictive distribution , the more uncertain the model is : H [ y|x , θ ] = − ∑ c p ( y = c|x , θ ) logp ( y = c|x , θ ) ( 3 ) • Bayesian Active Learning by Disagreement ( BALD ) ( Houlsby et al. , 2011 ) . Based on the mutual information between the input data and posterior , and quantifies the information gain about the model parameters if the correct label would be provided . I ( y , θ|x , θ ) = H [ y|x ; X , Y ] − Eθ∼p ( θ|X , Y ) [ H [ y|x , θ ] ] ( 4 ) • Variation Ratio ( Freeman , 1965 ) . Measures the statistical dispersion of a categorical variable , with larger values indicating higher uncertainty : VarRatio ( x ) = 1−max y p ( y|x , θ ) ( 5 ) As seen in Gal ( 2016 ) , for the above deterministic acquisition functions we can write the stochastic versions using the Bayesian MC-Dropout framework , where the class conditional probability p ( y|x , θ ) can be approximated by the average over the MC-Dropout forward passes . The stochastic predictive entropy becomes : H [ y|x , θ ] = − ∑ c ( 1 K ∑ k p ( y = c|x , θk ) ) log ( 1 K ∑ k p ( y = c|x , θk ) ) ( 6 ) K corresponds to the total number of MC-Dropout forward passes at test time . Equivalent stochastic versions can be obtained for all other acquisition functions .

Summary: The authors propose to use the combination of model ensemble and MC dropout in Bayesian deep active learning. They empirically show that there exists the mode collapse problem due to the MC dropout which can be regarded as a variational approximation. The authors introduce an ensemble of MC-Dropout models with different initialization to remedy this mode collapse problem.