Despite the promise of Convolutional neural network (CNN) based classification models for histopathological images, it is infeasible to quantify its uncertainties. Moreover, CNNs may su ffer from overfitting when the data is biased. We show that Bayesian–CNN can overcome these limitations by regularizing automatically and by quantifying the uncertainty. We have developed a novel technique to utilize the uncertainties provided by the Bayesian–CNN that significantly improves the performance on a large fraction of the test data (about 6% improvement in accuracy on 77% of test data). Further, we provide a novel explanation for the uncertainty by projecting the data into a low dimensional space through a nonlinear dimensionality reduction technique. This dimensionality reduction enables interpretation of the test data through visualization and reveals the structure of the data in a low dimensional feature space. We show that the Bayesian-CNN can perform much better than the state-of-the-art transfer learning CNN (TL-CNN) by reducing the false negative and false positive by 11% and 7.7% respectively for the present data set. It achieves this performance with only 1.86 million parameters as compared to 134.33 million for TL-CNN. Besides, we modify the Bayesian–CNN by introducing a stochastic adaptive activation function. The modified Bayesian–CNN performs slightly better than Bayesian–CNN on all performance metrics and significantly reduces the number of false negatives and false positives (3% reduction for both). We also show that these results are statistically significant by performing McNemar’s statistical significance test. This work shows the advantages of Bayesian-CNN against the state-of-the-art, explains and utilizes the uncertainties for histopathological images. It should find applications in various medical image classifications. Keywords: Bayesian Convolutional Neural Networks, Breast Cancer, Histopathological Imaging, Machine Learning, Uncertainty Quantification, t-SNE. 1. Introduction 1.1. Histopathological imaging for Breast cancer Breast cancer is the most common cause of cancer in women [1–3]. It has the highest incidence (43.3 per 100 000 population) than any other cancer and the highest mortality rate (15%) of all cancer deaths in women in 2012 [1]. Thus detection and diagnosis of breast cancer are vital in reducing the impact of the dis- ease. Histopathological imaging is considered the gold 1email: thiagara@mtu.edu 2email: ppkhairn@mtu.edu 3email: susantag@mtu.edu This paper published in IEEE TMI. DOI: 10.1109 /TMI.2021.3123300standard for breast cancer detection and diagnosis [4]. Histopathology is a diagnostic technique that involves microscopic examinations of tissues to study the sign of a disease. This method preserves the underlying ar- chitecture of the tissues thereby providing a significant contribution to the diagnosis of diseases. It is the only way to detect some of the diseases like lymphocytic in- filtration of cancer. Histopathological images have a lot of information and structure which makes them highly reliable in the diagnosis of diseases especially in almost all types of cancer [5]. 1.2. state–of–the–art machine learning algorithms for Histopathological imaging With the advancement in digital imaging, computer- aided diagnosis (CAD) is studied extensively in the 2 literature [5–11]. Machine learning algorithms such as conventional neural networks and deep neural net- works have shown tremendous potential in CAD ap- plications. Feature extraction in a conventional neural network played a significant role in the predictions of the neural network. Various methods of feature extrac- tion followed by classifiers such as support vector ma- chine, K-nearest neighbor, decision tree, Naive Bayes amongst others were explored in the literature for classi- fication and segmentation of breast cancer images [12– 18]. Though these methods performed reasonably well in terms of accuracy, manual feature extraction was a bottleneck in improving the results. Convolutional neural networks (CNNs) were the most successful in learning advanced features automati- cally from the input images. CNNs perform very well in classification, both binary [19–21] and multiclass classi- fications [4, 22], and metastasis detection [23] on breast cancer images. Apart from being able to perform clas- sification, CNNs were also successfully implemented in problems involving segmentation [24–26] and detecting regions of interest that contain in-depth discriminatory information for classification in large whole-slide im- ages [27–29]. CNNs also provided the scope to imple- ment transfer learning. In transfer learning, the param- eters learned for one data set are utilized to accelerate learning for another data set to solve a similar prob- lem. Transfer learning CNN (TL-CNN) performs the best and is considered as the state-of-the-art classifier for breast cancer images [30–32]. A grand challenge on breast cancer histology images was conducted to ad- vance the state–of–the–art in classifying these images. It was concluded that CNN was the most successful method to classify these breast cancer images [33]. The top performers [34–36] in this challenge used the ar- chitecture of an existing network such as Resnet [37], Densenet [38], Inception [39], VGG16 [40], etc and pre- trained these networks using ImageNet [41]. Uncertainty quantification provides a measure of trust in machine decisions and this metric is crucially impor- tant in medical applications [42]. Given the success of automatic classifiers for medical images, their un- certainty quantification is essential. However, there are only a few studies that report uncertainty quantification [43]. Although CNN performs better for image classifi- cation than other machine learning techniques in terms of accuracy, their parameters are deterministic and thus can not provide any measure of uncertainty in its pre- dictions. In addition, predictions based on deterministic CNNs might provide incorrect results and without any estimate of confidence, these might lead to undesirable consequences. To evaluate the confidence of predic-tions, uncertainty quantification is important [42]. The Bayesian CNN is an e fficient state–of–the–art machine learning technique to quantify uncertainties [44–46]. It endows the uncertainty quantification capability in the CNN framework with nominal additional computational effort. In a Bayesian–CNN, the weights and biases are ran- dom variables as opposed to deterministic variables used in CNN. Through the stochastic nature of the pa- rameters, Bayesian–CNN captures the variability in the data set and computes the uncertainties in its predic- tions. Towards this, Blundell et al. [44] introduced an efficient algorithm called “Bayes by backprop”. This algorithm can be used to learn the probability distribu- tion on weights and it was compatible with the conven- tional backpropagation scheme. They have also shown that it is possible to regularise the weights by mini- mizing a loss function known as the variational free energy and demonstrated that this method of regular- isation showed performance comparable with dropout on MNIST classification. Shridhar et al. [45, 46] ex- tended the “Bayes by backprop” algorithm to convolu- tional neural networks. The proposed Bayesian-CNN architecture was implemented for image classification, image super-resolution, and generative adversarial net- works. This method has shown performance as good as conventional CNN and in addition, provided uncer- tainty measures and regularisation. Kendall et al. [47] provided a Bayesian deep learning framework combin- ing aleatoric and epistemic uncertainties. The authors improved the model’s performance by 1 to 3% over its deterministic counterpart by reducing the e ffect of noisy data. The trade–o ffs between modeling aleatoric and epistemic uncertainties were also studied. Kwon et al. [48] proposed a method of quantifying uncertainties in classification problems using Bayesian neural networks. Bayesian networks are used in the literature to perform active learning. Uncertainties were utilized to add new samples in the training data to improve performance by Gal et al [43]. An accurate, reliable active learn- ing framework was introduced by Raczkowski et al [49] utilizing the variational dropout-based uncertainty mea- sures. 1.3. Present work: Quantification, explanation, and use of uncertainty, and the Modified Bayesian– CNN Bayesian–CNNs are gaining popularity due to their ability to provide uncertainties associated with the pre- dictions. Yet, there has been no known work to the au- thors’ knowledge that demonstrates the advantages of Bayesian–CNN over CNN to classify histopathological 3 images. In this work, at first, we perform uncertainty quantification for the classification of breast histopatho- logical images and show the advantages of Bayesian– CNN over CNN. We found that the Bayesian–CNN im- proves accuracy and reduces overfitting in comparison to CNN in addition to quantifying uncertainties. To ex- ploit the advantages of uncertainty quantification, we have developed a novel technique to divide the data into low and high uncertainty subsets. We show significant improvement in accuracy in the majority of data that has low uncertainty. A small fraction of the data that has higher uncertainty can be referred to the experts. Thus, uncertainty provides an avenue to determine which im- ages require human intervention. We explain the reason for this high uncertainty in a subset of data by project- ing the data into a low dimensional space (latent space) using t–distributed stochastic neighbor embedding (t- SNE). We choose a three-dimensional latent space to facilitate the visualization of data. To the best of the au- thors’ knowledge, such an explanation of uncertainty is not reported in the literature. We propose a new model to extend the Bayesian– CNN to further improve its performance and name it as the modified Bayesian–CNN. Adaptive activation functions have shown improvement in performance for both deep neural networks and convolutional neural net- works [50–52]. However such adaptive activation func- tions are not applied to Bayesian Neural networks. The novelty of the proposed adaptivity lies in the fact that the learnable parameter is random as opposed to deter- ministic parameters reported in the literature. Thus the activation function is now sampled from an ensemble of such functions based on the probability distribution of parameters learned from the data. The rest of the paper is organized as follows: in sec- tion 2 the data set and the methods used in this work are described; results obtained in the work and their analy- sis are presented in section 3 followed by conclusions in section 4. 2. Data set and methods 2.1. Data acquisition The breast histopathological images used in this work are from a publicly available data set [53]. These are im- ages containing regions of Invasive Ductal Carcinoma which is the most common subtype of all breast cancers [53]. These regions are separated from the whole slide images by pathologists. The original data set consisted of 162 whole mount slide images of Breast Cancer spec- imens scanned at 40x. From the original whole slideimages, 277,524 patches of size 50 ×50 pixels were ex- tracted (198,738 negative and 78,786 positive) and pro- vided as a data set for classification. 2.2. Data Preparation The data set consists of two classes: positive (1) and negative (0). The classification of the data was carefully done by experienced pathologists providing the ground truth for training. 20% of the entire data set was used as the testing set for our study. The remaining 80% of the entire data was further split into a training set and a val- idation set (80%-20% split) to perform hyperparameter optimization. The image size is 3 ×50×50 (D×H×W), where D is the depth (color channels), H is the height, and W is the width. The images were shu ffled and converted from uint8 to float format for normalizing. For most of the images, the majority of the pixel values are greater than 200. This makes the computation extremely expensive especially with the large size of data. Further, it leads to problems such as singularity /gradient explosion during the evaluation of loss function. In addition, the original images are such that the information that needs to be highlighted has low pixel values. To solve these prob- lems, we computed the complement of all the images (training and testing) and then used them as inputs to the neural network. The pixel-wise normalization and complement was carried out as pn=(255−p)/255. Where pis the original pixel value and pnis the pixel value after normalization and complement. 2.3. Methods The mathematical background for Bayesian–CNN and its uncertainty quantification [45], and for t-SNE [54] is revisited in this section. The implementation of stochastic adaptive activation on Bayesian–CNN is also described. 2.3.1. Bayesian Neural Networks The neural network can be defined as a probabilis- tic model P(y|x,w), where x∈Rpis an input to the network, y∈Υis the output. Where Υis a set of all possible outputs. The network consists of a set of trainable random parameters w. This set of parameters wis learned using a complete Bayesian approach. In this approach given training data D, the posterior distri- bution of weights P(w|D) is calculated using Bayesian inference for neural networks which involves marginal- ization over all possible values of w. Once the poste- rior distribution P(w|D) is obtained, predictions on the unseen data are obtained by taking expectations on the 4 predictive distributions. The predictive distribution of an unknown label byof a test data item bxis given by P(by|bx)=EP(w|D)[P(by|bx,w)]=R ΩwP(by|bx,w)P(w|D)dw. To estimate the posterior distribution we use Bayes’ Rule which gives: P(w|D)=P(D|w)P(w) P(D)(1) The term P(D|w) is the likelihood of the training data (D) given a parameter setting ( w). Assuming each training data to be independent and identically dis- tributed, the above term becomes the product of like- lihood, P(D|w)=QN n=1P(yn|w,xn)where, ( xn,yn) are the training data item and known label respectively. The prior P(w) is our belief about the distribution of weights without seeing the data. The term P(D) in the equation (1) is intractable which makes the pos- terior distribution P(w|D) intractable. The term P(D) involves marginalization over the weight distribution: P(D)=R ΩwP(D|w)P(w)dw. For variational inference [44], the posterior distribu- tion P(w|D) which is intractable is approximated with a tractable simpler distribution over the model weights q(w). If we assume our weight distributions to be Gaus- sian, this simpler tractable distribution has variational parameters θ, whereθ=(µ,σ). We fit the varia- tional parameters θsuch that q(w|θ)≈P(w|D). The variational posterior q(w|θ) is used instead of the in- tractable posterior P(w|D) for the inference, P(by|bx)= Eq(w|θ)[P(by|bx,w)]. Estimating q(w|θ)≈P(w|D) we can say that we have learned the distribution of weights given training data. To make the variational posterior q(w|θ) and the true posterior P(w|D) similar, we minimize the KL diver- gence between them: KL[q(w|θ)∥P(w|D)]=Eq(w|θ)" logq(w|θ) P(w|D)# =Z q(w|θ)logq(w|θ) P(w)dw+Z q(w|θ)log( P(D))dw −Z q(w|θ)log( P(D|w))dw (2) The term log( P(D)) makes the above equation in- tractable. Although intractable, it is constant. There- fore, minimizing the KL divergence can be defined as: min(KL[q(w|θ)∥P(w|D)])=min( KL[q(w|θ)∥P(w)] −Eq(w|θ)[log( P(D|w))] (3) The term KL[q(w|θ)∥P(w)]−Eq(w|θ)[log( P(D|w))] is called the Variational Free Energy (VFE) which is to beminimized. Exact minimisation of the cost function is computationally expensive, therefore it is approximated through a Monte Carlo sampling procedure as follows: F(D,θ)≈nX i=1[logq w(i)|θ −logP w(i) −logP D|w(i) ] (4) Where, w(i)denotes the ithMonte Carlo sample drawn from the variational posterior q(w(i)|θ). Each term in F(D,θ), the loss function that needs to be minimized, also known as the cost function has weights that are drawn from the variational posterior. For further details on the cost function, readers are referred to [44, 45]. To implement and compute the above cost we need three terms: 1) log of variational posterior, 2) log of prior (Gaussian or scale mixture), 3) log-likelihood of the data. The variational posterior ( q) is composed of in- dependent Gaussian distribution for each parameter. The sample of weights are obtained by sampling the unit Gaussian, shifting and scaling by the mean µ and a standard deviation σrespectively. To ensure that the standard deviation is always non-negative, it is expressed as σ=softplus (ρ)=log(1 +exp(ρ)), point-wise. Parameters, w, of the variational posterior, q, can be re-written in terms of a parameter–free noise as [ w=µ+log(1 +exp(ρ))◦ε] or [ w=µ+σ◦ε]. Where,◦is point-wise multiplication, εis a sample drawn from unit normal [44]. The steps for optimiza- tion are presented in supplementary document. 2.3.2. Bayesian Convolution Neural Networks The parameters of CNNs are filters or kernels which are to be learned during training. In the case of a Bayesian–CNN these kernels are represented by prob- ability distributions as shown in Fig. 1. Figure 1: Schematics of the convolution of Bayesian–CNN showing probability distributions of random parameters. During the training of Bayesian–CNN the reparam- eterization trick is applied on these filters or kernels, which are of shape h ×w×d. They are sampled from the 5 variational posterior q(w|θ) using the following equa- tion: wh,w,d=µh,w,d+log1+expρh,w,d◦εh,w,d (5) where h is the height, w is the width and d is the depth of the filter and◦represents point-wise multiplication. After the sampling from the variational posterior, simi- lar optimization steps as explained in the previous sec- tion are followed. 2.3.3. Uncertainty Quantification The uncertainty quantification becomes extremely important when dealing with the applications related to autonomous vehicles, medical imaging, etc. Bayesian deep learning makes it possible to quantify the uncer- tainties in the prediction as we have probability distribu- tion over weights. Taking an expectation of the predic- tive posterior probability distribution: Eq(w|θ)[P(by|bx,w)] gives us the most probable prediction of the unknown databx. The variance of the predictive posterior proba- bility distribution: Var q(w|θ)[P(ˆy|ˆx,w)] quantifies the un- certainties. There are two types of uncertainties: the Aleatoric and the Epistemic uncertainty. The variance of the predictive posterior probability distribution is the sum of both these uncertainties: Var q(w|θ)[P(by|bx,w)]=aleatoric +epistemic (6) The aleatoric uncertainty corresponds to the noise in the data set whereas the epistemic uncertainty corresponds to the variability of the model developed[45]. One of the promising approaches to quantify the uncertainties is explained in [48]. The uncertainties are obtained from the variance of the predictive posterior probability dis- tribution, Var q(w|θ)[P(by|bx,w)]=Eq(w|θ)h (y−E[y])2i =Z Ωwhh diag EP(by|bx,w)[by] −EP(by|bx,w)[by]EP(by|bx,w)[by]Ti q(w|θ)dw+Z Ωwhh EP(by|bx,w)[by]−Eqθ(by|bx)[by]i h EP(by|bx,w)[by]−Eqθ(Y|x)[by]iTq(w|θ)dw (7) The above expression is the sum of aleatoric and the epistemic uncertainties which is derived from the vari- ant of the law of the total variance [48] The first term of Eq. 7 is defined as the aleatoric uncertainty and the second term is the epistemic uncertainty. Due to the integral term in Eq. 7, it is intractable and requires approximations as fol- lows. The equation of the aleatoric uncertainty isdefined as:1 NPN n=1diagbpn−bpnbpT nwhere, bpn= pbwn= softmaxn fbwt(bx)o . And the equa- tion of the epistemic uncertainty is defined as: 1 NPN n=1bpn−pbpn−pTwhere, bpn= pbwn= softmaxn fˆWt(bx)o andp=PN n=1bpn N. The overall equation for the variance is: Var q(w|θ)[P(by|bx,w)]=1 NNX n=1diagbpn−bpnbpT n +1 NNX n=1bpn−¯pbpn−¯pT(8) The variance of the predictive distribution can be cal- culated by Eq. 8 which provide us with the confidence of the network in making predictions for a given image. 2.3.4. Adaptive activation The weights and bias of a perceptron in a neural net- work perform a linear transformation of the inputs. The output of this linear transformation is passed to an acti- vation function that determines if a particular perceptron is activated for a given input. A non-linear activation function is a key component of a neural network, which enables it to learn complex functions with a small num- ber of perceptrons. However, this nonlinearity of the ac- tivation function is known to introduce problems such as exploding or vanishing gradients. Thus, there is a trade- offbetween the learning capabilities vs training com- plexities of a perceptron, which can be optimized. To achieve this, we modify the Bayesian-CNN by introduc- ing a learnable stochastic activation function that adapts to the training data. Such adaptivity in the context of other machine learning models has been reported in the literature [50, 52], in which the learnable parameter is deterministic. In this work, we introduce a probabilistic learning parameter for the adaptivity of Bayesian-CNN. This adaptive activation function contains a probabilis- tic parameter that is learned during the training of the neural network. To this end, a trainable probabilistic hyperparameter ( α) is introduced in the activation func- tions of a Bayesian–CNN and the resulting network is the modified Bayesian–CNN. The details of the pro- posed adaptive activation function are given below. σ(αfk(xk−1)) (9) where fk(xk−1)=wkxk−1+bk σis the activation function αis the probabilistic hyperparameter that can be trained wandbare the weights and bias of the kthlayer and 6 xk−1is the output from the previous layer of the neural network. To incorporate the stochastic adaptive activation, the loss functionF(Eq. 4) is modified to include the addi- tional stochastic parameter α. Specifically, the set of trainable network parameters w={w,b}inFis ex- tended to W={w,b,α}. Thus the modified loss function eF(D,θ) is given as eF(D,θ)≈nX i=1[logq(W(i)|θ)−logP(W(i))−logP(D|W(i))] The parameter αis learned in the same way as the parameters was described in Sec. 2.3.1. A prior distribution (a mixture of Gaussian) for the parameter α is assumed and the posterior distribution is obtained via Bayes’ rule. This posterior distribution is approximated by a Gaussian variational posterior qαwith mean µα and standard deviation σα. The parameters of this variational posterior are updated through gradient descent as, µm+1 α=µm α−η∇µαeFmand σm+1 α=σm α−η∇σαeFm Where,ηis the learning rate. Refer to section II of the supplementary document for further details. In this work, an adaptive ReLu given by σ(γ)=max(0,γx) is used for the fully connected layers. The networks were trained, validated, and tested on GPUs using Apache MXNet on Python 3. Details of the computational platform are given in the supplementary document. 2.3.5. t-distributed stochastic neighbor embedding (t- SNE) t-SNE [54] is an unsupervised non-linear dimension- ality reduction technique that helps in visualizing high dimensional data by mapping it into a low dimensional space. In this method, as a first step, a conditional prob- ability ( pj/i) is assigned between data points xiandxj in a high dimensional space. This conditional proba- bility pj/irepresents the similarity of data points such that similar points in the high dimensional space have a higher probability than dissimilar points. One such measure of similarity is the Euclidean distance between a pair of data points. In the second step, a similar con- ditional probability ( qj/i) is defined for the low dimen- sional counterpart yiandyj. If the low dimensional representation of the data correctly models similaritybetween the points in the high dimensional space, the two conditional probabilities pj/iandqj/iwill be equal. Thus, in the third step, by minimizing the KL diver- gence between the two conditional probabilities a low dimensional map of the data points is obtained via t- SNE. In this work, we explain the high uncertainty of images by projecting them into a latent space via t-SNE. 3. Results and analysis This section presents classification results for histopathological data, its uncertainty quantification ob- tained through the proposed modified Bayesian–CNN (Section 2.3). It also provides an explanation of the predicted uncertainty via t-SNE. Further, the proposed model is analyzed and juxtaposed with Bayesian–CNN and transfer learning CNN (TL-CNN). 3.1. Hyperparameter optimization Hyperparameters and architecture for all the net- works considered here are chosen through hyperparam- eter optimization. A Tree-structured Parzen Estimator (TPE) algorithm is used which is a sequential model- based optimization approach [55]. In this approach, models are constructed to approximate the performance of hyperparameters based on historical measurements. New hyperparameters are chosen based on this model to evaluate performance. A python library Hyperopt [56] is used to implement this optimization algorithm over a given search space. Data is split into a training set and a validation set as explained in the Data preparation sec- tion 2.2. An optimization is performed to maximize the validation accuracy for di fferent configuration and hy- perparameter settings of the network. The architecture of TL-CNN follows VGG-16 [40]. Architectures for Bayesian–CNN and modified Bayesian–CNN are shown in Fig. 2. In the case of mod- ified Bayesian–CNN, the ReLU function after the FC 1 layer (see Fig.2) is changed into a stochastic–adaptive ReLU. Using stochastic-adaptive Relu in convolution layers as well increased the computational cost with- out any significant improvement in performance. More details are given in the supplementary document. The number of filters for convolution layers and the number of neurons in the fully connected layers are obtained through hyper-parameter optimization. The results of the optimisation are presented in Tables. 1 and 2. 7 Figure 2: Architecture for Bayesian–CNN and modified Bayesian– CNN. Conv represents convolution layers and FC represent fully con- nected layers. Number of neurons in FC 2 is fixed as 2 for binary classification. Optimized kernel size of all convolution layers is fixed as 3×3 with stride 1. Optimized kernel size of max pool layers is 2 ×2 with stride 2. Table 1: Selected hyperparameters Network Learning rateBatch SizeLayers learnt TL-CNN 0.001 64 2 Bayesian–CNN 0.001 128 All Modified Bayesian- CNN0.0001 64 All Table 2: Selected hyper parameters for network architecture Network con1 con2 con3 con4 con5 con6 FC1 Bayesian– CNN16 32 32 64 128 256 512 modified Bayesian- CNN32 64 64 128 128 128 256 3.2. Training, validation, and testing of networks The networks are trained using the images as inputs and the corresponding known classes as outputs. The cross-entropy loss and the variational free energy loss are minimized for TL-CNN and Bayesian networks re- spectively. The training is done until the loss converges or the validation accuracy starts to decrease. In the fol- lowing the details of training, validation, and testing for TL-CNN, Bayesian–CNN, and the modified Bayesian– CNN are described. 3.2.1. Convolutional neural networks Transfer learning is the state-of-the-art for CNNs and has consistently performed the best for medical image classification [33, 57]. In transfer learning, the learned parameters for one data set are utilized for other data sets to perform a similar task. Here we use the weights of VGG16 [40] architecture trained on Imagenet data set [41] for classification and perform transfer learning to classify the histopathological data. The accuracy for training and validation for each epoch is shown in Fig. 3 for TL-CNN. The accuracy increases with the epoch for the training data whereas the accuracy remains constantfor the validation data after the first few epochs. This problem is known as overfitting. Thus, TL-CNN fails to generalize its predictions to unknown validation data and overfits the training data. Beyond 40 epochs, the validation accuracy remains almost a constant while the training accuracy is still increasing, which is long before the training loss has converged. 0 10 20 30 40 50 60 Epoch0.840.860.880.90.92Accuracy Training accuracy Validation accuracy Figure 3: Training and validation accuracy of TL-CNN. We can clearly see the problem of overfitting as the training accuracy in- creases while the validation accuracy does not improve. The results presented for the TL-CNN henceforth are for the maximum validation accuracy. To alleviate the overfitting of the networks, reg- ularization strategies are used, such as L1orL2 regularization, cross-validation, early stoppage, and dropout [58–60]. However, these may require human intervention during training (such as parameter tuning, determination of early stoppage) or may lead to loss of information due to drop out [59]. In Bayesian networks the prior distribution on weights p(w) introduces regularisation of weights automati- cally. A Gaussian prior yields L2regularisation on the weights and a Laplace prior yields L1regularisation on the weights. In addition, due to the stochastic nature of the parameters, an average across multiple models is computed during training which introduces a regular- isation e ffect on the network. However, similar to the selection of a regularisation parameter in deterministic networks, the selection of prior parameters in Bayesian networks is challenging. Herein we show that through a Bayesian framework the limitations of overfitting is overcome automatically. 3.2.2. Bayesian–CNN and the modified Bayesian–CNN The accuracy of predictions for both training and val- idation data of the Bayesian and modified Bayesian- CNN is shown in Fig. 4 and Fig. 5 respectively. It is evident that the problem of overfitting is eliminated by using a Bayesian network as the accuracy of predictions 8 for both training and validation data increases with an increase in the number of epochs. 0 50 100 150 200 250 300 Epoch0.80.820.840.860.880.9Accuracy Training accuracy Validation accuracy Figure 4: Training and validation accuracy of Bayesian–CNN. We can see that both training and validation accuracy increases with epoch which implies automatic regularisation. 0 100 200 300 400 500 Epoch0.80.820.840.860.880.9Accuracy Training accuracy Validation accuracy Figure 5: Training and validation accuracy of modified Bayesian– CNN. The values of the accuracy of training, validation, and test data upon the convergence of the loss func- tion are provided in Table 3. It is seen that the problem of overfitting drastically reduces the accuracy for val- idation and test data in a TL-CNN whereas this prob- lem is not present for Bayesian–CNN and the modified Bayesian–CNN. The training, validation, and test accu- racies of the modified Bayesian–CNN is slightly higher than the Bayesian–CNN when the loss function is con- verged. We found that the modified Bayesian–CNN re- quires about the same time as that of Bayesian–CNN for the same batch size but twice the computational cost per epoch during the training as compared to TL-CNN. More details on the training times are presented in the supplementary document. We have performed a com- parative study of the Bayesian–CNN and the modified Bayesian–CNN on the MNIST data [61]. We found that the modified Bayesian–CNN shows a similar improve- ment in performance on the accuracy for the MNIST data set. Further details are presented in the supplemen- tary document.Table 3: A Comparison of TL-CNN, Bayesian–CNN and modified Bayesian-CNN Network Training Accu- racyValidation Accu- racyTesting Accu- racy TL-CNN 0.9078 0.8691 0.8676 Bayesian–CNN 0.8848 0.8817 0.8801 modified Bayesian- CNN0.8898 0.8856 0.8837 3.3. Quality assessment metrics The performance of the classifier is presented using the confusion matrix in Fig. 6. Details of the confusion matrix is provided in the supplementary document. (a) (b) (c) Figure 6: Comparison of confusion matrices for a) TL-CNN, b) Bayesian–CNN and modified Bayesian–CNN In addition to the improvement of accuracy, the mod- ified Bayesian–CNN remarkably reduces the number of false-negative predictions as seen in Fig. 6, which is a significant achievement. CNN is known to per- form poorly for biased data set [62]. Given the data set used in this work is highly biased to the negative class, the TL-CNN model fails to overcome this bias yield- ing more negative predictions. We demonstrate that the modified Bayesian–CNN, overcomes this limitation of the CNN and provides an unbiased result. The modified Bayesian–CNN reduces the false negative and false pos- itive predictions by about 13.7% and 10.4% respectively as compared to TL-CNN. The modified Bayesian–CNN reduces both the false negative and false positive by 3% as compared to Bayesian–CNN as seen in Fig. 6, demonstrating unbiased prediction. This is a remark- able advancement given that medical imaging data sets are usually biased towards negative labels. This signif- icant improvement in performance is achieved despite having only 1.11 million parameters as compared to 134.33 million for TL-CNN. The improvement in the performance of modified Bayesian–CNN is due to the adaptive nature of the ac- tivation function. The adaptive activation function with a probabilistic learning parameter works better because 9 through it the network trains an ensemble of activation functions where each activation function has its weights drawn from a probability distribution which is learned from the data. Learning the probability distribution for the activation parameter from the training data reduces the bias in the model by eliminating the assumption in the functional form of the activation. To quantitatively assess the performance of the pro- posed binary classification the following metrics are used in addition to the confusion matrix: Accuracy, Precision, Recall, F1-score, Cohen’s-kappa (CK) coe ffi- cient [63]. Cohen’s Kappa coe fficient varies between 0- 1 where: 0 – agreement is equivalent to chance and 1.0 – perfect agreement. Improvement in Cohen’s Kappa coefficient is achieved by using the modified Bayesian– CNN as seen from table 4 which implies that it works better than both Bayesian–CNN and TL-CNN for an un- balanced data set. The performance of the neural network classifiers is evaluated based on all of the above metrics and the re- sults are provided in table 4. The modified Bayesian– CNN performs slightly better than Bayesian–CNN and much better than TL-CNN on all the performance met- rics for this data set. Table 4: Performance metrics of TL-CNN and BCNN Network Recall Precision F1- scoreCK TL-CNN 0.7552 0.7722 0.7636 0.6717 Bayesian–CNN 0.7823 0.7917 0.7870 0.7036 modified Bayesian–CNN0.7888 0.7981 0.7934 0.7125 3.3.1. Statistical significance In order to ensure that the di fference in the re- sults obtained by the Bayesian–CNN and the modified Bayesian–CNN is not by chance, we performed a sta- tistical significance test. Importance of statistical sig- nificance tests is highlighted in [64, 65]. The McNe- mar’s test [66] is performed and the results show that the di fference in performance is statistically significant (p<0.001). Therefore, the Bayesian–CNN and the adaptive Bayesian CNN are statistically di fferent mod- els. The details and the results of the statistical signifi- cance test are provided in Sec. VIII of the supplemen- tary document. 3.4. Uncertainty quantification Uncertainty quantification informs how confident the network is in its predictions, which is crucial for medi- cal applications. Thus in addition to the accuracy met-rics, uncertainty is an important measure that can be es- timated naturally through the Bayesian approaches. On the contrary, the parameters of CNN are deterministic and large in number (134 million) that makes its uncer- tainty quantification infeasible. The parameters of the (modified) Bayesian–CNN are probability distributions whose variance provides an estimate of the uncertain- ties associated with the predictions. The larger the vari- ances the more uncertain the model is on its predictions. The change of the probability distribution of a parame- ter for the modified Bayesian–CNN is shown for dif- ferent epochs in Fig. 7. As the training continues, the standard deviation decreases increasing the confidence in predictions (see the supplementary document). -0.15 -0.1 -0.05 0 0.05 0.1 Weight parameter020406080100 Epoch 0 Epoch 10 Epoch 20 Epoch 30 Epoch 40 Epoch 50 Figure 7: Probability density of a parameter for di fferent epochs of the modified Bayesian-CNN. (a) (b) (c) (d) (e) (f) Figure 8: Sample images having negative labels and their uncertain- ties: a) TN [ A=9.25×10−3,E=6.50×10−6], b) TN [ A=1.8×10−2, E=5.65×10−5], c) TN [ A=4.44×10−3,E=2.53×10−6], d) TN [ A=2.77×10−1,E=5.42×10−3], e) FP [ A=3.19×10−1, E=4.59×10−3], f) FP [ A=3.74×10−1,E=5.57×10−3]. Aleatoric and epistemic uncertainties are given by A and E respectively. 10 (a) (b) (c) (d) (e) (f) Figure 9: Sample images having positive labels and their uncertain- ties: a) TP [ A=7.0×10−2,E=2.60×10−4], b) TP [ A=6.60×10−2 ,E=1.74×10−4], c) TP [ A=1.14×10−1,E=4.91×10−4], d) TP [A=4.72×10−1,E=2.03×10−2], e) FN [ A=4.64×10−1, E=9.37×10−3] f) FN [ A=4.74×10−1,E=6.26×10−3]. The uncertainty values associated with an individual image denote the confidence with which the network predicts the class label for that image. The aleatoric uncertainty corresponds to the noise in the data which cannot be reduced by improving the model. The epis- temic uncertainty represents the variability due to the model, which can be improved. A set of sample im- ages from the test data set, their predicted class and their aleatoric and epistemic uncertainty in prediction are shown in Fig. 8 and 9. The images with negative and positive ground truth labels are shown in Fig. 8 and Fig. 9 respectively. The important features appar- ent from these images are the texture density and color. It appears that the positive class has darker and denser images. It seems that the images with high epistemic uncertainty, Fig. 8(d,e,f) and 9(d,e,f) has features from both the classes. Therefore, the model gives high epis- temic uncertainty to those images indicating mixed fea- tures or mislabeling. Thus, the Bayesian approach pro- vides an avenue to identify which images should be re- ferred back to a human expert. 3.5. Use of uncertainty to improve performance We utilize the uncertainty to further improve the per- formance on a subset of data. The aleatoric uncertainty of the test set is normalised by the maximum and mini- mum values of aleatoric uncertainty of the training set. For every threshold of aleatoric uncertainty, we divide the test data into two subsets having uncertainty lower and higher than the threshold. The ratio of the num- ber of images in the low uncertainty subset is plotted against the uncertainty–threshold used to create the sub- set in Fig. 10(a). The accuracy corresponding to each ofthis subset are also plotted against the threshold. The number of false-negative and false-positive predictions are plotted for di fferent thresholds of aleatoric uncer- tainty in Fig. 10(b). For instance, if the threshold value is 0.6, the low uncertainty subset contains 77% of the test data and its accuracy is 94 .6% which is about 6% improvement in accuracy over the entire test data set. The remaining 23% of the test data, which has uncer- tainty higher than this threshold (0.6) may be referred to a human expert for a more accurate prediction. Fur- thermore, for this subset, the percentage of false nega- tives reduces from 6 .2% for the entire data set to 2 .7% and the percentage of false positives reduces from 5 .8% for the entire data set to 1 .5%, which are significant im- provements. We found that normalized uncertainty val- ues lesser than 0.8 provided significant improvement in performance for this data set. For other data sets this threshold on uncertainty can be set by looking at change in slope of the accuracy vs uncertainty plot. A similar study is done using the epistemic uncertainty and the results are presented in the supplementary document. Therefore, we show that uncertainty quantification can be used to improve the performance significantly for a subset of test data. 0 0.2 0.4 0.6 0.8 1 Normalized aleatoric uncertainty0.880.90.920.940.960.981Accuracy 00.20.40.60.81 Ratio of dataUncertainty 0.8 Accuracy 0.9246Uncertainty 0.6 Data 0.7696 Uncertainty 0.8 Data 0.8626 Uncertainty 0.6 Accuracy 0.9462 (a) Fraction of test data having uncertainty below the threshold and their accuracy plotted against the uncertainty–threshold. 0 0.2 0.4 0.6 0.8 1 Normalised aleatoric uncertainty01000200030004000No of false predictionsFalse Negative False Positive Uncertainty 0.8 FN2128 Uncertainty 0.6 FN 1495 Uncertainty 0.6 FP 805Uncertainty 0.8 FP 1484 (b) Number of false predictions in the low uncertainty subset plotted against the uncertainty–threshold. Figure 10: Performance improvements for a subset of test data that has uncertainty less than the uncertainty–threshold. Dotted Lines rep- resents two thresholds of aleatoric uncertainty. 11 3.6. Explanation of uncertainty by visualization in low dimension In the following, we provide an explanation of uncer- tainty by visualizing the test data in a low dimensional space (latent space). A dimensionality reduction is per- formed on the data using t-SNE to project the 7500- dimensional feature into 3 dimensions (3D). (a) Low uncertainty (b) Medium uncertainty (c) High uncertainty Figure 11: Low dimensional representation of the test data for three subsets using t-SNE. blue: negative class, red: positive classThe epistemic uncertainty ( E) evaluated through the Bayesian approach is normalized to range between [0 − 1]. The test data set is divided into three subsets: low (E≤0.01), medium (0 .01<E≤0.1) and high un- certainty ( E>0.1). t-SNE is performed for these three subsets. Low dimensional representations obtained via t-SNE are presented in Fig. 11 where each data point represents a test image. Fig. 11 explains the uncer- tainty associated with the individual images. The low uncertainty images have a clear separation between the positive and negative classes in the latent space. Thus the model classifies these images with high confidence. The images with medium uncertainty are clustered at different locations in the latent space however have sig- nificant overlap. Therefore the model classifies these images with lesser confidence. For the high uncertain images, the negative and the positive classes are not at all separable in the latent space. Thus the model can- not distinguish between the two classes for this subset, yielding poor accuracy and low confidence. 4. Conclusions In this study, we have quantified the uncertainty in the classification of histopathological images through Bayesian–CNN and compared its performance against the state-of-the-art. We have explained the uncertainty and utilized the uncertainty to improve the performance. To conclude this study, we summarize its main findings. Firstly, we have shown that Bayesian–CNN performs better than the state-of-the-art transfer learning CNN on all performance metrics and provides uncertainties in classifying breast histopathological images. Bayesian– CNN improves the test accuracy by 1.2% and reduces the false negative and false positive by 11% and 7.7% respectively as compare to TL-CNN, despite having only 1.86 million parameters as compared to 134.33 million for TL-CNN. Secondly, we explained the uncertainties in the test data by projecting them into a low-dimensional space. This low-dimensional projection enables visual inter- pretation of the data. The data structure in the low di- mensional feature space reveals that images with high uncertainty are not separable whereas the images with low uncertainty are clearly separable. Thirdly, using di fferent uncertainty thresholds, we demonstrate that the accuracy of Bayesian networks can be significantly improved on a large fraction of test data (6% improvement on 77% test data). Fourthly, we have proposed a novel stochastic– adaptive activation enabled Bayesian–CNN and call it 12 modified Bayesian–CNN. In the proposed model we in- troduce a probabilistic learnable activation function that adapts to the training data to improve predictive capabil- ities. We have shown that the modified Bayesian-CNN performs slightly better than the Bayesian–CNN in all performance metrics. Especially it can reduce the false negative and false positive predictions by 3% as com- pared to Bayesian–CNN. Fifthly, we have demonstrated that using the pro- posed modified Bayesian–CNN and the Bayesian– CNN, the problem of overfitting can be nearly elimi- nated without any recourse to regularization. That is the Bayesian approach work with almost equal accuracy for unknown and known data sets. This work improves the understanding of the un- certainties in Bayesian–CNN-based classification and helps to leverage these uncertainties to further improve its performance. These findings require further inves- tigation on larger data sets, however, they show poten- tial routes to improve upon the state–of–the–art classi- fier for a broad range of biomedical applications.