EVIDENCE-AWARE ENTROPY DECOMPOSITION FOR ACTIVE DEEP LEARNING
Year: 2019

Abstract:
We present a novel multi-source uncertainty prediction approach that enables deep learning (DL) models to be actively trained with much less labeled data. By leveraging the second-order uncertainty representation provided by subjective logic (SL), we conduct evidence-based theoretical analysis and formally decompose the predicted entropy over multiple classes into two distinct sources of uncertainty: vacuity and dissonance, caused by lack of evidence and conflict of strong evidence, respectively. The evidence based entropy decomposition provides deeper insights on the nature of uncertainty, which can help effectively explore a large and high-dimensional unlabeled data space. We develop a novel loss function that augments DL based evidence prediction with uncertainty anchor sample identification through kernel density estimation (KDE). The accurately estimated multiple sources of uncertainty are systematically integrated and dynamically balanced using a data sampling function for label-efficient active deep learning (ADL). Experiments conducted over both synthetic and real data and comparison with competitive AL methods demonstrate the effectiveness of the proposed ADL model.

heading: 1 INTRODUCTION
text: Deep learning (DL) models establish dominating status among other types of supervised learning models by achieving the state-of-the-art performance in various application domains. However, such an advantage only emerges when a huge amount of labeled training data is available. This limitation slows down the pace of DL, especially when being applied to knowledge-rich domains, such as medicine, biology, and military operations, where large-scale labeled samples are too expensive to obtain from well-trained experts. Meanwhile, active learning (AL) has demonstrated great success by showing that for many supervised models, training samples are not equally important in terms of improving the model performance (Settles, 2009). As a result, a carefully selected smaller training set can achieve a model equally well or even better than a randomly selected large training set.
An interesting question arises, which is whether DL models can be actively trained using much less labeled data. Recent efforts show promising results in this direction through Bayesian modeling (Gal et al., 2017) and batch model sampling (Sener & Savarese, 2018). However, as DL models are commonly applied to high dimensional data such as images and videos, a fundamental challenge still remains, which is how to most effectively explore the exponentially growing sample space to select the most useful data samples for active model training. Existing AL models usually leverage the model provided information, such as estimated decision boundaries or predicted entropy for data sampling. However, the deep structure and the large number of parameters of DL models make model overfitting almost inevitable especially in the early stage of AL when only very limited training data is available. As a result, the model may provide misleading information that makes data sampling from a high-dimensional search space even more difficult. Besides a high dimensionality, complex data may contain a large number of classes and data samples from certain classes may be completely missing. Such situations are quite common for domains, such as scientific discovery (e.g., gene function prediction) and anomaly detection. AL models should be able to effectively discover these out of distribution (OOD) samples for labeling in order to achieve an overall good prediction performance.
Uncertainty sampling has been one of the most commonly used pool-based AL models. In particular, a model chooses the data sample that it is least certain about. Thus, once the sample is labeled, model uncertainty can be significantly reduced. As an information-theoretic measure, entropy provides a
general criterion for uncertainty sampling. Some commonly used sampling methods, including least confident and margin based strategies, are equivalent to entropy-based sampling in binary classification (Settles, 2009). It is also natural and straightforward to generalize to multi-class problems.
A key challenge of entropy-based sampling for AL is that the predicted entropy may be highly inaccurate, especially in the early state of the AL. Such an issue may become more severe when training a neural network (NN)/DL active learner due to model overfitting as described above. Figure 1(a) shows the predicted entropy by an NN active learner trained using nine labeled data samples, which are in black color and evenly distributed in three classes. The standard softmax layer is used in the output layer to generate class probabilities over three classes, each of which is a mixture of two Gaussian’s. It turns out that all the data samples in the three small clusters located in the top left, top right, and bottom center, are wrongly predicted with high confidence, as indicated by the low entropy. As a result, data samples from these three clusters are less likely to be selected for labeling. In contrast, the data samples that are close to the center of the three major clusters are more likely to be selected. However, labeling these samples will have the effect of fine-tuning a wrongly predicted decision boundary, leading to a much higher (but less effective) labeling cost.
Figure 1(b) shows the the result from the proposed active deep learning (ADL) model. While the samples from the small clusters are still wrongly predicted due to lack of training data, they are predicted with a much lower low confidence as indicated by the high entropy. However, even with a more accurately predicted entropy, the active learner may still sample from the center of the three major clusters as it is still assigned a high entropy along with the areas that cover the three smaller clusters. By performing a fine-grained analysis of uncertainty under the subjective logic (SL) framework (Jøsang, 2016), we formally decompose entropy into two distinct sources of uncertainty: vacuity and dissonance, which are caused by lack of evidence and conflict of strong evidence, respectively. By putting the vacuity and dissonance as shown in Figures 1(c) and (d) together, it is interesting to see that we recover the entropy as shown in Figure 1(c), which empirically verifies our theoretical results. Entropy decomposition provides further insights on the sources on uncertainty, which is instrumental to guide the data sampling process. Intuitively, given the dataset in Figure 1, an effective sampling strategy will first choose samples from the three small clusters according to vacuity in the early stage of AL to properly establish the shape of the decision boundary. It can then fine-tune the decision boundary by sampling according to dissonance. Such an uncertainty-aware sampling strategy will be critical for a high-dimensional space with multiple competing classes where data samples are scarcely distributed and the decision boundary becomes more complicated.
Our major contribution is threefold: (1) decomposition of entropy through evidence-based theoretical analysis of belief vacuity and belief dissonance under the SL framework; (2) a multi-source uncertainty prediction model that accurately quantifies different sources of uncertainty through kernel density regularized evidence prediction; (3) an active deep learning model that systematically integrates different types of uncertainty for effective data sampling in a high-dimensional space. Extensive experiments are conducted over both synthetic and real-world data to demonstrate the effectiveness of the proposed ADL model.

heading: 2 RELATED WORK
text: Uncertainty Quantification in Belief/Evidence Theory: In the belief/evidence theory domain, uncertainty reasoning has been substantially explored such as Fuzzy Logic (De Silva, 2018), DempsterShafer Theory (DST) (Sentz et al., 2002), or Subjective Logic (SL) (Jøsang, 2016). Unlike the efforts made in ML/DL above, belief theorists focused on reasoning of inherent uncertainty in information
resulting from unreliable, incomplete, deceptive, and/or conflicting evidence. SL considered uncertainty in subjective opinions in terms of vacuity (i.e., lack of evidence) and vagueness (i.e., failure of discriminating a belief state) (Jøsang, 2016). Vacuity has been used as an effective vehicle to detect out-of-distribution queries through evidence learning, achieved under the typical DL setting with ample training samples (Sensoy et al., 2018). Recently, other dimensions of uncertainty have been studied, such as dissonance (due to conflicting evidence) and consonance (due to evidence about composite subsets of state values) (Jøsang et al., 2018).
Epistemic Uncertainty in Deep Learning: In DL, aleatoric uncertainty (AU) and epistemic uncertainty (EU) have been studies using Bayesian Neural Networks (BNNs) for computer vision. AU consists of homoscedastic uncertainty (i.e., constant errors for different inputs) and heteroscedastic uncertainty (i.e., different errors for different inputs) (Gal, 2016). A Bayesian DL (BDL) framework was presented to estimate both AU and DU simultaneously in regression (e.g., depth regression) and classification settings (e.g., semantic segmentation) (Kendall & Gal, 2017). A new type of uncertainty, called distributional uncertainty, is defined based on distributional mismatch between the test and training data distributions (Malinin & Gales, 2018).
Active Learning in Deep Learning: The common AL methods other than DL-based ones are surveyed in (Settles, 2009). There are limited efforts on actively training DL models for highdimensional data with a few exceptions. In (Wang & Shang, 2014), an AL model was developed for DL using three metrics for data sampling: least confidence, margin sampling, and entropy. A new approach combines recent advances in BDL into the AL framework to achieve label-efficient DL training (Gal et al., 2017). Another approach advances the AL development by introducing a cost-effective strategy to automatically select and annotate the high-confidence samples, which improves the traditional samples selection strategies (Wang et al., 2016). Data sampling in DL has also been approached as a core-set selection problem (Sener & Savarese, 2018), which requires a large batch to work well. Different from all existing works, the proposed ADL model decomposes the accurately estimated uncertainty into vacuity and dissonance and dynamically balances multisource uncertainty to achieve active training of DL models with much less labeled data.

heading: 3 EVIDENCE-AWARE ENTROPY DECOMPOSITION
text: As discussed earlier, a high entropy may be contributed by difference sources of uncertainty with distinct characteristics. In this section, we conduct a fine-grained theoretical analysis of different types of uncertainty that arise in the context of multi-class problems. The decomposition is conducted under the SL framework, which provides key building blocks for our theoretical analysis.

heading: 3.1 THEORY OF SUBJECTIVE LOGIC
text: SL is an uncertain probabilistic logic that is built upon probabilistic logic (PL) (Nilsson, 1986) and belief theory (BT) (Shafer, 1976) while making two unique extensions. First, SL explicitly represents uncertainty by introducing vacuity of evidence (or uncertainty mass) in its opinion representation, which addresses the limitation of using PL to model lack of confidence in probabilities. Second, SL extends the traditional belief function of the BT by incorporating base rates, which serve as the prior probability in Bayesian theory. The Bayesian nature of SL allows it to use second-order uncertainty to express and reason the uncertainty mass, where second-order uncertainty is represented in terms of a probability density function (PDF) over first-order probabilities (Jøsang, 2016). In particular, for multi-class problems, we use a multinomial distribution (first-order uncertainty) to model class probabilities and use a Dirichlet PDF (second-order uncertainty) to model the distribution of class probabilities. Second-order uncertainty enriches uncertainty representation with evidence information, which plays a central role in entropy decomposition as detailed later.
Subjective opinions (or opinions) are the arguments in SL. In the multi-class setting, the subjective opinion of a multinomial random variable y in domain Y = {1, ...,K} is given by a triplet
ω = (b, u,a), with K∑
k=1
bk + u = 1 (1)
where b = (b1, ..., bK)T , u, and a = (a1, ..., aK)T denote the belief mass distribution over Y, uncertainty mass representing vacuity of evidence, and base rate distribution over Y, respectively, and ∀k, ak ≥ 0, bk ≥ 0, u ≥ 0. The probability that y is assigned to the k-class is given by
P (y = k) = bk + aku, ∀k ∈ Y (2)
which combines the belief mass with the uncertain mass using the base rates. In the multi-class setting, ak can be regarded as the prior preference over the k-th class. When no specific preference is given, we assign all the base rates as 1/K.
In existing SL literature, there lacks a clear transition between the first order uncertainty given in equation 2 and the second-order uncertainty expressed as a Dirichlet PDF. Here, we make this transition more explicit by introducing a set of random variables p = (p1, ..., pK)T , where p is distributed on a simplex of dimensionality K − 1. We introduce a conditional distribution P (y = k|pk) = pk, which allows us to represent the marginal distribution in equation 2 by P (y) = ∫ P (y|p)p(p)dp. We define p(p) as a Dirichlet PDF over p: Dir(p|α), where α = (α1, ..., αK)T is K-dimensional strength vector, with αk ≥ 0 denoting the effective number of observations of the k-th class. SL explicitly introduces the uncertainty evidence through a non-informative weight W and redefine the strength parameter as
αk = rk + akW, with rk ≥ 0,∀k ∈ Y (3) where rk is the amount of evidence (or the number of observations) to support the k-th class and W is usually set to K, i.e., the number of classes. Given the new definition of the strength parameter, the expectation of the class probabilities p = (p1, ..., pK)T is given by
E[pk] = αk∑K j=1 αj = rk + akW∑K j=1 rj +W
(4)
where ak = 1/K. By marginalizing out p, we can derive an evidence-based expression of belief mass and uncertainty mass:
bk = rk S ∀k ∈ Y, u = W S , with S = K∑ k=1 αk (5)
SL categorizes uncertainty into two primary sources (Jøsang, 2016): (1) basic belief uncertainty that results from specific aspects of belief mass in isolation and (2) intra-belief uncertainty that results from the relationships between belief masses and uncertainty mass. Since we focus on the multiclass setting, no composite values (i.e., simultaneously assigned to multiple classes) are allowed. As a result, these two sources of uncertainty boil down to vacuity and dissonance, respectively, that correspond to vacuous belief and contradicting beliefs. In particular, vacuity of an opinion vac(ω) is captured by uncertainty mass u, which is defined in equation 5 and dissonance of an opinion (Jøsang et al., 2018) is defined as
diss(ω) = K∑ k=1 ( bk ∑ j 6=k bjBal(bj , bk)∑ j 6=k bj ),Bal(bj , bk) = { 1− |bj−bk|bj+bk if bibj 6= 0 0 if min(bi, bj) = 0 (6)
where Bal(bj , bk) is the relative mass balance function between two belief masses. The belief dissonance of an opinion is measured based on how much belief supports individual classes. Consider a binary classification example with a binomial opinion given by (b1, b2, u,a) = (0.49, 0.49, 0.02,a). Based on equation 6, it has a dissonance value of 0.98. In this case, although the vacuity is close to zero, a high dissonance indicates that one cannot make a clear decision because both two classes have the same amount of supporting evidence and hence strongly conflict with each other.

heading: 3.2 EVIDENCE-BASED ENTROPY DECOMPOSITION
text: By leveraging the second-order uncertainty representation, we formally show that the entropy of a predicted class distribution P (y) can be decomposed into vacuity and dissonance. Our major theoretical results indicate that the uncertainty of a high-entropy data sample may be caused by either lack of evidence (i.e., high vacuity) or conflict of strong evidence (i.e., high dissonance) but not both. By clearly identifying the sources of uncertainty instead of using them in a combined form as in entropy, the evidence based decomposition of entropy provides deeper insights on the nature of uncertainty, which provides important guidance for an AL model to more effectively explore a large and high-dimensional search space for efficient data sampling. Lemma 1. Dissonance maximization. Given a total Dirichlet strength S = CK, where C ≥ 1 and K is the number of classes, for any opinion ω on a multinomial random variable y, we have
max diss(ω) = 1− 1 C
(7)
Corollary 1. The dissonance diss(ω) is approaching (but not reaching) 1 when all the evidence rk’s are set to equal and S →∞; it reaches 0 when S = K:{
limS→∞ diss(ω) = 1 if r1 = ...rk... = rK diss(ω) = 0 if S = K
(8)
Lemma 2. Vacuity maximization. For any opinion ω on a multinomial random variable y, we have 0 ≤ vac(ω) ≤ 1 and the maximum vacuity is achieved when ∑K k=1 rk = 0. Theorem 1. Let y denote a multinomial random variable, ωy denote its opinion, S denote its total Dirichlet strength, and H[y] be the entropy of y. H[y] can be maximized under two different and non-overlapping conditions: (1) for S = K and assuming non-informative base rates, y∗ = arg maxH[y] ⇔ y∗ = arg max vac(ωy); (2) for S → ∞, y∗ = arg maxH[y] ⇔ y∗ = arg max diss(ωy).
A more intuitive interpretation of the main results in Theorem 1 is as following. A high-entropy data sample supported by a strong evidence (i.e., S K) is caused by a high dissonance (i.e., conflict of evidence); a high-entropy data sample supported by little evidence (i.e., S ≈ K) is caused by a high vacuity (i.e., lack of evidence). Through the second-order uncertainty representation, we offer an evidence based interpretation of entropy that allows us to identify two different sources of uncertainty that both cause a high entropy. The multi-source uncertainty will provide important information to design a fine-grained sampling function for AL, which will be detailed in next section.

heading: 4 MULTI-SOURCE UNCERTAINTY AWARE ACTIVE DEEP LEARNING
text: In order to best use the uncertainty information, the ADL model should first be able to provide an accurate uncertainty estimation based on very limited training data. This, coupled with the large number of parameters of the DL model, poses a fundamental challenge due to a higher risk of model overfitting. As shown earlier, inaccurate uncertainty estimation will cause the model to miss labeling important data samples that can help accurately detect the decision boundary if labeled.
In addition, since both vacuity and dissonance are derived from second-order uncertainty, solely predicting the class label or its distribution does not provide sufficient information for multi-source uncertainty prediction. Instead of predicting the class label distribution, the proposed ADL model directly estimates the supporting evidence (i.e., rk’s) for each class, which is a central element that can be used to quantify belief mass and uncertainty mass according to equation 5. To better address overfitting, we develop a novel loss function that augments DL based evidence prediction with uncertainty anchor sample identification through kernel density estimation (KDE). These anchor samples are unlabeled data that inform the ADL which areas of the data space are less explored. Optimizing this loss function will ensure that ADL predicts high vacuity over these areas. Furthermore, through KDE, these less explored areas are automatically ranked based on their data density. This nice property allows the ADL to effectively prioritize the sampling order over these areas and iterative visit them based on their data density. Finally, we introduce our novel sampling function that systematically integrates accurately estimated multi-source uncertainty for active deep learning.

heading: 4.1 UNCERTAINTY ANCHOR SAMPLE IDENTIFICATION
text: Let Xu and Xt denote the sets of unlabeled candidate and training samples, respectively. The probability density of the two populations with a kernel function k(·, ·) can be estimated as follows:
pu(x) = 1 |Xu| ∑ xn∈Xu k(x,xn), pt(x) = 1 |Xt| ∑ xn∈Xt k(x,xn) (9)
Since we aim to identify unlabeled anchor samples to inform the model areas in the space that are less explored by the training data, these samples should be from areas having a high density mass with respect to pu(x) but low density mass with respect to pt(x). The problem is formalized as:
max A⊆Xu λ ∑ x∈A pu(x)− ∑ x∈A pt(x) (10)
The first term ensures that the selected area has abundant candidate data points to sample so that it has lower risk of containing isolated noise. The second term makes sure the selected region is located OOD with respect to the current training data. The optimal set for equation 10 is given by:
A∗ = {x|λpu(x)− pt(x) > 0} (11) where λ ∈ [0, 1] is used to control the size of A∗ for given the candidate and training datasets.

heading: 4.2 MULTI-SOURCE UNCERTAINTY PREDICTION
text: The set of uncertainty anchor samples A∗ represents areas in the data space that are cohesively distributed far away from the current training data. As these data are essentially OOD with respect to the current training data, their predicted vacuity should be high, which implies low predicted evidence due to Lemma 2. We leverage this information by constructing an evidence strength loss, L(u)Evi, which forces the model to predict low evidence for xu ∈ A∗:
L(u)Evi(A ∗,Θ) = |1T(xu∈A∗)f(xi|Θ)| (12)
where 1(C) = 1 if C is true and 0 otherwise; ri = f(xu|Θ) is the output of the DL model, representing the predicted supporting evidence of xu, and Θ is the set of DL model parameters. Since we require rk ≥ 0, an activation layer (i.e., ReLu) is used to replace the softmax layer as commonly used in other NN classifiers. The evidence strength loss is the key component to our proposed overall loss function. Samples in A∗ act as anchors to provide the model a preview of certain areas that out of its current knowledge. The model is guided to put less belief mass on those areas, leading to more accurate uncertainty estimation and eventually benefit the multi-source uncertainty based data sampling. Furthermore, since the activation layer is used for model output, equation 12 essentially performs l1 regularization to last hidden layer’s weight matrix and bias vector. We want to emphasize that our approach demands no additional labeling cost. The anchor samples are dynamically detected according to the current training and put into use without their actual label being known.
We proceed to define our overall loss function. For training sample xi, let yi encode the ground-true class label k by setting yik = 1 and yij = 0,∀j 6= k. Let Cat(ŷi = k|pi(Θ)) be the likelihoood, where pi(Θ) ∼ Dir(pi|αi(Θ)) and αi(Θ) = f(xi|Θ) +Wai. We set the non-informative weight W = K and base rates aik = 1/K,∀k. The expected sum of squares loss is defined as
L(i)(Θ) = Epi∼Dir(pi(Θ)|αi(Θ))||yi − pi|| 2 2 = K∑ j=1 (y2ij − 2yijE[pij(Θ)] + E[pij(Θ)2]) (13)
Minimizing L(i)(Θ) has the effect of jointly minimizing the prediction error and the variance of pi (Sensoy et al., 2018), hence reducing the uncertainty. This can be seen using identity E[pij(Θ)2] = E[pij(Θ)]2 + Var(pij(Θ)) and rearranging the terms on the r.h.s. of equation 13. Our overall loss function is defined as:∑
xi∈Xt L(i)(Θ) + λ1 ∑ xu∈Xu L(u)Evi + λ2L2(Θ) (14)
where L2(Θ) is the standard L2 regularizer of the network parameters.

heading: 4.3 DATA SAMPLING FOR ACTIVE DEEP LEARNING
text: According to Lemma 2, a data sample’s vacuity is maximized when the model assigns zero evidence to all K classes. This indicates the model has never seen a similar data sample from training. Annotating samples with a large vacuity can help the ADL gain most new knowledge of the data space. It has the effect of guiding the model to explore the most important areas, which is especially critical for a high-dimensional data space. In AL, the true decision boundary can be easily skewed due to limited initial training. The vacuity-aware search helps the model fast converge to the true decision boundary without excessively sampling around the wrong one. Moreover, it is also effective to discover new types of classes whose instances have never exposed to the model, as shown in our experiments. According to Lemma 1, a data sample’s dissonance is maximized when the model assigns equally high (close to infinity) evidence to all K classes. These strong conflicting evidence received from different classes indicate the data sample is located near the decision boundary where multiple classes are heavily overlapped. Annotating samples with high dissonance helps the model further fine-tune the decision boundary, leading to better discriminative power.
We design a sample function that best leverages these two important and complementary sources of uncertainty to most effectively guide ADL. Intuitively, we would like ADL to rely more on vacuity in the early phase of AL, which can most effectively shape the decision boundary and avoid finetuning the wrong decision areas. As AL goes, dissonance should gradually gain a higher weight, which allows ADL to further fine-tune the decision boundary that has the right shape but is less accurate, aiming to maximize the discriminate power of the model. The sample function is given:
x∗ = arg max x∈Xu [diss(ω(x)) + βvac(ω(x)] (15)
where β is an annealing coefficient to gradually balance between vacuity and dissonance based on the rationale given above. More specifically, the importance of vacuity reduces as there are less “vacuous” areas in the data space w.r.t. the current training data. This implies that the training data can well approximate the entire data space. Thus, one natural way to quantify β is to use the inverse of mutual information KL(pu,t||pt), where the joint density distribution pu,t can be estimated as pu,t(x) =
1 |Xu|+|Xt| ∑ xn∈Xu∪Xt k(x,xn). In practice, calculating the mutual information for each
AL iteration is expensive. We use a heuristic surrogate: min max=minxu∈Xu maxxt∈Xt k(xt,xu) and set β = 1− dT if min max does not change within the past few AL iterations, where T denotes the current iteration of AL and d is a fixed decay rate (set to 1/100K in our experiments).

heading: 5 EXPERIMENTS
text: In this section, we report our experimental results on both synthetic and real-world data. The synthetic experiment aims to verify the key theoretical properties of ADL, including entropy decomposition and multi-source uncertainty prediction, and how these properties contribute to AL. The real-data experiment aims to compare ADL and its competitors. We focus on testing in classical AL environment, where the initial training set only includes limited samples from some classes with samples from other classes completely missing. In each AL iteration, we sample one or a small batch of data instances. This is fundamentally different than some recent DL based AL methods, such as (Sener & Savarese, 2018), which perform batch-mode sampling with a large batch size (larger than our entire labeled samples). All models uses the same DL architecture. For synthetic data, we adopt a 3-layer MLP with tanh for activation. For real data, we use LeNet with Relu for activation.

heading: 5.1 SYNTHETIC DATA
text: The synthetic experiment is designed to show: (1) whether ADL accurately captures different sources of uncertainty, and (2) whether accurately estimated uncertainty leads to better AL behavior. To mimic the existence of OOD, we generate three mixtures of Gaussian’s. Each mixture consists of a major and a smaller (i.e., OOD) clusters with 750 and 50 samples, respectively. We center the major Gaussian components from each class in the middle of the feature space and put their corresponding OOD components away from them. In Figure 1, we show that a classical DL model with a softmax layer provides very inaccurate uncertainty estimation. In contrast, the proposed ADL model not only provides accurate entropy prediction but also successfully decomposes it into vacuity and dissonance. Figure 2 shows the uncertain prediction result from EDL (Sentz et al., 2002), which can also provide evidence prediction but requires ample training data. Suffering from insufficient training, EDL is inaccurate in its entropy prediction, especially for the OOD clusters. While EDL does not provide entropy decomposition, we use its predicted evidence to compute vacuity and dissonance as shown in Figure 2. However, neither of them is accurately predicted as low vacuity is predicted for the three OOD clusters where there is no training data and high dissonance is predicted in areas with no nearby training data to show conflicting evidence.
Figure 3 shows the first time when ADL selects at least one data sample from each OOD area, high vacuity is assigned to an area with no training data but many unlabeled data. Meanwhile, high dissonance indicates that refining the decision boundary may be more instrumental to improve the model performance. A few iterations later, ADL starts to penalize vacuity. While vacuity is still accurately estimated (high in vacuous regions), it becomes less useful for sampling (since very few unlabeled data is located nearby). Two iterations later, penalizing on vacuity helps to choose data samples that significantly refine the decision boundary. The superior AL performance of ADL as shown in Figure 3 further confirms the effectiveness of ADL’s key properties as demonstrated above.

heading: 5.2 REAL DATA
text: The real-world experiment is conducted on three datasets, MNIST, notMNIST, and CIFAR-10, all of which have ten classes. To mimic the real-world AL scenario, we leave 2-5 classes out for initial training and there are 5 labeled samples for each available class. A good AL model is expected to discover samples of unknown classes in an early stage to effectively improve model accuracy. We compare the proposed model with EDL (Sensoy et al., 2018) (entropy, vacuity+dissonance), BALD (Gal et al., 2017) (epistemic), and softmax (entropy, random), where in the parenthesis are the uncertainty measurements used for sampling. Figures 4 and 5 show that ADL consistently outperforms other models on all three datasets. The advantages of ADL are twofold. First, entropy decomposition gives ADL flexibility to meet distinct sampling need at different AL phases. In an early stage, the fast accuracy improvement is achieved by the vacuity guided sampling where the most representative samples are labeled with high priority. Gradually, ADL switches to dissonance guided sampling to refine the decision boundary by labeling the most informative samples to improve its discriminative power. In contrast, sampling methods utilizing a unified uncertainty (e.g., epistemic uncertainty and entropy) lack such flexibility to adjust the sampling behavior, leading to either slow convergence or lower model accuracy. Second, compared with EDL, which can also perform evidence prediction, ADL is superior due to accurate uncertainty estimation using the effective loss function. For both synthetic and real data, we observe that ADL identifies samples from missing classes at least around 20% faster than using EDL and other models.

heading: 6 CONCLUSION
text: We present a novel active deep learning model that systematically leverages two distinct sources of uncertainty, vacuity and dissonance, to effectively explore a large and high-dimensional data space for label-efficient training of DL models. The proposed ADL model benefits from the evidencebased entropy decomposition that follows from our theoretical analysis of belief vacuity and belief dissonance under the SL framework. The multi-source uncertainty can be accurately estimated through a novel loss function that augments DL based evidence prediction with vacuity-aware regularization of the model parameters. Through dynamically balancing the importance of vacuity and dissonance, a sampling function is designed to first explore the critical areas of the data space and then fine-tune the decision boundary to maximize its discriminate power. Extensive experiments conducted over both synthetic and real data help verify the theoretical properties and empirical performance of the proposed ADL model.

heading: A APPENDIX

heading: PROOF OF LEMMA 1
text: Proof. Let Bjk denote Bal(bj , bk). Since 0 ≤ bk ≤ 1 (as S = ∑
k bk +K), we have 0 ≤ Bjk ≤ 1. In addition, Bjk = 1, if bj = bk 6= 0; Bjk = 0, if bjbk = 0. Thus, we have ∑ j 6=k bjBjk ≤∑
j 6=k bj , where the equality holds when Bjk = 1,∀j. Therefore, we have
diss(ω) = K∑ k=1 bk
[∑ j 6=k bjBjk∑
j 6=k bj
] ≤
K∑ k=1 bk (a) = ∑K k=1 rk S (b) = S −K S = 1− 1 C
(16)
where (a) is due to the definition of bk in equation 5 and (b) is due to the summation constraint in equation 1 and W = K.

heading: PROOF OF LEMMA 2
text: Proof. Using the definition of uncertainty mass in equation 5 and substituting W by K, we have
0 ≤ vac(ω) = K S
= K∑
k=1 rk +K ≤ 1 (17)
where equality is achieved when ∑K
k=1 rk = 0.

heading: PROOF OF THEOREM 1
text: Proof. For (1), (⇒) is easy to show as S = K implies ∑K
k=1 rk = 0 and vac(ωy∗) = 1; for (⇐), using equation 2 and non-informative base rates, we have P (y∗ = k) = 1/K,∀k, which achieves a maximum H[y∗] as logK.
For (2), we first prove (⇒). For y∗ = arg maxH[y], we have P (y∗ = k) = 1/K,∀k. Thus, (rk + akK)/S = 1/K,∀k. For S →∞, denote S = CK and we have rk/S + ak/C = 1/K,∀k. Let C →∞, we have rk/S → 1/K,∀k. Thus, we have y∗ = arg max diss(ωy) due to Corollary 1. To prove (⇐), diss(ωy∗) = 1 implies that r1 = ...rk... = rK and S →∞. Hence, limS→∞ P (y∗ = k) = limS→∞(rk + akK)/S = 1/K, which implies that y∗ = arg maxH[y] for S →∞.

heading: ADDITIONAL EXPERIMENTAL RESULTS
text: We obtain similar AL curves for notMNIST and CIFAR-10 when starting AL with 7 and 8 classes as with 5 and 6 classes, which are shown in Figure 6. In Figure 7, we also report the AL performance on the three datasets when there is no missing class. ADL still achieves the best performance in all cases with slightly less advantage than other models.

heading: EXPERIMENTAL SETTINGS
text: We choose the Adam optimizer to train ADL for 600 epochs and setting the learning rate to 0.001. The coefficient of evidence strength loss, λ1, is set to 0.005 (cross validated from {0.001, 0.005, 0.03, 0.05}). The coefficient of the L2 regularizer, λ2, is set to 0.05 (cross validated from {0.001, 0.005, 0.01, 0.03, 0.05, 0.08}). The λ for anchor sample identification in equation 10 is set to 0.005 (cross validated {0.001, 0.005, 0.03, 0.05}). We choose RBF as our kernel function with length scale set to 1 (cross validated from {0.01, 0.1, 1}).

heading: EFFECTIVENESS OF KDE BASED UNCERTAINTY ANCHOR SAMPLE IDENTIFICATION
text: In this section, we conduct additional experiment to evaluate the effectiveness of the KDE based uncertainty anchor sample identification method. Uncertainty anchor sample identification is an integral component of the ADL model, which aims to guide the model to be uncertain in the OOD areas with respect to the current training data (instead of providing the final prediction). Therefore, other more advanced kernel functions/similarity measures that are specifically designed for highdimensional data can be used for the same purpose without affecting the overall model. However, when choosing a specific technique, it is also important to consider both the quality of the data samples and efficiency as fast identification of these data samples is critical for AL which is usually performed in real time. Since the model is constantly changing as it continues to explore the data space, new uncertainty data samples need to be discovered in each AL iteration. We have conducted three additional experiments to demonstrate which technique can achieve such a good balance.
• We have compared KDE with the randomly selected anchor samples from unlabeled data and not using any anchor samples in Figure 8(a). KDE clearly outperforms random selection, which in turn performs better than not using any anchor samples. We further confirm the positive result by evaluating the min-max similarity between the unlabeled and training data. If KDE is able to identify anchor samples from the desired OOD regions of the feature space (although the estimated density in that region may not be very accurate), the sampling process would be guided correctly and the min-max similarity would increase in the next AL iteration as the result. Figure 8(b) compares the min-max similarity of KDE with random selection. The result shows that with KDE, the model covers the unlabeled feature space much more efficiently as AL moves forward.
• We have adopted the attention kernel as a more advanced distance metric to replace the RBF kernel in the proposed anchor sample identification component. The attention kernel is the major component in the matching network (Vinyals et al., 2016), where the spatial invariance is ensured by CNN and the dimensionality of the inputs is reduced through two correlated LSTM projections. However, the attention kernel (our current implementation) is much slower to compute as compared with KDE especially when facing a very large unlabeled pool as the entire candidate data samples need to be embedded every iteration when the training/testing data are changed along with AL. Thus, if the improvement is not significant (see Figure 8(a)) and when the efficiency becomes a bottleneck for a large unlabeled pool, the proposed KDE approach appears to be a good choice as it can provide a good balance between quality and efficiency, which is critical for AL.
• We have investigated the impact of the characteristic length scale used in RBF kernel on AL performance. Figure 8(c) shows that the ADL model performance is fairly robust to the length scale and only shows minor change with different choices.

heading: ABLATION STUDY
text: We have conducted a detailed ablation study to clearly demonstrate the effectiveness of each major technical components:
• Figure 9(a) compares proposed sampling method with other different sampling criteria: entropy, vacuity only, and dissonance only. The result confirms the effectiveness of the dynamically balanced sampling method. It is interesting to see that using vacuity alone performs quite well in the initial phase but only converges to a lower accuracy in the end. In contrast, using dissonance is slow to start but able to converge to a higher accuracy. The entropy curve roughly stays in the middle of the above two curves.
• The effectiveness of using the anchor samples has already been demonstrated in Figure 8(a) by comparing ADL with not using any anchor samples and randomly selected anchor samples from unlabeled data.
• Figure 9(b) shows the results using different vacuity/dissonance ratios but keeping fixed throughout the AL process. The dynamically balanced sampling method clearly outperforms the fixed weighting. This also demonstrates the usefulness of the proposed entropy decomposition theory. Since the sampling goal of AL changes with the accumulation of the labeled data, the optimal AL behavior can only be achieved by adaptively adjust the importance of vacuity and dissonance in the sampling function.
Finally, we have conducted batch-mode AL and reported the results in Figure 9(c). As can be seen, as the batch size increases, the performance decreases. This is expected as there is no special strategy to diversify the samples chosen in the same batch. We will leave this to our future work as the current model is not designed specifically for batch mode AL.
SAMPLE IMAGES CHOSEN BY ADL
In this section, we have visualized the image samples selected by ADL in the early and later stages of active learning to help better understand the role of vacuity and dissonance in data sampling. In order to better demonstrate the effectiveness of the vacuity measurement, we start active learning with 5 classes omitted from the initial training. Later we will see how does high vacuity at the early stage of active learning helps fast identify missing classes. Figure 10 shows the samples with highest vacuity selected by ADL in the first 30 AL iterations. The first four of them are from missing classes. This clearly demonstrates the effectiveness of using vacuity to explore the data space. As a result, data samples from the missing classes are quickly identified and being labeled. The last sample is from class ’3’, whose examples have already been exposed to ADL. However, the writing style of this sample is very different than other instances from the same class, which result in a high vacuity.
Figure 11 shows the samples with highest dissonance selected by ADL in the last 100 AL iterations. By observing their predicted belief mass, we find that the high dissonance result is due to the conflicting belief over multiple classes. For example, the first sample is confusing between classes ’4’ and ’6’; the second sample is confusing among classes ’5’,’6’, and ’8’; the third sample is confusing
among classes ’4’,’7’, and ’9’; the fourth sample is confusing between classes ’4’ and ’6’; and the Fifth sample is confusing between classes ’0’ and ’9’.

heading: SOURCE CODE
text: The code for this work can be found in https://drive.google.com/drive/folders/ 1imwnOahh8HtHK_g_HSTb4TCxZ7YG04ay?usp=sharing

Review summaries:
This paper propose an active deep learning model. By leveraging subjective Logic, they propose to decompose the entropy of a predicted class distribution into vacuity (lack of evidence) and dissonance (conflict of strong evidence). Instead of using the predicted class distribution, they estimate the supporting evidence for each class. In the actual data sampling stage, they first sample from those high-vacuity dense region, to shape the true decision boundary, and then gradually sample from those high-dissonance region to fine-tune the decision boundary. They show better performance than the baselines on both synthetic and real datasets. 

The authors consider active deep learning. They propose decomposing predictive entropy into a) vacuity (lack of evidence) and b) dissonance (contradictory evidence). They frame this in terms of "subjective logic". In practice this is achieved by having the NN output the parameters of a Dirichlet, which allows an additional degree of freedom describing variance/vacuity. Dissonance is defined in terms of the support of contradictory classes. To get improved estimates of vacuity they augment the loss with a term regularizing the Dirichlet parameters to be small (low precision) at unlabelled points with higher KDE(unlablled points) than KDE(labelled points). They propose initially weighting vacuity and later dissonance as AL proceeds. Encouraging results are presented on simulated 2D data, MNIST and CIFAR10.