Stop Guessing: Choosing the Optimization-Consistent Uncertainty Measurement for Evidential Deep Learning
Authors: Linye Li
Yufei Chen
Xiaodong Yue
Xujing Zhou
Qunjie Chen
Venue: ICLR 2026 Poster (2026)
Keywords: uncertainty estimation;
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Evidential Deep Learning (EDL) has emerged as a promising framework for uncertainty estimation in classification tasks by modeling predictive uncertainty with a Dirichlet prior. Despite its empirical success, prior work has primarily focused on the probabilistic properties of the Dirichlet distribution, leaving the role of optimization dynamics during training underexplored.  In this paper, we revisit EDL through the lens of optimization and establish a non-trivial connection: minimizing the expected cross-entropy loss over the Dirichlet prior implicitly encourages solutions akin to multi-class Support Vector Machines, maximizing decision margins. Motivated by this observation, we introduce the \emph{optimization-consistency principle}, which deems an uncertainty measure valid if its value decreases as samples approach the global optimum of the training objective. This principle provides a new criterion for evaluating and designing uncertainty measures that are consistent with the optimization dynamics. Building on this foundation, we further propose a novel measure, \emph{Margin-aware Predictive Uncertainty (MPU)}, which directly captures the separation between target and non-target evidence. Extensive experiments on out-of-distribution detection and classification-with-rejection benchmarks demonstrate the effectiveness of our propositions.

Published as a conference paper at ICLR 2026
STOP GUESSING: CHOOSING THE OPTIMIZATION- CONSISTENT UNCERTAINTY MEASUREMENT FOR EV- IDENTIAL DEEP LEARNING
Linye Li" Yufei Chen Xiaodong Yue² Xujing Zhou²
Qunjie Chen'
'School of Computer Science and Technology, Tongji University
Artificial Intelligence Institute, Shanghai University
(linyeli, yufeichen) eetonnji.edu...om cn, yswant edu muxixi@shu.er cn, cn, 2250115@tongji.ede
ABSTRACT
Evidential Deep Learning (EDL) has emerged as promising framework for un- certainty estimation in classification tasks by modeling predictive uncertainty with Dirichlet prior. Despite its empirical success, prior work has primarily focused on the probabilistic properties of the Dirichlet distribution, leaving role op- timization dynamics during training underexplored. In this paper, we revisit EDL through the lens of optimization and establish a non-trivial connection: minimiz- ing the expected cross-entropy loss over the Dirichlet prior implicitly encourages solutions akin to multi-class Support Vector Machines, maximizing decision mar- gins. Motivated by this observation, introduce the optimization-coonisttenneyy principle, which deems an uncertainty measure valid if its value decreases as samples approach the global optimum of the training objective. This principle provides a new criterion for evaluating and designing uncertainty measures that are consistent with the optimization dynamics. Building on this foundation, we further propose a novel measure, Margin-awarr Predictive Uncertainty (MPU), which directly captures the separation between target and non-target evidence. Extensive experiments on out-of -idtr--distribution detection and classification-witte rejection benchmarks demonstrate the effectiveness our propositions.
INTRODUCTION
In recent years, Evidential Deep Learning (EDL) (Sensoy al. 2018 Malinin Gales 2018 Char-
pentier & Günnemann (020)) has emerged as popular approach for classification tasks. By training single deterministic network to predict the parameters of Dirichlet distribution, EDL provides an elegant way to jointly capture aleatoric (data) and epistemic (model) uncertainty. Unlike methods that draw inspiration from Bayesian Neural Networks, such as MC-Dropout (Gal Ghahramani] 2016) and Deep Ensembles (Rahaman et al. 2021), which require multiple models repeated sam- pling to approximate posterior distributions, EDL. achieves efficiency with only single model and single forward pass at test time. This computational advantage, together with its strong empirical performance, has led to its widespread adoption various downstream tasks like open-set recog-
nition (Bao al. 202T Zhao al.) (003),, trusted multi-view classification (Han al. 2021 Liu
et al.
2022 2025a
c) and distrrbiist detection (Gao et al. 2025 Chen et al.
2023a
al.
2025
Fu al. 2026 Qu al. 2024).
Although EDL. has been extensively studied, prior work has almost exclusively analyzed it from a probabilistic perspective. For example, by designing priors, leveraging Fisher information, im-
posing constraints on Shannon entropy to improve performance (Bengs et al. 2022 2023) Wimmer
al. 2023 Juergens al. 2024 Liang al. 2019 Chen al. 2023b Qu al. 2024 Yoon & 2024). However, a critical gap remains: EDL fundamentally deep learning model, and the
Kim
"Corresponding author.
Code is available at httptpp://gtttbbb.oo///i/////

---

Published as a conference paper at ICLR 2026
behavior of deep learning models is strongly shaped by optimization processes such as loss function design and gradient dynamics. Thus, treating EDL merely as a universally applicable probabilistic estimator, while overlooking its optimization characteristics, provides an incomplete understanding.
In this paper, we revisit EDL from an optimization perspective and uncover a non-trivial theoretical connection: minimizing the expected cross-entropy loss under the Dirichlet prior implicitly drives the solution towards that of multi-class Support Vector Machine (SVM) (Crammer & Singer 2001 Hsu & Lin 2002). which maximizes the decision margin. This observation reveals that the notion of margin naturally emerges from the EDL objective, providing a more fundamental and optimization- aligned interpretation of uncertainty than conventional metrics derived solely from Dirichlet pa- rameters. Intuitively, samples with large margins are more certain, while those near the decision boundary are intrinsically uncertain. Building on this insight, we develop principled framework that bridges optimization and uncertainty estimation in EDL. Our contributions are summarized as:
Theoretical connection to maxximuu-margeee SVMs. We formally establish a connec- tion between Evidential Deep Learning (EDL) and the maximum-margin principle of Sup- port Vector Machines (SVMs). In particular, we prove that the Uncertainty-awarr Cross- Entropy (UCE) loss admits a margin- aware lower bound, offering new interpretation of EDL beyond its probabilistic formulation.
Optisira principle for uncertainty. We introduce the notion of optimizaaiion-consiieeer which defines an uncertainty measure as valid if its value mono- tonically decreases as a sample approaches the global optimum of the training objective. This principle bridges loss optimization with predictive uncertainty, providing a new crite- rion for evaluating uncertainty measures.
Margin-aware predictive uncertainty (MPU). Guided by the
consssstoonnc principle, we propose a novel uncertainty measure, Margin-awan Predictive Uncertainty (MPU), which explicitly aligned with the UCE loss and captures the margin between the dominant class evidence and the aggregated evidence of other classes.
Experimental validation. Extensive experiments on OOD detection and misclassification detection benchmarks demonstrate that aligning uncertainty estimation with optimization objectives substantially improves predictive reliability.
2
RELATED WORKS
2.1
SUBJECTIVE LOGIC AND EVIDENTIAL DEEP LEARNING
The theoretical foundation of Evidential Deep Learning (EDL) is closely related Dempster-Shafer Theory (DST) (Dempster, 1968) and Subjective Logic (Josang 2016), which extend conventional probability theory by explicitly modeling uncertainty. A central concept in this framework is the subjective opinion represents the uncertainty mass, and a = (al, "ul,...,ap, ,ax specifies the prior base-rate distribution. These quantities satisfy Σ by + u = 1 and Z==1= a3 = 1. Intuitively, b reflects direct evidence for each class, u quantifies the lack of evidence. and a serves as an uninformative prior. Build- ing upon this framework, for input æ. EDL (Sensoy al. 2018) maps the network parameterized with @ to logits z. followed by a non-negative activation function a, to get the evidence value, defined a
(b, u, a). where b = (bi,..., bx) denotes the belief masses, u [o. 1]
1, with total strength = - ag.. From these parameters, the belief and uncertainty masses can be computed as by e and u K/S. To train the model, EDL minimizes the expected empirical risk under the Dirichlet distribution, using losses such as the Brier Score, Type II Max- imum Likelihood loss, and the Expected Cross-entropy loss. A detailed derivation is provided in Appendix[A A However, prior work has typically optimized EDL models using a single loss function without formally establishing why that loss is particularly suitable. In this work, pected cross-entropy over the Dirichlet distribution, commonly referred to as the Uncertainty-aware
i.e., === <<( O)). Then, the corresponding Dirichlet parameters
focus the
Cross-Entropy (UCE) loss (Bilos et al. 2019 Sensoy et al. 2018 Charpentier et al. 2020 202T):
K
K au dp. =
K
()) =
Σ
Biy log(p:s)
B(x,)
1
Pij j=1
Σ 1=1
Ve (V(.)) vaar)) (1)
2

---

Published as a conference paper at ICLR 2026
where ci(-) denotes the digamma function and B(-) is the multivariate Beta function. We will show that optimization under this loss exhibits properties: the learned model parameters naturally align with the maximum-man margin solutions of classical multi-class Support Vector Machines (SVMs). This reveals a principled link between EDL and the well-established theory of max-margin learning.
2.2 UNCERTAINTY MEASURES WITH DIRICHLET DISTRIBUTIONS
In EDL, uncertainty is quantified by exploiting the probabilistic properties of the Dirichler distribu- tion. Several uncertainty measures have been proposed in previous works, each capturing different aspects of predictive reliability. First, the Vacuity of Evidence (VoE), defined as u K/S with EØ= atj. originates from Subjective Logic (Josang. 2016). It measures the overall lack of evidence in favor of any class, directly reflecting the igmorance of the model with respect to the pre- diction. Second, the Differential Entropy (DE) of the Dirichlet distribution, (Di (p a)). is adopted from information theory as global measure the dispersion of the predictive distribution, with higher entropy indicating greater uncertainty. Third, Mutual Information (MI), [ p). is adapted from Bayesian Model Averaging (Malinin & Gales) 2018 Gal et al.) (016) to isolate epis- temic uncertainty from total predictive uncertainty, quantifying how much knowledge of the model parameters reduces uncertainty in predictions. A detailed derivation of these measures is provided Appendix B Despite the availability of various measures, prior work has not established which is most effective. In this paper, we analyze and compare them from an optimization perspective for EDL models trained with the UCE loss.
PRELIMINARY
3.1 NOTATIONS
Following Juergens et al. 2024), we decompose the overall parameter set of the model, O, into the last-layer linear classifier W == [wi.... wx) Kxd and the parameters of the feature embed- ding function V that maps input æ into d-dimensional latent feature. The evidence for each class is defined as (x; +) = exp(2*)
expf u v (x)), where wy denotes the classifier vector associ- ated with class k. The exponential function exp(-) serves as non-negative activation, ensuring that the predicted evidence remains positive.
3.2 CRAMMER AND SINGER K- -CLASS SVM
In the following, we will analyze the relationship between EDL and classical multi-class support vector machine (SVM), namely the Crammer and Singer (C&S) K-class SVM. To this end, we first recall the primal formulation of this SVM and present the form of its optimal solution.
Definition The primal optimization problem for the Crammer and Singer multi-class SVM
(Crammer & Singer 2001 is defined
K w w
N
min E
&
3=1
1=
s.t.
w y. x. w. x; 8p. 11 Vi e [L, j Vi.
where 3 > 0 is the regularization constant, N is the number of training samples, & 2 0 are slack variables, y/ [K] denotes the ground-truth label of sample æ; ö is the Kronecker delta function 6p.g 1 p p- else 0 otherwise. A sample lies correctly outside the margin if & = 0, while & > indicates either margin violation or a misclassified sample. By considering the corresponding dual problem and applying the Karush Kuhn-Tucker (KKT) conditions (Bertsekas 1997). the optimal
classifier vectors can expressed as weighted sum of the training samples
w = 3-1
(8 Th.s)aa.
where The are the dual coefficients satisfying I........ , M. 2 and
Ei= Thi 1. A complete
derivation of the dual formulation and Eq. 3 is provided in Appendix C.1
The term (ōy. T/.33)
3

---

Published as a conference paper at ICLR 2026
646
120
627
1013
C&S
(b)
Figure I: Visualization of predictive uncertainty (Vacuity of Evidence), optimized linear classifiers W of EDL and C&S SVMs toy dataset in (a). We also trained One-vs-Rest (OvR) SVM SShhökoop 1997) on this dataset for comparison in (b).
acts sample contribution weight. Let's analyze this contribution: For the correct class (j yr), the weight is (1 7.,,)) sample æ; is easily classified (i.c., not a support vector), 16.9. - 1, and its contribution Wy, vanishes. This means easy samples do not define the classifier. For any incorrect class (j * y/), the weight is (-Nt.))- Th. > O, it implies z; is a support vector that is on or within the margin class j. The term ;; then this incorrectly active sample. In essence, the optimal SVM classifier wi is constructed by "pulling" towards challenging samples of its own class and "pushing" away from samples of other classes that violate the margin. The magnitude of these pulls and pushes is determined by the dual variables Maj which are non-zero only for the support vectors, where the most informative uncertain samples for defining the decision boundary.
"push" the classifier wy away from
4 MARGIN-AWARD PROPERTY OF EDL OPTIMIZED WITH UCE LOss
In this section, we will show that the classifiers W obtained by minimizing the UCE loss takes form closely resembling the weight vector of K-class SVM. This observation suggests that EDL implicitly performs margin-baseed classification. thus providing connection between probability- based uncertainty estimation and the classical principle of maximum margin.
4.1 MOTIVATION: ALIGNMENT OF EDL CLASSIFIERS WITH SVM OPTIMAL SOLUTIONS
To analyze the relationship between C&S SVM classifiers and those learned by an EDL model (via Eq. J.. we conducted a simple experiment. We assumed fixed, Gaussian-distribune features-an approach that enables direct comparison with SVMs. Importantly, these results are largely indepen- dent of the specific distribution, as the classifiers are mainly determined by support vectors. Further visualizations with different distributions are available Appendix
In Fig. [ we conduct experiments on a toy dataset consisting of three with an isotropic standard deviation of 0.3. We assume that the EDL. model's mapping V is the identity function. This allows to focus solely on the properties of the loss function with respect to the classifier weight vectors W. Green arrows represent the classifiers learned by the EDL model, while red arrows indicate the directions of the SVM classifier vectors. Black dotted lines correspond to the separating hyperplanes learned by the SVMs. We can make three main conclusions:
clusters
1. In Fig. the EDL classifiers closely align with the C&S SVM classifiers, suggesting an implicit margin-aware separation the EDL optimization with UCE loss.
2. In contrast, when comparing EDL with the classical one-vs-rest (OvR) SVM (Schölkopf 1997) as shown in Fig. [b the alignment less tight. While EDL and OvR classifiers share similar overall behavior this problem, the final OvR solution W is not as closely aligned with the EDL classifier as C&S SVM is, highlighting that the tight alignment specific to the C&S multi-class formulation.
4

---

Published as a conference paper at ICLR 2026
3. By visualizing the uncertainty adopted in EDL (Sensoy al. 2018) (i.e., the vacuity of evidence u = S))), we find that regions of high uncertainty correspond to the intersection of the negative half-spaces defined by the decision hyperplanes, as shown in Fig. la
These observations are heuristic, motivating to explore potential implicit connection between EDL and the C&S SVM, with the aim of gaining a deeper understanding of the underlying mecha- nism of EDL. In the following, we analyze the relationship between the optimization objectives of EDL and those of the C&S SVM, as well as their respective optimal solutions.
4.2 THE MARGIN-AWARE PROPERTY OF EVIDENTIAL DEEP LEARNING LOsS
Based on our previous observations, we first demonstrate that the UCE loss in Eq. satisfies margin-aware property by explicitly optimizing a margin between the logits for the ground-truth class and those of the incorrect classes.
Theorem 1. The Margin-Aware Property of UCE Loss. For an input sample x with correspond- ing ground-trute truth label y. the UCE loss LUCE can be oowereounded in the following form:
W (x) wy (((z)) = (
LUCE (I., V. W, v) 2
Σ jFy
(4)
where (tt) = log (1 (K 1) min (1, exp (t))). Let 2j w/ (xx for simplicity. The inequality achieves equality when 2, = 2/' for all j.j' * y. and (z, w. v) = Ejty (w.j To (xx) -w] (x)). The full derivation of the margin-aware lower bound is provided in Appendix C.2 Since @(l) is increasing function with respect to l, minimizing the UCE loss is equivalent to maximizing the margin-like quantity (K - 1) Σj 2j, which contrasts the ground-truth class output 2y simultancously to all the other outputs 2j for all j * y. Thus, we call Eq. margin-aware property. Maximizing M(x, y) encourages by to be positive and relatively large, while driving each (j + y) to be negative and relatively small.
Proposition 1. On the Analogy between EDL Gradient Dynamics and the SVM Dual Solution. The optimization dynamics of Evidential Deep Learning (EDL) classifiers reveal a striking parallel to the structure of the C&S SVM's optimal dual solution (Eq. 3 This suggests that EDL's learning process inherently mimics the margin-maximizing mechanism of SVMs. To demonstrate this, we analyze the gradient of the UCE loss with respect to a classifier weight vector wy. Assuming the Dirichlet strength for the correct class is large (ass, > 1), a condition typically met by using an exponential activation, the gradient contribution from each sample can be approximated as:
V Ww Cucu W, D
Σ (6ucs - big) L (xx), where b.
(5)
A complete derivation process is in Appendix C.3 Here, bi is the belief mass assigned to class j for sample i, and (2;; is the sample's feature embedding. The analogy becomes clear when compare the core components:
SVM Optimal Solution (Eq. 3
w x
Ei - "Ti,)) 5): Awy x -V m, C R Ei(gges bi.)) ( (xx;)
UCE Loss Gradient Update (Eq.
The belief mass by in the EDL gradient plays a role directly analogous to the dual coefficient Vsy in the SVM solution. Both quantify a sample's 's relationship to each class. Let's examine the update term (6 bz.)): For the correct class (j = yi),, the term is (1- b,,,.). where lower belief br.g: means higher uncertainty about the ground truth label. Since the gradient update small for confident samples (b.,g - 1), the learning process naturally focuses on samples where the model remains uncertain. For any incorrect class (j a y:), the term is (-b,,)). This is the negative belief in that wrong class. If the model incorrectly assigns high belief to class (b.j > 0), this term generates a strong gradient to "push" wi away from the sample's embedding V (x;). This mechanism is dynamic equivalent to the SVM's optimal solution, where C&S SVM identifies fixed set of support vectors (πy > 0) to define the boundary, and EDL's gradient descent continuously uses the belief masses (b:z) achieve similar outcome. Samples with high uncertainty (low bi,g,) high
5

---

Published as a conference paper at ICLR 2026
EEL Creas Enerpyy (Cood
Difornded Enooop
Harget
1300
010
Fapors Bieeace
Destue memdor
Figure 2: The desired alignment between the loss landscape and predictive uncertainty. The figure compares the UCE Loss landscape (left) with the predictive differential entropy (right). As sample moves from an ambiguous region (cyan square the origin) towards an ideal region for the target class Wo (orange triangle), both its loss and uncertainty should systematically decrease.
conflict (high br.j for * y:) act as "dynammc support vectors", exerting the strongest influence on the classifier updates. This demonstrates that EDL's optimization process inherently structured to promote large margin separation, akin to multi-class C&S SVM.
4.3 OPTIMIZATION- CONSIITTNNY PROPERTY OF UNCERTAINTY MEASURE
In the previous subsection, established that minimizing the UCE loss in EDL implicitly aligns the learned classifier with the maximum maximum-marrin solution of SVMs. This raises a natural question: which uncertainty measures are compatible with such an optimization process? While many uncer- tainty measures have been proposed in prior works (e.g., vacuity of evidence, differential entropy and mutual information), not all of them necessarily reflect the progress of the optimization dynam- ics. Intuitively, appropriate uncertainty measure should achieve its minimum value sample exactly when this sample aligns with the optimal solution of the underlying loss function curve. Otherwise, the measure may contradict the training objective and thus misrepresent predictive relia- bility. To formalize this requirement, we introduce the principle of Optimization Coniiis---nnssncccc.... which characterizes uncertainty measures whose values monotonically decrease the optimization drives a sample toward the global optimum of the UCE loss.
Property of Uncertainty. Let u(ar; W, T) be uncer- tainty measure induced by the network parameters set {W. V We say that a is optimization- consistent with respect the UCE loss if, for any pair of training sample (æ, y) and (x',y'), the following implication holds:
Theorem 2. Oppiiizattin-Conniitter
LUCE (x, y. W. V S LUCE (x', V. N,WWWW v
xx W. v S uux' (1, V
(6)
In other words, the optimization dynamics drive the sample closer to the global optimum of the objective loss, its predictive uncertainty should monotonically decrease. One envision the loss function as a surface, where individual samples map to distinct points on this surface. A meaningful uncertainty measure should yield lower values as samples closer the valley, i.c. the optimal point of the loss surface.
Let's further explore whether existing EDL-related loss functions characterized by this Oprimization-CConissenngg property. Intuitively, we discover that the Vacuity of Evidence, defined as u(x; W, D K/ Ex=ı exp(w, v (x) + 1), does not fulfill this property. A counter-example demonstrates this clearly. Suppose we have K 3 classes. For samples æ and x', both la- beled as
y 1, assuming their predicted Dirichlet distributions are a = (10, and a' (10, 10, 1), respectively. It is clear that the first sample has a lower loss value. However, according to the definition of Vacuity of Evidence, the second sample x' appears to have lower un- certainty 3/(10+1+1) > 3/010+ 10+1). Hence, conclude that the Vacuity of Evidence fails Soamizzatoo-Coosistennen
to meet the Optimization- Consistency property. We further discover that the Differential Entropy, defined as in Eq. satisfies the above-mentioned property.

---

Published as a conference paper at ICLR 2026
Proposition 2. Cotimizattin-Consitterrr of Differential Entropy. Let ENT( a)) denote the differential entropy of the Dirichlet distribution parameterized by a as
ENT(Dir(p) | a)) = pEAK-1 Dir(p a) log Dir(p x) dp
log B(a)) +
( (S- K))(S)
Σ - (0--1))(as)))
Then the differential entropy uncertainty measure is optimization-ccnniitenn with the UCE loss (The- orem 2). That is, for any two training samples (æ, y) and (a', ), we have
LUCE (z, W, v I Lucu(w', y', W, v = ENT(Dir(p a<(x))) < ENT((irrp)) a (ax)))
Complete proof can be in Appendix C.4 It should be noted that the above equation is one-sided only: while reducing the UCE loss reliably decreases entropy, entropy by itself is insufficient to de- termine the relative ordering of UCE loss across samples. To provide a more intuitive perspective on this concept, we conduct an experiment on three-class toy dataset. As demonstrated in Fig. [ we chart the loss contour of UCE and the corresponding differential entropy contour. Notably, points located closer to the global optimum location of the loss function (depicted by the orange triangle) exhibit a lesser uncertainty value compared to those positioned farther away from the global opti- mum (indicated by the cyan square). This observation underscores our assertion that the differential entropy measure displays optimization consistency with respect the UCE loss.
4.4 MARGIN- PREDICTIVE UNCERTAINTY
Inspired by the margin- aware property of UCE loss and the Optimization-Consiine Consistency Principle, we propose specially designed uncertainty specifically for UCE loss, which refer to Margin- Aware Predictive Uncertainty (MPU).
Proposition 3.
The Margin-Aware Predictive Uncertainty (MPU) is uncertainty measure principle and the margin-aware property of the UCE loss.
inspired by the We define the MPU by the margin of the Dirichlet parameters a
MPU(x)) (K - 1)0g Σaji aj.
where Ò denotes the predicted class label corresponding the highest probability in the model's output. Intuitively, this formulation measures the margin between the evidence for the predicted class and all other classes. Therefore, a higher MPU score signifies greater predictive certainty.
Characteristics and Advantages of MPU. To illustrate the characteristics and advantages of MPU over conventional uncertainty metrics, provide comparative analysis in Fig. 3 From left to right, the predictive distribution becomes more concentrated on the ground-trui class. According to the optimization consistency principle, the measured uncertainty should monotonically decrease. However, we observe the following behaviors from the conventional metrics: (1) VoE and MI re- main stagnant, failing to reflect the increasing certainty. (2) DE shows a moderate decrease, but its (3) MPU increases significantly from 4 to 28, providing sensitive and interpretable measure of confidence. Therefore, the MPU possesses several distinct advantages:
negative value is less intuitive.
Intuitive and Interpretable: MPU is more interpretable than Differential Entropy (DE). DE values are non-positive and lack an intuitive scale, making the practical difference between values like -1.9 and -2.9 unclear. In contrast, MPU starts at 0 for maximum uncer- tainty and increases with predictive confidence, offering a direct and monotonic measure.
Unlike MI, which remains stagnant, or DE, which shows only moderate vari- ation, MPU highly sensitive the concentration of a probability distribution. This is evidenced by its dramatic increase from 4 to 28 as model confidence grows, proving its superior capacity for discriminating between confidence levels.
Sensitivity:
Versatility: VoE is limited to OOD detection, it only captures uncertainty from evidence absence while ignoring the evidential distribution for a fixed total Dirichlet strength (S). In contrast, MPU is more versatile. It measures uncertainty from both insufficient evidence (for OOD detection) and conflicting class evidence (c.g., when top predictions are close), making effective for misclassification detection as well.
7

---

Published as a conference paper at ICLR 2026
a² (4,8,8)
a² (5,5,10)
a² (2,4,14)
a' (2,2,16)
(3,7,7)
(4,4,9)
(1,3,13)
(1,1,15)
0.75
0.350.35
0.45
0.05 0.15
0.050.05
VoE! 0.15
VoE 0.15
VoE 0.15
VoE* 0.15
DE -1.9862
DE 11.9977
-2.5261
-2.9224
MI' 0.048102
M[ 0.048103
M[² = 0.0468
MI* 0.0458
MPU
MP 10
MPU 22
MPU* 28
Figure 3: Illustration of MPU's advantages conventional uncertainty metrics Vacuity of Evi- dence (VoE), Differential Entropy (DE) and Mutual Information (MI).
EXPERIMENTS
In this section, we empirically validate our theoretical findings regarding the role of optimization- consistency principle uncertainty estimation. Specifically, we perform experiments on CIFAR-10 and 1IFAR-100, with
(OOD) detection and misclassification detection tasks.
Table 1: Summary of EDL-related methods, highlighting their loss functions, activation functions, post-hoc regularization on the predictive Dirichlet distribution, and additional training techniques.
Method
Loss Function
Activation Function
Post-Hoc Regularization
Additional Techniques
EDL
Brier Score
Softplus
KL Divergence
RED
T2ML
Exp Softplus
KL Divergence
Evidence Regularization
I-EDL
Brier Score
KL Divergence
Fisher Information Regularization
R-EDL
Brier Score
Softplus
KL Divergence
Adjust Prior
Re-EDL
Brier Score
Softplus
Adjust Prior
PostN
UCE
Exp
Entropy Regularization
Multiple Normalization Flow
NatPN
UCE
Exp
Entropy Regularization
Normalization Flow
Our
UCE
Exp
5.1 EXPERIMENTAL IMPLEMENTATION
Baseline Methods. We benchmark our approach against two sets of baselines: 1) Evidential meth- ods, including EDL. PostN, NatPN, and recent EDL, variants (RED, I-EDI,, R-EDL, Re-EDL); and 2) Classical uncertainty methods, like MC Dropout and DUQ. Following their original designs, we use Vacuity Evidence (VoE, defined as K/S) as the uncertainty measure for EDL and its variants, and Dirichlet strength (S) for PostN and NatPN. Table provides summary of these baselines.
Implementation Details. We conduct experiments on the CIFAR- 1IFAR-10/100 datasets, using SVHN Netzer et al.) 2011). GTSRB (Stallkamp al. 2012), Places365 (Zhou all, (201)). and Food101 Bossard et al. 2014)) as OOD datasets. Following Chen et al. (2025), we employ VGG16 as the backbone network. Models trained for 200 epochs using SGD optimizer with initial learning rate 0. decayed by cosine scheduler. Notably, our training objective relies solely on the UCE loss and does not involve any post-hoc regularization terms, such KL regularization, constrain the predicted Dirichlet parameters or suppress misleading evidence. Performance on OOD and misclassification detection is evaluated using the under the precision-recall curv (AUPR).
5.2 EXPERIMENTAL RESULTS ANALYSIS
MPU as the Best Companion for UCE Loss. In Table 2 we compare UCE loss combined with four different uncertainty measures: MPU (Margin-aware Predictive Uncertainty), DE (Differential
8

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison evaluated by classification accuracy and AUPR scores for OOD detection and misclassification detection on CIFAR-10. - X indicates using X as OOD data. Re- sults are averaged over runs, with mean and standard deviation reported. The bold numbers indi- cate the best performance, while the underlined numbers represent the second-best method.
Method
-SVHN
CIIAAROO
GTTSRB
-PPaccs365
-ooo
Cls Acc
Mis Detect
MC Dropout
78.4013.88 88
85.3910.58
83.51=2.32
67.6342.34
76.07=2.58
90.16=0.23
98. 08006
EDL
12.
87.13-0.26
84.57=1.26
70.4610.77
80.18=0.69
88.48=0.32
98.74=0.07
DUQ
81.44=4.63
85.38+0.37
83.35=4.30
66.20+33.11
75.87=4.40
89,39=0.13
97,98=0.34
PostN
83.76+0.46
87.07=00.94
84.83+1667
71.79++2.36
78.83+2.53
87.82=0.06
97.46-0.06
NatPN
83.5620.38
88.05=0.75
84.76-2.03
71.14:22.06
79,44=2.63
87.73=0.09
97.53=0.05
RED
82.85+2.35
87.84=0.54
85.30=3.31
70.78+22.33
79.91=1.92
89,43=0.28
98.82=0.09
T-EDL.
84.97+22.11
86.31+0.32
84.79+2.13
68.92+1.21
77.75=0.27
88.38+0.15
98.71=0.11
R-EDL
85.00+1.222
87.73++0.31
87.25
71.97:+0.99
79.64+2.36
90.09
98, 0.09
Re-EDL
89,94=1440
88.31=0.116
90.53+2204
13.4
80.83=1.72
90.13=0.25
98.81=0.05
Our VoE
48.96+8.900
6...45+1.73
69.64+-2.18
43.21++3.00
45.65+2.26
96.63+0.322
Our /w MI
84.28:12.60 60
86.73-10.40
86.35
68.61111.33
1.58
99.09-00088
Our /w DE
87,32:1 1.20
88.11-0.334
87.30
70.9111.58
18.64=1440
93.35-00.25
90.01
Our /w MPU
87.36
88.92--0.31
88.71+1.85
7..82.1.24
79.79 1.35
99.41 0.04
A (MPU vs VoL)
38.40
422.47
+19.06
+29.69
434.14
+2.78
Table 3: Comparison of uncertainty measures for OOD and misclassification detection CIFAR- 100, evaluated by AUPR and classification accuracy. PostNt indicates that the pre- tre-trained feature encoder frozen and only the density estimator is trained follow Kirichenko et (2020).
CIFAR
CIFAR DE
Method
MPU
DE
MI
VolE
MPU
MI
VoB
EDL
65.02=7.882
65.6317.92
65.7017.93
65.71117.93
69.42-4.41
69.5811462
69.6014.64
69.60.14.66
2-EDL
40.72215.22
18.10:28..22
42.71=16.18
42.26=15.72
18.42=15.26
53.00=11.76
51.50:10.72
10.29=10.57
R-EDL
64.91=5.76
65.3515.88
65.6911590
65.74.15.86
73.61=5.59
73.83:55.58
73.98:15.54
73.98.15.51
Re-EDL
65.20=6603
65.97=6.04
66.0826.03
66.09-16.03
72.97=4.20
73.14=4.41
73.15=24.43
73.15:24.44
PostN
58.62-6603
58.6245.94
26.8444.11
26.84+5.41
54.49-8.13
56.28-7.65
36.58+4.69
53.81+6688
PostN
45.84 2.58
49.25
25.72 1.44
4..1241.54
71.36 1.77
6..69
40.71
71.65 11.65-11.886 1.86
Our
60.46=6.85
57.76+88332
44.4148.48
49.74466.3
76.06-1699
7173-44.76
61.92.+491
56.05-43.38
CIFARI00
PPaaces65
CHFARI00
Method
MPU
DE
MI
VoB
MPU
DE
MI
VoB
EDL
45.45=1447
52
52551.252
45.52+1.52
68 68.54=2.19
68.87-2.46
68.89+2.51
68 89--2.51
Z.EDI.
31.16+ 10.22
28.34+-7.25
76-77-5
26.53-4-.37
46.95+ 19.68
42.06++14.00
38.90+-11.42
38.40 + 10.9
R-EDL
45.66=145
45.70=1444
15.71+11.46
45.71+1.46
70.15-083
70.54-0.82
70.76+0.78
70.77:10.76
ED.
43.88 -210
43.92+-2.13
43.9142,14
43.9142.14
69.99
70.46- 1.46
70.50+11.48
70.50+ 1.48
PostN
31.88=3.40
53.04+4.554
18.05+3.11
31.6443.79
47.55=5.98
19-4947.84
22.7944.45
46 46.85-45.82
PostN
38.41 1.23
39.5241.29
19.961-1.14
38.01 1:54
56.33 1.90
56.87 1.81
26.09 +..00
55.72441.75
Our
49.62 1.05
45.0111440
33.2710091
27.83.10.37
68.34_0.89
66.7512200
$5.40
44.72.1..27
CTFAR 100 -CIFARIO
CIFAR-100
Mis Detect
MPU
DE
MI
VoB
MPU
DE
MI
VoB
EDL
1..75.1.022
11.70.1.10
72.67.11.12
72.67.11.12
90.5610.32
90.48.10.33
10.33
2-EDL
60.29111U01
57.01
55.5426.58
55.29:1.33
60.29:11.01
57.0128.08
55.5416.588
55.29-16.33
R.EDL
Re-EDL
71.69=1.49
71.551163
71.5221.65
71.5111665
90.92-0.26
90.36:00.99
90.26:20.29
90.26:10.29
65.15
44.16.1..56
78.45.1..5
PostN!
68.07=1.87
68.27+1445
44.8711.65
67.4011.42
88,57=1.58
89.27=1.53
61,94:1.25
87,87:1.15
76.05=0.49
74.55+0.44 48
65.77+0.88
58.72+1.35
94.34-0.25
90.34=0.90
80.71+1.11
1..41+1.113
Entropy), MI (Mutual Information), and VoE (Vacuity of Evidence) on CIFAR-10 OOD detection and misclassification detection tasks. Several key observations emerge from this comparison.
(1) The combination of VoE with UCE loss and an exponential activation performs the worst overall, it violates the previously discussed the known overconfidence issue of UCE reported in EDL, which explains why subsequent EDL related studies have avoided this combination.
wwwniiiittiiooosistttttite principle. This finding corroborates
(2) MPU achieves the best performance when combined with UCE loss, consistently outperforming the other uncertainty measures. Among the alternatives, Differential Entropy, which also satisfies optimization consistency, provides the second-best results. This supports our claim that the effec- tiveness of uncertainty measure critically depends its consistency with the loss function.

---

Published as a conference paper at ICLR 2026
Smopy
Vasual Intoraaoo
Vacilty
Figure 4: t-SNE visualization of feature distributions for IDD (CIFAR-10) 10) and OOD (SVHN) samples, along with their predicted uncertainties under different measures.
(3) While not always top-ranked for OOD detection, UCE+MPU is highly competitive, often achiev- ing second-best performance. For in-distribution tasks, UCE with exponential activation delivers the highest accuracy, while MPU excels at misclassification detection. Notably, simply replacing VoE with MPU as the uncertainty measure boosts AUPR scores by 38.40, 22.47, 19.06, 26.69, and 34, 14 across OOD datasets.
UCE+MPU Becomes More Effective with Increasing Number of Classes. We further bench- evaluated different methods combined with various uncertainty measures on both OOD detection and misclas- sification detection. Our results show that UCE+MPU consistently achieves the best performance on certain OOD datasets, including GTSRB, Places365, and CIFAR-10, In addition, MPU delivers the strongest results in misclassification detection on CIFAR-100, underscoring its effectiveness in more challenging, large-class classification scenarios.
marked several state-of-the-art EDL methods on CIFAR-100) As shown in Table
Uncertainty Ranking Exists Only within UCE Loss. A consistent ranking uncertainty measures exists when training with UCE loss: MPU > DE >> MI > VoE. This ordering does not hold for other methods based on the Brier score loss (e.g. Z-EDL, R-EDL, and Re-EDI.). Therefore, previous works typically report the performance of their methods across all uncertainty measures, but they cannot determine which uncertainty measure is optimal for their models.
Visual Comparison of Uncertainty Measures. To further understand the behavior of different un- certainty measures, we visualize the 2D feature distributions of ID and OOD samples using t-SNE, along with their predicted uncertainties from model trained with UCE loss. As shown in Fig. 4 MPU assigns higher uncertainty to OOD and misclassified samples, with clearer separation com- pared to other measures. Among them, Differential Entropy. which satisfies the optimization sistency property, performs well, while Mutual Information and Vacuity of Evidence show smaller uncertainty differences between ID and OOD samples. This visualization further highlights MPU's effectiveness in enhancing discernibility, particularly for OOD and misclassified instances.
CONCLUSION
Summary. In this work, we revisited the reliability of uncertainty measures in evidential deep learning (EDL) from an optimization perspective. We argued that a desirable uncertainty measure should be consistent with the model's optimization dynamics, i.e., samples closer the global optimum should naturally exhibit lower uncertainty. Based on this principle, we proposed new uncertainty measure tailored for UCE loss and demonstrated its effectiveness through extensive experiments. Our findings highlight the fundamental link between uncertainty quantification and optimization, suggesting that designing uncertainty measures with optimization consistency in mind serve promising direction for future research in EDL and beyond.
Deficiencies and Future directions. Although our method yields strong results, it has notable limi- tations. On datasets with fewer classes (e.g., CIFAR-IO), 10), it underperforms Re-EDL trained with the Brier score, indicating that the UCE loss may not be universally optimal. This motivates exploring alternative losses such as the Brier score and examining how different losses and regularization terms (c.g., entropy minimization, KL divergence) jointly affect optimization and uncertainty estimation. Both theoretical and empirical studies in these directions valuable avenues for future work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS AND AUTHOR CONTRIBUTIONS
This work was supported by the National Natural Science Foundation of China (No. 62472315, 62476165).
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our research, provide comprehensive details of our methodol- ogy, experiments, and results. Detailed derivations and proofs for our theoretical claims are pro- vided in the Appendix: objective functions
presented in Appendix A uncertainty measures in Appendix B the optimal solution of the SVM dual problem Appendix [C. and the margin-aware property of UCE loss in Appendix C22 All code related to model training and feature visualization included the supplementary materials.
REFERENCES
Robert B Ash. Information theory. Courier Corporation, 2012.
Wentao Bao, Qi Yu, and Yu Kong. Evidential deep learning for open set action recognition, In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 13349-13358, 2021.
Peter L Bartlett, Michael Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-156 2006.
Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Pitfalls of epistemic uncertainty quan- tification through loss minimisation. Advances in Neural Information Processing Systems, 35: 29205-29216, 2022.
Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. On second-order scoring rules for epis- temic uncertainty quantification. In International Conference on Machine Learning, pp. 2078- 2091. PMLR. 2023.
Dimitri P Bertsekas. Nonlinear programming. Journal of the Operational Research Society, 48(3): 334-334, 1997.
Marin Biloš, Bertrand Charpentier, and Stephan Günnemann. Uncertainty on asynchronous time event prediction. Advances in Neural Information Processing Systems, 32, 2019.
Lukas Bossard, Matthieu Guillaumin, and Lue Van Gool. Food-101-mining discriminative compo- nents with random forests. In European conference on computer vision, pp. 446-461. Springer, 2014.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without ood samples via density- pseudo-counts. Advances in neural information processing systems, 33:1356-1366 2020.
Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, and Stephan Günnemann. Natural posterior network: Deep bayesian uncertainty for exponential family distributions. arXiv preprint arXhv:2105.04471, 2021.
Daniel Zügner Charpentier, Bertrand and Stephan Günnemann. Posterior network: Uncertainty esti- mation without ood samples via density- based pseudo-counts. counts. In Advances Neural Information Processing Systems 33. Curran Associates, Inc., 2020.
Mengyuan Chen, Junyu Gao, and Changsheng Xu. Uncertainty dual-evidential learning for weakly-supervised temporal action localization. IEEE transactions on pattern analysis and ma- chine intelligence,
2023a.
Mengyuan Chen, Junyu Gao, and Changsheng Xu. R-edl: Relaxing nonessential settings of idential deep learning. The Tweifth International Conference on Learning Representatiinss 2023b.
11

---

Published as a conference paper at ICLR 2026
Mengyuan Chen, Junyu Gao, and Changsheng Xu. Revisiting estial and nonessential settings of evidential deep learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597-1607. PmLR, 2020.
Koby Crammer and Yoram Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of machine learning research, 2(Dec):265-22) 2001.
Arthur P Dempster. A generalization of bayesian inference. Journal of the Royal Statistical Society:
Series (Methodological), 30(2):205 232, 1968.
Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ann Heng. Uncertainty estimation by fisher information-based evidential deep learning. In International Conference Machine Learning, pp. 7596-7616. PMLR, 2023.
Wei Fu, Yufei Chen, Yuqi Liu, and Xiaodong Yue. D-edl: Differential evidential deep learning for robust medical out-of-distribution detection. Medical Image Analysis, 109: 103888, 2026. ISSN 1361-8415. doi: http:///teetcceeeeeeeeeeeeeraeaa 5111111336611111545577777
103888. URL https: www.
sciencedirect. www.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation:
Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059.
PMLR, 2016.
Yarin Gal et al. Uncertainty in deep learning. 2016.
João Gama, Indrè Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A
survey concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
Junyu Gao, Mengyuan Chen, and Changsheng Xu. Learning probabilistic presence-absence evi- dence for weakly audio-visual event perception. IEEE Transactions Pattern Anal- ysis and Machine Intelligence, 2025.
Yonatan Geifman and Ran El- Yaniv. Selective classification for deep neural networks. Advances in neural information processing systems, 30, 2017.
Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view clas-
sification. In International Conference on Learning Representations, 2021. URL https:
//openzeview. wwt.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729- 2020.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprint 1903 2019.
Chih-Wei Hsu and Chih-Jen Lin. A comparison of methods for multiclass support vector machines. IEEE transactions on Neural Networks, 13(2):415-425, 2002.
Audun Jøsang. Subjective logic, volume 3. Springer, 2016.
Mira Juergens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Is epis- temic uncertainty faithfully represented by evidential deep learning methods? arXiv preprint arXiv:2402.09056. 2024.
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of-distribution data. Advances in neural information processing systems, 33:20578-2053 2020.
Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: large video database for human motion recognition. In 201/ International conference on computer vision, pp. 2556-2563. IEEE, 201
12

---

Published as a conference paper at ICLR 2026
TASET SHIFT IN MACHINE LEARNING. Dataset shift in machine learning, 2009.
Linye Li, Yufei Chen, and Xiaodong Yue. Vicinal label supervision for reliable aleatoric and epis- temic uncertainty estimation. In The Thirty-ninth Annual Conference Neural Information Pro- cessing Systems, 2025. URL https::/p///m. /tt.p::e//ppeeeview.
Tengyuan Liang, Tomaso Poggio, Alexander Rakhlin, and James Stokes. Fisher-rao metrie, ge- ometry, and complexity of neural networks. In The 22nd international conference on artificial intelligence and statisfics, pp. 888-896. PMLR, 2019.
Wei Liu, Xiaodong Yue, Yufei Chen, and Thierry Denoeux. Trusted multi-view deep learning with opinion aggregation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7585-7593, 2022.
Wei Liu, Yufei Chen. and Xiaodong Yue. Enhancing multi-view classification reliability with adap- tive rejection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 18969-18977, 2025a.
Wei Liu, Yufei Chen, and Xiaodong Yue. Enhancing testing-time robustness for trusted multi- view classification in the wild. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15508-15517, 2025b.
Wei Liu, Yufei Chen, Xiaodong Yue, Changqing Zhang, and Shaorong Xie. Enhancing reliability in medical image classification of imperfect views. IEEE Transactions Circuits and Systems for Video Technology, 2025c.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 212-220, 2017.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Ng, et al. Reading digits natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, 2011.
Deep Shankar Pandey and Qi Yu. Learn accumulate evidence from all training samples: theory and practice. In International Conference on Machine Learning, pp. 26963-2262200 26999. PMLR, 2023.
Jingen Qu, Yufei Chen, Xiaodong Yue, Wei Fu, and Qiguang Huang. Hyper-opinion evidential deep learning for disttii-ittiiution detection. Advances in Neural Information Processing Systems, 37:84645 84668. 2024.
Rahul Rahaman et al. Uncertainty quantification and deep ensembles. Advances in neural informa- lion processing systems, 34:20063-20075, 2021.
Bernhard Schölkopf. Support vector learning. PhD thesis, Oldenbourg München, Germany, 1997.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. Advances in neural information processing systems, 31, 2018.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: dataset 101 human actions classes from videos in the wild. arXiv preprint arXiv: 12/2.040 2012.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Bench- marking machine learning algorithms for traffic sign recognition. Neural networks, 32:323-332
2012.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5265-5274, 2018
13

---

Published as a conference paper at ICLR 2026
Lisa Wimmer, Yusuf Sale, Paul Hofman, Bernd Bischl, and Eyke Hüllermeier. Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? In Uncertainty in artificial intelligence, pp.
PMLR, 2023.
Xiaoyu Yang, Yufei Chen, Xiaodong Yue, Shaoxun Xu, and Chao Ma. T-distributed spherical feature representation for imbalanced classificatton.. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 10825-10833, 2023.
Xiaoyu Yang, Jie Lu, and En Yu. Adapting multi-modal large language model to concept drift from pre-training onwards. In The Thirteenth International Conference on Learning Representations, 2025a. UtLhttts: / http://pppneeeeeeeeeeeeee net/ http://wew..ntt/ooooddd/d2220000 forum?id=b220k22Gm.
Xiaoyu Yang, Jie Lu, and En Yu. Walking the tightrope:
Autonomous disentangling beneficial and detrimental drifts in non-stationary custom-tuning. In The Thirty-nimth Annual Conference Neural Information Processing Systems, 2025b. URLhttps: httttp:://openeviie...
S0-1BAiQmAFS
Xiaoyu Yang, Lijian Xu, Hongsheng Li, and Shaoting Zhang. One leaf reveals the season: Occlusion-baze contrastive learning with semantic-award aware views for efficient visual represen- tation. In Forty-second International Conference on Machine Learning, 2025c. URL https: //proceedings.u www.gn.......................... press (erres//v6)) v267 Sttrees//267//ag2255e2 /yaggnaang...... html
Taeseong Yoon and Heeyoung Kim.
Uncertainty estimation by density aware evidential deep learning. Forty-first International Conference on Machine Learning, 2024. URL https:
Chen Zhao, Dawei Du, Anthony Hoogs, and Christopher Funk. Open set action recognition via multi-label evidential learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22982 22991, 2023.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligene, 40(6):1452-1466 2017.
14

---

Published as a conference paper at ICLR 2026
THE USE OF LARGE LANGUAGE MODELS (LLMS)
We acknowledge the of Large Language Model (LLM) as ceeeral-purpost writing assistant in preparing this manuscript. Its role was confined to improving grammar and language, and rephrasing sentences for clarity. All content was authored, and all LLM suggestions were critically reviewed and approved, by the human authors, who retain full responsibility for the final manuscript.
A DERIVATION OF THE OBJECTIVE FUNCTIONS
We present step- step-by-ster step analytical derivations of the UCE loss, the Brier score loss, and the Type- III maximum likelihood loss. The derivation the Brier score loss follows directly from the EDL (Sen- soy et al. 2018). while the derivations of the UCE loss and the Type-Il maximum-likelinee loss are original to this work and are provided in full detail.
A.1 DERIVATION OF UNCERTAINTY-AWARE CROSS ENTROPY (UCE) LOSS
Consider K-class classification problem, where the model predicts a Dirichlet distribution Dir(p; (x) over the categorical probabilities p; (Pii...., ..k) AK-1, with parameters a; = (a.t...., a.k). The Uncertainty Cross-Entropy (UCE) loss defined as the expected cross-entropy loss under the Dirichlet distribution:
K Σ
Lucr( (0) = JPEEAK-1
Mi log(pi;)
Dir(p: (a;) dp:.
(10)
where yi is the one-hot label vector and the Dirichlet pdf
Dir(p:/at
1I
Pu ay-1
B(a.)
Il
Psj
ouy
dp:
(11)
B(a,)
AK-1
the multivariate Beta function. Differentiating B(x) with respect to ag and exchanging the order of differentiation and integration yields
(B(() dan
ll f=1
Pj
Ø5 log pk dp.
(12)
JA
Dividing both sides by B(a), obtain
Ep-D(a) [log(p)]] Log(Pa)) / JAK-1
1 0B(a)
log D(p ap
(13)
B(a))
don
Comparing Eq. with Eq. 12 we have
E Boy = v(ax) +SS).
(14)
Finally, the loss can be written as the Dirichlet expectation of the negative log- ligeliiklhhood::
(UC = --- Vis Dp-
(15)
j=1
which, upon substitution of the above identity, gives
(uCE(O)= Σ Mis (v((a)))) v'()))
Σ Mis (v(S)) +(aa))
(16)
15

---

Published as a conference paper at ICLR 2026
A.2 BRIER SCORE
The Brier Score loss takes the expected squares error loss between sampled probability p and label y follows:
CBiir (
/
11 -Pill
1
Pij
-1
dp:
D.EAK-1
B(xx)
j=1
K
Σ
E [v/ 2yssprs + +
Σ
(vlg
2y, Elpul + Elp
j=1
j=1
Using the identity [ = E Elpis] + Var(pis). we gete
K Σ
Crier (0)
((y)) 2 + Var(p.))
5=1
k
(yij asr/St) ait (S: - Oiis)
Σ
(17)
S + 1)
mm
k
Pag (1 - P(s) S: +1
Σ
(ui De,)2
j=1
Li is the bias term, which measures the squared difference between the predicted mean probability Piy and the true label Vıj. reflecting the accuracy of the prediction. L is the variance term, which captures the uncertainty of the prediction arising from the Dirichlet distribution over class probabilities, reflecting the model's confidence. Thus, the Brier Score loss naturally decomposes into error term and an uncertainty term, providing unified way jointly account for prediction accuracy and epistemic uncertainty in evidential deep learning.
A.3 TYPE II MAXIMUM LIKELIHOOD LOSS
The Type Maximum Likelihood (T2ML) loss, also known as the marginal likelihood loss, is derived by maximizing the likelihood of the observed labels y; under the predictive Dirichlet distri- bution parameterized by a, Intuitively, it encourages the model to assign higher Dirichlet concen- tration to the correct class, while naturally accounting for the uncertainty encoded in the distribution over class probabilities. Formally, the loss defined as
K /II
(I) log
II
Paj
dp:
Σ j=l
yis (log S; -.- llogarr)) (18)
B(x,)
j=1
Ignoring the sample index for simplicity, consider
[ DEAK
K II J=1
P3
2 dp.
(19)
B(a))
Since the integrand is proportional to a Dirichlet density with parameters a + y, we can rewrite
K PEAK-1 B(a)
B(a y)
P dp
My tgy
(20)
B(a)
j=1
Thus, the loss can be expressed as
B(a + y) B(a)
(T) = -llog
(21)
16

---

Published as a conference paper at ICLR 2026
Expanding the Beta functions in terms of Gamma functions gives
B(a) I I(as) B(a + y) I I(ay + Uj) I(S + 1)
(22)
I(S)
Because y is one-hot, if yk for the true class & and yj 0 for j * k, have
B(a + y) I(ax + +1) r(S) B(x)
(23)
I(ax) T(S + 1)
Using the Gamma identity [[= + z [(=), this simplifies to
B(a + y) ac B(a) S
(24)
Therefore,
(e) = log aK
log S log Ogg
(25)
S
Finally, reinstating the one-hot form of y, write
K
LTIML (+)
Vij
log S; log a:y) = log
log Oskk:
(26)
j=1
which completes the proof.
B DERIVATION OF THE UNCERTAINTY MEASURE
In this section, we provide detailed derivations of the uncertainty measures used in our work, includ- ing mutual information, differential entropy, and vacuity evidence. These measures constitute fundamental background concepts, and their formulations have been summarized in prior works such as PriorNet (Malinin Gales 2018), EDL (Sensoy et al., 2018) and R-EDL (Chen et al. 2023b). We include them here for completeness and to maintain self-contained presentation.
Mutual Information. A fundamental identity in information theory that the Shannon entropy of random variable X can be additively decomposed into the mutual information between X and Y, and the conditional entropy of given Y [Ash] 2012):
H(X) = I(X; H(X Y)
(27)
Follow this idea, Malinin Gales 2018) propose a method to explicitly model and decompose predictive total uncertainty into two components: data uncertainty (aleatoric uncertainty) and dis- tributional uncertainty (epistemic uncertainty). The total uncertainty in the prediction is measured by the Shannon entropy of the expected categorical distribution conditioned
H [p(lp)) Ep-Dic(a)ll(y p)]
log
Oj
(28)
5
Alcatoric uncertainty (or data uncertainty) corresponds to the expected entropy of the categorical distributions sampled from the Dirichlet prior, commonly referred as the conditional entropy
K
Ep-Dir() H[[p( p)ll Ep
Σ j=1
Pj log Pj
Σ
aj
(vla, + +1) - +SS + 1))
(29)
=
+(S+1)
Σ
a
(a, 11)
17

---

Published as a conference paper at ICLR 2026
where (-))) is the digamma function. For epistemic uncertainty, it is measured by the mutual infor- mation between predictions and the Dirichlet parameters as
I(y: p) = [Bp-Dir((>) [p(y|p)] (pnpD)i(
Ep-
p)]]-
(30)
This mutual information quantifies how much of the total uncertainty arises from uncertainty in the model parameters (i.e., distribution uncertainty. It is defined as
categorical distributions), and thus reflects epistemic
Ty.
H E Did(Daaallllii
R Dir(ax) [[(())P)
Epistemic Uncertainty
Total Uncertainty
Alcatoric Uncertainty
K
aj
In
aj
aj s
(vaa, + 1) + 1))
(31)
5
==1
==1
0 In ( Σ
vaa 1) + (5 (1+
Differential Entropy. Differential entropy is also a prevalent measure of epistemic uncertainty, where lower value indicates that the model yields sharper distribution, and a higher value means a more uniform Dirichlet distribution. It is defined as
ENi(Dir (p a)) PEAK-1
Dir(p log Dir(p | a) dp,
(32)
The closed-fore form expression is given by
K
ENT(Dir(p) a)) = log B(a) + (S- ((8--K)aa K))(S) - Σ (az 1))((a))
(33)
Vacuity of Evidence. EDL (Sensoy al. 2018 RED Pandey Yu 2023), I-EDL (Deng al.
2023). R-EDL (Chen et al. 2023b). and H-EDL Quu et al. 22224). which are grounded in Subjective Logic (ooanng 2016) and Dempster-Shafee Theory (Dempster 1968), represent uncertainty using evidence mass values. Subjective Logic (Jøsang 2016) provides principled framework for model- ing predictive uncertainty by interpreting the output of neural network as an opinion-a structured discrete set of classes. Unlike conventional classifiers that output
representation of uncertainty categorical probabilities, EDL models produce non-negative evidence values e = [é1, e ex] for each of the K classes. These evidence values parameterize Dirichlet distribution Dir(ax), where Oj = + 1. In Subjective Logic, an opinion over a finite domain is characterized by three components: the belief mass by. the base rate aj, and the uncertainty
u, satisfying:
K
by -u- a, Epp,, and
by +u=1
(34)
where Py denotes the probability assigned class j. These quantities relate to the Dirichlet param- eters as follows: The belief mass by is proportional to the evidence for class k:
K
by =
e s
where
Σ
(e,
+
aj j=1
Σ
(35)
j=1
The base rate d is typically assumed to be uniform, i.c., ag 1/K. The uncertainty mass is defined as:
K
K S
(36)
15=10 022
This uncertainty mass is referred to as vacuity of evidence in EDL literature, and it quantifies the degree of epistemic uncertainty due to lack of evidence. When the total evidence low (e.g., under out-of-distribution ambiguous inputs), S becomes small and approaches 1, indicating that the model abstains from committing belief to any specific class. Conversely, high total evidence yields low vacuity, reflecting confident predictions based on strong feature-basee based support. This opinion- based interpretation highlights the epistemic nature uncertainty in EDL and differentiates from aleatoric uncertainty captured by distributional spread in conventional probabilistic models.
18

---

Published as a conference paper at ICLR 2026
C PROOF OF THE MARGIN-BASED THEOREM AND PROPOSITION
C.I DERIVATION OF OPTIMAL SOLUTION FOR C&S MULTI-CLASS SVM
Definition 1. The primal optimization problem for the Crammer and Singer multi-class SVM
CCammer & Singer 2001) is defined as
K
N
min
8
w
W
2
(37)
j=1
1=1
S.t.
20 1/4 22 w; x; Bun 1-1 E., [1, E [1,
where 3 > 0 is a regularization constant, and N the number of training samples. The N slack variables GEIN) quantify the degree of classification error. å the Kronecker delta function 6p.9 the inequality constraints become & 2 O. For each sample 2;,
ifp= q else 0 otherwise. For E, measures the minimal adjustment needed for the model satisfy the classification constraint. If data point lies outside the margin or is correctly classified, then & = O: otherwise, & > O. indicating that the sample is either misclassified or within the incorrect classification margin. By minimizing the sum of these slack variables and weight norm, Crammer and Singer's K-class SVM seeks balance between accurate classification and model complexity. The primal optimization problem in Eq. 37 can be solved by applying the Karush-Kuhn-Bi Tucker (KKT) theorem (Bertsekas) 1997), which provides the necessary conditions for optimality constrained optimization problems. By introducing a set of dual variables 1, for the constraints, construct the Lagrangian function and derive the corresponding dual optimization problem:
max min L(w, E. n) w.
s.t. The 20 Vi N1, E [1
(38)
where
1 2
L(w, E, n)
Σ m
Wi
Σ
&
Th. [w 5; w. x. 53.5 + E.]
(39)
We can solve for optimal wy as function of 1 by finding the saddle point by differentiating wy and Es, require
0C
The = 0
(40)
0E,
Then, we get Ex Thi4 1, E [N]. Similarly, for wj, we require
K
ac
= Bwg
Mijji
Σ
Thig
æi
(41)
Ow;
1=1
1=1
J Wi + :3e =
(42)
Finally, the optimal form of wy becomes:
wi 8-1
Tijai
(43)
where y: 7); denotes the difference between the one-hot label distribution concentrating on the ground-truth label and the distribution y), obtained by the optimization problem. In Eq. 43 the optimal w, is expressed
linear combination of [tg[N], with weights given by Taj. Since for 2;, the constraints 7).1..... "/.K 2 O and Σ Mi.j = 1, Vi E [N] hold, the composition of wy can interpreted probabilistically. Specifically, x; considered support pattern if and only the correct label y:- Therefore, the classifiers w, for j E [] are constructed using patterns whose distribution 7 deviate from the true label information (i.e. whose labels uncertain (Crammer & Singer 2001)..
its corresponding distribution is not concentrated
19

---

Published as a conference paper at ICLR 2026
C.2 DERIVATION OF MARGIN-AWARI PROPERTY OF EDL LOSS
Before proceeding with the proof, we first introduce Lemma 1.
Lemma 1. For any b > 0, it holds that (/(b) */(a) 2 log(b) log(a).
Proof of Lemma 1. Define the auxiliary function k((x) = (xx) - (x) for x > To prove the
claim, it suffices show that h(b) h(() whenever a, which is equivalent showing that
h(x) is monotonically non-decreasing function. This can be verified by examining its derivative h())):
d de (v(()) - log(z)) (z)
h' (x())
(44)
dx
Here, (1) is the trigamma function. A standard integral representation of the trigamma function
(z) 00 te at
l- dt.
(45)
well-known inequality for the exponential function is 1 e- S for all t 2 0. Since both sides positive for t >> 0, taking reciprocals reverses the inequality:
(46)
c
Substituting this bound into the integral for (zxx) gives
te-
1
x
dt
e dt
(47)
0
x
Thus, (xxx) 2 which implies
1x(x) = (x()) 2 0.
(48)
Since its derivative is non-negative, the function h(x)) is monotonically non-decreasing.
Let y denote the ground-truth label. Since ay (as contains ay plus other positive terms), follows that h(S) 2 M(yy). That is,
((S) log(S) 2 vaay - log(a,).
(49)
Rearranging this inequality yields the desired lower bound for the loss function:
LUCE +) = (S) (aa)) 2 log((S) log(ay) log atyy
(50)
Theorem 1. The Margin-Aware Property of UCE Loss. It can be shown that the UCE loss LuCE can be lower-bounded in the following form:
LUCE (x. W. D 2 o
Σ jzY
w (x) - w. ((x)) = =
w. ( -MM
(51)
where (tt) = log (1 + (K 1) min (1, exp (t))), and the inequality achieves equality when 2j = Zyz for all j.J' * y. and M (x, yi W, v = E
jサy (w.)
(x)) w (z)).
Proof of Theorem 1. First, we transform the expression into has lower bound with the form log(S)- - log(a,) with Lemma 1, which is lower-bounded by the logarithm of the ratio S/ay. Now, let's analyze the argument of the logarithm, ay
2j=19 aj
Oy Ej Oj
Ejty αj
= 1+
(52)
Qg
Ogy
Oy
ay
20

---

Published as a conference paper at ICLR 2026
So our lower bound becomes:
( 2 log 11+ Ejst aj
(53)
We adopt the exponential activation function, i.e., o(2) exp(2) and 0j =exp(z)) + 1. Then, the ratio term becomes:
Ejty aj
Ejzy(exp(z)) + ( exp(2,)) + (K - 1)
(54)
ay
exp(zy) +1
exp(2y) +
We can rewrite Σ exp(2))
Σ exp((z) =
Σ
exp(((z) -yy + +zy = exp(zy)
Σ
exp(2)- -zy)
(55)
jØV
For convex function & (here, o(x) = )²), we have the Jensen's inequality that:
(xx)
(56)
m
Applying this to our sum over the non-target classes due the convexity of exponential function, we have
Σ
exp (z, -2y) 2 exp
K - zn))
E
(57)
K
JFY
Let's define the M = (ej - 2ny) The inequality becomes:
Σ
exp exp (zj zn) (K - exp
c)
M
(58)
K K-1
Now, substitute Eq. 58 back into Eq. 55
we have
M
Σ exp(22) exp(2y) 2y)
Σ
exp(2; - -2y 2 exp(2x)( - exp
exp
(59)
K K-1
j
jzy
Plugging this result back into Eq. 54
we have
Ejsy 03
(Eitv (xp(2))) + (K - 1)
exp (X cxp
K-1
+
+(K- 1)
2
ay
exp(2y) +1
exp( (2y)
+
(60)
exp(zy) exp
+1
k-1
(K-
exp(2 (pp(
+1
This expression still depends on By. To obtain bound of the form f(M), must find lower bound for this expression that is independent of zy. Let's introduce the term
exp(zy) exp
M
+1
K
g(ay: M)
(61)
exp(z,) +1
Let = exp(z1) and C = exp( M )- The expression is aC+1 a+1 The derivative with respect to the derivative positive, and the function increasing in (and thus 2y). Its minimum value is approached as + -cc, which 1. If M < O, then C < 1, the derivative is negative, and the function is decreasing in (and thus 2y). Its minimum value is approached zy - oo, which is C exp( M k=)). As Zy (i.e., - oc), the term approaches
C-1 If M O. then C (u+1))=
21

---

Published as a conference paper at ICLR 2026
CC C exp K-
As Zy + -oo (i.c., u - 0), the term approaches = 1. Combining these
two cases, can establish a uniform lower bound:
(zy exp
K- M +1
M
min exx
K-1
(62)
exp(22g 2yy +
Therefore, have lower bound for our ratio that depends only the margin M:
Ejty aj
(x=))
M
[K - 1) min
exp
(63)
K
Cty
Then, we get the final lower bound for the loss function:
LUCE( ( 2 log 1 + Σizy az
z log (1+ (K - min (1xpp
Eitules Bzy)
Cly
K -
(64)
Let (t)) = log (1 + (K- 1) min (1, (t)).. It is straightforward to verify that Φ(t) is non- decreasing function of by computing its first derivative. Therefore, we have rigorously shown that the UCE loss function LUCE( (@) admits lower bound of the form Φ EEjy(e zy)) where is non non-decreasing function. This implies that minimizing the UCE loss inherently promotes increasing the margin between the logit of the class and those of the incorrect classes, thereby aligning with a desirable property of classification losses.
C.3 GRADIENT ANALYSIS FOR THE CLASSIFIER VECTORS WITH UCE LOSS
Proof of Proposition We aim to show that when optimizing an EDL classifier W with the UCE loss, the gradient update contributed by each sample is analogous that in the dual problem of C&S SVMs, where it is expressed as a linear combination of the sample features and uncertain samples contribute more strongly the update
N Σ
WH, LUCE W, W
(8 biz) (xx)), bij
(65)
S,
The gradient descent update uses the negative gradient, -v.,, C. Suppose that the final gradient update is given by the accumulated contributions of all N samples, we have
OL; Sw,
- - doaj v (xx,)
V Ww Luck(ww)
Σ
Σ
Ozis
i=1
(66)
Douj
dais Ozgz
N
(-th (Si)
Jass Dzis
1h (ais)
Us (Si)
(xx; +
Σ i=
T (xx)
0zij
i=1,8=5
d. In (z) denotes the trigamma function, which is strictly decreasing and convex on (0), and ky. is the Kronecker delta. When 2 >>>>> 1, we have (1(()) s =. we assume that Oi,y; and S. > 1, this is typically easy achieve in practice, since we exponential activation function, and the Dirichlet concentration on the correct class is usually much larger than the prior of 1. Consider the two terms that contribute the classifier w,:
where (z) = 25
(1) The sum over samples with yi = jj:
Σ ド=
(Win (ar))
doaj Ozij
wbn (Si) daij
Ozig ((x.).
(67)
Substitute Caug/0aa ogj 1, we have
Σ
( 1) v (xx).
(68)
22

---

Published as a conference paper at ICLR 2026
Then apply the approximations (as 2/1 (air)
for aij > 1), we have
ass
2 (ais) (ass - 1) 2
Oūj -1 Oajj
v1((.) (aij 1) 2
Ozj
(69)
S
then
-b1 (a15 1) 1 (xx)
Σ
Oij 1
Cij -
v ((,).
(70)
aij
Éy
(2) The sum over samples with Vi + j:
Σ
(- (-(((5)) (St)
daus Dzys
)
(71)
Substituting do:j /Bzij Osi - and using the same approximation for t/1 (S:) 1),
Σ
(- - (5:) - 1)) ((x)) R
Oijj
(x.).
(72)
S
Combine the sums and write both cases in single sum over all First, observe that for the = j case the coefficient (after approximation) is
0uj -
03j -
aaj 1
(73)
Olig S
S.
with the assumption that oij > 1 for y/ j. And for the ys # case is
Okjj
(74)
S
Define the scalar biz by
1
bij
(75)
S
as in Subjective Logic (Jøsang 2016), which is the belief mass that supports the j-th singleton. Hence both contributions can written as the single sum for w
V my LUCE W.
(B - bis (x.))
(76)
C.4 PROOF OF OPTIMIZATION CONSISTENCY FOR DITFERNNTAL ENTROPY
Proposition 2. Optimization- Conessstonnnniiinnnnnyco of Differential Entropy. Let ENT(Dir(p) a)) denote the differential entropy of the Dirichlet distribution parameterized by a as
ENT(Dir(p) x)) = - pEAK-) Dir(p a)) log Dir(p ax) dp
(77)
log B(a) + - - Σ (ay- 1)v(az))
where B(a)) = x T(ay) Then the uncertainty measure defined by differential entropy is I(S)
optinizationn-coniitten with respect to the UCE loss in the sense of Theorem 2. That is, for any two training samples (æx, y) and (ar, "',),, we have
Lvcr(æ, y. W, v < Luer(a', V. W, D
ENT(Dir(p a(x))) < ENT(Dir(p a(x))))
(78)
Proof of Proposition 2. From Theorem
the UCE loss for sample (æ, y) satisfies
Lucr( v) 2 ob -M(2z))) M( Σ (zy 2z),
(79)
3#
23

---

Published as a conference paper at ICLR 2026
where 2j 3 L (az) and ((t) log (1 + (K min(),)xp(())) By the monotonicity of o, for any two samples (z. 3) and (æ', y'):
( v) LuCE(æ'' n/)
M( v) M(x',
(80)
Thus, a smaller UCE loss corresponds to a larger logit margin. Then, consider the differential entropy H(x) == ENT(Dir(p a)) with the parameterization aj = exp(zz) 1. Then the
gradient with respect to 2, for Vj E [K]
TH = 0H (aj - 1).
(81)
Dzj
daj
As
log B(a) = v[aa)) -55),
(82)
da;
0 [S k))(S) (SS) (S (S),
(83)
duy
0
[ (a: 1))((;;]] -v((,)) - (a, 1){2(aa))
(84)
dag
where thn (-) is the trigamma function, satisfying 1h (x) > 0 and strictly decreasing for x > 0. Then, have
OH da;
= (S (5))(((((5) (a. - 1){((a))
(85)
and
0H ((S K)) (ar - (a;) 1). 0zj
(86)
Since a, = cxx(22) + > 1, we have (az- - 1) > 0, and the sign of au determined by the expression:
0s,
A, = (S (S) (aj- 1) (aj).
(87)
Consider the function h(x) (I 1) (z) for > The function h(z) is monotonically increas- ing, and as x > co, h(x) > 1. We analyze the sign of Ay:
For the class y: When the margin M increases, zy increases, leading an increase in Oy- Since S= Ex=
=
j= aj and the margin increase typically implies ag dominates the sum,
we have ~ ay. Then:
A, ~ (a, K)Vi - (ay - 1a,, (1 - K (a,)(a))))))))))))))))))))))))) < 0,
(88)
since K 2 2 and eh (ag) > Therefore, Jzy H.
0, meaning that increasing my decreases
For incorrect classes * y: When the margin M increases, 2j decreases, leading to a; decreasing toward 1. As ay + 1, (aj - (ay) + 0, while (S K) K))1(S) > 0 (since S > K), so A₂ > 0. Therefore, 8t > 0. meaning that decreasing 2j decreases H. dz;
Thus, when the margin M increases (i.c., increases or zj decreases for j * y), the differential entropy H decreases. By Theorem 1, a smaller UCE loss implies larger margin, which in turn implies smaller differential entropy. Therefore:
LLCE(E, v, W, 4 Luce(a', v'. W, v)
M( 2 M(x',v')
(89)
H(a(=)) S H(a(x')),
which completes the proof of optimization consistency.
24

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison evaluated by classification accuracy and AUPR scores for mis- classification detection on CIFAR-10 10 and CIFAR-1O-C with different level of noise settings.
Method
CIFAR 10
CIFAR 10.C Level
I
MPU
DE
M1
Voli
MPU
DE
MI
EDL
88.484032
98. 74
98.74
98.74
98,74 98,74
84,644504
97.95=a11
97.95=a.1
97.94
97.94
Z-EDL
88.3840015
98. 12
98.7140.1
98.70=013
98.70+812
84.01=847
97,77=318
97.77=018
97.75+a18
97.75+018
R-EDI.
90.09
98. 98.98.am
20
98. 2n09
98 gos
86. 15233
98. 17.00s
98. 172005
98. 17
98. 17.00s
Re EDL
10
98. 98.76.00
98..7
98.69
18.81
86.34
17.931011
97.92..11
97.831004
1018
Our
93.41:0.44
99.43:am
39.34
99.12aue1
26.66
89.26:8.19
98.70=axu
98.59
98.32aa.
94.79a056
CIFAR-
Level 2
CIFAR-I00
Method
Acc
MPU
DE
MI
Voll
Acc
MPU
DE
MI
Vol
EDL
a0.47.1.m
96.79.017
96.791017
96.78.uux
96.78.aa17
76.08
95.35
95.35
95.33
95.33
I-EDL
79.294002
96.42+0.0
96.4240.0
96.38=nup
96.38+200
14
94.72+05:
94.72=05:
94.67=+0.51
94.66+0551
R-EDL
81,7340.5
96. 99
96.
96.99a0114
96.992014
96.99=114
77.24=052
95.51
95.5
+5.51
95.51=345
=5.51
Re-EDI.
81.95
96. 70
96.681907
a6657
96.55
1m
95. 19
17
95.03
25.01
Our
84.25
97,50
17.38
97.02
92.322,01
79.04_..5
96.03..27
95.88 joaz7
95.44]jo.1
89.67.10.8
CITAR-1O IO.C.O..C... Level 4
CIFAR
Method
Acc
MPU
DE
M1
VoE
Acc
MPU
DE
M1
Vol
EDI.
70.3410m
93.03,097
93.031097
13.01
62.07.114
88.961114
88.96.114
88.92.1
88.92.1
Z-EDL
69.07.1
92.30107e
92.30
92.23:104
92:231a24
60.40::116
87.75
87.75
87.65
87.64.:11
R-EDL
71.41+091
93.11=100
93.11+100
93.11=180
93.11=103
62.42=143
88 88.66+1.94
88.661104
88.66*194
88 38.66411.
Re-EDL
71.84
92, 92,944208
92.92aa85
92.77=058
92.74=031
63 1.2
89.03=1142
89.01=344
88, 48
88 80+1
72.88.aa1
93.89.a55
93.741057
93.20g001
86.13
63.89
89.62.1116
89.52a117
88.78:11
79.65.a1
D ADDITIONAL EXPERIMENTS
UCE MPU Achieves More Robust Performance Under Noisy Settings. Following prior work Charpentier et al.] (2020); [hen et al. (2023b); Deng et al. (2023). we evaluate our model under noisy data conditions. In particular, we follow Charppenier et al. (2020) and adopt the CIFAR- 10-C benchmark HHendrycks & Dietterich (2019), which contains 15 types of image corruptions (c.g., Gaussian noise, blur, brightness) each applied at 5 severity levels. Table reports the classifi- cation accuracy and misclassification detection performance CIFAR-10 and CIFAR-IO-C across all severity levels. As the corruption severity increases, classification accuracy inevitably degrades, making reliable misclassification detection especially critical. We observe that method con- sistently surpasses previous EDL-based approaches across all corruption levels, both in terms of classification accuracy and misclassification detection. Notably, our proposed MPU achieves the strongest misclassification detection performance among all competing methods.
Video-modality Verification. To evaluate our method in more challenging setting, we follow Bao et al.) (2021) and perform experiments on the video-based open-set action recognition task,
treating UCF-101 (Soomro et al. (012)) known classes and HMDB-51 (Kuehne et al. 201 I) as unknown. We report the Open maFi and Open Set AUC metrics as defined in Bao et al. (2021), and all baseline results are taken from the same source. As shown in Table our method (w/ MPU) achieves higher Open maF1 and AUC scores compared to prior approaches. In Fig. 5 we plot the average Open maFl scores under varying openness by gradually introducing HMDB-5T test samples as unknowns. The results show that our MPUJbased method consistently outperforms all baselines, and the performance gap remains large even as the openness level increases.
E DECISION BOUNDARY VISUALIZATION
We further compare the decision boundary learned by EDL with the UCE loss to those of C&S SVMs on three synthetic datasets from sklearn, namely two moons, concentric circles, and a linearly separable dataset. As shown in Fig. 9 we observe that for mostly linearly separable datasets, EDL exhibits strong alignment with the SVM. In contrast, for the nonlinearly separable concentric circles dataset, notable discrepancy is observed, reflecting the distinct behavior of EDL in capturing nonlinear decision boundaries. Moreover, the classification accuracy achieved by EDL with the UCE loss is comparable that of the C&S SVM across all three datasets.
25

---

Published as a conference paper at ICLR 2026
Table Performance of the video-modality ting using the 13D backbone on JCF-101, eval- uated with HMDB-51 as unkown.
MPU (Ours)
79.0
78.5
UCF-101 HMDB-5L
E 78.0
Method
Open maF1
AUC
77.5
OpenMax
67.85-10.12
74.34
77.0
MC Dropout
71.13:10.15
75.07
BNN SVI
71.57=0.17
74.66
76.5
SoftMax
73.19:10.17
75.68
76.0
RPL
71.48-t0.15
75.20
DEAR
77.24+-0.18
77.08
Openness (96)
100
Our (w/ VoE)
76.32:10.16
76,09
Our (w/ MI)
76.97+-0.18
76.86
Figure 5: Open macro-FI scores against vary- ing Openness with different uncertainty mea- sure when HMDB-51 as OOD dataset.
Our (w/ DE)
77.23+0.17
77.07
Our (w/ MPU)
78.31
77.67
(a) MPU
(b) Differential Entropy
(c) Mutual Information
(d) Vacuity Evidence
Figure 6: Comparison of different uncertainty measures a toy dataset trained with the UCE loss. For visual consistency across panels, all uncertainty measures, including the normalized MPU, use the same au colormap (red = low uncertainty, blue high uncertainty). MPU assigns the high- est uncertainty at the global center, where samples do not belong to any class, making well-suited for out-of-distribution detection. It also yields elevated uncertainty near class decision boundaries, which is beneficial for misclassification detection. Differential Entropy exhibits a similar trend, showing high uncertainty at the global center; however, it also produces undesirably high uncer- tainty within in-distribution regions. Mutual Information and Vacuity of Evidence behave similarly, showing high uncertainty only near the global center. This is expected, both reflect epistemic uncertainty, which becomes large when evidence for all classes is low.
FUTURE WORK
Several promising directions emerge from our optimization-contnn view of Evidential Deep Learn- ing (EDL). First, while our analysis focuses classification under stationary distributions, extend- ing the aptinization-consistenee principle to non-stationary environments is an important avenue. In practical deployments, models frequently encounter distribution shifts and concept drift (Gama al.) 2014 LEARNiNG 2009). Understanding how evidential objectives interact with drift adaptation and continual learning mechanisms (Yang al., 2025b a) may provide deeper insight into the sta- bility of predictive uncertainty. Second, our proposed Margin-aware Predictive Uncertainty (MPU) explicitly models class-wist evidence separation. Future work could explore tighter connections between evidential uncertainty and representation geometry, building on studies of feature norm dy- namics, angular margins, and hyperspherical embeddings (Liu al. 2017 Wang et al.] 2018 Yang et al.) 2023). Such perspectives may clarify how margin formation and evidence strength jointly govern predictive confidence. Third, recent advances in representation learning highlight the im- portance of view construction, invariance, and robustness to partial observations (Chen et al. 2020 He et al. 2020 Yang et al.) 2025c). Integrating these ideas with evidential training objectives may yield more stable and interpretable uncertainty behavior. Finally, minimization and maximum-maree solutions suggests broader opportunities at the intersection of margin theory, calibration, and rejection learning (Bartlett et al. 2006. Geifman & El- Yaniy 2017)
theoretical link between UCE
26

---

Published as a conference paper at ICLR 2026
(a) MPU
(b) Differential Entropy
(c) Mutual Information
(d) Vacuity of Evidence
Figure 7: Uncertainty measures on a toy dataset trained with the Brier score loss. First, we observe that the classifier vectors learned with the Brier score (green arrows) not tightly aligned with the C&S SVM solutions (red arrows), in contrast to the alignment observed under the UCE loss. This suggests that the implicit connection between EDL models and C&S SVM classifier geometry appears specifically under the UCE loss, rather than being a general property of all losses. addi- tion, MPU still exhibits its characteristic behavior: high uncertainty the global center, followed by elevated uncertainty along class decision boundaries. This indicates that MPU remains applicable models trained with the Brier score, although its effectiveness weaker than under the UCE loss, due to the absence of the margin aware structure that UCE naturally induces.
Figure 8: Predicted class probabilities of a logistic regression model on toy dataset. The leftmost plot shows the final classification and confidence (max probability), while the three plots the right show the individual probability surfaces for each class (0, 1, and 2).
Input data
Nearest Neighbors
Linear SSMM
BBF SMM
Gaussian Process.
EDL
Figure 9: Comparison of the decision boundaries of several classifiers, on three synthetic datasets (two moons, concentric circles, and linearly separable dataset), showing the distinct characteristies of each model's boundary.
Developing unified frameworks that jointly optimize margin, calibration, and abstention remains an exciting direction.
27


Reviews:
[{"review_id": "VPfkEbQ5WZ", "reviewer": "Reviewer_PQRx", "summary": "Evidential Deep Learning (EDL) is currently a promising approach for modeling predictive uncertainty in classification tasks. However, previous works on uncertainty estimation have largely overlooked the role of optimization dynamics.This paper first provides a clear and insightful analysis showing that the optimization objective of EDL implicitly encourages SVM‐like solutions.Based on this observation, the authors introduce the optimization‐consistency principle, which states that an uncertainty measure is valid if its value decreases as samples approach the global optimum of the training objective.Finally, motivated by this principle, the paper defines a new uncertainty metric termed Margin-aware Predictive Uncertainty (MPU). Extensive experiments demonstrate the effectiveness of this metric.", "questions": "Please refer to the Weakness part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper convincingly establishes the connection between EDL and SVM, supported by rigorous theoretical proofs and reasonable assumptions.\n2 The proposed Optimization-Consistency Property is intuitively sound and well aligned with optimization dynamics.", "weaknesses": "1. In terms of content organization, the authors devote a large portion of the paper to explaining the relationship between EDL and SVM, as well as the optimization-consistency property, but do not provide a sufficiently detailed discussion of the proposed uncertainty metric itself—its characteristics, advantages, and deeper connections to the preceding theory.\n2. The relationship between the Optimization-Consistency of Differential Entropy and the final MPU metric is not clearly explained.\n3. Neither the main text nor the experimental section specifies what training loss was used. Did the authors employ only the UCE loss? Was the proposed MPU-related loss incorporated into training?\n4. Some comparable works, such as I-EDL and R-EDL, include experiments under few-shot and noisy settings, but such evaluations are missing in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ED3nd2AzYZ", "reviewer": "Reviewer_N3WT", "summary": "This paper revisits Evidential Deep Learning (EDL) from an optimization perspective, revealing that minimizing the expected cross-entropy under a Dirichlet prior implicitly leads to margin-maximizing behavior similar to multi-class SVMs. Based on this insight, the authors propose an optimization-consistency principle, which defines a valid uncertainty measure as one that decreases when samples approach the training objective’s optimum. Guided by this principle, they introduce a new uncertainty metric—Margin-aware Predictive Uncertainty (MPU)—that explicitly reflects class separation. Experiments on out-of-distribution detection and classification-with-rejection tasks show that MPU achieves strong empirical performance, supporting the paper’s theoretical findings.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The paper presents strong theoretical originality by revisiting Evidential Deep Learning (EDL) from an optimization perspective and uncovering its intrinsic connection to the maximum-margin principle in Support Vector Machines (SVMs). This offers a fresh and insightful theoretical interpretation of EDL beyond its traditional probabilistic formulation.\n\n2. The introduction of the optimization-consistency principle provides a novel and principled criterion for evaluating and designing uncertainty measures. This concept effectively bridges the gap between optimization objectives and predictive uncertainty, contributing to a deeper understanding of uncertainty modeling.\n\n3. The proposed Margin-aware Predictive Uncertainty (MPU) metric is well-motivated and theoretically grounded. By explicitly capturing the margin between target and non-target evidence, it aligns uncertainty estimation with the underlying training dynamics.\n\n4. The experimental validation is comprehensive, covering both out-of-distribution detection and classification-with-rejection benchmarks. The results convincingly demonstrate the empirical benefits of aligning uncertainty estimation with optimization objectives.", "weaknesses": "1. Some theoretical analyses rely on relatively strong assumptions, such as convergence to or proximity to global optima, which may not always hold in practical deep learning scenarios.\n\n2. The experimental scope remains somewhat limited to classification tasks. Further evaluation in more complex or multimodal settings would strengthen the generality and applicability of the proposed framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B8QPq2HpwE", "reviewer": "Reviewer_WeXo", "summary": "The paper presents the finding that performing evidential deep learning on a cross entropy loss maximizes the decision margin, thereby enables a learning regime similar to a support vector machine. The paper introduces a lower bound to the cross-entropy variant of an evidential loss which has demonstrable margin maximization properties. The paper also identifies a monotonicity relationship between this loss and the uncertainty score learned by the evidential model. Linking these two outcomes, the paper arrives at the central claim that EDL works well because it learns uncertainty scores that are inversely proportional to the distance of the data points to the decision boundaries. Combined with the inherent max-margin property, it learns accurate predictors with reliable uncertainty estimates. The paper develops a new learning rule based on its theoretical findings. In particular, it suggests training EDL with the cross-entropy loss using exponent function for output activations and without post-hoc KL-based regularization. The paper demonstrates in comprehensive experiments that this learning rule can reach competitive accuracies and uncertainty quantification scores.", "questions": "* Is the monotonicity relationship suggested in Proposition 1 one sided only? Is the order between sample pairs preserved also in the backward direction from entropies to UCE losses?\n * Is the claim about the lack of an alignment between EDL and OvR in Figure 1 (b) a bit of an exaggeration? The two model still appear to work quite similarly in this problem and the uncertainties created by EDL are similar to the left panel.\n * Figure 2 appears to suggest that the uncertainty should increase as the sample moves from the center to, say, bottom left corner as it moves away from the observation and climbs in the loss landscape. However the in left panel it appears to decrease. Is this not against the main argument of the paper?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper's presentation is particularly clear. It illustrates key notions both as visuals as in Figures 1 and 2 and with simple and intuitive analytical results such as Proposition 1.\n * The theoretical findings prescribe a learning rule and this rule is shown to perform competitively on a diverse set of experiments. These experiments contain a comparison to a wide set of baselines that cover a representative set of earlier EDL variants as well as alternative approaches.\n * Section 5 links experiment results to concrete take-home messages, such as the existence of an uncertainty ranking within the studied UCE loss.\n * The paper identifies the limitations of the contribution openly and precisely, such its not being equally applicable to small class sizes.\n * Proposition 1 and the gradient analysis provided in Appendix C.3 are novel and they support the main claim of the paper appropriately.", "weaknesses": "* My main concern is that the results reported in Tables 2 and 3 do not demonstrate a concrete improvement in accuracy or uncertainty-based scores (e.g. MisDetect and AUPR). This weakens the claim that the knowledge sought about the max-margin character of EDL has a utility.\n * A large portion of the theoretical material is either prior work (whole Appendices A and B) or rather straightforward conclusions such as Lemma 1 of Appendix C.2. \n * I also have a few critical questions below which stops me from suggesting a clear accept at this stage, as they have a big impact on the main claim of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iLRrPgwxCi", "reviewer": "Reviewer_6f5n", "summary": "The paper derives a connection between evidential deep learning and multi-class support vector machines when the former considers an entropy-based objective.\nAdditionally, it proposes a new uncertainty measure and evaluates the proposal on a range of vision-based experiments.", "questions": "- Q1: How would the corresponding Figure 1 look for any other margin-based classifier, e.g., the baselines in this approach or a simple multi-class logistic regression, as well as for the remaining uncertainty measures (DE, MI, MPU)?\n- Q2: Do the authors have an intuition as to why PostN and NatPN, which optimize essentially the same objective, perform so much worse? Have all hyperparameters been tuned properly for each baseline?\n- Q3: Why are the UCE-based baselines omitted from Table 3? This seems to be a major omission, especially given the claim in l454 that UCE losses consistently induce an order in the uncertainty measures. This would require results showing that the same holds for these baselines.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "- The margin-aware predictive uncertainty (MPU) measure is relatively well motivated.\n- The approach provides improvements on some vision-based datasets.\n- An implementation is available.", "weaknesses": "- Section 2.1 claims that the UCE's similarity to SVMs is a \"unique property\" without further evidence. The proof simply shows that there exists a lower bound, not that such a bound cannot exist for other cases.\n- In general, the connection to SVMs remains rather vague, both in the theoretical contribution, which merely shows that one can interpret a bound in that way, and in the simple visual examples that show similar results. It also lacks a further step explaining why such a connection would be interesting and what can be learned from the fact that margin-based approaches look similar.\n- The experiments consist simply of training a regular EDL approach and evaluating different metrics.\n\n### Minor weaknesses\n- l036 \"classical Bayesian methods\": MC-Dropout can be interpreted as following a certain Bayesian approach, and ensembles are often used as a way of uncertainty quantification, but neither is a \"classical\" Bayesian method in the sense that a Bayesian neural network would be.\n- The citation system is broken throughout the paper; please use `citet` and `citep` correctly and consistently.\n\n\nIn summary, the paper shows some initial potential, but both the theory and the empirical results seem rather half-finished in their current state.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)