A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation
Authors: Dennis Thomas Ulmer
Venue: ICLR 2022 Submitted (2022)
Keywords: uncertainty estimation
prior networks
posterior networks
conjugate priors
classification
regression
evidential deep learning
dirichlet

Abstract: Popular approaches for quantifying predictive uncertainty in deep neural networks often involve a set of weights or models, for instance via ensembling or Monte Carlo Dropout. These techniques usually produce overhead by having to train multiple model instances or do not produce very diverse predictions. This survey aims to familiarize the reader with an alternative class of models based on the concept of Evidential Deep Learning: For unfamiliar data, they admit “what they don’t know” and fall back onto a prior belief. Furthermore, they allow uncertainty estimation in a single model and forward pass by parameterizing distributions over distributions. This survey recapitulates existing works, focusing on the implementation in a classification setting. Finally, we survey the application of the same paradigm to regression problems. We also provide a reflection on the strengths and weaknesses of the mentioned approaches compared to existing ones and provide the most central theoretical results in order to inform future research.

Under review as a conference paper at ICLR 2022
A SURVEY ON EVIDENTIAL DEEP LEARNING FOR SINSLE-PASS UNCERTAINTY ESTIMATION
Anonymous authors Paper under double-blind review
ABSTRACT
Popular approaches for quantifying predictive uncertainty deep neural networks often involve a set of weights models, for instance via ensembling or Monte Carlo Dropout. These techniques usually produce overhead by having train multiple model instances do not produce very diverse predictions. This sur- vey aims to familiarize the reader with an alternative class of models based on the concept of Evidential Deep Learning: For unfamiliar data, they admit "what they don't know" and fall back onto a prior belief. Furthermore, they allow un- certainty estimation in single model and forward pass by parameterizing distri- butions over distributions. This survey recapitulates existing works, focusing on the implementation in classification setting. Finally, survey the application of the same paradigm to regression problems. We also provide a reflection on the strengths and weaknesses of the mentioned approaches compared to existing ones and provide the most central theoretical results in order to inform future research.
INTRODUCTION
Many existing methods for uncer- tainty estimation leverage the cept of Bayesian Model Averaging, that approaches such Monte Carlo (MC) Dropout (Gal & Ghahramani 2016). bayes-by-backprop (Blundell it al. 2015) or ensembling (Lak- shminarayanan al.] 2017) can be
Prior
Posterior
Classification
Variational
Joo et al. (2018)
Requires OOD
Malnin Geles (2019)
Chen el (2018)
Nandy el al. (2020)
Malnnn Gales (2018)
Shen et al. (2020)
Sensoy at al. (2020)
Sernay et al. (2018)
grouped under (Wilson & Izmailov 2020). This involves the approxi- mation of an otherwise infeasible to compute integral using Monte Carlo samples for instance from an aux- iliary distribution or in the form of ensemble members. This implies the following problems: Firstly, the qual- ity the MC approximation depends on the veracity and diversity of samples from the weight posterior. Secondly, the approach often involves increasing the number of parameters in model or training more model instances altogether. Recently, new class of models has been proposed to side-step this conundrum by using a different factorization of the posterior predictive distribution. This allows compute uncertainty in a single forward pass and set of weights. Furthermore, these models are grounded in concept coined Evidential Deep Learning: For out-of-distribution (OOD) inputs, they fall back onto prior, often expressed as knowing what they don't 'I know.
(2019)
Knowledge Distillation
Flow-based
Telligkardds (2019)
Malrrin et al. (2020b)
Cherpentier
(2020)
Regression
Malinin et al. (2020s)
Charpentier et (2021)
Amini et al. (2020)
Figure 1: Taxonomy of surveyed approaches.
Our contributions are as follows: We summarize the existing literature and group these approaches, critically reflecting on their advantages and shortcomings alike, as well as how they fare compared to other, related methods. This survey aims to both serve an accessible introduction to this model family to the unfamiliar reader as well as an informative overview, in order to promote more applica- tions outside the uncertainty estimation literature. We also provide collection of the most important theoretical results for the Dirichlet distribution for Machine Learning, which plays a central role in many of the discussed approaches. We give overview over all discussed work in Figure T

---

Under review as a conference paper at ICLR 2022
2
BACKGROUND
(a) Ensemble
(b) Confident prediction
(c) Data uncertainty
(d) Model uncertainty
(e) Distributional uncertainty
(l) Representation gap (OOD)
Figure 2:
Examples of the probability simplex for K 3 classification problem, where every corner corresponds to class and every point to categorical distribution. Brighter colors correspond higher density. (a) Ensemble of discriminators. (b) (e) (Desired) Behavior of Dirichlet in different scenarios by Malinin & Gales (2018). (f) Representation gap by Nandy et al. (2020).
We first familiarize the reader with the necessary prerequisites for the rest of the survey, including Bayesian Model Averaging and the alternative approach of Evidential Deep Learning in Section 2.2 along with short introduction to the Dirichlet distribution in the next section.
2.1 THE DIRICHLET DISTRIBUTION
The Beta distribution commonly used prior for a Bernoulli likelihood, which be used to formulate binary classification problem. The Dirichlet distribution arises as a multivariate gener- alization of the Beta distribution for multi-class classification problem and is defined follows:
K
I I(ax), I(ao) ag
K
1
a)
B(a)
api
ag E
R
(I)
B(a)
where I'(-) denotes the gamma function, a generalization of the factorial the real numbers, K the number of categories or classes, and B(-) is called the beta function. For notational convenience, we also define K {1,.... K} as the set all classes. The distribution characterized by its concentration parameters the sum of which, often denoted as ap, is called the precision The distribution becomes relevant for applications using neural networks, considering that most modern networks for classification use softmax function after their last layer produce categorical distribution classes, for which the Dirichlet conjugate prior. The class probabilities can be expressed using vector μ [0, 1] s.t. ,4 = P(y k|z). Then, using Dirichlet prior for categorical likelihood, due to its conjugacy, produces a Dirichlet posterior with parameters (,, given data set D (2; ) of N observations with corresponding labels:
'The precision is analogous the precision of Gaussian, where larger og signifies sharper distribution.
2

---

Under review as a conference paper at ICLR 2022
N
K
K
plu ID, a) x (D)
1
at-1
Pk
Pk
((x)
k=1
(2)
K II
(E lx:=k) 1
K
K
Netog-l = Dir(1e
B(a))
Pk
B(a)
1k
k=1
where B is vector with 34 = 0% + Ng. with N₂ denoting the number of observations for class k and 1 being the indicator function. Intuitively, this implies that the prior belief encoded by the initial Dirichlet is updated using the actual data, sharpening the distribution for classes for which many instances have been observed. This distribution constitutes a distribution over categorical distributions over the K -1 probability simplex, multiple instances of which are shown in Figure 2 corner
Each point on the simplex corresponds to a categorical distribution, with the proximity indicating high probability for the corresponding class. [Figure 2a displays the predictions of an ensemble classifiers as point cloud on the simplex. Using Dirichlet, this finite set of distributions can be extended to continuous density over the whole simplex (Figures tb 21
2.2 PREDICTIVE UNCERTAINTY IN NEURAL NETWORKS
In probabilistic modelling, uncertainty is commonly divided into aleatoric and epistemic uncertainty (Der Kiureghian Ditlevsen) 2009: Hüllermeier Waegeman 2021). The former refers to the un- certainty that is induced by the data- generating process, and which e.g. might create unresolvable overlap in class distributions The latter describes the uncertainty about the optimal model parame- ters (or even hypothesis class), reducible with an increasing amount of data, less and less possible models become a plausible fit. These two notions resurface when formulating the posterior predic- tive distribution of classifter for new data point x
p(y1x) x) = / P(y x, p()
(3)
Aleatoric Epistemic
For large number of real-valued parameters 0 like in neural networks, this integral becomes in- tractable to evaluate, and thus is usually approximated using Monte Carlo samples with the afore- mentioned problems of potential computational overhead and approximation errors. Malinin & Gales (201)) thus propose to factorize Equation further::
/ʃ P p dddd P(y) p(u|x, 0) dp (D) 5(0
ply x)
Alcatoric Distributional
Epistemic
In the last step, we replace p(0 [T3) by a point estimate using the Dirac delta function, i.e. a sin- gle trained neural network, get rid of the intractable integral. Although another integral remains, retrieving the uncertainty from this predictive distribution actually has a closed-form analytical so- lution for the Dirichlet (see [Section 3.2p. The advantage of this approach further that it allows us to differentiate uncertainty about data point because it lies in a region of considerable class overlap (Figure 2c) from it differing from the training distribution entirely (Figure 2e). Assuming that a point estimate of the parameters suffices prevents one from estimating epistemic uncertainty like earlier works, as discussed in the next section. However, there are works like (Haussmann et al. (2019); Zhao et al. (2020) that combine both approaches.
UUnless additional features are added. 2 Nooe that the predictive distribution Eqquatio (3) recovers the common case for single network predic- tion where P(y) x, 0) xs P(y] Ø). Mathematically. this is expressed hy replacing the posterior p(O (D) by
delta distribution like in Equation (4) where all probability density single parameter configuration.
3

---

Under review as a conference paper at ICLR 2022
3 DIRICHLET NETWORKS
We will show in Section 3.1 how neural networks can parameterize Dirichlet distributions, while Section 3.2 reveals revels how such parameterization can be exploited for efficient uncertainty estimation. The remaining sections enumerate different examples from the literature parameterizing either a prior (Section 3.3.1p posterior Dirichlet distribution (Section 3.3.2) according to Eqqations (1)
and
3.1 PARAMETERIZATION
For classification problem with K classes, neural classifier is usually realized function fe : RD - R* mapping to logits for each class given an input x RD. Followed up by softmax function, this then defines categorical distribution over classes with a vector fe A4 ply k| x, 0). The same architecture can be used without any major modification to in- stead parameterize Dirichlet distribution, as in Equation (1) In order classify a data point x, a categorical distribution is created from the predicted concentration parameters of the Dirichlet as follows (this definition arises from the expected value, see Appendix A.I
Okk Or
a = [o());
9 arg max PI, Pl PK KEK
(5)
As discussed in Section 3.3.2 this process is very similar when parameterizing Dirichlet posterior distribution, except that in this case, a term corresponding to the class observation in inlEquation (2) added to every concentration parameter well.
3.2 UNCERTAINTY ESTIMATION WITH DIRICHLET NETWORKS
Let us now turn our attention on how to estimate the different notions of uncertainty laid out in Sec- tion 2.2 within the Dirichlet framework. Although stated for the prior parameters a, the following methods can also be applied to the posterior parameters ,3 as well without loss of generality.
Data (aleatoric) uncertainty. For the data uncertainty. the data distribution p(yl (4) (similar to previous works like e.g. Gal & Ghahramani 2016). As the entropy captures the "peakiness" of the output distribution, lower entropy indicates that the model is concentrating all probability mass on single class, while high entropy stands for a more uniform distribution the model thus is undecided about the right prediction. For Dirichlet networks, this quantity has closed-form solution (for the full derivation, refer to Appendix B.I.:
evaluate the expected entropy of
K
E p(x xx0)
H ((y) = Σ ask
vbaa + +1 vla + +1)
(6)
where si denotes the digamma function, defined as (xxx
d log ((x), and the Shannon entropy. dir
Model (epistemic) uncertainty. As saw in Section 2.2 computing the model uncertainty in
the classical sense via the weight posterior p(0 (D) like in Blundell et al. (2015); 5) Gal Ghahramani (2016); Smith & Gal (0018) is usually not done in the Dirichlet framework (with exceptions such
as Haussmann et al.) 2019; Zhao et al. 2020). Nevertheless, the defining property of Dirichlet networks is that epistemic uncertainty expressed in the spread of the Dirichlet distribution (for instance in FFiure 2 (d) and (e)). Therefore, the epistemic uncertainty can be quantified considering the concentration parameters that shape this very same distribution: Charpentier et al. (2020) simply consider the maximum 0g as score akin to the maximum probability score by Hendrycss & Gimpel while /ensoy al. (2018) compute it by K/ >/>x (as, + 1) or simply ar (Charpentier ct al. 2020). In both cases, the underlying intuition is that larger ag produce a sharper density, and thus indicate increased confidence in prediction.
'The only thing to note here is that the every as has to be strictly positive, which can for instance be enforced by using additional ReLU function (and adding small value, e.g. like in /Sensyy al. 2020) the output or predicting log og instead (Sensoy et al. 2018 Malimin & Gales 2018).
4

---

Under review as a conference paper at ICLR 2022
Distributional uncertainty. Another appealing property of this model family is to distinguish uncertainty due to model underspecification (Figure 2d from uncertainty due to alien inputs (Fig-) ure 2ch. In the Dirichlet framework, the distributional uncertainty difference between the total amount of uncertainty and the data uncertainty, which can be expressed in terms of the mutual information between the label y and its categorical distribution p.:.
be deduced by computing the
[[ (y.4 x, ID] H
EpCH X.D) Plu mb]
Epcu x.D) H (P(l
Total Uncertainty
Data Uncertainty
Given that [| Appendix A.1 1 and assuming the point estimate p( x. D) p(u x 0)
to be sufficient (Malinin & Gales 2018),
obtain expression very similar to Equation (6)
log
v(ax + +1 (aa0 +1)
3.3 EXISTING APPROACHES
The properties we discussed in previous sections are desirable traits, as they simplify the process of obtaining different uncertainty scores. However, it is important to note that the behaviors of the Dirichlet distributions in Figure aar idealized. In the empirical risk minimization framework that neural networks are usually trained in, Dirichlet networks are not incentivized to behave in the depicted way per Thus, when comparing existing approaches for parameterizing Dirichlet priors (Section 3.3.1) and posteriors (Section 3.3.2) we mainly focus on the different ways that authors try tackle this problem by means of loss functions and training procedures. Due to spatial constraints, we refrain to present all the different ideas in detail and instead only highlight some of them, summarizing the rest in informal manner We give overview over the discussed works in Tables 2nd@in in the respective sections.
3.3.1
PRIOR NETWORKS
The key challenge in training Dirichlet networks comes in the form of ensuring both high classi- fication performance and the intended behavior under foreign data inputs. For this reason, most discussed works follow a loss function design using two parts: One optimizing for task accuracy for the former goal, the other one for flat Dirichlet distribution for the latter.
As their main objective, Tiligkaridss (2019) derive a generalized lp loss (see Appendix B.3). but using local approximation of the Rényi divergence w.r.t uniform Dirichlet for the regularization term in order to ensure higher uncertainties for misclassified examples [ Zhao et al. (2020) similarly use le loss in the context of Graph Neural Networks (GNNs), but adapt Kullback-Leibler (KL) regularization term to incorporate information about the local graph structure instead of referring uniform prior, as well as knowledge distillation loss. Haussmann al. (2019) optimize the
model using a negative log- ligeliiklihhood (NLL) loss and derive a regularizer from PAC bounds.
Instead of enforeing the flatness of the Dirichlet by itself, Malinin & Gales (2018) instead explicitly maximize the KI. divergence to a uniform Dirichlet on OOD data points. Further, they instead utilize another KI. term to train the model on predicting the correct label instead of 1p norm. However, as the KI. divergence is not symmetrical, Malinin & Gales (2019) argue that the reverse counterparts of both loss terms actually have more appealing properties in producing the correct behavior of the predicted distribution (see Appendix B.5). Nandy et al. (2020) refine this idea further, stating that
"Even though the term prior and posterior network have been coined by Malinin & Gales (2018) and Charpentier et al. (2020) their respective approaches. we use them in the following as an umbrella term for all methods targeting a prior or posterior Dirichlet.
'For more details, refer the reader to Appendix A for general derivations concerning the Dirichlet distri- bution. dedicate Appenddx B to more extensive derivations of the different loss functions and regularizers and give detailed overview over their mathematical forms in Appenddix C
Thhe Dullback-L-eibler divergence can he seen special case of the Rényi divergence (van Erven & Harremoës 2014), where the latter has stronger information-thrente underpinning.
5

---

Under review as a conference paper at ICLR 2022
Table 1: Overview over prior networks for classification. (x) OOD samples were created via tem- perature scaling inspired by Liang et al. (2018). ID: In-distribution.
Method
function
Architecture
Requires 0OD samples?
Prior network MMalinin & Gales 2018
ID KL wat smoothed label & OOD KL w.r.t uniform prior Reverse KL of Malinin & Gales 2018
MLP / CNN
Prior networks Malinin & Gales 2019
CNN
Information Robust Dirichlet Networks Tsiligkaridis 2019
l, norm w.n. one-hot label & Approx. Rényi divergence w.r.. uniform prior
CNN
Dirichlet via Function Decomposition (Bilos et al 2019
Uncertainty Cross-entropy & mean variance regularizer
RNN
Prior network with PAC Regularization
Negative oogliikelhood loss PAC regularizer
BNN
(Haussmam 2019
Ensemble Distribution Distillation
Knowledge distillation objective
/INN/CNN
Malinin et al.
2020b
Prior networks with representation gap
TD OOD Cross-entropy precision regularizer
/LP/
Nandy et al.
2020
Prior RNN Shen et al. 2020
Cross-entropy + entropy regularizer
RNN
W
Graph-based Kemel Dirichlet distribution estimation (GKDE) Zhao 2020
la w.r.t one-hot label
GNN
KL reg. with node-level distance prior & Knowledge distillation objective
even in this framework high epistemic and high distributional uncertainty (Figures 2dd 2e) might be confused, and instead propose novel loss functions producing a representation gap (Figure 21 check Appendix forr the final form), which aims to be more easily distinguishable. Lastly, Malinin al. (2020b) show that prior networks can also be distilled using an ensemble of classifiers and their predicted categorical distributions (akin to learning Figure 2e from Figure 2a), which does not require regularization at all (but training the ensemble).
An application to Natural Language Processing can be found in the work of Shen et al. (2020), who train their recurrent neural network for spoken language understanding using a simple cross-entropy loss and entropy regularizer. However, [Biloš al. (2019), who apply their model to asynchronous event classification, note that the standard cross-entropy loss only involves a point estimate of a cate- gorical distribution, discarding all the information contained in the predicted Dirichlet. For this rea- son, they propose an uncertainty- aware cross- cooss-entropy (UCE) loss instead, which has a closed-form solution in the Dirichlet case (see Appendix B.6 They further regularize the mean and variance for OOD data points using an extra loss term.
3.3.2 POSTERIOR NETWORKS
Table Overview over posterior networks for classification. OOD samples were created via (f) the fast-sign gradient method (Kurakin et al.) or (3) using a Variational Auto-Encoder (VAE; Kingma Welling 2014
NLL: Negative logelikeli CE: Cross-entropy.
Method
Loss function
Architecture
Requires ODD samples?
Fvidential Deep Leaming (Sensoy et al. 2018
lg norm tn one-hot label
CNN
X
KL w.et. uniform prior
Variational Dirichler (Chen et al. 2018
ELBO + Contrastive Adversarial
CNN
(V))
Belief Matching Joo
2020
ELBO
CNN
X
Posterior networks Charpentier et al. 2020
Uncertainty CE (Bilok al 2019 2019 Entropy regularizer
MLP CNN
X
Norn. Flow
Generative Evidential Neural Networks (Sensoy et al. 2020
Contrastive NLL + KL between
CNN
(
uniform Dirichlet of wrong classes

---

Under review as a conference paper at ICLR 2022
choosing Dirichlet prior, due to its conjugacy to the categorical distribution, induces a Dirichlet posterior distribution. Like the prior in the previous section, this posterior can be parameterized by neural network. The challenges hereby two-fold: Account- ing for the number of class observations N₁ that make up part of the posterior density parameters G (Equation (2)). and, similarly prior networks, ensuring the wanted behavior on the probability simplex for in- and out-of-distribttion inputs.
As elaborated on in Section 2.1,
Sensoy et al. (2018) base their approach on the Dempster-Shafer theory of evidence (Yager & Liu 2008 lending its name to the term "Evidential Deep Learning") and its formalization via subjective logic (Audun, 2018). In doing so, an agnostic belief in form of a uniform Dirichlet prior is updated using the aforementioned (pseudo-Jcounts, which are predicted by neural network. [Sensoy et al. (2018) train their model using a straightforward l2 loss between the predicted Dirichlet and the one-hot encoded class label (Appendix B.4), as well as a regularization term consisting of the KI. divergence w.r.t. a uniform Dirichlet:
T(K)
K
[L(p(ula)) p( 1)] log ((a)
Σ (ax ()(((ax) (bao))
In follow-up Sensoy et al.
2020 train
N1o.1
.1to
similar model using contrastive loss with arti- ficial OOD samples from Variational Autoen- coder (Kingma & Welling 2014), and a KL- based regularizer similar to that of Tiiligkariis (2019). Charpentier et al. (2020) also set a to uniform prior, but obtain class observations N₂ from the training set and scale them by the probability of input's latent representation under a normalizing flow SuFF:Reeende & Mo- hamed, 2015) with parameters Φ and one flow instance per class (see Figure 3):
Normalizing Flow p(ale:0)
Is(x!!t
1s(xc
gPPt pPeeMlp
Input
Latent space
Prediction
Figure 3: Schematic of posterior network, taken from Charpentier et al. (2020). An encoder fe maps inputs to a latent representation z. NFss then model class-condiiional densities, which are used together with the prior concentration to produce the posterior parameters.
8x az + No p(z ly k: b); z fo(x
This has the advantage of producing low prob- abilitics for strange inputs like the noise in Fig- ure 3] which in turn translate to low concentra- tion parameters of the posterior Dirichlet, as it falls back onto the uniform prior. The model is then optimized using the same uncertainty-aware cross-entropy loss as in Biloš et al. (2019) with an additional entropy regularizer.
Another route lies in directly parameterizing the posterior parameters B. Because it is infeasible to model the posterior this way due to an intractable integral, this leaves us to instead model an approximate posterior using variational inference methods, which is exactly the approach of Joo et al.] (2020) and [Cen et al. (2018). As the KL divergence between the true and approximate posterior is infeasible to estimate as well, the variational methods usually optimizes the evidence lower bound (ELBO) instead. For the Dirichlet family, the ELBO has an analytical solution (we refer the reader to Appendix A.3 for derivation of the expression):
B(B)
K
LELBO xb3,)=(33))))))))))))())))) - log B(r)
((6))
'A NF a deep generative model, estimating a density the feature space by mapping a simple Gaus- sian in a latent space by a series of invertible, bijective transformations. The probability of an input can then be estimated by its latent encoding w.r.t. the simple Gaussian and the change-of variaabe formula, traversing the flow in reverse. Instead of mapping from the feature space into latent space, the flows in Charpentier et al.) (202)) map from the encoder latent space into separate, second latent space.

---

Under review as a conference paper at ICLR 2022
4 EVIDENTIAL DEEP LEARNING FOR REGRESSION
Table 3: Overview over Evidential Deep Learning methods for regression.
Mcthod
Parameterized distribution
Loss function
Model
Deep Evidential
Normal-Inverse Gamma Prior
Negative log-likelihood loss KL W.r.t. uniform prior
MLP/ CNN
Regression
(Amini et 2020
Regression Prior Network
Normal-Wishart Prior
Reverse KL
MLP/ MNP/CNN
MMlinin al. 2020a]
Malinin & Gales, 2019
Natural Posterior Network CCarpentier al. 2021
Inverse-x Posterior
Uncertainty Cross-entrupy (Bioos et al. 2019
MLP/ CNN Norm. Flow
Entropy regularizer
Because the Evidential Deep Learning framework provides such appealing properties, the question naturally arises of whether it can be extended to regression problems as well. The answer is yes, although the Dirichlet distribution is not an appropriate choice in this case. It is very common to model a regression problem using a normal likelihood (Bishop) 2006). As such, there are multiple potential choices for a prior distribution. The methods listed in Table either choose the Normal- Inverse Gamma distribution (Amini et al. Charpenner et al. [2021) inducing a scaled inverse- x² posterior (Gelman et al. 1995) as well as a Normal Wishart prior (Malnnin et al. 2020a). We
will discuss these approaches in turn.
Amini et al. (2020) model the regression problem as a normal distribution with unknown mean and variance N (y; (u, a²), and as such use a normal prior for the mean with μ NN N(r,oáv o²x-1) and an inverse Gamma prior for the variance with a² ~ (a, resulting in a combined Inverse- Gamma prior with parameters 7, v, a. 33. These are then predicted by different "heads" of neural network. Aleatoric and epistemic uncertainty can then be estimated using the expected value of the variance well the variance of the mean, respectively, which have closed form solutions under this parameterization. The model is optimized using a negative lig-likel objective along with an evidence regularizer, akin to the entropy one for Dirichlet networks. In the work of Charpentier ee al. (202T), the authors generalize the approach behind the posterior networks by Charpentier et al. (2020) to different distributions from the exponential family, keeping architecture and loss function the same. Depending on the distributions used however, the UCE loss by Bilos et (2019) takes on a different form. Malinin et al. (2020a) can be seen as the multivariate generalization of the work of Amini et al. (2020), where a combined Normal- -ishart prior is formed to fit the now multivariate normal likelihood. Again, the prior parameters are the output of a neural network, and uncertainty be quanitified in a similar way. For training purposes, they apply the reverse KL objective of Malinin & Gales (2019) as well as the knowledge distillation objective of Malinin et al. (2020b).
RELATED WORK
The need for the quantification of uncertainty in order to earn the trust of end-users and stakeholders has been a key driver for research (Bhatt et al.) (202T). Unfortunately, standard neural discriminator architectures have been proven to possess unwanted theoretical properties w.r.t. to OOD inputs [0] (Hein et al.) 2019; Ulmer & Cinà 2020) and lacking calibration in practice (Guo et al., 2017).
popular way to overcome these blemishes is by quantifying (epistemic) uncertainty by aggregating multiple predictions by networks in the Bayesian Model Averaging framework (Jeffreys] 1998 Wil-
son & Izmailov] 2020), Laplace approximations (Kristiadi et al. [000 Daxberger et al. (0221) vari-
ational methods (Gal & Ghahramani] 2016) Blundell et al.) 2015), ensembling (Lakshminarayynan
al.) (2017) or mixtures of the latter two (Pearce al. 2020 Wilson & Izmailov 2020). Nev-
ertheless, many of these methods have been shown not to produce diverse predictions (Wilson &
"The form of the Normal-Inverse Gamma posterior and the Normal Inverse-y posterior are interchangable using some parameter substitutions (Murphy| (207).
"eaaree et al. (2021) argue that some insights might partially be mislead by low-dimensions intuitions, and that empirically OOD data in higher dimensions tend to be mapped into regions of higher uncertainty.
8

---

Under review as a conference paper at ICLR 2022
Fort et al., 2019) and to deliver subpar performance and potentially misleading un- certainty estimates under distributional shift (Ovadia et al. 2019; Masegosa
Izmailov 2020
2020 Wenzel et al.
2020 Izmailov et al. 2021 a02lape b), raising doubts about their efficacy.
The methods in Section 3.3 and Section cca be seen as single-pass alternatives that avoid approxi- mating the predictive distribution in Eqqution (3) Monte Carlo estimates. The proposed Posterior Network (Charpentier et al.) 2020) 2021) can furthermore be seen as related to another, competing approach, namely the combination of neural discriminators with density estimation methods, for instance in the form of energy- models (Grathwohl et al. Elflein et al. 2021) or other hybrid architectures (Lee al.
2018 Mukhoti al. 2021).
Some of the discussed models have already found a variety of applications, such as in autonomous
driving (Capellier al. 2019; Liu et al. [201]] Petek et al. 2021). medical screening (Ghesu al.)
2019
Gu et al. 2021), molecular analysis (Soleimany et al. 2021). and open set recognition (Bao
et al. 2021).
6
DISCUSSION
Despite their advantages, the last chapters have highlighted key weaknesses of Dirichlet networks as well: In order to achieve the right behavior of the distribution and thus guarantee sensible uncertainty estimates, some approaches Malinin & Gales (2018) 2019); Nandy et al. (2020); Malinin et al. (2020a) require out-of -iito---dttribution data points during training. This comes with two problems: Such data is often not available or in the first place, or cannot guarantee robustness against other kinds of unseen OOD data, of which infinite types exist in real-valued feature space Indeed, Kopetzki eet al. (202T) found OOD detection to deteriorate across a family of Dirichlet- based models under adversarial perturbation and OOD data points. One possible explanation for this behavior might lie in the insight that neural networks trained in the empirical risk minimization framework might learn spurious but highly predictive features (Ilyas al., 2019 Nagarajan al., (021). This way, inputs stemming from the training distribution might be mapped to similar parts of the latent space as data points outside the distribution even though they have (from a human perspective) blatant semantic differences, simply because these semantic features were not useful to optimize for the training objective. This can result ID and OOD points having assigned similar feature representations by network, phenomenon has been coined "feature collapse" (Nalisnick et al. 2019: van Amersfoort ct al. 2021 Havtorn et al. 202T). One strategy to mitigate (but not solve) this issue has been to enforce constraint
the smoothness of the neural network function (Wei et al., 2018 van Amersfoort et al.] 2020; [021]] Liu et al., (2020), thereby maintaining both a sensitivity to semantic changes in the input and robustness against adversarial inputs (Yu al., 2019). Nevertheless, this question remains an open area of research and the impact on Evidential Deep Learning methods
underexplored.
7 CONCLUSION
This survey has given an overview over contemporary approaches for uncertainty estimation using neural networks to parameterize conjugate priors or the corresponding posteriors instead of likeli- hoods, with focus on the Dirichlet distribution appealing theoretical properties allowing for uncertainty estimation with minimal computational overhead, rendering them as a viable alternative to existing strategies. We also emphasized practical problems: In order to nudge models towards the desired behavior in the face of unseen or out-of- distribution samples, the design of the model architecture and loss function have to be carefully considered. At the moment, the entropy regularizer seems to be sensible choice in prior networks when OOD data is not available. Combining discriminators with generative models like normaliz- ing flows like in (Charpentier et al.) 2020 202T), embedded in a sturdy Bayesian framework, also appears as an exciting direction for practical applications. In summary, we believe that recent ad- vances show promising results for Evidential Deep Learning, making viable option in the realm of uncertainty estimation to improve safety and trustworthiness in Machine Learning systems.
classification context. We highlighted their
The same applies the artificial OOD data in Chen al. 2018 Shen al. 2020 g Sensoy al. 2020

---

Under review as a conference paper at ICLR 2022
REFERENCES
Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Balcan, and Hsuan- -ienn
Jsang Audun. Subjective Logic: A formalism for reasoning under uncertainty. Springer, 2018.
Wentao Bao, Qi Yu, and Yu Kong. Evidential deep learning for open set action recognition, In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2021.
Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Gauthier Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nach- man, Rumi Chunara. Madhulika Srikumar, Adrian Weller, and Alice Xiang. Uncertainty as form of transparency:
Measuring, communicating, and using uncertainty. In Marion Fourcade, Benjamin Kuipers, Seth Lazar, and Deirdre K. Mulligan (eds.), AIES' '21: AAAI/ACM Conference on AI, Ethics, and Society, Virtual Event, USA, May 19-21, 2021, pp. 401-413. ACM, 2021.
Marin Biloš, Bertrand Charpentier, and Stephan Günnemann. Uncertainty on asynchronous time event prediction. In Advances in Neural Information Processing Systems, pp. 12851-12860, 2019.
Christopher M Bishop. Pattern recognition. Machine learning, 128(9), 2006.
Charles Blundell, Julien Comebiie,, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint 1 2015.
Edouard Capellier, Franck Davoine, Véronique Cherfaoui, and You Li. Evidential deep learning for arbitrary LIDAR object classification the context of autonomous driving. 2019 IEEE Intelligent Vehicles Symposium, IV 2019, Paris, France, June 9-12, 2019, pp. 1304-1311. IEEE, 2019.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without OOD samples via density-based pseudo-counts. CoRR, abs/2006.0922 2020.
Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, and Stephan Günnemann. Natural posterior network: Deep bayesian predictive uncertainty for exponential family distribu- tions. arXiv preprint arXiv:2105.04477 2021.
Wenhu Chen, Yilin Shen, Hongxia Jin, and William Wang. A variational dirichlet framework for out-of-distribution detection. arXiv preprint arXiv:1811.07300. 2018
Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless bayesian deep learning. arXiv preprint crXiv:2/06.14800 2021.
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter? Structural safety, 31(2):105-112 2009.
Sven Elflein, Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. On detection with energy- based models. arXiv preprint arXiv:2107.08785, 2021.
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape per-
spective. arXiv preprint arXiv: 191200277,, 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In International conference on Machine Learning, pp. 1050-1059, 2016.
Andrew Gelman, John Carlin, Hal Stern, and Donald B Rubin. Bayesian data analysis. Chap- man and Hall/CRC, 1995.
10

---

Under review as a conference paper at ICLR 2022
Florin C. Ghesu, Bogdan Georgescu, Eli Gibson, Sebastian Gündel, Mannudeep K. Kalra, Raman- deep Singh, Subba R. Digumarthy, Sasa Grbic, and Dorin Comaniciu. Quantifying and lever- aging classification uncertainty for chest radiograph assessment. In Dinggang Shen, Tiamming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap. and Ali R. Khan (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2019 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part VI, volume 11769 of Lecture Notes in Computer Science, pp. 676-684. Springer, 2019.
Will Grathwohl, Kuan-Chich Wang, Horn-Kerrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.
Ang Nan Gu, Christina Luong, Mohammad H. Jafari, Nathan Van Woudenberg, Hany Girgis, Purang Abolmaesumi, and Teresa Tsang. Efficient echocardiogram view classification with sampling-fee free uncertainty estimation. In J. Alison Noble, Stephen R. Aylward, Alexander Grimwood, Zhe Min, Su-Lin Lee, and Yipeng Hu (eds.), Simplifying Medical Ultrasound Second International Work- shop, ASMUS 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings, volume 12967 of Lecture Notes in Computer Science, pp. 139-148. Springer, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.). Proceedings of the 34th International Con- ference Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 2017.
Manuel Haussmann, Sehastian Gerwinn, and Melih Kandemir. Bayesian evidential deep learning with pac regularization. arXiv preprint arXiv: /906 1906.00866, 2019.
Jakob Drachmann Havtorn Jes Frellsen, Søren Hauberg, and Lars Maalóe. Hierarchical vaes know what they don't 't know. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th Inter- national Conference Machine Learning, ICMI. 202 1, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 41 4117-4128, PMLR, 2021.
Matthias Hein, Maksym Andrüushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 41-50. Computer Vision Foundation IEEE, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribure examples in neural networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn., 10(3))0477-55) 2021.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Buc, Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- fion P Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 125-136, 2019.
Pavel Izmailov, Patrick Nicholson, Sanae Lotfi, and Andrew Gordon Wilson. Dangers of bayesian model averaging under covariate shift. arXiv preprint arXiv:2106. 11905, 2021a.
Pavel Izmailov, Sharad Vikram, Matthew D. Hoffman, and Andrew Gordon Wilson. What bayesian neural network posteriors really like? In Marina Mcila and Tong Zhang (cds.), Pro- ceedings of the 38th International Conference on Machine Learning, ICML. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 4629-46400 PMLR, 2021b.
Harold Jeffreys. The theory of probability. OUP Oxford, 1998.
11

---

Under review as a conference paper at ICLR 2022
Taejong Joo, Uijung Chung, and Min-Gwar Sco. Being bayesian about categorical probability. CoRR, abs/2002.07965, 2020.
Michael Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. An introduction to variational methods for graphical models. Machine learning, 37(2):183-233) 1999.
Diederik P. Kingma and Max Welling, Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Reprexentations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
Anna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, and Stephan Günnemann. Evaluating robustness of predictive uncertainty estimation: Are dirichlet-based models reliable? In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 5707-5718. PMLR, 2021.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just a bit, fixes overconfidence in relu networks. Proceedings of the 37th International Conference on Machine Learning, ICMI. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 5436-5446. PMLR, 2020.
Morton Kupperman. Probabilities of hypotheses and informatiio-ssatiiiiis in sampling from exponential class populations. Selected Mathematical Papers, 29(2):57, 1964.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. Advances in neural information processing systems, pp. 6402-6413, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 7167-7177, 2018.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of detection neural networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
image
Jiyy Lin. On the dirichlet distribution. Mater's 'sse's Report, 2016
Jeremiah Z. Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax- and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: An- nual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020.
Zhijian Liu, Alexander Amini, Sibo Zhu, Sertac Karaman, Song Han, and Daniela L. Rus. Efficient and robust lidar- based end-to-end navigation. In IEEE International Conference on Robotics and Automation, ICRA 2021, Xi' Xi'an, China, May 30- June 5, 2021, pp. 13247-13254, IEEE, 2021.
Andrey Malinin and Mark J. F. Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Infor- mation Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, pp- 7047-7058, 2018.
Andrey Malinin and Mark J. F. Gales. Reverse di-divergence training of prior networks: Improved uncertainty and adversarial robustness. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, pp. 14520-14531, 2019.
12

---

Under review as a conference paper at ICLR 2022
Andrey Malinin, Sergey Chervontsev, Ivan Provilkov, and Mark Gales. Regression prior networks. arXiv preprint arXiv:2006. 11590, 2020a.
Andrey Malinin, Bruno Mlodozeniec, and Mark J. F. Gales, Ensemble distribution distillation. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020b.
Andrés R. Masegosa. Learning under model misspecification: Applications to variational and en- semble methods. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florine Bal- can, and Hsuan- Lin (eds.), Advances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Jeffrey W. Miller. (ml 7.7.a2) expectation of dirichlet random variable, 2011.
URL https:
77 www.yoottube.comm
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty. arXiv preprint arXiv:2102.11558 2021.
Kevin P Murphy. Conjugate bayesian analysis of the gaussian distribution. def. 1(2o2):16, 2007.
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of out-of-distribution generalization. In 9th International Conference Learning Repre- sentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
Eric T. Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Görür, and Balaji Lakshminarayanan. Do deep generative models know what they don't 't know? 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Jay Nandy, Wynne Hsu, and Mong Li Lee. Towards maximizing the representation gap between in-domain & out-of-distribution examples. Advances in Neural Information Processing Systems, 33, 2020.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Process- ing Systems, pp. 13991-14003 2019.
Tim Pearce, Felix Leibfried, and Alexandra Brintrup. Uncertainty in neural networks: Approxi- mately bayesian ensembling. In International Conference on Artificial Intelligence and Statistics, pp. 234-244, 2020.
Tim Pearce, Alexandra Brintrup, and Jun Zhu, Understanding softmax confidence and uncertainty. arXiv preprint 1XXi:22006049722 2021.
Kürsat Petek, Kshitij Sirohi, Daniel Büscher, and Wolfram Burgard. Robust monocular localization in sparse hd maps leveraging multi-task uncertainty estimation. arXiv preprini arXiv:21 10563, 2021.
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 1 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 1530-1538. JMLR.org, 2015.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. In Advances in Neural Information Processing Systems, pp. 3179-3189, 3189, 2018
Murat Sensoy, Lance M. Kaplan, Federico Cerutti, and Maryam Saleki. Uncertainty deep classifiers using generative models. In The Thirty-Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty- Secrnd Innovative Applications of Artificial Intelligence Confer- ence, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, FAAI 2020, New York, NY, USA February 7-12, 2020, pp. 5620-56222222 AAAI Press, 2020.
13

---

Under review as a conference paper at ICLR 2022
Yilin Shen, Wenhu Chen, and Hongxia Jin. Modeling token-level uncertainty learn unknown concepts in SLU via calibrated dirichlet prior RNN. CoRR, abs/2010.08101, 2020.
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detec- tion. In Proceedings of the Thirty-Fourth Fourth Conference on Uncertainty Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pp. 560-569, 2018.
Ava P Soleimany, Alexander Amini, Samuel Goldman, Daniela Rus, Sangeeta N Bhatia, and Con- nor W Coley. Evidential deep learning for guided molecular property prediction and discovery. ACS central science, 7(8):1356-137) 2021.
Theodoros Tsiligkaridis. Information robust dirichlet networks for predictive uncertainty estimation. arXiv preprint arXiv: 11X1v:1910.04819, 2019.
Dennis Ulmer and Giovanni Cinà. Know your limits: Uncertainty estimation with relu classifiers fails at reliable ood detection. arXiv preprint S.Xi::2012.05329, 2020.
Joost Amersfoort, Lewis Smith, Yee Whye Tch, and Yarin Gal. Uncertainty estimation using a single deep deterministic neural network. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 9690-979999 9700. PMLR, 2020.
Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. On feature collapse and deep kernel learning for single forward pass uncertainty. arXiv preprint arXiv:2102. /1409, 2021.
Tim Erven and Peter Harremoës. Rényi divergence and kullback-leibler divergence. IEEE Trans. Inf. Theory, 60(7):3797-3820) 2014.
Xiang Wei, Boqing Gong. Zixia Liu, Wei Lu, and Liqiang Wang. Improving the improved training of wasserstein gans: A consistency term and its dual effect. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Confer- ence Track Proceedings, 2018
Florian Wenzel, Kevin Roth, Bastiaan S Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really? arXiv preprint arXiv:2002.02405. 2020.
Wikipedia.
Exponential
family,
2021.
URL
https:/
ily Moment-
Accessed
September
2021.
Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Floring Bal- can, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Ronald R Yager and Liping Liu. Classic works of the Dempster-Shafer theory of functions, volume 219. Springer, 2008
Fuxun Yu, Zhuwei Qin, Chenchen Liu, Liang Zhao, Yanzhi Wang, and Xiang Chen. Interpreting and evaluating neural network robustness. In Sarit Kraus (ed.), Proceedings of the Eiigtt-Eighth International Joins Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10- 16, 2019, pp. 4295- ijcai.org, 2019.
Xujiang Zhao, Feng Chen, Shu Hu, and LinHHee Cho. Uncertainty aware semi-supervised learning on graph data. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florin Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Con- ference Neurul Information P Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020.
14

---

Under review as a conference paper at ICLR 2022
A FUNDAMENTAL DERIVATIONS
This appendix section walks the reader through generalized versions recurring theoretical re- sults using Dirichlet distributions in Machine Learning context, such as their expectation in Ap- pendix A. their entropy in Appendix and the Kullback-Leibler divergence between two Dirich- lets in Appendix B.3
A.1 EXPECTATION OF DIRICHLET
Here, we show results for the quantities E[j²}] and Ellog For the first, one such derivation is given by Lin 2016), however instead adapt the more elegant solution by Miller (2011)
E [[mx]]
f
((ao)
II
ak Pk
dun dux
14k
I [(ax)
k=1
Moving P
06-1 out of the product:
((ao)
II u AW
dul ..l...de ..duk
(8)
I
For the next step, define set of Dirichlet parameters with Br ak and AK' + k Si = 0g.. Now we can see that for those new parameters, Bu is defined as 3o = Σイ 3x = Cip- So by virtue of the Gamma function's 's property that [(ag 1) (an)) replacing all terms in the normalization factor will yield
f
OCk 00
T(3))
II
K
BN PE
dp duk a₂
1(3k))
where the last step we obtain the final result, since the Dirichlet with new parameters 8g must nevertheless integrate 1, and the integrals do not regard aę Oug- For the expectation E [lo [k]]
we first rephrase the Dirichlet distribution terms the exponential family (Kupperman 1964). The exponential family encompasses many commonly-used distributions, such the normal, expo- nential, Beta or Poisson, that all follow the form
p(x;7) = (x) exp (1 u(x) - A(n))
with natural parameters n7 sufficient statistic u(x), and log-partition function A(n). For the Dirich-
let distribution, Wikipedia (021) provides the sufficient statistic as u(fz) = [log p1--..5 #g]'" and the log-partition function
K
A(a) = Σ log ((ox) - log I(a.)
(9)
1=1
By Wikipedia (021), also find that by the moment-ggenerainng function that for the sufficient statistic, its expectation can be derived by
0A(n)
E [((())
(10)
0mp
Therefore evaluate the expected value of log μ4 (i.c the sufficient statistic) by inserting the definition of the log-partition function in Eqquation into Equation (10)
i Day
Eloog pn]
log l((4) log I'(a)) (aa))) (aao))
(11)
which corresponds precisely to the definition of the digamma function (b(x)) log [(x:).
15

---

Under review as a conference paper at ICLR 2022
A.2 ENTROPY OF DIRICHLET
The following derivation is adapted from 2020) well.
Lin 2016), with the result stated in Charpentier et al.
H E (())g(()))))
K II
= E log
B(a)
1k
av
k=1
-EE
log B(a) +
(ake 1) log Mk
k=1
K Σ
log B(a)=
(ak
[)o u4]
Using Equation (11)
K Σ
log B(a)
(ax ())-))((((n)) vaa))
K
= log B(a) +
Σ (ax - 1)bb(au)
Σ (ax 1)p(ak)
k=1
k=1
K
log B(a)= + (ao (oo(
Σ (ak 1)b((a))
A.3 KULLBACK--AIRBER DIVERGENCE BETWEEN TWO DIRICHLETS
The following result is presented using an adapted derivation by Lin 2016) and appears in Chen et al. (208) Joo et al. (2020) as starting point for their variational objective (see Appendix B.7). In the following we use Dir(p: ox) to denote the optimized distribution, and Dir(u: the reference or target distribution.
KL plu m)
=E log
p(u xx) p(
E
log p( ax)
E log 17)
log ((a) +
(ax 1) Mi
k=1
K
E
log B(7) +
Σ
(7k- - 1) log jo
k=1
Distributing and pulling out B(a) and B(y) out of the expectation (they don't depend on μ4):
B(7)
K
log
E
(ak 1) log pm -77 1) log pa
B(x))
B(>)
log
Σ (a - 7k) log u4
B(x)
Moving the expectation inward and using the identity [[ux]] v(ax) sb(ao) from Appendix A.1
B(y)
K
=
+ (ax ( v(ao))
B(a)
k=l
16

---

Under review as a conference paper at ICLR 2022
The KL divergence is also used by some works regularizier by penalizing the distance to uniform Dirichlet with y (Sensoy et al. 2018). In this case, the result above can be derived to be
r(K) B(a)
KL p( p((1aa) pl [1] = log
+ Σ akk - -)(((())-((((()
where the log T(K) term can also be omitted for optimization purposes, since it does not depend on ON.
B ADDITIONAL DERIVATIONS
In this appendix we results relevant in Machine Learning context. These include derivations of expected entropy (Appendix B.1 and mutual information (Appendix B.2) as uncertainty metries for Dirichlet networks. Also, we derive a multitude of loss functions, including the 1.0 norm loss of Dirichlet w.r.t. one-hot encoded class label Appendiix B.3] the /2 norm loss in Appendix B.4, as well as the reverse KI. loss by Malinin & Gales) (2019), the UCE objective [Bllos et al. (2019); Charpentier et al. (2020) and ELBO Shen et al. (2020 Chen et al. (2018) as training objectives Appendices B.5 to B.7)
B. DERIVATION EXPECTED ENTROPY
The following derivation is adapted from Malinin & Gales
(008) appendix section C.4.
H [(u1 p(u (0ulxx00
E Pplu x.0)
Σ aa log po du
--E plu x, 0) (un log yu )
Inserting the definition of p(11) DD):
I(ow)
/
Me log Me II p4
d
7 I(aw)
Singling out the factor pu:
K
I(ao)
/ Mi log ph
= =-
[awRR) I(aw)
II K'E
M
-kk
as
d
Adjusting the normalizing constant (this is the same trick used in Appendix A.I)
I(ao + +1
log uh
II (EKY(R)
My
ay-1
14 d
oo
I(an 1) 1)Ileek)(1)) I(aw))
k=
Using the identity [B[log mx] v(aa)) ((ao)
[Equation (11)
agk
(ba + 10) vb(00 + +1)
00
We can accommodate the extra factor (4 by adding to its concentration parameter, adjusting the whole normalizing constant, and thus obtaining expectation of log #4 distribution which includes this very factor. Therefore, the resulting Digamma functions are also adjusted to this new distribution.
new Dirichlet
17

---

Under review as a conference paper at ICLR 2022
B.2 DERIVATION OF MUTUAL INFORMATION
We start from the expression in Equation (7)
1 1 x, DD H BpiN x. P(ui m0]] Bp(u xxD)
1 [Pv] m)]
Given that E [[ Do is sufficient (Malinin & Gales] 2018), can identify the first term the standard Shannon en- tropy >E K
Appendix A. and assuming that point estimate p(u x, D) p(u x. 0)
k= 14 log Mk 2k=1 ag log at Furthermore, the second part already derived in
ax
Appendix B.1
and thus obtain:
OO log K
K
Olk
Maxx + +1) 35(000 + +1))
+
Ca
1=1
Otj
log
v(ax + 1) <b + +1)
a
an
B.3 1o0 NORM DERIVATION
In this section we follow elaborate on the derivation of Tsiligkarids (2019) deriving a generalized Lp loss, upper-bounding the 30 loss. This in tumn allows to easily derive the l2 loss used by [Sensoy jet al. (2018); IZhao et al.] (2020). Here we assume the classification target y to be provided in the form of one-hot encoded label y [1y=1...., 1g=
Ep(n ( [lly y -- 1 [loo] S E Epin xx,0) [lly 44 [11p]
1/p
s Eppla I xx,) [|ly -e [1
(E((1-M,)P) My) Blug] 1/P
Repfoy
In order to compute the expression above, we first realize that all components of μ are distributed according to Beta distribution Beta(o, ,8) (since the Dirichlet is multivariate generalization of the beta distribution) for which the moment geneentggnnraatng function is defined as follows:
T(a + ())
a(a p) 3)
E E(pº]
I'a +Ca++++Blt B)
I'(a + +
Given that the first term in Appendix B.3 is characterized by Beta(oo agg (gy) and the second one by Beta( agg - as), we can evaluate as follows:
Epiu xx,0 [|ly - |1o0 <
T(ao 0y + p) - 96 966
P(au + p)T + a0 - am)
+
T(ao 95 + p 16) ((00 aay)
D( + 00 (r))((a))
kzy
(T(ao ay + ()oo)))
T(ax + p)T((o))
[( + p - aay)
T(p + an)
kgy
= - ( + +p) T(ao
I(ao) r(ao)_
Ekeg ag +
I(ax +p) I(ax)
+
r
Exjp akk
18

---

Under review as a conference paper at ICLR 2022
B.4 l2 NORM LOSS DERIVATION
Here we present an adapted derivation by Sensoy et al. (0188) for the (2-norm loss to train Dirichlet networks. Here we again an one-hot vector for label with y (1y=ly.e 1y= ky=x]
K
Bp(u x.0)
[1y-
11
1
E
ykk )x
(12)
E
1 y= 21 A/1 pkk
(13)
K
y 21 2ly=xBllu E
(14)
Using the identity that B[p = Elux] 1 + Var(ux))
K
12
yonk 2 + E[aa]? + Var(ww)
(15)
k=
11=k- Var(ux)
(16)
Finally, we use the result from Appendix A.l and the result that Var(uw) 2016):
(c(00-ot) ogloot1)
(sce Lin
Olk ao
ax(a0 ock)
1 yy=k
(17)
ag + 1)
B.5 DERIVATION OF REVERSE KL LOSS
Here re-state and annotate the derivation of reverse KL loss by Malinin & Gales (2019) in more detail, starting form the forward KL loss by Malinin Gales (2018).
Σ 1 1y plu (a) plu x. ])]
Ep(x,))
K
1 y / pll ai) log plus (xx) p( 0) d de
Ex(x,)
Σ
A=1
Writing the expectation explicitly:
Σ 1 y= / plula log plu (x)
Σ ply k, x)
d duxx
plu x, 0)
K
p(u (x) p(11 x, 0)
p(x) P(y = =/(x)
Σ 1y=k
p(ulla log
1x
k=1
p(H (x)
Σ P(y=
Σ 1 y=kk
p(ula))ogg log
d
p(11 0)
A==1
Adding factor in log, collapsing double sum:
K
p(u p(u(a:) (x) Σk P(y = k|x)
= EEp
Σ
P(y = 1/ x))x
log
p( x, 0) 5 P(y k(x)) d
19

---

Under review as a conference paper at ICLR 2022
Reordering, separating constant factor from log:
K Ply = k) x|x)p(ue())
Ek Ply = kl x)p((u (0)
Ep(x)
log
p(1e 0)
k=1
K Σ
log
P(y K| x1xx)
du
k=1
K
Bp(x)
KL
P = k|x)p( (a)
p( x, 0)
k=1
Mixture Dirichlets
where we can see that this objective actually tries to minimizes the divergence towards a mixture of K Dirichlet distributions. In the case of high data uncertainty, this is claimed incentivize the model to distribute mass around each of the corners of the simplex, instead of the desired behavior shown Figure 2c Therefore, Malinin Gales (2019) propose swap the order of arguments in the KL Kiediiee,, resulting in the following:
Ex(x)
Σ
P(y k| x) KL plu x, 1)lp [m)]
K
pvy x|x) / pi 0) log p(pe x. 0) plu à) du
Bp(x)
Reordering:
/ plulx x 0)
K
p( x, 0) p( (a)
-p(x))
Σ P(y x/x))ogg
dl
kml
K
-Epx)
E Epic (x,0)
Σ
P(y = k x) x, 0)
Σ
P(y k| log logp(m (a)
k=1
k=1
K [I x, 0) p(o=h)))))
K
Bp(x)
/ p(u|xx Xx, p(u|xx0)
log
-log log
II p(u a P(o=st1))) d
k=1
Pp(x / |((1 ()) log p(p x, D) p(u=H|x))
ag 1) )))0))))))))))) ) d
-log
II B(a))
16 II
kml
k-=l
p(y=k)
Ep(x)
p(u|x x, 0)
log (x(lm(x,00) log
II
ag -1
du
((x)
k=1
k=1
Bp(x)
/ plul (lulxx001 0) 0)(
(118 (p(44
- log
II
K
> Ply
x)
dd
B(a)
#=1
K
Ep(x) KL. | x, 0)1l(u[a))
where
Σ k=1
ply kl x)
Therefore, instead of a mixture of Dirichlet distribution, we obtain a single distribution whose pa- rameters are mixture of the concentrations of each class.
20

---

Under review as a conference paper at ICLR 2022
B.6 UNCERTAINTY-AWARE CROSS-ENTROPY LOSS
The uncertainty-aware cross-entropy loss in Bilos et al. (2019): Charpentier et al. (2020) has the form
LucE Ep(n xx0) llolgg(()))) [i] EBlog MyJ v(aa) ((a))
as ply) /1) is given by the true label in form of delta distribution, we can apply the result from Appendix A.1
B.7 SVIDENCE-LOWER BOUND FOR DIRICHLET POSTERIOR ESTIMATION
The evidence lower bound is a well-known objective to optimize the KL-divergence between an approximate proposal and target distribution (Jordan al. 1999 Kingma & Welling 2014). We derive it based on Chen et al. (2018) the following for the Dirichlet case with a proposal distri- bution p( x, 0) the target distribution p( ly). For the first part of the derivation, we omit the dependence on 3 for clarity.
log p( x. 0)
p( x, 0)p(y) log
KL [p(p(1] x, 0)[[p(H)))))) [y]] = Epial xx,) 1x,0)
Epiu xx,)
pi4a ly)
p(1,, y)
Factorizing p(p, 3) P(yl p)p(H), pulling out p(v) as it doesn't depend on pc:
log p(u x, Ø) P p))(())
+ p(y)
log p(11 x, 0)
1x.0
Epccl x.0) loog P(yl 4)] + +ply)
p(1))
- KL[p(K x,
(.00 [p]()]]]] Epiu x [log P(yl [4]
Now note that the second part of the result is the uncertainty-aware cross-entropy loss from Ap- pendix B.6 and re-adding the dependence of p(u) on 7, we can re-use our result regarding the KL-divergence between two Dirichlets in in/Appendix An3 thus obtain:
LEBOO (3,))) - ((6) log B(3)
K (B, -B ((6))
(18)
B(>)
k=1
which is exactly the solution obtained by both Chen (0018 and Joo al. (2020).
C OVERVIEW OVER Loss FUNCTIONS
In Tables and 5 compare the forms of the loss function used by Evidential Deep Learning methods for classification, using the consistent notation from the paper. Most of the presented results can be found in the previous/Appendee details about the objective of Nandy et al. (2020).
A andd Appendix B. We refer the original work for
21

---

Under review as conference paper at ICLR 2022
Table 4: Overview over objectives used by prior networks for classification.
Method
Loss function
Regularizer
Comment
Prior networks
log Bal -amifa dx)((((s)) v(aa))
P(D log Bla)
22 1)(((ax))
((co))
Target concentration parameters à are created using a label smoothing approuch.
Malinin & Gules 2018
{ (K
1)e if y
i... jb
if $
Together with setting de as a hyperpanmmeter, Og
14.00
Prior networks Malinin Gales, 2019
Biài log Pias
Ennifa fau 6=)((((ss) -V(oo)
BiA log Biaj
Eds - -x)((((a)) -((on)
Similar to above, a! 1- +
for in-distrbbtion and ai 1.=8 + where we have hyperparameters set die 0.01 and Own O. Then finally, â Et ply x(x) and a E ply kJ x)ax.
r( Erno 10)
Information Robust Dirichlet Networks Tsiligkarids 2019
Doon) ((ont))
1ox+pp ((ao)
2 Ennu(an (0))
( is the polygamma function defined as (400 = #vxx).
r E (Ireo a))
Dirichlet Function Decomposition Bilos et al. 2019
p(aa)) wloo)
Au So pa(r) + do 1o dd(r))
Factors and Ae are treated hyperparameters that weigh first term pushing for logit while pushing the variance in the first term to
lomu
kl pin printoollbin 1) logo
Prior network with PAC Reg.
log [
oa
The expectation in the loss function is evaluated using parameter samples from a weight distribution. [0, 1].
Haussmann 2019
Ensemble Distribution Distillation
v(aa)) Σi sl(a)) + to E 5-8 logs's x mlm))
The objective uses predictions from a trained ensemble with purameters 01.... Ozz.
Malinin al. 2020b
Prior networks with
log * a(ae)
De-1 A log ua Ae M olaw)
The main objective being optimized on in-distribution. the regularizer out-of-distribuiion data. Aia and Aua weighing terms and a denotes the sigmoid function.
representation gup
Nandy et al. 2020
Prior RNN
Σx=1 Limp log
log B(a) + (dn K)v(ao) Ex (au 1)0(as)
Here, the entropy regularizer operates
scaled version concentration parameters à = (Ix W) ax where W is learned.
Shen et al. 2020
Graph based Kemel
oz > (ly=e-
(o0 oz) of
log Blo) Dtmi(ee (w)(vlow) ((oo))
corresponds to uniform prior including some information about the local grüph structure. The authors also use an additional knowledge distillation objective. which was omitted
Dirichlet dist, est. (GKDE)
on
Zhao et 2020
doesn't 't related the Dirichlet.

---

Under review as conference paper at ICLR 2022
Table 5: Overview over objectives used by posterior networks for classification.
Method
Regularizer
Comment
Evidential Deep Learning (Sensoy al. 2018
X (Just
(ou-ma F3
log (VK) S -imn(o - 1)(((a)) - +l(ou))
Variational Dirichlet
v(2,)) v((6)
log DE (x)((()))))))))))))))))))))))))) v(%))
33
Chen et al. 2018
Belief Matching
v(A,) v(())
8:8) log (ig)
+ (8 - (w((()))))))))))))))))))))))))) volol)
Joo et 2020
Posterior networks
v(3,) v(())
log B(a) + (0o Ki)t(a))) EE (ca - 1)v(a))
Charpeatier 2020
Generative Evidential
Σ [Bpacoo ([0x(1([[((x))]] Bau00 Dog(1 =(fo(()11)
log rIA)
Exnylon 1)((aa)) -(oo))
The discriminative using samples, generated by a VAE The regularizer is taken over all classes eucluding the class (also by a--).
Neural Network
(Sessy al.


Reviews:
[{"review_id": "mJdLHol7juC", "reviewer": "Reviewer_eCYd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**:\n* In general, the paper is well written, structured, and easy to understand.\n* There is significant value in surveying evidential deep learning techniques as there has been some parallel work in this space that is not often connected. This survey covers and categorizes a large number of papers under the umbrella of evidential deep learning.\n* Fig. 1 is an excellent visualization and categorization of approaches in this space.\n* I liked the classification of the literature into prior and posterior methods throughout the survey.\n\n**Weaknesses**:\n* A significant portion of the survey repeats ideas from the Prior Network paper (Malinin and Gales, 2018) with some typos (e.g., in Eqs. 3 and 4, the condition on the dataset as per the Prior Network paper has been omitted). I am missing further insight and takeaways beyond the existing collection of papers surveyed. The discussion details some potential avenues for future work, and there are some insights in the footnotes. It would be good to bring these footnote insights into the paper and expound on them (particularly since there is enough space). However, as it stands, the survey does not provide sufficient additional insight or context for the ideas presented to form its contribution.\n* The representation gap (Nandy et al., 2020) and its resulting visualization were not described in sufficient detail.\n* Overall, it appears that there was insufficient material for a 9 page paper as equations were extremely spaced out (e.g., Eqs. 3 and 4), and even then the paper did not reach the page limit, making it look unpolished.\n* In general, the presentation of the paper needs improvement. There were frequent typos (e.g., 'We also to provide', 'e. 4', 'uncertaint-aware') and inconsistent formatting for referencing figures, sections, and equations (i.e., capitalization and abbreviation differences). There were some notational issues, as well. For example, it was confusing to see the definition $\\mu_k = P(y = k \\mid x, \\theta)$, with and without $\\theta$, and then to have the distribution $p(y \\mid \\mu)$. Furthermore, the references list is unpolished: repeating conference names, listing urls, etc.\n\n**Suggestions for improvement**:\n* My main recommendation is to work into the survey further insights and context from beyond the surveyed papers. For example, the origin of the term *evidential deep learning* comes from evidential theory [1], which relates to the Dirichlet distribution through the ideas in subjective logic [2]. It would be good to further highlight that prior networks collapse the uncertainty over the model parameters, as this insight is important and worth spending more space on in the paper.\n* The Sensoy et al. AAAI paper [3] is missing from the survey, and should be incorporated.\n* Additional investigation into the advantages and disadvantages of the different metrics used to evaluate the performance of evidential deep learning methods with respect to OOD inputs and uncertainty calibration would be valuable.\n* Since this field does not have an unreasonable number of papers, the authors should consider running some of their own experiments to provide empirical conclusions, comparing methods that were not already compared in existing work (e.g., (Charpentier et al., 2020) and (Sensoy et al., 2020)).\n* Lastly, the paper should be polished and proofread. I recommend the use of the cleveref package in LaTeX with the 'capitalise' option to avoid inconsistencies in referencing figures, equations, and sections. \n\n[1] A. P. Dempster. A generalization of Bayesian inference. Classic works of the Dempster-Shafer Theory of Belief Functions, pages 73–104, 2008.\n\n[2] A. Jøsang. Subjective Logic: A formalism for reasoning under uncertainty. Springer, 2016.\n\n[3] M. Sensoy, et al. Uncertainty-Aware Deep Classifiers Using Generative Models. In AAAI, 2020.", "summary_of_the_paper": "The proposed paper surveys recent work in evidential deep learning, and provides some potential directions for future work in this space. Evidential deep learning is a set of techniques for calibrating uncertainty of neural networks through second-order distributions.", "summary_of_the_review": "The idea of a survey on evidential deep learning is excellent and valuable to researchers. However, in its current state, I do not believe the paper is ready for publication at ICLR. With improvements to the broader insights and takeaways from the survey as well as paper presentation, this work will become a good contribution to the academic community.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "ZF2zI654ThV", "reviewer": "Reviewer_8cR8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Why is Sensoy et al. 2018 called ‘prior networks’ (it seems the original papers did not referred to their method as prior network either). According to the equation in Section 3.3.1, shouldn’t the prior be the flat Dirichlet? What is needed to be trained is more like an approximate posterior network that takes as input a new data x. It would be helpful for readers to elaborate on this.\n\nIn Equation 5, should $\\mu_0, …, \\mu_K$ be $\\mu_1, …, \\mu_K$? Otherwise, $\\mu_0$ will always be the largest by definition, which makes on sense. \n\nFor Equation 6, is some term related to $B(\\alpha)$ missing?\n\nThe argument that the model uncertainty can simply replaced by some function of $\\alpha$ is not very convincing, given that the network will directly output $\\alpha$. It would be helpful if more details and intuition can be provided. \n\nAs a survey, this manuscript seems to miss a lot of important related work on probabilistic neural networks and Bayesian deep learning [a-f]. \n\nMinor:\n\nP6: fig. 2 -> Fig. 2\n\n\n\n[a] Natural-Parameter Networks: A Class of Probabilistic Neural Networks, NIPS 2016\n[b] Feed-forward Propagation in Probabilistic Neural Networks with Categorical and Max Layers, ICLR 2018\n[c] Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation, CVPR 2019\n[d] Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks, ICML 2015\n[e] Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks, ICML 2020\n[f] A Survey on Bayesian Deep Learning, ACM Computing Surveys, 2020\n", "summary_of_the_paper": "This paper surveys a collection of existing works that the author frames as evidential deep learning. For the classification case, evidential deep learning tries to train a network to output the parameters of a Dirichlet distribution, hoping the one could directly obtain the data uncertainty and model uncertainty from some functions of the output parameters. The authors provide a relatively comprehensive review of recent work in this direction. Besides classification, evidential deep learning for regression is also discussed briefly. ", "summary_of_the_review": "Overall, the paper is informative and did a relatively good job in introducing the basic concepts as well as the motivation of evidential deep learning. However, it seems there are still some important related works that are missing from the references. Besides, the paper in its current form looks more like a review rather than a comprehensive thorough survey, partly because of the space constraints in ICLR. Therefore I am not sure that ICLR is the right venue for it. Perhaps it is more suitable as a long journal where more details and taxonomy could be included. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "H8urSpN6gb", "reviewer": "Reviewer_vS5D", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In my opinion, the paper did a very good job of collecting papers on evidential deep learning and  providing a brief overview of their ideas. As a person unfamiliar with this topic, I got interested and proceeded to read some of the references as I didn't fully grasp the motivation and all the details from a rather short introduction. The paper could have been more enjoyable if it wasn't limited to 9 pages and was written in a form of a tutorial with more intuition for all the equations. For example, it took me some time to interpret Eq.4,6,7. What bothers me about evidential deep learning is that the choice of the regularization term often seems arbitrary. I wish this paper had some deeper insights on  this matter. \n\nI haven't seen survey papers published at conferences before, so I'm hesitant to recommend its acceptance. It would have been different if the paper, in addition to being a survey, presented some fundamental insights into the nature of these methods, which could count as a novel contribution.\n\nI think it can become a well-cited journal or arxiv paper that can get people interested in the topic of evidential deep learning if the background sections are turned into a self-contained tutorial, and the survey parts include some additional motivation and intuition regarding the design choices that each of the papers make.    \n\nExtra remarks:\n- should Eq.2 be $p(\\mu | D, \\alpha)$ instead of $p(\\alpha | D, \\mu)$?  \n- section 3.2 typo: quanitified\n- section 3.3.2 typo: uncertaint-aware\n\n \n\n  ", "summary_of_the_paper": "The paper is a survey of methods in evidential deep learning. It gives a brief motivation for this set of methods, explains a general framework, and describes previous works for classification and regressions tasks in varying amount of details. As a survey paper, it doesn't introduce novel ideas, but gives a useful overview for people who would like to start working on this topic. ", "summary_of_the_review": "Contributions of this paper are not novel, which is normal for a survey paper. \nI think survey papers are not suitable for conferences as they need to be evaluated using a different set of criteria. \nTherefore, my recommendation would be to reject this paper purely based on its type. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "0Dvrf8sLebb", "reviewer": "Reviewer_H7TT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a survey paper without any conceptual claims of novelty. Its main purpose is to make an emerging family of machine learning models more accessible to the community. I find this goal sensible and important.\n\nThe paper presents the technical material very clearly and provides a well-justified dichotomy of the evidential deep learning model family.\n\nI think the statement in Footnote 1 is a bit unnecessary confusion. The precision score of the Dirichlet distribution corresponds simply to the \"precision\" of a Gaussian distribution, which is customarily defined as the inverse of the covariance.\n\nThere exist few pieces of work that use Dirichlet priors on class probabilities, which are missed by this survey paper. For instance,\n\nJ. Gast and S. Roth, \"Lightweight Probabilistic Deep Networks\", CVPR, 2018\n\nM. Haussmann et al., \"Bayesian Evidential Deep Learning with PAC Regularization\", AABI, 2020\n\nThe paper could clarify the qualitative difference between distributional uncertainty and representation gap better. It looks to me like they both quantify the same source of uncertainty: whether a test-time samples is coming from a different distribution than the training-time samples. They only differ in the quantitative aspect: how much the model is confident that a sample is out-of-domain. I am missing the rationale behind introducing a new category of uncertainty only for the sake of this quantitative difference.\n\nThe paper claims to collect some theoretical properties of the Dirichlet distribution that could be useful for machine learning. However, going through the paper and the appendix, I am not able to identify theoretical content that provides any new insights on the use of Dirichlet distribution for machine learning. The presented calculations seem to have been taken from the original papers in their vanilla form and some additional properties, such as expected L-infty norm and moment generating function are rather textbook material.", "summary_of_the_paper": "The paper presents a survey of evidential deep learning, a family of machine learning methods that suggest accounting for various forms of uncertainties via a hierarchical predictive model whose hyperprior parameters are a function of input observations. The paper classifies evidential deep learning approaches into categories and points out the strengths and weaknesses of each category. It also presents some theoretical properties of the Dirichlet distribution that could be relevant for machine learning tasks.", "summary_of_the_review": "Having said that the paper touches the important topic of making a new model family more accessible to the audience, I am afraid I do not think it does it in a way that would add sufficient value on top of one reading the material from the original papers. I think this way because the paper adopts most of the content verbatim from the original sources and does not add a level of abstraction that helps the reader draw new insights that are not straightforward from the original sources. Nor does it highlight overlooked positive or negative properties of the evidential model and nor does it present any empirical outcomes that one would find surprising. Under these conditions, I am not able to recommend an accept for this paper in its current shape.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper surveys a collection of existing works that the author frames as evidential deep learning.

While the paper has been recognized as a nicely written survey, all reviewers have raised the major concern that the paper does not have a sufficient academic contribution compared to the surveyed papers. In particular, novelty appears to be limited as the paper does not offer novel views into the surveyed subfield.

Given the strong consensus among reviewers, I recommend rejecting this paper.
Decision: Reject