Random-Set Neural Networks
Authors: Shireen Kudukkil Manchingal
Muhammad Mubashar
Kaizheng Wang
Keivan Shariatmadar
Fabio Cuzzolin
Venue: ICLR 2025 Poster (2025)
Keywords: Uncertainty quantification
Classification
Deep Learning
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Machine learning is increasingly deployed in safety-critical domains where erroneous predictions may lead to potentially catastrophic consequences, highlighting the need for learning systems to be aware of how confident they are in their own predictions: in other words, 'to know when they do not know’. In this paper, we propose a novel Random-Set Neural Network (RS-NN) approach to classification which predicts *belief functions* (rather than classical probability vectors) over the class list using the mathematics of *random sets*, i.e., distributions over the collection of *sets* of classes. RS-NN encodes the 'epistemic' uncertainty induced by training sets that are insufficiently representative or limited in size via the size of the convex set of probability vectors associated with a predicted belief function. Our approach outperforms state-of-the-art Bayesian and Ensemble methods in terms of accuracy, uncertainty estimation and out-of-distribution (OoD) detection on multiple benchmarks (CIFAR-10 vs SVHN/Intel-Image, MNIST vs FMNIST/KMNIST, ImageNet vs ImageNet-O). RS-NN also scales up effectively to large-scale architectures (e.g. WideResNet-28-10, VGG16, Inception V3, EfficientNetB2 and ViT-Base-16),
exhibits remarkable robustness to adversarial attacks and can provide statistical guarantees in a conformal learning setting.

Published as a conference paper at ICLR 2025
RANDOM-SET NEURAL NETWORKS
Shireen Kudukkil Manchingal
Muhammad Mubashar
Kaizheng Wang".
Keivan Shariatmadar
Fabio Cuzzolin'
1 "Scooo of Engineering, Computing and Mathematics, Oxford Brookes University, UK 2 MM-Group and DistriNet Division, Department of Computer Science, KU Leuven, Belgium LLMS Division, Mechanical Engineering, KU Leuven *Flanders Make @ KU Leuven (19185895, 19230664, fabio. www.olinzeeeecooooo @brookkes (kaizheng wang, keivan. /oroo:ocuuz.iii//rroook..... aaaeessaccuu ac.uk
@kuleuven.bbe
ABSTRACT
Machine learning is increasingly deployed in safety-critical domains where neous predictions may lead to potentially catastrophic consequences, highlighting the need for learning systems be aware of how confident they in their own predictions: in other words, "to know when they do not know". In this paper, we propose novel Random-Set Neural Network (RS-NN) approach to classification which predicts belief functions (rather than classical probability vectors) over the class list using the mathematics of random sets, i.c., distributions over the col- lection of sets of classes. RS-NN encodes the 'epistemic" uncertainty induced by training sets that are insufficiently representative or limited in size via the size of the convex set of probability vectors associated with predicted belief func- tion. Our approach outperforms state-of-the-art Bayesian and Ensemble methods in terms of accuracy, uncertainty estimation and out-of-distribuiie (OoD) detec- tion on multiple benchmarks (CIFAR-10 vs SVHN/ntel-Image MNIST vs FM- NIST/KMNIST, ImageNet vs ImageNet-O). RS-NN also scales up effectively large-scale architectures (e.g. WideResNNt-228- 10, VGG16, Inception V3, Effi cientNetB2 and ViT-Base- exhibits remarkable robustness to adversarial at- tacks and can provide statistical guarantees in a conformal learning setting.
INTRODUCTION
Machine learning models often struggle to provide reliable predictions when confronted with unfa- miliar data (Guo et 2017a Ovadia et al.) 2019 Minderer et al.) (02T), may be noisy samples (Papernot et al. 2016) deliberately designed to deceive models, out-of-distribution data (OoD) beyond the model's training distribution. An ideal learning system, in opposition, should be aware of how confident it is in its own predictions, acknowledge the limits of its knowledge and gauge these limitations to make informed decisions by modelling the epistemic uncertainty associated with it, in essence, 'to know when it does not know'. Epistemic uncertainty pertains to uncertainty associated with our ignorance about the underlying data generation process (Kendall & Gal 2017 Hüllermeier & Waegeman 2021 Manchingal & Cuzzolin, (022). Within machine learning, major source of epistemic uncertainty arises from the limited representativeness of the available training data, con- strained in both quantity and quality.
In this paper, we propose new approach to classification which models epistemic uncertainty using a random set approach (Molchanov) 2005 2017). As they assign probability values to sets of out- comes directly, random sets can naturally model the fact that observations often come in the form of sets (in particular, when missing data occurs), and accommodate ambiguity, incomplete data, and probbaabbiiittc uncertainties. As classification involves only a finite list of classes, we model uncertainty using belief functions (Shafer 1976), the finite incarnation of random sets (Cuzzolin, 2018), whose theory (Shafer 1976) is, fact, generalisation of Bayesian inference (Smets)) 1986). Classical (discrete) probabilities can be seen special belief functions, and Bayes' rule as special case of the Dempster's 'ss rule of combination (Dempster) 2008) originally proposed for aggregating belief functions. For readers less familiar with the topic, in SA.] we recall the distinction between classical probabilities and belief functions and the way they handle uncertainty more detail. Fig.
*Corresponding author.

---

Published as a conference paper at ICLR 2025
1 contrasts the inference processes of our Random-Set Neural Network (RS-NN) and of a Bayesian Neural Network (Jospin et al.] 2022). In hierarchical Bayesian inference (top). a posterior distri- bution over the network's s weights is learned from a training set. At prediction time, a predictive distribution is generated in the target space (left) by sampling from this weight posterior (middle), which amounts second-order probability distribution there (Hüllermeier & Waegeman, 2021 A single (mean) prediction is then typically derived by Bayesian Model Averaging (BMA) (Hocting al. 1999) (right), while uncertainty is measured by the entropy of the mean prediction and the variance of the predictive distribution.
Bayesian Neural Network (BNN)
Bayesian
Predictive Entropy. 18p9
Epsilon: 0.05
True Label:
Model Averaging
3
Predictions: RS-NN
Variance, o
Probubility simplex distribution (Bayesian prodictive posteriors
Second-order
Bayessan meam peediction fs
(66) 33.59%
CNN 14: 99.01%
inference)
Uncertainty measures
miA mLÁgi
Probability
Pignistic
Pignistic Entropy, H(Be(P)
Epsilon: 0.01
True Label: 3 Predictions:
Ak midge
Function
3
Credal set width
RS-NN
Random-sct space representation
Probsbility simplex with credal set induced
Pignistic prediction
[3', 6') 48.25% CNN
BetP
by belief functions
14:: 96.74%
Random-Set Neural Network (RS-XN)
Figure 1: Inference in Bayesian Neural Network (top) as op- posed to a Random-Set Neural Network (bottom). with corre- sponding measures of uncertainty and their sources. The triangle represents the set of probability vectors (probability simplex) one can define on the target space (e-g.. a set of classes).
Figure 2: Confidence scores of RS-NN and CNN for FGSM adversarial attack for different perturbations (e = 0.05, 0.01) on MNIST dataset.
In Random-Set Neural Network (Fig. [ bottom), each test input at inference time is mapped to a belief function (Shater 1976) on the collection of classes C. This belief function is encoded by a mass value m(A) [0, assigned to finite budget O {Au C C} of sets of classes (left). The most relevant such sets are identified from training data by fitting Gaussian Mixture Model (GMM) to the labelled data and computing the overlap among the resulting clusters (Spruyt, 2013). Such a predicted belief function is mathematically equivalent to a convex set of probability vectors (credal ser (Levi 1980, Cuzzolin 22088) on the class list (middle). A pointwise prediction (right) can then be obtained by computing the centre of mass of this credal set, termed pignistic probubility (Smets) Kennes 1994). In RS-NN, uncertainty can expressed using either the entropy of the pignistic prediction (analogously to the Bayesian case), the width of the credal set prediction (Antonucci & Cuzzolin 2010). We find empirically that the pignistic entropy better separates in in-distribution (iD) and out-of distribttii (OoD) samples than Bayesian entropy does (see Tab. 2 the width of the credal prediction, as it encodes the epistemic uncertainty about the prediction itself, is empirically much less correlated with the confidence score than entropy (see $E.5.3).
Using random-set representation prevents the need discard information, challenge observed in Bayesian Model Averaging (Hinne et al. 2020 Graefe et al. (013). While Bayesian inference re- quires defining prior distributions for model parameters even in the absence of relevant information, in belief theory priors are not required for the inference process, thus avoiding the selection bias risks that can seriously condition Bayesian reasoning (Freedman 1999). Based on our extensive ex- periments on multiple datasets, including large-scale ones, RS-NN not only demonstrate superior accuracy compared state-of-the-art Bayesian and Ensemble models (Sec. [4.2), but also arguably better encode the epistemic uncertainty associated with the predictions (Sec. 4..4) and better dis- tinguish in in-distribution and duitt--dittiibttion data (Sec. 4.3). Furthermore, RS-NNs effectively circumvent the tendency of standard networks to generate overconfident incorrect predictions, as illustrated in Fig. 2 in an experiment on Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2014) adversarial attacks MNIST (LeCun & Cortes 2005). CNN misclassifies with high confi- dence scores of 99.01% % and 96. 66.74% while RS-NN shows lower confidence at 32.59% and 48.25% 2.%.
Contributions. Firstly, we propose a novel Random-Ser Neural Nenvork (RS-NN) approach based on the principle that deep neural network predicting belief values for sets of classes, rather than individual classes, has the potential to be a more faithful representation of the epistemic uncertainty induced by the limited quantity and quality of the training data. RS-NN is 'wrapper' technique
2

---

Published as a conference paper at ICLR 2025
that can be applied on top of any existing baseline network model, by just changing the output layers and loss function. Statistical guarantees can also easily be provided for RS-NN predictions by applying conformal prediction budgeting method for efficiently selecting a limited budget of relevant sets of classes for the task at hand given the available training data, by fitting Gaussian Mixture Models to the labelled training points and computing their clusters. This overcomes the exponential complexity of vanilla random- set implementations, ensures the scalability of the approach to large datasets, and helps the network learn by limiting the available degrees of freedom ($E.7). Thirdly, we introduce two new meth- ods for assessing the uncertainty associated with random-set prediction: the Shannon entropy of the pignistic prediction and the width of the credal prediction itself, which prove to be more ro- bust uncertainty measures. For instance, pignistic entropy provides a clearer distinction between in-distribution (iD) and out-of-distribution (OoD) entropy (Fig. [8] $E.5.T), while credal set width excels in separating iD and OoD samples, particularly for challenging datasets like ImageNet vs. ImageNet-O (Tab. D. Finally, we present large body of experimental results (based on a fair com- parison principle in which all competing models are trained from scratch) which demonstrate how RS-NN outperforms both ofate-of-the-ar Bayesian (LB-BNN (Hobbhahn et al.) [022). FSVI (Rud-) ner et al. ) and Ensemble (DE [Lakshminarayanan et al., 2017), ENN (Osband et al.) 2024)) methods terms of: (i) performance (lest accuracy, inference time) (Sec. 4.2): (ii) results on vari- ous out-of- (OoD) benchmarks (Sec. 4.3) including CIFAR- 10 vs. SVHN/Intel-
the pignistic probabilities (see $A.4). Secondly,
outline
MNIST vs. FMNIST/KMNIST,, and ImageNet vs. ImageNet-O; (iii) ability to provide reliable mea- sures of uncertainty quantification (Sec. 4.4) in the form of pignistic entropy and credal set width, verified on OoD benchmarks; (iv) scalability to large-scale architectures (WideResNet--8-
In- ception V3, EfficientNet B2, ViT-Base- 16) and datasets (e.g. ImageNet) (Sec. 4.5). Additionally,
we show how RS-NN is robust to adversarial attacks ( ($E.4) and noisy data ($E.3), ). and circumvents the overconfidence problem in CNNs ($E.)) A qualitative assessment of entropy vs credal set width
is given in EE..3 and in-distribution vs out- -uitt--distribution entropy scores are shown in Fig. 18
Paper outline. We first recall the notions of random sets, belief functions, credal and pignistic predictions (Sec. 2 We explain the RS-NN approach, loss function and uncertainty representation in Sec. 3 Sec. 4 provides a large body of empirical evidence supporting our approach and discusses its limitations. Sec. 5 concludes and outlines future work. Appendix disdususss RS-NN learning process and statistical guarantees, $B reviews further related work, $C dddecribess all algorithms in detail while CCnntains a wealth of additional experimental results and all implementation details.
Related Work. The machine learning community has recognised the challenge of estimating un- certainty in model predictions, leading to the development of several Bayesian approximations (Gal & Ghahramani 2016) CCarpentter et al., 2020), evidential Dirichlet models (Sensoy et al.) 2018
Gao et al.| (024) and conformal prediction (Shafer & Vovk 2008 Papadopoulos et al. 20088] Bala-
subramanian et 2014 Vovk 2012; Angelopoulos & Bates [202]). Some methods rely on prior knowledge (Fortuin 2022). ). whereas others require setting a desired threshold on predictions (An-
ggllopoulss & Bates, 2021 J. Some (Baron) 1987; Hüllermeier & Waegeman [02T] have argued that
classical probability is not equipped to model 'second-level uncertainty on the probabilities them selves. This has led to the formulation of numerous uncertainty calculi (Cuzzolin) 2020), including
possibility theory (Dubois & Prade 1990), probability intervals (Halpern (017]) credal sets (Levi 1980,, random sets (Nguyen, 1978) or imprecise probability [Walley] (191).
Bayesian approaches, pioneered by |Bnntine & Weigend (1991) and others (MacKay 1992 Neal 2012). are dominant in uncertainty estimation. Notable techniques include R-BNN (Reparameter- isation) (Kingma et al.) 2015), variational inference with reparameterisation and Laplace Bridge Bayesian approximation (LB-BN) BB-BNN) (Hobbhahn et al. 2022), which uses the Laplace Bridge to map between Gaussian and Dirichlet distributions. Various approximations of full Bayesian inference exist, such Markov chain Monte Carlo (MCMC), function space BNNs (Sun et al. 2019) such as function-space variational inference (FSVI) (Rudner et al. 2022). and Dropout Variational Infer- ence (Gal & Ghahramani 2015). In our experiments, we do not consider older Bayesian models such as R-BNN, MCMC and Dropout VI, since they have been superseded by more performing approaches and are computationally expensive to train on larger datasets and architectures. Recent work addresses challenges in computational cost (Hobbhahn et al., 2022 Daxberger et al. 2021) and model priors (Tran et al.] (020)). Despite their advantages, Bayesian models face challenges when the model prior is misspecified. Ensemble-base approaches, such as Deep Ensembles (DE) (Lakshminarayanan et al. (017) and Epistemic Neural Networks (ENN) (Osband et al., 2024), ef- ficiently estimate uncertainty by leveraging multiple models. However, the computational cost of
3

---

Published as a conference paper at ICLR 2025
training ensembles, especially for large models, is often impractical. Our approach mitigates these challenges by eliminating the need for both inference- time sampling and prior selection, reducing computational complexity compared to Bayesian inference and lowering training time compared to Ensembles, as demonstrated in our experiments in Sec. More related work is given in $
2 RANDOM SETS AND BELIEF FUNCTIONS
Random sets. A die is a simple example of a (discrete) random variable. Its probability space is defined on the sample space θ {facel, face ..... face 6}. where elements are mapped to the real numbers 1, ..2..... 6, respectively. Now, imagine that faces 1 and 2 are cloaked, and we roll the die. How do we model this new experiment, mathematically? Actually, the probability space has not changed (as the physical die has not been altered, its faces still have the same probabilities). What has changed is the mapping: since we cannot observe the outcome when a cloaked face is shown (assuming that only the top face is observable), both face 1 and face 2 (as elements of @) are mapped to the set of possible values {1, 2} on the real line I (Fig. 3). Mathematically, this is random set (Matheron 1975; Kendall 1974, Nguyen 1978 Molchanov] 2005), i.e., set-valued random variable, modelling random experiments in which observations come in the form of sets.
faoe
values
Credat set plaj-1
2
face S
Bedjey sull maties) D4-00
Probsbility simplex
Béer
Mass values
2
TAItFy Aill miicaag
#ts
Figure 3: The random set associated with cloaked die which faces and 2 are not visible.
Figure 4: A belief function is equivalent to credal set with boundaries determined by lower bounds (Eq. 4 on probability values.
Belief functions. Random sets have been proposed by Dempster (0088) and Shafer 1976) as a mathematical model for subjective belief, alternative to Bayesian reasoning. Thus, on finite domains (e.g., for classification) they assume the name of belief functions. While classical discrete mass functions assign normalised, non-negative values to elements 0 E of their sample space, belief function independently assigns normalised, non-negative mass values to its subsets A C O:
m(A)= O, Σ m((A) 1, AA E P())
The belief function associated with mass fúnction m measures the total mass of the subsets of each 'focal set' A. Mass functions can be recovered from belief functions via Moebius inversion (Shafer 1976), which, in combinatorics, plays role similar to that of the derivative:
(el(A) Σ m(B), m(A) Σ (-1) Bel(
BCA
BCA
{cr, C2, 02) and let P(0) be its power set (collection of all subsets). As shown in Fig. 4 one can define a mass function on P(O) as: m((ei)) = 0.5, m((ca)) sets (unspecified) equal to zero. Note that m is normalised: Σ EBCe = 1. By Eq. 2 the
Example. Consider a sample space (class list) e - C
0.1, m({ci, o2)) = 0.4 (Fig. 4 top), and the masses for all other
belief value of the composite class A = {c2, 03} is: Bel( (el( cs)) ) m((ca)) 0.1 (Fig.
left). Similarly, the belief value of composite class (c1,02) can by computed as Bel( (2)) m({ci)) + m((ci, = 0.5 + 0.4 = 0.9.
Pignistic probability. Given a belief function Bel, its pignistic probability is the precise probability
distribution obtained by re-distributing the mass of its focal sets A to its constituent elements, E A:
m(AA)
Bed = Σ
T4
A30
Smets (2005) originally proposed use the pignistic probability for decision making using belief functions, by applying expected utility to it. Notably, the pignistic probability is geometrically the centre of mass of the credal set (see Fig. 4 associated with a belief function (Cuzzolin 2018).
Credal prediction. As anticipated, RS-NN is designed to predict belief function (a finite random set) on the set of classes. A belief function, in turn, associated with a convex set of probability
distributions (a credal set eLvi 1980. Zaffalon & Fagiuoli] 2003) Cuzzolin 2010: Antonucci &
Cuzzolin 2010) Cuzzolin 2008) on the same domain. This is the set:
Cre =P:: θ [0, Be) < P(A)).
4

---

Published as a conference paper at ICLR 2025
of probability distributions P on ② which dominate the belief function each focal set A. The size of the resulting credal prediction measures the extent of the related epistemic uncertainty (see Sec. 3.3) arising from lack of evidence (see $E.5.3). The use of credal set size as measure of epistemic uncertainty is well-supported in literature (Hüllermeier & Waegeman 2021: Bronevich & Klir (008), it aligns with established concepts of uncertainty such as conflict and non-specificity
Yager, [008 Kolmogorov] (965). A credal prediction, by encompassing multiple potential distri- butions, reflects the model's acknowledgment of this uncertainty. A wider credal set indicates higher uncertainty, as the model refrains from committing to specific probability distribution due to lim- ited conflicting evidence. In contrast, a narrower credal set implies lower uncertainty, signifying a more confident prediction based substantial, consistent evidence.
3 RANDOM-SET NEURAL NETWORK
3.1 APPROACH
Representation. As shown in Fig. (b), RS-NN predicts for cach input data point belief function, rather than a vector of softmax probabilities as in a traditional CNN. For N classes, a 'vanilla' RS-NN would have 2 outputs (as 2 is the cardinality of P(C)), each being the belief value (Eq- E P(C) corresponding to that output neuron. Our architecture the final layers (Fig. (b), in grey), acting
2 of the focal set of classes
wrapper applicable top of any baseline representation layers from existing models (Fig. (b), in blue). This enables the integration of pre-trained networks while fine-tuning only the final decision layers. Hence, RS-NN
tocuses primarily
is easily scalable to any model architecture, as demonstrated in Sec. 4.5 Tab.
Given a training datapoint with true class attached, its ground truth is encoded by the vector bel = [Bel(A), e }(C)} of belief values for each focal set of classes A E P(C). Bel(A) is set iff the true class contained in the subset A, otherwise This corresponds full certainty that the element belongs to that set and there is complete connidence in this proposition (see A.2 for an example).
Be(( bel
Pignistic Entropy Hi
Predicted
Pignistic Probability BetP
Inpot (eg image)
-1
Fitting Guuseian Moddls (GMM) each class
Uncertainty estimase
Crodal set Cir
Top K relevant subsess
Inference
playi
Minimise Eyes
Training
vehicle)
Gar. track
Belief encoded ground trath of most relevant subsets
c P(C)
Peature vector
Overlapping clusters over F(C)
a = CU A Ark
Del(A)
bel
(a)
(b)
Figure 5: RS-NN model architecture. (a) Budgeting: Given collection C. of N classes, the top K are selected from the powerset TP(C) (Algorithm in IC.ID and added to the singleton classes form the budget O. (b) Training and Inference: Ground truth classes are encoded belief vectors bel and used to predict belief function bel for each training data point by minimising the loss Ls to train the output layers (in grey) producing bel. Mass values in and pignistic probability estimates Bet are computed from the predicted belief function. Uncertainty is estimated described in Sec. 3.3
relevant (focal) sets of classes (AI,...,AA) AK)
Budgeting. To overcome the exponential complexity of using 2 sets of classes (especially for large N), fixed budget of K relevant non-singleton (of cardinality > 1) focal sets are used. These focal sets are obtained by clustering the original classes fitting ellipses over them, and selecting the top K focal sets of classes with the highest overlap ratio (Fig. 5 (a)). This is computed as the intersection over union for each subset A in PCC): overlap(
, AK E P(C). Clustering is performed on feature vectors of images generated by any feature extractor trained on the original classes C. In our experiments, we used features from a trained standard ResNet50 model. These feature vectors are further reduced to 3 dimensions using t-SNE (t-Distributed Stochastic Neighbor Embedding) (van der Maaten & Hinton 2008) before applying a Gaussian Mixture Model
A
Note that the helief encoding of ground-truth is not related to label smoothing. It maps ground-truth labels from the original class space set space without adding noise, thus preserving label "hardness".
5

---

Published as a conference paper at ICLR 2025
(GMM) to them. Ellipsoids (Spruyt 2013), covering 95% of data, are generated using eigenvectors and eigenvalues of the covariance matrix E. and the mean vector He. Vc E C, Pe ~ (are (KrecHHe He, Ex) obtained from the GMM to calculate the overlaps. To avoid computing degree of overlap for all 2 subsets, the algorithm is early stopped when increasing the cardinality does not alter the list of most-overlapping sets of classes. The K non-singleton focal sets obtained, along with the N original (singleton) classes, form our network's ' budget of outputs O = C C CAA........ Ax} J. E.g., in 100-class scenario, the powerset contains 2100 subsets (10 possibilities), Setting budget of K = 200, for instance, results in 100 + K = 300 outputs, a far more manageable number.
As shown in our experiments, budgeting also helps the network converge without overfitting. While, in theory, a complete belief function model would be more powerful, in practice a larger number of focal sets impedes the learning process. As shown in Tab. T6 E..7 RS-NN with a limited number of well-representative sets often performs better than RS-NN with a full power set of classes, and is more efficient at uncertainty estimation. The budgeting step is a one-time procedure, applied to any given dataset before training. It requires just 2 minutes for CIFAR-10, 7 minutes for CIFAR- 100, and 60 minutes for ImageNet (== 1.1M images, 1000 classes).
Further, our budgeting procedure is not specific to t-SNE, but use any dimensionality reduc- tion technique (Maćkiewicz & Ratajczak] 1993). For instance, Uniform Manifold Approximation and Projection (UMAP) (Mclnnes et al.,, 2018) requires approximately 1 minute for the CIFAR- 10 dataset, 2 minutes for CIFAR-100, and around 23 minutes for ImageNet to generate embeddings (see Tabs. [7 18 in $E.8 for a t-SNE vs UMAP ablation study). This is including the overlap computation which only takes a few seconds and is efficiently parallelised across 150 CPU cores.
3.2 LOSS FUNCTION
A random-set prediction problem is mathematically similar to the multi-label classification problem, for in both cases the ground truth vector contains several Is. In the former case, these correspond to sets all containing the true class; in the latter, to all the class labels attached to same data point. Despite the different semantics, we can thus adopt as loss Binary Cross-Entropy (BCE) (5) with sigmoid activation, to drive the prediction of a belief value for each focal set in the identified budget:
beize
CBC
Σ O
Bel, log( (A), + (1 Bel, log(1 - (A))
bosse i=1
AEO
(5)
Here, is the index of the training point within a batch of cardinality baizes A is a focal set of classes in the budget O, (A) is the A-th component of the vector bel, encoding the ground truth belief values for the i-th training point, and (A) is the corresponding belief value in the predicted vector bel, for the same training point. Both bel, and bel, are vectors of cardinality [O] for all i. A valid belief function satisfies Eq. [ which states that mass values derived from belief functions should be non-negative and should sum up to (Shafer 1976). To ensure this, we incorporate a mass regularization term M, and a mass sum term M, in the loss function:
beise
buisc
M,
max (A)), M. = max 0,
Σ
(m((A)-))
(6)
Dsize i=1 AEO
baize
1=1
AEO
M, encourages non-negativity of the (predicted) mass values ri)A), A These mass values are obtained from the predicted belief function bel via the Moebius inversion formula (Fq. 2 For it to be valid, the sum of the masses of the predicted belief function must be equal to 1 (Eq. D. which is encouraged by the mass sum term M.. All loss components LBCE. M, and M, are computed during batch training. Two hyperparameters, and B, control the relative importance of the two mass terms, yielding as the total loss for RS-NN:
LRS = LBCE + aM, + 8M,.
(7)
The regularisation terms aim to penalise deviations from valid belief functions, in line with, e.g., the way training time regularisation in neurosymbolic learning encourages predictions to be common- sense (Giunchiglia al. (023). In general, soft constraints (Márquez-Neila shown to not be inferior to hard ones. Still, when a and 3 are too small, this may not be sufficient to ensure that predictions are valid belief functions. In such cases P post-training, we set any negative
al., 2017) have been
2 At any rate, improper belief functions are normally used in the literature Denoeux 2021 e.g. for condi- tioning (uzzzolin 2020).

---

Published as a conference paper at ICLR 2025
masses to and add the 'universal" set of all classes to the final budget. This subset is assigned all the remaining mass, ensuring that the sum of masses across all focal sets in O equals 1. This approach mimics classical approximation schemes (e.g. Cuzzolin (2220), Part III..
3.3 ACCURACY AND UNCERTAINTY ESTIMATION
Pignistic prediction. The pignistic probability (Eq. 3 is the central prediction associated with belief function (seen as a credal set): standard performance metrics such as accuracy can then be calculated after extracting the most likely class according to the pignistic prediction (e.g. in (E.5.I). Notably, the RS-NN architecture and training mechanism are designed to facilitate set- based learning by incorporating class set information during training. When pignistic probabilities computed during inference, they reflect the learning derived from the masses and beliefs of various subsets, making it more reliable than traditional softmax probabilities, as shown in $E.2
Entropy of the pignistic prediction. The Shannon entropy of the pignistic prediction BetP can then be used as a measure of the uncertainty associated with the predictions, in some way analogous to the entropy of Bayesian mean average prediction:
HRS E BelP(c) log BelP(c).
A higher entropy value (8) indicates greater uncertainty in the model's predictions.
Size of the credal prediction. As discussed above, and further elaborated upon in SE.3.3 sensible measure of the epistemic uncertainty attached to random-set prediction bel is the size of the corresponding credal set. Several ways exist of measuring the size of a convex polytope such as a credal set (Sale et al. 2023). Given the upper and lower bounds to the probability assigned to each class by distributions within the predicted credal set Cre (Eq. 4
P(c) max P(c), P(c)
min P(c).
PeCre propose a simple way to measure the size of the credal set the difference between the lower and upper bounds (Eq. (9) associated with the most likely class, according to the pignistic prediction. Note that the predicted pignistic estimate Pet falls within the interval [L(c), P(c)] for each
PeCre
class c, not just for the most likely class ė. Its width P(c) P(c) indicates the epistemic uncertainty associated with the prediction (AA.)).
4 EXPERIMENTS
4.1
IMPLEMENTATION
Datasets. Our experiments are performed on multi-class image classification datasets, including MNIST (LeCun & Cortes 2005), CIFAR-10 (Krizhevsky et al. 2209), Intel Image (Bansal 2019).
CIFAR- 100 (Krizhevsky) 2012), and ImageNet (Deng et al.) 2009). For out-of-distribution (OoD) experiments, we assess several in-distribution (iD) vs OoD datasets: CIFAR-10 vs SVHN (Net- zer et al. 2011//ntel-Image (Bansal) 2019). MNIST vs F-MNIST (Xiao et al. 2017MK-MNIST (Clanuwat al. 2018). and ImageNet vs ImageNet-O (Hendrycks et al.J 2021). The data is
split into 40000: 0000: 10000 samples for training, testing, and validation respectively for CIFAR- 10 and CIFAR- 100, 10000: 10000 samples for MNIST, 13934:3000: 100 for Intel Image, 1172498:50000:100 10866 for ImageNet. For OoD datasets, use 10,000 testing samples, except for Intel Image (3,000) and ImageNet-O (2,000). Training images are resized to 224 x 224 pixels.
Baselines, backbone and training details. Our baselines include state-of-the-art Bayesian meth- ods L.BBNNN (Hobbhahn et al.) 2022) and FSVI (Rudner et al. 2022). Ensemble classifiers DE (5 ensembles) (Takshminarayanann et al.] (017)) and ENN (3 ensembles) (Osband et al., 2024), and standard CNN (see Sec. 4.2 Our bascline models are similar to those used in the most recent work (Cinquin& & Bamler Daxberge et al. 2021 Wu & Williamson 2024) in uncertainty estima- tion. All models, including RS-NN, are trained ResNet50 (on NVIDIA A 1100 80GB GPUs) with a learning rate scheduler initialized at le-3 with 0.1 decrease at epochs 80, 120, 160 and 180. Stan- dard data augmentation (Krizhevsky et al. 2012), including random horizontal/vertical shifts with magnitude of 0. and horizontal flips, is applied to all models.
ResNet50, excluding the top classification layer, serves as the common architecture. ResNet was originally designed for classification on the ImageNet dataset (1000 classes). To accommodate reduced number of classes as in smaller datasets (c.g., CIFAR- 10/MNIST with 10 classes, Intel Image with 7 classes, and CIFAR-100 with 100 classes), two additional dense layers (1024 and 512
7

---

Published as a conference paper at ICLR 2025
Table I: Test accuracies (%) and inference time (ms) for uncertainty estimation over 5 consecutive
runs across methods and datasets. Average and standard deviation are shown for each experiment.
Daaneets
MNNST
CHIAR-10
Intel Image
CHFAR-100
ImageNerr (lop-1)
ImageNet (Top-5)
Inference time (mR)
RS-AN (ours)
90. 71 + 0.08
95.53 + 0.09
96.22 + 0.08
71.81 + 00007
79.92
94.47
1.91 0.02
LB-BNN Habdlnae al 2022
50.56
2: 2.0.4
89.35 = 0.8t
50.49 :: 0.42
39.50 1.96
72.48
90 5G
7.11 # 0.80
ESVL
50.15 2.0.0
80.29 0.00
58.92 a 0.33
58.34 0.00
62.56
84.00
860.26 d. 0.76
DE
2017
50.25 0.00
22.73 0.94
91.38
1 0.1:1
70.53 0.07
78.77
94.37
19163.30 = 3.37
ENW
50,07 1.0.1
91.55 0.00
91.49
0.399
08.02 0.25
71.82
89.48
3.10 a: 0.03
CNN
50.12 1 0.04
92.08
1 0.44
90.89 1 0.30
05.50 0.08
78.56
04.34
1.91 0.03
neurons, ReLU activation) are added to ResNet50. Similar techniques are commonly applied in
deep learning to adapt model architectures to datasets (He et al., (016;] (Zagorukko & Komodakis, 2016). The output layer of RS-NN on ResNet50 has the same number of units as the number of (selected) focal sets I and uses sigmoid activation, since the ground truth encoding resembles multi-label classification problem (see Sec. 3.2). For all other models, the final output layer simply consists of softmax activation for multi-class classification. All the models were trained from scratch for 200 epochs (recommended by most), with a batch size of 128. Our objective is to ensure a fair comparison across all models for all experiments. Tuning each model separately to maximise performance would not guarantee that, as some models use pre-trained weights while others train for larger number of epochs, which could result in the over-training of another model. We used pre- trained weights for all models only when trained on ImageNet for efficiency. For all other training hyperparameters (e.g, optimizer), we use what is specified for each model in their respective papers (more in SD Tab. 3. Consequently, the performance metrics reported in the original papers may vary from those presented here, as they were obtained under different training conditions.
Training the RS-NN. In the clustering phase, we obtain features directly from a pre-trained ResNet50 classifier. We use 50 CPU cores for t-SNE dimensionality reduction, and 150 CPU cores for computing the overlap of classes. We set a budget K of 20 focal sets (ablation study on K in (E.7) for CIFAR-1O1 10/ MNIST/ Intel Image, 200 for CIFAR- 100 and 3000 for ImageNet. RS-NN is trained from scratch on ground-truit truth belief encoding of sets using the Lns loss function (Eq. 7 over 200 epochs, with a batch size (bsize) of 128 and a = 3 = le 3 as hyperparameter values.
Experiments. Firstly, we assess the test accuracy (%) and inference time (ms) of all baselines on all the multi-class classification datasets. Tab. provides a comparison of test accuracies and inference times of RS-NN against the baselines. Our second set of experiments (Sec. 4.3) concerns out-of- distribution (OoD) detection. Tab. 2 shows our results on OoD detection metrics AUROC (Area Under Receiver Operating Characteristic curve) and AUPRC (Area Under Precision- Recall curve) for all the models on the iD vs OoD datasets listed above. We also report the Expected Calibration Error (ECF) for all models on datasets CIFAR-10. MNIST and ImageNet (see Tab. (). In third set of experiments (Sec. 4.4]. we test the uncertainty estimation capabilitics of RS-NN using both the entropy of the pignistic prediction and the size of the predicted credal set. Tab. 2 presents the predicted pignistic entropy of RS-NN alongside entropies of all baselines on iD and OoD datasets. Tab. shows credal set widths for RS-NN predictions across the same datasets. In our final set of experiments, we explore the scalability of RS-NN (Sec. (.5) to large-scale architectures, employing it on models such as WideRessNet-2S-10 10, VGG16, Inception V3, EfficientNetB2 and ViT-Base-16. Further, to underscore the model's ability to leverage transfer learning, we train and test on pre- trained ResNet50 model initialised with ImageNet weights (Tab.
Additional experiments concern obtaining statistical guarantees for RS-NN using non-conformity
scores A) evaluating the robustness of RS-NN to adversarial attacks (...4). noisy and rotated in- distribution samples (Appendix $E.3). showing how RS-NN circumvents the overconfidence prob- lem in CNNs (EE.)). Results showing how credal set width is less correlated with confidence scores than entropy are discussed in in $E.5.3 We also conduct ablation studies on a and ($E.6) and the
number of non-singleton focal sets K ( EE.7 which show that the best accuracy is obtained at small
values (le- 3) of and B, and for K 20 (on the CIFAR-10 10 dataset).
4.2 COMPARISON WITH THE OFATE-OF- ON ACCURACY
Tab. [ shows that RS-NN outperforms state-of-the-art Bayesian (LB-BNN, FSVI) and Ensemble (DE, ENN) methods in terms of test accuracy (%) across all datasets. Experiments on inference time (ms) per sample, conducted over 5 runs (a single forward propagation) on the CIFAR-10 dataset, show that RS-NN and CNN have the fastest inference times among all models (Tab. [] Albeit CNN and DE are close to RS-NN in performance (e.g., on ImageNet), CNN tends to be over- confident in incorrect predictions (S..2). while DE has significantly longer training (Tab. 6 $D) and inference times (Tab. D] The pignistic predictions derived from predicted belief functions are more reliable and accurate than predicted softmax probabilities of CNN and DE. It is important to
8

---

Published as a conference paper at ICLR 2025
Table 2: OoD detection performance and uncertainty estimation for models trained on ResNet50 on CIFAR-10 10 vs SVHN/Intel Image, MNIST vs -MNIST and ImageNet vs ImageNet- O. Evaluation metries include AUROC/AUPRC (OoD); Entropy of predictions (uncertainty) and Expected Calibration Error (ECE).
Indiooittaionn (i7)
Olt Oft-di-dionteem (OuDh)
Dutset
Model
Uscattaisty
Estrogy
BCE (1)
AUROCC
AUPPCC (f)
Formay (f)
AUROCC (19
AUPRCC (1)
Image
Fermyy (f)
RS-NS
53.55
Pymidi: uneapy
0,088
0.0684
91.91
1.152
8.8
1.537 0.7.00
Estropy
ESVI
501.9
Prodecive
0.563
0.0001
50.50
0.43 =
0.250 +000000000 07077
CIPAR-10
DE
92.73
Mean Ermog
0.154
2:24:-000 0.367
0.0453
58.94
$1.58
0.930 = 0.554
94.55
76.36
1.316 2 0.555
BXN
91.55
Mese Exines
0.125-5
0.523
0.3698
52:20 13.70
$5.05
0.867
2 0.814
66.60
58.00
0.000
2 0.378
CNN
92.00
Soltmat Encopy
0.304
58.11
01.3
0.050 0.430
$7.70
$6.04
0.739
0.475
FIMNIST
K-MINITT
RS-NK
09:71
Pigsidic craops
0U010
8:111
0,0029
93.89
93.08
0.550 22 0.777
96.75
98.65 68
0.740 2: 0.017
BNN
39.58
Predictive Easooyy
0.001
0:3082
5016
50..0
0.442
35.01
56.05
Hasmyy
MINSS
DE
59.23
Mean Frrney
0.031
0.155
0.0001
69.50
0.581
95.81
0.554 017155
BKN
30.07
Mexs Erroge
IUI22
0.127
0.3001
51.70
52.00
0.318 = 00.664
20.94
50.65
0.538 = 0.677
CNN
95.90
Softnax Entay
IL025
8 0.135
0,2002
58.77
24.14
0.275 = 00.26
34.00 05
58.04
0.616 5 0..55
Imeog//res11
AUROC
AUPRI
Eanogy 3,675 = 8.77)
RS-NN:
79.02
Pigsidt: emooy
2 +: 105
0.1416
00.35
65. 16
LB-BNN
72.65
Prodictive Entroyy
2.03
0.5812
41.05
30.00
2: 0.125
ImageNet
DE
77.77
Mean Eraops
1215
0.1040
50.37
1.313
BNCN
71.57
Mean Errmps
1 4-1 1-15119
0.5161
54.07
13 T3
1.617 +: 1.507
CNN
78.00
Suftnax Estroyy
0.986 6: 1.385
0.4064
54.25
48.77
0,575 :: 11.512
note that the comparison with standard CNN is not based on its role as an uncertainty method, but as benchmark, underscoring RS-NN's effectiveness in achieving high accuracy compared to other uncertainty baselines. Test accuracy for RS-NN is calculated by determining the class with the high- est pignistic probability for each prediction and comparing it with the true class. This proves that, by modelling the epistemic uncertainty about the prediction, we take better into account possible distribution shifts at test time as a result, the central prediction of the set of probabilities (credal set) associated with the predicted belief function is more likely to be closer to the ground truth. Note that the results in the FSVI paper (Rudner et al., 2022) are based on ResNet18. Although using a pre-trained ResNer50 could enhance FSVT's performance, it would provide unfair advantage over the other baseline models in our paper, which were all trained from scratch.
4.3 OUT-OF-DISTRIBUTIIN (OoD) DETECTION
We evaluate our out-of distriddittiittinnnn (OoD) detection (Tab. 2 against baselines using AUROC and AUPRC scores. OoD detection identifies data points deviating from the in-distribution (iD) train- ing data by measuring true and false positive rates (AUROC) and precision and recall trade-offs (AUPRC), indicating the model's ability to handle unfamiliar data (further detailed in CC.3) As shown in Tab. 2 RS-NN greatly outperforms Bayesian, Ensemble and standard CNN models in OoD detection, with significantly higher AUROC and AUPRC scores for all iD vs OoD datasets, es- pecially on difficult datasets like ImageNet and ImageNet-O (see $E.II. Even high-accuracy models like CNN and DE struggle to differentiate between ImageNet and ImageNet-O. While DE shows comparable AUROC scores on CIFAR-I0 vs. SVHN and MNIST vs. LAMNIST/K-MNIS" to RS- NN, it has lower AUPRC. While AUROC assesses a model's 's ability to distinguish between iD) and OoD samples, AUPRC measures both the precision of correctly identified OoD samples and the recall of detected OoD samples, making it more informative metric. Tab. also reports the various models' Expected Calibration Error (ECE) (detailed in $C.2). A low ECE indicates that the model's s confidence scores align closely with the actual likelihood of events. RS-NN exhibits the lowest ECE indicating cell-calibrated probabilistic predictions.
4.4 UNCERTAINTY ESTIMATION
Entropy. Tab. 2 shows the mean and variance of the entropy distributions for both iD and OoD datasets. Lower entropy values for iD datasets indicate a confident prediction as the model is trained on familiar data, while higher entropy for OoD datasets reflects the model's uncertainty with unseen data. RS-NN exhibits both low entropy for iD datasets and high entropy for OoD ones. The percent- age mean iD/OoD shift in entropy is most pronounced in RS AS-,N, especially evident in CIFAR-10 10 vs SVHN/Intel Image and MNIST vs F-MNIST/K-MNISTT. For ImageNet vs ImageNet-O, CNN exhibits high entropy for both iD and OoD datasets while RS-NN maintains desirable iD vs OoD entropy ratio. In fact, Fig. [8] [E.. show that RS-NN has the best iD vs OoD entropy ratio for all datasets. This ideal behavior, along with superior OoD detection (Sec. 4.3), is a good indication that RS-NN is better at estimating uncertainty induced by domain shift (see $E.5.I).
Credal set width. Fig. shooss a density plot of estimated credal set widths, the difference between the lower and upper probability bounds in Eq. 9 for correctly and incorrectly classified samples of
9

---

Published as a conference paper at ICLR 2025
Table 3: Credal set width for RS-NN on iD vs OoD datasets: CIFARIO vs SVHN/Intel Image, MNIST vs F. MNIST/K-MNIST and ImageNet vs ImageNet-O.
Dersily Credal Sei Wish Correct and Incereed Classfiiations
incoriectly Classifed Coreccil Chasiffed
In-disribution (ID)
Off-o-diatibuution (OnD)
CIFARIO 0.007 1.0044
SVIEN
0.260 1 0.322
Ineel Image
0.557 0.366
MNIST 0.001 + +00.33
F-MNIST
0.070 + 067
K-MNIST 0.103 + 0.200
ImageNer 0.238 + 0.266
ImageNes-O
0.272 + 0.276
Table 4: Adaptability to large-scale model architectures with test accuracy (%) and parameters (in million) re-
ported on CIFARIO.
Srodal Wid
Morkd
Profiiined
WRN-25-10
IncepV3
ENc:B2
Wi--Bamc
Test
94.08
92.79
Figure 6: Width of credal predictions for CIFAR-IO test data (correctly clas-
Params (M)
CNN
RS-NN
2,60
7.47
30.7
$7.0
15.11
15.12
31.21
81.22
7.71
7.72
D.5
852
sified, blue; incorrectly classified, red).
CIFAR-I0 test data. Incorrect predictions (red) correlate with larger credal set widths, indicating higher epistemic uncertainty, and are more dispersed in the plot. Correct predictions (blue) concen- trate around smaller values, exhibiting smaller credal intervals as expected. Tab. reports the credal set widths of RS-NN predictions for iD vs OoD datasets. Larger intervals can be observed for OoD datasets. The credal set widths for ImageNet vs ImageNet-O are not as distinct given the nature of these datasets, as detailed in 1 Note that credal set width is not directly comparable to the entropy, variance or mutual information of other models, as such metries have distinct semantics.
4.5 SCALABILITY TO LARGE-SCALE ARCHITECTURES
Tab. shows the scalability of RS-NN to larger model architectures and its ability to leverage transfer learning. RS-NN outperforms standard CNNs across various large-scale model architec- tures, including WideResNet28- 10 (WRN-28- 10), VGG16, Inception (In, EfficientNetB2 (ENetB2), and Vision Transformer (ViT-Base- 16), highlighting its versatility and ease in adopting different architectures, and the generality of the random-set concept. Notably, using a pre-trained ResNet50 (Pre-trained R50) model with ImageNet weights, RS-NN achieves higher accuracy on CIFAR-10 10 compared to using only the architecture (no pre-trained weights).
4.6 LIMITATIONS
The budgeting step can be time consuming but, for a given dataset, it is one-time procedure prior to training. For large datasets, t-SNE and GMM can be applied to representative samples rather than the entire dataset, the purpose is merely to identify the most relevant sets, dramatically reducing computational demands. To further optimise the method, instead of t-SNE, budgeting can utilize various methods such as autoencoders (Ghasedi Dizaji et al., 2017 Guo et al., 2017b) or PCA (Abdi) & Williams] 2010) for dimensionality reduction. However, despite the initial time investment, the efficiency gained from having a budget applicable to any training scenario is substantial. Another limitation is manually setting the number of focal elements K; a dynamic strategy adjusting K based on overlap would enhance flexibility and effectiveness. The budgeting procedure can also be adapted to work with multiple sources of data or a stream of data ($E.8). Alternative methods such as using sparse mass functions (Itkina et al., 2020. Chen et al. 2021). could provide a quantitative basis for determining the subsets Ag to consider.
CONCLUSION
This paper proposes novel Random-Set Neural Network (RS-NN) for uncertainty estimation in clas- sification predicting belief functions. Our random-set representation is foundational approach that acts as a versatile wrapper, applicable any model architecture and classification task (e.g, text). To our knowledge, this concept, along with budgeting for optimal selection of sets, is unprecedented. RS-NN outperforms state-of-the-an uncertainty estimation models and the standard CNN in per- formance and out-of-distribution OoD tests, and scales seamlessly to large-scale architectures and datasets. Given such results, our approach exhibits significant potential impact for safety-critical applications such medical diagnostics and autonomous driving, where uncertainty estimation and OoD detection is crucial. Future work will explore alternative methods for ensuring the validity of belief functions without mass regularisation terms, such integrating Semantic Probabilistic Layers (Ahmed et al., 2022) to enforce logical constraints in probabilistic predictions, or applying softmax over the computed masses guarantee non-negativity (see (..6).. A parameter-level representation of RS-NN and the extension of this concept to regression will also be explored.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
To ensure reproducibility, provide detailed information the datasets, baseline models, and the training procedure for RS-NN and baselines used in our experiments in Sec. 4.1 under the subheadings 'Datasets' and "Baselines, backbones, and training details' respectively. Additional hyperparameters for RS-NN are provided under "Training the RS-NN" (Sec. [4.1]. and for baseline models in $D For convenience,
summarize the training hyperparameters for all models in Tab. 5 Additionally, the training times for each model are included in Tab. ( The code for training RS-NN (train. pynb), including pre-trained models (for faster evaluation), evaluation script deval.ipynkl and configuration files are provided as supplementary materials. We also provide yml file set up the environment to run the experiments.
ACKNOWLEDGEMENT
This work has received funding from the European Union's Horizon 2020 Research and Innovation program under Grant Agreement No. 964505 (E-pi).
REFERENCES
Hervé Abdi and Lynne Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459, 2010.
Taiga Abe, Estefany Kelly Buchanan, Geoff Pleiss, Richard Zemel, and John P Cunningham. Deep ensembles work, but they necessary? Advances in Neural Information Processing Systems, 35:33646-33660, 2022.
Charu Aggarwal, Xiangnan Kong, Quanquan Gu, Jiawei Han, and S Yu Philip. Active learning: A survey. In Data Classification: Algorithms and Applications, pp. 371-605. CRC Press, 2014.
Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van den Broeck, and Antonio Vergari. Seman- tic probabilistic layers for neuro-symbolic learning. Advances in Neural Information Processing Systems, 35:29944-29959, 2022.
Alexander A Alemi, Ian Fischer, and Joshua V Dillon. Uncertainty in the variational information bottleneck. arXiv preprint arXiv: 1807,00906, 2018.
Anastasios N Angelopoulos and Stephen Bates. A gentle introduction conformal prediction and distribution-fre uncertainty quantification. arXiv preprint arXiv:2107.075// 2021.
Alessandro Antonucci and Giorgio Corani. The multilabel Naive Credal Classifier. International
Journal of Approximate Reasoning, 83:320-336, 2017. ISSN 0888-613X. doi: https://doi.
org/10. 1t1.1110166..jaar2211..000 10.000. URL https: //www///wwwwww wwwencedirect....m..mm cot/ crom/science)
atticlee/p1//008811113338.
Alessandro Antonucci and Fabio Cuzzolin. Credal sets approximation by lower probabilities: appli- cation to credal networks. In Computational Intelligence for Knowledge-Bass Based Systems Design: 13th International Conference on Information Processing and Management of Uncertainty, IPMU 2010, Dortmund, Germany, June 28-July 2, 2010. Proceedings 13, pp. 716-725. Springer, 2010.
Thomas Augustin, Frank PA Coolen, Gert De Cooman, and Matthias CM Troffaes. Introduction to imprecise probabilities, volume 591. John Wiley & Sons, 2014.
Vineeth Balasubramanian, Shen-Shyang Ho, and Vladimir Vovk. Conformal prediction for reliable machine learning: theory, adaptations and applications. Newnes, 2014.
Punect Bansal. Intel image classification. Available on https://www. kaggle. com/punerr6000fmmtll classffictin,,,,,nnnn, Online, 2019.
Daniel Barbara and Ping Chen. Tracking clusters evolving data sets. In FLAIRS, pp. 239-243, 2001.
11

---

Published as a conference paper at ICLR 2025
Jonathan Baron. Second-order probabilities and belief functions. Theory and Decision, 23:25-36, 1987.
Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Pitfalls of epistemic uncertainty quan- tification through loss minimisation. Advances in Neural Information Processing Systems, 35: 29205-29216, 2022.
Andrey Bronevich and George Klir. Axioms for uncertainty measures on belief functions and credal sets. In NAFIPS 2008-2008 Annual Meeting of the North American Fuzzy Information Processing Society, pp. 1-6. IEEE, 2008.
Wray L. Buntine and Andreas Weigend, Bayesian broo-pronnnntion. Complex Syst., 5, 1991. URR.http:://ap www.stmiiiiii..o. cccholschhhaal ar org/CorpusTD: 14814125
Thomas Burger, Oya Aran, and Alice Caplier. Modeling hesitation and conflict: a belief-based approach for multi-class problems. In 2006 Sth International Conference on Machine Learning and Applications (ICMLA'00) pp. 95-100. IEEE, 2006.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts. Advances in neural information 2020.
processing systems, 33: 1356-136111
Alain Chateauneuf and Jean- Yves Jaffray. Some characterizations of lower probabili- ties and other monotone capacities through the use of Möbius inversion. Mathemati- cal Social Sciences, 17(3):263-283,
1989. ISSN 0165-4896. doi: https://oi..og
101600165-4496618999000555 brt.cleep11/011545666
URL https://wwwt www.eceeeeeeeeeeeeooooeeeeeeeeee rect. com/ http/scient science/
Phil Chen, Mikhal Itkina, Ransalu Senanayake, and Mykel J Kochenderfer. Evidential softmax for sparse multimodal distributions in deep Generative models. Advances in Neural Information Processing Systems, 34:1 1565-11576, 2021.
Tristan Cinquin and Robert Bamler. Regularized kl-divergence for well-defined function-space vari- ational inference in bayesian neural networks. arXiv preprini Sttt:::20060431117 2024.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical Japanese literature. arXiv preprint arXiv: 2018.
Giorgio Corani and Marco Zaffalon. Learning reliable classifiers from small or incomplete data sets: The naive credal classifier 2. Journal of Machine Learning Research, 9:581-621, 04 2008. doi: 10.1145/1 145/1390681 1390701.
Fabio Cuzzolin. On the credal structure of consistent probabilities. In European Workshop on Logics in Artificial Intelligence, pp. 126-139 139. Springer, 2008.
Fabio Cuzzolin. Credal semantics of Bayesian transformations in terms of probability intervals.
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 40(2):421-432, 2010.
Fabio Cuzzolin. Visions of a generalized probability theory. arXiv preprint arXiv:1810.10334 2018.
Fabio Cuzzolin. The Geometry of Uncertainty: The Geometry of Imprecise Probabilities. Ar- tificial Intelligence: Foundations, Theory, and Algorithms. Springer International Publish- ing, 2020. ISBN 9783030631536. URL pettps://books http:/ooooooollllloo.//// sccoglee.c..ut//ooks2i...
Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless Bayesian deep learning. Advances in Neural Informa- tion Processing Systems, 34:20089-20103, 2021.
Arthur P Dempster. Upper and lower probabilities induced by a multivalued mapping. In Classic works the Dempster-Shafer theory of belief functions, pp. 57-72. Springer, 2008
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255. leee, 2009.
12

---

Published as a conference paper at ICLR 2025
Thierry. Denoeux. A neural network classifier based Dempster- Shafer theory. JEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 30(2):131-150, 2000. doi: 00009//468.8330994
Thierry Dencexx Constructing belief functions from sample data using multinomial confidence regions. International Journal of Approximate Reasoning, 42(())228-255) 2006.
Thierry Denoeux. A k-nearest neighbor classification rule based on Dempster-Shafes theory. In Roland Yager and Liping Liu (eds.), Classic Works of the Dempster-Siee Shafer Theory of Belief Functions, volume 219 of Studies in Fuzziness and Soft Computing, pp. 737-760. Springer, 2008.
Thierry Denocux. Distributed combination of belief functions. Information Fusion, 65:179-191,
2021.
Thierry Dencex. An evidential neural network model for regression based on random fuzzy num- bers. In International Conference Belief Functions, pp. 57-66. Springer, 2022.
Javier Diaz-Rozo, Concha Bielza, and Pedro Larrañaga. Clustering of data streams with dynamic gaussian mixture models: An iot application in industrial processes. IEEE Internet of Things Journal, 5(5):3533-3547 2018.
Didier Dubois and Henri Prade. Consonant approximations belief functions. International Jour- nal of Approximate Reasoning, 4:419-449, 1990.
Zied Elouedi, Khaled Mellouli, and Philippe Smets. Decision trees using the belief function theory. In Proceedings of the Eighth International Conference on Information Processing and Manage- ment of Uncertainty in Knowledge-based Systems (IPMU 2000), volume pp. 141-148 Madrid, 2000.
Zied Elouedil, Khaled Melloulil, and Philippe Smets. Classification with belief decision trees. In International Conference on Artificial Intelligence: Methodology, Systems, and Applications, pp. 80-90. Springer, 2000.
Rebecca Fay, Friedhelm Schwenker, Christian Thiel, and Günther Palm. Hierarchical neural net- works utilising dempster-shafer evidence theory. In Friedhelm Schwenker and Simone Marinai (eds.), Artificial Neural Networks in Pattern Recognition, pp. 198-209, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-37952-2.
Vincent Fortuin. Priors in Bayesian Deep Learning: A Review. International Statistical Review, 90, 05 2022. doi: 10.11 1/insr. 12502.
David Freedman. Wald lecture: On the Bernstein-von Mises theorem with infiniteddimensionall parameters. The Annals of Statistics, 1714):1119-114 1999.
Yarin Gal and Zoubin Ghahramani. Bayesian convolutional neural networks with Bernoulli approx- imate variational inference. urXiv preprint arXiv: /50Xvv:5506022 1506.02 058, 2015.
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation:
Representing model
uncertainty in deep learning. In International Conference Machine Learning, pp. 1050-1059. PMLR, 2016.
Junyu Gao, Mengyuan Chen, Liangyu Xiang, and Changsheng Xu. A comprehensive survey on evidential deep learning and its applications. arXiv preprint arXiv:2409.04722 2024.
Stefano Gasperini, Jan Haug, Mohammad-AR Nikouci Mahani. Alvaro Marcos-Ramiro Nassir Navab, Benjamin Busam, and Federico Tombari. Certainnet: Sampling-free ffree uncertainty esti- mation for object detection. IEEE Robotics and Automation Letters, 7(2):698-705, 2021.
Yonatan Geifman and Ran El-Yaniv. arXiv:1005.08500, 2017.
Selective classification for deep neural networks.
Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference computer vision, pp. 5736-5745, 2017.
13

---

Published as a conference paper at ICLR 2025
Elconora Giunchiglia, Mihaela Cätălina Stoian, Salman Khan, Fabio Cuzzolin, and Thomas Lukasiewicz. Road-r: The autonomous driving dataset with logical requirements. Machine Learn- ing. 12(0))3221-329) 2023.
Wenjuan Gong and Fabio Cuzzolin. A belief-theoretical approach to example-based pose estimation. IEEE Transactions on Fuzzy Systems, 26(2):598-611, 2018. doi: 10.1 10.110//FFU2222177.2666666
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 142.:544,,,6522, 2014.
Andreas Graefe, Helmut Küchenhoff, Veronika Stierle, and Bernhard Riedl. Limitations of ensem- ble Bayesian model averaging for forecasting social science problems. International Journal of Forecasting, 31(3):943-95,, 2015.
Derek Greene and Pádraig Cunningham. Practical solutions to the problem of diagonal dominance in kernel document clustering. In Proc. 23rd International Conference on Machine learning (ICML'06), pp. 377-384. ACM Press, 2006.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017a.
Xifeng Guo, Xinwang Liu, En Zhu, and Jianping Yin. Deep clustering with convolutional au- toencoders. In Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Purt Il 24, pp. 373-382. Springer, 2017b.
Joseph Y. Halpem. Reasoning About Uncertainty. MIT Press, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adver- sarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15262-15271, 2021.
Max Hinne, Quentin F Gronau, Don van den Bergh, and Eric-Jan Wagenmakers. A conceptual introduction Bayesian model averaging. Advances in Methods and Practices in Psychological Science, 3(2):200-215, 2020.
Marius Hobbhahn, Agustinus Kristiadi, and Philipp Hennig. Fast predictive uncertainty for clas- sification with Bayesian deep networks. In Uncertainty Artificial Intelligence, pp. 822-832.
PMLR, 2022.
Jennifer A Hoeting, David Madigan, Adrian E Raftery, and Chris T Volinsky. Bayesian model averaging: a tutorial (with comments by m. clyde, david draper and george, and a rejoinder by the authors. Statistical science, 14(4):382-417, 1999.
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning. 1 1103))457--06, 2021.
Masha Itkina, Boris Ivanovic, Ransalu Senanayake, Mykel J Kochenderfer, and Marco Pavone. Evidential sparsification of multimodal latent spaces in conditional variational autoencoders. Ad- vances in Neural Information Processing Systems, 33:1 13:15535-10246 2020.
Laurent Valentin Jospin, Hamid Laga, Farid Boussaid, Wray Buntine, and Mohammed Bennamoun. Hands-on bayesian neural networks tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2):29 48, 2022.
Alex Kendall and Yarin Gal. What uncertainties do need in Bayesian deep learning for computer vision? 103.04770.. 2017.
David G. Kendall. Foundations of a theory of random sets. In F. Harding and D. G. Kendall (eds.), Stochastic Geometry, pp. 322-376. Wiley, London, 1974.
14

---

Published as a conference paper at ICLR 2025
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local Reparameteri- zation trick. Advances in Neural Information Processing Systems, 28, 2015.
Andrei N Kolmogorov. Three approaches to the quantitative definition of information. Problems of information transmission, 1(1):1-7, 1965.
Andrei N. Kolmogorov and Albert T. Bharucha-Res Foundations of the theory of probability: Second English Edition. Courier Dover Publications, 2018.
Anna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, and Stephan Günnemann. Evaluating robustness of predictive uncertainty estimation: Are dirichlet-based models reliable? In International Conference on Machine Learning, pp. 5707-5718. PMLR, 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 2012.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. CIFAR-10 (Canadian Institute For Ad- vanced Research). Technical report, CIFAR, 2009. URL htt.h:ttt/:/wwwt wttp:/wwww.....ooooee..eeooo. http://wwwww.....ooooooooooo
"kriz/cffar html
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25, Curran Associates, Inc., 2012. URL
neurpps cc paper. files/paper/2012/ File/ c39 399 F12-3399825555677777555242666333
Brian Kulis and Michael I Jordan. Revisiting k-means: New algorithms via bayesian nonparamet- rics. arXiv preprint arXiv: /eXiv://11.000 1.00322, 2011.
Hicham Laanaya, Arnaud Martin, Driss Aboutajdine, and Ali Khenchaf. Support vector regression of membership functions and belief functions Application for pattern recognition. Information Fusion, (144)3338-333 350, 2010.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predic- tive Uncertainty Estimation using Deep Ensembles. Advances in Neural Information Processing Systems, 30, 2017.
Yann LeCun and Corinna Cortes. The MNIST database of handwritten digits. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2005. doi: 10.
2009/CVR..205..1111 URL htttttttt:///eeee http:///eeeeeppppoc ieee ieee. http:/ooo//ooooooooeeeeeoo (14672222
Isaac Levi. The enterprise of knowledge: An essay knowledge, credal probability, and chance.
The MIT Press, Cambridge, Massachusetts, 1980.
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in neural information processing systems, 33:7498-7512 2020.
Zhunga Liu, Jean Dezert, Grégoire Mercier, and Quan Pan. Belief C-means: An extension of fuzzy C-means algorithm in belief functions framework. Pattern Recognition Letters, 33(3):291-300, 2012.
David J. C. MacKay, A Practical Bayesian Framework for Backpropagation Networks. Neural Computation, 4(3):488 472, 05 1992. ISSN 0899-7667. doi: (1t.::222mccco1992.444444 URL
https://doi org. 166//ecc..... 1992.4.3 448
Andrzej Mačkiewicz and Waldemar Ratajczak. Principal components analysis (pea). Computers & Geosciences, 19(3)3303-342, 1993.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.
Towards deep learning models resistant adversarial attacks. stat, 1050(9), 2017.
15

---

Published as a conference paper at ICLR 2025
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018.
Andrey Malinin and Mark Gales. Reverse KI.-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
Andrey Malinin, Bruno Mlodozeniec, and Mark Gales, Ensemble distribution distillation. In Inter- national Conference on Learning Representations, 2019.
Shireen Kudukkil Manchingal and Fabio Cuzzolin. Epistemic deep learning. arXiv preprint 27X10 07600,66099 2022.
Shireen Kudukkil Manchingal, Muhammad Mubashar, Kaizheng Wang, and Fabio Cuzzolin. A unified evaluation framework for epistemic predictions. The 28th International Conference on Artificial Intelligence and Statistics, 2025. URL httph: /tttps::/openreview. www.:enee/eeeeee fortmfor
kXCOsdF8KN
Pablo Márquez-Neikl Mathieu Salzmann, and Pascal Fua. Imposing hard constraints on deep net- works: Promises and limitations. arXiv preprint (Tf6:1700.0022.. 2017.
Ryan Martin and Chuanhai Liu. Inferential models: A framework for prior prior-free posterior proba- bilistic inference. Journal of the American Statistical Association, 108(501):3013111 2013.
Ryan Martin and Chuanhai Liu. Inferential models: reasoning with uncertainty. CRC Press, 2015.
Georges Matheron. Random sets and integral geometry. Wiley Series in Probability and Mathemat- ical Statistics, New York, 1975.
Leland Mclnnes, John Healy, and James Melville. Umap: projection for dimension reduction. arXiv preprint arXiv: 1802.030, 2018.
Uniform manifold approximation and
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34: 1662--1544,, 2021.
Ilya Molchanov. Random sets and random functions. Theory of Random Sets, pp. 451-552, 2017.
Ilya S Molchanov. Theory of random sets, volume 19. Springer, 2005.
Thomas Mortier, Marek Wydmuch, Krzysztof Dembezyúski, Eyke Hüllermeier, and Willem Waege- man. Efficient set-valued prediction in multi-class classification. Data Mining and Knowledge Discovery, 35(4):1435-1465 2021.
Jishnu Mukhoti,, Andreas Kirsch, Joost Amersfoort, Philip HS Torr, and Yarin Gal. Deep deter- ministic uncertainty: A simple baseline. arXiv preprint arXiv:2 11582, 2021.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, Spain, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), June 2015.
Hung T. Nguyen. On random sets and belief functions. Journal of Mathematical Analysis and Applications, 65:531-542. 1978.
Vu-Linh Nguyen, Sébastien Destercke, Marie-Hélène Masson, and Eyke Hüllermeier. Reliable multi-class classification based on pairwise epistemic and aleatoric uncertainty. In Proceedings of the Sevenh- International Joint Conference Artificial Intelligence, IJCAI-18, pp- 5089-5095. International Joint Conferences on Artificial Intelligence Organization, 7 2018. doi: 10.24963/ijcai. 201 8/706. URI.bttps://dan http:t/se//aaa..ooo//000.///////
201 20t.iaa....011088
16

---

Published as a conference paper at ICLR 2025
Ilia Nouretdinov, Tom Melluish, and Volodya Vovk. Ridge regression confidence machine. In Pro- ceedings of the Eighteenth International Conference on Machine Learning, pp. 385-392. Morgan Kaufmann, 2001.
lan Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. Epistemic neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in nearal information processing systems, 32, 2019.
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Tapio Elomaa, Heikki Mannila, and Hannu Toivonen (eds.), Machine Learning: ECML 2002, pp. 345-356, Berlin, Heidelberg, 2002a. Springer Berlin Heidelberg. ISBN 978-3-540-36755-4
Harris Papadopoulos, Vladimir Vovk, and Alexander Gammerman. Qualified prediction for large data sets in the case of pattern recognition. In ICMLA, pp. 159-163, 2002b.
Harris Papadopoulos, Alex Gammerman, and Volodya Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the 26th IASTED International Conference on Artificial Intelligence and Applications, AIA '08, pp. 64-69, USA, 2008. ACTA Press. ISBN 9780889867109.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European Sympo- sium on Security and Privacy (EuroS&P), pp. 372-387, 2016.
Janis Postels, Mattia Segu, Tao Sun, Luca Sieber, Luc Van Gool, Fisher Yu, and Federico Tombari. On the practicality of deterministic epistemic uncertainty. arXiv preprint arXh::2 2021.
Kostas Proedrou, Ilia Nouretdinov, Volodya Vovk, and Alex Gammerman. Transductive confidence machines for pattern recognition. In Tapio Elomaa, Heikki Mannila, and Hannu Toivonen (eds.), Machine Learning: ECML 2002, pp. 381-390, Berlin, Heidelberg, 2002. Springer. ISBN 978-3- 540-36755-0.
Rahul Rahaman et al. Uncertainty quantification and deep ensembles. Advances in neural informa- fion processing systems, 34:20063-20017 2021.
Galina Rogova. Combining the results of several neural network classifiers. Classic Works of the Dempster-Shafer Theory of Belief Functions, pp. 683-692, 2008
Tim GJ Rudner, Zonghao Chen, Yee Whye Tch, and Yarin Gal. Tractable funetion-space space variational inference in Bayesian neural networks. Advances Neural Information Processing Systems, 35: #2686-22698, 2022.
Yusuf Sale, Michele Caprio, and Eyke Hüllermeier. the volume of credal set good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence, pp. 1795-1804. PMLR, 2023.
Craig Saunders, Alexander Gammerman, and Volodya Vovk. Transduction with confidence and credibility. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelli- gence, IJCAI '99, pp. 722-726, San Francisco, CA, USA, 1999. Morgan Kaufmann Publishers Inc. ISBN 1558606130..
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifi- cation uncertainty. Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 3183-3193, Red Hook, NY, USA, 2018. Curran Associates Inc.
Armin Seyeditabari, Narges Tabari, and Wlodek Zadrozny. Emotion detection in text: a review. arXiv preprint arXiv: 1806.00674 2018.
17

---

Published as a conference paper at ICLR 2025
Glenn Shafer. A mathematical theory of evidence, volume 42. Princeton university press, 1976.
Glenn Shafer and Vladimir Vovk. tutorial on Conformal Prediction. Journal of Machine Learning Research, 9(3), 2008.
Philippe Smets. Bayes" theorem generalized for belief functions. Proc. ecai-86, 2:169-171, 1986.
Philippe Smets. Decision making in the TBM: the necessity of the pignistic transformation. Inter- national Journal of Approximate Reasoning, 38(2):133-147 2005.
Philippe Smets and Robert Kennes. The transferable belief model. Artificial intelligence, 66(2): 191-234, 1994.
Vincent Spruyt. How to draw a covariance error ellipse. 2014. Computer Vision for Dum- (Aviillve Online-hrtp//wwww. visiondummny. com/201 40w.44//eaeerrrrllllssee-reeesee covariance-mutrix// 2013.
Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Günnemann. Graph posterior network: Bayesian predictive uncertainty node classification. Advances in Neural Information Processing Systems, 34: 18033-18048, 2021.
Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional variational bayesian neural networks. arXiv preprint arXiv:1903.05779 2019.
Zheng Tong, Philippe Xu, and Thierry Denoeux. An evidential classifier based on Dempster-Shafer theory and deep learning. Neurocomputing. 450:275-293, 2021.
Salsabil Trabelsi, Zied Flouedi, and Pawan Lingras. Classification systems based on rough sets under the belief function framework. Int. J. Approx. Reason.,
2011.
Ba-Hien Tran, Simone Rossi, Dimitrios Milios, and Maurizio Filippone. All you need is a good functional prior for Bayesian deep learning. arXiv preprint 2rXiv:2011.12822 2020.
Dennis Thomas Ulmer, Christian Hardmeier, and Jes Frellsen. Prior and posterior networks: A survey on evidential deep learning methods for uncertainty estimation. Transactions on Machine Learning Research, 2023.
Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using single deep deterministic neural network. In International conference on machine learning, pp- 9690-9700. PMIR, 2020.
Laurens van der Maaten and Geoffrey Hinton.
Visualizing data using t-sne. Journal of Ma- chine Learning Research, (86):2579-26 2008. URL http:://mml...orgpppprrrr
maandeemmattenn html
Patrick Vannoorenberghe and Philippe Smets. Partially supervised learning by credal em approach. In Lluís Godo (ed.), Symbolic and Quantitative Approaches to Reasoning with Uncertainty, pp. 956-967, Berlin, Heidelberg. 2005. Springer Berlin Heidelberg. ISBN 978-3-540-33 55540-3188---- 18888-0.
Vladimir Vovk. Cross-conformal predictors. Annals of Mathematics and Artificial Intelligence, 74, 08 2012. doi: 10.10077/104472-01--997
Vladimir Vovk and Ivan Petej. Venn-abers predictors. arXiv preprint arXiv:1211.0025, 2012.
Vladimir Vovk, Glenn Shafer, and Ilia Nouretdinov. Self-calibrating probability forecasting. In S. Thrun, L. Saul, and B. Schölkopf (eds.), Advances in Neural Information Processing Sys- fems, volume 16. MIT Press, 2003. URL httphtttts/////rceeee
www. seurrips.oo htppsccc/paprr. Paper...
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer Science Business Media, 2005.
Peter Walley. Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, New York, 1991.
18

---

Published as a conference paper at ICLR 2025
Peter Walley. Towards unified theory of imprecise probability. International Journal of Approxi- mate Reasoning, 24(2-3):125-455 2000.
Anton Wallner. Maximal number of vertices of polytopes defined by f-probabilities. In Fabio Gagliardi Cozman, R. Nau, and Teddy Seidenfeld (eds.), Proceedings of the Fourth In- ternational Symposium on Imprecise Probabilities and Their Applications (ISIPTA 2005), pp- 388 395, 2005.
Luhuan Wu and Sinead Williamson. Posterior uncertainty quantification in neural networks using data augmentation. In International Conference on Artificial Intelligence and Statistics, pp. 3376- 3384. PMLR, 2024.
Mike Wu and Noah Goodman. A simple framework for uncertainty in contrastive learning. arXiv preprint ar arXhv:2010.02053 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.000 2017.
L. Xu, A. Krzyzak, and C.Y. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man, and Cyhernetics, 22(3):418- 435, 1992. doi: 10. 10.1109/2..155433
Ronald R Yager. Entropy and specificity in mathematical theory of evidence. Classic works the Dempster-Shafer theory of belief functions, pp. 291-310, 2008
Marco Zaffalon. The Naive Credal Classifier. Journal of Statistical Planning and Inference J STATIST PLAN INFER, 105:5-21, 06 2002. doi: 10.1016/5037--3755000
Marco Zaffalon and Enrico Fagiuoli. Tree-based credal networks for classification. Reliable com-
puting, 96):487-5509 2003.
Sergey Zagoruyko and Nikos Komodakis. 105.:1160 2016.
Wide residual networks.
arXiv preprint
19

---

Published as a conference paper at ICLR 2025
APPENDIX: RANDOM-SED NEURAL NETWORKS
This appendix provides a comprehensive overview of Random-Set Neural Networks (RS-NN), in- cluding theoretical underpinnings ($A), related work (. algorithms (C)) implementation details (DD). additional experimental results ( which addresses critical questions to elucidate the func- tionality and robustness of RS-NN, and application to text classification (sT).
A THEORETICAL EXPLANATION OF RS-NN
In this section, we provide overview of belief functions (AA.I). describe the RS-NN learning mechanism ($ and show the relationship between pignistic and credal predictions ($A.3). and provide statistical guarantees for RS-NN by applying conformal prediction to the predicted pignistic probabilities ($A.4).
A.1 CLASSICAL PROBABILITIES VS BELIEF FUNCTIONS
In this section, we explain the difference between classical probability measures and aod-additive measures, such as random sets. Random sets (Matheron, 1975 Kendall 1974, Nguyen 1978]
Molchanov 2205)) are non-additive measures (i.e., the additivity property does not hold). Their additional degrees of freedom allow to express the "epistemic' uncertainty about probability values
themselves. In fact, an entire field on 'imprecise probabilities' (Augustin et al. 2014 Walley 2000)
exist which deals with these kinds of measures.
In a standard classification task, where each input is classified into one of several mutually exclusive classes, the classical probability framework does assume that each class is distinct and independent. The output of a softmax model provides the probabilities for each class, and these probabilities sum to 1. This approach works well when you confident about the classification, for you have sufficient evidence to assign a clear probability to each class. However, it does not capture the uncertainty or ambiguity that exists when the evidence (as provided by the training set) is insufficient (e.g., because you did not sample similar points at training time, or because the test data is affected by distribution shift).
In Sec. 1 of the paper, we highlight that relying solely on confidence measures (like softmax proba- bilities) is insufficient for a comprehensive analysis of model predictions. Fig. 2 shows that standard neural networks are often overconfident (Nguyen et al. 2015) even for misclassífications. Our ap- proach uses random sets (which, in the finite case, assume the name of belief functions), as detailed in Sec. 2, to offer robust framework for assessing prediction reliability.
Random sets, as the name suggest, assign probability mass values to sets independently, and are therefore more general than classical probability measures (which are special case of random set). As a result, belief functions offer a way to handle uncertainty and make decisions when information is incomplete or ambiguous. While in classical probability, the sum of probabilities for individual events and their unions adheres to strict additive rules, belief functions relax these constraints to better capture uncertainty. In particular, the belief in the union of two hypotheses, Bel((A.B)) can be greater than the sum of the beliefs in the individual hypotheses, Bel((A)) + Bel((B) i.e, Bel([A)) Be)( - Bel((A.))). This inequality captures the idea that, in some situations, the available evidence may support the true class being in the set {A, BJ. without being enough to distinguish between the two alternatives. If (el((A.B) > Bel( (A}) Bel( it indicates that the model has significant uncertainty or confusion about distinguishing between these classes for a particular input.
Unlike classical probability theory. belief functions operate in higher space than probability vec- tors, i.e, it operates in a framework that generalizes classical probability theory. Suppose we know with 100% certainty that an event is either A or B. In a classical probability setting, we might say P(A) = 0.5 and P(B) = 0.5, or we might assign different probabilities to A and if we have some rea- son to favor one over the other. With belief functions, we can represent the same scenario differently. We could assign a belief value of I to the set (A, B} and 0 to both A and B individually (Bel( (A}) 0, Bel({ A, B}) 1). This means that while we know the outcome must be either A or B. we are entirely uncertain about which one it is. Alternatively, we could have Bel((A}) = 0.5,
Bel(
Bel( (B) 0.5. Bel((A, A, B))= 1, reflecting different state of knowledge evidence, where we' 're
20

---

Published as a conference paper at ICLR 2025
somewhat more informed but still not fully certain. In the behavioural interpretation of probability [Wlley] 2000), the belief value of an event A is the upper bound to the price one is willing to pay for betting outcome A.
Consider the following example (Cuzzolin 2020): ple- -ee-,eter, John, and Mary- are suspects. Our hypothesis space is therefore θ {Peter, John, Mary)- A witness testifies that the person he saw was a man, supporting the proposition A = (Peter, John] C O. However, the witness was tested, and the machine reported a 20% chance that he was drunk when he gave his testimony. As a result, we assign 80% belief to the proposition A, representing our uncertainty about whether any of the two could be the murderer.
Suppose there is a murder, and three peo-
In classical probability, Kolmogorov's additive probability theory (Kolmogorov & Bharucha- Reid 2018) forces us to specify support for individual outcomes, i.c. to distribute this 80% probabil- ity between Peter and John individually, even when the evidence (our data) supports set proposi- tions. This is unreasonable an artificial constraint due to a mathematical model that is not general enough. In the example, we have do not have enough evidence to assign this 80% probability to either Peter or John, nor information on how to distribute it amongst them. The cause is the additivity constraint that probability measures are subject to. This constraint reflects a broader lim- itation of measure- -teeorrtcc probability, which, while effective in many scenarios, struggles with second-order uncertainties and incomplete information. Therefore, we have several other methods to estimate uncertainty, such as the Bayesian method, or using degrees of belief, such as Evidential and belief function methods (see (Sensoy al. (2018), Sec. 3).
This means that belief functions allow us to model situations where we have uncertainty not just about which class an input belongs to, but also about whether we have enough evidence to distin- guish between certain classes. This added flexibility is what makes belief functions suitable for single-label classification tasks.
A.2 RS-NN LEARNING MECHANISM
Here we wish to expand on the rationale behind our choices for ground-truth representation bel, loss function and training, and the way in which RS-NN learns sets of outcomes.
Recall from Sec. (Bel(A), A E P(C) where Bel(A) is the belief function of a focal set A in the power set of classes C. In our method, Bel(A) is 1 if the true class is in subset A and 0 otherwise. Conse- quently, the belief-encoded ground truth bel will include multiple occurrences of 1. For instance, digit-classification task such MNIST, if {3] the true class, the belief-encoded ground truth would contain Is in all sets where {3} is present, such as {1. 3). {0, 3). {1, 2, 3}, and so forth, 0 for sets not containing (3) at all, such as f0. 1). [1, 23. (0. 1, 2) etc.
that the ground-truth for RS-NN is the belief-encoded vector bel
Since we assign precise labels for belief containing different focal sets. our model begins by pe- nalizing predictions that differ from the observed label in the same manner, regardless of the set's composition or relationship to the true label. Using MNIST as an example, this means that the loss incurred for predicting label (3) is equivalent to predicting label (3. 7), as both sets contain the true label. This equivalence in losses might seem counterintuitive at first glance. However, despite the identical loss values, the probabilities output by the sigmoid activation function will vary due to differences in the input logit values for each label. For instance, if the correct label is {3), the loss for predicting (3) or {3, 7} would be the same. Still, the loss for predicting (7) would differ, allowing the model to discern the set structure during training.
During the training process, the model learns to capture and understand these relationships by ob- serving patterns and dependencies in the training data. As the model optimizes its parameters based on the training objective (e.g., minimizing the loss function), it gradually adjusts its internal rep- resentations to better reflect these relationships. We use as the basis for our loss function Binary cross-entropy with sigmoid activation. It allows the model to predict the presence or absence of each label separately as binary classification problem, producing probabilities between 0 and for each class. While the model is unaware that it is learning for sets of outcomes, we leverage this technique to extract mass functions and pignistic predictions from the learnt belief functions. By re-distributing masses to original classes, we obtain the best pignistic predictions that are often more accurate than standard CNN predictions, attributing to the higher accuracy in Tab. |
21

---

Published as a conference paper at ICLR 2025
A.3
PIGNISTIC PROBABILITY AND CREDAL PREDICTION
6.6
0.4
Samples 60
100
Figure 7: Upper and lower bounds (Eq. 9), in dotted red, to the probability of the predicted most likely class according the pignistic prediction (in solid red) for 100 samples of CIFAR-10.
As they have belief values as lower bounds, credal sets induced by belief functions have peculiar shape (see Fig. right, for case with 3 classes). Their vertices are induced by the permutations of the elements of the sample space (for us, the set of classes) (Chateauneul & Jattray 1989, Cuzzolin 2008). Given one such permutation, e-g., (c2, Ca, C1 Cs) a set of 4 classes, the corresponding probability vector (vertex of the credal prediction) assigns to each class the mass of all the focal sets containing it, bul not containing any class preceding in the permutation order (Wallner 2005).
For additional illustration, the lower and upper bounds (Eq. 9 to the prohability of the top predicted class are plotted in Fig. [ together with the pignistic probability, for 100 samples of CIFAR-10. The bounds Eq. 9 can be efficiently computed from the finite number of vertices of the credal prediction
(Cuzzolii 2008 Fig.
A.4 STATISTICAL GUARANTEES
To complement the extensive discussion the main paper, we provide here an in-depth discussion on how statistical guarantees can be provided in an RS-NN framework.
Plugging RS-NN into conformal learning. Indeed, RS-NN can be employed as 'underlying model' in inductive conformal learning framework which builds empirical cumulative dis- tribution of the 'non-cofformity" scores of a set calibration samples, and at test time outputs the set of labels whose empirical CDF is above desired significance level (e.g., 95%).
Given test input and the associated predictive belief function Bel(clæ) (the output of RS-NN), we could, for instance, set as non-conformity score
s(x, c) = Pl(cx)) - Bel(C
{c](2)
(10)
(i.e., label is 'non-conformal" if its predicted plausibility, which is defined PI(A) 1 Bel(O), A) and has the semantic of an upper probability bound (Shafer 1976). is low), and compute predictive regions in the usual way:
T(x) {c C: >c)
where
p = (z),,,) s(1,, s(x,c))
+ ((x), s(z,,))) =(,c)))ss s(x,c))
++1
a+1
(Iz, Gj) is the j-th calibration point, 9 is the number of calibration points, and u ~ U(0, 1) (the uniform distribution the interval (0, 1)).
Experiments on conformal guarantees. Thus, we conducted experiments on CIFAR- 10 dataset to provide conformal prediction guarantees with non-conformity scores as given in Eq. TO The goal to assess the reliability of classification predictions by determining a threshold of non-conformity scores that covers 95% of the data.
To achieve this, con-conformity scores are obtained for all classes, and a threshold is calculated such that it contains 95% of the data. This threshold determines the coverage of the classification,, which refers to the proportion of predictions that actually contain the true outcome.
https://cml rhul. ac wwtp.aac...ccopa211111.... http://aaw...aaaooooooooooeoeaaa
22

---

Published as a conference paper at ICLR 2025
This prediction threshold is determined by finding the percentile corresponding to 1 - where is set to 0.05 to achieve 95% coverage. Fig. showss the non-confornity scores for all the calibration data, j 1000 (Angelopoulos & Bates 202I) (using the following split: 40000:10000:9000 10000:9000: 1000 training, testing, validation and calibration data points respectively for CIFAR-10), with a cut-off threshold that ensures 95% coverage.
The conformal prediction sets for two sample inputs are shown in Fig. along with their predicted probabilities.
Prediction sets: ('cat', 'horse', 'truck")
Prediction sets: ('cat', 'dog"} [9.49130123] [0.41791199] label: cat
20296496 [0.54583361] [e. 24168862] label: horse
Figure 8: Conformal prediction sets by RS-NN the CIFAR-10 dataset. The predicted probabilities for each class is shown.
0.8
95th percentile conformality threshold
[non-contormalliy))
0.6
0.2
0.0
200
400
600
800
1000
Sample by si)
Figure 9: Conformal prediction threshold for CIFAR-10 indicating the boundary beyond which predictions are considered non-conformal.
Figure 10: Coverage per class of RS-NN for CIFAR-10 dataset.
Figure 11:
Average size of the predictive sets generated for each class of CIFAR-10.
Figs. 10 and II show the coverage and average set size for each class on the test dataset. Coverage the proportion of prediction sets that actually contain the outcome, and average size is the
23

---

Published as a conference paper at ICLR 2025
average number of predicted classes per instance. Higher numbers represent more overlap between the classification regions of different classes.
Summarizing the results, (1) the coverage averages across classes over the desired threshold. This indicates that the conformal prediction method is providing reliable prediction intervals that contain the true class label with the specified confidence level; (2) The spread or variability in set sizes across different samples provides insights into how well the prediction sets adapt to the difficulty of the samples. Ideally, the prediction sets should dynamically adjust based on the difficulty of each example, with larger sets for more challenging inputs and smaller sets for easier ones. Fig, [ that the average set size for classes 2, 3, 4, and 6 is considerably larger than for the rest, indicating that the model found samples from these as challenging inputs.
Alternative approaches to statistical guarantees. In the future we plan explore the possibility of generalising the empirical CDF at the basis of conformal learning in a belief function / random set representation, aiming to retain statistical guarantees. Given the complexity of this enterprise, we are working towards this in separate paper.
Finally, an intriguing alternative approach (and one that is more achievable in the short term) is the study of confidence intervals in belief functions representation, such as the one we employ in RS-NNs. In fact, recent studies have been looking at extending the notion of confidence interval hhtpp:: / //en.nn/nn// wikipedia org/ wwikkii/ooffidenee interval) belief functions, un- der the name of confidence structures
765766666 66)3533),, which generalise standard confidence distributions and generate "frequency-caliirattee" belief functions.
Liu and Martin, in particular, have developed an Inferential Model (IM) approach which produces belief functions with well-defined frequentist properties (Martin & Liu| 2013 2013). An alternative approach relies on the notion of "predictive" belief function (Denœeux (006). which, under repeated sampling, is less committed than the true probability distribution of interest with some prescribed probability.
B RELATED WORK
Researchers have recently taken some tentative steps to model uncertainty in machine learning and artificial intelligence. Scientists have adopted measures of epistemic uncertainty (Kendall & Gal, 2017) to refuse or delay making decision (Geifman & El-Yaniv] 2017), take actions specifically directed at reducing uncertainty (as in active learning (Aggarwal et al. 2014)), or exclude highly- uncertain data at decision making time (Kendall & Gal
et al. (025).
Imprecise probability. Significant work has been done in a credal set setting. Notably, (Zaffalon 2002 Corani& & Zaffalon 2008) have proposed the Naive Credal Classifier (NCC) as extension of the naive Bayes classifier to credal sets, where imprecise probabilities are included in models in the form of sets of classes. (Antonucci & Corani 2017) have presented graphical models which gen- eralise NCCs to multilabel data. Expected utility maximisation algorithms, such as Bayes-optimal prediction (Mortier et al. (021), and classification with reject option for risk aversion (Nguyen al., 2018) are also based on set-valued predictions.
Classification with partial data has been studied by various authors (Vannoorenberghe & Smets 2005), while large number of papers have been published on decision trees in the belief function Tramework (Elouedil et al., 2000). Significant work in the neural network area was conducted in (Denoeux, 2000) and (Fay et al. 2006). Classification approaches based on rough sets were proposed in (Trabelsi et al.J (011). Ensemble classification (Burger al. (0066) is another area in which uncertainty theory has been quite impactful, as the problem is one of combining the results of different classifiers. Important contributions have been made by (Xu et al.. 1992) and (Rogoval 2008). Regression, on the other hand, has only recently been considered by belief theorists (Laanaya ct al. 2010 Gong & Cuzzolin, 2018: Denauux 2022). For tasks like image classification in large datasets, most of these approaches have low performance metrics including training time and test accuracy.
Evidential approaches. Within a proper epistemic setting, significant amount of work has been done by Denoeux and co-authors, Lid et al. (2012), on unsupervised learning and clustering in particular in the belief function framework. Quite lot of work has been done ensemble classifica-
24

---

Published as a conference paper at ICLR 2025
tion in the evidential framework (Xu al. 1992) (in particular for neural networks (Rogova 2008
decision trees (Elouedi et al. Madrid, 2000), K-nearest neighbour classifiers (Denceux 2008). and more recently evidential deep learning classifiers able to quantify uncertainty (Tong al. (02II). Tong et al. (Tng et al., 2021) proposes a convolutional neural network based on Dempster-Shafed theory called the evidential deep-classifier employs utility functions from decision theory to assign utilities on mass functions derived from input features to produce set-valued observations. Another recent approach by Sensoy et al. (Sensoy et al. 2018) proposes an evidential deep learning classifier to estimate second-order uncertainty in the Dirichlet representation. This work is based on subjec- tive logic and learning to form subjective opinions by minimizing the Kullback-Leiblee divergence over uniform Dirichlet distribution.
These methods represent predictions as Dirichlet distributions, as explored various works over the last few years (Sensoy et al. 2018 Malinin & Gales [018 2019; Malinin et al.) 2019 Charpentier al. 2020). However, many commonly used loss functions for these networks are flawed, as they fail to ensure that epistemic uncertainty diminishes with more data, violating basic asymptotic assumptions (Bengs et al., 2022). Additionally, some approaches require out-of-distribution (OoD) data for training. which may not always be available and doesn't guarantee robustness against all types of OoD data. Studies (Ulmer et al.) (2023) have shown that OoD detection degrades in certain models under adversarial conditions, and even techniques like normalizing flows (NFs) in posterior networks (Charpentier et al., [2020), while effective, can struggle with OoD data when relying on learned features (Kopetzki al.) 202 Stadler al. 2021).
Conformal prediction. Conformal prediction (Vovk et al. 2005) provides a framework for esti- mating uncertainty (Shafer & Vovk 2008) by applying threshold on the error the model can make to produce prediction sets, irrespective of the underlying prediction problem. Different variants of conformal predictors are described in papers by Saunders et al. (Saunders al. 1999), Nouretdinov et al. (Nouretdinov et al. (2001), Proedrou et al. (Proedrou et al. 2002) and Papadopoulos et al. (Papadopoulos al. 2008).
Since the computational inefficiency of conformal predictors posed problem for their use in neu- ral networks, Inductive Conformal Predictors (ICPs) were proposed by Papadopoulos et al. (Pa- padopoulos et al., (002a)) (Papadopoulos et al. 2002b). Venn Predictors (Vovk et al., (003), cross- conformal predictors (Vovk 2012) and Venn-Abers predictors (Vovk & Petei] 2012) were introduced in distribution-fre free uncertainty quantification using conformal learning. As show above, RS-NN is compatible with conformal prediction, as a possible underlying model. In the future we plan to extend our random-set representation conformal learning, by replacing cumulative distribution functions with random sets.
uncertainty quantification. Deterministic Uncertainty Quantification (DUQ) meth- ods estimate epistemic uncertainty by analyzing latent representations or using distance-sensitte functions instead of softmax (Alemi et al. 2018: Wu & Goodman 2020 Liu et al. 20200 Mukhoti et al. 2021; Van Amersfoort et al.) 2020). These methods have been applied to areas like object detection (Gasperini et al.] (021)) but focus mainly on OoD detection, overlooking calibration how well uncertainty reflects model performance under shifting distributions. Calibration is crucial for safe deployment but remains underexplored, as DUQS are typically tested only on simple tasks and datasets, leaving their effectiveness in more complex scenarios unproven (Postels et al. (021))
Deterministic
Uncertainty estimates are poorly calibrated under distributional shifts, especially when compared to scalable Bayesian methods. Regularization techniques based on feature space distances, such as bi-Lipschitz regularization (used by most of these models), do not effectively improve OoD detec- tion or calibration (Postels et al. (021). This is due to the limitations of distance metrics in high- dimensional data, such as images. Additionally, DUQ methods differ from RS-NN by modeling epistemic uncertainty in the input space rather than the prediction space. They assess uncertainty based on whether the model's 's input is an iD or OoD sample. Because of this difference in how epistemic uncertainty is modeled, cannot directly compare RS-NN to DUQ methods.
C ALGORITHMS
This section outlines the algorithms integral to the implementation and evaluation of budgeting ($C.T). expected calibration error (ECE) ($2.), area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC)
25

---

Published as a conference paper at ICLR 2025
C.I ALGORITHM FOR BUDGETING
For RS-NN with N classes, generating 2 outputs is computationally infeasible due to exponential complexity. Instead, choose K relevant subsets (Ax focal sets) from the 2- possibilities.
To obtain these K focal subsets, we extract feature vectors from the penultimate layer of a trained standard CNN with N outputs. We then apply t-SNE for dimensionality reduction 3 dimensions. Note that our approach is agnostic, as t-SNE could be replaced with any other dimensionality reduc- tion technique, including autoencoders.
Next, we fit a Gaussian Mixture Model (GMM) to the reduced feature vectors of each class, Using the eigenvectors and eigenvalues of the covariance matrix Σ, and the mean vector He for each class we define an ellipsoid covering 95% of the data. The lengths of the ellipsoid's ' principal axes are computed as lengthe. = 2v7.8154; where X; the jth eigenvalue. The scalar 7.815 corresponds to a 95% confidence interval for chi-square distribution with 3 degrees of freedom. The class ellipsoids are plotted in 3D space and the overlap of each subset the power set N is computed. As it is computationally infeasible to compute overlap for all 2 subsets, cardinality 2 and use early stopping when increasing the cardinality further does not alter the list of pess-lovela sets of classes. We choose the top-K subsets (Ak) with the highest overlapping ratio, computed as the intersection over union for each subset.
start doing from
Algorithm Budgeting Algorithm
1: Input: D -Training data with N classes, C - The set of classes, K Number of non-singleton focal sets
2: Output: Set containing N + K focal
3: Initialization
4: Extract feature vectors using trained CNN
5: Apply t-SNE for dimensionality reduction to 3 dimensions
6: for each class do
7: Fit GMM to the reduced feature vectors for that class to obtain 44a and Eg
8: Define an ellipsoid covering 95% data using Me and Eg
9: end for
10: most most_ wwwerlappinggg..ooooooo.... sets + Initialize an empty list for non-singleton focal sets Ai...., Ak
11: Set current_care cardinalit +
12: while current www.ontttcadinaaaaii S N do
13:
Compute overlaps for subsets of cardinality current cariiniltt overlap(A) A
A
Select top-K subsets with highest overlap
16:
Update most overlapping sets
if no change in most_overlapping_set then
18:
break 19: end
20: current. cardinality current cardinality +
21: end while
Combine selected non-singleton focal sets with N singleton sets
23: O t CU [AI,..,,KK]
24: return O
C.2 ALGORITHM FOR ECE
Expected Calibration Error (ECF) computed by comparing the average confidence and accuracy within each bin. The steps involved are as follows:
For each bin, calculate the absolute difference between the average confidence and the accuracy.
2. Weight each difference by the proportion of instances in that bin compared to the total number of instances.
3. Sum up these weighted differences across all bins to get the final ECE value.
26

---

Published as a conference paper at ICLR 2025
Algorithm 2 Expected Calibration Error (ECE)
Require: confidences: List array of confidence scores predicted by the model predictions: List
or array predicted class labels vueelabels List or array of true class labels Number of bins for binning confidence scores
Ensure: ECE: Expected Calibration Error
1: Step 1: Normalize Confidences Normalize the confidence scores to ensure they are in the
range [0, l].
2: Step 2: Binning Divide the confidence scores into num hins bins.
3: Step 3: Initialize Arrays Initialize arrays bin fin.accurayy, bin.confidence, and weights with ze- ros.
4: Step 4: Populate Arrays
5: for each bin do
6:
Identify instances falling into the bin
7: Calculate mean accuracy and mean confidence within the bin
8: Update bin_accuracy, bin_confidence, and weights
9: end for
10: Step 5: Calculate ECE Calculate the Expected Calibration Error using the populated arrays.
ECE Σ
(bin accuracy, bin. fonn.confiiee confidecc))) weighls;
The formula for ECE is given by:
ECE
(Accuracy, Confidence:) Weight,
where:
Accuracy, is the accuracy within the i-th bin.
Confidence, the average confidence within the i-th bin.
Weight, the proportion of instances in the i-th bin compared to the total number of instances.
the total number of bins.
The final ECE value represents how much the model's estimated probabilities differ from the true (observed) probabilities. A low ECE indicates well-calibrated probabilistic predictions, demonstrat- ing that the model's confidence scores align closely with the actual likelihood of events. RS-NN exhibits the lowest ECE as shown in Tab
C.3 ALGGORITH FOR AUROC, AUPRC
-utttf-diittibuttin (OoD) detection involves the identification of data points that deviate from the in-distribution (iD) data on which a model was trained. This process relies on assessing the model's uncertainty, particularly its epistemic uncertainty, which reflects the model's 's lack of knowledge or confidence in making predictions. When exposed to OoD data, which differs significantly from the training data, the model tends to exhibit higher epistemic uncertainty.
AUROC (Area Under the Receiver Operating Characteristic Curve) and AUPRC (Area Under the Precision-Recall Curve) are evaluation metrics commonly used to assess the performance of binary classification models, providing insights into their ability to distinguish between positive and nega- tive instances. In the context of evaluating uncertainty estimation in machine learning models, these metrics quantify how well the model separates iD samples from OoD samples. AUROC is derived from the Receiver Operating Characteristic (ROC) curve, which illustrates the trade-off between True Positive Rate (TPR) and False Positive Rate (FPR) across various classification thresholds.
The AUROC value represents the area under this curve and ranges from 0 to 1, with higher values indicating better discriminative performance. AUPRC is based on the Precision- Recall curve, which
27

---

Published as a conference paper at ICLR 2025
Algorithm 3 Algorithm for AUROC, AUPRC
Require: uncertainty _id:iitttyiiiiiiooooo Uncertainty scores for in-distribution samples. uncertainty Uncer- tainty scores for out-of-distribution samples.
Ensure: (fpr, tpr, thresholds): ROC curve metries.
1: (precision, recall, pre.thresholds): Precision Recalll curve metrics.
2: auroc: Area Under the Receiver Operating Characteristic curve.
3: aupre: Area Under the Precision-Recall curve.
4: Step I: Concatenate uncertainties;
5: uncertainties + concatenate/ (uncertainty Connaaeeaaeeuncertaannyy
.idd, pncertainty_oodl
6: Step 2: Create and combine labels;
7: labels <- serostuncertainty.iid _iid.shapeee)) ood_labels < onef(uncerrantyyooossasaa((()))) labels t- concatenate labels, ood labels)
8: Step 4: Calculate ROC curve;
9: (fpr, tpr, thresholds) <- roc curve uncertainties)
10: Step 5: Calculate AUROC;
11: auroc f- wauuusccoreeabaaae uncertainties)
12: Step 6: Calculate Precision- Recall curve;
13: (precision, recall, prc_threshiee sre_thressolds + precision recall www.:rraeaaalaaaeaaeeeeeeeeeeeea curve(labels, uncertainties)
14: Step 7: Calculate AUPRC;
15: aupre f- average precision clabel,,,,ls, uncertainties)
plots Precision against Recall at different classification thresholds. Precision measures the accuracy of positive predictions, while Recall quantifies the ability to capture all positive instances. AUPRC calculates the area under this curve and provides complementary perspective, particularly valuable when dealing with imbalanced datasets.
2y-09 ABC Curce
Ictel raage Carve
SW6N PRC Curve
Incil Imrage PAC Curve
IS-SK
15433 (0x)
PMM 10 6014
21
ELA1
Mes 91.77
Tass folttve Nata (TRR)
Faito Poeitho Rata (TRR)
Rocal
Rocal
Figure 12: Receiver Operating Characteristic (ROC) and Precision- Recall Characteristic (PRC) curves for RS-NN, LB-BNN, ENN, and CNN evaluated on the SVHN and Intel Image OoD datasets for CIFAR-10.
F-VWIST Curse
K-MISST RCC Carve
PRC Curce
K-RLEST PBC Curce
80
70 23
CSD
CUD
1936
9.4
3.8
Figure 13: Receiver Operating Characteristic (ROC) and curves for RS-NN, LB-BNN, ENN, and CNN evaluated on the SVHN and Intel Image Ool datasets for MNIST.
Characteristic (PRC)
Figs. 12 and 13 plot each model's performance, illustrating the trade-offs between True Positive Rate and False Positive Rate (ROC curve), Precision and Recall (PRC curve) for CIFAR-10 10 (Fig. 12 and MNIST (Fig. [3].. The left two plots depict the AUROC curves, where the blue curve representing RS-NN outperforms others, indicating superior discrimination between true positive and false positive rates. Similarly, on the two right plots displaying Precision-Recall Curve (PRC), RS-NN exhibits the highest curve, emphasizing its precision and recall performance. These results showcase RS-NN's effectiveness in distinguishing in-distribution and out-of-distribution samples.
In real-world scenarios, encountering OoD instances is inevitable, making reliable OoD detection essential for safety-critics applications to avoid erroneous decisions on unfamiliar data. Recognis- ing unfamiliar data signals the model about situations beyond its training, allowing it to acknowledge its own limitations and ignorance, which in turn enhances its uncertainty estimation.
28

---

Published as a conference paper at ICLR 2025
D IMPLEMENTATION DETAILS
Laplace Bridge Bayesian approximation (LB-BNN) (Hobbhahn et al. 2022) uses the Laplace Bridge to map efficiently between Gaussian and Dirichlet distributions and enhance computational effi- ciency. In our experiments, we obtain multiple samples from the LB-BNN posterior and aver- age these samples using Bayesian Model Averaging (BMA). Function-Space Variational Inference (FSVI) (Rudner et al. (022) utilizes variational inference to derive an approximation of the posterior distribution over the function space. FSVI proposes approximating distributions over functions as Gaussian by linearizing their mean parameters and derived tractable and well-defined variational posterior. Deep Ensembles (DEs) (Lakshminarayanan et al. 2017) and Epistemic Neural Networks (ENNs) (Osband et al. 2024) are ensemble methods that train ensembles of models to efficiently estimate uncertainty. The mean and variance of ensemble predictions are calculated and the mean is softmaxed to obtain predictions from DE and ENN. The training hyperparameters for all models are outlined in Tab. S
Table 5: Training Hyperparameters for All Models
Parameter
Value
Architecture
ResNet50 (excluding top classification layer) 1024 and 512 neurons (ReLU activation) Sigmoid (multi-label classification)
Additional Dense Layers
Output Layer Activation (RS-NN)
Output Layer Activation (Other Models)
Softmax (multi-class classification)
Learning Rate
Initial: le-3
Learning Rate Scheduler
Decrease by 0.1 at epochs 80, 120, 160, 180
Training Epochs
200
Batch Size Optimizer
128
RS-NN: Adam, LB-BNN: Adam, ENN: Adam, DE: Adam, FSVI: SGD
CIFAR-I0: 40,000
CIFAR-100: 40,000
Training Dataset Sizes
MNIST: 50,000
Intel Image: 13,934
ImageNet: 1,172,498
CIFAR-10: 10,000
CIFAR-100: 10,000
Testig Dataset Sizes
MNIST: 10,000
Intel Image: 3,000
ImageNet: 2,000
Testing Samples for OoD Datasets
10,000 (except Intel Image: 3,000)
Input Image Size Data Augmentation GPU
224 224
Random borizontal/vertical shifts (magnitude 0. horizontal flips NVIDIA A100 80GB
We report the training times (in minutes) for all models on the CIFAR-10 dataset in Tab. 6 Tab. 6 shows that FSVI has the longest training time, followed by RNN and DE. Compared to LB- BNN, RS-NN has an additional 5 minutes the training duration, while CNN has the shortest training time. However, CNNs do not provide uncertainty estimation, and RS-NN has more reliable uncertainty and Ool metrics in comparison to LB-BNN.
The significant training time required for Deep Ensembles (DE) often does not justify their uncer- tainty estimates (Abe et al., 2022 Rahaman et al. 2021). Despite the computational cost, DE is sometimes only as good as other uncertainty estimation models, which can be trained much faster. Other approaches, like methods with calibration techniques (c.g., temperature scaling or Bayesian approaches),, can provide similar performance without the substantial overhead that DE entails.
Table 6: Training time (min) for all models on the 100 dataset.
RS-NN
LB-BNN
FSVI
DE
ENN
CNN
Training time (min)
113.23
107.900
1518.35
426.66
712.302
85.333
29

---

Published as a conference paper at ICLR 2025
As mentioned in Sec. 4.2 the results for FSVI are different than what was reported in (Rudner et al. 2022). The lower performance with ResNet-50 in our experiments may be attributed to the model being optimized specifically for ResNet-18. We noticed the unexpectedly low results and adhered to their suggested optimizer, SGD, instead of Adam (used in all other models), to achieve optimal performance, mentioned in Tab. Additionally, to enhance performance, porated the context points from ImageNet and CIFAR-100 as recommended in Rudner et al. (2022). Furthermore, we observed that FSVI uses more extensive data augmentation than our standardized setup. For instance, their data augmentation includes random crops and higher magnitude random flips (0.5), while we only used random horizontal/vertica flips of magnitude 0.1, potentially con- tributing to the performance gap. While increasing model capacity by using ResNet-50 (25.6M parameters) over ResNet-18 (11.7M parameters) should theoretically improve performance, this not always the case in practice. ResNet-50 requires more careful optimization and is more prone to overfitting, particularly when used with smaller datasets or suboptimal augmentation. This explains why ResNet-50 did not consistently outperform ResNet- 18 in our implementation of the compared methods.
also incor-
E ADDITIONAL EXPERIMENTAL RESULTS
In this section, we will address the following questions based on our experimental findings:
Why are most models not able to differentiate between ImageNet and ImageNet-O in terms of entropy?
Q1:
Why does RS-NN achieve better accuracy than CNN? What the overconfidence problem in CNN?
Q2:
Q3: How does RS-NN handle perturbed data (noisy, rotated)?
Q4: What empirical evidence supports the claim that RS-NN is more robust against adversarial attacks than traditional CNNs?
Q5: Why is the credal set considered good measure of epistemic uncertainty?
Q6: What is the importance of the mass regularizers in the loss function Las (Sec. 3.2 Why it important to properly tune the regularization parameters and J3, and what impact does this have on the model's performance?
Q7: What are the effects of budgeting on RS-NN? How was the number of budgeted focal K chosen? How does budgeted RS-NN differ from standard RS-NN with all the subsets in the power set?
Q8: How does UMAP measure as a faster alternative to t-SNE for budgeting in RS-NN? How can the budgeting procedure be adapted for continuous data streams?
E. IMAGENET VS IMAGENET-O
In Tab the clear separation between iD and OoD entropy is evident for RS-NN, yet for ImageNet vs ImageNet-O, all other models struggle to distinguish between the iD vs OoD entropy and the OoD dataset almost falls within the standard deviation of the iD samples. Also, in Tab. the credal set widths for CIFAR-10 vs SVHN/Intel Image and MNIST vs F-MNISTTK-MNNNN -NIST are distinct, but ImageNet ImageNet-O is not too different.
To better understand this, is important to consider how ImageNet-O is generated. ImageNet-O is a dataset of adversarially filtered examples for ImageNet duttrf-distributtion detectors. It is created by
30

---

Published as a conference paper at ICLR 2025
removing all the ImageNet-IK samples from the full ImageNet-22K dataset. The remaining sam- ples, which do not belong to ImageNet- 1 classes, are classified by a trained model. The samples classified by the model as ImageNet- IK classes with high confidence becomes ImageNet-O. Hence, in general, difficult for models to make the iD vs Ool distinction as evident by uncertainty measures in Tab.
We conducted experiments on Credal Set Width and Pignistic Entropy measures of RS-NN using a ViT-B-16 (Vision Transformer) backbone, as shown in Tab. 7 The pignistic entropy and credal set width show good distinction between iD and OoD measures, unlike the uncertainty measures from other baseline models Tab, 2
Table 7: Credal set width and entropy for RS-NN using ViT-B-16 model on ImageNet ImageNet- O datasets.
Datascts
Credal Set Width
Pigmistic Entropy
Softmax Entropy
ImageNet (iD) (L)
0.1082 + 0.1926
2.7401 2.3678
2.3442 1.2243
RS-NN
CNN
ImageNet-O (OoD) (t)
0.1948 1 0.2455
4.4238 1 2.5744
2.8619 + 1.5884
Table OoD detection and uncertainty estimation performance for vs OoD dataset: CIFAR-10
vs CIFAR-100.
In-distribution Entropy ($)
Credal Set Width (1)
CIFAR-100
Dataset
Model
AUROC (1)
AUPRC (1)
Entropy (t)
Credal Set Width (f)
CIFAR-10 RS-NN
0.0802 + 0.297 0.0061
0.039
88.01
$4.93
0.5778 + 0.729
0.0721 + 0.165
This is specifically due to the peculiar relation between ImageNet and ImageNet-O, and not general problem with the models not being able distinguish samples when they belong the same image distribution.
To support our claim, conducted experiments using CIFAR-10 (iD) CIFAR-100 (OoD) as shown in Tab. g below, and the results indicate that our model distinguishes between these two datasets well, evident through the AUROC, AUPRC, entropy and credal set width. Therefore, the issue seems to be specific to ImageNet and ImageNet-O, likely due to the class imbalance in ImageNet. As per the general trend for ImageNet in Tab. [ all models behave uncertainly for higher number of classes which makes the difference between iD and OoD less extreme. Although, RS-NN still performs better than other models on ImageNet.
E.2 RS-NN vs CNNs: ACCURACY AND CONFIDENCE
Table 9: Standard CNN fails to perform well when tested on an noisy (rotated) sample of MNIST test data. The predicted results show how a standard CNN predicts the wrong class with a high confidence score (99. 99.95%), while the RS-NN model predicts the correct class with 49.7% confidence and high entropy 2.1626.
True Label
Predictions
Do
Class 0.9995
Class 0.0002
[6', '4'
Beliet values
'8". 1
0.7853
0.7150
['6',
['''_ '8", (1")
Mass values
0.2079
0.1999
Pignistic Probubility
0.4978
0.2294
Class 0.0001
Class 1.4e-05
Class 1.1e-06
[18] 131)
(9' '8'
66' '8'
0.5529
0.4092
0.6357
[16]
('8''33 (3)
[8'
0.0961
0.05147
0.1959
O. 0.1002
0.0459
Entropy 0.0061
Entropy 2.1626
Credal set width 0.582
Accuracy. RS-NN consistently achieves higher accuracy than standard CNN (ResNet50) on all datasets as shown in Tab. [ This appears attest that, RS-NN which does not require prior assumptions and more data driven, it can better adapt and capture the inherent complexity of the data (as sets), contributing to its superior performance in test accuracy
random-set framework
31

---

Published as a conference paper at ICLR 2025
various datasets (Tab. D.. The training hyperparameters for RS-NN and CNN are the same, since the training procedure is quite similar.
Confidence. We further analyse how RS-NN overcomes the overconfidence problem in CNNs. Tab shows an noisy example of a rotated MNIST image with a true label '8', The standard CNN makes incorrect prediction ('6') with 99.99 99.95% confidence and low entropy of 0.0061, showcasing ooTab limitation in relying solely on confidence scores and entropy of predicted probabilities. RS- NN provides a correct prediction (class '8') with confidence of 49.7% and pignistic entropy of 2. 1626 and credal set width of 0.482. Crucially, the RS-NN's prediction process considers more than just the predicted class, taking into account both the confidence of the pignistic probability and the entropy. In this example, despite predicting the correct class ('8"), the RS-NN demonstrates low confidence (49.7%) and high entropy (2. 1626). This holistic approach provides a more nuanced and reliable understanding of the model's uncertainty.
Incorrectly Classified Incorrecty Classifiad RS NN
2.5
Frequency
2.0
1.5
1.0
0.5
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Confidence scores:
Figure 14: Confidence scores for Incorrectly Classified samples of RS-NN and CNN
A distribution of confidence scores for RS-NN in the same noisy experiment are shown in Fig. for both incorrectly classified and correctly classified samples. Fig. 14
we show how a standard CNN has high confidence for incorrectly classified samples whereas RS NN exhibits lower confidence and
higher entropy (Fig. 16) for incorrectly classified samples. Fig. T6 displays the entropy distribution
for correctly and incorrectly classified samples of CIFAR-10, 10, including in-distribution, noisy, and rotated images. Higher entropy is observed for incorrectly classified predictions across all three cases.
12.5
Incorrectly tly Classified
12.5
Pncoeccyly Classified
12.5
Incorrectly Classified
Correctly Classified
Correctly Classified
Correctly Classified
10.0
10.0
5.0
5.0
5.0
2.5
2.5
2.5
0.50
1.00
0.00
0.25
1.00
0.00
0.25
Confidence scores: CIFAR10
Confidence scores: Noisy CIFAR1O
Confidence scores: Rotated CIFAR10
Figure 15: Confidence scores of RS-NN on C00AR-10, Noisy CIFAR-10, and Rotated CIFAR-10
E.3 ROBUSTNESS TO NOISY AND ROTATED DATA
We split the MNIST data into training and test sets, train the RS-NN model using the training test, and test the model noisy and rotated out-of-distribun test data. This is done by adding random noise to the test set of images to obtain noisy data and rotating MNIST images with random degrees of rotation between 0 and 360.
32

---

Published as a conference paper at ICLR 2025
Correctly Classilied
Correctlyy Classlied
Correctly Classified Incorrectly Classified
Incorrectly Classified
Incorrectly Classifted
3-
2-
3
0
Entropy:
Entropy: Noley CIFAR10
Entropy: Rotated FARRI
Figure 16: Entropy distribution of RS-NN on CIFAR-10, Noisy CIFAR-I0, and Rotated CIFAR-10
Table 10: Standard CNN fails to perform well when tested noisy noisy and rotated samples of MNIST test data. The predicted results show how a standard CNN predicts the wrong class with a high confidence score, whereas the RS-NN model predicts the right class with varying confidence scores.
Standard CNN Predictions Belief RS-NN Predictions
True Label =
Belief values
Mass values
Pignistic
Class 0.628
['2"]
0.985
[22)
0.982
0.982
Class 2 0.232
[22) 10', '1 0.981
[',', '8'}
0.009
8 0.007
Class 3 0.117
['21, 0.980
['7', '0', '1'
0.002
0.100
Class 8 0.015
{22', 54"}
0.979
('')
0.002
0.004
True Label =3
Belief values
Mass values
Pignistic
Class 0.969
['3", '5"} 0.772
{'3))
0.361
0.427
Class 5 0.018
(3") '3')
0.637
['7', '8'} 0.134
8 0.205
Class 9 0.006
['6', '5)
0.620
['0r, '6")
0.104
0.115
Class 2 0.003
(3')
0.540
['8') 0.084
0.080
True Label =
Belief values
Mass values
Pignistic
Class 2 1.0
['2', 0.999
{'1')
0,556
1 0.566
Class 1 3.39e-08
[111_ (9") 0.958
('2")
0.423
2 0.423
Class 6 1.03e-10
(11') 0.924
('1', '9') 0.020
9 0.010
Class 3 4.92e-11
(51) 0.825
('6')
4.31e-05
6 4.31c-05
True Label 9
Belief values
Mass values
Pignistic
Class 5 0.988
{'7', '5', '9'}
0.831
{'7,, '9 0.171
9 0.699
Class 2 0.010
(97) ['5', '9") 0.576
0.706
['3"] 0.134
7 0.023
Class 3 0.0004
['9') 0.132
0.004
Class 7 0.0002
['3', '9") 0.558
['7', '8') 0.102
0.001
Tab 10 shows predictions for noisy and rotated noisy MNIST samples. In cases where a standard CNN makes wrong predictions with high confidence scores, Random-Sed NN manages to predict the correct class with varying confidences verifying that the model is not overcofident in uncertain cases. For example, noisy sample with true class '3' has a standard CNN prediction of class '8' with 96.9% confidence, while RS-NN predicts the correct class ('3') with 42.7% 12.7% confidence. Similarly, for rotated "9', the standard CNN predicts class "5' with 98.8% confidence whereas RS- NN predicts the correct class ("''} with 69.9% confidence. For rotated '1', the standard CNN predicts class '2" with 100% confidence.
Tab. I1 shows the test accuracies for standard CNN, RS-NN, LB-BNN and ENN at different scales of random noise. RS-NN shows significantly higher test accuracy than other models as the amount of noise added to the test data increases.
The test accuracies for standard CNN, RS-NN, LB-BNN and ENN on Rotated MNIST images are shown in Tab 12 The samples are randomly rotated every 60 degrees, -180 to - -120 to -600
33

---

Published as a conference paper at ICLR 2025
Table 11: Test accuracies($5 for RS-NN, standard CNN, LB-BNN (Bayesian) and ENN (Ensemble)
on noisy samples of MNIST. 'Scale' represents the standard deviation of the normal distribution from which random numbers are being generated for random noise.
Noise (scale)
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Standard CNN
93.40%
79.08%
79.15%
58.33%
40.19%
28.15%
28.59%
RS-NN
96.90%
85.36%
85.91
68.33 %
51.46 %
37.18 %
38.05 %
LB-BNN
98.47%
95.26 %
80.18%
61.56%
43.28%
31.41%
24.55
ENN
97.81%
90.76%
75.41%
58.99%
45.70%
36.97%
31.51
Table 12: Test accuracies(%) for RS-NN, standard CNN, LB-BNN (Bayesian) and ENN (Ensemble) on Rotated MNIST out-of-distribution (OoD) samples. Rotation angle is random between the values given.
Rotation (angle)
-180/-120
-120/-60
-60/0
0/60
60/120
120/180
0/360
Standard CNN
36.41%
21.86%
74.41%
80.80%
23.89%
37.53%
45.86%
RS-NN
37.84%
23.54%
78.44%
81.46%
26.31%
47.71% 37.83%
38,48%
LB-BNN
37.56%
20.19%
75.12%
77.67%
23.18%
46,07
ENN
36.40%
19.43%
71.70%
78.59%
18.70%
36.38%
44.14
-600 to 09, etc. A fully random rotation between 0 and 360 also shows higher test accuracy for RS-NN at 47.71% when compared standard CNN with test accuracy 45.86%.
E.4 ROBUSTNESS TO ADVERSARIAL ATTACKS
We conducted experiments on adversarial attacks using the well-known Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2014). FGSM is popular approach for generating adversarial examples by perturbing input images based on the sign of the gradient of the loss function with respect to the input. Mathematically, the perturbed image x' is computed
sign( a (0, 2, v)),
(11)
where r is the original input image, (epsilon) is small scalar representing the magnitude of the perturbation, J/ is the loss function, 0 are the model parameters, and is the true label of the input image.
Epsilion:
Epsilons
Epsilon:
Onggnal
Neturses
Orignzl
Pyttrbed
Origzal
Inturbad
3
3
3
3
Epsilon: 0.03
Epsilon: 0.04
Epsilon: 0.05
Ofgiinll
lerturte2
at:ha
Rettrted
Oftinal
Remurat2
3
3
3
3
3
3
Figure 17: Examples perturbed images generated using FGSM adversarial attacks on the MNIST dataset for different epsilon values. Epsilon values range from 0 to 0.05.
In experiments, applied FGSM adversarial attacks the MNIST dataset for standard CNN, Random-Sa NN (RS-NN), LB-BNN (Bayesian) and ENN (Ensemble) models. This involved per- turbing images from the MNIST dataset with noise generated using FGSM based the gradient of
34

---

Published as a conference paper at ICLR 2025
the loss function (namely, the cross entropy loss for CNN and the CB-RS loss, Sec. 3.2 for RS-NN).
Table 13: Test accuracies of CNN, RS-NN, LB-BNN and ENN models under FGSM adversarial attacks on MNIST and CIFAR-10 10 dataset for different epsilon values. Epsilon values range from 0 to 0.05.
Epsilon (6)
Dataset
Model
0.005
0.01
0.02
0.03
0.04
0.05
CNN
99.12
98.25
94.82
72.62
49.75
38.08
32.43
RS-NN
99.71
98 46.46
95.84
91.90
90.62
90.10
89.72
MNIST
LB-BNN
99.58
99.07
98.64
97.15
80.51
47.65
34.25
ENN
99.07
98.56
92.98
51.51
35.04
27.03
9.47
Test acc. (%)
CNN
92.08
41.54
20.88
9.18
5.81
4.43
4.05
RS-NN
93.53
63.42
61.73
61.67
61.53
61.12
60.35
CIFAR-10
LB-BNN
89.95
23.95
10.54
6.17
5.96
6.26
6.47
ENN
91.55
62.29
42.64
24.12
16.39
12.78
11.20
Subsequently, we evaluated the models' predictions on these perturbed images and computed their test accuracies. Tab. T3 presents the test accuracies of CNN, RS-NN, LB-BNN and ENN models under FGSM adversarial attacks the MNIST dataset for different values of
For CIFAR-1O, RS-NN demonstrates higher robustness to the attack compared to all other models with increased perturbations, while for MNIST, LB-BNN has higher accuracy at lower values but drops significantly for larger values. RS-NN circumvents the FGSM attack and shows consistent accuracy values for higher This is because FGSM for given input image is dependent the gradient of the loss, whereas RS-NN makes precise correct predictions with loss/gradient zero and very confident about these predictions. As result, the input image does not get perturbed and is correctly classified by RS-NN irrespective of the value.
Fig. 17 shows examples of perturbed images generated using FSGM adversarial attacks applied to the MNIST dataset for various epsilon values ranging from 0 to 0.05.
Additionally, conducted experiments using the Projected Gradient Descent (PGD) adversarial attack (Madry et al. 2017), which is a more iterative and stronger attack compared to FGSM. PGD generates adversarial examples by iteratively perturbing the input image in the direction of the gra- dient of the loss function, followed by projection step to ensure that the perturbation stays within a predefined range, typically defined by a norm ball around the original image. Mathematically. the perturbed image r' is computed as follows:
x 0) x, x (1+1) Proj (1 a sign (0,(0,xx)) ()))),
(12)
where x0() represents the image at the t-th iteration, a is the step size, é is the maximum allowable perturbation (i.e., the size of the B(x, c) ball), and Proj denotes the projection operation that ensures the perturbed image x' stays within the e-ball around the original image x:
B(x, c) fa' || x sc.
(13)
After number of iterations, typically denoted as T, the final perturbed image x' is obtained. This it- erative process makes PGD a more powerful adversarial attack compared FGSM. Tab. p4ereeett the test accuracies of CNN, RS-NN, LB-BNN and ENN models under the PGD adversarial attack on MNIST and CIFAR-10 10 datasets for different values of e.
On MNIST, LB-BNN does well for 0.005 and 0.01, but CNN, LB-BNN and ENN suffer significant drops in accuracy at higher e values. RS-NN performs better at higher e values. On CIFAR-10. 10, RS-NN significantly outperforms all models at all values, while the other models show a significant drop in performance, even at = 0.005. As explained earlier, RS-NN makes precise predictions with gradient zero and, therefore, is unaffected by PGD.
35

---

Published as a conference paper at ICLR 2025
Test accuracies of CNN, RS-NN, LB-BNN and ENN models under PGD adversarial attacks on MNIST and 10 datasets with 1/255 and number of iterations = 10. Epsilon values range from 0 to 0.05
Table 14:
Epsilon (c)
Datasel
Model
0.005
0.01
0.02
0.03
0.04
0.05
CNN
99.12
98.90
55.44
14.03
8.11
6,58
6.58
RS-NN
99.71
99.13
93.39
91.06
89.25
89.07
89.02
MNIST
LB-BNN
99.58
99.35
99.01
33.67
16.35
13.64
13.64
ENN
99.07
98.43
78.09
20.64
9.94
3.78
0.93
Test acc. (%)
CNN
92.08
23.82
2.47
0.04
0
RS-NN
93.53
60.23
59.98
59.97
59.97
59.97
59.97
CIFAR-10
LB-BNN
89.95
6,28
0.51
0.55
0.59
0.59
0.59
ENN
91.55
33.95
4.40
0.15
0.02
0
E.5 UNCERTAINTY ESTIMATION FOR RS-NN
In this section, we discuss the uncertainty measures of RS-NN: pignistic entropy ($E.5.1) and credal set width ($E.5.2). and their correlation with confidence scores E.5.3)
E.5.1 ENTROPY OF PIGNISTIC PREDICTIONS
The plots shown in Fig. 18 provide comparative analysis of the entropy values across various machine learning models under two conditions: In-Distribution (iD) and Out-of-Distribution (OoD) datasets. Each subplot represents a different dataset CIFAR-10 10, MNIST, and ImageNet - high- lighting the performance of multiple models, including RS-NN, LB-BNN, FSVI, DE, ENN, and CNN.
Fig. 18 illustrates that RS-NN exhibits the best iD vs OoD entropy ratio among the evaluated baseline models. Specifically, RS-NN maintains significantly lower entropy values for iD datasets, indicating a strong ability to recognize familiar patterns while showing elevated entropy for OoD datasets. DE and CNN closely follow RS-NN but struggle to effectively differentiate entropy for the ImageNet dataset. LB-BNN shows more uncertainty regarding iD samples compared to OoD samples. FSVI demonstrates the weakest distinction between iD and OoD.
Enyroyy Comsarison for OPAR1O
Entroyy Compor Son for VNIST
Ertroay Comparison Imagetet
[imapetie-0))
1,21
RS-488
15A
0064
CHNl
680
R5 404
10.800
C99
RS-481
18-800
8514
18
Moods
Modets
Nodel
Figure 18: Entropy comparison of all models for iD and OoD datasets. The plots illustrate the performance of various models, with error bars indicating the standard deviation of entropy values.
Tab. 15 shows sample predictions by RS-NN: belief functions, masses, and pignistic predictions for given samples of CIFAR- 10. The predicted belief, mass values, pignistic probabilities, and entropy are illustrated for two CIFAR-10 predictions. In the top figure, corresponding to the true label "horse," " the model makes a highly confident prediction with 99.9% confidence and a low entropy of 0.0017. Conversely, the bottom figure, associated with the true label "cat," represents an uncertain prediction with 33.3% confidence and a higher entropy of 2.6955. It's important to note that the second image is slightly unclear and poor in quality.
36

---

Published as a conference paper at ICLR 2025
Table 15: The predicted belief, mass values, pignistic probabilities, and entropy for two CIFAR- 10 predictions. The figure on the top (True Label = "horse") is a certain prediction with 99.9% confidence and low entropy of 0.0017, whereas the figure the bottom (True Label "cat") uncertain prediction with 33.3% confidence and higher entropy of 2.6955.
Sample
Belief
Mass
Pignistic
Entropy
('brrse' "bird") 0,9999402
("hozse') "eat', "truck')
0.9999175
horse 0.9998833
00001040148
Thorse', 'dog') 0,9999225
6.859753e-05
trick
1.5826409--05
(horse')
'ship', "bird")
4.094290c-05
3.3859180c-05
Phorse', 'deer")
0,9998697
"hocee' "bird")
2,250525e-05
bird
3 (53
('cat'. 'truck" 7.0380207c-05
('dog')
1.717869c-05
ship 2.30605-1e-055
('deee', 'cat")
0,4787728
('deec')
0,3104962
0.3332411
2.6955228261
'deer's 'alrpaane'' "bird")
0.4126398
"oat', (trock))
0.1762222
cal
0.2230723
("horse', 'deez'> ('deer', "bird")
0.37322957
"dog'. "bird")
0,0998060
horse
0.1153417
0,3658997
"horee', "bird")
0,0954350
1.866245
('dees'. "
0.3651531
("bied')
0.0524873
deg
0.1039505
Fig. 19 shows the entropy for two samples of CIFAR-10. 10, one is slightly uncertain image, whereas the other is certain with high belief values and lower entropy. Both results are shown for K = 20 classes, additional to the singletons.
Iheeched Chow: (vei)
Prokicel enc (veg)
Eve thoie out
Cor
Figure 19: Entropy. predicted class and belief value graph for samples of CIFAR- dataset.
20
Unouranna
55
RS NN
ENN LS-BNN
Degris
0
3
3
M
Figure 20: Uncertainty (entropy) of MNIST digit '3' rotated between -90 and 90 degrees. The blue line indicates RS-NN estimates low uncertainty between -30 to 30, and high uncertainty for further rotations.
Qualitative results of entropy. Fig. 20 depicts entropy- uncertainty estimates for rotated out-of-disttibution (OoD) MNIST digit samples. All models accurately predict the true class -0%, (8,, and 30". RS-NN and L.-BNN consistently exhibit low entropy in these scenarios,
37

---

Published as a conference paper at ICLR 2025
IID MNIST
CIFARIO
0OD
Fashion-
00D = SVHN
38-
0OD
KucushijiMMNS)
000 Intel-Image
Donsity
Density
0.6
3.5
25
1.5
2.
5.0
0.5
1.0
2.0
3.0
Entropy
Entropy
Figure 21: Entropy Distributions for RS-NN on Figure 22: Entropy Distributions for RS-NN on MNIST vs Fashion-MNISTIN SaA---NNISSKKKuusij--MNNSS CIFAR-10 vs SVHN/Intel-Imag
while ENN shows higher entropy for correct classifications at these angles. As the rotation angle increases, indicating more challenging scenarios, all models predict the wrong class. Notably, LB- BNN fails to exhibit a significant increase in entropy for these incorrect predictions. ENN performs relatively better than RS-NN 60% and 90 rotations but has previously demonstrated high entropy levels even for accurate predictions at relatively minor rotations -30 and +30 degrees. Overall, RS-NN provides more reliable measure of uncertainty.
Figs. 21 and 22 show the entropy distributions for RS-NN on MNIST (iD) vs Fashion-MNIST JuZUSKJJJJJJJ (OoD), and CIFAR-IO (iD) vs SVHN (OoD))ntel-Image (OoD), respec- tively. There is a clear iD vs OoD shift in entropy for RS-NN, as detailed in Sec. 4.4
The plots show that RS-NN consistently exhibits larger iD vs OoD entropy ratio for all datasets. For iD data, the entropy values are generally lower, indicating that these models exhibit a higher level of confidence when making predictions within familiar datasets. Conversely, the OoD side shows significantly higher entropy values for most models, particularly notable with the SVHN and Intel datasets, suggesting that the models struggle to generalize when faced with unfamiliar data. The error bars represent the standard deviation of the entropy values.
E.5.2 CREDAL SET WIDTH
Crodal set u dth for Inco noctly Clasiifed Samples
Credal sctwidch for Comrectyy Classifie Samples
0.2
0.6
64
0.2
0.7
0.8
Figure 23: Credal set widths for individual classes of CIFAR-10 dataset. For Incorrectly Classified
samples (top). For Correctly Classified samples (bottom).
Fig. 23 plots the credal set widths for incorrectly classified (left) and correctly classified (right) samples over each class e in CIFAR-10, 10, respectively. The box represents the interquartile range between the 25th and 75th percentiles of the data encompassing most of the data. The vertical line inside the box represents the median and the whiskers extend from the box to the minimum and maximum values within certain range, and data points beyond this range considered outliers
38

---

Published as a conference paper at ICLR 2025
plotted individually as dots circles. The box plots here shown for 10,000 samples of CIFAR-10 dataset where most samples within the incorrect classifications have higher credal set widths indicating higher uncertainty in these samples, especially for classes 'bird', 'cat', 'deer', and "dog".
and are
E.5.3 ENTROPY VS CREDAL SET WIDTH
There has been considerable research on uncertainty measures for credal sets. One of the earlier works by Yager 2008) makes the distinction between two types of uncertainty within credal conflict (also known randomness discord) and non-specificity, Non-specificity essentially varies with the size of the credal set (Kolmogorov 1963). Since by definition, aleatoric uncertainty refers to the inherent randomness or variability the data, while epistemic uncertainty relates to the lack of knowledge information about the system (Hüllermeier & Waegeman, 2021), conflict and non-specificity directly parallel these concepts. These measures of uncertainty are axiomatically justified (Bronevich & Klir 2008).
Figs. 24 and 25 depict the relationship between the entropy of RS-NN pignistic prediction and the associated confidence level (Fig. 24). and between credal set width and confidence (Fig. 25 for the CIFAR- 10 (left) and SVHN/Intel Image (right) datasets. The distribution of iD predictions (left) for both tests show concentration at the top left, indicating high confidence and low entropy credal set width. Conversely, OoD predictions (middle, right) exhibit a more dispersed pattern.
ID: CIFARIO
OoD: SVHN
OnD: Intel
977
Confidence Score
Frequency
53
5.25
0.0
3.75
2.00
1.75
2.62
Entropy
Entropy
Entropy
Figure 24: Entropy vs Confidence score on iD (left) vs OoD (right) datasets. For CIFAR-10, most predictions are concentrated top left of the plot indicating lower entropy and higher confidence in the predictions. For SVHN and Intel Image datasets, predictions are more distributed.
ID: CIFAR10
OcD: SVHN
OoD: intel Image
0.75
Confidence Score
0.
0.25
5.0
2:75
2,80
2.62
1:75
2.02
Credal Width
Credial Set Width
Credal Set Wiath
Figure 25: Credal Set Width Confidence score on iD (left) vs OoD (right) datasets. For CIFAR- 10, confidence scores are high and credal set width is small. For SVHN and Intel Image datasets, credal set width varies for each prediction and less reliant on confidence score.
Entropy, reflecting prediction uncertainty, quite correlated with confidence, both iD and OoD tests. In contrast, as it considers the entire set of plausible outcomes within belief function rather than single prediction, credal set width better quantifies the degree of epistemic uncertainty inher- prediction. As result, credal set width less dependent the concentration of predictions
39

---

Published as a conference paper at ICLR 2025
and is more reflective of the overall uncertainty encompassed by the model. Fig.
25 shows that,
unlike entropy, credal with is clearly not correlated with confidence.
E.6 ABLATION STUDY ON HYPERPARAMETERS a AND 3
The regularization serves the purpose of ensuring that our model generates valid belief functions, which are constrained by maintaining a sum of masses equal to I and ensuring non-negativity. These terms are designed to penalize deviations from valid belief functions. However, it is crucial not to assign too much weight to this term, as excessively penalizing deviations may hinder the model's ability to accurately classify data points. For example, in a Variational Auto Encoder (VAE), if we assign too much weight to the KL divergence term in the loss, the model may prioritize fitting the latent distribution at the expense of reconstructing the input data accurately. This imbalance can lead to poor reconstruction quality and suboptimal performance on downstream tasks.
Hence, it is essential to properly tune the values of and 3 to ensure that these regularization terms play a meaningful role the training. The ablation study on hyperparameters is conducted to examine the impact of and / on the model's accuracy. It demonstrates that small values suffice for these parameters.
In Fig. 26), we show the test accuracies for different values of hyperparameters and 3 in Cn-RS ns loss function (Eq.7). Hyperparameters a and / adjusts the relative significance of the regularization terms M, and M, respectively in LB- ns8 loss. The test accuracies dataset with a fixed number of focal sets K = 20 and varying a (blue) and / (red) values, a/3 [0.001, 0.005, 0.006, 0.009, 0.01, 0.015, 0.020, 0.025, 0.03, 0.04, 0.05]. Test accuracy the highest when a and B equals le 3 3.
calculated for CIFAR-10
In our experiments across various datasets and architectures, we found that a value of 1e for the hyperparameters yields satisfactory results. This includes architectures ranging from ResNet-50 to Vision Transformers (ViT-Base-16) and datasets ranging from MNIST (+x 60,000 images of 10 classes) to ImageNet (= 1.1M images of 1000 classes). While conducting parameter search for each dataset could potentially lead to further optimization, is worth noting that, many cases, this step can be omitted without sacrificing performance significantly. For further optimization one can, of course, perform a tailored parameter search per dataset, but this not quite necessary.
94.0
Max Accuracy: 93.53
a
0,001
93.5
Test Accuracy
93.0
92.5
92.0
91.5
91.0
0.00
0.01
0.02
0.03
0.04
0.05
Hyperparameters a
Figure 26: Test accuracies RS-NN on the CIFAR-IO dataset using ResNet50, for a fixed K 20 and different values of hyperparameters and 8 in the loss function.
An alternative loss with valid belief functions. A straightforward way of enforcing the positivity of the masses is to apply softmax over the masses computed the BCE loss. This ensures that the masses are non-negative and sum to one. After obtaining the softmaxed masses, one can compute the belief functions from these normalized masses and minimize the loss between this computed belief and the original unnormalized belief.
The random set loss LRs now becomes,
Lns LBCE LBCEnerm
40

---

Published as a conference paper at ICLR 2025
where LECE is the BCE loss between belief function logits and ground truth, and EBC is the BCE loss between the normalized belief function logits reconstructed from mass logits that have been softmaxed.
We implemented this simple method on ResNet50 RS-NN on the CIFAR-10 dataset and obtained accuracy of 92.47%. This is quite close to original accuracy of 93.53% with the mass regu- larizations. This mirrors results the literature showing that soft constraints work as well as hard ones in practice (Márquez-Neila et al. 2017).
E.7 ABLATION STUDY ON THE NUMBER OF FOCAL SETS
90.0
85.0
82.5
Acruracy
77.5
75.0
70.0
20
60
100
500
1013
Figure 27: Ablation study on number of non-singleton focal sets K on CIFAR- 10 dataset using ViT-Base- 16 (with a = 8 = le 3). The maximum value of K can be 1013 for 10 classes (after excluding the singletons and empty set).
The number of non-singleton focal sets K to be budgeted is a hyperparameter and needs to be studied. A smaller value of K can lead to more similar results to classical classification while larger value of K can increase the complexity. Therefore, we conducted an ablation study on K on the CIFAR-10 dataset. We found that small value of K (comparable to the number of classes) works best and even performs better than K = 0 (when there are no non-singleton focal sets) (see Fig. 27). Hence, using set prediction along with the proposed budgeting algorithm not only helps induce uncertainty quantification but also improves the performance of the model. Note that performance comparison
made in terms of accuracy. Smet's Pignistic Transform (Smets) 2005) was used to compute class-wise probabilities from the predicted belief function. Fig. 27 shows an ablation study on different number of focal sets K for CIFAR-10 on the Vision-Traes Transformer (ViT-Base-16) 16) model with a = 3 le - 3.
To further support our claim, we compare a budgeted RS-NN with standard RS-NN (full 2 sets) on the CIFAR-10 datasets using ResNet50 as the backbone. In Tab. [6 below, we report the test accuracy, OoD detection metrics (AUROC. AUPRC), pignistic entropy and credal set width for the standard RS-NN vs budgeted RS-NN. The standard RS-NN has 1024 (220) sets since CIFAR-10 has 10 classes, and the budgeted RS-NN has K 20 focal (non-singleton) sets, so 10 + 20 30 input sets. Tab. 16 shows that budgeted RS-NN performs better than the standard model, especially iD vs OoD entropy, and AUROC and AUPRC scores.
As we say in the main paper, while, in theory, using the full power set is ideal for uncertainty estimation, in practice, having all those degrees of freedom might make more difficult for the network learn. When model is confronted with an overwhelming number of input sets, it may struggle to identify meaningful patterns amidst the noise created by less relevant or redundant combinations. This excessive flexibility can hinder the network's ability to converge effectively, as
41

---

Published as a conference paper at ICLR 2025
it may become trapped in local minima overfit to the training data. This can dilute the model's capacity to generalize well to unseen data, thereby complicating the learning of the underlying relationships between features and the set structure of the data.
Table 16: Comparison of accuracy, AUROC, AUPRC, pignistic entropy, and credal set width for standard RS-NN vs budgeted RS-NN on the CIFAR-10 dataset.
In-distribttion (ID)
Duf-of-iitrriutiivn (Cho)
Ditaset
Model
Test accuracy (9) (7)
BCE (1)
SVHN
Intel Imaage
AUROC'(T) (t>
AUPRCC ct>
AURUC' (t)
AUPKC (f)
Standard KS-NN Rudgeted RS-NN
92.85 93. 53
000G01
93.39
91.28
16.84
89 89.53
0.0484
94.91
93.72
97.39
90 27
CIFAR- 10
SVIN
Intel Irsuge
Entrupp GD) (L)
Credal Set Width (I)
Eniropy (1)
Crodal Sei Width (1)
Eniropy (
Credal Set Wsath (1)
Standard RS NN
0. 0.266
1. 0.80
0,048 1. 0.15
1.205 1. 1.20
0.406 1 0.07
1,450 1 0.71
0.669 1 0.21 0.587 0.367
Budletted RS-NK
0. 0,088 0,308
0.007 :: 0.044
1.132 0.855
0,200 0.522
1.517 1. 0.740
E.8 ON THE FEASIBILITY OF BUDGETING OF SETS
Tabs. 17 and 18 show that replacing t-SNE with UMAP for budgeting in the RS-NN model yields comparable performance in terms test accuracy, out-of-distribution (OoD) detection, and uncer- tainty estimation. Both t-SNE and UMAP are capable of embedding data in a 3D space, with UMAP employing a faster algorithm that utilizes five nearest neighbors and a minimum distance of 0.9. This approach ensures that the integrity of the data structure is maintained while significantly reducing computational costs. UMAP offers an efficient alternative without compromising the model's 's ef- fectiveness. To make the generation of embeddings for budgeting of focal sets more efficient, can utilize faster alternatives like UMAP instead of t-SNE, without compromising the overall model performance.
Table 17: Test accuracy and OoD detection (AUROC, AUPRC) results on RS-NN where budgeting is done using t-SNE vs budgeting done on UMAP.
In-distribution (ID)
Out-of-distribution (OoD)
Test accuracy (%) (t)
SVHN
Intel Image
Dataset
Model
BCE (L)
AUROC (1)
AUPRC (t)
AUROC (t)
AUPRC (1)
RS-NN (t-NEE)
93.53
0.0484 0.0482
94.91
93.72
97.39
90.27
CIFAR-10
RS-NN (UMAP)
92.97
94.31
92.46
96.22
89.66
Table 18: Entropy and credal set width results on RS-NN where budgeting done using t-SNE vs budgeting done on UMAP. The goal is to minimize both metrics for in-distribution (iD) data while increasing them for out-of-distribution (OoD) data.
distribution (iD)
Ofs-off-istiibtioon (OcD)
Dataset
Model
Entropy (
Credul Set Wiltt (})
SVHN
Intel Image
Entropy (1)
Credal Set Width (1)
Entropy (1)
Crodal Sei Width (T)
CIFAR-10
RS-NK (-SNE)
0.088 :: 0. 303
0.007 0.044
1.132 0.855
0. 1.260 +: 0.322
1.517
0,740
0.587 0.367
RS-NN (UMAP)
1.119 0.887
3.374
1.423
0.762
0.582 0.442
A 2D visualization of the ellipses formed by calculating the principle axes and their lengths using eigenvectors and eigenvalues obtained from GMMs, for each class e (Spruyt 2013) is shown in Fig.
28
The budgeting procedure can also be modified to accommodate multiple data sources or a continuous stream of data. t-SNE can be replaced by an autoencoder (this will need to be trained first) or PCA to add continual inference capabilities in the dimensionality reduction step. Similarly, Gaussian-based dynamic probabilistic clustering (GDPC) (Diaz-Rozo et al.) 2018) can be used instead of GMM clustering to handle a continuous stream of data. GDPC works by first initialising a GMM and then updating its parameters as it sees new data. In alternative, (Kulis & Jordan| 2011) 1) have proposed an interesting approach to learn a GMM from multiple sources of data by maintaining a local and global mixture model. Both these approaches can be plugged into our budgeting framework with
42

---

Published as a conference paper at ICLR 2025
30
-14
Figure 28: 2D visualization of the clusters of 10 classes of CIFAR- dataset and the ellipses formed by RS-NN based on the hyperparameters of Gaussian Mixture Models.
little modifications to add continual learning capabilities to it. The computation of cluster overlaps remains the same, overlapping scores and the resultant focal sets will need be updated as the clusters evolve. However, a cluster tracking strategy (Barbara & Chen 200T) can be employed so that the overlap assessment step is only re-done when sufficient drift has been detected in the clusters. All the proposed changes are efficient enough to not have a significant effect on the overall time of budgeting.
F APPLICATION ON TEXT CLASSIFICATION
The random-set approach can be applied on any classification task. this section, we detail experi- ments performed on text classification.
Dataset. We chose the BBC text dataset (Greene & Cunningham, 2006), which consists of various news categories such tech, business, sport, and entertainment, and used this data for training model. The task is to classify the given text into one of these categories. The dataset structured with columns: category and text, where the category the label, and the text the content to be classified. The text data will be preprocessed and fed into the model to predict the category of each text.
Model. To train our model, we leveraged pre-trained BERT model available on TensorFlow Hub. Specifically, we used the small BERT (L 444 512 8) model, which is lightweight version of BERT. The model is designed to take text input, preprocess with BERT's preprocessing layer, pass through BERT's encoder to generate embeddings, and then use a dropout layer. The final layer is a fully-connected layer with sigmoid activation for the RS-NN BERT classifier (RS-NN BERT), and fully-connected layer with softmax activation for the standard BERT classifier (CNN BERT).
Training. We trained these models on the BBC text dataset, fine-tuning the BERT model as part of the training process. Both models were trained for 10 epochs using Adam optimizer, batch size of 32, and learning rate of 3e-5. RS-NN BERT has all the sets of classes excluding the null set and full set (25 = 32 = 30 classes) and CNN BERT has the 5 original classes.
detection. For OoD detection, we use the Emotion Detection from Text (Seyeditabari et al.] 2018) dataset which contains tweets annotated with emotional labels. The dataset includes three columns: tweetid, sentiment, and content, where sentiment represents the emotion behind each tweet. The dataset includes 13 emotion classes such as anger, fear, joy, love, sadness, and surprise, etc, aiming identify emotional expressions in text.
43

---

Published as a conference paper at ICLR 2025
Experimental results. In Tab. 19 below, show the test accuracy, out-of-distribution (OoD) metrics (AUROC, AUPRC), and the in-distribution (iD) vs OoD entropy for RS-NN BERT and CNN BERT. RS-NN achieves higher overall performance with significantly higher AUROC and AUPRC scores, highlighting the efficiency of the model at differentiating between iD and Ool) samples. RS-NN has higher OoD entropy than CNN, but also has higher iD entropy than CNN, which indicates that the uncertainty in predictions is higher as it is a small dataset.
Table 19: Test accuracy, AUROC, AUPRC, iD vs OoD entropy for RS-NN BERT and CNN BERT (both fine-tuned on BERT).
Emotion Detection (OoD)
Dataset (iD)
Model
Test Acc. (%) (1)
iD Entropy
AUROC (t)
AUPRC (f)
Entropy (f)
RS-NN BERT
96.85
0.541
0.196
95.19
95.93
1,510 0.611
BBC Text (iD)
CNN BERT
94.15
0.027
0.125
56.71
57.54
0.059 = 0.191
44


Reviews:
[{"review_id": "ipfFRX5AGG", "reviewer": "Reviewer_8ggx", "summary": "This paper focuses on a novel approach called Random-Set Neural Network (RS-NN) for classification tasks. The RS-NN approach aims to predict belief functions over a class list using random sets, which can represent uncertainty induced by limited training data. RS-NN can be presented as a wrapper technique that can enhance existing baseline models by predicting beliefs for sets of classes. Specifically, it efficiently selects relevant sets of classes using GMMs and introduces measures for assessing uncertainty in random-set predictions. In the experiments, RS-NN is compared with Bayesian and Ensemble methods, demonstrating better performance in uncertainty estimation on multiple benchmarks.", "questions": "1. Please clarify the validity of belief functions in Eq. 7.\n2. Please discuss and compare with some recent uncertainty works in the experiments.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and well-organized.\n2. The issue of uncertainty in neural networks is crucial. The proposed method can be easily extended to other baselines, which could benefit the model trustworthiness community.\n3. The proposed method is much more efficient than other baselines, making the proposed method more accessible and feasible for adoption.\n4. The evaluation is sufficient. The effectiveness of the proposed method is validated on various models and datasets.", "weaknesses": "1. One concern lies in the formulation of Eq. 7, where the validity of belief functions is achieved by the involvement of loss terms $M_s$ and $M_r$. However, according to the ablation studies in Figure 26, a large hyperparameter of $\\alpha$ and $\\beta$ could lead to a decrease in accuracy, which makes it a trade-off between the accuracy and validity of belief functions. How to balance this trade-off is not well-explained in this paper. Although the authors mentioned that any negative masses will be set to zero in post-training, the gap caused by this process is not discussed or explained.\n2. Another concern lies in the comparison. Most recent model calibration works that focus on uncertainty have not been discussed or compared in this paper, such as [a,b,c,d]. Furthermore, the evaluation of adversarial robustness could be weak. It is necessary to conduct some evaluation on CIFAR-10 under a stronger PGD attack to demonstrate the robustness.\n\n[a]. Calibrating Deep Neural Networks using Focal Loss. NeurIPS 2020.\n\n[b]. Improving model calibration with accuracy versus uncertainty optimization. NeurIPS 2020.\n\n[c]. Local Calibration: Metrics and Recalibration. UAI 2022.\n\n[d]. Dual focal loss for calibration. ICML 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OLzzXtkPBZ", "reviewer": "Reviewer_KLiR", "summary": "The authors introduce a novel Random-Set Neural Network (RS-NN) approach for epistemic uncertainty estimation in classification. RS-NN can be applied on top of an existing neural network and predicts distributions (belief functions) over collections of sets of classes and makes use of pignistic prediction to estimate the entropy. The authors theoretically motive their work and empirically show, that RS-NN outperforms state-of-the-art Bayesian and Ensemble uncertainty estimation methods on multiple metrics and datasets.", "questions": "I have compared the reported results for LB-BNN, FSVI, and Deep Ensemble with their sources and was surprised to find large differences in the reported metrics compared to the results reported in the manuscript in Table 2 (OoD detection performance and uncertainty estimation).\n\nWhile the manuscript claims \"Note that the results in the FSVI paper (Rudner et al., 2022) are based on a pre-trained ResNet18\", having read the FSVI paper and supplementary, I was not able to confirm this. See also Appendix D.3 of FSVI[1]:\n\n> In this experiment, a standard ResNet-18 network architecture was used. All models are trained for 200 epochs with a mini-batch size of 128 using SGD with a learning rate of 5 × 10−3, momentum (with momentum parameter 0.9), and a cosine learning rate schedule with parameter 0.05\n\nThe reported metrics use ResNet-50 everywhere, while LB-BNN, FSVI, and Deep Ensemble use a ResNet-18 on CIFAR-10. Using ResNet-50 (25.6M parameters) should in principle only improve the results as we increase model capacity, compared to ResNet-18 (11.7M parameters).\n\n**CIFAR-10**\n\n| Method                      | Model                                             | Test accuracy (%) | ECE $\\downarrow$ | AUROC (SVHN) $\\uparrow$ |\n|-----------------------------|---------------------------------------------------|:-------------------:|:------------------:|:-------------------------:|\n| RS-NN                       | ResNet-50                                         | 93.53             | 0.0484           | 94.91                   |\n| LB-BNN (reported)           | ResNet-50                                         | 89.95             | 0.0585           | 88.14                   |\n| LB-BNN (source)             | ResNet-18                                         | **95.40**         | 0.3090           | 96.50                   |\n| FSVI (reported)             | ResNet-50                                         | 80.29             | 0.0521           | 80.59                   |\n| FSVI (source)               | ResNet-18 ($p_{x_{C}} = \\text{rand. monochrome}$) | 93.35             | 0.0340           | 94.76                   |\n| FSVI (source)               | ResNet-18 ($p_{x_{C}} = \\text{CIFAR-100}$)        | 93.57             | 0.0260           | 98.07                   |\n| FSVI Ensemble (source)      | ResNet-18 ($p_{x_{C}} = \\text{rand. monochrome}$) | 95.19             | **0.0130**       | **99.19**               |\n| Deep Ensemble (reported)    | ResNet-50                                         | 92.73             | 0.0482           | 93.84                   |\n| Deep Ensemble (FSVI source) | ResNet-18                                         | 95.13             | 0.0190           | 98.04                   |\n\nI appreciate, that the authors took the time and effort to run the compared methods themselves, improving comparability in principle. Nevertheless, it surprises me, that the original (highlighted as *source* in the table above) results for LB-BNN, FSVI, and DE all *outperform* RS-NN in test accuracy, ECE, and AUROC of OoD detection by a significant margin. I have focused on the CIFAR-10 results, as this was a dataset with SVHN as OoD dataset used in all three sources.\n\nCould it be, that the compared methods are, as always, are sensitive to hyperparameters, and the authors probably have unintentionally picked suboptimal hyperparameters for those methods by accident? Or am I missing some important detail?\n\nWhile I generally really like the manuscript and experimental evaluation, the above leaves me questioning the validity of the comparisons and the claims of \"beating state-of-the-art methods\" in all experiments. I usually do not particularly care about beating the state-of-the-art, but if a method happens to be misrepresented and suddenly goes from being the best method to being the worst method in the comparison, I get interested as to why that is (as I said, I'm probably just missing an important detail -- I'm not implying that the authors did misrepresent anything on purpose). I will happily raise my score to 6 or 8 if the authors can clarify these discrepancies.\n\n[1] Tractable Function-Space Variational Inference in Bayesian Neural Networks; Tim G. J. Rudner, Zonghao Chen, Yee Whye Teh, Yarin Gal. https://arxiv.org/pdf/2312.17199\n\n**Misc. Questions**:\n- L80: \"the entropy of a pignistic prediction is more diverse than that of a Bayesian prediction across in-distribution and out-of-distribution data (see Tab. 2)\" -> What does \"more diverse\" mean?\n- E.4, Tab. 13: Why was the influence of $\\varepsilon$ not evaluated against LB-BNN and ENN as the authors did in Table 12 for different rotations of MNIST and Table 11 for different perturbation noise levels?\n- L212, Eq. (4): How is the set of probability distributions $P$ on $\\Theta$ selected/constructed?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The manuscript is generally well written and in most parts easy to follow. Examples and visualizations are provided where necessary or helpful for the reader.\n- The authors deliver a good example for belief functions in Section 2 which easy to follow, and greatly helps to understand the previous definitions.\n- Visualizations in Figure 1, 3, 4 and 5 are well crafted and complement the understanding of the described procedures.\n- The experimental evaluation is detailed and covers multiple methods, datasets and aspects to investigate. Most experiments were run multiple times, the standard deviation was reported in addition to the mean result.\n- The proposed method impressively beats most of the previous state-of-the-art results in terms of accuracy, uncertainty estimation, and out-of-distribution detection while keeping the inference time of the base model (note, that this does not include the additional time one-time cost for the budgeting step).\n- The appendix contains a lot of additional experiments and in-detail explanations and recaps for relevant methods. While being relatively long (23 pages), the authors managed to insert references to the appendix in the main body of the manuscript at many relevant places. I found myself looking into the appendix more than usual compared to other papers with long appendices (which I see as positive).", "weaknesses": "The experimental results solely focus on image classification tasks. What about a different modality? Tabular data? Text data? The proposed method is not specific to image data and can be applied to arbitrary models and modalities with classification tasks.\n\nNot having worked with belief functions yet, it was at first unclear to me, that Eq. (1) goes over *all possible subsets A* since L185 talks about \"mass values to *its* subsets\", implying that $m(\\cdot)$ is a function which is assigned / is only valid on its own set of subsets. Suggestion: make this consistent with the use of the powerset notation used in the directly following example (L193) and replace $\\forall A \\subset \\Theta$ in the middle of Eq. (1) with $A \\in \\mathbb{P}(\\Theta)$ at the end of the equation.\n\n\nMinor Errors and Suggestions (these did not impact the score):\n- L30: \"how confident ti\" -> \"how confident it\"\n- L100: \"can applied\" -> \"can be applied\"\n- L103: \"we outline an **ingenious** budgeting method\" -> In my opinion, the authors should refrain from calling their own contributions \"ingenious\".\n- L183 - L187: This part is a bit confusing, stating \"While [...] mass functions does X, a belief function does Y:\", then following with an equation of the mass function (the sentence implies, that the reader will now be shown the definition of a belief function, which only comes in the equation after the next paragraph).\n- L269 and multiple times in the appendix: Wrong quotation marks around words were used (\"foobar\" instead of ``foobar'').\n- L305: The definition of $M_r$ should appear directly after introducing $M_r$ in the text. I guess the authors squashed this into Eq. (6) together with $M_s$ to safe space, but it would improve readability to separate both definition, since right now the text talks about $M_s$ (\"we also add a mass sum term $M_s$, namely:\") and then shows a definition of $M_s$ *and* $M_r$. Since the camera-ready version does not allow an additional page, the authors could also rewrite the text preceding Eq. (6) to make this more clear.\n- L432, Tab. 2: The highlighted results (bold-face) for \"In-distribution Entropy\" for ENN on ImageNet is wrong and should be FSVI (lowest), and for \"Out-of-distribution Entropy\" for RS-NN on F-MNIST is wrong and should be DE (highest).\n- L1362: \"to a 95The class\" -> \"to a 95% confidence interval. The class\" (?) -- I guess the \"%\" was not escaped and commented out the rest of the line in LaTeX.\n- L2144, E.7, Tab. 15: The highlighted results (bold-face) for \"Entropy (OoD)\", \"Credal Set Width (OoD)\" on SVHN and \"Credal Set Width (OoD)\" on Intel Image are flipped (in all cases \"Budgeted RS-NN\" was highlighted as \"winner\" while \"Standard RS-NN\" achieved a better performance).\n- E.4, L1830: The authors speak of \"Tab. 1 in the anonymous PDF linked to this response\". This reads like a leftover from a previous submission in which this part of the appendix was a response to a review.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9CNh03I6Gj", "reviewer": "Reviewer_N6Y1", "summary": "This paper introduces Random-Set Neural Networks (RS-NN), a novel approach to classification that predicts belief functions over sets of classes rather than probability vectors. The key innovation is using random sets and belief functions to model epistemic uncertainty arising from limited training data. RS-NN extends any baseline neural network by modifying its output layer to predict belief values for relevant sets of classes, identified through an efficient \"budgeting\" procedure that selects the most informative class combinations. The approach measures uncertainty through both the entropy of the \"pignistic\" (central) probability prediction and the width of the predicted credal set (set of compatible probability distributions). Through extensive experiments on multiple benchmarks, the authors claim RS-NN outperforms state-of-the-art Bayesian and ensemble methods in accuracy, out-of-distribution detection, uncertainty quantification, and adversarial robustness, while being computationally efficient. The approach is applied to different network architectures and provides statistical guarantees through conformal prediction.", "questions": "1. Please address the point raised in section Weaknesses.\n2. The authors claim \"Current machine learning models struggle to provide reliable predictions when confronted with unfamiliar data (Guo et al., 2017a)\". The paper is from 2017. How did the authors infer no progress has been made since ?\n3. The authors claim \"RS-NN uses a random-set framework which does not require prior assumptions and is more data driven\". Both models are classifying the same data, hence the claim that one is more data driven than the other does not hold and additional details are required to make the text clear.\n4. The authors provided code to replicate results. When running eval.ipynb, line:\n\nmodel = load_model(selected_model, selected_dataset, model_type = \"CNN\")\n\nthrew error:\n\nOSError: No file or directory found at saved_models/CNN_resnet50_cifar10.keras\n\nThe weights file was found to be missing. Can the file be provided ?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "This paper describes a method in uncertainty estimation for deep learning, using the Random-Set Neural Network (RS-NN). The work discusses a theoretical framework using random sets and belief functions to model epistemic uncertainty, develops an innovative \"budgeting\" technique to make the approach computationally tractable, and proposes new uncertainty measures based on pignistic entropy and credal set width. The paper presents comprehensive empirical validation across multiple datasets and architectures showing improved performance to state-of-the-art Bayesian and ensemble methods in accuracy, out-of-distribution detection, and uncertainty quantification. The presentation is clear, with explanations of complex concepts supported by helpful visualizations and detailed appendices. The work is significant for machine learning deployment in safety-critical domains, offering a wrapper technique applicable to existing architectures while providing uncertainty estimates and robustness to adversarial attacks. The theoretical foundations in random set theory, combined with practical engineering solutions like the budgeting procedure, make this work both mathematically rigorous and implementable in practice.", "weaknesses": "The authors claim in the abstract \"In this paper, we propose a novel Random-Set Neural Network (RS-NN) approach to classification\nwhich predicts belief functions\" and go on to cite Manchingal & Cuzzolin, 2022 (w.r.t. epistemic uncertainty) where on pg 18 it is \"discussed under Sec. 6, the performance and evaluation metrics for random set neural networks\", as well as several mentions of belief functions throughout the earlier paper. The works seem closely related hence the claim to originality in the more recent work seems questionable, and a clarification in order.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)