Hyper-opinion Evidential Deep Learning for Out-of-Distribution Detection
Authors: Jingen Qu
Yufei Chen
Xiaodong Yue
Wei Fu
Qiguang Huang
Venue: NeurIPS 2024 poster (2024)
Keywords: Out-of-distribution Detection
Evidential Deep Learning
Uncertainty Estimation.
Primary area: probabilistic_methods

Abstract: Evidential Deep Learning (EDL), grounded in Evidence Theory and Subjective Logic (SL), provides a robust framework to estimate uncertainty for out-of-distribution (OOD) detection alongside traditional classification probabilities.However, the EDL framework is constrained by its focus on evidence that supports only single categories, neglecting the other collective evidences that could corroborate multiple in-distribution categories. This limitation leads to a diminished estimation of uncertainty and a subsequent decline in OOD detection performance.Additionally, EDL encounters the vanishing gradient problem within its fully-connected layers, further degrading classification accuracy.To address these issues, we introduce hyper-domain and propose Hyper-opinion Evidential Deep Learning (HEDL). HEDL extends the evidence modeling paradigm by explicitly integrating sharp evidence, which supports a singular category, with vague evidence that accommodates multiple potential categories.Additionally, we propose a novel opinion projection mechanism that translates hyper-opinion into multinomial-opinion, which is then optimized within the EDL framework to ensure precise classification and refined uncertainty estimation.HEDL integrates evidences across various categories to yield a holistic evidentiary foundation for achieving superior OOD detection. Furthermore, our proposed opinion projection method effectively mitigates the vanishing gradient issue, ensuring classification accuracy without additional model complexity. Extensive experiments over many datasets demonstrate our proposed method outperforms existing OOD detection methods.

Hyper-opinion Evidential Deep Learning for Out-of-Distribution Detection
Jingen Qu
Yufei Chen"
School of Computer Science and Technology Tongji University, Shanghai, China newcityštonge i. edu. cn
School of Computer Science and Technology Tongji University, Shanghai, China yufeichen@tongji edu. cn
Xiaodong Yue Artificial Intelligence Institute Shanghai University, Shanghai, China. yswantfly@shu. edu. cn
Wei Fu
School of Computer Science and Technology Tongii University, Shanghai, China cs os_fuwei@outlook.o com
Qiguang Huang
School of Computer Science and Technology
Tongji University, Shanghai, China 17535438tongji. edu.cn
Abstract
Evidential Deep Learning (EDL), grounded in Evidence Theory and Subjective Logic (SL), provides a robust framework to estimate uncertainty for out-of- distribution (OOD) detection alongside traditional classification probabilities. However, the EDL framework is constrained by its focus on evidence that supports only single categories, neglecting the other collective evidences that could cor- roborate multiple in an-iitributiin categories. This limitation leads to a diminished estimation uncertainty and a subsequent decline OOD detection performance. Additionally. EDL encounters the vanishing gradient problem within its fully- connected layers, further degrading classification accuracy. To address these issues, introduce hyper-domain and propose Hyper-opinion Evidential Deep Learning (HEDL). HEDL extends the evidence modeling paradigm by explicitly integrating sharp evidence, which supports singular category, with vague evidence that accommodates multiple potential categories. Additionally, we propose a novel opinion projection mechanism that translates hyper-opinion into multinomial- opinion, which is then optimized within the EDL framework to ensure precise classification and refined uncertainty estimation. HEDL integrates evidences across various categories to yield a holistic evidentiary foundation for achieving superior OOD detection. Furthermore, effectively mitigates the vanishing gradient issue, ensuring classification accuracy without additional model complexity. Extensive experiments over many datasets demonstrate our proposed method outperforms existing OOD detection methods.
proposed opinion projection method
Introduction
Deep Learning (DL) models have been widely adopted many real-worke applications 25 However, these models are trained under the implicit assumption that the training and test data are
57
64
"correpponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

drawn from the same distribution 70], leading to overconfident predictions 445 Thus when DL model encounters an input that differs from its training data, it may be overconfident with wrong prediction, bringing rise to the out-of-distribution (OOD) problem. The resolution of the OOD problem is of utmost importance, and researchers have devoted significant attention to studying the intricacies of OOD detection 5 16, 19
30 [3 43
To address OOD problem, a variety of methods have been developed in DL 51 Some researchers apply post-processors to the base classifier to generate an uncertainty score for OOD detection. These post-hoc methods only take effect at inference phase and are easy to use, but rely on the performance of the pretrained model. Others propose training methods that involve training- time regularization, which require more computational resources. To train an uncertainty- aaware model without additional computation, a recent search leverages Evidence Theory and Subjective Logic (SI.) with DNNs 54 called Evidential Deep Learning (EDI.) [55] estimation in neural networks which represents the degree of 'unknown' in opinion. It modifies the existing DL structure slightly and allows neural network to quantify the uncertainty for OOD detection with a well-defined theory framework. Evidential models have been extended to many areas
7 EDL offers uncertainty
such as open set recognition (2, classification 33 32 [2 36 33], multi-view learning 72 68 23 34
The EDL models face several challenges, with one primary issue arising from the theoreti- cal framework. The evidence in multinomial- opinion in EDL exclusively supports singleton sets, which contains only category. In other words, EDL only captures the evidence which supports single category and rejects others. As a result, EDL is unable to effectively leverage vague evidence, such as features supporting composite set containing multiple categories. As Figure [sooss, EDL suffers from performance degradation in the face of ambiguous samples.
Uncertiity zli
class
class 21
Traditional EIN.
Hyper-opinion EDI
0.6
Hort
In addition, the parameters of colle-connected layer in EDL models are facing vanishing gra- dient problem when number of category in datasets rises [49]. Vanishing gradient in EDL. leads to failure in classification of several cat- egories. To mitigate this problem, Pandey al. [49 introduce regularization techniques. However, these efforts yield unsatisfactory re- sults in real world OOD detection tasks.
bellef
bellef
Belief and uncertainty masses across varying levels of In-distribution sample vagueness. As sample gets vaguer, EDL tends to extract minimal quantity of sharp evidence, results in ele- vated uncertainty estimation. HEDL demonstrates the capability to extract vague evidence as sample vagueness increases, thereby maintaining lower un- certainty levels.
Figure 1:
To train an evidential model maintaining clas- sification accuracy and providing reliable certainty estimation for OOD detection, we incorporate EDL. with hyper-opinion and pro- pose Hyper-opinion Evidential Deep Learning (HEDL). While EDL is built upon multinomial- opinion in a basic domain, hyper-opinion represents the opinion in the hyper-domain, which includes the basic domain and the composite sets. Through the concepts of composite set, HEDL is able to learn from vague evidence ignored by EDL. HEDL provides an effective mechanism for quantify- ing evidence that supports composite sets, thereby enhancing the differentiation of OOD data and classification accuracy. Our major contributions can be summarized as follows:
We introduce evidential representation within the hyper-domain, which integrates sharp evidence that supports a singular category, with vague evidence that accommodates multiple potential categories, establish a more comprehensive and accurate evidentiary foundation.
We develop hyper-opinion framework within the hyper-domain and propose a novel opinion projection. This method transfers hyper-opinion to multinomial-opinoon, allocating evidence to each category precisely and mitigating the vanishing gradient problem, while preserving computational efficiency.
2

---

Data
hoppr
mralicorani opteice
Ivilngs
(g Hyperopleise Seitef
Do Uyerdon Projedics
Melrrorttseppiee Optisiceisn
Od0 the
JVC.all I... FTR
projeced bve bypeoopace
(53
200)
Figure 2: Framework of HEDL.. HEDL framework is composed of three integral components. The first part transfers the extracted features to evidence and models them with in hyper-opinion framework. Subsequently, the second component projects the hyper-opinion to multinomial-opine the framework optimizes the opinion to attain precise classification and to furnish robust uncertainty estimations for OOD detection.
Ultimately,
Our proposed Hyper-opinion Evidential Deep Learning (HEDL) procures more exhaustive evidence, which refines the precision of uncertainty estimation, and consequently enhances the performance of OOD detection while maintaing ID classification accuracy.
We carry out experiments over multiple challenging datasets validate the OOD detection in HEDL outperforms existing OOD detection methods.
2
Related Work
Uncertainty based OOD Detection
Accurately quantifying predictive uncertainty DL models is crucial for recognizing out-of- distribution (OOD) samples. Traditional softmax-based models provide confidence estimation through class posteriors, which are inversely correlated with predictive uncertainty[IG] Several methods applicable to pre-trained classifiers that output class posteriors using softmax have been p 14 53
Detector for Neural Networks [31] and Mahalanobis Distance 30 Besides, deep ensemble is technique developed for uncertainty quantifi- cation (2), which constructs an ensemble of neural networks and measures uncertainty based on the agrremenn/disaareeemnnt across the ensemble components [13]. However, this approach significantly increases the scale of model parameters, leading to high computational and storage complexity. Alter- natively, neural networks based Bayesian statisties called Bayesian neural networks [12 4 42 is raised to to quantify different uncertainties in Bayesian formalism. Bayesian methods normally apply approximation address the intractability issue in marginalization of latent variables. And such methods require sampling for uncertainty quantification, leading to expensive computations. A recent research effort has summarized OOD detection methods and established OOD benchmark [69]
37 60 18), including Out-of-Distributie
2.2 Evidential Deep Learning
EDL introduces conjugate higher-order evidential prior for the likelihood distribution that enables the model to capture the evidence vacuity as predictive uncertainty. The training of an EDI. model can be regarded as an evidence-collecting process. Researches on multiple applications with EDL have been done, c.g., Dirichlet prior is introduced over the multinomial likelihood for evidential classification 2 73 I evidential models for regression[]] 48), adversarial robustness 27 and calibration 63 Most existing methods built upon EDL are trained on evidential losses conjunct with regularization of the evidence to guide the evidence vacuity, i.e., uncertainty. behavior 447 [56]. Some EDL models combine with the idea of outlier exposure [7 that provides access of OOD data guide the evidence learning process EDL models[40) 41
In this work, we focus evidential models for classification and OOD detection, and consider settings where no extra regularization and OOD data are used during model training to make the proposed approach more broadly applicable to practical real-world situations.

---

Proposed Method
Our method's framework is depicted in Figure 2 which operates under the assumption of no prior information.
3.1 Hyper-opinion Belief
Subjective Logic (SL) is a theory of uncertain reasoning based on probability theory and belief theory in domain X. which represents the set of exclusive possible states of variable situation. introduces the concepts of belief mass and uncertainty mass to describe the degree of belief and uncertainty about event.
Traditional EDL is built upon multinomial-opinion within domain X in SL and domain a limited portion of hyper-domain R(X), where (Xx) is the powerset of X.
R(X)= = (X)) [x),, (0))-
(I)
Let us consider a domain with cardinality of SL provides a belief mass by representing the belief degree and base rate ag representing the prior information for each singleton & = 1,.., and an overall uncertainty mass of u. The three compose multinomial-opinion w (b, u, a). belief' mass and uncertainty mass sum up to one, eg.,
K
Σ by 1
u u20 and by 2 for k = 1= K.
(2)
k=1
Our method models the evidence in hyper-domain R(X) with hyper-opinion, which provides belief mass b"' RX),, representing the belief degree of set z. Along with a and the three compose hyper-opinion w = (b*, a and the hypernomial belief mass distribution also follows the additivity requirement:
6" R(X) + |0,
Σ
6" =1
(ER(X)
Hyper opinion allows belief mass to be divided into two types called sharp belief mass and vague belief mass. Belief mass that only supports a specific singleton is called sharp belief mass, eg., k X. it discriminates between this and other singletons. EDL built upon the cetiinomialloppnnii only offers sharp belief mass estimation. Considers domain X of K mutually exclusive singletons, for each singleton & = 1,..., K, sharp belief mass is
by
Ak E X.
Belief mass assigned a composite set x E C(X). where C(X) R(X)XX, represents vague belief mass because it expresses cognitive vagueness. It supports the truth of multiple singletons in X simultaneously. Vague belief mass can allocated singleton as
Σ
a(k)))b
a(k/z) = Eiera
ak
Vk X E
(5)
PEC(X)
where a(k|z) is relative base rate. When prior information is available, a(k)z) can be simplified to x=|
a(k]>)
Vk E X, Vr C(X),
(6)
where |2| is the cardinality of r. Then in hyper-opinion a belief mass bdd for set is computed using the evidence for the set. Let " 2 be the evidence derived for the set æ, then the belief b" and the uncertainty are computed as
KW prior and u S
S
Σ DER(X)
e + KW prior.
By introducing hyper-opinion, vague beliefs that assigned to composite sets can be take into con- sideration, which better measure comprehensive evidence and estimate uncertainty more accurately.
In practice, we activate the features extracted by the neural network evidence hyper-domain, and build them within hyper-opinion to distinguish sharp belief and vague belief. This allows the model to maintain vagueness among similar in-distribution categories, thereby ensuring that the uncertainty remains low.

---

3.2 Opinion Projection
projection from hyper-opinion multinomial-opinion needed to realize the projected probability of each singleton. Therefore we introduce novel opinion projection implementation that projects belief mass from hyper-opinion into multinomial-opine with by and by that can be calculated by Eq. 4 and Eq. [] following
by = byg
b vk X
(8)
We activate the features extracted by neural network for ascertaining non-negative evidence within the hyper-domain. After associate evidence with belief in hyper-opinion, we determine the set each belief mass supports as mentioned in section 3.1 and project the belief mass from hyper-opinion to multinomialopinion.
Specifically, we apply unit step activation function to the parameters of the fully connected layer, eg., Heaviside function
1
(x) = 0, elsc.
It offers access to matrix W = HW),,, where W corresponds to the weight matrix of the fully connected layer. WS represents the information set each belief mass supports.
Assume there are K singletons and N belief masses supporting different sets. it offers a matrix W K For belief mass 6 supporting set x, W is a vector that contains information about which singlctons belong to the set x.
Once the each belief mass supports has been identified, projecting hyper-opinion to multinomial- opinion is straightforward. For each belief mass within the hyper-opinion, we can compute its relative base rate to each singleton, and allocate belief mass accordingly. For a singleton k, its total projected multiommialoppiionn belief mass is
by
Σ ZER(X
(b W
(10)
HW
ag W. 2,k
WP
A X, R(X),
(11)
Σ
(a,
Σ (a. WS)
where a is the base rate. Without any prior information, Eq.
II can be simplied to
WS a.k.
WP
k x
(12)
WS 23
To date, have successfully delineated the process of projecting belief mass from hyper-opinion to a mulinomial-opinion within a neural network framework. In practical terms, this projection executed by applying a linear transformation to the output of the fully connected layer. This transformation facilitates the allocation of belief mass to the respective singletons in the multinomial- opinion. Consequently, the incremental computational complexity associated with our method is constant as O(1).
WPB
b-o D.W.G.. G(W,
(13)
W
where o is the output of fully-connected layer and W, WP, 5 are all detached variables, making G(W, b² constant during one training epoch.
The output after opinion projection represents the projected multinomial-opinion in EDL, which has the equivalent meaning in EDL and can be optimized with the same techniques. We used an example to show why the uncertainty estimation of HEDL outperforms EDL in Appendix A
3.3 Multinomial-pinionn Optimization
By building evidence within hyper-domain and projecting hyper-opinion belief mass into multinomial- opinion belief mass, we construct flow that can be optimized in multinomial-opinoon framework to obtain the comprehensive evidence and accurate uncertainty estimation for OOD detection, which similar to traditional EDL.
5

---

As the sum of evidence ExER(X) e² and uncertainty remain the same during the projection, we can pass the belief mass in the form of evidence to simplify the calculation. Therefore the projected probability distribution derived from the projected multinomial-opinion can correspond an expected probability distribution derived from a Dirichlet distribution parameterized by a
(b, u, a) ++ Dér (P a
(14)
0g = Ex + ag W prior = S + 04 W prior
The Dirichlet distribution a probability density function (pdf) for possible values of the probability mass function (pmf) P and is given by:
K
Dir (P (P))=
B(x)
P
(15)
In projected multinomial-opinion the expected probability for the hthh singleton calculation is
(16)
5
which allows to be optimized by the loss function defined in EDL
(((0) = / Σ 5=1
K
K dp. Σ yis b(a))) j=1
yos log(piz)
II 5=1
au P.j
(17)
B(as)
where (-) is the digamma function, is one-hot vector encoding the ground-truth class of observation 24 with yky and yık for all + j. and a; be the parameters of the Dirichlet density on the predictors.
At this point, we have established the complete framework of HEDL. spanning all stages ranging from input processing to classification and uncertainty estimation. Our method objective has the following proposition the Appendix B
By establishing the framework of HEDL, we comprehensively extract the sharp and vague evidence each sample contains and allocate preciously, thereby enabling accurate classification. Moreover, comprehensive evidence contributes to improved uncertainty estimation and subsequently enhances the performance of OOD detection.
4 Experiment
In this section, we describe our experimental setup and demonstrate the effectiveness of our method on wide range of OOD evaluation benchmarks and the most widely used metric AUROC is adopted (1 10 37). of our approach.
We also conduct an ablation analysis that leads to an improved understanding
4.1 Setup
In-distribution Datasets. We use the CIFAR- 28 200-201 656 ID data.
CIFAR- 100 28), Flower-102 46 and CUB-
Datasets. For the OOD test datasets, we use three common benchmarks SVHN 44 Textures (6), Places365[74], that are used in Openood-benchmark 69 There is no overlapping between ID datasets and OOD datasets.
Evaluation Metries. We measure the following metrics: 1) FPR95 measures the false positive rate (FPR) when the true positive rate (TPR) is equal 95%. Lower scores indicate better performance, 2) AUROC measures the area under the Receiver Operating Characteristic (ROC) curve, which displays the relationship between TPR and FPR. The area under the ROC curve can be interpreted as the probability that a positive ID example will have a higher detection score than negative OOD example.
3) AUPR measures the area under the Precision- rreialln-Recall (PR) curve. The PR curve is created by plotting precision versus recall. AUROC is the most common metric 52 21
10 37 and we use AUROC as the main metric for OOD detection performance while accuracy measures performance of detecting ID samples. Our goal is to detect more OOD samples while maintaining ID classification performance.

---

Table I: Comparison of OOD detection performance between HEDL and other baselines with CIFAR- 10 and CIFAR-100 as ID dataset. All values are percentages. t indicates larger values are better, and + indicates smaller values are better. The bold are superior results.
OOD Dasasets
ID data
SVHN AUPRT
Testures AUPKT
Place365
Average AUPRT
FPR95
AUROCT
SPR95L
AUROCT
FPX95
AUPRT
AUROCT
PPR95L
AUROUT
Acc.f
CIFAR-10
MSP
51.87
78.
90.5
91.25
72
57.64
70.24
89,03
56.47
79.90
89:
95.
ODIN 31
67.92
42.13
73. 13.53
51 10
82:
50.70
50.51
50.27
82.55
56.51
58.22
78.86
95.06
upen 26
99.39
33.90
$3.56
98.21
61.48
42.22
99.64
19.55
36.55
99.02
38.31
44.12
95.06
GradNurm
78.89
53.91
VIM
14.41
93.76
97.23
20.78
97.
06
47.52 .2.5
72.83
90.08
27.57
87.98
95.06
KNN66
33.32
92.31
95.13
46.01
95.93
92.77
43.78
$0.15
91.82
41.04
89.47
93.23
95,06
DICE 1X
67.78
73.19
66.43
67 48
BS
14
56.06
57.52
84.43
63.78
72.03
83,66
95.06
Rani
68.15
59.71
94.66
55.97
75.87
95.06
AS
83.54
89.06
73:46
$4.59
72.85
77.45
77.89
91.04
79,89
82.04
85.32
76.93
95.06
S11 XAl
62.74
94.46
86. 58
84 60
77.28
$1.57
76.36
94.88
82.89
74.57
88.87
83.61
95.06
GEN 53
28.14
96.37
91.90
40.74
84.71
90.14
47.03
96.67
38.64
92.58
90..2
95.06
MCDrspon [P)
92.67
91.74
52.47
$9.98
G-ODIN [9
8.42
96.65 06.63
98.4
23.32
96 01
94.51
39.80 80
73.49
91,10
23.84
89.39
94.67
94.70
6S
17.56
97.75
95.15
28.95
82.99
90.71
34.76
96.38
89.56
27.09
92.57
91.82
91.15
MOS PD
90.85
70.55
$1.09
85. 56
90.89
52.91
71.74
78.67
74.15
K2.71
80,03
59.38
94.83
29.91
83.73
93.82
57.38
92:72
91.26
45.57
80.13
91.27
95.82
LogitNorm(S)
5.30
97.70
98.90
30.94
96.32
94.30
31.17
$8.11
94.76
22.47
98.04
95.97
94.30
EDL [1
11,56
88.60
93.92
19:93
99.07
95. 70
19.36
93.15
96.54
16.96
93.61
95.39
95.72
RED 40
65.75
29.85
86.49
71.54
46.84
95.80
HEDI,(Ous)
8.43
94.09
96 96
19.15
99.19
96.23
19.08
90.14
95.71
15.55
94.47
96.27
95.66
CIFAR-100
MSP 10
83.69
60.76
76.04
83.83
85. 24.24
76.93
81.24
62.39
79.04
82.91
69.46
77.47
27.25
ODIN 31
78.37
86.67
76.77
openGAN 26
83.96
60.85
78.68
86-31
80.
73.53
39,87
70:15
86-21
59.96
74.
27.25
GrdNNorm
69.90
89.45
76.95
92.51
56:77
64.38
55.32
39.78
69,09
85.91
78.33
70.41
77.25
VIM [6
82.79
72.82
81 30
55.90
92.15
S7.41
83.85
56.24
75.76
74.18
73.74
81. 16.46
77.25
KNN
82.21
66.40
81.71
DICE
79.90
90.33
77.70
80.75
62.76
80.15
80,40
71.37
79.28
21.25
RanicPoat
58
58.49
83.40
72. 14
66.87
52.43
(9. 40
77.42
83.74
63.82
67.59
75.19
65.45
77.25
ASH
46,00
92.97
85.00
61.27
68:97
50.72
62.95
91.48
78.76
56.74
84.47
81.09
77.25
SHE 01
90.85
80.99
73.29
60.57
76.99
77.25
78.
91.90
80.25
82.26
50.1
27.25
MCDoop 12
71.63
67.44
81.31
S0.16
86.01
71.93
79.52
61.34
79.20
71.31
71.00
79.44
75.85
G-ODIN [9
71.62
79.80
86.13
58.01
93.01
K5..35
78.67
$5.45
75.15
69 44
76,09
84.21
74.46
CSI [0
80.24
MDS
90.58
74.48
59.42
96.32
92.64
71.87
93.18
78.64
55.09
76.98
VOS
98.62
56.36
68.90 99
94-54
76.22 20
68.33
97.81
43.20
68.21
96.09
$8.59
68.51
77.20
Log Nooon x7
79.16
75.57
83.03
87.06
79.08
71.53
80.20
63.10
79.84
82.14
72.58
13
76.34
EDL 54
93.05
76.51
71.40
RED
90.00
62.75
76.41
56.01
96
11
64.75
71.40
74.58
82.05
80.36
HEDL/Oas))
39.56
89.22
93.46
61.97
96.85
85,98
63.89
81.14
89,32
55.14
89.07
89.59
80.40
Implementation Details. We follow the experiment settings outlined in OpenOOD66]] 69 We use ResNet- 1 13 for CIFAR-10 and 100. For more intricate datasets that are not included in OpenOOD 69. such as fine- -in-igri datasets Flower-102 and CUB-200-2011 1, we employ ResNet- 34(15) for enhanced feature representation. All experiments are implemented with PyTorch 50) and carried out with NVIDIA GeForce RTX 3090 GPU. We use the standard data split for all datasets, and the number of training epochs is 100, the initial learning rate is 0.0001 with Adam 39, and the batch size is 128. At test time, all images are resized to 224 x 224. For HEDL model, we first train the feature extractor with softmax layer for 90 epochs and then train in HEDL framework for 10 epochs. HEDL does not introduce any additional hyperparameters, thereby climinating the need for extensive hyperparameter tuning, and W prrior is set to I for HEDL.
We compare our method with several classical and tfte-of-the-aa OOD detection methods. Specifically, we compare our method with post-hoc inference methods and training methods. From MSP(16) to GEN are post-hoe inference methods, which affect OOD detection performance only and do not change model accuracy. The others are training methods. We excluded methods that required auxiliary OOD data due to the practical real-world situations consideration. We leverage selected experimental results from OpenOOD 69 to demonstrate the efféctiveness of our approach.
Baseline Methods.
7

---

Table 2: Ablation experiment results Flower- 102 and CUB-200-2011. Results show that EDL fails to extract evidence fully. HEDL without projection can extract comprehensive evidence to distinguish ID and OOD samples but fails to classify ID categories. HEDL can further assign evidence correctly and obtain accurate classification.
Flower 102
CUB 200-2011
Average ODD performance
IDD data
Average OOD perfomaance
ID data
Multinomial-og opinion Hyper opinion
Opinion projection
[PRS5]
AUPRT
AUROCT
Acc.t
[PR95]
AUPR?
AUROCY
Ace:f
14.86
95.94
97.42
83.75
30.29
91.18
94.35
75,82
100,00
66.95
67.23
66.84
98.03
71.80
75.27
59.87
97.61
V
3,98
98.73
99.07
388
98.91
74.62
4.2 OOD Detection Results
The comparative results on CIFAR-I0 and CIFAR-100 are detailed in Table D and the results on Flower-102 and CUB-200-2011 are shown in Appendix [ For each model, we utilize three OOD datasets, thereby aiming to achieve more realistic and generalized outcomes. We reveal a common challenge: when confronted with more complex data scenarios, training methods struggle maintain both accuracy and OOD detection capabilities simultancously. However, HEDL consistently achieves better OOD detection performance than existing state-of-the-art OOD detection methods while preserving the accuracy of ID classification, even under complex data scenarios. Notably, HEDL accomplishes this enhancement without additional regularization strategies hyperparameters. indicating strong generalization ability on different datasets, it also avoids incurring higher computational costs. The experimental training time analysis of HEDL can be found in Appendix
4.3 Gradient Analysis
The gradient norms of fully-connected layer pa- rameters over EDL and HEDL during training is shown in Figure 3 alongside the final accu- racy for each category. The sum of these gra- dient norms has been normalized for compar- ative analysis.
observed that the gradient norms for several parameters within the fully- connected layer of the EDL model remain zero throughout the training process, which corre- lates with a significantly lower final accuracy for these categories. This outcome is indicative of the vamishing gradient problem. Conversely, HEDL does not experience this issue, demon- strating that our proposed method effectively circumvents the challenge of vanishing gradi- ents within the fully-connected layer.
Gradant
Figure 3: The sum of gradient norms within the fully-connected layer for each category CIFAR- 100 throughout the training process.
4.4 Ablation Study
EDL suffers from notable decline in both ID accuracy and OOD detection when facing proportional rise in the volume of vague evidence. In contrast, HEDL demonstrates the capability to consistently extract comprehensive evidence and maintain its performance regardless of the dataset scale.
We investigate the performance of our method with ablation experiments on two challenging fine- grained datasets. The fine-grained datasets contain more vagueness among categories and can better prove the effectiveness our methods. We conducte ablation experiments on the effects of hyper- opinion and opinion projection, respectively. Note that opinion projection can only be built upon hpper-opinion.
Figure 4 illustratess the uncertainty distribution of ID and OOD samples across different datasets for EDL., HEDL. without opinion projection, and HEDL itself. Notably, on the latter three more complex datasets, the approaches based on hyper-opinion exhibits distinct performance advantage. It is also

---

FD
HEDI win projection
HFDI
In Badio Oui-t
D-D butoon Ju.-d:Orontsilon
Uineertal
Uncerra Fty
Uncersalty Tyy
(a) CIFAR-10, 10, the overlap between ID and OOD is 20%, 23%, and 18% for EDL, HEDL w/o projection, and HEDL, respectively.
HEDL projection
HEDL
W.Dietrauat Dut doirationn
Distr button Dy www.ooiitttooo
Det teation Decresctre
Dannday
Up
Untertar fy
(b) CIFAR-100, the overlap between ID and OOD 62%, 45% and 41% EDL, HEDL w/o projection, and HEDL,, respectively.
projection
In Butto Out-ot O dtiication
In buboo Ou Duaiit-ntttttt
Ora Ibstion
12-
Dut Chttriiit
3.8
Unoomta
Unoar
Unoertal
(c) Flower-102, the overlap between and OOD 71%, 26% and 29% for EDL, HEDL w/o projection, and HEDL, respectively.
projection
Batmauss Out of Oudrication
Dietibatton Our Dxconiation
ttatton Out of Detritution
Dasetry
14
(d) 200-200-2011, the overlap between ID and OOD 50%. 20% and 17% for EDL, HEDL w/o projection, and HEDL, respectively.
Figure 4: The normalized density distribution of normalized uncertainty for ID and OOD samples across differing datasets.
worth observing that, in these datasets, instances of ID data with maximum uncertainty are present in the EDL model. This phenomenon can be attributed to the failure of extracting evidence of those categories due to the vanishing gradient problem.
2 shows that evidence built on hyper-opinion can be considered comprehensively, leading to accurate uncertainty estimation and above baseline OOD detection performance. But without the correct projection from hyper-opinion to multinomiallopinion, vague evidence can not be assigned precisely, leading to inaccurate classification.
Conclusion
In this paper, propose Hyper-opinion Evidential Deep Learning (HEDL), novel approach designed to generate precise uncertainty estimation for caatof-Distributtinn (OOD) detection. Our method encapsulates comprehensive representation of evidence within hyper-opinion, which allows model to preserve its vagueness among In- In-Distribution categories to reject OOD data.
Additionally, by projecting hyper-opinion to multinomial-opinion HEDL circumvents the vanishing gradient problem encountered in the fully-connected layers of traditional EDL. This projection is opti- mized within an established framework, yielding accurate and reliable evidence. Notably, our method

---

accomplishes superior OOD detection performance while simultaneously upholding classification ac- curacy without incurring additional computational complexity. Extensive experimental results across numerous datasets substantiate the efficacy of the proposed Hyper-opinion Evidential Deep Learning.
Limitations and societal impact. Our proposed HEDL method achieves best performance by transfering learning on pre-trained models. In future work, it is necessary to reduce the dependence on pre-trained models and explore alternative approaches. This work aims improve the safety of deep learning models, which tends to benefit wide range of applications of AI in social life.
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (No. 62173252, 62476165).
References
[1] Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression. Advances in Neural Information Processing Systems, 33: 33:14927-14997 2020.
[2] Wentao Bao, Qi Yu, and Yu Kong. Evidential deep learning for open set action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13349- 13358, 2021.
[3] Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1563-1572, 2016.
[4] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613-1622, PMLR, 2015.
[5] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Informative outlier matters: Robustifying out-of-distributie detection using outlier mining. 2020.
[6] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and paftern recognition, pages 3606-3613, 2014.
[7] Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ann Heng. Uncertainty esti- mation by fisher information-based evidential deep learning. arXiv preprint 2xXi::3303000455 2023.
[8] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple ac- tivation shaping for distr--distribution detection. In International Conference on Learning Representations, 2022.
[9] Xuefeng Du, Zhaoning Wang. Mu Cai, and Yixuan Li. Vos: Learning what you don't 't know by virtual outlier synthesis. In International Conference on Learning Representations, 2021.
[10] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of of -isttriuuttnnn detection. Advances in Neural Information Processing Systems, 34:7068-70s: 177068 2021.
[11] Wei Fu, Yufei Chen, Wei Liu, Xiaodong Yue, and Chao Ma. Evidence reconciled neural network for
detection in medical images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 305-315. Springer, 2023.
[12] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050-1059. PMLR, 2016.
[13] Mudasir Ganaie, Minghui Hu, AK Malik, M Tanveer, and PN Suganthan. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022.
10

---

[14] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International conference on machine learning, pages 1321-1330. PMLR, 2017.
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[16] Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-o--distribution examples in neural networks. In International Conference on Learning Representations, 2016.
[17] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, 2018.
[18] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling -uitt--disttibuttion detection for real- world settings. In International Conference Machine Learning, pages 8759-8773 PMLR, 2022.
[19] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out- data. In Proceedings of the
of-distribution image without learning from IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 10951- 2020.
[20] Rui Huang and Yixuan Li. Mos: Towards scaling disti--distiiuttion detection for large semantic space. 2021 icee. In CVF Conference Computer Vision and Pattern Recognition (CVPR) pp. pages 7706-8715,2200 2021.
[21] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34: 677-689, 2021.
[22] Bingbing Jiang, Chenglong Zhang, Yan Zhong, Yi Liu, Yingwei Zhang, Xingyu Wu, and Weiguo Sheng. Adaptive collaborative fusion for multi-view semi-supervised classification. Information Fusion, 96:37-50, 2023.
[23] Bingbing Jiang, Xingyu Wu, Xiren Zhou, Anthony G Cohn, Yi Liu, Weiguo Sheng, and Huanhuan Chen. Semi-supervised multi-view feature selection with adaptive graph learning. IEEE Transactions on Neural Networks and Learning Systems, 35(3):3115-362,, 2024.
[24] Audun Jøsang. Subjective logic, volume Springer, 2016.
[25] Uday Kamath, John Liu, and James Whitaker. Deep learning for NLP and speech recognition, volume 84. Springer, 2019.
[26] Shu Kong and Deva Ramanan. Opengan: Open-set recognition via open data generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 813-822, 2021.
[27] Anna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, and Stephan Günnemann. Evaluating robustness of predictive uncertainty estimation: Are dirichlet-hased models reliable? In International Conference on Machine Learning, pages 5707-5718. PMLR, 2021.
[28] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[30] Kimin Lec, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribttio samples and adversarial attacks. Advances in neural information processing systems, 31, 2018
[31] Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of dut-o---ist image detection in neural networks. In International Conference Learning Representations, 2018.
11

---

[32] Xinyan Liang, Yuhua Qian, Qian Guo, Honghong Cheng, and Jiye Liang. Af: An association- based fusion method for modll classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44( 44(12):9236 9254, 2021.
[33] Xinyan Liang, Pinhan Fu, Qian Guo, Keyin Zheng, and Yuhua Qian. Dc-nas: Divide-anc conquer neural architecture search for multi-modie classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13754-13762, 2024.
[34] Wei Liu, Xiaodong Yue, Yufei Chen, and Thierry Denoeux. Trusted multi-view deep learning with opinion aggregation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7593, 2022.
[35] Wei Liu, Yufei Chen, Xiaodong Yue, Changqing Zhang, and Shaorong Xie. Safe multi-view deep classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8870-8878, 2023.
[36] Wei Liu, Yufei Chen, and Xiaodong Yue. Building trust in decision with conformalized multi-view deep classification. In ACM Multimedia 2024, 2024.
[37] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. thergy-based out- out-of- detection. Advances in neural information processing systems, 13:21464-21444 2020.
[38] Xixi Liu, Yaroslava Lochman, and Christopher Zach. Gen: Pushing the limits of softmax-bant based out-of-distribution detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11946-23955 2023.
[39] Nya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference Learning Representations, 2018.
[40] Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018
[41] Andrey Malinin and Mark Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
[42] Aryan Mobiny, Pengyu Yuan, Supratik K Moulik, Naveen Garg, Carol C Wu, and Hien Van Nguyen. Dropconnect is effective in modeling uncertainty of bayesian deep networks. Scientific reports, 11(1:5558,, 2021.
[43] Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable out-of-distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages
2020.
[44] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[45] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427-436, 2015.
[46] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision. graphics & image processing, pages 722-729. IEFF, 2008
[47] Deep Shankar Pandey and Qi Yu. Multidimensional belief quantification for label-efficient meta-learning In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 44390- 2022.
[48] Deep Shankar Pandey and Qi Yu. Evidential conditional neural processes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9389-9397, 2023.
[49] Deep Shankar Pandey and Qi Yu. Learn to accumulate evidence from all training samples: theory and practice. In International Conference on Machine Learning, pages 26963-26989. PMLR, 2023.
12

---

[50] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[51] Tim Pearce, Felix Leibfried, and Alexandra Brintrup. Uncertainty in neural networks: Approxi- mately bayesian ensembling. In International conference on artificial intelligence and statistics, pages 234-2 244..44. PMLR, 2020.
[52] Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out- of-issiiiiitttionn detection. Advances in neural information processing systems, 32, 2019.
[53] Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. In International Conference on Machine Learning, pages 8491-8501. PMLR, 2020.
[54] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31, 2018.
[55] Glenn Shafer. A mathematical theory of evidence, volume 42. Princeton university press, 1976.
[56] Weishi Shi, Xujiang Zhao, Feng Chen, and Qi Yu. Multifaceted uncertainty estimation for label- efficient deep learning. Advances in neural information processing systems, 33:17247-17257, 2020.
[57] Shashi Pal Singh, Ajai Kumar, Hemant Darbari, Lenali Singh, Anshika Rastogi, and Shikha Jain. Machine translation using deep learning: An overview. In 2017 international conference on computer, communications and electronics (comptelix), pages 162-167. IEEE, 2017.
[58] Yue Song, Nicu Sebe, and Wei Wang. Rankfeat: Rank- feature removal for -ustrf-dist detection. Advances in Neural Information Processing Systems, 15:17885-17888 2022.
[59] Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for -uittf-distributton detection. In European Conference on Computer Vision, pages 691-708. Springer, 2022.
[60] Yiyou Sun, Chuan Guo, and Yixuan Li. React: nat-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144-151 2021.
[61] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribttion detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827-20840. PMLR, 2022.
[62] Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33: 33:11833-11 1111-18111880 11522, 2020.
[63] Christian Tomani and Florian Buettner. Towards trustworthy predictions from deep neural networks with fast adversarial calibration. I Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 9886-9896, 2021.
[64] Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, Eftychios Protopapadakis, et al. Deep learning for computer vision: A brief review. Computational intelligence and neuroscience, 2018, 2018
[65] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2 dataset. 2011.
[66] Haoqi Wang, Zhizhong Li. Litong Feng, and Wayne Zhang. Vim: virtual-logit matching. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921-4930, 2022.
with
[67] Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In International Conference on Machine Learning, pages 23631-23644 1-24444. PMLR, 2022.
13

---

[68] Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, and Xiyue Gao. Reliable conflictive multi- view learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16129-16137, 2024.
[69] Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang. Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of- distribution detection. Advances in Neural Information Processing Systems, 35:32598- 32611,, 2022.
[70] Nanyang Ye, Kaican Li, Haoyue Bai, Runpeng Yu, Lanqing Hong, Fengwei Zhou, Zhenguo Li, and Jun Zhu. Ood-bencis Quantifying and understanding two dimensions of -iitr-idittiittioon generalization. In Proceedings of the IEEF/CVE Conference on Computer Vision and Patlern Recognition, pages 7947-7958, 2022.
[71] Jinsong Zhang. Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, Shi Han, Dongmei Zhang, et al. cut-of-distribution detection based on in-distribution data patterns memorization with modern hopfield energy. In The Eleventh International Conference on Learning Representations, 2022.
[72] Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Di Hu, et al. Multimodal fusion on low-quality data: A comprehensive survey. arXiv preprint arXiv:2404.1894 2024.
[73] Xujiang Zhao, Feng Chen, Shu Hu, and Jin-Hee Cho. Uncertainty aware semi-supervised learning on graph data. Advances in Neural Information Processing Systems, 13:12227-12836, 2020.
[74] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452- 1464, 2017.
14

---

A An Example within EDL and HEDL
(rewet contle Arngeras.pponn
hypar-spttiin
maliiomiaal
ournect & loverr LEABI14AA
Tome 1-
tava catte
0115
Operien Prepettes
-0077
Senarme 0115
7 pas 1- 1096
0.154
- (308655-359
0284
snceratily
My = (01:
evidence
8.345
T Feuats
Faily Conaanted
(-31,- -3 10122
0.11
0.605X
B.12
Seature
evilence
lyper opiaiin
multiionial epinion
Figure 5: Example an image been classified by EDL and HEDL. When confronted with vague samples, HEDL leverages the incorporation of vague evidence, which culminates in enhanced accuracy for classification and more precise uncertainty estimations, thereby fortifying OOD detection capabilities.
sample displaying the classification process of EDL and HEDL shown in Figure 5 EDL tends to ignore diminish the amount of vague evidence to get sharper belief mass. The loss of evidence leads to increased uncertainty and potential misclassification. the vague evidence, thereby achieving improved estimations of uncertainty and more accurate classification results. Notice that when there is no evidence supporting a set æ, then ex = 0.
contrast, HEDL framework reserves
B Gradient Vanishing Analysis
Proposition By building evidence on hyper-opinion and then projecting muliinomial-opinton, we avoid the vanishing gradient problem in fully- layer in traditional EDL.
Proof 1. Consider the neural network forward propagation in EDL
02 = Wz + bias,
(18)
ek (eI,(Oo); Wprion
(19)
0g ex
(20)
K
K
())
Σ Mi (v(i)) - vaa,)) Ø=1
(21)
where bias stands for the bias of the fully-connected layer, = represents the feature extracted by the neural network. We can write expressions for all partial derivatives follows:
Ook OW
Oak dek
= 2,
1,
(22)
OC
y/
Uk
(23)
Dox
52
(i + S)²
2
agt)
gt
i=1
dek
0
Og
(24)
Dok
otherwise.
Therefore by the chain rule, we can calculatethe the gradient w.r.t. W as:
ac ac day de, doz AL dep z. aw
(25)
day deg Dog aw Day Ook
15

---

Obviously when exists oμ I 0, vk E X, vanishing gradient problem is unavoidable in traditional EDL. To ensure that proposed HEDL is not associate with similar problem, considering the forward propagation of HEDI.:
ok Wz bias,
(26)
cx - GW b
(27)
a ck W Wprior
(28)
where G(W, b²) can be calculated by Eq. and Eq. and the gradient w.r.t W is calculated by chain rule:
dc OC dag deg dox
(29)
aw dax de, dor OW
where 0c Oaz doz are known items that won't cause vanishing gradient problem. Consider Dag." de,," DW
deg do,G(W b*, = G(W,b"),
(30)
Dog dok
where W, b are all detached variables that are irrelevant variables in this partial derivative item, implying that G(W,b") remains constant during the backward process.
ac dL G(W, G(W,b")z.
(31)
aw daz
Consequently, the opinion projection successfully circumvents the vanishing gradient problem in the fully-connected layer.
C Experiment Results on Flower-102 and CUB-200-2011
Table details the comparative results fine- fineggraieed datasets Flower- 102 and CUB-200-2011 On more complex fine-grained datasets, HEDL consistently demonstrates superior performance in OOD detection.
Table 3: Comparison of OOD detection performance between HEDL and other baselines with Flower- 102 and CUB-200-22 ID dataset.
Flower- 102
CUB-200-2011
Method
Average OOD performance
ID data
Average OOD performance
ID data
FPR95
AUPRI
AUROCT
Acc.1
FPR95
AUPR
AUROCT
Acc.
MSP 16
14.86
95.94
97.42
83.75
30.29
91.18
94.35
75.82
ODIN 31
4.36
97.63
98.22
83.75
21.92
89.92
96.22
75.82
V1M
6.34
96.70
97.94
83.75
6.71
97.27
98.26
75.82
GradNorm
5.38
97.11
98.81
83.75
32.08
97.68
95.22
75.82
K1
18.45
88.83
95.30
83.75
14.35
88.63
97.40
75.82
DICE 59
4.64
97.62
98.95
83.75
25.82
88.83
96.00
75.82
RankFeat 58
96.57
76.62
60.98
83.75
74.68
83.38
71.09
75.82
ASH
5.16
97.54
98.84
83.75
15.82
92.75
97.07
75.82
SHE(71
11.69
93.96
97.79
83.75
22.94
96.14
96.18
75.82
GEN 38
5.25
97.55
98.85
83.75
15.88
92.74
97.06
75.82
MCDropout 22
14.77
96.22
97.41
83,98
42.46
87.08
91.76
75.83
G-ODIN 19
56.92
69.88
82.12
24.30
29.51
85.13
93.85
66.74
VOS 9
39.17
84.52
90.11
78.08
35.98
83.93
89.86
75.92
LogitNorm 67
41.07
80.34
85.65
77.41
22.69
91.69
95.99
74.84
54
100.00
66,95
67.23
66.84
98.03
71.80
75.27
59.87
RED 49
95.87
80.10
76.45
84.63
36.01
94.58
94.89
76.30
HEDL(Ours)
3.98
98.73
99.07
84.13
3.82
97.80
98.91
74.62
16

---

D Experiment Analysis of Computational Complexity
Table presents the average training time per epoch of EDL and HEDL compared with MSP on different datasets, all under identical training conditions. The results indicate that the implementation of HEDL does not incur additional computational complexity.
Table 4:
Average training time per epoch of EDL and HEDL compared with MSP on different datasets, + indicates more time, and indicates less time.
Method
Cifar10
Cifar100
Flower-102
CUB-200-2011 1
EDI
+3.78%
-1.93%
+1.21%
+2.14%
HEDI.
+1.62%
+1.02%
-0.74%
+3.57%
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction provide a comprchensive overview of the contributions, the scope, and the method of paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitation of the proposed method in the paper, namely that the best performance of our method depends on the performance of the pre-trained model we adapted.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
18

---

Answer: [Yes]
Justification:
We provide a complete and correct proof for the theoretical results the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: This paper fully discloses all the information needed to reproduce the main experimental results, including the theoretical and practical implementation, and training
details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code of this paper is included in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc.
Shbllac/guiless/oddeebbbm.ssss htt.:/ummmiiiiioooooooooo.. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ cc/ppi.cc guwdes/ddeeeee CooeeSumeeeeeeeeeeeeee ssi onyylicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: This paper specifies all the training and test details necessary to understand the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
This paper presents a series of replicable experiments conducted across multiple datasets, proving the statistical significance of the experiments of our proposed method.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: This paper provides sufficient information on the computer resources needed reproduce the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: btetps://neuripee http:///....ppp..itatccccceeeeee
Answer: [Yes]
Justification:
The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper discusses the positive societal impacts of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
21

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The creators of the data and models used in this paper are properly credited and are the license and terms use explicitly mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide the details of our code part of our submissions via structured templates, along with documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "qc6PxFk4fQ", "reviewer": "Reviewer_hxrE", "summary": "This paper provides a method for Out-of-Distribution detection called Hyper-opinion Evidential Deep Learning which is based on Evidential Deep Learning. It models the evidence on the hyper-domain, considering the extra vague evidence containing multiple possible categories. The measurement of vague evidence makes HEDL have more accurate uncertainty estimation. This paper also proposes an opinion projection method to mitigate the gradient vanishing problem in EDL. Experiments on several datasets show that the proposed method outperforms the current Out-of-Distribution detection methods.", "questions": "Please answer the questions in Weakness.", "limitations": "Please see Weakness", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(1)The paper has clear motivation for each part of the proposed method and the method has a solid theoretical foundation.\n\n(2)The proposed method achieves SOTA results compared with other methods.", "weaknesses": "1.Eqs. 13, 14, and 15 are hard to understand. The authors can detail on the meanings between the different W's.\n\n2.Why HEDL is trained on a pre-trained feature extractor and how to ensure the fairness of comparative evaluations with other methods?\n\n3.Theoretical proof of how HEDL solves vanishing gradient problem is missing. How is the EDL gradient vanishing point determined in Figure 3 and does this phenomenon consistently result in the vanishing gradients for certain fixed classes during each iteration?\n\n4.Adding experiments on model complexity comparison can be beneficial to illustrate the computation complexity of HEDL.\n\n5.More state-of-the-art methods (CIDER[1] and NECO[2]) can better demonstrate the effectiveness of proposed method. \n\n[1] https://openreview.net/forum?id=aEFaE0W5pAd\n\n[2] https://openreview.net/forum?id=9ROuKblmi7\n\n I’m mostly concerned about 2 and 3. If the authors can address these questions, I may consider raising my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer hxrE,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our paper and providing us with valuable feedback. Here are our replies.\n\n> Eqs. 13, 14, and 15 are hard to understand. The authors can detail on the meanings between the different W's.\n\n$W$ is a matrix of $N \\times K$ shape, representing the weight of the fully connected layer.\n\n$W^s$ is a 0/1 matrix obtained by applying the Heaviside function to each value in the $W$ matrix, representing the set of each piece of evidence supports on the hyper-domain.\n\n$W^p$ normalizes the value of each row of $W^s$ according to prior information, representing the opinion projection weight matrix.\n\n> Why HEDL is trained on a pre-trained feature extractor and how to ensure the fairness of comparative evaluations with other methods?\n\nBecause HEDL extracts both vague and sharp evidences, it converges slowly in the early stages of training. Therefore, applying the softmax layer to pre-train a feature extractor and then fine-tuning it with HEDL can achieve faster convergence and better model results.\n\nRefer to **Experiment Implementation Details**. For all comparative experiments, we apply the same model structure and initial training parameters, as well as the total number of training epochs, thus ensuring the fairness.\n\n> Theoretical proof of how HEDL solves vanishing gradient problem is missing. How is the EDL gradient vanishing point determined in Figure 3 and does this phenomenon consistently result in the vanishing gradients for certain fixed classes during each iteration?\n\nWe provided theoretical proof of how HEDL solves vanishing gradient problem in **Appendix B**. \n\nFigure 3 shows that in each training process, the parameters whose gradient norm sum is 0 are the gradient vanishes points. \nThe gradient vanishing classes in each training are not fixed, and will change according to the initial parameters.\nDue to lacking of extracting evidence training caused by gradient vanishing, those gradient vanishing classes suffer from low classification accuracy and high uncertainty.\n\n> Adding experiments on model complexity comparison can be beneficial to illustrate the computation complexity of HEDL.\n\nWe had provided the experimental data with training time for each model(softmax/EDL/HEDL) in **Appendix D**. Both theoretical proof and experimental results show that HEDL does not increase the computational complexity of the model.\n\n> More state-of-the-art methods (CIDER[1] and NECO[2]) can better demonstrate the effectiveness of proposed method.\n\nThanks for your suggestion. We planned to compare with CIDER in our experiments, since CIDER does not report the classification accuracy in its original paper, we did not put it the main text. As for NECO, we keep the same experimental settings and model parameters as in our paper to reproduce the experimental results. The results are shown in the table below. It proves that our method is better than the current SOTA methods. If our paper is accepted, we can add these experimental results to our paper if necessary.\n\n| Method | CIFAR-10  |           |           | CIFAR-100 |           |           |\n|:------ |:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n|        | FPR95     | AUPR      | AUROC     | FPR95     | AUPR      | AUROC     |\n| CIDER  | 21.94     | **95.16** | 95.33     | 58.21     | 87.99     | 84.61     |\n| NECO   | 31.28     | 92.34     | 94.68     | 64.92     | 86.36     | 83.74     |\n| HEDL   | **15.55** | 94.47     | **96.27** | **55.14** | **89.07** | **89.59** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R3n5O5Zm1W", "reviewer": "Reviewer_jaEe", "summary": "This paper introduces an out-of-distribution detection method based on evidential deep learning. This method models the evidence in hyper-domain, and the hyper-opinion in Subjective Logic is used to replace the multinomial-opinion in traditional evidential deep learning. The Hyper-opinion Evidential Deep Learning considers the vague evidence ignored in traditional Evidential Deep Learning, so as to achieve better uncertainty estimation effect. The OOD detection performance of proposed method exceeds the current SOTA OOD detection methods while maintaining the classification accuracy.", "questions": "1.\tIn Figure 4, the uncertainty distribution of In-Distribution and Out-of-Distribution data is similar between HEDL w/o projection and HEDL. This observation raises the question of the contribution of the opinion projection. Could the authors further explain the specific role and impact of the opinion projection in the whole method?\n2.\tCould the authors clarify the conceptual difference between a 'Dirichlet hyper distribution' and a standard 'Dirichlet distribution' in Equation 8? From my understanding, it seems that the two distributions equal. If there is a nuanced difference, please provide an explanation to delineate the unique characteristics of the 'Dirichlet hyper distribution' as employed in your method.", "limitations": "Yes, limitations and social impact are discussed in the main paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tHEDL achieves SOTA OOD detection performance, while maintaining classification accuracy.\n2.\tAs the number of categories in a dataset increases, the traditional EDL framework's performance significantly deteriorates. In contrast, HEDL can consistently extract comprehensive evidence and maintain its performance, regardless of the dataset's scale.\n3.\tHEDL does not introduce additional computational complexity of the model.\n4.\tHEDL can mitigate the vanishing gradient problem in EDL theoretically and practically.", "weaknesses": "1.\tThe origin of the sample uncertainty depicted in the upper portion of Figure 1 is not clearly defined. It is imperative to clarify whether the data presented comes from actual experimental outcomes or if it is merely illustrative example. If the data is based on real-world results, please provide an explanation of how to quantify the samples vagueness.\n2.\tThe paper does not specify the value of $W_{prior} $ nor does it analyze its role as a hyperparameter. It is essential to elucidate whether $W_{prior} $ is equivalent to EDL. Please provide a detailed numerical definition of $W_{prior} $.\n3.\tFigure 2 can be described more clearly. In the second module ‘Opinion Projection’ of the lower part, the positioning of the right bar appears to be more logically associated with the 'Multinomial-Opinion Optimization' section. To improve the figure's comprehensibility and accuracy, it is recommended to reposition the right bar accordingly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer jaEe,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our paper and providing us with valuable feedback. Here are our replies.\n\n> The origin of the sample uncertainty depicted in the upper portion of Figure 1 is not clearly defined. It is imperative to clarify whether the data presented comes from actual experimental outcomes or if it is merely illustrative example. If the data is based on real-world results, please provide an explanation of how to quantify the samples vagueness.\n\nThese data comes from the experimental results of CIFAR-100. We randomly select two categories (kangaroos and dinosaurs in Figure 1 for example), and calculate the vague evidence ratio of each sample in HEDL. Next we calculate the uncertainty of the samples in EDL and HEDL models respectively.\n\n> The paper does not specify the value of $W_{prior}$ nor does it analyze its role as a hyperparameter. It is essential to elucidate whether $W_{prior}$ is equivalent to EDL. Please provide a detailed numerical definition of $W_{prior}$.\n\nIn all our experiments, the value of $W_{prior}$ is set to 1, the same as in EDL, denoting no prior information of the evidence.\n\nThis clarification is mentioned in section 3.1, if the paper is accepted, we will add this definition to the experimental setup too.\n\n> Figure 2 can be described more clearly. In the second module ‘Opinion Projection’ of the lower part, the positioning of the right bar appears to be more logically associated with the 'Multinomial-Opinion Optimization' section. To improve the figure's comprehensibility and accuracy, it is recommended to reposition the right bar accordingly.\n\nThanks for your suggestion. The 'Multinomial-Opinion Optimization' section is about the optimization of the Dirichlet distribution associates with a multinomial-opinion. The right bar in ‘Opinion Projection’ represents the belief mass allocation from hyper-opinion to multinomial-opinion, which is more related to opinion projection. If the paper is accepted, we will make Figure 2 more clear.\n\n> In Figure 4, the uncertainty distribution of In-Distribution and Out-of-Distribution data is similar between HEDL w/o projection and HEDL. This observation raises the question of the contribution of the opinion projection. Could the authors further explain the specific role and impact of the opinion projection in the whole method?\n\nAs shown in **Table 2**, the evidence needs to be projected from hyper-opinion to multinomial-opinion for accurate classification. The opinion projection operation can ensure the correct allocation of vague and sharp evidence, thereby obtaining accurate classification results.\n\n> Could the authors clarify the conceptual difference between a 'Dirichlet hyper distribution' and a standard 'Dirichlet distribution' in Equation 8? From my understanding, it seems that the two distributions equal. If there is a nuanced difference, please provide an explanation to delineate the unique characteristics of the 'Dirichlet hyper distribution' as employed in your method.\n\nThe Dirichlet hyper distribution is built upon hyper-domain. \nInstead of building evidence for singletons in a domain, Dirichlet hyper distribution treats the set as a singleton element in hyper-domain. \nThus Dirichlet hyper distribution is able to model the evidence for set (support multiple singletons in the same time), while Dirichlet distribution is limited to model evidence for singletons only. Modeling evidence for set enables Dirichlet hyper distribution to take vague evidence into consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RcbhewHVRW", "reviewer": "Reviewer_Y3V6", "summary": "This paper studies the problem of out-of-distribution (OOD) detection. Traditional Evidential Deep Learning framework collects sharp evidence that supports a single category while ignoring vague evidence that supports multiple categories, leading to inaccurate uncertainty estimation and decreased OOD detection performance. The authors introduce hyper-domain and propose Hyper-opinion Evidential Deep Learning. With hyper-opinion, HEDL explicitly models evidences as sharp and vague evidences that support single categories and multiple categories respectively. HEDL extends the framework of EDL and establishes more accurate uncertainty estimation for OOD detection. Experiments on several datasets demonstrate the effectiveness of the proposed method.", "questions": "1.Whether the opinion projection process that translates hyper-opinion into standard multinomial-opinion, leads to a loss of uncertainty information? Please provide a detailed discussion on this matter.\n2.Whether HEDL encounters the issue of exponential explosion, a challenge mentioned in subjective logic as the number of categories expands. Could the authors address this potential concern and discuss any mechanisms to manage such an increase in complexity?", "limitations": "The limitations discussed in the manuscript do not appear to significantly constrain the applicability of the method.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "-The proposed method is sound in theory. HEDL extends the framework of EDL, establishes a more accurate uncertainty estimation framework for OOD detection. It’s quite novel and enlightening.\n-Without additional regularization terms nor computational complexity, it maintains the simplicity and generalizability.\n-This paper has sufficient experiments to demonstrate the effectiveness of the proposed method and each module. It achieves better performance compared with SOTA methods.", "weaknesses": "-To ensure fairness, the value of W_{prior} should be explicitly stated and consistent with EDL, cause the value of W_{prior} has been proved to be crucial to the effectiveness of EDL model [1].\n-The authors may consider incorporating the KL divergence as a loss term to enhance the model's performance. \n-In the section when proofing the existence of the gradient vanishing problem in EDL. It would be beneficial to delineate the conditions where o_k is less than zero occurs.\n-The paper could benefit from a wider analysis of the impact of the other two loss functions mentioned in EDL, e.g., the MSE loss and the Log loss, when applied to the HEDL model. \n\n[1] Chen M, Gao J, Xu C. R-EDL: Relaxing Nonessential Settings of Evidential Deep Learning[C]//The Twelfth International Conference on Learning Representations. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Y3V6,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our paper and providing us with valuable feedback. Here are our replies.\n\n> To ensure fairness, the value of $W_{prior}$ should be explicitly stated and consistent with EDL, cause the value of $W_{prior}$ has been proved to be crucial to the effectiveness of EDL model [1].\n\nIn all our experiments, the value of $W_{prior}$ is set to 1, the same as in EDL, denoting no prior information of the evidence.\n\nThis clarification is mentioned in section 3.1, if the paper is accepted, we will add this definition to the experimental setup too.\n\n> The authors may consider incorporating the KL divergence as a loss term to enhance the model's performance.\n\nWe tried to add KL divergence as a constraint yet obtained worse model effect. In fact, HEDL will extract and retain vague evidence, while KL divergence tends to constrain the generation of vague evidence. The conflict between them thus affecting the effect of the model.\n\n> In the section when proofing the existence of the gradient vanishing problem in EDL. It would be beneficial to delineate the conditions where $o_k$ is less than zero occurs.\n\nEq. 27 shows that in each back-propagation process, for categories that are not ground-truth, the gradient values of the fully connected layer weights will experience large gradient descent. For some categories (determined by the initial parameters of the model), after undergoing multiple large gradient descents, the fully connected layer weights are degraded, resulting in the output $o_k$ being less than 0.\n\n> The paper could benefit from a wider analysis of the impact of the other two loss functions mentioned in EDL, e.g., the MSE loss and the Log loss, when applied to the HEDL model.\n\nThanks for your suggestion. The results of applying the other two loss functions are as follows. Different loss functions have limited effect on the model performance.\n\n| Method | CIFAR-10 |     |     |     | CIFAR-100 |     |     |     |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|     | FPR95 | AUPR | AUROC | Acc | FPR95 | AUPR | AUROC | Acc |\n| MSE | 18.27 | 92.66 | 95.16 | 95.47 | 56.14 | 89.15 | 88.65 | 80.67 |\n| Log | 15.33 | 93.64 | 95.71 | 95.62 | 52.44 | 89.70 | 89.15 | 80.39 |\n| Digamma | 15.55 | 94.47 | 96.27 | 95.66 | 55.14 | 89.07 | 89.59 | 80.40 |\n\n> Whether the opinion projection process that translates hyper-opinion into standard multinomial-opinion, leads to a loss of uncertainty information? Please provide a detailed discussion on this matter.\n\nIn the process of projecting hyper-opinion to multinomial-opinion, vague evidence is allocated to specific singletons, while the total evidence mass does not change. At the same time, the uncertainty of each singleton and the overall uncertainty remains unchanged. Due to Eqs. 10 and 18, there is no uncertainty loss in the opinion projection process.\n\n> Whether HEDL encounters the issue of exponential explosion, a challenge mentioned in subjective logic as the number of categories expands. Could the authors address this potential concern and discuss any mechanisms to manage such an increase in complexity?\n\nInstead of building the corresponding evidence for each element, we build the extracted features as evidence in the hyper-domain. The number of evidence we extract are determined by the feature dimensions. So the number of evidence does not change with the number of categories, which avoids the exponential explosion problem.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z2w1ZLqqJw", "reviewer": "Reviewer_cb7H", "summary": "This manuscript takes a significant step forward in the realm of evidential deep learning by considering a more holistic hyper-opinion evidence framework.  The approach presented offers a novel perspective for optimizing evidential deep learning models, notably enhancing their ability to detect out-of-distribution instances without additional settings. This enhancement is particularly evident when the models are applied to complex datasets.", "questions": "I had some concerns on the setup of the ablation experiment. What implement the authors take to transform from hyper-opinion to multinomial-opinion without opinion projection?", "limitations": "The limitation is understandable and the method does not present any potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The manuscript is commendable for presenting a fresh and innovative approach to the framework of evidential deep learning. It is noteworthy that while previous research has mainly concentrated on enhancing the settings in evidential deep learning, there has been an oversight in addressing the structural deficiencies of the framework itself.\n\nThe authors have introduced an interesting concept by treating the features extracted by the network as evidence when applying the theory to practical scenarios. The authors also give full mathematical proof on the problems they solved. This approach grounds in a solid theoretical foundation based on subjective logic, demonstrates simplicity in its application and shows impressive results in experimental validation.", "weaknesses": "The manuscript exhibits some shortcomings in the handling of certain details. The hyper-domain does not encompass the set itself and the empty set; however, in practice, due to the treatment of features as evidence, the possibility of evidence representing these two sets cannot be ruled out. Furthermore, the subjective logic stipulates that evidence from the same subset should appear at most once, yet the approach presented in this paper does not guarantee full compliance with this condition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer cb7H,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our paper and providing us with valuable feedback. Here are our replies.\n\n> The manuscript exhibits some shortcomings in the handling of certain details. The hyper-domain does not encompass the set itself and the empty set; however, in practice, due to the treatment of features as evidence, the possibility of evidence representing these two sets cannot be ruled out. Furthermore, the subjective logic stipulates that evidence from the same subset should appear at most once, yet the approach presented in this paper does not guarantee full compliance with this condition.\n\nDue to the opinion projection without prior information, both the full set and the empty set contribute the same to each category. For full set, it contributes $\\frac{1}{K}$ (where K is the number of categories) evidence to each category. For empty set, it contributes none evidence to any category. Therefore both situation will not affect the model accuracy.\n\nBefore the opinion projection, there may exists multiple pieces of evidence supporting the same set. But these evidences will be accumulated as one piece of evidence, which meets the condition of Subjective Logic. As in the 'Hyper-opinion Belief' section of Figure 2, the belief mass for one set appears at most once.\n\n> I had some concerns on the setup of the ablation experiment. What implement the authors take to transform from hyper-opinion to multinomial-opinion without opinion projection?\n\nWe first model the evidence on hyper-domain to form the hyper-opinion, and then use the fully connected layer for opinion assignment. ReLU activation is applied for the fully connected layer weights and the bias is set to 0 to ensure the non-negative assignment of hyper-opinion evidence.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces Hyper-opinion Evidential Deep Learning (HEDL), an extension of the Evidential Deep Learning (EDL) framework, for out-of-distribution detection. The method addresses the limitations of EDL by incorporating both sharp evidence (supporting single categories) and vague evidence (supporting multiple categories). The authors introduce an opinion projection mechanism that mitigates the vanishing gradient issue encountered in EDL. The method is shown to outperform existing OOD detection methods across various datasets.

All reviewers acknowledged its significant contributions, sound theoretical foundation, and solid experimental validation. They consistently rated the paper positively, with all ratings falling within the weak accept-to-accept range.

The main strengths of the paper:
The paper presents a novel extension to the EDL framework, enhancing its capability for OOD detection by effectively modeling both sharp and vague evidence. The introduction of the opinion projection mechanism is a meaningful contribution, addressing the vanishing gradient issue and improving classification accuracy without increasing model complexity. Extensive experiments demonstrate that HEDL outperforms state-of-the-art methods in OOD detection, confirming the practical relevance of the proposed approach.
The paper is grounded in solid theoretical foundations and includes rigorous mathematical proofs to support the proposed innovations.

Weaknesses:
Some equations were initially difficult to understand, and the paper could benefit from additional clarifications and improvements in the presentation of certain concepts before the camera ready. Concerns were raised about the fairness of comparative evaluations due to the use of a pre-trained feature extractor. The authors clarified that consistent experimental conditions were applied across all methods.
Minor concerns were noted about the clarity of some figures and the explanation of certain terms, which were addressed in the author rebuttal.

Discussion: The authors effectively addressed all concerns raised by the reviewers during the rebuttal process. They provided additional explanations, clarified ambiguous points, and presented further experimental results, which satisfied the reviewers and led to an overall positive reception of the paper. 

All in all, given the strong theoretical contributions, robust experimental validation, and positive feedback from reviewers, I recommend accepting this paper. The paper is expected to have a positive impact in the area of probabilistic methods and OOD detection, contributing valuable insights to the NeurIPS community.
Decision: Accept (poster)