Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer
Authors: Xinyue Hu
Zhibin Duan
Bo Chen
Mingyuan Zhou
Venue: ICLR 2025 Poster (2025)
Keywords: Factor Analysis
Uncertainty Estimation
explainable AI
Bayesian Last Layer
Primary area: interpretability and explainable AI

Abstract: Although deep neural networks have demonstrated significant success due to their
powerful expressiveness, most models struggle to meet practical requirements for
uncertainty estimation. Concurrently, the entangled nature of deep neural net-
works leads to a multifaceted problem, where various localized explanation tech-
niques reveal that multiple unrelated features influence the decisions, thereby un-
dermining interpretability. To address these challenges, we develop a Bayesian
Nonnegative Decision Layer (BNDL), which reformulates deep neural networks
as a conditional Bayesian non-negative factor analysis. By leveraging stochastic
latent variables, the BNDL can model complex dependencies and provide robust
uncertainty estimation. Moreover, the sparsity and non-negativity of the latent
variables encourage the model to learn disentangled representations and decision
layers, thereby improving interpretability. We also offer theoretical guarantees
that BNDL can achieve effective disentangled learning. In addition, we developed
a corresponding variational inference method utilizing a Weibull variational in-
ference network to approximate the posterior distribution of the latent variables.
Our experimental results demonstrate that with enhanced disentanglement capa-
bilities, BNDL not only improves the model’s accuracy but also provides reliable
uncertainty estimation and improved interpretability.

Published as a conference paper at ICLR 2025
ENHANCING UNCERTAINTY ESTIMATION AND INTER- PRETABILITY VIA BAYESIAN NON-NEGATIVE DECI-
SION LAYER
Xinyue Hu" Zhibin Duan'-2, Bo Chen Mingyuan Zhou²
National Key Laboratory of Radar Signal Processing, Xidian University. Xi an, 710071, China.
2 School of Mathematics and Statistics, Xi' an Jiaotong University, Xi' an, Shaanxi, China.
3 McCombs School of Business, The University of Texas at Austin, Austin, TX 78712
sayyuhuu228@gmas
com zbduan@xjjun edu, cn
bchenêmail xi widian... edu. en mingyuan. zhou@mccombs utexas. edu
ABSTRACT
Although deep neural networks have demonstrated significant success due to their powerful expressiveness, most models struggle meet practical requirements for uncertainty estimation. Concurrently, the entangled nature of deep neural net- works leads to multifaceted problem, where various localized explanation tech- niques reveal that multiple unrelated features influence the decisions, thereby un- dermining interpretability. To address these challenges, Non-negative Decision Layer (BNDL), which reformulates deep neural networks conditional Bayesian non-negative factor analysis. By leveraging stochastic latent variables, the BNDL can model complex dependencies and provide robust uncertainty estimation. Moreover, the sparsity and non-negativity of the latent variables encourage the model to learn disentangled representations and decision layers, thereby improving interpretability. We also offer theoretical guarantees that BNDL can achieve effective disentangled learning. In addition, we developed a corresponding variational inference method utilizing a Weibull variational in- ference network to approximate the posterior distribution of the latent variables. Our experimental results demonstrate that with enhanced disentanglement capa- bilities, BNDL not only improves the model's accuracy but also provides reliable uncertainty estimation and improved interpretability.
develop Bayesian
INTRODUCTION
Over the last decade, deep neural networks (DNNs) have achieved significant success and have been widely applied across various research domains (LeCun et al., 2015). As these applications expand, quantifying uncertainty in predictions has gained importance, especially for Al safety (Amodei et al.) 2016). A key goal of uncertainty estimation is to ensure that neural networks assign low confidence to test cases poorly represented by training data or prior knowledge (Gal & Ghahramani 2016). One approach to this challenge is Bayesian neural networks, which treat model parameters dom variables. Although progress has been made in developing approximate inference methods for Bayesian neural networks (Li al. 2016; Louizos & Welling 2017 [hi al.) (017)), computa- tional scalability continues to pose significant obstacle. Alternatively deterministic methods such as deep ensembles (Lakshminarayanan
al.) (017) and dropout (Gal & Ghahramani] 2016) have
been proposed. However, these approaches necessitate running full DNNs multiple times during the testing phase, resulting in high computational costs and increased inference time (Fan al. (021))
In addition to uncertainty estimation, there is growing demand for interpretability in DNNs. aimed helping users understand model decisions through various tools. Advanced techniques have been developed to provide localized explanations by identifying key features or regions influencing out-
"Equal contribution.
Corresponning author.

---

Published as a conference paper at ICLR 2025
puts (Olah et al. 2017 Yosinski et al. 2015). However, a key challenge in this field is that neurons within well-trained DNNs tend to be multifaceted (Nguyen et al.) 2016). meaning they respond to multiple, unrelated features. This phenomenon may arise from the entangled nature of DNNs, wherein multiple features are utilized for various tasks. To address this challenge, significant efforts have been made in the literature, including the use of specialized regularizers aimed at promoting feature disentanglement (Nguyen et al., (2016), and the employment of sparse linear decision layers select the most important features (Wong et al. 2021). While these approaches have led to no- table advancements, they warrant further exploration from both practical and theoretical standpoints. Specifically, the method of employing a sparse linear decision layer may impair task performance due to the loss of essential dimensional information. Furthermore, while empirical results suggest that sparsity enhances model interpretability and mitigates the challenges posed by multifaceted neurons (Moakhar et al. 2023), these claims lack rigorous theoretical support.
While deep neural networks (DNNs) face inherent limitations, non-negative factor analysis (NFal-notably Poisson variants (Zhou et al.
strengths in sparse concept dis- entanglement and uncertainty-awarr modeling through stochastic latent variables (Lee & Seung, 1999). Thus, compelling motivation exists to transfer the good properties of NEA to DNN. How- ever, their paradigm divergence poses challenges: NFA relies on shallow linear architectures, while DNNs employ deep non-linear hierarchies. key insight lies in decomposing DNNs into non-linear feature extractors and linear decision layers, where recent studies show the latter is sufficient for capturing uncertainty (Kristiadi et al., 22000] DDulliawlla al., 2023b Joo et al., 2020 Parekh al.) 2022). Further, [Wong et al. (2021) demonstrates that leveraging a sparse decision layer can en- hance model's interpretability through innovative techniques. This line of inquiry motivates the use of NEA as a linear decision layer, integrated with DNNs as feature extractors, to enhance both uncertainty estimation and model's s interpretability.
With the considerations above, we developed a Bayesian Non-negative Decision Layer (BNDL), designed to empower DNNs with enhanced interpretability and uncertainty estimation capabilities. Specifically, under the categorical likelihood, the label is factorized into gamma-distribune score matrix (local latent variables) and a corresponding gamma-distributed factor loading matrix (global latent variables). The former represents the latent representation of the observation, while the latter captures the interaction between the latent variables and the label. Given the challenge of intractable posterior distributions for the latent variables, introduce deep Weibull varia- tional neural network to effectively approximate the gamma-distributed latent variables (Zhang et al.] 2018). All parameters trained using stochastic gradient descent (SGD) within variational infer- ence framework. Furthermore, we provide theoretical guarantées for the model's disentanglent lement capabilities, which enhances its interpretabiliiyy Additionally, our complexity analysis indicates that the increase in computational effort is minimal during both the training and uncertainty testing phases. To assess the efficacy of the proposed model, we conducted evaluations on a wide range of benchmark datasets using image classification tasks. The experimental results demonstrate that the proposed approach consistently outperforms standard classification models and offers superior uncertainty estimation. The main contributions of the paper can be summarized as follows:
factor
We develop flexible Bayesian Non-negative Decision Layer (BNDL) for deep neural networks, empower its interpretability and uncertainty estimation capabilities.
The complexity analysis shows that the computational overhead introduced by BNDL is minimal compared to DNNs. Further, we take theory analysis to verify its disentangled properties.
We assessed the effectivvness of BNDL across multiple datasets, including CIFAR-1O, CIFAR- and ImageNet-Ik. BNDL not only preserves even enhances baseline per- formance but also facilitates uncertainty estimation and improves the interpretability of neurons.
2 RELATED WORK
2.1 UNCERTAINTY ESTIMATION
Existing research in supervised learning has focused on modeling conditional distributions beyond the mean, particularly for predictive uncertainty. Ensemble methods (Liu et al., 2021 Lakshmi-
2

---

Published as a conference paper at ICLR 2025
X
0
0
p(vlx)
p(0)x)
p(01x)
P(v|0, s)
q [x)
X
Φ
X
(a) DNNs
(b) Extend DNNs with
(c) Generative Model of BNDL
(d) Inference Model of BNDL
Illustration of the graphical models. I(a): the predictive process of output Y for the baseline Deep Neural Network; I(b): the generative model of DNNs with introducing stochastic latent variable 0: I(c): the generation model of the Bayesian non-negative decision layer; and I(d); corresponding approximate inference for latent variables
Figure 1:
narayanan et al.
combine neural networks with stochastic outputs to quantify uncertainty, while Bayesian neural networks (BNNs) use distributions over network parameters to reflect model
2017)
Adams 2013 Kingma et al. 2015 Gal & Ghahramani 2016 Tomezak et al. 2021). However, BNNs are complex and difficult to train. In
plausibility (Blundell et al. 2015 Hemándezz-obate
contrast, Bayesian Last Layer (BI.L) methods, which focus uncertainty only on the output layer,
offer a simpler, more efficient alternative (Oher & Rasmussen 2019 Watson et al. [021] Daxberger
itt al. 2021 Kristiadi et al. 2020 Harrison et al.|| 20024) BLL techniques, such as Ober & Rasmussen (2019)' 's noise marginalization and Daxberger et al. (02Tl'ss use of Laplace Approximations (LA), improve probabilistic predictions. Recent advancements like retraining objectives (Weber et al, 2018) and variational improvements (Harrison et al., 2024 [Wtssnn al., (222]) have expanded BLL's application in uncertainty estimation tasks. Another relevant approach is evidential deep learning (EDL), which uses higher-order conditional distributions, like the Dirichlet distribution, for uncertainty estimation (Sensoy et al., 2018;; Malinin & Gales 2018). EDL has shown effec- tiveness in tasks such as classification (Sensoy et al.) 2018) and regression (Malinin et al. 2020). Furthermore, Amiii et al. (2020) proposed training networks to infer hyperparameters of evidential distributions, enabling scalable uncertainty learning. Building on BLL principles, BNDL models the decision layer as Bayesian generative model, using a gamma prior to enhance feature sparsity and disentanglement, improving both uncertainty estimation and interpretability.
2.2 INTERPRETABILITY TOOLS FOR DEEP NEURAL NETWORK
The goal of neural network interpretability is to identify the mechanisms underlying DNN's decision-making processes. The related research ranges from approaches which link abstract con- cepts structural network components, such specific neurons, for example via visualization (Yosinski et al. 2015 Nguyen et al. 2016), to approaches which aim to trace individual model outputs on per-sampl basis such as local surrogates (Ribeiro et al., 2016) and salience maps (Si- monyan et al. (013). However, as noted by various recent studies, these local attributions can be easy to fool or may otherwise fail capture global aspects of model behavior (Adebayo et al.) 2018 Leavitt & Morcos 2020 Wong et al. 2021). A major confounder for interpretability is that neurons in a well-trained DNN are often multifaceted (Nguyen et al. 2016 Moakhar et al. 2023), responding to various, often unrelated, features. In contrast, our approach ensures that the identified high-level concepts-i.ed the deep features utilized by the sparse decision boyer-full, determine the model's behavior.
BAYESIAN NON-NEGATIVE DECISION LAYER
This section first reformulates the traditional DNNs as a latent variable model (Sec. 3.1 and then provides a detailed description the proposed Bayesian non-negative decision layer, which consists of the generative model (Sec. (3.2 and the variational inference network (Sec. 3.3). followed by the description of the proposed variational inference algorithm (Sec. 3.4).
3

---

Published as a conference paper at ICLR 2025
3.1 PRELIMINARIES ON DEEP NEURAL NETWORKS
We first adopt a latent variable model to re-examine the DNNs, which are commonly tackled by training dopain-specifie neural networks with a sigmoid or softmax output layer. Dataset samples are mapped deterninistically by a neural network / to real vector = = f(x),, which is transformed in the softmax layer to a point on the simplex A a discrete distribution over class labels E y:
exp(zwy + bby p 2) Evey exp(z + by/
(I)
Where the Wy and by represent the weights and biases of the final fully connected layer. Thus, the classification process can be viewed as generative process for label y. as shown in Fig. [a). Previous works (Dhuliawala et al. 2023a Joo al. 2020) have shown that a softmax classifier is special case of Equation 2
p x)
p(y =)p(= |xx))
The neural network f. up to the softmax layer, models p(z|1r) delta distribution fz-((x). with the softmax input representing a sample from p(z[zz, and p(yl2z) defined by the softmax layer. While the softmax output can be viewed as a categorical distribution, the limited randomness from the output layer is often insufficient for capturing complex dependencies (Chung et al.) 2015). Ad- ditionally, DNNs with softmax layer face overconfidence issues (Guo et al. 2017] 7 Kristiadi et al.) 2020 Liu al., 2020), complicating uncertainty estimation. Furthermore, the data feature and parameter Wy are often entangled as dense vectors, leading to multifaceted phenomena in local- ized explanations (Nguyen et al., 2016). These challenges motivate the development of a Bayesian non-negative decision layer.
3.2 BAYESIAN NON-NEGATIVE DECOMPOSITION LAYER
Building on the latent variable model of the softmax classification task, we can further reformu- late DNNs Non-negative Factor Analysis, referred to the Bayesian Non non-negative Decision Layer (BNDL). Firstly, to better capture complex dependence and alcatoric uncertainty, referring to the inherent randomness in the data that cannot be explained away, erative model of original DNNs with stochastic latent variables by modeling latent representation with a distribution. Thus, illustrated in Fig. [b), the Eq.
intuitively extend the gen-
2 is improved
p x)) = / p(u p)p(@ xx)
To further account for epistemic uncertainty, which refers to the uncertainty inherent in the model itself, we treat the weights of the final fully connected layer as stochastic latent variables. The generative model then defined follows, and its graphical model shown in Fig. [e):
ply x)
p(y 0, 2 p(@ ())p)))))))
(4)
1..
This formulation bears similarities to factor analysis Yu et al. 2008), where both 0 and + in clas- sical DNNs are commonly sampled from Gaussian distribution, making the generative model akin to Gaussian factor analysis. However, while Gaussian factor analysis can effectively evaluate uncer- tainty, it struggles to achieve disentangled representation learning with dense latent variables (Moran et al. (021), which is crucial for model interpretability (Nguyen et al.) 2016).
Considering the gamma distribution possesses both non-linearity and non-negativity, we use the gamma distribution the prior distribution for 0 and 4. This choice allows for a more comprehen- sive capture of complex relationships within the model (Zhou et al., 2015 Duan et al., 2021 2024), while also accommodating the characteristics of sparse non-negative variables, thereby enhancing the disentanglement and interpretability of the learned representations and corresponding decision layers & (Lee & Scung 1999). Specifically, given data samples x, and their corresponding one-hot label y E RL, where C is the number of classes, we can factorize y, under the category likelihood
as follows:
yj 03 Category (0,0), 0g a. Gamma (Jo(zz), 1),
Gamma (1, (5)
4

---

Published as a conference paper at ICLR 2025
where 0 I is the factor score matrix, each column of which encodes the relative importance of each atom sample æ;; and E R.KCC' is the factor loading matrix, each column of which is factor encoding the relative importance of each term. Intuitively, in a classification problem, the k-th column of & E R K'xC denoted as Øb E RR. represents the k-th distribution across all classes. Although in our formulation the prior of the latent variable 9 is modulated by the input æ.j. the constraint can be easily relaxed to allow the latent variable to be statistically independent of the input variable (Sohn al., 2015) Kingma et al. 2014). Therefore, we can simply define the data-independent prior of the latent variable 0 as fo(I,)
3.3 VARIATIONAL. INFERENCE NEURAL NETWORK
We have constructed the generative process y. which includes the parameters 0 and &, in Sec. 3.2 Due to the intractable posterior in BNDL, we build Weibull variational inference to approximate the posterior of 0 and T
Weibull Approximate Posterior: While the gamma distribution appears suitable for the posterior distributiinn due to its encouragement of sparsity and adherence to the nonnegative condition, directly reparameterizing the gamma distribution can result in high noise (Zhang al., 2018 Kingma & Welling 2014 Knowles 2015 Ruiz et al. 2016 Naesseth et al. (2017). and using the REINFORCE method for gradient estimation may lead to large variance (Williams 1992). Hence, we use the reparameterizable Weibull distribution ZZang et al.) 2018) to approximate the posterior for the gamma latent variables, mainly due to the following considerations: 2), the Weibull distribution has simple reparameterization so that it is easier to optimize; id) the Weibull distribution is similar to a gamma distribution, capable of modeling sparse, skewed and positive distributions. Specifically, the latent variable (eibull(k:; A) can be easily reparameterized
x = X In(1 - e)) & ~~ Uniform(O, 1).
(6)
Where X and l are the scale and shape parameter of Weilbull distribution respectively; iit), The KL divergence from the gamma Weibull distributions has an analytic expression as:
KL (Weibull(k; X) Gamma(a, (0) = y - a log X + log k: + 3A1 (1-
k
- a aog f + log ((a)
where y is the Mallh-MMachrronii constant.
Local latent variables Inference Network: As shown in Fig. [d), the variational inference net- work construct the variational posterior:
q (0, 2;) Weibull (ky,))
(8)
where the inference network can be defined as
ky = Softplus (fi))) A5 = ReLu (fx(,)) /exp 1/kk) h fNN(N)
(9)
where h, an extracted feature with deep neural networks, such ResNet, which can be seen as a deep feature extractor; Let f.(.) denotes the neural network, where fNN is the feature extractor, encompassing all layers from the input layer to the penultimate layer, and fx and fx are the network to infer the scale and shape parameters of Weibull distribution, respectively. The Softplus function, defined as log(1 + exp(-)). ). is applied element-wise non-linearity to ensure positive Weibull shape parameters. The Weibull distribution is used to approximate the gamma-distributed conditional pos- terior, and its parameters (())) E R and x k are inferred by the bottom-up data information using the neural networks.
Global latent variables inference Network: For the same reason, we also use Weibull distributions to approximate the posteriors of global latent variables & E R xC formulated as
-) Weibull (ko. Xo)
(10)
where the inference network can be expressed as:
k +
Softplus(W:
(1) 4
Relu Relu(W2)//xp) 1/ks)
(11)
5

---

Published as a conference paper at ICLR 2025
Note that W1 and W2 are randomly initialized matrices with dimensions matching T.
Connection with Non-Negative Matrix Factorization: Equations 8 and ensure that B = and [[$) Xg. Therefore. instead of sampling 0, and & from their respective distributions, substituting their expectations makes the mapping equivalent to that of standard non-negative matrix factorization, resulting in y >3 >, In other words, if we let the shape parameter & of the Weibull distribution approach infinity-impe implying that the variance of the latent variables approaches zero, and the distribution collapses into point mass concentrated the expectation, then the proposed stochastic decision layer reduces to non-negative matrix factorization. For further discussions on NMF, please refer to Appendix A.3
3.4
VARIATIONAL INFERENCE
For BNDL, given the model parameters referred to Z. which consist of the parameters in the generative model and inference network, the marginal likelihood the dataset (X, Y) is defined as:
/ [I p (ys 0,, + (0,)
/
p (Y
(12)
The inference task is to learn the parameters of the generative model and the inference network. Sim- ilar to VAEs, the optimization objective of the BNDL can be achieved by maximizing the evidence lower bound (ELBO) of the log-likelihood as:
= ((Y)
Σ
ee, my)
[Imp (y/ θz. $)
1=1
(13)
J
In (03 x.))] p (0,)
E E In 2 ( -) p (
E E0 1 Σ
where the first term is the expected ligeliieldoood of the generative model, which ensures reconstruc- tion performance, and the last two term is the Kullback -ullback-Leibler (KL) divergence that constrains the variational distribution (-) be close to its prior p(-). The parameters in the Generalized GBN can be directly optimized by advanced gradient algorithms, like SGD (Kingma & Ba 2015).
Complexity analysis Modifying the last layer of base deep neural networks minimally increases parameter count, resulting in negligible space complexity. The time com- plexity of ResNet is dominated by its convolutional layers and can be expressed as: o ELach x c(ll H x W()) KOO k.). where H, W, K K. Cnn and Cout the input height, width, kernel size, input channels, and output channels, respectively. In BNDL, the added time complexity from KL divergence computations for local and global latent variables ( and O(C(L)) out x C). respectively. which are negligible compared to ResNet's overall complexity. Unlike traditional ensemble methods and dropout approaches, which require multiple full-network runs to assess uncertainty, BNDL performs a single forward pass to infer the variational posterior, followed by sampling for uncertainty estimation, greatly reducing computational costs.
THEORETICAL GUARANTEES FOR BNDL
We provide theoretical guarantees for BNDL from the perspective of identifiable features. As de- scribed in Sec. [.3, BNDL can be viewed as Non-negative Matrix Factorization (NMF) problem. From this perspective, its objective function p (Y (Φ, 0 X)) can be further reformulated as:
min 020,020 22
0Φ 2
(14)
In the realm of non-negative matrix factorization, many studies have aimed to establish the iden- tifiability and uniqueness of decomposition 0 up to permutation and scaling. Recently, Gillis & Rajkó (2023) demonstrated that subset of the columns of 0 and & can be identified and made unique under more relaxed conditions. We will show how BNDL adheres to the criteria set forth in Gillis & Rajkó (202)), thereby enabling the learning of partially identifiable features.

---

Published as a conference paper at ICLR 2025
Proposition! 1 ((Gillis & Rajkó 2023)). The k-th column of 0 is identifiable under the assump-
tions:
Selective Window: There exists row of ④. say the j-th, such that Φ = ae(xy for
a > where e/), represents the k-th standard row vector in vector space.
Sparsity Constrain: The k--th column of a contains at least 1 entries equal to zero, where r is the rank Y
The selective window assumption states that the column in 0 corresponding to Φ(j.:) is unique in the dataset, which reasonable many applications Gillis 2020), e.g., 0 can represent latent classes in classification task, where Wang et al. (2024) achieves full identifiability by assuming each latent class corresponds to a unique sample. Under this assumption, it suffices to have a single latent class with unique sample, making it more feasible and easier satisfy. For the sparsity constraint, the use of a gamma prior and a ReLU activation function in & within the BNDL frame- work, outlined in and TT enforces sparsity during the training process. Moreover, instead of relying on the parameter norm (Wong et al. 2021), which often leads to performance degradation, we propose a more scalable and effective adaptive activation function achieve sparsity. Specifi- cally, we employ fxx ReLU(( o),, where ar is predefined constant, set as a hyperparameter. This approach offers more flexible mechanism for inducing sparsity without sacrificing model per- formance. Similarly, we can demonstrate the partial identifiability of db as it is often considered to be the transpose of (Fu et al., 2017 HaaoChen et al., 2021}. In conclusion, BNDL follows the aforementioned assumptions, and its optimization objective promotes the partial identifiability of the learned features and decision layer, thereby enhancing their disentanglement capability. We validated our theoretical guarantees through experiments presented in Sec. 5.2 More details for the theoretical guarantees can be found in Appendix A.2
EXPERIMENTS
Experiment Setup Following Wong al. (2021). 1) we analyze the following models: (a) ResNet trained on ImageNet- Ik (Deng et al., 2009; Russakovsky et al. 2015) and Places-10 subset of Places365 (Zhou al. 2017)) ). and ResNet-18 trained on CIFAR-10//00 Krrzhevsky 2009); (b) ViT-based model (Dosovisskiy et al. 2021) with pretrained weights from Hugging Face" Baselines are detailed in Sec. |.I] We evaluate accuracy and uncertainty for quantitative analysis and use LIME for qualitative insights into BNDL predictions. Setup details, including hyperparameters and datasets, are provided in Appendix A. Our code available at http://github.com/XYine You WwW.XYHHuH222NNNMM
Uncertainty evaluation metric. We estimate uncertainty using a hypothesis testing approach (Fan al. 2021). This method provides interpretable p-values, enabling practical deployment for binary uncertainty decisions. A prediction's 's certainty is determined by comparing its p-value against threshold. To evaluate uncertainty estimates, (PAvPU) metric (Mukhoti & Gal 2018). which defined as PAvPU = (Nae + nju V(nge + Nau + Nic nad where Nlacs Plan Niee and Niu represent the counts of accurate-certain, accurate-uncertain, inaccurate-certain, and inaccurate-uncen that the model reliably produces accurate predictions with high certainty and inaccurate ones with high uncertainty.
the Patch Accuracy
Patch Uncertainty
samples, respectively. Higher PAvPU values indicate
Sparsity Measurement We measure the sparsity of the final decision layer weights. Since the weights in our method are non-negative, we follow the approach of (Wong et al., 2021) and (Wang al. 2024), considering weights greater than 1 x 10 as non-sparse (denoted inon) and the remaining values as sparse (denoted as Leparse ). The sparsity is calculated as nnz = hnon/ (hnon + Isparse), where smaller value indicates a higher level of sparsity.
5.1 CLASSIFICATION PERFORMANCE
Specifically, for each dataset, we conducted the following experiments:

---

Published as a conference paper at ICLR 2025
Table 1: Overall model accuracy across different datasets, with BNDL being our method. We use ResNet-50 as the baseline for ImageNet-Ik Ik, and ResNet-18 as the baseline for CIFAR-10 and CIFAR-100. Vit refers sit-basepaatchl6-224
Model
CIFAR-10
CIEAR-100
ImageNet-Ik
ACC
PAYPU
ACC
PAvPU
ACC
PAvPU
ResNet
94.98 0.12
74.62 + 0.23
75.33 0.14
MC Dropout
94.54 0.03
78.83 1 0.12
78.12 1 0.06
64.41 1 0.22
75.98 0.08
76.50 = 0.02
BM
94.07 + 0.07
93,98
75.81 1: 0.34
77.13 1 0.67
CARD
90.93 0.02
91.11 1 0.04
71.42 1 0.01
71.48 1 0.03
76.20 0.00
76.29 1 0.01
ResNet-BNDL
95.54 = 0.08
95.58 = 0.20
79.82 + 0.13
81.1 = 0.21
77.01 = 0.14
77.66 = 0.03
ViT-Base
95.51 0.03
84.15 + 0.03
80.33
ViT-BNDL
96.34 0.04
97.01
0.02
85.16 1. 0.03
86.37 1. 0.11
81.29 0.02
82.50
0.03
(a) Performance and Uncertainty Evaluation We replaced the decision layers in the network with BNDL and performed supervised training from scratch. The results are shown in Table [ The baseline models are grouped into two categories: 1) Uncertainty estimation networks, including Bernoulli MC Dropout & Ghahramani] 2016), BM (Joo et al., 2020) and CARD (Han et al.,
2022] 2) Dense decision layer baselines: including ViT-Base (Dosovitskiy et al.) 202T) (We used the pretrained weights for the weebbass-ptcchlle 16 only, modifying the decision layer for continued training.) and ResNet (He et al.) 2016). is important to note that the goal of BNDL not to achieve substantial improvement in the model's performance, but rather to preserve the model's performance while enhancing its interpretability and uncertainty estimation capabilities.
(b) Impact of Sparsity. We replaced the decision layers of pre-trained model with BNDL, froze the existing feature layers, and fine-tuned only the parameters of the BNDL. The results across different datasets are illustrated in Fig. where compare BNDL (shown in blue) with the Debuggable Network (Wong et al. 2021 (shown in orange), both of them utilize the same backbone with different sparse decision layers.
5.1.1 PERFORMANCE AND UNCERTAINTY EVALUATION
In T. show accuracy and PAvPU. Our model reports the mean and variance across 5 differ- ent random seeds, while the results of other models are reported from previous papers available. Since we directly used the source' to test ImageNet-Ik, the variance the table. We can observe that: 1) By leveraging stochastic latent variables to capture complex de- pendencies, BNDL consistently outperformed all datasets and demonstrated improved performance across various widely-used architectures, including ResNet and VIT. 2) The integration of BNDL en- dowed the model with the capability for uncertainty estimation, as evidenced by the improvements in PAvPU metrics when compared to several strong baselines. 3) BNDL. exhibits scalability and can be extended to larger datasets, such as ImageNet- Ik, as demonstrated by the complexity analysis.
is not provided in
High Uncerteinty
Uncertainty
Siae Lhickte
Dastee
Pacii Truii
Magui
Tost Aoowaay (70
Dovel Reat
Red Wiw
Mibile Houss
Uncertatady
Figure 2: The lefumost line chart illustrates the average uncertainty and accuracy across subsets of the ImageNet test set. The middle and right panels sample images from the subsets with the highest and lowest uncertainty, as defined by the curve. The top shows the original images with ground truth labels, while the bottom row displays the model's predictions alongside LIME visualizations.
8

---

Published as a conference paper at ICLR 2025
Relation between Uncertainty and Accuracy We conducted an ablation study explore the lationship between prediction uncertainty and downstream performance. Using ResNet-BNDL, we sorted the ImageNet test set by evaluation uncertainty into 10 subsets. For each subset, calcu- lated the average accuracy and uncertainty, then plotted the results in Fig. [ We also selected images from the highest and lowest uncertainty subsets and visualized their LIME explanations, showing the most influential features of the activations on the right side of the figure. The line chart illustrates clear negative correlation between uncertainty and accuracy: higher uncertainty corresponds to lower accuracy. This suggests that the model provides reliable uncertainty estimates, helping to avoid potential misclassifications. In the visualization, we observe that the model made correct pre- dictions for images with low uncertainty, while for images with high uncertainty, the visualizations reveal the causes of misclassification, egg., the image of wine bottle, the model primarily fo- cused on the wine glass filled with red wine in the background, leading to a misclassification as red wine, The Uncertainty Vs Acc results for additional datasets and ViT-BNDL are provided in the Appendix A.3
5.1.2 IMPACT OF SPARSITY
T-Apeady
1-deariyy
1-Ppeaily
(a) Cifar-10
(b) Imagenet- 1k
(c) Cifar-100
(d) Places-10 10
Sparsity-accure accuracy trade-offs for BNDL and Debuggable Network WWng et al. 2021 Each point on the curve represents a BNDL classifier. Horizontal dashed lines indicate the fully dense accuracy for each network. The x-axis shows the proportion of non-sparse weights, with higher values indicating denser distributions, while the y-axis represents test accuracy on each dataset.
Figure 3:
The results of fine-tuning on the original model are shown in Fig. 3 For each dataset, we control the sparsity according to the activation function mentioned in Eq. 11 resulting in the Sparsity vs. accuracy curve. The blue lines represent our method, while the orange lines represent the Debuggable Network (Wong et al. 2021), which also employs the idea of a sparse decision layer. It can be observed that at the same level of sparsity, our model overall outperforms the Debuggable Network across different datasets. Additionally, across different datasets, the decision layer can be made substantially sparser-by by up to two orders of magnitude- wwihh minimal impact accuracy (cf. Fig. (())). For instance, when the sparsity the ImageNet- classifier classification accuracy still reaches 75.7%.
classiffe is 0. 0.0024, the network's
5.2 INTERPRETABILIT EVALUATION
Disentangled representation learning To validate the disentanglement on real-world data, we adopt an unsupervised disentanglement metric SEPIN @k (Do & Tran 2019). SEPIN measures how each feature 0, is disentangled from others 0z: by computing their conditional mutual informa- tion with the top l: features, i.e., SEPIN k: 1/k Σ - T(x, 0,, (r:)) which estimated with InfoNCE lower bound (Oord et al. 2018) implemented following Wang et al. 2024).
Table 2: Feature disentanglement score on Imagenet-lk, where @k denotes the top-k dimensions. Values are scaled by 102, use ResNet-50 as the baseline for BNDL
SEPIN @1
SEPIN@ 10
SEPING @ 100
SEPIN @ 1000
SEPIN @all
ResNet50
1.50 + 0.02
1.03 + 0.01
0.60 + 0.01
0.31 + 0.01
0.23 + 0.01
ResNet50-BNDD --NDL
2.59 0.03
2.12 0.01
1.30 0.01
0.65 + 0.01
0.44 0.01

---

Published as a conference paper at ICLR 2025
Arctic Fox
Candle
Labrador
Drake
Payphone
Trafisybux
Egypdan Gat
RosNet 50
The LIME visualizations for BNDL and ResNet-50, focusing on the largest 0 for each image, show that BNDL's features align more closely with the semantic meaning of true labels, sug- gesting more disentangled representations. As we selected the top- 10 super-pixels for visualization, the results may include some less significant super-pixels; this issue alleviated when we reduce the number of top-k super-pixels.
Figure 4:
As shown in Table 2 BNDL features exhibit much better disentanglement than ResNet-50 across all top-k dimensions. The advantage is pronounced when considering the top features, learned features also contain noise dimensions. This verifies the disentanglement learned features, as analyzed in Sec. 4 BNDL indeed provides better feature disentanglement real-world data. The disentanglement results on other datasets can be found in Appendix A.3
Visualization We visualized the feature representation O of BNDL and the baseline model (ResNet-50) for the same images in ImageNet- Ik, as illustrated in Fig. 4 Specifically, we se- lected the feature with the highest activation for each image and applied in line with the approach used in Fig. 2 The top row of Fig. f shows the true categories of the corresponding images, the second row presents our visualization results, and the third row displays the visualization results of the baseline model. Overall, the visualization results of BNDL are more semantically meaningful compared to those of ResNet-50. For instance, in the image BNDL successfully captures parts of the candle, while ResNet-50 only identifies the cake. Similar observations occur in other categories, and we provide additional visualization results Appendix A.3 This finding aligns with the conclusions drawn suggesting that BNDL has learned more identifiable features through the constraint of sparsity.
LIME visualization,
candle,
6 CONCLUSION
We introduce BNDL as a simple and scalable Bayesian decision layer that excels both uncertainty estimation and interpretability. while maintaining or improving accuracy across a range of tasks, cluding large-scale applications. With an efficient parameterization of the covariance-dependenn variational distribution, BNDL enhances the flexibility DNNs with only a slight increase in mem- ory and computational cost. We demonstrate the broad applicability of BNDL on both ResNet- based and ViT-based models and show that BNDL achieves superior performance compared to these base- lines. Notably, we provide both practical and theoretical guarantees for BNDL's ability to learn more disentangled and identifiable features. Based on these results, we believe BNDL can serve as an efficient alternative to decision layer in the versatile tool box of modules.
REPRODUCILTLY STATEMENT
The novel methods introduced in this paper accompanied by detailed descriptions (Sec. 3 and their implementations are provided at http:///itubb.oo//XX///22222 12
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work of X. Hu, Z. Duan, and B. Chen was supported in part by the National Natural Science Foundation of China under Grant U21B2006; in part by Shaanxi Youth Innovation Team Project: in part by the Fundamental Research Funds for the Central Universities QTZX23037 and QTZX22160; and in part by the 111 Project under Grant B B18039.
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Alexander Amini, Wilko Schwarting, Ava Soleimany, and Danicla Rus. Deep evidential regression. Advances in neural information processing systems, 33:14927-149973 2020.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete problems in ai safety. arXiv preprint arXiv: 160.6600000655,220
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. International conference on machine learning, pp. 1613-1622. PMLR, 2015.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, and Yoshua Ben- gio. A recurrent latent variable model for sequential data. Advances in neural information pro- cessing systems, 28, 2015.
Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortlesss bayesian deep learning. Advances in Neural Information Processing Systems, 34:20089-20103, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Shehzaad Dhuliawala, Mrinmaya Sachan, and Carl Allen. Variational classification. Trans. Mach. Learn. Res., 2024, 2023a. URI. httts: www.nttiscchooooooooo.o..cccctce wttpoo/oalr..rrorooosooooot
258740937
Shehzaad Dhuliawala, Mrinmaya Sachan, and Carl Allen. Variational classification. arXiv preprint arXiv:2305.10406 2023b.
Kien Do and Truyen Tran. Theory and evaluation metrics for learning disentangled representations. arXiv preprint arXiv: / 09...000...90 00960999...........9099990 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition scale. In 9th International Conference on Learning Representations (ICLR 2021), 2021. URL https:
//pp///eeeee.eeeeeee https::/openreview.nrt
Zhibin Duan, Dongsheng Wang, Bo Chen, Chaojie Wang, Wenchao Chen, Yewen Li, Jie Ren, and Mingyuan Zhou. Sawtooth factorial topic embeddings guided gamma belief network. In Interna- tional Conference on Machine Learning, pp. 2903-291 PMLR, 2021.
Zhibin Duan, Tiansheng Wen, Muyao Wang, Bo Chen, and Mingyuan Zhou. non-negative vae: the generalized gamma belief network. arXiv preprint arXiv:2408. 200.22888.33888 2024.
Xinjie Fan, Shujian Zhang, Korawat Tanwisuth, Xiaoning Qian, and Mingyuan Zhou. Contextual dropout: An efficient sample-dependent dropout module, In International Conference on Learn ing Representations, 2021.
net/ wiwtorummiidcct.o __aM_
Xiao Fu, Kejun Huang. and Nicholas D. Sidiropoulos. On identifiability of nonnegative ma- trix factorization. IEEE Signal Processing Letters, 25:328-332 332, 2017. URL https: //ttps
semant cscho

---

Published as a conference paper at ICLR 2025
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
Nicolas Gillis. Nonnegative matrix factorization. SIAM, 2020.
Nicolas Gillis and Róbert Rajkó. Partial identifiability for nonnegative matrix factorization, SIAM Journal on Matrix Analysis and Applications, 44(1):27-52, 2023.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Xizewen Han, Huangjie Zheng, and Mingyuan Zhou. Card: Classification and regression diffu- //pii www.sentiicccholrr..cooom
sion models. ArXiv, abs/2206.007755, 2022.
CoorpuID::2467700
Jeff Z.
HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self- supervised deep learning with spectral contrastive loss. In Neural Information Processing Sys- tems, 2021. URL httl:/ttp://atttt
semant cccholar wewear.rg//CCoo /tgg/oorprsssrr ID: 235367888
James Harrison, John Willes, and Jasper Snoek. Variational bayesian last layers. arXiv preprint arXiv:2404.11599 11599, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
770-778, 2016.
José Miguel Hernández-Lobato and Ryan Adams. Probabilistic backpropagation for scalable learn- ing of bayesian neural networks. In International conference on machine learning, pp. 1861- 1869. PMLR, 2015.
Patrik O Hoyer.
Non-negative matrix factorization with sparseness constraints. Journal of machine learning research, 5(9), 2004.
Kejun Huang, Nicholas D Sidiropoulos, and Ananthram Swami. Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition. IEEE Transactions on Signal Processing, 62(1):211-224 2013.
Taejong Joo, Uijung Chung, and Minji Seo. Being bayesian about categorical probability. In Interna- http://app..aaatteee
lional Conference on Machine Learning, 2020. URL
org/ CorpusID:21117191
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http:
7/ aaaaiiv.com org abs 1412,412220 6980
Diederik P Kingma and Max Welling. Stochastic gradient vb and the variational auto-encoder. In Second international conference on learning representations, ICLR, volume 19, pp. 121, 2014.
Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. Advances in neural information processing systems, 27, 2014.
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameteri- zation trick. Advances neural information processing systems, 28, 2015.
David Knowles. Stochastic gradient variational bayes for gamma approximating distributions. arXiv preprint arXiv: 4rXiv:1509.0/631, 2015.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just bit, fixes overconfidence in relu networks. In International conference on machine learning, pp. 5436- 5446. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2025
A Krizhevsky. Learning multiple layers of features from tiny images. Master's 's thesis, University of Tront, 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances neural information processing systems, 30, 2017.
Matthew L. Leavitt and Ari Morcos. Towards falsifiable interpretability research. arXiv preprint arXiv:2010.1200 12016, 2020.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436-44, 2015.
Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factor- ization. Nature, 401(6755):788-79 - 1999.
Chunyuan Li, Changyou Chen, David Carlson, and Lawrence Carin. Preconditioned stochastic gradient langevin dynamics for deep neural networks. Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Deep ensembling with no over- head for either training testing: The all-round blessings of dynamic sparsity. arXiv preprint arXiv:2106. 14568, 2021.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detec- tion. Advances in neural information processing systems, 33:21464 21:21464-21475, 2020.
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In International Conference on Machine Learning, pp. 2218-2227. PMLR, 2017.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018
Andrey Malinin, Sergey Chervontsev, Ivan Provilkov. and Mark Gales. Regression prior networks. arXiv preprint arXiv:2006. 1t0.0006..199990 11590, 2020.
Arshia Soltani Moakhar, Eugenia lofinova, and Dan Alistarh. Spade: Sparsity-guided debugging for deep neural networks. arXiv preprint 2rXiv:23 2023.
Gemma E Moran, Dhanya Sridhar, Yixin Wang, and David M Blei. Identifiable deep generative models via sparse decoding. arXiv preprint arXiv:21 10804, 2021.
Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmenta- tion. arXiv preprint arX1v:111.112099, 2018
Christian Naesseth, Francisco Ruiz, Scott Linderman, and David Blei. Reparameterization gradients through acceptance-rejeccion sampling algorithms. In Artificial Intelligence and Statistics, pp- 489-496 PMLR, 2017.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks. arXiv preprint arXiv: 1602.03666, 2016.
Sebastian W Ober and Carl E Rasmussen. Benchmarking the neural linear model for regression. In Second Symposium on Advances in Approximate Bayesian Inference, 2019.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11);e7, 2017.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. Xiv preprint arXiv: 2018.
13

---

Published as a conference paper at ICLR 2025
Jayneel Parekh, Sanjeel Parekh, Pavlo Mozharovskyi, Florence d' Alchh Buc, and Gaël Richard. Post-hoc interpretability for audio networks with nmf. Advances in Neural Information Processing Systems, 35:35270-35288 2022.
Listen to interpret:
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Francisco R Ruiz, Titsias RC AUEB, David Blei, et al. The generalized reparameterizatioo gradient. Advances in neural information processing systems, 29, 2016.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-522. 2015.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 2017.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. Advances in neural information processing systems, 31, 2018.
Jiaxin Shi, Shengyang Sun, and Jun Zhu. Kernel implicit variational inference. arXiv preprint arXiv:1705. 101 19, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015.
Marcin Tomezak, Siddharth Swaroop, Andrew Foong, and Richard Turner. Collapsed variational bounds for bayesian neural networks. Advances in Neural Information Processing Systems, 34: 25412-25426, 2021.
Yifei Wang, Qi Zhang, Yaoyu Guo, and Yisen Wang. Non-negative contrastive learning. In The Twelfth International Conference on Learning Representations, 2024.
Joe Watson, Jihao Andreas Lin, Pascal Klink, Joni Pajarinen, and Jan Peters. Latent derivative bayesian last layer networks. In International Conference on Artificial Intelligence and Statistics, pp. 1198-1206. PMLR, 2021.
Noah Weber, Janez Stare, Arpit Mittal, Roi Blanco, and Lluís Mårquez. Optimizing over a bayesian last layer. In NeurlPS workshop Bayesian Deep Learning. 2018.
Ronald J Williams. Simple statistical crudient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229-221 -269992561111 1992.
Eric Wong, Shibani Santurkar, and Aleksander Madry. Leveraging sparse linear layers for de- buggable deep networks. In International Conference on Machine Learning, pp. 11205-11216. PMLR, 2021.
Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506. 06579, 2015.
Byron M Yu, John P Cunningham, Gopal Santhanam, Stephen Ryu, Krishna V Shenoy, and Ma- neesh Sahani. maussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. Advances in neural information processing systems, 21, 2008
Hao Zhang, Bo Chen, Dandan Guo, and Mingyuan Zhou. Whai: Weibull hybrid autoencoding inference for deep topic modeling. arXiv preprint 103.v.180..01328, 2018.
14

---

Published as a conference paper at ICLR 2025
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452-1485 2017.
Mingyuan Zhou, Lauren Hannah, David Dunson, and Lawrence Carin. Beta-negative binomial pro- cess and Poisson factor analysis. In Artificial Intelligence and Statistics, pp. 1462-1471, PMLR, 2012.
Mingyuan Zhou, Yulai Cong, and Bo Chen. The Poisson gamma belief network, Advances Neural Information Processing Systems, 28, 2015.
15

---

Published as a conference paper at ICLR 2025
A
APPENDIX / SUPPLEMENTAL MATERIAL
A.1 EXPERIMENTAL SETTING
All experiments are conducted Linux servers equipped with 32 AMD EPYC 7302 16-Core Pro- cessors and 2 NVIDIA 3090 GPUs. Models are implemented in PyTorch version 1.12.1, scikit-learn version 1.0.2 and Python 3.7. The CIFAR-IO, CIFAR-IOO, and ImageNet-IK datasets we used are all publicly available standard datasets. As for Places-10, it is a subset of Places365 (Zhou et al.) 2017] containing the classes "airport terminal", "boat deck", "bridge". "butcher's shop", "chureh- outdoor", "hotel room", "laundromat", "river", "ski slope" and "volcano". We chose this dataset to ensure fair comparison with DebuggableNetworks (Wong et al., 2021), as they also selected this subset.
Training from scratch setup For ResNet-18 on CIFAR-IO and CIFAR-100, we set the batch size to 128, learning rate to 0.1, training epochs to 150, and weight decay to Se-4. For ResNet-50 on ImageNet-Ik, 1k, we set the batch size to 256, weight decay to le-4, epochs to 200, and learning rate to 0.1. For the calculation of PAvPU, we sampled 20 times to obtain 20 different logits, and then selected the top-2 classes based on their mean logits, performing two-sample t-test to obtain the p-value. Following the approach used by CARD and MC Dropout models, we set the p-value threshold to 0.05 to determine whether sample is classified as certain.
Fine-tuning setup For Places-10, 10, we set the learning rate to 0.1, batch size to 128, and epochs to 100. For ImageNet-Ik, Ik, CIFAR-10 10, and CIFAR-100, we set the learning rate to 0.001 and epochs to 200.
Sparsity Vs Accuracy in Sec. 5.1.2 We follow the default settings of (Wong et al. 2021) when running the Debuggable baselines, and run BNDL using the settings from the fine-tuning setup. It is worth mentioning that in Debuggable Networks, the sparsity of the decision layer is controlled via the elastic net, which adjusts the sparsity of decision layer weights through the regularization path, this kind of parameter norm often leads to performance degradation. In contrast, BNDL increases the sparsity of the decision layer by using a gamma distribution as the prior. Additionally, we control the sparsity of the weights by applying an activation function to the weights, w' ReLU((w a)) where predefined constant set as a hyperparameter.
Visualization tool (LIME) Traditionally, LIME is used to obtain -pstance-specific explana- tions i.e., to identify the super-pixels in a given test image that are most responsible for the model's prediction. our setting, we follow this intuition and use the following two step-procedure obtain LIME-based feature interpretations: (i) We randomly select image category and then randomly choose K images from that category. For each image, we identify the feature with the maximum activation for visualization. Among them, our definition of maximum activation is that select the maximum weight of the category index corresponding to the image in Φ and select the feature neuron multiplied by this weight. (ii) Run LIME on each of these examples identify relevant super-pixels. At a high level, this involves performing linear regression to map image super-pixels to the (normalized) activation of the deep feature (rather than the probability of specific class as is typical).
A.2 THEORETICAL GUARANTEES FOR BNDL
Problem Formulation As described in Sec. 3.3 BNDL can be viewed as a Non-negative Matrix Factorization (NMF) problem. From this perspective, its objective function PY ( 0, X)) can be further reformulated as:
min 20 200
||Y
0Φ 2
(15)
This property enables NMF to accurately recover the ground truth factors that generated the data. Following the definition in Gillis & Rajkó 2023), we first define an exact NMF (that is, an errorless reconstruction) as follows:
Definition 1. (Excat NMF of size r) Given a nonnegative matrix E RR,,, the decomposition 0$ where 0 E Rex and & E Brxn is exact NMF Y of size ifY 0Φ.
16

---

Published as a conference paper at ICLR 2025
The formally defined identifiability of an Exact NMF is as follows:
Definition 2. (Identifiability of Exact NMF) The Exact NMF of 0 D. of size r is identifiable if and only if for any other Exact NMF of Y 0 of size v, there exists a permutation matrix E {0, 1 and nonsingular diagonal scaling matrix D such that:
0 == 0-IIDD and D
(16)
Intuitively, Definition 2 indicates that all columns of 0 and + must be identifiable. Achieving this typically requires very stringent conditions, such as the requirement for both 0 and + to satisfy the so-called sufficiently scattered condition (SSC) (Huang et al. 2013). Therefore, we concentrate on the partial identifiability of BNDL, which similarly ensures the identifiabiliyy and uniqueness of a subset of columns of 0 and & under more relaxed conditions.
Partially Identifiable Features To demonstrate that BNDL is partially identifiable, we first present the definition of partial identifiability in exact NMF.
Definition 3. (Purtial identifiability in Exact NMF) Let Y = 0. be an exact NMF of of size r. The k-th column of 0, identifiable if and only if for any other Exact NMF of Y = 0D of size r, there exits on index set j and scalar a > 0 such that:
((:,)) (:.k;,k)
(17)
Similarly, can define the identifiability of the kth column of T. using symmetry.
Previous work Gillis & Rajkó 2023) has shown that Definition can hold under two relatively relaxed assumptions.
Proposition 2. Giills & Rajkō 2023) Let Y = 0. where O. E IRT and 4. E Rrxa with rank(Y) r.. Without loss of generalit, assume Y, t. and O are column stochastic. The k-th column of 0. is identifiable if it satisfies the following two conditions:
(Selective Window) There exists a row of 2., say the j-th, such that 2.(j.)) aeti)) for a
(Sparsity Constrain) There exists subset ofr columns Y,, namely (i, such that rank(Y J)) = = and for all j E T:
Fo. (O. n Fo. (R(:.)) =
(18)
which means that the minimal face containing the k-th column 0. does not intersect with the minimal faces containing the columns of Y J).
For the Selective Window assumption: Intuitively, this assumption means that the column in 0 (latent class) corresponding to the (i.;) appears uniquely in the dataset, which is reasonable in many applications (Giliss 2020). E.g., in classification tasks (Wang al. 2024), the authors achieve full identifiability by positing that each latent class has a unique sample. In the context of selective window assumption,
only need to assume the presence of a single latent class with a unique sample to satisfy the selective window assumption, which makes it more feasible and easier to achieve. For the Sparsity Constrain: This condition implies that the k-th column of &. contains at least r - 1 entries equal to zero, namely, (= k) = 0. Due to the use of gamma prior and a ReLU function in & within BNDL, as shown in 5 and rrespectively, & is enforced be sparse during the training process. Additionally, in Sec. 5.1.1 we demonstrate the high sparsity of BNDL, for instance, the 1-sparsity of decision weights for ImageNet only 0.04. Only a small portion of the weights have a decisive impact the final results, indicating that BNDL satisfies the sparsity constraint.
In summary, our theory demonstrates that BNDL satisfies the assumptions outlined in 2 facilitating the partial identifiability of the learned features.
A.3 ADDITIONAL EXPERIMENTAL RESULTS
Uncertainty Evaluation We provide additional ablation study results on Fig. 5 with experimental settings consistent with Sec. 5.1.I For plotting the curves, we used B-spline interpolation to gener- ate smooth curves, setting k = (cubic spline). The line charts illustrate clear negative correlation
17

---

Published as a conference paper at ICLR 2025
between uncertainty and accuracy: higher uncertainty corresponds to lower accuracy. This suggests that the model provides reliable uncer- tainty estimates, helping to avoid potential misclassifications.
Taa Acouracc (10
Aorurnnr (80
Fent PestAnnrery
(a) ResNet-18 Cifar-10
(b) ResNet-18 Cifar- 100
(c) VIT Imagenet- Ik
Figure 5: Uncertainty Vs Test Acc Curve on other datasets and model.
Table 3: Feature score Cifar-100, where @k denotes the top-k dimensions. Values scaled by 10a
Cifar-100
SEPIN
SEPIN@ 10
SEPIN@ 100
SEPIN@ 1000
SEPIN@al
ResNet18
3.17 0.06
2.40 0.03
1.79 0.03
1.21 0.01
1.21 + 0.01
BM
3.19
0.02
1.17
1.28 0.02
1.24 + 0.01
ResNet18-Ne 8-NMF
3.21 =
2.95 +
1.87 0.03
1.33 + 0.02
1.23 + 0.03
BNDL
3.91 0.06
2.77 0.03
1.69 0.03
1.69 0.02
Disentangled Measurement Disentanglement Result on BNDL, BM, ResNet-NMF and ResNet18. Among them, ResNet-NMF represents framework where non-negativity constraints applied to both the weights of the decision layer and the input features, transforming the problem into one of non-negative matrix factorization. It can be observed that BNDL consistently achieves the best disentanglement metrics, aligning with the conclusions drawn in Section 5.2 of the paper. This suggests that BNDL has successfully learned more identifiable features through the constraint of sparsity.
Table 4: Accuracy and weight sparsity results of Cifar-100, where ResNet-NMF applied non- negativity constraints to both the input features and weights of decision layer.
Cifar-100
Accurarcy
1-Sparsity
ResNet18
74.62 0.23
0,97 = 0.02
ResNet18-NMM
76.73 0.16
0.23 0.01
ResNetl8-BNee
79.82 0.13
0.12 = 0.01
Additional Discussion on the relation of BNDL and Non-negative Matrix Factorization We now provide a more detailed explanation of the relationship between NMF and interpretability. Specifically. the non-negativity constraint in NMF ensures that the decomposition of the data ma- trix can be interpreted as an "additive combination" rather than complex mathematical operation involving cancellations between positive and negative terms. This characteristic aligns more closely with how humans naturally understand patterns real-world data. Therefore, both BNDL and Wang ctt al. (2024) leverage the principles NMF by modeling the final layer of the network as an NMF problem, employing non-negativity constraints to enhance interpretability.
However, there remain key differences between BNDL and Wang et al. (2024), which include the following: 1) NCL remains a point-estimation model that enforees negaaiivtt constraints by applying a ReLU activation function to the features. In contrast, BNDL probabilistically models the features non-negative distributions, which enables uncertainty estimation something that NCL.
18

---

Published as a conference paper at ICLR 2025
does not accommodate; 2) Although NMF naturally provides some degree of disentanglement, it is often insufficient for handling complex data (Hoyer] 2004). necessitating additional constraints to improve this effect. Unlike NCL, which does not introduce such extra constraints, BNDL applics gamma prior to the factors in matrix decomposition, further enhancing the sparsity and non-linearity of these factors. This additional constraint strengthens both the disentanglement and interpretability of the model.
Regarding (Duan al. 2024). it also recognizes the additive property introduced by non-negativity, which aids in the disentanglement of the network. It leverages the non-negativity and sparsity of the gamma distribution to design a Variational Autoencoder (VAE) generative model, achieving better disentanglement performance compared to Gaussian- VAE. In general, while BNDL shares some similar tools with (Wang et al. 2024) and (Duan al. 2024). there are notable differences in their network designs and architectures, as well as the distinct objectives they serve in different tasks.
we incorporated NMF into a supervised learning framework and conducted experiments on CIFAR- 100. Consistent with the experimental settings in the paper, ResNet-NMF adopts the ResNet18 baseline model. The experiments were performed on an NVIDIA 3090 GPU. We ran five exper- iments with different random seeds and computed the mean and variance of the results. The ex- perimental outcomes are presented the Table 3 and From the experiments on accuracy and disentanglement, the following observations be made: (1) Incorporating NMF into the network enhances its performance. By introducing non-negativity constraints on both weights and features, ResNet-NMF achieves more sparse weights and improved disentanglement performance. (2) BNDL consistently achieves the best results on the CIFAR-IOS dataset. This can be attributed to its prob- abilistic modeling and the additional sparsity and nonlinear constraints introduced by the gamma prior. These factors lead to greater weight sparsity and further improved disentanglement, aligning with our discussion.
We visualized the feature representation O of BNDL and the baseline model (ResNet-50) for the same images in ImageNet-Ik, as illustrated in Figures 8 to I2 Specifically, selected the feature A with the highest activation for each image and applied the LIME method using the top-10 super-pixels for visualization, in line with the approach used in Fig. 2 We have added comparative visualization results of BNDL and several models on ImageNet- Ik, including the uncertainty estimation model BM (Joo et al. 2020) and the sparse decision-layer model Debuggable Networks (Wong et al., (021), the results are shown in Fig. 6 The top row of Figures shows the true categories of the corresponding images, the second row presents our visualization results, and the third row displays the visualization results of the baseline model. Overall, the visualization results of BNDL are more semantically meaningful compared to those of ResNet-50.
Additional Visualization Results
We also included the results of post-hoc explanations using GradCAM(Selvann et al.) 2017). which is based on the concept of Class Activation Mapping. It aims to generate heatmaps by exploiting the relationship between the feature maps of specific layers in the neural network and the final prediction of the classification task, visually highlighting the regions that contribute the most to the prediction of particular class. In our experiment, we used the most activated class for each image to generate explanations, with the results shown in Fig. 7 It can be observed that BNDL tends to generate more focused heatmaps. For example, in the first image, BNDL only focuses on the region of the dog, while ResNet50 also attends to the surrounding ground. These results further demonstrate that BNDL is inclined to generate more disentangled features.
19

---

Published as a conference paper at ICLR 2025
Argiope aurantia
Hay
Lampshade
Lorikeet
Water ouzel
Egyptian cat
ResNet
Debuggable
BM
BNDL
Figure 6: The LIME visualization results for BNDL RResNet-50 Debuggable Networks and BM, focusing on the largest 0 for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels. This suggests that BNDL has learned more identifiable features.
Class Label: Saint Bernard
ResNet 50
ResNet 50 BNDL
Figure 7: We applied GradCAM to visualize ResNet50 and BNDL, and the corresponding heatmaps are shown above. The ground truth label for the visualized image is "Saint Bernard." It can be observed that the BNDL visualization more focused, while ResNet exhibits multifaceted behavior. For instance, in the fourth image, ResNet attends to both the person and the dog, whereas BNDL is more disentangled, focusing solely on the dog.
20

---

Published as a conference paper at ICLR 2025
Arctic fox, white fox, Alopex lagopus
ResNet 50 BNDL
ResNet 50
Figure 8: The LIME visualization results for BNDL and ResNet-50, focusing on the largest for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels compared to ResNet-50. This suggests that BNDL has learned more identifiable features.
Egyptian cat
ResNet BNDL
ResNet
Figure 9: The LIME visualization results for BNDL and ResNet-50, focusing on the largest 0 for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels compared to ResNet-50. This suggests that BNDL has learned more identifiable features.
21

---

Published as a conference paper at ICLR 2025
loupe, jeweler's loupe
ResNet 50 BNDL
ResNet
Figure 10: The LIME visualization results for BNDL and ResNet-50, focusing on the largest for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels compared to ResNet-50. This suggests that BNDL has learned more identifiable features.
Redbone
ResNet 50 BNDL
ResNet 50
Figure 11: The LIME visualization results for BNDL and ResNet-50, focusing the largest 0 for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels compared ResNet-50. This suggests that BNDL has learned more identifiable features.
22

---

Published as a conference paper at ICLR 2025
trolleybus, trolley coach, trackless trolley
ResNet 50 BNDL
ResNet 50
Figure 12: The LIME visualization results for BNDL and ResNet-50, focusing on the largest 0 for each image, demonstrate that BNDL's feature visualization aligns more closely with the semantic meaning of the true labels compared ResNet-50. This suggests that BNDL has learned more identifiable features.
23


Reviews:
[{"review_id": "WZay2ZMonE", "reviewer": "Reviewer_Wv5o", "summary": "The manuscript introduces a Bayesian Nonnegative Decision Layer (BNDL) for deep neural network classifiers, with the intent of reformulating them as a factor analysis. This is shown to enhance the interpretability and uncertainty-estimation capabilities of the networks, at least on the examined datasets.", "questions": "- How do we know the distinction between epistemic and aleatoric uncertainty? The epistemic uncertainty should go to zero for large data limit. Is this the case in the present modeling? \n- In Sec. \"uncertainty evaluation metric\", how are the various n_ac,  n_au, n_ic, n_iu actually computed?\n- Eq. 9: what is the difference between f_NNand f_lambda?\n- in section 3.2, the switch from Gaussian to Gamma distribution is unclear. Is the Gaussian distribution used in this work? It seems not but the sentence \"Both θ and Φ are sampled from a Gaussian distribution\" points otherwise. \n- in sparsity measurement a threshold of 10^-5 is defined for the weights. Shouldn't this come from the analysis of the distribution of weights, rather than just providing a number?\n-in table 1, it seems the application of BNDL to ResNet introduces a significantly better improvement than when applied to ViT. Can the reason of this be understood? \n\nTypos:\n- row 178, \"uncertainty-refers\" should not have a hyphen\n- 282 Killback–Leibler \n- 345 \"descirbed\"\n- 355 \"we uses\" \n- 371 \"Perforamce\"", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The idea is rigorous and the implementation introduces a minimal overhead over an existing network.", "weaknesses": "The presentation should be improved, see questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kLzXImaBcn", "reviewer": "Reviewer_4YUN", "summary": "The authors focus on improving the interpretability of deterministic neural net, by inserting a probabilistic layer that provides a non-negative factorization for an interpretable classification.\nThey prove (partial) identifiability and evaluate the method on several image classification benchmarks.", "questions": "- Q1: The authors provide a theoretical complexity analysis. What is the practical runtime increase compared to a simple fully-connected layer?\n- Q2: Can the authors provide greater detail on their relatio nto Wang et al. (2024)? There is a strong relation in the method (NMF) and aim to improve interpretability, yet within this paper it is only ever mentioned in passing without being fully introduced nor compared against. The same holds, e.g., for Duan et al. (2024).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The method is generic enough to be added to an arbitrary deep architecture\n- It performs well under varying experimental settings and architectures and can keep/improve upon its non-interpretable counterpart while greatly improving interpretability  \n\n\nOne caveat that should be noted is that I am not too familiar with the current state of the art in interpretability research. As presented, the results look significant, but they might not be.", "weaknesses": "- The experiments are limited to a small set of interpretability baselines. \n- Sec 3.1 \"We first adopt a Bayesian perspective to re-examine the DNNs\". This framing is rather crude. The vague fact that one could interpret the input to a softmax as a delta distribution over a latent variable alone is not enough to call something Bayesian. A Bayesian approach requires a well-specified prior combined with a posterior inference. Simply making a model (indirectly) probabilistic is not Bayesian.\n- l218 \"it lacks reparameterization and cannot be optimized\" \nYou can always rely on what is often known as the REINFORCE approach, i.e., $\\nabla_a E_{x\\sim q_a(x)}[f(x)] = E_{x \\sim q_a(x)}[f(x)\\nabla_a \\log q_a(x)]$. However, you usually don't want to do this as it will punish your gradients with a huge variance, which is why your proposal is much more stable and sane. But as long as you have a density you could in theory do it. \n- (12) the left hand side should be p(Y|X), as you marginalize on the rhs over $\\theta$ and $\\Phi$\n- Given the Bayesian framing of the paper a short discussion or mention of what is known as last-layer BNNs is missing. These combine a deterministic network trunk with a Bayesian inference over the last layer of a neural net. See, e.g., the references in Harrison et al. (2024). (This is not necessarily the best reference for this research direction that has been growing recently, but one whose references can serve as guidance for a more generic reference.)  \n- In a similar direction goes the field of evidential deep learning also known as prior networks, where a prior to the last layer is inserted in a different way. See, e.g., Sensoy et al. (2018) for classification or Amini et al. (2020) and Malinin et al. (2020) for regression. Both research directions, i.e., EDL and LL-BNNs have a different aim than the authors' proposal but rely on similar mechanics.\n- Regarding overconfidence in l167, I would have expected a reference to the first main study in that direction by Guo et al. (2017). (At least to my knowledge.)\n- In Thm 1 $e_{(k)}$ is not introduced\n\n_____\n_Amini et al., Deep Evidential Regression (2020)_  \n_Guo et al., On Calibration of Modern Neural Networks (2017)_    \n_Harrison et al., Variational Bayesian Last Layers (2024)_    \n_Malinin et al., Regression prior networks (2020)_    \n_Sensoy et al., Evidential Deep Learning to Quantify Classification Uncertainty (2018)_    \n\n\n\n\n\n### Typos\nThe paper contains a lot of typos and missing articles, which should be fixed in a thorough proofreading round. A subset of these are\n- l81 Furthermore, we provide\n- l105,l107 (and maybe others) the citation style is broken use \\citep and \\citet correctly, please follow the style guide\n- most equations, e.g., (1), (2) lack proper punctuation\n- l206 $\\mathbb{R}_+$\n- l241 \"where $h_j$ is an extracted feature\n- l272 of the log-likelihood\n- l345 are described in\n- l440 misclassification, e.g., in the\n- A lot of references are broken, e.g., Dosovitskiy et al. (2020) is a published paper, so is Kingma & Ba's Adam etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bGdRUUK2Dl", "reviewer": "Reviewer_hX73", "summary": "The authors propose a Bayesian neural network where the final layer is modeled as a non-negative matrix factorization (NMF), i.e. $y \\sim \\Phi\\Theta$, the motivation being that such a model would provide both predictive uncertainty estimates (because we learn a posterior distribution) and interpretability (because we learn a sparse factorization for $y$). Both $\\Phi$ and $\\Theta$ are modeled as Gamma distributions, and approximated variationally using the Weibull distribution. The authors evaluate their model (on accuracy, uncertainty, and sparsity) on CIFAR and ImageNet.", "questions": "The authors should respond to my questions in the Weaknesses section. I have no further questions, although I want to note that there are a fair number of typos in the paper that the authors should clean up.\n\nE.g. I believe the first term inside the integral of Eq (4) should be $p(y|\\Phi, \\Theta)$?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The methodology and description of the model and inference process is soundly written. Modeling as the factorization matrices as Gamma distributions should, in theory, encourage sparsity. The variational inference process that is described makes sense. \n\n2. Section 4 is an interesting addition to the paper, which describes how the factorization matrices are (partially) identifiable under certain assumptions, and how the author's model satisfies such criteria.\n\n3. I appreciate the sanity testing for uncertainty/accuracy correlation in Section 5.1.1.", "weaknesses": "My main criticisms would relate to the experimental results:\n\n1. It is not clear to me why it is necessary to compare to non-Bayesian/point-estimate models, considering that the goal of the paper is to provide better uncertainty estimates. As such, I am not sure the ViT results are especially meaningful, i.e. it is unclear to me what I should be comparing ViT-BNDL to.\n\n2. From Table 1, the PAvPU numbers for the ResNet model do not seem to be a huge improvement over the competing methods, especially the recent approaches (BM and CARD). \n\n3. Is there a reason why sparsity values are not shown for the competing approaches in Section 5.1.2?\n\n4. It is not clear to me that the interpretability evaluation metric in Section 5.2 is correct or useful. Specifically, why is unsupervised disentanglement important for ImageNet and CIFAR? Disentanglement does not imply interpretability, and disentangled features are not necessarily the correct ones to learn either (e.g. a spurious feature will be disentangled from a salient feature, but it doesn't mean we want to learn the former).\n\n5. Relatedly, why do we not compare to competing approaches in Section 5.2? Would the authors be able to report the metric in Table 2 and the visualizations in Figure 4 for the models that BNDL was compared to earlier?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OpnjSMEICx", "reviewer": "Reviewer_dpRV", "summary": "The manuscript suggests using Bayesian non-negative decision layer for improving model's uncertainty evaluation and sparsity (disentanglement), with no (statistically significant) loss of accuracy.", "questions": "The results on both uncertainty and disentanglement use one metric only each. Would it be possible include further metrics?\n\nMore verbal details on exactly how all experiments were performed would be appreciated.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Suggested BDNL seems advantageous for uncertainty evaluation and disentangled representation learning.\n\nThe authors try to perform theoretical analysis of their method.\n\nFor most paragraphs ZeroGPT score was 0%, for some 4% and 8%. Thus rather human-written.", "weaknesses": "Literature overview: why you do not cite works on DNNs and non-negative factor analysis in the interpretability framework, these are probably the closest works to your manuscript and constitute the core of your work? E.g.:\nhttps://proceedings.neurips.cc/paper_files/paper/2022/hash/e53280d73dd5389e820f4a6250365b0e-Abstract-Conference.html\n\nTheorem 1 is not a theorem (please check any statistical/ML literature, like AoS or NeurIPS for what is a theorem), neither is its \"proof\" is a proof, this is just a discussion. I would suggest you change the presentation.\n\nThe improvement of the performance does not seem to be significant at all, which is Ok, but the PAvPU might still seem questionable. How many receptions have been performed, what are your p-values? How, e.g., these numbers in Table 1 with +/- in front were calculated?\n\nThere are typos in the manuscript (which means it is human-written).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)