Islands of Confidence: Robust Neural Network Classification with Uncertainty Quantification
Authors: Sibylle Hess
Tianjin Huang
Wouter Duivesteijn
Venue: Submitted to ICLR 2023 (2023)
Keywords: uncertainty quantification
neural collapse
deep learning

Abstract: We propose a Gaussian confidence measure and its optimization, for use in neural network classifiers. The measure comes with theoretical results, simultaneously resolving two pressing problems in NN classification: uncertainty quantification, and robustness. Existing research in uncertainty quantification mostly revolves around the confidence reflected in the input feature space. Instead, we focus on the learned representation of the network and analyze the confidence in the penultimate layer space. We formally prove that, independent of optimization-procedural effects, a set of centroids always exists such that softmax classifiers are nearest-centroid classifiers. Softmax confidence, however, does not reflect that the classification is based on nearest centroids: artificially inflated confidence is also given to out-of-distributions samples that are not near any centroid, but slightly less distant from one centroid than from the others. Our new confidence measure is centroid-based, and hence no longer suffers from the artificial confidence inflation of out-of-distribution samples. We also show that our proposed centroidal confidence measure is providing a robustness certificate against attacks. As such, it manages to reflect what the model doesn't know (as demanded by uncertainty quantification), and to resolve the issue of robustness of neural networks.

Under review as a conference paper at ICLR 2023
ISLANDS OF CONFIDENCE: ROBUST NEURAL NET- WORK CLASSIFICATION WITH UNCERTAINTY QUAN- TIFICATION
A Anonymous authors Paper under double-blind review
ABSTRACT
We propose Gaussian confidence measure and its optimization, for use in neural network classifiers. The measure comes with theoretical results, si- multancously resolving two pressing problems in Deep Neural Network clas- sification: uncertainty quantification, and robustness. Existing research in uncertainly quantification mostly revolves around the confidence reflected in the input feature space. Instead, we focus on the learned representation of the network and analyze the confidence in the penultimate layer space. procedural effects, a set of centroids always exists such that softmax classífiers are nearest- centroid classificrs. Softmax confidence, however, does not reflect that the classification is based on nearest centroids: artificially inflated confidence is also given to out-of-distrrbutions samples that are not near any centroid, but slightly less distant from one centroid than from the others. Our new confidence measure is ventroid-basede and hence longer suffers from the artificial confidence inflation of distrf-iisttiiutionn samples. We also show that proposed centroidal confidence measure is providing robustness certificate against attacks. As such, it manages to reffect what the model doesn't know (as demanded by uncertainty quantification), and resolve the issue of robustness of neural networks.
We formally prove that, independent of optimization cewpiiiatioo-roocduurr
1 INTRODUCTION
The last layer of state-of-the-art neural networks computes the final classifications by ap- proximation through the softmax function (Boltzmann, 1868). This function partitions the transformed input space into Voronoi calls, each of which encompasses a single class. Con- coptually, this is equivalent to putting number of centroids in this transformed space, and clustering the data points in the dataset by proximity to these controids through k-mcans. Several recent papers posed that exploring a relation between sofumax and k-means can be beneficial (Kilinc& & Uysal, 2018; Peng el al., 2018; Schilling et al., 2018).
The current state of scientific knowledge the relation between k-means and softmax is empirical. In this paper, we theoretically prove that softmax is a centroid-based classifier, and we derive a centroid-based robustness certificate. This certificate motivates the usage of a confidence mcasure", the Gauss confidence, which reflects the distance of obscrvations to their assigned centroids. Gauss confidence therefore expresses the uncertaintics of the model; moreover, it indicates the vulnerabiliiies to attacks.
We show that our Gauss networks can match (median absolute difference: 0.45 percentage points) the test accuracy of softmax networks, but at lower confidence (as desired); both outperform the competing DUQ networks (van Amersfoort has many classes. The lower confidence also results in Gauss networks being much less susceptible to adversarial attacks. Hence, the islands of confidence as illustrated in the rightmost plot of Figure reflect reality much better than the confidence landscapes of existing methods (cf. other two plots in Figure 1).
al., 2020) when the dataset.
'previously published informally (non-peer-revieeed)) in January 2020 (NN al., 2020)

---

Under review as a conference paper at ICLR 2023
Softmax networks
DUQ networks
Proposed Gauss networks
10
(
12
40.
08
-06
Confoent
04
10
02
00
Figure 1: Classification confidence plots of actual decision boundaries/ aareas of a = 2- dimensional penultimate layer space for three distinct Deep Neural Networks. The hand- written numbers indicate the MNIST class predictions in (J.....4).
RELATED WORK
Peng et al. (2018); Kiline & Uysal (2018) were among the first to observe that the penul- timate layer has surprising clustering properties. The phenomenon that samples gather around their centroid in the transformed feature space recently gained popularity under the term neural collapse (Kothapalli et al., 2022; Papyan et al., 2020). This motivates an adaptation of the confidence measure. Softmax confidence has the well-documented issue that it extrapolates with unjustified high confidence data. points which are very far from the training data (Gal, 2016): softmax is overconfident. The consequences are observable in the very confident classification of noisy, entirely randomly generated images (Nguyen et al., 2015), and the effect that softmax probabilities are badly calibrated (Guo et al., 2017). Figure illustrates that this effect is not. surprising: in the transformed feature space of the penultimate layer, points that do not lie directly on the decision boundaries are all confidently assigned to class. Softmax confidence has capability express what the model doesn't 't know.
2.1 QUANTIFYING UNCERTAINTIES OF DNNs
To remedy the softmar issuc, various methods have been proposed. Although networks with centroid-based confidence intuitively make sense, training them is major challenge. When using the softmax cross-entropy loss, pushing a transformed feature vector ((x) away from one class region means that it is pushed towards other class regions. Employing a troidal confidence, one could push a point away from all centroids. This introduces trivial global optima of common loss functions, which does not lead to well-performin classifier. To this end, Wen et al. (2016); Pang et al. (2019) minimize the distance of transformed feature points to given (fixed) centroids. Wan et al. (2018) employ centroidal cross en- tropy loss, which directly optimizes for a nearest centroid classifier, but which does not increase the density around reids. Likewise, Lebedev et al. (2018) define a confidence measure based on the normalized RBF kernel function. Hobbhahn et al. (2022); Mukhoti et al. (2021) propose a retraining of only the last layer to reflect uncertainties based on multivariate Gaussian representation of confidences. Closest to our approach are Determin- istic Uncertainty Classifiers (DUQs) (van Amersfoort al., 2020), which directly optimize for multivariate Gaussian confidence measure (cf. Figure 1). The authors employ binary cross-entropy loss, which is given for one-hot encoded labels Y* mxmxe for m samples and classes by
1 mc
(BCE(FDUQ. Y*)
VjA log (fDUQ(X) (x,)) + (1 Y5A) ) log (1 fDUq(x)))
j=1
fDUq(x); = exp (-(o(x) Z.N) T ( 2.+))

---

Under review as a conference paper at ICLR 2023
Here, fDUQ: RR + [0, 1] returns the prediction confidences, Z E R gathers the centroids by its columns and Σg Rdxd is scaled covariance matrix. BCE loss works well for data with a moderate number of classes. With increasing class complexity, DUQ models converge to the global optimum where all training data is infinitely far from all centroids (||E~'I + 0).
Related to methods that use RBF- or Gaussian mixture-based uncertainty mcasures are DUE (van Amersfoort et al., 2021) and SNGP (Liu et al., 2022), which train a network with a final Gaussian processes layer. Yet, also other distributions are suitable to reflect uncertainties. For example, prior networks (Malinin & Gales, 2018; 2019) reflect the training data domain via Dirichlet distribution.
2.2 ROBUSTNESS OF DEEP NEURAL NETWORKS (DNNs)
The existence of adversarial examples demonstrates that SOTA doep learning models have warped inner representations of classes properties. The fact that adding humanly indistinguishable noise changes the classification from bus to an ostrich is worrying. Con- sequently, a lot of research focuses performing and preventing these attacks. The result a circular development of attacks and defenses (Akhtar & Mian, 2018; Carlini et al., 2019; Athalye et al., 2018), which creates need for certifiable robustness. A certificate provides, c.g,, a lower bound on the magnitude of adversarial perturbation required to change the class. Existing cortificates rely bounds of the global Lipschitz constant, limiting the effect of any, arbitrarily large perturbation (Szegedy et al., 2014; Qian & Wegmann 2019; Gouk et al., 2018). Experimental and theoretical evaluations show that global Lipschitz bounds overly curtail the expressiveness of DNNs, yielding underfitting classifiers (Huster et al., 2018). The expressiveness of DNNs, their ability to approximate almost any func- tion (Tu et al., 2017), is an important property which distinguishes feature transformations learned by DNNs from popular alternatives. Hence, we require a control of the local Lip- schitz constant (restricting only the effect of small perturbations). Unfortunately, this is currently only possible for DNNs having one hidden layer (Hein & Andriushchenko, 2017).
Many uncertainty quantification methods regularize towards bi-Lipschitz functions. Bi- Lipschitz functions preserve distances in the output to degree, that is specified by the Lipschitz constant. This prevents that all data (also onto the same confidently assigned regions. DUQ uses gradient penalization, and DUE, SGNP, and DDU spectral normalization (Gouk et al., 2021). Unfortunately, gradient penalization is slow and it only bounds the local Lipschitz constant. for the data points it is optimized on. In comparison, spectral normalization is faster. However, applying spectral normalization such that it doesn't 't notably hurt the performance does not bound the Lipschitz constant enough to provide relevant robustness guarantees.
data) is mapped
Another way to increase robustness is the maximum margin approach. Enlarging the margin between classes in the transformed feature space should require larger perturbances push an example over the decision boundary (Liu et al., 2016; Pang et al., 2019). The success of this approach stlll depends on the Lipschitz constant of the feature transformation, requiring controlling the Lipschitz constant while maintaining expressiveness (Tsuzuku et al., 2018). With Gauss networks we show that a centroidal confidence measure provides a guarantee on the robustness which depends on both the Lipschitz constant and the margin.
3 WHY SOFTMAX IS ACTUALLY CENTROID-BASED CLASSIFIER
Consider a feedforward network, mapping points in n-dimensional space to a e-dimensional probability vector (with e the number of classes), computing predictions by the function
Jam (x) softmax (WT ((x) b).
Here, ((x) E Rd returns the output of the penultimate layer. The last layer is linear; its weights are represented by the matrix W IRdxe and bias b Re.

---

Under review as a conference paper at ICLR 2023
feature space
Input layer
Pemultimate Output
1.5
layer
layer
21
v(x)1
1
|
W11
softmax(W" 6(x) ) b)1
0.5
22
Way
0
23
0.5
1
1.5
2
W
softmax (aannxxWWTolos b).
transformed feature space
To.
Wdc
2
S
((x)
ZIZZ
d-dimensional transformed feature space (depiction for d=2)
Ha A)
Z
Z1
X
0
1.5
(a) A feed-forward neural network, computing a feature trams- formation & and the representation of classes in the transformed feature space by convex cones, induced by centroids Z.....,, Z.e.
(b) Gauss confidence certi- fies robustness: minimum distance must be bridged to change the predicted class.
Figure 2: The role of Gauss confidence for Deep Neural Networks.
3-1 SOFTMAX AND INHERENT CLUSTERING PROPERTIES
Let D= = v1)............................. (Xm. bm)} denote our training data of data points x, R" and their corresponding class labels yj (1. We denole by ||-I| the vector 12-nom or the matrix Frobenius norm and by |-| the vector 4-norm. Figure 2a illustrates the result of Theorem 1, showing that softmax-basee
classifiers are nearest centroid classifiers.
Theorem Let W E Raxe be the matriz of weights between the penultimate layer and the output layer. Let TR" IRd be the DNN output of the penullimate layer. If the network computes predictions over the (soft)max function with bias term b such that = arg max; Φ(x) W. + by and if W has at least rank of r Z c, then there exist a class centroids Z. E IRd such that every point x is assigned to the class having the nearest centroid:
y = arg min |lo(x) 2.
Proof. We show that for any dataset and network there cxists a set of centroids such that the classification does not change when classifying according to the nearest centroid.
We gather the outputs of the penultimate layer in the matrix reflecting the transformed feature vectors rows, or conversely, as columns in DJ:
DJ (Ø(x1)
((xm)) E Rdxm
We define Z = V T where v E Rd and 1, E [1] is a constant one vector. The (soft) Jma classification of all data points in D is then given by the one-hot encoded matrix Y E ma that optimizes the objective
argmax (W DT b1))) arg Din]| YZT 112 tr((211) 2 TT)
Full derivation this equivalence is given in Appendix A. The matrix E d indicates set of centroids by its columns. The first term of Equation (1) is minimized if Y assigns

---

Under review as a conference paper at ICLR 2023
the class with the closest centroid to each data point in D. Hence, if we show that there exists a vector v' E R such that the second term of Equation (1) is equal to zero (given D and W) then we have shown what wanted to prove. Since |yy| 1 (every point is assigned to exactly one class), the matrix y is diagonal matrix, having the number of data points assigned to each class on the diagonal: y dia,g(Y.
Hence, the
trace term on the right of Equation (1) equals
(2b1, Z' Z) = Σ (2bg. || 2 2u T W. -Ivvll 1=1
(2)
Full derivation of this equivalence is given in Appendix B. We define the vector u e R such that U 6y- / [| 12 The right term of Equation (2) is constant for a vector v satisfying
uk = v W.. for < S c. That is, we need to solve the following equation for v:
u u= W v VEU V.
Since the rank of W is e (full column rank), this equation has a solution. is given by the SVD of W UEVT, where U Raxe is left orthogonal matrix (U = 1), E Rex is diagonal matrix having only positive values, and V Rexe is orthogonal matrix VV VV = 1). Setting v = =V--V u, this vector solves the equation.
The objective miny |D 1 is the k-means cluster assignment objective in matrix factorization form. Hence, class predictions correspond to Voronoi tesselation of Rd. Classification accuracy is unaffected by the distance of the points 6(x) to the centroid, as long as they are in the correct Voronoi cell. Softmax confidence is high for points maximizing the inner product o(x) L.k, where Z. is the center of the predicted class, since
softmax(W" (x)()) - b)n exp(o(x) W + bk)
exp(o(x)T Tv)
Li= exp(o(x) W., + by)
exp(o(xx v)
exp(o(x)] 2.k + bb))
softmax ((x) b)R- Σi cxp(o(x) Z. + by)
Hence, softmax confidence is high for points (xx) aligning with the direction of their class center Z. and having a large norm in the transformed feature space. However, the empirical obscrvation of neural collapse indicates that networks do in fact not map the training and lest data arbitrarily far away from their centroids but rather into their neighborhood. Thus, one could attach semantic meaning to the distance of transformed samples class roids.
3.2 A CENTROID-BASED ROBUSTNESS CERTIFICATE
In addition to the potential reflection of learned representations of a neural network, the distance of points to their centroid also determines the robustness of network, as the following result shows (illustrated in Figure 2b).
Thcorem 2. Let x E RR" be a data point with predicted class and let Z Rdxe be the centroid matriz. Assume & Rn + Rd is Lipschitz continuous with modulus Lg. Any distortion E R changing the prediction of point A to another class I + k has then minimuu size of
Z.
Z [xx + A) - Z [xxx)) - -Z.
| 2
Ls
Proof. Let x, A and Z be as described above. We derive from the triangle incquality and from the Lipschitz continuity the following inequality:
[ + A) Z+ < 0 + A) o(x)]l + [|o(x) - Z
L A + [xx(x) Z.
(3)
The triangle inequality also yields:
2| 7 S [| + -) Zill + 11 + A) 7.
Subtracting ||x(x + A) Z.Ill yields a lower bound on the distance | Zull, which we I
apply in Equation (3) to obtain the final bound the distortion A.

---

Under review as a conference paper at ICLR 2023
Theorem provides three explanations for the phenomenon that predictions of neural net- works can be flipped by small perturbations: 1) the Lipschitz modulus is large, 2) class centroids are close each other, and 3) point centroid. Case remains an open problem: this paper will not provide a manner to suitably control the Lipschitz constant. Case motivates maximum margin approach where the centroids are supposed to be as far away from each other as possible. Case 3 motivates novel confidence score, which reflects the distance to the centroid. In the desirable situation where the Lipschitz constant is small and centroids are far away from each other, small perturbations only result in prediction change at least one of the points x or + A is lar from its centroid. Hence, confidence score reflecting the distance lo the centroid is in alignment with the theoretical robustness guarantee of Theorem 2. Such a confidence score expresses not only the model's uncertainties, but also its vulnerability to attacks.
+ A is not mapped close to their class
4 GAUSS CONFIDENCE AND GAUSS NETWORKS
Theorem 2 motivates the direct optimization for good centroids which are in current networks only indirectly learned. To do so, we employ natural choice for a confidence measure which reflects the proximity to the cluster centroids via the Gaussian RBF--ernel function.
Definition 1. Given a function ø IR + Rd and a. centroid matrix E Rdxxe, we define predictions with Gauss confidence by the function fga (x), where for h (1....,c)
(x=(x;)) = <xpp(~>>l0(* - 11 E (0,
The parameter y R. determincs how close the transformed samples have to be to their centroid in order to gel a high confidence. The Gauss confidence is a special case of the multivariate Gaussian confidence (used, e.g., by DUQ networks) where the covariance matrix for class is equal to Σg = 1/2. This might. seem restrictive, but this choice has two advantages. First, the margin between classes is more easily controllable over the distance between centroids when using Gauss confidence. For example, we can see in Figure 1 that the centroids of class and 3 are comparatively far away, but due to the multivariate confidence, the margin between the confidently assigned areas is small. Second, the required storage for the covariance matrix in multivariate confidences increases vastly with the dimensionality of the penultimate layer space. For example, for ResNet-50 architecture, the covariance matrix has a dimensionality of 22488x x x 2048. We can also not decrease the required storage by employing low rank approximation of Σ-1, since this would introduce directions in which the confidence is always equal to one. In turn, the Gauss confidence requires only to store the c-dimensional vector 7.
4.1 TRAINING GAUSS NETWORKS
We replace softmax with the Gauss confidence. While the weights W of traditional net- works indirectly determine the class centroids Z, our network learns the centroids directly, represented by the matrix W = Z, connecting the penultimate with the last layer.
Based on the result of Theorem 2, we aim for robust networks which map well-classifabbl points close to the corresponding centroid and points which are difficult to classify further away from all centroids. In terms of clustering, we aim to minimize the atthin-class-scatiie while maximizing the between--lass-sattee
We propose novel loss function that is based on the Gauss confidence and achieves both: it optimizes for feature transformation that map samples close to their centroid and centroids far away from each other. To this end, we introduce a parameter vector å E (D.01.0.9]:
1 m
Iga (xa(X))))
Lga (fga; Y*)
log
log (Sea 6))
fxa(X; ExpL, fea (x
m
The product indicates here the element-wii wise (Hadamard) product. The left term of the loss resembles cross-entropy loss. We introduce here the vector to simulate higher confidences

---

Under review as a conference paper at ICLR 2023
Table Test accuracy (ace) and average confidence (conf) of MNIST, Cifar-10, Cifar-100 and ImageNet for softmax, softmax with adversarial training (+AT), DUQ and the proposed Gauss networks, using LeNct and ResNet (He al., 2015) architoctures.
MNIST
Cifar-10 ResNet18
Cifar-100 ResNet18 accf
ImageNet ResNet50 acct
LeNet
Network
acct
conf
acct
conf
conf
conf
Softmax
99.3%
0.99
94-4%
0.98
76.4%
0.86
75-7%
0.81
Softmax+AT
99.0%
0.99
79-7%
0.63
56.8%
0.50
DUQ
98.4%
0.87
94-3%
0.94
2.6%
0.35
Gauss
99-3%
0-93
94-3%
0.91
75.8%
0.61
73.8%
0.65
for the wrong classes during training. This pushes centroids further away from samples that do not belong to their class, and hence increases the margin. The second term resembles negative log likelihood and increases the sample density around the centroids. The inclusion of 8 forces the network to have higher sample density (and hence smaller confidence ball around the centroid) when is large, and hence the margin around that class is supposedly small. To optimize for small values of 6, we further employ a weight decay. We optimize the paramcters of the network, as well as the paramcters and 8, with SGD.
While the entsopy term pushes towards perfect training data classification, our pro- posed kga loss is more sensitive to the initialization than softmax models. We make use of the effect of neural collapse and our proven connection of softmax ner and nearest- centroid classifiers, and employ warm start: we train softmax network as usual for some epochs, then we compute the class centroids and determine the initial centroid matrix for the Gauss confidence. We then train the Gauss network for the remaining epochs with lga.
5 EXPERIMENTAL RESULTS
We evaluate the proposed Gauss networks with respect to robustness and a suitable reflection of classification confidences by the Gauss confidence. For this purpose, we compare popular network architectures with the refined Gauss variant on MNIST (Lecun et al., 1998), Cifar- 10 and -100 (Krizhevsky, 2009), and ImageNet (Russakovsky al., 2015). As competitors, we choose Softmax networks (with a linear classifier and softmax confidence), DUQ models, and adversarially trained (AT) models (Madry al., 2018). We also compare with respect to OOD detection with the method DDU, where we train a Gaussian mixture layer top of the Softmax model to obtain an additional uncertainty measure. Our Pytorch implementation with scripts for training and attacking is publicly available".
5-1 CLASSIFICATION PERFORMANCE
We evaluate the feasibility of our proposed loss function to attain similar test set accuracies as softmax models. Parameter settings of our experiments are given in Appendix C.
Table 1 summarizes the performance results. For every dataset, we report the test accuracy and the average confidence of the predictions. Gauss networks achieve oquivalent accuracies and similarly high confidences as softmax networks on MNIST and Cifar-1o. On Cifar- 100, we observe that. the accuracy drops bit but more notably, Gauss networks are less confident on these classes. While softmax networks are rather overconfident (the confidence higher than the accuracy), Gauss networks tend to be less confident. This effect recurs on ImageNet: Gauss networks lose a few percentage points of accuracy w.r.t. softmax networks, but the latter have a higher confidence than accuracy. DUQ networks can compete on the MNIST and Cifar-1o datasets, but its training of Cifar-10o failed: the BCE loss pushes here the embedded training samples away from all centroids. In addition, the employed gradient penalty of DUQ networks quadruples the runtime. Consequently, didn't 't optimize DUQ for ImageNet. Note that the runtime for one epoch is the same for Gauss and softmax
A7E7

---

Under review as a conference paper at ICLR 2023
Table 2: Results of Attacks on MNIST and Cifar-1o. We report the rate of successful attacks (rate) and the average confidence of the adversarial samples (conf) for each attack. The displayed attacks are Carlini & Wagner (C&W) (Carlini & Wagner, 2017), Fast Gradient Sign Attack (FGSM) (Goodfellow et al., 2014), Projected Gradient Descent (Madry et al., 2018) with 6x- (lxPGD) and with 1--norm (l2PGD). For all attacks we set 0.1.
C&W
FG)
(GD( conf.
12 (GD(o.1) conf.
Datasct
Network
rate]
conf.
ratel
conf.
rate]
rate]
Softmax
39%
0.81
12%
0.75
23%
0.80
1%
0.54
Softmax+AT
11%
0.83
3%
0.78
4%
0.83
0%
0.62
MNIST
DUQ
20%
0.69
38%
0.67
92%
0.91
0%
Gauss
2%
0.12
1%
0.41
20%
0.52
o%
0.00
Softmax
93%
0.84
76%
0.84
93%
1.00
40%
0.96
Cifar-10
Softmax+AT
81%
0.63
59%
0.44
78%
0.70
3%
0.36
DUQ
97%
0.73
0%
0.00
26%
0.67
44%
0.8g
Gauss
9%
0.44
43%
0.66
88%
0.72
21%
0.8g
nets. We observe that adversarial training comes at cost. Accuracy for AT on MNIST is comparable to its competitors, but drops by 14 and 20 percentage points on Cifar-10 and Cifar-1oo, respectively, when compared to the plain Softmax model.
5.2 ROBUSTNESS TO ATTACKS
We evaluate the robustness of models by means of four attack methods: Carlini & Wag- ner (C&W) (Carlini & Wagner, 2017), Fast Gradient Sign Attack (FGSM) (Goodfellow et al., 2014), Projected Gradient Descent (Madry al., 2018) with the too (GD) and l2 (LPGD) norms. For our attacks, we employ the advertorch library (Ding et al., 2019). One of the theoretical strengths of the Gauss and DUQ confidence the possibility to identify outliers. Since the softmax confidence of the predicted class is always larger than e: we also specify that a point is considered as an outlier if the confidence of the predicted class is smaller than d: if we have ten classes, then the outlier threshold is 0.1. We train all networks with the same number of epochs.
We investigate to which extent Gauss networks provide an inbuilt robustness (cf. Theorem 2) on the MNIST and Cifar-10 datasets. Table 2 reports the fraction of images for which the attacks lead to misclassification, and the average confidence of the network for these misclassifications. For both measures, lower is better. For all attacks on both datasets, the Gauss network outperforms the softmax network in terms of both attack success rate and confidence in the adversarial examples. Gauss networks are less or equally susceptible to C& W and FGGMM(o.) attacks than the adversarially trained network. Notably, the AT models are trained to defend against the foe PGD attack, hence it comes at no surprise that AT models are more robust to PGD attacks than Gauss networks. DUQ is sometimes more, sometimes less robust than the softmax model. On Cifar-10, DUQ is notably more robust to FGSM and foe PGD attacks. Here, the attacks seem to overshoot: decreasing the allowed perturbation size actually increases the attack success rate (cf. Appendix E). A qualitative analysis of the adversarial examples is given in Appendix D.
5-3 OOD DETECTION
Ideally, good confidence measures are able to reflect what the model knows, i.c., what has been trained on. This reflection of arcas of confidence can be used for OOD detec- tion. Figure 3 displays the Gauss confidence against three uncertainty measures on Fashion MNIST trained networks with MNIST being the OOD data (top) and on Cifar-10 trained models with SVHN (Netzer et al., 2011) being the OOD data (bottom). We train Fashion MNIST on the LeNet architecture and Cifar-10 ResNet18. Our competitors softmax with entropy uncertainty (high entropy indicates high uncertainty). DDU (Mukhoti al., 2021) using the log of the of the Gaussian mixture pdfs certainty measure, and
8

---

Under review as a conference paper at ICLR 2023
Softmax
DDU
DUQ
Gauss
-10
15
15
10
6
density
10
10
4
5
5
2
0
0
0
0
0.5
1.5
2
0
0.5
0
0.5
1
-1033
-00
30
30
20
3
density
20
20
2
10
10
0
0
0
0
0.5
1.5
1,000
0.5
0.5
entropy
log sum probs
conf fDUQ
conf fga
Fashion MNIST
MNIST (OOD)
Cifar-10
SVHN (OOD)
Figure 3: Histograms uncertainty measures and confidences for models trained on Fashion MNIST (top) and Cifar-1o (bottom), deployed also out-of-distribuutin data.
DUQ networks' confidence. We train the DDU softmax model as recommended with spec- tral normalization on the ResNet-18 architecture, using a soft Lipschitz bound of 3. The plots indicate that no measure delivers a perfect split between OOD and in-distribution data. However, we can observe a distinguishing characteristic of Gauss networks: using threshold of 0.5 confidence separates the majority of OOD samples from the majority of in-distribution samples. For all other uncertainty measures, the cut-off value that separates the distributions is less consistently evident.
6 CONCLUSIONS
We introduce a loss function to optimize for neural networks with Gauss confidence, cen- troidal confidence measure for Deep Neural Networks, instead of the standard softmax clas- sification and the recently proposed alternative of DUQ networks (van Amersfoort et al., 2020). We prove that softmax-based classifiers are nearest centroid classifiers (cf. Theo- rem 1). Softmax classification accuracy is typically insensitive to the distance of the points to its centroid: as long as a point lies in the correct Voronoi cell, softmax confident of its decision. We prove that
certify the robustness of the network through these dis- tances (cf. Theorem 2). We build on this centroid-based robustness certificate to define our Gauss confidence (ef. Definition 1). Its islands of confidence, illustrated in the rightmost plot of Figure 1, are in contrast to the wider confidence regions generated by the competitors.
Gauss networks can achieve comparable accuracies softmax networks (cf. Table 1), but are more robust against adversarial attacks (cf. Table 2). When the softmax networks enhanced with adversarial training, they are more robust. against PGD attacks than Gauss networks, but Gauss networks are more robust against C&W and FGSM attacks and achieve better accuracies. In some cases, DUQ networks match Gauss networks in their robustness (though Gauss networks are more robust on the Cifar-1o dataset), but DUQ networks cannot achieve the same accuracy as Gauss or softmax networks when the dataset. encompasses > 10 classes. In future work, we intend to improve Gauss networks by allowing for multiple centroids per class. This would allow for classes that are not one monolithic whole: they could consist of several parallel subsets, even hicrarchical concepts.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks on deep learning in computer vision: A survey. TEEE Access, 6:14110-1410,, 2018. ISSN 2169-3536. doi: 10.1109/
ACCESS.2018.. 2CCCESS.018..2073355
Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using a single deep deterministic neural network. Proceedings of the 37th International Conference on Machine Learning, TCML, 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedingg of Machine Leurning Research, pp. 9690 9700. PMLR, 2020. URL http:/ http:///poceeeiig...o mlr. http:eeedinn......prrsssssaaaaee press/v11g/vant www.aaaesffooroo...oooo htmll.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning, 1CML 2018, Stockholmssüüsson, Stockholm, Sweden, July 10-15, 2018, pp. 274 283, 2018.
Ludwig Boltzmann. Studien über das gleichgewicht der lebendigen kraft zwischen bewegten materiellen punkten. Wiener Berichte, 58:517-5660 560, 1868.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE symposium security and privacy (sp); pp. 39-57 57. IEEE, 2017.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dim- itris Tsipras, Ian J. Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. CoRR, abs/2902.06705. 2019. URL 06705.
Gavin Weiguang Ding, Luyu Wang, and Xiaomeng Jin. AdverTorch vo.1: An adversarial robustness toolbox based on pylorch. arXiv preprint orXiv:192..07623, 2019.
Yarin Gal. Uncertainty in deep learning section 1.5. University of Cambridge, 2016.
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint orXim:142..677 2014.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J. Cree. Regularisation of
neural networks by enforcing lipschitz continuity. CoRR, abs/18o..0n 2018.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110:393-415 2021.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern noural networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1321 1330, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual lcarning for image recognition. arXiv e-prints, art. EXXivi512.03336 Dec 2015-
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems, pp. 2266-2276, 2017.
Marius Hobbhahn, Agustinus Kristiadi, and Philipp Hennig. Fast predictive uncertainty for classification with bayesian deep networks. In Uncertainty in Artificial Intelligence, pp. 822 832. PMLR, 2022.
Todd Huster, Cho-Yu Jason Chiang, and Ritu Chadha. Limitations of the lipschitz constant as a defense against adversarial examples. In Joint. European conference on machine learning and knowledge discovery in databases (ECML PKDD), pp. 16-2g. Springer, 2018.
Ozsel Kiline and Ismail Uysal. Learning latent representations in neural networks for cluster- ing through pseudo supervision and graph- activity regularization. Proc. ICLR, 2018.
10

---

Under review as a conference paper at ICLR 2023
Vignesh Kothapalli, Ebrahim Rasromani, and Vasudev Awatramani. Neural collapse: A review on modelling principles and generalization. ArXiv, abs/2206.04041, 2022.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report (4), 7, University of Toronto, 2009.
Vadim Lebedev, Artem Babenko, and Victor Lempitsky. Impostor networks for fast fine- grained recognition. arXiv preprint arXit:1806.00911 05217 2018.
Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-band based learning applied document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Jeremiah Liu, Shreyas Padhy, J. Ren, Zi Lin, Yeming Wen, Ghassen Jerfel, Zachary Nado, Jasper Snock, Dustin Tran, and Balaji Lakshminarayanan, A simple approach to improve singlo-model deep uncertainty via distanco-awareness. ArXiu, abs/2205.00400 2022.
Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolutional neural networks. ICML, volume 2, pp. 7, 2016.
Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. The expressive power of neural networks: A view from the width. In Advances in neural information processing systems, pp. 2231-6239. 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant. to adversarial attacks. Tn International Conference on Learning Representations, 2018.
Andrey Malinin and Mark John Francis Gales. Predictive uncertainty estimation via prior networks. In Neural Information Processing Systems, 2018.
Andrey Malinin and Mark John Francis Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. In Newral Information Pro- cessing Systems, 2019.
Jishnu Mukhoti, Andreas Kirsch, Joost R. van Amersfoort, Philip Н. S. Torr, and Yarin Gal. Deep deterministic uncertainty: A simple baseline. volume abs/ abss/106.1es 210/6.106.2 2021.
Yuval Netzer, Tao Wang, Adam Coates, A. Bissacco, Bo Wu, and A. Ng, Reading digits in natural images with unsupervised feature learning. 2011.
Anh Nguyen, Jason Yosinski, and Jell Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427-430 2015.
NN, NN, and NN. Reference scrubbed for double-blind reviewing purposes. 2020.
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross-entropy loss for adversarial robustness. In International Conference on Learning Representations, 2019.
Vardan Papyan, X. Y. Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences of the United States of America 117:24652 - 24663, 2020.
Xi Peng, Joey Tianyi Zhou, and Hongyuan Zhu. k-meansnet: When k-means meets differ- entiable programming. CoRR, abs/1808.0772g 2018. URL 07292.
Haifeng Qian and Mark N. Wegman. L2-nonexpansive neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
11

---

Under review as a conference paper at ICLR 2023
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3)221-252, 2015-
Achim Schilling, Claus Metzner, Jonas Rietsch, Richard Gerum, Holger Schulze, and Patrick Krauss. How deep is deep moungh"-uuanifyingg class separability in the hidden layers of deep neural networks. arXiv preprint arX10:811.01153. 2018.
Christian Szegedy, Wojciech Zaremba, llya. Sutskever, Joan Bruna, Dumitru Erhan, lan J. Goodfellow, and Rob Fergus. Intriguing properties neural networks. In International Conference on Learning Representations (ICLR), 2014. URL http://arxiv atgg/bbb//1/22 6199.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: scalable certification of perturbation invariance for deep neural networks. In Advances in Neural Information Processing Systems, pp. 6541-6550, 2018.
Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. On feature collapse and deep kernel learning for single forward pass uncertainly. arXiv preprint arXio:2102.11400 2021.
Weitao Wan, Yuanyi Zhong, Tianpeng Li, and Jiansheng Chen. Rethinking feature distri- bution for loss functions in image classification. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 9117 9126, 2018.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European conference on computer vision, pp. 499- 515. Springer, 2016.
12

---

Under review as a conference paper at ICLR 2023
A DERIVATION OF EQUATION (1)
We employ the fact that the m-dimensionall constant one vector equal to 1m = Yles
because 1, 1y.1 Using this relation, obtain Equation (4).
arg max tr DJ m)))
Y
arg D tr(Yb1,
(4)
arg argmiin| D|| -tr(Y DJ) 2tr(1mm D tr
(5)
= arg. DI 2tr(Y 1.00 )DT) - 1r(2b1, rT)
(6)
arg DI 21 DJ) + ur(V ZY 2Y - -r(Y ZYT) 11(211
(7)
mig D YZ - + z == z))
(8)
In Equation (5) we change the maximization the minimization of the negative objective function times two. Furthermore, we add the constant terms D| and - trmmmmmm DT), that do not change the optimizer Y. Using again the fact that 1m Ylc. we can write the two terms in the middle of Equation (5) one in Equation (6). In Equation (7), we complete the square, arriving at the objective of Equation (8).
B DERIVATION OF EQUATION (2)
The diagonal elements of the matrix 261 - are given by 2bg + Z. Z.x, which yields Equation (9). We then insert the definition of Z. = W. + in Equation (10), expand the term and simplify Ei-lYal m, arriving at Equation (11).
Σ (261) TT 2) Yx
(2bk + Z.)(a))VVell
(9)
(2br (W.x + T))(W., =))l
(10)
26p + ||W, 20 W. + |10|
k=
(2b + [ 2 20 T W. + +11013mm
(11)
km
C EXPERIMENTAL SETUP
We employ for Gauss networks a warmup of 10 epochs on MNIST and Cifar-10, 30 epochs on Cifar-1oo, and 60 epochs on ImageNet. MNIST is trained for 60 epochs, while the other datasets are trained for 100 epochs (including the warmup). For softmax and Gauss networks we employ a similar learning rate schedule, and we use the advised learning rate schedule for DUQ networks (as provided for Fashion MNIST and Cifar-1o). We train the LeNet architecture with a learning rate of 0.1 for softmax nets and 0.02 for Gauss nets, which is decayed by factor of 10 at epoch 40. We train the Nes-eet-18 architecture with a learning rate 0.1 for softmax nets and 0.02 for Gauss nets, which is then docayed by factor of 10 at epoch 60 and 8o. The AT models are trained with the I.D attack. We employ a standard setting with 10 iterations in the inner loop and e = 0.1 for MNIST and 8 for CIFAR-10/100.

---

Under review as a conference paper at ICLR 2023
(0.92)
(0.92)
(0.97)
(0.96)
(0.78)
5m (1.00)
(a.
4
9
3
4
4
G
3
4
(0. (0.94)
96) duq
80) duq
(6.96)
(0.80)
D
0
4
4
0
4
ga (0.93)
ga (0.77)
ga (0.84)
(0.91)
ga (0.95)
(0.81)
(0.98)
ga (0.90)
ga (0.79)
(0.75 75)
4
4
4
S
9
4
4
7
3
Figure 4: Adversarial examples achieved with the 1x PGD attack on MNIST with confidence 0.7. Above each image you find the network (sm: softmax, duq: DUQ, ga: Gauss), the erroncously predicted class, and the confidence of the prediction in parentheses.
iny
inh 3.11
Top 2.50
MACT
(LCB)
RAITNA
dettaa DIRT
CA TAAAIAA
dest NAC
A. BAR
3395
C
Figure 5: Adversarial examples achieved with the too attack on Cifar-10 with confidence 0.7. Above each image you find the network (sm: softmax, duq: DUQ, ga: Gauss), the erroneously predicted class, and the confidence of the prediction in hesent
D CIFAR-10 ADVERSARIAL EXAMPLES
The most successful Gauss network attacks are the Lx PGD attack on MNIST and Cifar-10. Confidently assigned adversarial examples are displayed in Figures 4 and 5. Adversarial examples on MNIST indicate that Gauss networks often confuse a four with a nine; these also are the classes with the nearest centroids (distance of 9-5: average centroid distance is 12.1 + 1.4). The adversarial examples on Cifar-1o are less conclusive. While some of the adversarial examples for Gauss net are justifiable, other adversarial examples hardly make sense. This indicates that our proposed confidence measure does not yet provide a definite safeguard against learning shortcuts, but it provides a first step towards indications of model representations and knowledge.
E EFFECT OF THE PARAMETER IN ATTACKS
We plot Figure the attack success rate against the parameter e that determines the bound on the perturbation size. As expected, the higher the parameter e, the higher the attack success rate for softmax and Gauss networks. In contrast, we observe that DUQ models are for the FGSM and PGD attacks more susceptible to small perturbations of the input. This observation suggests that those attacks possibly overshoot, such that larger perturbations result. in images that mapped outside of the confidently assigned region.

---

Under review as a conference paper at ICLR 2023
FGSM
60 PGD
PGD
attack success rate (%
100
100
100
80
80
80
60
60
60
40
40
40
20
20
20
0
0
0
5
10-2
10
0.1
0
5
10-2 -
0.1
0
0.2
0.4
6
SOFTMAX
DUQ
GAUSS
Figure 6: Variation of parameter c, comparison of attack success rates (the lower the better) on Cifar-10 trained networks for the FGSM, PGD and &gPGD attack.
In comparison, the empirical analysis of the effect of further indicates that the proposed Gauss works are indeed well protected against small perturbations in the input.


Reviews:
[{"review_id": "UPY33yXzyqG", "reviewer": "Reviewer_D1u1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on uncertainty quantification and robustness. The authors find that existing work in uncertainty quantification mostly revolves around the confidence reflected in the input feature space. The authors focus on the learned representation of the network and analyze the confidence in the penultimate layer space. The proposed new confidence measure is centroid-based and hence no longer suffers from the artificial confidence inflation of out-of-distribution samples. \n", "summary_of_the_review": "See Strength And Weaknesses:\n\n------------ after rebuttal ------------\n\nThe authors' response addressed some of my concerns. But I still suggest adding more baselines and out-of-distribution detection experiments in the future version to demonstrate the effectiveness of the proposed method. Therefore, I will keep my score.", "strength_and_weaknesses": "Strengths:\n\n1. The authors formally prove that independent of optimization-procedural effects, a set of centroids always exists such that softmax classifiers are the nearest centroid classifiers.\n2. The authors proposed a new confidence measure that is centroid-based and hence no longer suffers from the artificial confidence inflation of out-of-distribution samples. \n3. The proposed centroidal confidence measure provides a robustness certificate against attacks.\n\n\nWeaknesses:\n\n1. What is the motivation of Section 3? What is the connection between uncertainty and robustness?\n2. Lack of important related work of uncertainty quantification, such as [1, 2].\n3. It is better to add more baselines to demonstrate the effectiveness of the proposed method.\n4. Since the paper focuses on uncertainty quantification, it may necessary to show some experiments result about out-of-distribution detection.\n\n\n[1] Malinin, Andrey, and Mark Gales. \"Predictive uncertainty estimation via prior networks.\" Advances in neural information processing systems 31 (2018).\n\n[2] Malinin, Andrey, and Mark Gales. \"Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness.\" Advances in Neural Information Processing Systems 32 (2019).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RhGEewQapz", "reviewer": "Reviewer_SMCi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new method for measuring confidence and increasing the robustness of neural networks. The core of the method is replacing the last linear layer and softmax with learnable per-class Gaussians (centroids).   \nTo address convergence problems, the paper proposes simplified Gaussians and adding negative log-likelihood with a margin to a loss.\nThe method is backed by proof that classification with centroids is at least as expressive as linear layer classification.", "summary_of_the_review": "While I have concerns about the experimental part, the paper is coherent and original overall.", "strength_and_weaknesses": "Strengths\n* The paper is well-written, the explanation is clear and the flow is good.\n* The proposed method appears naturally, backed up by a sound theoretical background.\n* Though it's closely related to another method, the one proposed in the paper feels simpler, requiring fewer resources and being more robust at the same time.\n\nWeakness\nMy main concern is that experiments feel shallow\n* First of all, the authors refer to DUQ and make comparisons mostly with it. But DUQ by itself focuses on out-of-distribution detection, not robustness, so I would expect out-of-distribution experiments. If the robustness is expected to be the strongest point for the method, then I would expect some baseline from this area, e.g. adversarial training or improved Lipschitz constant with methods like spectral normalization.\n* The authors mention restricting of Lipschitz constant and it's a part of a proof condition, but it's not addressed in the implementation and experiments. What's more concerning, local Lipschitz constant restriction with spectral normalization is part of the DUQ method, but I feel that it wasn't used in the paper this way. The provided repository doesn't contain the code for it, so I have some concerns about the validity of the provided results.\n* I'm not sure what conclusion should we make from confidence in table 1. Mean confidence is not a very meaningful metric, as with postprocessing like temperature scaling you can get almost arbitrary values for it. The calibration plot would be much more meaningful in this case. Particularly, one can argue that having mean confidence 0.87 with 0.99 accuracy is strongly under-confident.\n* While the statement in theorem 1 seems valid, there are a few issues in derivation, I believe it could use some polishing. E.g., missed opening bracket in eq.1. The sign for b seems wrong as well there, should be '-' in the third line of the appendix.  It's stated that the second term eq.1 could be made zero, but in fact, it is only shown to be constant.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9N4Q40A3G3", "reviewer": "Reviewer_gCc4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes modeling the predictive confidence of the DNN model using RBF-style output layers (Definition 1). This choice is justified by a theoretical argument on the connection between softmax classifier and the  neares centroid classifier, for which a robustness guanrantee can be derived (Section 3). The paper then conducted vision experiments based on standard benchmark and architecture (ResNet-18/50 on MNIST/CIFAR/ImageNet), ans shows accuracy improvement over DUQ, and adversarial robustness improvement over softmax classifier. ", "summary_of_the_review": "This is an interesting work that theoretically establishes the connection between softmax classifier and centroid classifier, empirically proposed an improved formulation of RBF-style last-layer classifier than outperforms its similar predecessors (i.e., DUQ), and brings the consideration of adversarial robustness into the picture. \n\nHowever, the work is slightly disconnected from the current uncertainty literature in that it does not sufficiently review or compare with its contemporary methods (e.g., SNGP, DUE) that handles Lipschitz control and uses distance-based classifier, and empirically its accuracy performance is not on par with the other uncertainty methods, and its uncertainty performance is not rigorously evaluated with standard performance metrics (e.g., ECE). Adding literature review and additional baseline comparison should address these issues to some extent.", "strength_and_weaknesses": "Strength:\n\n* A interesting work that improves the single-model uncertainty methods based on RBF-style classifier (i.e., DUQ).\n* Author investigated (both theoretically and empirically) the implication of adversarial robustness of the distance-based approach.\n\nWeakness:\n\n* Performance is lacking on large-scale benchmarks. There are several single-model methods developed and tested on, e.g., ImageNet with ResNet50 (e.g., see MIMO or SNGP on Uncertainty Baselines: https://github.com/google/uncertainty-baselines/tree/main/baselines/imagenet), which are generally competitive or outperform standard softmax classifier in Table 1. This may somehow discount the emprirical utility of Gauss classifier, as it still incurs a hard uncertainty v.s. utility tradeoff. This somehow leads me to question the utility of centroid-based classifier in more difficult tasks (CIFAR-100 and ImageNet).\n\n* Missing evaluation metric for uncertainty performance: In Table 1, please consider evaluate model uncertainty using a standard evaluation metric such as ECE.\n\n* Missing treatment of Lipschitz constant control / related literature review. As authors commented, Lipschitz constant is important for guanrantee model's uncertainty and robustness performance. However, there are a stream work in the single-model uncertainty literature (e.g., [SNGP](https://arxiv.org/abs/2205.00403), [DUE](https://arxiv.org/abs/2102.11409), [DDU](http://www.gatsby.ucl.ac.uk/~balaji/udl2021/accepted-papers/UDL2021-paper-022.pdf) etc) that tackles this by combining spectral normalization with distance-based classifier, which generally outperforms DUE and works well for high-dimensional benchmarks. Please consider include them into related work, and optionally absorbing some of these baseline techniques into methodology or include these previous work as baselines.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject