Revisiting Explicit Regularization in Neural Networks for Reliable Predictive Probability
Authors: Taejong Joo
Uijung Chung
Venue:  (2021)
Keywords: deep learning
predictive uncertainty
explicit regularization

Abstract: From the statistical learning perspective, complexity control via explicit regularization is a necessity for improving the generalization of over-parameterized models, which deters the memorization of intricate patterns existing only in the training data. However, the impressive generalization performance of over-parameterized neural networks with only implicit regularization challenges the importance of explicit regularization. Furthermore, explicit regularization does not prevent neural networks from memorizing unnatural patterns, such as random labels. In this work, we revisit the role and importance of explicit regularization methods for generalization of the predictive probability, not just the generalization of the 0-1 loss. Specifically, we analyze the possible cause of the poor predictive probability and identify that regularization of predictive confidence is required during training. We then empirically show that explicit regularization significantly improves the reliability of the predictive probability, which enables better predictive uncertainty representation and prevents the overconfidence problem. Our findings present a new direction to improve the predictive probability quality of deterministic neural networks, which can be an efficient and scalable alternative to Bayesian neural networks and ensemble methods.


Under review as a conference paper at ICLR 2021
REVISITING EXPLICIT REGULARIZATION IN NEURAL NETWORKS FOR RELIABLE PREDICTIVE PROBABILITY
Anonymous authors Paper under double-blind review
ABSTRACT
From the statistical learning perspective, complexity control via explicit regular- ization is necessity for improving the generalization of overparameterized mod- els, which deters the memorization of intricate patterns existing only in the train- ing data. However, the impressive generalization performance of overparameter- ized neural networks with only implicit regularization challenges the importance explicit regularization. Furthermore, explicit regularization does not prevent neural networks from memorizing unnatural patterns, such as random labels. In this work, we revisit the role and importance of explicit regularization methods for generalization of the predictive probability, not just the generalization of the 0-1 loss. Specifically, we analyze the possible cause of the poor predictive prob- ability and identify that regularization of predictive confidence is required during training. We then empirically show that explicit regularization significantly im- proves the reliability of the predictive probability, which enables better predictive uncertainty representation and prevents the overconfidence problem. Our findings present a new direction to improve the predictive probability quality of determinis- tic neural networks, which can be an efficient and scalable alternative to Bayesian neural networks and ensemble methods.
INTRODUCTION
As deep learning models have become pervasive in real-world decision decision-systems, systems, the importance of producing reliable predictive probability is increasing. In this paper, we call predictive probabil- ity reliable if it is well-calibrated and precisely represents uncertainty about its predictions. The calibrated behavior refers to the ability to match its predictive probability of an event to the long- term frequency of the event occurrence (Dawid] 1982). The reliable predictive probability benefits many downstream tasks such as anomaly detection Malinnnn & Gales 2019), classification with re- jection Lakshminaaayaaaal et al. 2017), and exploration in reinforcement learning (Gal & Ghahra- mani 2016). More importantly, deep learning systems with more reliable predictive probability can provide better feedback for explaining what is going on, situations when its prediction becomes uncertain, and unexpected anomalies to users. Unfortunately, neural networks prone to be over- confident and lack uncertainty representation ability, and this problem has become fundamental concern in the deep learning community.
Bayesian methods have innate abilities to produce reliable predictive probability. Specifically, they express the probability distribution over parameters, in which uncertainty in the parameter space is automatically determined by data (MacKay 1992. Neal 1993). Then, uncertainty in prediction can be represented by means of providing rich information about aggregated predictions from different parameter configurations such entropy and mutual information. From this perspective, deter- ministic neural networks selecting a single parameter configuration that cannot provide such rich information naturally lack the uncertainty representation ability. However, the automatic determi- nation of parameter uncertainty in the light of data, i.c., posterior inference, comes with prohibitive computational costs. Therefore, the mainstream approach for improving the predictive probabil- ity quality has been an efficient adoption of the Bayesian principle into neural networks (Gal &
Ghahramani] 2016 Ritter al. 2018 Teye al. 2018 Joo et al. 2020a).
Thulasidasan et al.) 2019) has dis- covered the hidden gems of label smoothing Szegedy et al. 2016), mixup (Zhang et al. 2018 11 and
Recent works Lakshminarayanan et al. 2017 Müller et al. 2019

---

Under review as a conference paper at ICLR 2021
adversarial training (Goodfellow et al.) 2015), which improve the calibration performance and the uncertainty representation ability. These findings present a new possibility of improving the reliabil- ity of the predictive probability without changing the deterministic nature of neural networks. This direction is appealing because it can be applied in a plug-and-plat fashion to the existing building blocks. This means that they can inherit the scalability, computational efficiency, and surprising gen- eralization performance of the deterministic neural networks, for which Bayesian neural networks
often struggle (Wu al. 2019 Osawa et al.
2019 Joo et al. 2020a).
Motivated by these observations, we investigate a general direction from the regularization perspec- tive to mitigate the unreliable predictive probability problem, rather than proposing new constructive heuristics or discovering hidden properties of specific methods. Our main contribution is twofold. First, we present new direction for alleviating the unreliable predictive hehavior, which readily applicable, computationally efficient, and scalable to large-scale models compared to Bayesian neu- ral networks or ensemble methods. Second, our findings provide a novel view of the role of explicit regularization in deep learning, which improves the reliability of the predictive probability.
ANALYZING THE CAUSE OF UNRELIABLE PREDICTIVE PROBABILITY
2.1 BACKGROUND
We consider a classification problem with i.i.d. training samples D = (xx11, ( i=1 drawn from unknown distributions Px, whose corresponding tuple of random variables is (x, y). We denote
X' as an input space and y as a set of categories {1.2. K}. Let fw X + Z be neural
network with parameters W where 2 IR is a logit space. On top of the logit space, the softmax RRK + A-1 normalizes the exponential of logits:
exp(fE W (z))
(a)
exp (x))
where we let of (x) 02 (x)) for brevity. (f W (x) is often interpreted as the predictive
probability that the label of æ belongs to class k (Bridle 1990)
The probabilistic interpretation of neural network outputs gives the natural minimization objective for classification-tth the cross-entropy between the predictive probability and the one-hot encoded la- bel: IcE(y, @ (x)) - Es (k) log w (x). where ( an indicator funetion taking one if E A and zero otherwise. By minimizing the cross-entropy (or equivalently maximizing the log-likelihood) with stochastic gradient descent (SGD) (Robbins & Monro 1951) or its variants, modern neural networks achieve the surprising generalization performance.
As the demand for neural networks in wal-world decision-making is emerging, reliable predictive probability has been of interest in the machine learning community. One important quality of pre- dictive probability is calibrated behavior. Specifically, based on the notion of calibration in classical forecasting problem (Dawid 1982), the perfectly calibrated model can be defined follows:
ply = ki W (x) = p) = Pk. vp E K-1 E (1,2, , ,K}
(2)
Here note that the calibrated model does not necessarily be ones producing s (x) = p(y = kxx).
In practice, expected calibration error (ECE) (Naeini et al., 2015) is widely used for calibration performance measure. ECE on dataset D can be computed by binning predictions into M-groups based on their confidences" and then averaging their calibration scores by:
M
9. DJ
Σ
acc(G,) conf(G.)
(3)
where G; = fz i/M < max; ow (x) < (1 i)M, E DT); acc(G,) and conf(g.) are average accuracy and confidence of predictions in group G., respectively.
'Throughout this paper, the confidence (or predictive confidence) at æ refers to max; Φ (ar), which is different from the confidence in statistics literature.
2

---

Under review as a conference paper at ICLR 2021
Another metric for evaluating the reliability of predictive probability is based on predictive entropy, which evaluates how well is the model aware of its ignorance. To this end. predictive entropy is mea- sured on samples that the model is ignorant of, such as misclassified out-of-distin of-iistrbbuution (OOD) samples. For the classifier having reliable predictive probability, we expect the high uncertainty of Φ (x) such samples, i.e., the answer "I don't know."
However, several recent findings show that the resulting neural net- work produces unreliable predictive probability, which interprets the softmax output as the "predictive probability" implausible (Gal & Ghahramani 2016). For example, Figure [T] illustrates the unreliable predictive behavior of ResNet (He et al., (016):: the network produces outputs with high confidence in misclassified examples (Figure [ (up- sam- ples, albeit the samples belong to none of the classes it learned (Fig- ure u (lower).
1.0
Softmax
0.8
Suftmax tamp. scalling
Acourecy
0.2
per)) and provides low predictive entropy on out-of-distribute
0.0
0.00
0.25
0.50
0.75
1.00
Confidence
One simple yet effective solution for improving the predictive proba- bility's s quality is temperature scaling (Guo et al. 2017). which adjusts the smoothness of the softmax so that the resulting predictive proba- bility maximizes the log-like likelihood on unseen dataset D':
Softmax
0.8
Softmax scalling
Deralty
4.4
exp(fy (x)/r) Exexp(s, (a:)/7)
max
log
(4)
0.0
0
4
Entropy
where W is a fixed pretrained weight and T is a temperature con- trolling the smoothness of the softmax output. This simple method makes the softmax output more reliable predictive probability. For instance, the predictive confidence matches its actual accuracy well, and the predictive entropy on out-of-distributin samples significantly increases (Figure
Figure 1: Reliability curve (upper) and pre- dictive uncertainty on out-of--istribun samples (lower) of ResNet.
2.2 CLOSER LOOK AT THE LOG-LIKELIHOOD ON UNSEEN SAMPLES.
Motivated by the success of the temperature scaling, we aim to find the relationship between the log- likelihood and the calibration performance. To this end, let s be a binary random variable indicating whether model correctly classifies a sample. Then, can derive the following upper bound by using the law of total expectation:
Exy [log 0g (x)] = Ex [pm)x(s 1)),1,=111xx [log of (x)] + Pspp 0)1Ey|o=0xx [log by
W
(x)]]
S Sxx [P' x = 1) log oh (x) + Ps/x(( = 0) log(1 W (x)) (5)
where mx is the predictive class such that mx arg J (x), Ps|x(s = 1) ply
arg max, Om, (x) xx), and the inequality comes from the fact that Eyjos=xx [log o, (x)] S
max log 6 (x) < log(1 - w (x)) The upper bound can be thought of as the probabilistic measure of the calibration performance. Specifically, suppose neural network produces an answer with probability " (x) and refuses to answer otherwise. Then, the upper bound becomes the di- vergence between the model's ability to correctly predict the sample and the model's 's willingness to answer. When we consider the inequality in equation i] applied to the empirical mean on the test dataset, this inequality clearly explains why the temperature scaling is helpful; it increases a lower bound of the calibration error with modified confidence by closer to its accuracy.
More importantly, the inequality in equation exxplains the impacts of the cross-entropy minimiza- tion the behavior of the neural network. Specifically, suppose (xxx))> [log] oy for some dataset D. Then, can be shown that Psix(s 1) + 1 and & W (x) + 1 for all (x, 2) E D. This holds because we have ming di (x) > 1/K for any x and the minimum of p log + (1 - p) log(1 - q) are (p.g) + (1,
W (x)] + 0
W
(p.g) + (0, Therefore, for high-capacity models that can make the log- likeeloooood on the training dataset Der close to zero, e.g., deep neural networks. its behavior
the training set will converge to configuration that corrects all samples with perfect confidence. Figure 2 illustrates this phenomenon in ResNet trained on CIFAR- 100: Ex [log [W]
0 (a) and (xx [p = 1)] - (b) as the training continues.
3

---

Under review as a conference paper at ICLR 2021
1.0
-2
0.6
30
Ivalidi
0.4
-3
Max log prob. Itrain)
Max (valid)
0.2
Accurasy (train) (valid:
nom (train)
(valid)
100 Epoch
150
200
50
100
150
200
50
100
150
200
Epoch
Epoch
(a)
(b)
(c)
Figure 2: Monitoring changes in the behavior of ResNet during training on CIFAR-100. In (a), LL indicates the log-likelihood Ex, [log o, (x)] and max log-prob indicates Ex [loo Pme (x)]. In
(c), the L norm is approximated with respect to empirical distributions of training samples and validation samples.
Why are these convergences problematic? We observe that function's 's properties, which depends only on x, evaluated on two different datasets are very close to each other if they are drawn from the same distribution, unlike values that depend on the external randomness y x; that is, (Ex-D,r(g(3))
Ex' [x(x))] < (xx,))_D,, [h(x, y)] - Exx \'x''''>D..ud [h(x', y')l for some functions g(-) and
h(-). For example, the empirical mean of the maximum log- see-prbbability log @ (x) (Figure 2 (a)
significantly similar to those values on unseen samples compared to the log-likelihood (Figure 2 (a)) and the accuracy (Figure 2 (b))-
and 1,2 norm of fw (Figure (c) on training samples
We conjecture that the log-likelihood maximization with a high-capacity neural network naturally results in a high calibration eπror on unseen samples due to this discrepancy; that is, it produces perfect confidence on unseen samples as it does on the training samples where cannot produce the perfect accuracy.
Fortunately, this result also indicates that restricting the predictive confidence W (x) on training samples will directly impact the predictive confidence on unseen samples and therefore can reduce the calibration error. For this reason, we explore various ways to restrict confidence and show their efficacy on a wide range of tasks for the rest of the paper.
CONFIDENCE CONTROL FOR RELIABLE PREDICTIVE PROBABILITY
The goal of this section is to examine the effectiveness of restricting predictive confidence on im- proving the reliability of predictive probability. Therefore, we apply several existing regularizers to the (pre-activation) ResNet (He al. 2016) trained CIFAR (Krizhevsky et al. 2009), which is one the most prevalent basis models in many state-of-the-arr architectures (Huang et al. 2017 Xie et al.) 2017). We also present the VGG (Simonyan & Zisserman 2015) as representative of models without residual connection in appendix B in which we observe results similar to ResNet. We follow the standard training strategy presented in He al. (2016) except for the initial learning rate warm-up, clipping the gradient when its norm exceeds one, and making an extra validation set of 10,000 samples split from the training set. We describe detailed setup in appendix A
We mainly use the ECE for measuring the reliability of the predictive probability. In addition. used negative log-likelihood (NLL), following the common practice (Lakshminarayanar 2017 Ovadia et al.] 2019 Guo et al. 2017). Here, NLL evaluates how well the predictive probability explains the test data D whose optimal score is achieved if and only if ø w (x) perfectly matches p(y(x).
al.,
3.1 CONFIDENCE CONTROL BY WEIGHT DECAY
The simplest way to constrain the confidence may conceivably be using stronger weight decay (Krogh & Hertz 1992), which encourage the production of less extreme outputs by shrink- ing weights. Therefore, we first explore the confidence control by varying the weight decay ratio X, conjecturing that the weight decay ratio used in the base model, e.g., X 0.0001 in ResNet,
"This norm is evaluated on the function space by || f ||2= ( ff (m) (d)) 1/2 which we will discuss with more detail in Section 3.2
4

---

Under review as a conference paper at ICLR 2021
is too small to prevent overconfident predictions. Figure (upper) illustrates the impact of À on generalization performance and calibration performance. We can see that ECE decreases to some extent the decay ratio increases. However, the ECE improvement becomes inversely proportional to generalization performance improvement when the decay ratio is larger than 0.001. This means that improving the reliability with strong weight decay is against the primary goal of supervised learning.
We further investigate this undesirability by monitoring the L function norm of fw (cf. equation 7 under different weight de- cay ratios. This analysis is motivated by the fact that temperature scaling modifies only the L" norm of fW as it divides logits with a sealar. Under the single-label classification model, this measure is useful proxy for the predominance of the maximum predictive probability, To set the desired value of the T,2 norm, we "leak" the test set Dd and then apply temperature scaling to the neural network trained with À = 0.0001.
100
Ernor
50
ECE
Error rate
10
20
10-1
10-1
10-3
10"
Welght
Figure 3 shows that the SGD with various decay ratios finds only a trivial solution or an infeasible solution to the following optimiza- tion problem:
0.01
0.023
20-
0.023
0.0301
tema -CRP
min min En [ ( W
s.t. || f ||2/|| f W /T" ll2
(6)
20
where Eoll] is the empirical mean on D. W is the trained weights under X 0.0001, and 7* Figure 3: Impact of the weight decay ratio on ECE and accuracy (upper) and the
arg, Epr llog {a(s (x)/r)] This means that re-
ducing the predictive confidence to the desired level without sacrificing generalization performance is significantly challeng- ing optimization problem. Specifically, Figure (lower) shows that
2 norm (lower).
the 2 norm goes to zero under the decay ratio À > 0.003, which means that all weights collapse to zero, i.e., the trivial solution. This happens when the magnitude of the decay ratio overwhelms the magnitude of the weight update by the gradient of the cross-entropy, e.g., at approximately the epoch 50 under À = 0.003 (Figure B (lower)). However, ratios of 0.001 and 0.0001 do not suffer from the weight collapse, but the scale of the I norm under such ratios is significantly higher than
|l [ /T" Il2. which corresponds to infeasible solutions (Figure 3 (lower)). These results may seem natural because the weight decay does not explicitly consider the constraint about predictive confidence. Therefore, we explore an alternative method that adds a regularization loss explicitly concerning predictive confidence, c.g., the constraint in equation 6
3.2 DIRECT CONFIDENCE CONTROL BY EXPLICIT REGULARIZZTIIN
In this subsection, we examine two types of regularizers that directly constrain the predictive confi- dence on training samples.
Regularization in the function space. The first approach regards fw as an element of LP(
space. I space is the space of measurable functions with the norm:
w 11-= 11MMM))))) 1/p
<0
Here, we note that the norm is computed with respect to the input generating distribution Px, which allows concerning how the function fw actually behaves on the data manifold. Since Px is un- known, so it is approximated by the Monte-Carlo approximation with minibatch samples. Then, the approximate function norm can be computed by f I1 1 Ev, I5, (1.. By penalizing the complexity with the LP norm, the continuous increase in the leading entry of logit towards infinity, or continuous decreases in the nonleading entry to negative infinity, can be prevented. In this paper, examine fW || and fW 1/ regularization losses
Regularization in the probability distribution space. The second approach regards fW (x) as random variable and then minimize its distance the bell-shaped target distribution. Since the
5

---

Under review as a conference paper at ICLR 2021
Table 1: Experimental results under various regularization methods ResNet. Arrows the metrics represent the desirable direction. We searched four hyperparameters for each method and chose the best hyperparameter based on validation accuracy (cf. appendix A Values represent mean obtained from five repetitions, and all values are rounded to two decimal places.
CIFAR-10
CIFAR-100
method
Acc f
NL..
ECF. +
f
w
1a
Acc
NLL
ECE 4
|| f
lla
Vanilla
94.17
0.29
4.06
20.12
74.64
1.31
13.95
43.81
f 11
94.32
0.25
2.89
6,81
76.28
1.27
7.77
9.07
94.38
0.23
3.27
8.19
75.84
1.07
5.52
10.55
SW SWI v))
94.46
0.23
2.12
5,4
76.27
1.1
7.02
9.67
PER
94.30
0.23
2.89
6.94
76.23
1.15
4.67
7.56
bell-shaped distribution has a peak mass at zero and the same density on points with the same distance to zero, this regularization can make logits have small values and make each entry of logits have a similar scale, which can prevent neural nets from producing overconfident answers, Among various bell-shaped distributions, we choose the standard Gaussian for the target distribution due to its simplicity and popularity. As metric in the probability distribution space, we use the sliced Wasserstein distance of order one because of its computational efficiency and ability to measure the distance between probability distributions with different supports, which is useful when dealing with the empirical distribution. We refer to Peyré et al. (2019) for more detailed explanations about this metric. Specifically, given minibatch samples D' = (x()) vm 1==1, let an empirical measure of logits be uD (A) = m Σ, (x)((()())) and the standard Gaussian measure on Z be v(A) =
K77 JA exp (-2 2 || z 119) dz. Then, the sliced Wasserstein distance can be computed by:
1 Σ 1 (00. (12 ()) drdA(0)
W ,V) SW SWI
(x)
(8)
gg
where 2() (w A is a uniform measure on the unit sphere SK-1, and pi is a measure obtained by projecting "B at angle 0. Projected error function regularization (PER) (Joo et al. 2020b) simplifies the computation of the SW (u. 2/ by applying the Minkowski inequality to the above equation. As result, the gradient of PER resembles the gradient of Huber loss (Huber 1964) in the projected space, which allows combining advantages of both the L norm and the 1,- norm as well as capturing the dependency between logits of each location by a projection operation (Joo cet al. 2020b).
Image classification with ResNet. Table [ lists the experimental results, in which both regulariza- tion in the function space and the Wasserstein probability space successfully controlled the confi- dence, e.g., reduced the L norm of ResNet by at least 34% on CIFAR-10 and 68% on CIFAR- 100. We note that regularization methods can constrain the confidence without compromising the gener- alization performance; actually, all regularization methods achieves small but consistent improve- ments of test rates. We also note that the of the Frobenius norm of weights often increases compared to the vanilla method and changes only at most 2% when it decreases, which again shows the undesirability of adjusting the weight decay ratio for confidence control.
More importantly, the predictive probability's 's reliability significantly improves under all considered measures compared to the vanilla method. For instance, the regularization methods reduce the NLL of ResNet by at least 13% CIFAR-10 and 6% on CIFAR-100 and reduce the ECE of ResNet by at least 19% on CIFAR- 10 and 41% on CIFAR- 100. These improvements are comparable or better than those of temperature scaling. For instance, ResNet with temperature scaling gives an NLLI of 1.15 and an ECE of 8.41 on CIFAR-100. Here, we split the test set into two equal-size sets-a performance measurement set and a temperature calibration set-and measure the performance after temperature scaling with the calibration set, and repeat the same procedure by reversing their roles. We note that more realistic evaluation requires drawing the temperature calibration set from the training set. In this case, its performance decreases as cannot fully exploit the entire dataset during training.
Predictive uncertainty. We also investigate the usefulness of explicit regularization on the uncer- tainty representation ability misclassified samples and OOD samples. Since the model is ignorant

---

Under review as a conference paper at ICLR 2021
regubbriattnn Wenng
logtt reguloitation (e-t9t) ccp
Correct (o3s)) Whong (h-ctit)
0.0
Wrong in Indiiiitttonn Our -oo butoon
Currect (n-didrtutton)
000
Erireyy
Grerpy
Ehimpy
Entraey
Vanilla
Deep ansomble
MC-droooutt In (.2)
MC-dropeut (p m
Correctinddtt
1.00
dermectivd
(indss))) o02
Whongth
oce
2.3
0.30
2.5
2
9.0
0.00
010spy
Figure 4: Density of predictive uncertainty on CIFAR-100 (in-distribution) and SVHN (OOD). The upper figures illustrate explicit regularization methods, and the lower figures illustrate the vanilla method, ensemble methods, and Bayesian neural networks.
Table 2: Misclassification detection and OOD detection task performances based on NBAUCCo.5 (higher is better). MC-D (p) represents MC-dropout with probability p, and Ens represents deep ensemble.
Task
Vanilla
MC-D (0.2)
MC-D (0.3)
Ens.
||
[ 113
SW SWiWW,, v)
PER
Miscls
1.55
3.33
5,94
0.59
9.53
6.85
5.10
10.24
OOD
15.98
22.14
31.28
28.46
55.51
31.47
31.63
49.77
of misclassified samples and OOD samples they do not belong to any category, the neural network should produce the answer of "I don't 't know" for these samples. Figure the predictive un- certainty of ResNett-50 with respect to CIFAR- 100 -iddistribution) and SVHN (Netzer et al.) 201T) (OOD). The vanilla method's 's predictive uncertainty on SVHN and misclassified samples remains in the somewhat confident region, albeit less confident compared to those on CIFAR-100 In contrast, explicit regularization successfully gathers mass of predictive uncertainty for both SVHN samples and misclassified samples around the maximum-entropy region.
We compare the uncertainty representation abilities of regularization methods to those of Bayesian neural networks and ensemble methods. Specifically, we use the scalable Bayesian neural network, called MC-dropout (Gal & Ghahramani] 2016), because other methods based on variational infer- ence (Graees 2011 Blundell et al., 2015 Wu et al.) 2019)) or MCMC (Welling & Tch 2011 Zhang al. 2020) require modifications to the baseline including the optimization procedure and the chitecture, which deters a fair comparison. We searched a dropout rate over (0.1, 0.2, 0.3, 0.4, 0.5 and used 100 number of Monte-Carll samples test time, i.e., 100x more inference time. We also use the deep ensemble Lakshminarayanan et al.) 2017) with 5 number of ensembles, i.e., 5x more training and inference time. Figure shows that the regularization-bann -baedd methods produce signifi- cantly better uncertainty representation than the MC-dropout and deep ensemble; even though both deep ensemble and MC-dropout can move mass on less certain regions, the positions are still far from the highest uncertainty region, unlike the regularization-bee baased methods. Finally, we note sev- eral approaches (Malinin & Gales, 2019 Papadopoulos et al., (2019) explicitly maximize uncertainty on OOD training samples for OOD detection tasks. However, we do not consider such approaches, assuming we have no access to OOD samples during training.
Misclasiifcattin/OODD detection. We also evaluate the uncertainty representation ability for misclassified and out-of-distributtion samples. To this end, we use recently proposed normalized bounded area under the calibration curve (NBAUCC) (Kong al. (202)). which handles the disad- vantage of classical measures such as AUROC and AUPRC that cannot take the calibration property into account. Specifically, NBAUCC with the upper bound of confidence threshold for misclassified
7

---

Under review as a conference paper at ICLR 2021
Table 3: Experimental results under various regularization methods BERT. Arrows the metrics represent the desirable direction. Values represent mean obtained from five repetitions, which are rounded to decimal places.
No-Newsgroup
Web of science
method
Ace f
N
ECE +
f
||
Acc
NLL.
ECE +
|| f 1/2
Vanilla
84.51
0.70
6.40
9.71
81.28
0.94
8.26
20.13
f
84.71
0.61
4.67
6.02
81.68
0.90
7.02
8.89
85,04
0.64
3.24
5.11
81,83
0.94
5.79
7.70
SW SW
84.76
0.67
3.78
4.98
81.44
7.22
11.33
PER
85.02
0.65
4.50
6.21
81.36
0.87
8.19
13.21
80
Accuracy
1.6
NLL
20
ECE
Accuracy
2.5
NLLL
20
ECE
75
Early Stoping
PBR
2.5
Unear Scaling
70
Sartee
1.5
10
65
300
200
300
1.00
200
300
70
32
2048
128
2048
32 128 2028
Training Epochs (paselin::
Batch (baselinn: 128)
(a) Early stopping
(b) Varying noise stochastic gradient
Figure 5: Effects of implicit regularization methods accuracy, negative log- log-likelihood, and ECE.
or OOD samples, denoted by r, is computed by:
NBAUCC, M
F
M
where (I) computes F1 score by regarding samples with predictive confidence higher than as correct (resp. in-distribution) samples and incorrect (resp. OOD) samples otherwise; M is a predetermined step size. We present the misclasificattinnOODD detection performance in Table 2 where all regularization methods significantly improve detection performances based on their better predictive uncertainty representation abilities.
Document classification with BERT. Finally, we perform additional experiments on the natural language processing domain in order more thoroughly evaluate the effectiveness of explicit reg- ularization. Inspired by the recent finding (Kong et al. 2020). we perform document classification on 20 newsgroup dataset (Socher et al., 2012) and web of science dataset (Kowsari al. 2017). We obtain a classifier by adding one linear layer on top of the BERT (Devlin et al.) 2018). We provide training details in Appendix [ As consistent with the findings in the image classification task, all explicit regularization methods improve the reliability of predictive probability with consistent gains test accuracy (Table
3.3
CAN THE IMPROVED RELIABILITY BE ATTRIBUTED TO IMPROVED GENERALIZATION?
Considering that the log- likeliieli is the training objective, the improved reliability may be caused by the improved generalization due to regularization effects. To clarify this point, we investigate the impact of improved generalization via implicit regularization mechanisms (early stopping and exploitation of noise from SGD) to ResNet-50 on CIFAR-100. Figure 5 (a) shows that the early stopping does result in dramatic improvements in all measures compared PER. Instead, NLL and ECE slightly improve under longer training due to increased classification accuracy. Consider- ing that the L norm of logits achieves a significantly high value the early stage of training (cf. Figure (b)). this undesirable result may seem natural. Next, we examine the effects of varying batch sizes under the baseline learning rate or linear scaling rule (Goyal et al., 2017). in which smaller batch size increases the noise of the stochastic gradient. In Figure (b). we observe that in- creasing the noise level by using smaller batch sizes benefits generalization performance. However, it does not benefit and even worsens the predictive distribution quality (NLL and ECE). In addi- tion, we observed that these implicit regularizers fail to produce high predictive entropy on OOD samples, unlike explicit regularization. These observations corroborate the efficacy of confidence control in improving the reliability of predictive probability, which highlights the importance of explicit regularization.
8

---

Under review as a conference paper at ICLR 2021
4 RELATED WORK
Recent works show that joint modeling of a generative model p(x) along with a classifier p x or p(x,)) y) directly, helps to produce reliable predictive probability (Alemi et al. 2018 Nalisnick et al. 2019 Grathwohl et al. 2020). Specifically, (Alemi et al. (2018) argue that modeling stochastic hidden representation through the variational information bottleneck principle (Alemi et al. (017) allows representing better predictive uncertainty. This can be related to the effectiveness of en- semble methods, which aggregate representations of several models, on enhancing predictive un- certainty representation and confidence calibration (Lakshminarayanan et al., [017]] Ovadia et al.) 2019 AAhhukha et al.) 2020). In this regard, hybrid modeling and ensemble methods share a similar principle to the Bayesian methods, concerning the stochasticity of the function. However, this paper concentrates on explicit regularization for controlling the predictive confidence, which is fundamen- tally different from previous works focusing on building the stochastic representation.
Other works concentrate on the structural characteristics of neural networks. Specifically, Hein al. (2019) identify the cause of the overconfidence problem based on an analysis of the affine compositional function, e.g., ReLU, The basic intuition behind this analysis is that one can always find multiplier À to an input æ, which makes neural network produce one dominant entry on Ax. Verma& & Swami 2019) point out that the region of the highest predictive uncertainty under the softmax forms a subspace in the logit space, so the volume of an area representing high predictive uncertainty would be negligible. However, our approach suggests that these structural characteris- ties' inherent flaws can be easily cured by adding an explicit regularization term without changing the existing components of neural networks.
From the perspective of the statistical learning theory (Vapnik) 1995), a regularization method min- imizing some form of complexity measures, e.g., Rademacher complexity (Bartlett al., 2005) or VC-dimension Vaannik & Chervonenkis 2015), is a "must" to achieve better generalization of overparameterized models by preventing memorization of intricate patterns existing only in training samples. However, the role of capacity control with explicit regularization is challenged by many observations in deep learning. Specifically, overparameterized neural networks achieve impressive generalization performance with only implicit regularizations contained in the optimization proce- dures (Hardt et al.] 20166] Li et al. 2019) or the structural characteristics (Gunasekar et al. 2018 Hanin & Rolnick 2019; Luo et al. 2019). Moreover, Zhang et al. 2017) show that explicit regular- ization cannot prevent neural networks from easily fitting random labels that cannot be generalized to unseen examples. Therefore, the importance of capacity control with explicit regularization seems to be questionable in deep learning. In this work, we reemphasize its importance, presenting a dif- ferent view on the role of regularization in terms of generalization of the predictive probability, not solely on better accuracy.
CONCLUSION
In this works, we show the usefulness of the explicit regularization on improving the reliability of predictive probability, which presents a novel view the role and importance of explicit regular- ization. Specifically, our extensive experimental results show that several existing regularization methods improves calibration, predictive uncertainty representation, misclassificatioonO000 detec- tion performances, and even the test accuracy. Compared to the temperature scaling method, the eegularization-based approach can be more widely applicable in more general settings such con- tinual learning and online learning as it does not requires an additional hold-out dataset. In addition, the regularization-baat based approaches provided small but consistent gain in generalization perfor- mances, while the temperature scaling does not impact the generalization performances. Compared to marginalization-ban basedd approach such as Bayesian neural networks and ensemble methods, the regularization-bue based approach is appealing in terms of computational efficiency and scalability. De- spite these advantages, the regularization methods are limited that they cannot utilize more so- phisticated uncertainty quantification methods based on stochastic representation on the predictive probability space, such as mutual information measuring epistemic uncertainty (Smith & Gal 2018), due to its deterministic nature. We leave this limitation an important future direction of research, which may be solved by more expressive parameterization, e.g., (Wilson et al., 2016 Skafte et al. 2019 Malinin & Gales 2019 Joo al. 2020a)

---

Under review as a conference paper at ICLR 2021
REFERENCES
Alexander A Alemi, lan Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference Learning Representations, 2017.
Alexander A Alemi, lan Fischer, and Joshua V Dillon. Uncertainty in the variational information bottleneck. arXiv preprint wwXiv::87..00900 2018.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. In International Conference Learning Representations, 2020.
Peter L Bartlett, Olivier Bousquet, Shahar Mendelson, et al. Local Rademacher complexities. The Annals of Statisticss, 53(44)1497-15333 2005.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. International Conference on Machine Learning. 2015.
John S Bridle. Probabilistic interpretation of feedforward classification network outputs, with rela- tionships to statistical pattern recognition. In Neurocomputing pp. 227-236. Springer, 1990.
A Philip Dawid, The well-calibrated Bayesian. Journal of the American Statistical Association, 77 (379):605-610) 1982.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800 2018.
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning. 2016.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint 176x:17766.0677,,2000 2017.
Will Grathwohl, Kuan-Chich Wang, Jöm--Herik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference on Learning Representations, 2020.
Alex Graves. Practical variational inference for neural networks. In Advances Neural Information Processing Systems, 2011.
Suriya Gunasekar, Jason D Lee, Daniel Soudry, and Nati Srebro. Implicit bias of gradient descent on linear convolutional networks. In Advances in Neural Information Processing Systems, 2018.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
Boris Hanin and David Rolnick. Deep relu networks have surprisingly few activation patterns. In Advances in Neural Information Processing Systems, 2019.
Moritz Hardt, Benjamin Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpass- ing human-level performance on imagenet classification. In IEEE International Conference on Computer Vision, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference Computer Vision, pp. 630-645, 2016.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In JEEE Conference Computer Vision and Pattern Recognition, 2019.
10

---

Under review as a conference paper at ICLR 2021
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Peter Huber. Robust estimation of location parameter. The Annals of Mathematical Statistics, 35 (1):73-101) 1964.
Taejong Joo, Uijung Chung, and Min-Gwan Sco. Being Bayesian about categorical probability. In International Conference on Machine Learning, 2020a.
Taejong Joo, Donggu Kang, and Byunghoon Kim. Regularizing activations in neural networks via distribution matching with the Wasserstein metric. In International Conference on Learning Representations, 2020b.
Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. Calibrated lan- guage model fine-tuning for in-and -ustt--diitriuutiin data. In Conference on Empirical Methods in Natural Language Processing, 2020.
Kamran Kowsari, Donald E Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S Ger- ber, and Laura E Barnes. Hdltex: Hierarchical deep learning for text classification. In 2017 16th IEEE international conference on machine learning and applications (ICMLA), pp. 364-371. IEEE, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Anders Krogh and John A Hertz. A simple weight decay can improve generalization. In Advances in Neural Information Processing Systems, 1992.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
Yuanzhi Li, Colin Wei, and Tengyu Ma. Towards explaining the regularization effect of initial large learning rate in training neural networks. In Advances in Neural Information Processing Systems, 2019.
Ping Luo, Xinjiang Wang, Wenqi Shao, and Zhanglin Peng. Towards understanding regularization in batch normalization. In International Conference on Learning Representations, 2019.
David JC MacKay. A practical Bayesian framework for backpropagation networks. Neural Compu- tation, 4(3):448-477) 1992.
Andrey Malinin and Mark Gales. Reverse KL-divergence training of prior networks: Improved uncertainty and adversarial robustness. In Advances in Neural Information Processing Systems, 2019.
Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, 2019.
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba-
bilities using Bayesian binning. In AAAI Conference on Artificial Intelligence, 2015.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Hybrid models with deep and invertible features. In International Conference Machine Learn- ing, 2019.
Radford M Neal. Bayesian learning via stochastic dynamics. In Advances in Neural Information Processing Systems, 1993.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
11

---

Under review as a conference paper at ICLR 2021
Kazuki Osawa, Siddharth Swaroop, Mohammad Emtiyaz E Khan, Anirudh Jain, Runa Eschenhagen, Richard E Turner, and Rio Yokota. Practical deep learning with Bayesian principles. In Advances in Neural Information Processing Systems, 2019.
Yaniv Ovadia, Emily Fertig, Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, 2019.
Eestotelis-AAgells Papadopoulos, Mohammad Reza Rajati, Nazim Shaikh, and Jiamian Wang- Outlier exposure with confidence control for out-of-distribution detection. arXiv preprint arXiv:1906.0350, 2019.
Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport. Foundations and Trends in Machine Learning, 1 115--6):355-677, 2019.
Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable Laplace approximation for neural networks. In International Conference on Learning Representations, 2018.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The Annals of Mathemat- ical Statistics, pp. 400-407, 1951.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
Nicki Skafte, Martin Jørgensen, and Søren Hauberg. Reliable training and estimation of variance networks. In Advances in Neural Information Processing Systems, 2019.
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detec- tion. arXiv preprint 1rXiv:8803.08533 2018.
Richard Socher, Yoshua Bengio, and Christopher D Manning. Deep learning for nip (without magic). In Tutorial Abstracts of ACL 2012, pp. 5-5. 2012.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Mattias Teye, Hossein Azizpour, and Kevin Smith. Bayesian uncertainty estimation for batch nor- malized deep networks. In International Conference on Machine Learning. 2018.
Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Micha- lak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances Neural Information Processing Systems, 2019.
Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer-Veriie Verlag, 1995.
Vladimir N Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. In Measures of Complexity, pp. 11-30. Springer, 2015.
Gunjan Verma and Ananthram Swami. Error correcting output codes improve probability estimation and adversarial robustness of deep neural networks. In Advances Neural Information Process- ing Systems, 2019.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient Langevin dynamics. In International Conference Machine Learning, 2011.
Andrew G Wilson, Zhiting Hu, Ruslan R Salakhutdinov, and Eric P Xing. Deep kernel learning. In International Conference on Artificial Intelligence and Statistics, 2016.
Anqi WU, Sebastian Nowozin, Edward Meeds. Richard E Turner, José Miguel Hernández- and Alexander L Gaunt. Deterministic variational inference for robust Bayesian neural networks. In International Conference on Learning Representations, 2019.
12

---

Under review as a conference paper at ICLR 2021
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In IEEE Conference on Computer Vision and Pattern Recog- nition, 2017.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Rep- resentations, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. Mixup: Beyond empiri- cal risk minimization. In International Conference Learning Representations, 2018.
Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson. Cyclical stochastic gradient MCMC for Bayesian deep learning. In International Conference on Learning Representations, 2020.
13

---

Under review as a conference paper at ICLR 2021
A DETAILED EXPERIMENTAL SETUP
ResNet base setup. We trained ResNet for 200 epochs by SGD with momentum coefficient 0.9, minibatch size of 128, and weight decay ratio 0.0001; weights were initialized by He initialization (He et al. 2015); an initial learning rate was 0.1, and decreased by factor of 10 at 100 and 150 epochs. We also used the standard augmentation presented in He et al. (2015).
VGG setup. We trained VGG by re-using the ResNet setup for convenience, except increasing the weight decay ratio 0.0005 as in Simonyan & Zisserman (2015).
BERT setup. Our configuration is based on Kong et al. 2020). Specifically, the model is trained for 5 epochs by Adam optimizer with learning rate 0.00005 and minibatch size of 32.
Hyperparameters. We searched four regularization loss coefficients for each method, and chose best one based on validation set accuracy ( The search spaces were: {0.1, 0.03, 0.01, 0.0033 for I, norm; (0.03, 0.01, 0.003, 0.001) for 1,2 norm; 10.1, 0.03, 0.01, 0.003) } for sliced Wasserstein regularization; (1.0, 0.3, 0.1, 0.03) for PER (10x lower coefficient for CIFAR-10).
Sliced Wasserstein regularization and PER involve the integral over the unit sphere, which is eval- uated by Monte-Carlo approximation. In this paper, we used 256 number of evaluations, following
Joo et al. 2020b).
Table 4: Best hyperparameters for each configuration
Regularizer
VGG-16
VGG-16
ResNet-50
ResNet-50 & CIFAR-100
& CIFAR-10
& CIFAR-100
& CIFAR-10
fW 111
0.01
0.03
0.01
0.01
0.003
0.01
0.003
0.01
SWI(E PER
0.001
0.03
0.001
0.01
0.003
1.0
0.03
1.0
B VGG RESULTS
As consistent with the results of ResNet, all regularization losses improves NLL., ECE, and accuracy (Ta except L regularization on CIFAR- However, the improvements are less significant compared to ResNet because the small capacity of VGG makes the vanilla method produces less confident answers and then less vulnerable to the confidence penalty. This can be inferred from that values of) f ||2 of VGG reduced by almost 50% compared to those of ResNet.
Table 5: Experimental results under various regularization methods. Arrows on the metrics repre- sent the desirable direction. Values represent + a obtained from five repetitions, and all values are rounded to decimal places.
Model & Data
Regularizer
Acc 1
NLL 1
ECE 1
fW llz
VGG-16 & CIFAR-10
Vanilla f
92.97 +0.20
0.35 10.01
4.96 10.16
9.62 10.05
93.07 +0.08
40.01
4.1
6.62 +0.01
f fW
93.06 +0.03
4.58 +0.07
70.03
SW1(upg v)
93.13 #0.06
10.0
1.900
5.4
PER
93.1 +0.17
0.31 +0.01
4.79 +0.13
8.49 +0.03
VGG-16
Vanilla
71.96 10.12
1.400.01
16.9 10.09
22.98 10.31
CIF
72.71 10.17
1.44
11.37 40.18
9.27 10.1
f
72.68 10.16
1.31
10.79 +0.28
9,28 10.11
SWi v)
72.26 #0.23
1..01
12.59 10.34
9.72 40.02
PER
72.89 +0.24
1.34 +0.01
6.47 +0.12
7.37 +0.05
14


Reviews:
[{"review_id": "wySQifgFACp", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main contribution of this paper is to propose new regularization methods in deep neural networks that produce well-calibrated probability scores. The authors argue that regularization is better than post-processing, such as temperature scaling, because temperature scaling would require a separate dataset for calibration. In addition, regularization is added to the loss so it does not alter other components of the neural network. There are two forms of regularization that the authors propose: (1) regularizing in the function space, and (2) in the probability space.  Interestingly, they show that both regularization methods yield well-calibrated scores, which cannot be attributed to minimizing the norm of the weights alone.\n\nWhile the contribution above is interesting, it is unfortunate that the paper digresses to unrelated topics and is quite hard to understand in a few places. Some of its claims can be misleading as well. For example, the title of the paper and its abstract suggest that regularization during training is \"required\" but the authors only show that there exists certain forms of regularization that are *sufficient* but not necessary. In fact, as the authors mention, a simple technique like temperature scaling performs equally well as their proposed method. In addition, some of the claimed contributions are not novel or new. It is known that probability scores of neural networks are not well-calibrated because the cross entropy loss and the expressive power of neural nets lead neural nets to make overconfident predictions on the training sample. So, the first contribution stated in Page 2 is not a new contribution to be precise.\n\nIn Section 2, the authors look into the fact neural nets make confident predictions on the training sample. This entire section is chatty, imprecise, and is not necessary for the main contribution of the paper (which is in Section 3). Some issues with Section 2 include: \n- The authors speak of \"optimal confidence\" without defining it. I originally thought that the optimal confidence of predicting y=1 is itself the true probability p(y=1|x) but Figure 2(c) shows that the authors have a different definition in mind, which they do not explain. \n- The authors use a cyclic argument to explain why neural nets are overly confident on their predictions. In the paragraph below Eq 3, he authors argue that given that *we know* that neural nets perfectly fits the training examples, the upper bound on the loss in Eq 3 suggests that the predictions must match with the true labels. But that's a cyclic argument because you used the conclusion as a premise. \n- There is a lengthy discussion about a \"divisor\" (alpha) that is not defined formally in the paper. \n- In the text, the authors state that the x-axis of the Figure 2(c) is the test accuracy, written as E_{y|x} 1_y(m_x), but the legend says it is the \"correct probability.\" Which one is it? \n- What is LL in Figure 2(a)? I could not find it in the text or the figure caption. \n\nAs mentioned earlier, the primary contribution is Section 3.2. The experimental results are interesting. But, the authors later mention that a simple technique like temperature scaling would achieve similar results. Given that temperature scaling does not require any hyper-parameter to tune, I am not sure if the proposed method would be useful in practice. Unfortunately, the authors do not show how sensitive their results are to their hyper-parameters. \n\nRegarding out-of-distribution detection, I personally do not agree that it should be used as a method for evaluating confidence scores. Suppose the data comes from a mixture of two Gaussians and you learn the mixture using expectation maximization. That algorithm would give good estimates of the probability scores because it estimates the means and covariance matrices of two Gaussian densities. However, if the distribution changes, i.e. we now have a different mixture of Gaussians, then the algorithm would still make many confident predictions that are wrong simply because the distribution has changed. That does not mean the algorithm is not well-calibrated to the distribution it was trained on.\n\nIn summary, the paper makes interesting contributions. It proposes a new method of learning well-calibrated probability scores using new forms of regularization but it is suitable for publication in its present form. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "sxz4vgMOuim", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to improve the \"reliable predictive probability\" when using a single deterministic deep net. The paper proposes explicit regularization methods to achieve this goal.\n\nOverall, I found the paper hard to follow, often unclear in what the goal is, jumping quickly from one point to another one. Also many crucial technical steps seem incorrect or at least not well motivated/discussed. A few examples : \n\n1. What do the authors exactly mean by “reliable” predictive probabilities, which is at the core of this paper? In the introduction, calibration, overconfidence of the model and uncertainty representation (like in Bayesian statistics) are discussed. Does “reliability” refer to all of this at once, or is it some other concept? How is it defined? Given that this is not clear to me, it makes it hard for me to understand if  the paper provides some approaches to improve it.\n\n2. The inequality in Eq 3 could be discussed in some detail. My understanding is that each \\phi_k is replaced by 1-\\phi_mx in the second part, i.e., the maximum possible probability 1-\\phi_mx is used for each misclassification. This seems very extreme to me.  Moreover, these “new probabilities“ do not sum to one any longer: \\phi_mx + (K-1) * (1-\\phi_mx) = 1+ (K-2) * (1-\\phi_mx) >> 1, which grows linearly with the number of classes K (if \\phi_mx <1), which seems weird. I wonder if this extremely loose bound itself causes the high cost of misclassification that is later “derived” as a key problem.\n\n3. It would also be good to discuss why it is useful to consider an upper bound on a quantity that gets maximized (log likelihood). Maximizing a lower bound would seem more natural.\n\n4. The divisor \\alpha_x,y is introduced in Eq 3 as a mechanism to achieve equality. As I understand Eq 3, however, \\alpha_x,y cannot be a function of y at the location where it is introduced in Eq 3, as it is  outside of the expectation E_y|x. It would be good to clarify this before deriving further insights.\n\n5. The L2-norm of function f^W is discussed in several places before it is defined in Eq 5.\n\n6. Below Eq 4, \\tau^\\star is defined in a maximization problem. However,  \\tau can be moved outside of the expectation: E[ \\log(\\phi/\\tau)]=  E[\\log(\\phi)] - \\log(\\tau), hence this is trivially maximized when \\tau approaches 0, which does not make sense to me.\n\nApart from that, also the language of the paper could be greatly improved.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "4koJmpQ62l", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "There has been an ongoing debate about the role and importance of explicit and implicit regularization in deep learning. This paper attempts to address this issue by arguing that explicit regularization is required for the generalization of predictive probabilities, which may not be observed under the 0-1 loss. The paper provides some discussion and numerical evidence to support the claim.\n\nAlthough the paper makes some interesting remarks, the idea of looking at the predictive probability rather than classification error is certainly not new. This may be part of the story, but is unlikely to a central one. In particular, this does not explain why overparametrized models may still generalize well in the regression setting. The only theoretical analysis in the paper is based on eq. (3), which is almost trivial and does not reflect any characteristic of deep models. I find the arguments totally heuristic and not convincing enough to justify the necessity of explicit regularization. In terms of methodology, the paper examines only complexity control through weight decay and two existing explicit regularizers and does not propose any new regularization strategy.\n\nMinor comments:\n1) The predictive probability should not be referred to as “predictive confidence.” In particular, “level of predictive confidence” and “confidence control” may be confused with the standard use of “confidence” in “level of confidence” and “confidence interval.”\n\n2) The coined terms “stochastic switch” and “deterministic score” are also misleading. The score usually means the derivative of the log-likelihood.\n\nUpdate: I appreciate the authors’ response and, in particular, providing more explanations about eq. (3) (now eq. (5)). However, the explanations are still largely heuristic and based on unproven claims. To reflect the authors’ efforts to improve the paper, I am increasing my rating from 3 to 4.", "extra_scores": {}, "extra_text": {}}, {"review_id": "2yfWfAHjwz6", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies how explicit regularization affects the reliable predictive probability (calibration) of classification tasks. An analysis of the log-likelihood is presented which motivates the use of explicit regularization. Next, two regularization terms are proposed to improve the predictive probability, and experiments on CIFAR – 10/100 show that these regularization terms improve both the accuracy and the calibration error.\nI think that studying how different regularization techniques affects calibration is an interesting direction. The empirical results of this paper look promising and show that indeed adding explicit regularization terms improves both the accuracy and the calibration.\nI do have some concerns regarding the theoretical analysis, and the experimental setting:\n1) Regarding section 2 (theoretical analysis), I am not sure how this analysis motivates that the cause of unreliable predictive probability is due to the lack of regularization. In more details:\na. The analysis of the log-likelihood is very unclear. How is the inequality of eq (3) turned into an equality? What is the \\alpha_{x,y} term, is there a formula that defines it? \nb. It is claimed that improving the expectation in the third sentence of page 4 is challenging due to many complex factors, therefore it is not directly connected to improving this expectation on unseen samples. Why is it true?\nc. The temperature scaling method introduced there is used to improve calibration, how it is related to explicit regularization?  \nd. What is the bottom line of this section? In other words, how the analysis presented there motivates that what should help the calibration is explicit regularization (which seems to be the main claim of the paper)? \n2) I find the experimental setup a bit lacking in two aspects:\na. The experiments are done only on CIFAR – 10 and CIFAR – 100. Since this is mainly an empirical paper, I would expect that the claims would be tested on more datasets, especially the experiments presented in Table 1. \nb. The only method that is compared to the proposed regularization terms is Vanilla training. It is hard to evaluate this way whether the proposed regularization terms are really what best improves the calibration. I think it would be better to compare in Table 1 to other methods such as L_1 regularization, dropout, temperature scaling, or other known regularization/calibration techniques.\n\nTo conclude, I suggest widening the experimental setting of the paper by testing on more datasets and comparing it to other known techniques, otherwise, it is not clear whether the techniques proposed in this paper really improve on other known techniques. Regarding the theoretical analysis, I suggest making it clearer by adding rigorous theorems/propositions and explicitly stating how the analysis suggests that explicit regularization can improve the calibration.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "-b7cJnuUZAX", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update: After reading the other reviews/responses, I think there are persistent concerns with the breadth of experiments and the substantiveness of the contribution; although the manuscript is somewhat improved by the authors' updates, I'm keeping my score at 5.\n\nThis paper examines the effect of explicit regularization on the quality of predicted probabilities from neural network classifiers. Empirical results show that explicit regularization substantially improves predictive uncertainty on CIFAR10/100, as measured by Negative Log Likelihood and Expected Calibration Error. Additional results demonstrate the effectiveness of explicit regularization on separating out-of-distribution (OOD) data, with CIFAR as in-distribution and SVHN as OOD.\n\nThe paper is well-grounded theoretically, and well-written and organized. The experiments show convincing results on the ability of explicit regularization to improve the quality of uncertainty of neural network classifiers. The baselines chosen are sensible (temperature scaling for an in-distribution test set, ensembles/MC dropout/temperature scaling for the OOD tasks).\n\nThe datasets and modalities tested are very limited, and the paper would be stronger if it showed experimental results on additional datasets besides CIFAR and SVHN, and additional modalities besides images. In particular, Nalisnick et al. 2018 (“Do deep generative models know what they don’t know?”) showed that models trained on CIFAR assign high likelihood to examples from SVHN, which I think calls into question how well results on this pair of datasets generalize.\n\nThe entropy diagrams in Figure 4 give a good qualitative indication of OOD detection, but the results would be strengthened by a quantitative measure such as AUROC/AUPRC. Also, Figure 4 is missing a legend.\n\nI believe the title overreaches a little -- the results of the paper are interesting and compelling, but they fall short of demonstrating that regularization in deep learning models is “required” for reliable predictive probability.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary of reviews and discussions: Reviewers were overwhelmingly negative on this paper due to a variety of factors: unclear writing, heuristic motivation, overpromising in the title while underdelivering on results. Although the authors responded to the reviewers' feedback, and some reviewers increased their score to acknowledge the author's work, they remained unconvinced overall and no reviewer argued strongly for acceptance.
Decision: Reject