Revisiting Explicit Regularization in Neural Networks for Reliable Predictive Probability

1 INTRODUCTION . As deep learning models have become pervasive in real-world decision-systems , the importance of producing a reliable predictive probability is increasing . In this paper , we call predictive probability reliable if it is well-calibrated and precisely represents uncertainty about its predictions . The calibrated behavior refers to the ability to match its predictive probability of an event to the longterm frequency of the event occurrence ( Dawid , 1982 ) . The reliable predictive probability benefits many downstream tasks such as anomaly detection ( Malinin & Gales , 2019 ) , classification with rejection ( Lakshminarayanan et al. , 2017 ) , and exploration in reinforcement learning ( Gal & Ghahramani , 2016 ) . More importantly , deep learning systems with more reliable predictive probability can provide better feedback for explaining what is going on , situations when its prediction becomes uncertain , and unexpected anomalies to users . Unfortunately , neural networks are prone to be overconfident and lack uncertainty representation ability , and this problem has become a fundamental concern in the deep learning community . Bayesian methods have innate abilities to produce reliable predictive probability . Specifically , they express the probability distribution over parameters , in which uncertainty in the parameter space is automatically determined by data ( MacKay , 1992 ; Neal , 1993 ) . Then , uncertainty in prediction can be represented by means of providing rich information about aggregated predictions from different parameter configurations such as entropy and mutual information . From this perspective , deterministic neural networks selecting a single parameter configuration that can not provide such rich information naturally lack the uncertainty representation ability . However , the automatic determination of parameter uncertainty in the light of data , i.e. , posterior inference , comes with prohibitive computational costs . Therefore , the mainstream approach for improving the predictive probability quality has been an efficient adoption of the Bayesian principle into neural networks ( Gal & Ghahramani , 2016 ; Ritter et al. , 2018 ; Teye et al. , 2018 ; Joo et al. , 2020a ) . Recent works ( Lakshminarayanan et al. , 2017 ; Müller et al. , 2019 ; Thulasidasan et al. , 2019 ) has discovered the hidden gems of label smoothing ( Szegedy et al. , 2016 ) , mixup ( Zhang et al. , 2018 ) , and adversarial training ( Goodfellow et al. , 2015 ) , which improve the calibration performance and the uncertainty representation ability . These findings present a new possibility of improving the reliability of the predictive probability without changing the deterministic nature of neural networks . This direction is appealing because it can be applied in a plug-and-play fashion to the existing building blocks . This means that they can inherit the scalability , computational efficiency , and surprising generalization performance of the deterministic neural networks , for which Bayesian neural networks often struggle ( Wu et al. , 2019 ; Osawa et al. , 2019 ; Joo et al. , 2020a ) . Motivated by these observations , we investigate a general direction from the regularization perspective to mitigate the unreliable predictive probability problem , rather than proposing new constructive heuristics or discovering hidden properties of specific methods . Our main contribution is twofold . First , we present a new direction for alleviating the unreliable predictive behavior , which is readily applicable , computationally efficient , and scalable to large-scale models compared to Bayesian neural networks or ensemble methods . Second , our findings provide a novel view of the role of explicit regularization in deep learning , which improves the reliability of the predictive probability . 2 ANALYZING THE CAUSE OF UNRELIABLE PREDICTIVE PROBABILITY . 2.1 BACKGROUND . We consider a classification problem with i.i.d . training samples D = { ( x ( i ) , y ( i ) ) } N i=1 drawn from unknown distributions Px , y whose corresponding tuple of random variables is ( x , y ) . We denote X as an input space and Y as a set of categories { 1 , 2 , · · · , K } . Let fW : X → Z be a neural network with parameters W where Z = RK is a logit space . On top of the logit space , the softmax σ : RK →4K−1 normalizes the exponential of logits : φWk ( x ) = exp ( fWk ( x ) ) ∑ i exp ( f W i ( x ) ) ( 1 ) where we let φWk ( x ) = σk ( f W ( x ) ) for brevity . σk ( fW ( x ) ) is often interpreted as the predictive probability that the label of x belongs to class k ( Bridle , 1990 ) The probabilistic interpretation of neural network outputs gives the natural minimization objective for classification–the cross-entropy between the predictive probability and the one-hot encoded label : lCE ( y , φW ( x ) ) = − ∑ k 1y ( k ) log φ W k ( x ) , where 1A ( ω ) is an indicator function taking one if ω ∈ A and zero otherwise . By minimizing the cross-entropy ( or equivalently maximizing the log-likelihood ) with stochastic gradient descent ( SGD ) ( Robbins & Monro , 1951 ) or its variants , modern neural networks achieve the surprising generalization performance . As the demand for neural networks in real-world decision-making is emerging , reliable predictive probability has been of interest in the machine learning community . One important quality of predictive probability is calibrated behavior . Specifically , based on the notion of calibration in classical forecasting problem ( Dawid , 1982 ) , the perfectly calibrated model can be defined as follows : p ( y = k|φW ( x ) = p ) = pk , ∀p ∈ 4K−1 , k ∈ { 1 , 2 , · · · , K } ( 2 ) Here note that the calibrated model does not necessarily be ones producing φWk ( x ) = p ( y = k|x ) . In practice , expected calibration error ( ECE ) ( Naeini et al. , 2015 ) is widely used for calibration performance measure . ECE on dataset DT can be computed by binning predictions into M -groups based on their confidences1 and then averaging their calibration scores by : M∑ i=1 |Gi| |DT | |acc ( Gi ) − conf ( Gi ) | ( 3 ) where Gi = { x : i/M < maxk φ W k ( x ) ≤ ( 1 + i ) /M , x ∈ DT } ; acc ( Gi ) and conf ( Gi ) are average accuracy and confidence of predictions in group Gi , respectively . 1Throughout this paper , the confidence ( or predictive confidence ) at x refers to maxk φWk ( x ) , which is different from the confidence in statistics literature . Another metric for evaluating the reliability of predictive probability is based on predictive entropy , which evaluates how well is the model aware of its ignorance . To this end , predictive entropy is measured on samples that the model is ignorant of , such as misclassified or out-of-distribution ( OOD ) samples . For the classifier having reliable predictive probability , we expect the high uncertainty of φW ( x ) on such samples , i.e. , the answer “ I don ’ t know. ” However , several recent findings show that the resulting neural network produces unreliable predictive probability , which interprets the softmax output as the “ predictive probability ” implausible ( Gal & Ghahramani , 2016 ) . For example , Figure 1 illustrates the unreliable predictive behavior of ResNet ( He et al. , 2016 ) : the network produces outputs with high confidence in misclassified examples ( Figure 1 ( upper ) ) and provides low predictive entropy on out-of-distribution samples , albeit the samples belong to none of the classes it learned ( Figure 1 ( lower ) ) . One simple yet effective solution for improving the predictive probability ’ s quality is temperature scaling ( Guo et al. , 2017 ) , which adjusts the smoothness of the softmax so that the resulting predictive probability maximizes the log-likelihood on unseen dataset D′ : max τ ∑ ( x , y ) ∈D′ log exp ( fWy ( x ) /τ ) ∑ j exp ( f W j ( x ) /τ ) ( 4 ) where W is a fixed pretrained weight and τ is a temperature controlling the smoothness of the softmax output . This simple method makes the softmax output more reliable predictive probability . For instance , the predictive confidence matches its actual accuracy well , and the predictive entropy on out-of-distribution samples significantly increases ( Figure 1 ) . 2.2 A CLOSER LOOK AT THE LOG-LIKELIHOOD ON UNSEEN SAMPLES .. Motivated by the success of the temperature scaling , we aim to find the relationship between the loglikelihood and the calibration performance . To this end , let s be a binary random variable indicating whether a model correctly classifies a sample . Then , we can derive the following upper bound by using the law of total expectation : Ex , y [ log φWy ( x ) ] = Ex [ ps|x ( s = 1 ) Ey|s=1 , x [ log φWy ( x ) ] + ps|x ( s = 0 ) Ey|s=0 , x [ log φWy ( x ) ] ] ≤ Ex [ ps|x ( s = 1 ) log φ W mx ( x ) + ps|x ( s = 0 ) log ( 1− φ W mx ( x ) ) ] ( 5 ) where mx is the predictive class such that mx = argmaxk f W k ( x ) , ps|x ( s = 1 ) = p ( y = argmaxk φ W mk ( x ) |x ) , and the inequality comes from the fact that Ey|s=0 , x [ log φWy ( x ) ] ≤ maxk 6=mx log φ W k ( x ) ≤ log ( 1− φWmx ( x ) ) . The upper bound can be thought of as the probabilistic measure of the calibration performance . Specifically , suppose a neural network produces an answer with probability φWmx ( x ) and refuses to answer otherwise . Then , the upper bound becomes the divergence between the model ’ s ability to correctly predict the sample and the model ’ s willingness to answer . When we consider the inequality in equation 5 is applied to the empirical mean on the test dataset , this inequality clearly explains why the temperature scaling is helpful ; it increases a lower bound of the calibration error with modified confidence by τ closer to its accuracy . More importantly , the inequality in equation 5 explains the impacts of the cross-entropy minimization on the behavior of the neural network . Specifically , suppose E ( x , y ) ∼D [ log φWy ( x ) ] → 0 for some dataset D. Then , it can be shown that ps|x ( s = 1 ) → 1 and φWmx ( x ) → 1 for all ( x , y ) ∈ D. This holds because we have mink φWk ( x ) ≥ 1/K for any x and the minimum of p log q + ( 1 − p ) log ( 1 − q ) are ( p , q ) → ( 1 , 1 ) or ( p , q ) → ( 0 , 0 ) . Therefore , for high-capacity models that can make the log-likelihood on the training dataset Dtr close to zero , e.g. , deep neural networks , its behavior on the training set will converge to a configuration that corrects all samples with perfect confidence . Figure 2 illustrates this phenomenon in ResNet trained on CIFAR-100 : Ex∼Dtr [ log φWmx ] → 0 ( a ) and E ( x , y ) ∼Dtr [ ps|x ( s = 1 ) ] → 1 ( b ) as the training continues . Why are these convergences problematic ? We observe that function ’ s properties , which depends only on x , evaluated on two different datasets are very close to each other if they are drawn from the same distribution , unlike values that depend on the external randomness y|x ; that is , |Ex∼Dtr [ g ( x ) ] − Ex′∼Dval [ g ( x′ ) ] | |E ( x , y ) ∼Dtr [ h ( x , y ) ] − E ( x′ , y′ ) ∼Dval [ h ( x′ , y′ ) ] | for some functions g ( · ) and h ( · ) . For example , the empirical mean of the maximum log-probability log φWmx ( x ) ( Figure 2 ( a ) ) and L2 norm of fW ( Figure 2 ( c ) ) 2 on training samples are significantly similar to those values on unseen samples compared to the log-likelihood ( Figure 2 ( a ) ) and the accuracy ( Figure 2 ( b ) ) . We conjecture that the log-likelihood maximization with a high-capacity neural network naturally results in a high calibration error on unseen samples due to this discrepancy ; that is , it produces perfect confidence on unseen samples as it does on the training samples where it can not produce the perfect accuracy . Fortunately , this result also indicates that restricting the predictive confidence φWmx ( x ) on training samples will directly impact the predictive confidence on unseen samples and therefore can reduce the calibration error . For this reason , we explore various ways to restrict confidence and show their efficacy on a wide range of tasks for the rest of the paper .

Summary: The main contribution of this paper is to propose new regularization methods in deep neural networks that produce well-calibrated probability scores. The authors argue that regularization is better than post-processing, such as temperature scaling, because temperature scaling would require a separate dataset for calibration. In addition, regularization is added to the loss so it does not alter other components of the neural network. There are two forms of regularization that the authors propose: (1) regularizing in the function space, and (2) in the probability space.  Interestingly, they show that both regularization methods yield well-calibrated scores, which cannot be attributed to minimizing the norm of the weights alone.