PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

1 INTRODUCTION . A key challenge facing deep neural networks is that they do not produce reliable confidence estimates , which are important for applications such as safe reinforcement learning ( Berkenkamp et al. , 2017 ) , guided exploration ( Malik et al. , 2019 ) , and active learning ( Gal et al. , 2017 ) . We consider the setting where the test data follows the same distribution as the training data ( i.e. , we do not consider adversarial examples designed to fool the network ( Szegedy et al. , 2014 ) ) ; even in this setting , confidence estimates produced by deep neural networks are notoriously unreliable ( Guo et al. , 2017 ) . One intuition for this shortcoming is that unlike traditional supervised learning algorithms , deep learning models typically overfit the training data ( Zhang et al. , 2017 ) . As a consequence , the confidence estimates of deep neural networks are flawed even for test data from the training distribution since , by construction , they overestimate the likelihood of the training data . A promising approach to addressing this challenge is temperature scaling ( Platt , 1999 ) . This approach takes as input a trained neural network fφ̂ ( y | x ) —i.e. , whose parameters φ̂ have already been fit to a training dataset Ztrain—which produces unreliable probabilities fφ̂ ( y | x ) . Then , this approach rescales these confidence estimates based on a validation dataset to improve their “ calibration ” . More precisely , this approach fits confidence estimates of the form fφ̂ , τ ( y | x ) ∝ exp ( τ log fφ̂ ( y | x ) ) , where τ ∈ R > 0 is a temperature scaling parameter that is fit based on the validation dataset . The goal is to choose τ to minimize calibration error , which roughly speaking measures the degree to which the reported error rate differs from the actual error rate . The key insight is that in the temperature scaling approach , only a single parameter τ is fit to the validation data—thus , unlike fitting the original neural network , the temperature scaling algorithm comes with generalization guarantees based on traditional statistical learning theory . Despite the improved generalization guarantees , these confidence estimates still do not come with theoretical guarantees . We are interested in producing confidence sets that satisfy statistical guarantees while being as small as possible . Given a test input x ∈ X , a confidence set CT ( x ) ⊆ Y 1Our code is available at https : //github.com/sangdon/PAC-confidence-set . ( parameterized by T ∈ R ) should contain the true label y for at least a 1− fraction of cases : P ( x , y ) ∼D [ y ∈ CT ( x ) ] ≥ 1− . Since we are fitting a parameter T to based on Zval , we additionally incur a probability of failure due to the randomness in Zval . In other words , given , δ ∈ R > 0 , we aim to obtain probably approximately correct ( PAC ) confidence sets CT ( x ) ⊆ Y satisfying the guarantee PZval∼Dn ( P ( x , y ) ∼D ( y ∈ CT ( x ) ) ≥ 1− ) ≥ 1− δ . Indeed , techniques from statistical learning theory ( Vapnik , 1999 ) can be used to do so ( Vovk , 2013 ) . There are a number of reasons why confidence sets can be useful . First , they can be used to inform safety critical decision making . For example , consider a doctor who uses prediction tools to help perform diagnosis . Having a confidence set would both help the doctor estimate the confidence of the prediction ( i.e. , smaller confidence sets imply higher confidence ) , but also give a sense of the set of possible diagnoses . Second , having a confidence set can be useful for reasoning about safety since they contain the true outcome with high probability . For instance , robots may use a confidence set over predicted trajectories to determine whether it is safe to act with high probability . As a concrete example , consider a self-driving car that uses a deep neural network to predict the path that a pedestrian might take . We require that the self-driving car avoid the pedestrian with high probability , which it can do by avoiding all possible paths in the predicted confidence set . Contributions . We propose an algorithm combining calibrated prediction and statistical learning theory to construct PAC confidence sets for deep neural networks ( Section 3 ) . We propose instantiations of this framework in the settings of classification , regression , and learning models for reinforcement learning ( Section 3.6 ) . Finally , we evaluate our approach on three benchmarks : ResNet ( He et al. , 2016 ) for ImageNet ( Russakovsky et al. , 2015 ) , a model ( Held et al. , 2016 ) learned for a visual object tracking benchmark ( Wu et al. , 2013 ) , and a probabilistic dynamics model ( Chua et al. , 2018 ) learned for the half-cheetah environment ( Brockman et al. , 2016 ) ( Section 4 ) . Examples of ImageNet images with different sized ResNet confidence sets are shown in Table 1 . As can be seen , our confidence sets become larger and the images become more challenging to classify . In addition , we show predicted confidence sets for ResNet in Table 2 , as well as predicted confidence sets for the visual object tracking model in Table 3 . Related work . There has been work on constructing confidence sets with theoretical guarantees . Oftentimes , these guarantees are asymptotic rather than finite sample ( Steinberger & Leeb , 2016 ; 2018 ) . Alternatively , there has been work focused on predicting confidence sets with a given expected size ( Denis & Hebiri , 2017 ) . More relatedly , there has been recent work on obtaining PAC guarantees . For example , there has been some work specific prediction tasks such as binary classification ( Lei , 2014 ; Wang & Qiao , 2018 ) . There has also been work in the setting of regression ( Lei et al. , 2018 ; Barber et al. , 2019 ) . However , in this case , the confidence sets are fixed in size—i.e. , they do not depend on the input x ( Barber et al. , 2019 ) . Furthermore , they make stability assumptions about the learning algorithm ( though they achieved improved rates by doing so ) ( Lei et al. , 2018 ; Barber et al. , 2019 ) . The most closely related work is on conformal prediction ( Papadopoulos , 2008 ; Vovk , 2013 ) . Like our approach , this line of work provides a way to construct confidence sets from a given confidence predictor , and provided PAC guarantees for the validity of these confidence sets . Indeed , with some work , our generalization bound Theorem 1 can be shown to be equivalent to Theorem 1 in Vovk ( 2013 ) . In contrast to their approach , we proposed to use calibrated prediction to construct confidence predictors that can suitably be used with deep neural networks . Furthermore , our approach makes explicit the connections to temperature scaling and as well as to generalization bounds from statistical learning theory ( Vapnik , 1999 ) . In addition , unlike our paper , they do not explicitly provide an efficient algorithm for constructing confidence sets . Finally , we also propose an extension to the case of learning models for reinforcement learning . Finally , we build on a long line of work on calibrated prediction , which aims to construct “ calibrated ” probabilities ( Murphy , 1972 ; DeGroot & Fienberg , 1983 ; Platt , 1999 ; Zadrozny & Elkan , 2001 ; 2002 ; Naeini et al. , 2015 ; Kuleshov & Liang , 2015 ) . Roughly speaking , probabilities are calibrated if events happen at rates equal to the predicted probabilities . This work has recently been applied to obtaining confidence estimates for deep neural networks ( Guo et al. , 2017 ; Kuleshov et al. , 2018 ; Pearce et al. , 2018 ) , including for learned models for reinforcement learning ( Malik et al. , 2019 ) . However , these approaches do not come with PAC guarantees . 2 PAC CONFIDENCE SETS . Our goal is to estimate confidence sets that are as small as possible , while simultaneously ensuring that they are probably approximately correct ( PAC ) ( Valiant , 1984 ) . Essentially , a confidence set is correct if it contains the true label . More precisely , let X be the inputs and Y be the labels , and let D be a distribution over Z = X × Y . A confidence set predictor is a function C : X → 2Y such that C ( x ) ⊆ Y is a set of labels ; we denote the set of all confidence set predictors by C. For a given example ( x , y ) ∼ D , we say C is correct if y ∈ C ( x ) . Then , the error of C is L ( C ) = P ( x , y ) ∼D [ y 6∈ C ( x ) ] . ( 1 ) Finally , consider an algorithm A that takes as input a validation set Zval ⊆ Z consisting of n i.i.d . samples ( x , y ) ∼ D , and outputs a confidence set predictor Ĉ . Given , δ ∈ R > 0 , we say that A is probably approximately correct ( PAC ) if PZval∼Dn [ L ( Ĉ ) > where Ĉ = A ( Zval ) ] < δ . ( 2 ) Our goal is to design an algorithm A that satisfies ( 2 ) while constructing confidence sets C ( x ) that are as “ small in size ” as possible on average . The size of C ( x ) depends on the domain . For classification , we consider confidence sets that are arbitrary subsets of labels C ( x ) ⊆ Y = { 1 , ... , Y } , and we measure the size by |C ( x ) | ∈ N—i.e. , the number of labels in C ( x ) . For regression , we consider confidence sets that are intervals C ( x ) = [ a , b ] ⊆ Y = R , and we measure size by b − a—i.e. , the length of the predicted interval . Note that there is an intrinsic tradeoff between satisfying ( 2 ) and average size of C ( x ) —larger confidence sets are more likely to satisfy ( 2 ) . 3 PAC ALGORITHM FOR CONFIDENCE SET CONSTRUCTION . Our algorithm is formulated in the empirical risk framework . Typically , this framework refers to empirical risk minimization . In our setting , such an algorithm would take as input ( i ) a parametric family of confidence set predictors C = { Cθ | θ ∈ Θ } , where Θ is the parameter space , and ( ii ) a training set Zval ⊆ Z of n i.i.d . samples ( x , y ) ∼ D , and output the confidence set predictor Cθ̂ , where θ̂ minimizes the empirical risk : θ̂ = arg min θ∈Θ L̂ ( Cθ ; Zval ) where L̂ ( C ; Zval ) = 1 n ∑ ( x , y ) ∈Zval I [ y 6∈ C ( x ) ] . Here , I [ φ ] ∈ { 0 , 1 } is the indicator function , and the empirical risk L̂ in an estimate of the confidence set error ( 1 ) based on the validation set Zval . However , our algorithm does not minimize the empirical risk . Rather , recall that our goal is to minimize the size of the predicted confidence sets given a PAC constraint on the true risk L ( θ̂ ) based on the given PAC parameters , δ ∈ R > 0 and the number of available validation samples n = |Zval| . Thus , the risk shows up as a constraint in the optimization problem , and the objective is instead to minimize the size of the predicted confidence sets : θ̂ = arg min θ∈Θ S ( θ ) subj . to L̂ ( Cθ ; Zval ) ≤ α . ( 3 ) At a high level , the value α = α ( n , , δ ) ∈ R≥0 is chosen to enforce the PAC constraint , and is based on generalization bounds from statistical learning theory ( Valiant , 1984 ) . Furthermore , following the temperature scaling approach ( Platt , 1999 ) , the parameter space Θ is chosen to be as small as possible ( in particular , one dimensional ) to enable good generalization . Finally , our choice of size metric S follows straightforwardly based on our choice of parameter space . In the remainder of this section , we describe the choices of ( i ) parameter space Θ , ( ii ) size metric S ( θ ) , and ( iii ) confidence level α ( n , , δ ) in more detail , as well as how to solve ( 3 ) given these choices .

Summary: This paper propose to estimate the confidence sets for deep neural networks with PAC guarantees by combining calibrated prediction and generalization bounds from learning theory. Reliable confidence set estimation can be very useful for applications in safe-critical domains, and a paper advancing knowledge in this area is certainly welcome. The paper is clearly presented and well motivated. In addition to give a direct generalization bound that trades off between realizable setting and the VC bound, it also proposes an algorithm for predict the confidence set in practice. Experimental results in image classification and reinforcement learning illustrate the effectiveness of the proposed algorithm. One thing that may need further clarify is that the derived bound and algorithm is general in that it can be applied to any estimator f_\theta using 4 and 5. What kind of structural characteristic of neutral nets that are explicitly exploited in the algorithm in particular? Will this consideration further tighten the bound? Small typo: under Equation 5, Z^\prime_{train} is oftentimes the Z_{val}.