Locally Valid and Discriminative Prediction Intervals for Deep Learning Models

Crucial for building trust in deep learning models for critical real-world applications is efficient and theoretically sound uncertainty quantification , a task that continues to be challenging . Useful uncertainty information is expected to have two key properties : It should be valid ( guaranteeing coverage ) and discriminative ( more uncertain when the expected risk is high ) . Moreover , when combined with deep learning ( DL ) methods , it should be scalable and affect the DL model performance minimally . Most existing Bayesian methods lack frequentist coverage guarantees and usually affect model performance . The few available frequentist methods are rarely discriminative and/or violate coverage guarantees due to unrealistic assumptions . Moreover , many methods are expensive or require substantial modifications to the base neural network . Building upon recent advances in conformal prediction [ 13 , 33 ] and leveraging the classical idea of kernel regression , we propose Locally Valid and Discriminative prediction intervals ( LVD ) , a simple , efficient and lightweight method to construct discriminative prediction intervals ( PIs ) for almost any DL model . With no assumptions on the data distribution , such PIs also offer finite-sample local coverage guarantees ( contrasted to the simpler marginal coverage ) . We empirically verify , using diverse datasets , that besides being the only locally valid method for DL , LVD also exceeds or matches the performance ( including coverage rate and prediction accuracy ) of existing uncertainty quantification methods , while offering additional benefits in scalability and flexibility . 1 Introduction . Consider a training set Strain = { ( Xi , Yi ) } Ni=1 and a test example ( XN+1 , YN+1 ) , all drawn i.i.d from an arbitrary joint distribution P , with ( Xi , Yi ) ∈ X × Y for some X ⊆ Rd and Y ⊆ R. We are interested in the problem of predictive inference : On observing Strain and XN+1 , our task is to construct a prediction interval ( PI ) 1 estimate Ĉ ( XN+1 ) that contains the true value of YN+1 with a ( pre-specified ) high probability . 1Several recent deep learning papers use “ Confidence Interval ” and “ Prediction Interval ” interchangeably . We stick to the conventional statistical usage . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The construction of actionable PIs involves two general challenges : First , Ĉ should be valid , meaning that if the specified probability is 1− α , we expect Ĉ ( XN+1 ) to cover YN+1 at least 1− α of the time . Moreover , Ĉ should be discriminative i.e. , we expect Ĉ ( XN+1 ) to be narrower for confident cases and vice-versa . The width of the PI Ĉ ( XN+1 ) is thus quantification of the uncertainty . Figure 1 illustrates these notions , with more details in Section 2.2 and 2.3 . While deep learning ( DL ) models have demonstrated impressive performance over a range of complicated tasks and data modalities , it has remained difficult to quantify the uncertainty for their predictions . For DL predictions to be actionable , uncertainty information is however indispensable , especially in domains like medicine and finance [ 2 ] . Apart from requiring validity and discrimination as discussed earlier , two additional challenges exist specifically for DL models . Obviously , any uncertainty estimation method needs to finish reasonably fast to be useful , so the third challenge is scalability . The fourth challenge is accuracy : The uncertainty estimation should not decrease the prediction accuracy of the DL model . Post-hoc methods are ideal because they usually do not interfere with the base NN prediction at all . These four requirements together constitute a set of essential desiderata for uncertainty quantification in DL . Existing uncertainty estimation methods for DL rarely address more than one or two of the above requirements . Credible intervals given by posteriors of approximate Bayesian methods such as [ 43 , 15 ] , deep ensemble [ 20 , 44 ] and Monte-Carlo dropout [ 12 ] are not valid in the frequentist sense [ 7 ] . Most existing methods also interfere with the original model design , loss function and/or training , which could be expensive and decrease the model performance ( as verified in our experiments ) [ 12 , 1 , 7 , 20 ] . To address these requirements , we leverage recent advances in conformal prediction and the classical idea of kernel regression . Conformal prediction , pioneered by Vovk [ 40 ] , is a powerful approach for constructing valid PIs . The most popular split conformal methods usually leverage prediction errors from a hold-out set to construct Ĉ ( XN+1 ) , which would be valid if a future data point ( XN+1 , YN+1 ) follows the same distribution as data in the hold-out set . This framework is particularly suitable for deep learning due to its distribution-free nature , and has motivated many recent uncertainty quantification efforts in deep learning for both classification and regression tasks [ 1 , 23 , 3 , 11 ] . However , most conformal methods are only marginally valid [ 25 , 40 , 21 , 4 ] . Moreover , less-thanmeticulous applications to DL can break distributional assumptions and theoretical validity , as in the case of [ 1 ] ( see discussion in Appendix ) . We however seek to construct a PI conditioning on the input ( similar to the third PI in Fig . 1 ) . Some recent advances ( [ 13 , 33 ] ) examine the possibility of “ approximately ” conditionally valid PIs . While these methods can not be directly applied to DL due to efficiency and performance considerations , their methodological and theoretical contributions serve as major inspirations for us to develop a highly flexible and practical method . Summary of Contributions : We propose Locally Valid Discriminative Prediction Intervals ( LVD ) , a simple uncertainty estimation method for deep learning which combines recent advances in conformal prediction and the classical idea of kernel regression . LVD applies to almost all DL models , and is the first method that satisfies all four aforementioned requirements : • Validity : LVD has frequentist coverage guarantee ( not just marginal , but approximately conditional ) . • Discrimination : The width of the PIs given by LVD adapts to the risk/uncertainty level of XN+1 . • Scalability : LVD is lightweight , adding limited overhead to the base DL model . • Accuracy : LVD is post-hoc without requiring model retraining , and does not affect the base performance of the DL model . We must note that while the theoretical foundation for guaranteeing `` validity '' is mostly based on [ 13 ] , LVD addresses several challenges to satisfy the other three requirements . The code to replicate all our results can be found at https : //github.com/zlin7/LVD . 2 Preliminaries . 2.1 Learning Setup and Assumptions . We assume data and response pairs ( X , Y ) ∈ X × Y have a joint distribution denoted P , with the marginal distributions of Y and X and the conditional distribution Y |X denoted as PY , PX , and PY |X , respectively . Further , we will define Zi : = ( Xi , Yi ) for concision . Assuming that we already have an algorithm ( with all the training protocols folded in ) , such as a Deep Neural Network ( DNN ) , that provides a mean estimator µ̂NN ( x ) : X 7→ Y . Given a target coverage level 1− α ∈ ( 0 , 1 ) , our task is to also construct a prediction interval estimator function Ĉα ( x ) : X 7→ { subset of Y } that has the validity and discrimination properties as defined below . 2.2 Validity ( Frequentist Coverage ) . There are several ( related ) notions for a PI to be valid – marginal , conditional , and local . Given target level 1−α , we say Ĉα has the marginal coverage guarantee ( or , equivalently , is marginally valid ) if P { YN+1 ∈ Ĉα ( XN+1 ) } ≥ 1− α ( 1 ) where the probability is taken over the training data and ( the unseen ) ( XN+1 , YN+1 ) . A limitation of marginal coverage is that it is not conditioned on XN+1 . A more desirable , albeit stronger , property would be conditional coverage at 1− α : P { YN+1 ∈ Ĉα ( XN+1 ) |XN+1 = x } ≥ 1− α for almost all x ∈ X . ( 2 ) Here the probability is taken over the training data and YN+1 ( with XN+1 fixed ) . It is thus clear that conditional coverage implies marginal coverage but not the other way around . Indeed , a Ĉα with marginal coverage property only implies a 1 − α chance of being accurate on average across all data points ( marginalizing over XN+1 ) i.e . there might be a sub-population in the data for which the coverage is completely missed . Unfortunately , it is impossible to achieve distribution-free finitesample conditional coverage ( Eq . 2 ) in a non-trivial way . Indeed , it is known that a finite-sample estimated Ĉα ( x ) can not achieve conditional coverage , unless it produces infinitely wide prediction intervals in expectation under any non-discrete distribution P [ 39 , 22 , 5 ] . It is thus reasonable to instead seek approximate conditional coverage . As might be apparent , there is considerable freedom in defining an appropriate notion of “ approximate ” , depending on specific tasks and domains . However , a sufficiently general-purpose and natural notion involves using a kernel function K : X × X 7→ R and a center x′ ∈ X , like the relaxation given in [ 33 ] : ∫ P { YN+1 ∈ Ĉα ( x′ ) |XN+1 = x } K ( x , x′ ) dPX ( x ) ∫ K ( x , x′ ) dPX ( x ) ≥ 1− α ( 3 ) with the probability ( P in the integral ) taken over all training samples and YN+1 , with ( XN+1 , YN+1 ) ∼ P̃ = P̃X × PY |X . Here P̃X is just the distribution re-weighted by the kernel with a center x′ , defined by dP̃X ( x ) dx ∝ dPX ( x ) dx K ( x ′ , x ) . Instead of choosing x′ beforehand , if we let the center be XN+1 and fold the integral into P like in [ 13 ] , we arrive at the definition of local coverage : P { ỸN+1 ∈ Ĉα ( XN+1 ) |XN+1 = x′ } ≥ 1− α . ( 4 ) Here the probability integrates over all training data and an additional ( X̃N+1 , ỸN+1 ) ∼ P̃ defined above . Intuitively , this definition means Ĉα ( XN+1 ) is valid “ on average ” within a small neighborhood of XN+1 . Note that Eqs . 1 and 2 reduce to Eq . 4 with K being constant and delta functions , respectively . In the rest of the paper , we will call Ĉα marginally/conditionally/locally valid if it satisfies Eq . 1/2/4 respectively , and we will pursue finite-sample local validity . 2.3 Discrimination . The idea of discrimination is simple : If the error of our prediction µ̂ ( x ) is high for an input x , the PI should be wide , and vice versa . Formally , following [ 1 ] , we require E [ W ( Ĉ ( x ) ) ] ≥ E [ W ( Ĉ ( x′ ) ) ] ⇔ E [ ` ( y , µ̂ ( x ) ) ] ≥ E [ ` ( y′ , µ̂ ( x′ ) ) ] . ( 5 ) Here the expectation is taken over the training data , W is a measure of the width of the PI , and ` is a loss function such as MSE . This property can be verified ( as shown in Section 4 ) by checking how well W ( Ĉ ( x ) ) could predict the magnitude of the error . Discrimination could be considered a measure of efficiency , as a good Ĉ could “ save ” some width when the expected risk is low . However , it only makes sense to compare efficiency if all else is equal ( i.e . two marginally valid PIs estimators with the same error ) . Note that although discrimination could be related to conditional/local validity , they are not the same - e.g. , a PI that is always infinitely wide is conditionally valid , but not discriminative . Our goal is to achieve both local validity and discrimination without making any assumptions about the underlying distribution P ( i.e. , in a distribution-free setting ) . As noted in Section 1 , our method should also run fast and not affect the performance of underlying neural network model µ̂NN .

Summary: The authors the Locally Valid and Discriminative (LVD) method to construct confidence intervals. The idea has three steps. (1) First is to take the embeddings from a neural network and train a kernel regression model on the task, learning a Gaussian kernel $K$. (2) Second is to, on a separate held-out "conformal" set, compute the residuals between the model and ground truth labels. (3) Third is to, at inference time, compute the empirical desired $(1-\alpha)$ interval on the "conformal" set residuals, where each residual is re-weighted according to $K(x', x)$ where $x'$ is the query of interest. LVD has the appealing property that it is compatible with any (potentially pre-trained) deep learning model, with no further further training necessary.  The authors prove that under appropriate assumptions LVD achieves *local coverage*, i.e. coverage not only marginal over the entire feature space, but at a particular query $x$. Empirically they demonstrate that LVD improves upon popular neural network predictive uncertainty methods on the well-known UCI datasets. They evaluate in terms of Mean Absolute Deviation, Marginal Coverage Rates, and Tail Coverage Rates (which is defined as the coverage rate for the left and right 10% of the tails of labels $y$).