Understanding the Under-Coverage Bias in Uncertainty Estimation

1 Introduction . This paper is concerned with the problem of uncertainty estimation in regression problems . Uncertainty estimation is an increasingly important task in modern machine learning applications—Models should not only make high-accuracy predictions , but also have a sense of how much the true label may deviate from the prediction . This capability is crucial for deploying machine learning in the real world , in particular in risk-sensitive domains such as medical AI [ 15 , 29 ] , self-driving cars [ 47 ] , and so on . A common approach for uncertainty estimation in regression is to learn a quantile function or a prediction interval of the true label conditioned on the input , which provides useful distributional information about the label . Such learned quantiles are typically evaluated by their coverage , i.e. , probability that it covers the true label on a new test example . For example , a learned 90 % upper quantile function should be an actual upper bound of the true label at least 90 % of the time . Algorithms for learning quantiles date back to the classical quantile regression [ 35 ] , which estimates the quantile function by solving an empirical risk minimization problem with a suitable loss function that depends on the desired quantile level α. Quantile regression is conceptually simple , and is 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . theoretically shown to achieve asymptotically correct coverage as the sample size goes to infinity [ 34 ] or approximately correct coverage in finite samples under specific modeling assumptions [ 46 , 60 , 56 ] . However , it is observed that quantile regression often under-covers than the desired coverage level in practice [ 53 ] . Various alternative approaches for constructing quantiles and confidence intervals are proposed in more recent work , for example by aggregating multiple predictions using Bayesian neural networks or ensembles [ 24 , 37 ] , or by building on the conformal prediction technique to construct prediction intervals with finite-sample coverage guarantees [ 68 , 66 , 39 , 53 ] . However , despite these advances , a more fundamental understanding on why vanilla quantile regression exhibits this under-coverage bias is still lacking . This paper revisits quantile regression and presents a first precise theoretical study on its coverage , in a new regime where the number of samples n is proportional to the dimension d , and the ratio d/n is small ( so that the problem is under-parametrized ) . Our main result shows that quantile regression exhibits an inherent under-cover bias under this regime , even in the well-specified setting of learning a linear quantile function when the true data distribution follows a Gaussian linear model . To the best of our knowledge , this is the first rigorous theoretical justification of the under-coverage bias . Our main contributions are summarized as follows . • We prove that linear quantile regression exhibits an inherent under-coverage bias in the wellspecified setting where the data is generated from a Gaussian linear model , and the number of samples n is proportional to the feature dimension d with a small d/n ( Section 3 ) . More quantitatively , quantile regression at nominal level α ∈ ( 0.5 , 1 ) roughly achieves coverage α− ( α− 1/2 ) d/n regardless of the noise distribution . To the best of our knowledge , this is the first rigorous characterization of the under-coverage bias in quantile regression . • Towards understanding the source of this under-coverage bias , we disentangle the effect of estimating the bias and estimating the linear coefficient on the coverage of the learned linear quantile ( Section 4 ) . We show that the estimation error in the bias can have either an undercoverage or over-coverage effect , depending on the noise distribution . In contrast , the estimation error in the linear coefficient always drives the learned quantile to under-cover , and we show this effect is present even on broader classes of data distributions beyond the Gaussian linear model . • We perform experiments on simulated and real data to test our theory ( Section 5 ) . Our simulations show that the coverage of quantile regression in Gaussian linear models agrees well with our precise theoretical formula as well as the α− ( α− 1/2 ) d/n approximation . On real data , we find quantile regression using high-capacity models ( such as neural networks ) exhibits severe under-coverage biases , while linear quantile regression can also have a mild but non-negligible amount of under-coverage , even after we remove the potential effect of model misspecification . • On the technical end , our analysis builds on recent understandings of empirical risk minimization problems in the high-dimensional proportional limit with a small d/n , and develops new techniques such as a novel concentration argument to deal with an additional learnable variable in learning linear models with biases , which we believe could be of further interest ( Section 6 ) . 1.1 Related work . Algorithms for uncertainty estimation in regression The earliest methods for uncertainty estimation in regression adopted subsampling methods ( bootstrap ) or leave-one-out methods ( Jackknife ) for assessing or calibrating prediction uncertainty [ 52 , 63 , 58 , 25 ] . More recently , a growing line of work builds on the idea of conformal prediction [ 55 ] to design uncertainty estimation algorithms for regression . These algorithms provide confidence bounds or prediction intervals by post-processing any predictor , and can achieve distribution-free finite-sample marginal coverage guarantees utilizing exchangeability of the data [ 50 , 66 , 39 , 53 , 33 , 67 , 69 , 68 , 11 ] . Further modifications of the conformal prediction technique can yield stronger guarantees such as group coverage [ 10 ] or coverage under distribution shift [ 9 ] under additional assumptions . Our under-coverage results advocate the necessity of such post-processing techniques , and are complementary in the sense that we provide understandings on the more vanilla quantile regression algorithm . Quantiles and prediction intervals can also be obtained by aggregating multiple predictors , such as using Bayesian neural networks [ 41 , 24 , 32 , 44 , 42 ] or ensembles [ 37 , 49 , 28 , 45 ] . These methods offer an alternative approach for uncertainty estimation , but do not typically come with coverage guarantees . Theoretical analysis of quantile regression Linear quantile regression with the pinball loss dates back to the late 1970s [ 34 ] . The same work proved the asymptotic normality of the regression coefficients in the n → ∞ , fixed d limit . Takeuchi et al . [ 60 ] studied non-parametric quantile regression using kernel methods , and provided generalization bounds ( with the pinball loss ) based on the Rademacher complexity . Meinshausen [ 46 ] studied non-parametric quantile regression using random forest and showed its consistency under proper assumptions . Christmann and Steinwart [ 19 ] , Steinwart et al . [ 56 ] established a “ self-calibration ” inequality for the quantile loss , which , when combined with standard generalization bounds , can be translated to an estimation error bound for quantile regression . These works all focus on bounding the parameter or function estimation error , which can be translated to bounds on the coverage bias , but does not tell the sign of this coverage bias as we do in this paper . We also remark that conformalization can be used in conjunction with quantile regression to correct its coverage bias [ 53 ] . Uncertainty quantification for classification For classification problems , two main types of uncertainty quantification methods have been considered : outputting discrete prediction sets with guarantees of covering the true ( discrete ) label [ 70 , 71 , 38 , 7 , 12 , 18 , 17 ] , or calibrating the predicted probabilities [ 51 , 72 , 73 , 37 , 26 ] . The connection between prediction sets and calibration was discussed in [ 27 ] . The sample complexity of calibration has been studied in a number of theoretical works [ 36 , 27 , 54 , 30 , 40 , 8 ] . Our work is inspired by the recent work of Bai et al . [ 8 ] , which showed that logistic regression is over-confident even if the model is correctly specified and the sample size is larger than the dimension . High-dimensional behaviors of empirical risk minimization There is a rapidly growing literature on limiting characterizations of convex optimization-based estimators in the n ∝ d regime [ 21 , 13 , 23 , 31 , 57 , 61 , 20 , 62 , 43 , 59 , 16 ] . Our analysis builds on results for unregularized Mestimator derived in [ 62 ] and generalizes theirs in certain aspects ( see also [ 23 , 20 , 31 ] ) . 2 Preliminaries . In this paper we focus on the problem of learning quantiles . Suppose we observe a training dataset { ( xi , yi ) } ni=1 drawn i.i.d . from some joint distribution P on Rd × R , where xi ∈ Rd is the input features and y ∈ R is the real-valued response ( label ) . Let F ( t|x ) : = P ( Y ≤ t|X = x ) denote the conditional CDF of Y |X . Our goal is to learn the α- ( conditional ) quantile of Y |X : q ? α ( x ) : = inf { t ∈ R : F ( t|x ) ≥ α } . For example , q ? 0.95 ( x ) is the ground truth 95 % quantile of the true conditional distribution Y |X , and can be seen as the “ ideal ” 95 % upper confidence bound for the label y given the features x . Throughout this paper we work with upper quantiles , that is , α ∈ ( 0.5 , 1 ) ( some typical choices are α ∈ { 0.8 , 0.9 , 0.95 } ) ; by symmetry our results hold for learning lower quantiles as well . Coverage For any learned quantile function f̂ : Rd → R , the marginal coverage ( henceforth “ coverage ” ) of f̂ is the probability of y ≤ f̂ ( x ) on a new test example ( x , y ) : Coverage ( f̂ ) : = P ( x , y ) ( y ≤ f̂ ( x ) ) = Ex [ P ( y ≤ f̂ ( x ) |x ) ] . ( 1 ) For learning the α-quantile ( α > 0.5 ) , we usually expect Coverage ( f̂ ) ≈ α , i.e . f̂ ( x ) covers the label y on approximately α proportion of the data , under the ground truth data distribution . We say that f̂ has under-coverage if Coverage ( f̂ ) < α and over-coverage if Coverage ( f̂ ) > α . Note that these two notions are not symmetric : Over-coverage means that the learned upper quantile f̂ ( x ) is overly conservative ( higher than enough ) , and is typically tolerable ; In contrast , under-coverage means that f̂ ( x ) fails to cover y with α probability , and is typically considered as a failure . We remark that while there exist more fine-grained notions of coverage such as conditional coverage [ 10 ] , the ( marginal ) coverage is still a basic requirement for any quantile learning algorithm . Quantile regression We consider quantile regression , a standard method for learning quantiles from data [ 35 ] . Quantile regression estimates the true quantile function qα ( · ) via the pinball loss [ 34 , 56 ] ` α ( t ) = − ( 1− α ) t1 { t ≤ 0 } + αt1 { t > 0 } . ( 2 ) Note that in the special case of α = 0.5 , we have ` 0.5 ( t ) = |t|/2 , and thus the pinball loss strictly generalizes the absolute loss ( for learning medians ) to learning any quantile . Given the training dataset and any function class { fθ : θ ∈ Θ } ( e.g . linear models or neural networks ) , quantile regression solves the ( unregularized ) empirical risk minimization ( ERM ) problem θ̂ = arg min θ∈Θ R̂n ( θ ) : = 1 n n∑ i=1 ` α ( yi − fθ ( xi ) ) . ( 3 ) ( We take θ̂ as any minimizer of R̂n when the minimizer is non-unique . ) Let R ( θ ) : = E [ R̂n ( θ ) ] denote the corresponding population risk . It is known that the population risk over all ( measurable ) functions is minimized at the true quantile q ? α = arg minf R ( f ) under minimal regularity conditions ( for completeness , we provide a proof in Appendix B.1 ) .

Summary: This paper gives theoretical insights into the under-coverage phenomenon often observed in quantile regression, i.e., for a continuous distribution, the probability of observing  a true label below a learned  $\alpha$-quantile is often below $\alpha$, which is consider a failure in estimating the uncertainty. Two important results are shown. The first one provides a characterization of the under-coverage in the setting of a Gaussian linear model in the under-parameterized high-dimensional proportional limit (i.e. $d,n \rightarrow \infty$ , $d/n\rightarrow\kappa$ with small $\kappa$) independently of the noise distribution. The experimental results show that the formulas match pretty well the observed coverage in realistic situations of dimension around 100, on synthetic and real data. The second important result shows that the source of the under-coverage stems essentially from the error in estimating the high-dimensional linear coefficient, under some assumptions on the underlying source.