Understanding the Under-Coverage Bias in Uncertainty Estimation

1 Introduction . This paper is concerned with the problem of uncertainty estimation in regression problems . Uncertainty estimation is an increasingly important task in modern machine learning applications—Models should not only make high-accuracy predictions , but also have a sense of how much the true label may deviate from the prediction . This capability is crucial for deploying machine learning in the real world , in particular in risk-sensitive domains such as medical AI [ 15 , 29 ] , self-driving cars [ 47 ] , and so on . A common approach for uncertainty estimation in regression is to learn a quantile function or a prediction interval of the true label conditioned on the input , which provides useful distributional information about the label . Such learned quantiles are typically evaluated by their coverage , i.e. , probability that it covers the true label on a new test example . For example , a learned 90 % upper quantile function should be an actual upper bound of the true label at least 90 % of the time . Algorithms for learning quantiles date back to the classical quantile regression [ 35 ] , which estimates the quantile function by solving an empirical risk minimization problem with a suitable loss function that depends on the desired quantile level α. Quantile regression is conceptually simple , and is 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . theoretically shown to achieve asymptotically correct coverage as the sample size goes to infinity [ 34 ] or approximately correct coverage in finite samples under specific modeling assumptions [ 46 , 60 , 56 ] . However , it is observed that quantile regression often under-covers than the desired coverage level in practice [ 53 ] . Various alternative approaches for constructing quantiles and confidence intervals are proposed in more recent work , for example by aggregating multiple predictions using Bayesian neural networks or ensembles [ 24 , 37 ] , or by building on the conformal prediction technique to construct prediction intervals with finite-sample coverage guarantees [ 68 , 66 , 39 , 53 ] . However , despite these advances , a more fundamental understanding on why vanilla quantile regression exhibits this under-coverage bias is still lacking . This paper revisits quantile regression and presents a first precise theoretical study on its coverage , in a new regime where the number of samples n is proportional to the dimension d , and the ratio d/n is small ( so that the problem is under-parametrized ) . Our main result shows that quantile regression exhibits an inherent under-cover bias under this regime , even in the well-specified setting of learning a linear quantile function when the true data distribution follows a Gaussian linear model . To the best of our knowledge , this is the first rigorous theoretical justification of the under-coverage bias . Our main contributions are summarized as follows . • We prove that linear quantile regression exhibits an inherent under-coverage bias in the wellspecified setting where the data is generated from a Gaussian linear model , and the number of samples n is proportional to the feature dimension d with a small d/n ( Section 3 ) . More quantitatively , quantile regression at nominal level α ∈ ( 0.5 , 1 ) roughly achieves coverage α− ( α− 1/2 ) d/n regardless of the noise distribution . To the best of our knowledge , this is the first rigorous characterization of the under-coverage bias in quantile regression . • Towards understanding the source of this under-coverage bias , we disentangle the effect of estimating the bias and estimating the linear coefficient on the coverage of the learned linear quantile ( Section 4 ) . We show that the estimation error in the bias can have either an undercoverage or over-coverage effect , depending on the noise distribution . In contrast , the estimation error in the linear coefficient always drives the learned quantile to under-cover , and we show this effect is present even on broader classes of data distributions beyond the Gaussian linear model . • We perform experiments on simulated and real data to test our theory ( Section 5 ) . Our simulations show that the coverage of quantile regression in Gaussian linear models agrees well with our precise theoretical formula as well as the α− ( α− 1/2 ) d/n approximation . On real data , we find quantile regression using high-capacity models ( such as neural networks ) exhibits severe under-coverage biases , while linear quantile regression can also have a mild but non-negligible amount of under-coverage , even after we remove the potential effect of model misspecification . • On the technical end , our analysis builds on recent understandings of empirical risk minimization problems in the high-dimensional proportional limit with a small d/n , and develops new techniques such as a novel concentration argument to deal with an additional learnable variable in learning linear models with biases , which we believe could be of further interest ( Section 6 ) . 1.1 Related work . Algorithms for uncertainty estimation in regression The earliest methods for uncertainty estimation in regression adopted subsampling methods ( bootstrap ) or leave-one-out methods ( Jackknife ) for assessing or calibrating prediction uncertainty [ 52 , 63 , 58 , 25 ] . More recently , a growing line of work builds on the idea of conformal prediction [ 55 ] to design uncertainty estimation algorithms for regression . These algorithms provide confidence bounds or prediction intervals by post-processing any predictor , and can achieve distribution-free finite-sample marginal coverage guarantees utilizing exchangeability of the data [ 50 , 66 , 39 , 53 , 33 , 67 , 69 , 68 , 11 ] . Further modifications of the conformal prediction technique can yield stronger guarantees such as group coverage [ 10 ] or coverage under distribution shift [ 9 ] under additional assumptions . Our under-coverage results advocate the necessity of such post-processing techniques , and are complementary in the sense that we provide understandings on the more vanilla quantile regression algorithm . Quantiles and prediction intervals can also be obtained by aggregating multiple predictors , such as using Bayesian neural networks [ 41 , 24 , 32 , 44 , 42 ] or ensembles [ 37 , 49 , 28 , 45 ] . These methods offer an alternative approach for uncertainty estimation , but do not typically come with coverage guarantees . Theoretical analysis of quantile regression Linear quantile regression with the pinball loss dates back to the late 1970s [ 34 ] . The same work proved the asymptotic normality of the regression coefficients in the n → ∞ , fixed d limit . Takeuchi et al . [ 60 ] studied non-parametric quantile regression using kernel methods , and provided generalization bounds ( with the pinball loss ) based on the Rademacher complexity . Meinshausen [ 46 ] studied non-parametric quantile regression using random forest and showed its consistency under proper assumptions . Christmann and Steinwart [ 19 ] , Steinwart et al . [ 56 ] established a “ self-calibration ” inequality for the quantile loss , which , when combined with standard generalization bounds , can be translated to an estimation error bound for quantile regression . These works all focus on bounding the parameter or function estimation error , which can be translated to bounds on the coverage bias , but does not tell the sign of this coverage bias as we do in this paper . We also remark that conformalization can be used in conjunction with quantile regression to correct its coverage bias [ 53 ] . Uncertainty quantification for classification For classification problems , two main types of uncertainty quantification methods have been considered : outputting discrete prediction sets with guarantees of covering the true ( discrete ) label [ 70 , 71 , 38 , 7 , 12 , 18 , 17 ] , or calibrating the predicted probabilities [ 51 , 72 , 73 , 37 , 26 ] . The connection between prediction sets and calibration was discussed in [ 27 ] . The sample complexity of calibration has been studied in a number of theoretical works [ 36 , 27 , 54 , 30 , 40 , 8 ] . Our work is inspired by the recent work of Bai et al . [ 8 ] , which showed that logistic regression is over-confident even if the model is correctly specified and the sample size is larger than the dimension . High-dimensional behaviors of empirical risk minimization There is a rapidly growing literature on limiting characterizations of convex optimization-based estimators in the n ∝ d regime [ 21 , 13 , 23 , 31 , 57 , 61 , 20 , 62 , 43 , 59 , 16 ] . Our analysis builds on results for unregularized Mestimator derived in [ 62 ] and generalizes theirs in certain aspects ( see also [ 23 , 20 , 31 ] ) . 2 Preliminaries . In this paper we focus on the problem of learning quantiles . Suppose we observe a training dataset { ( xi , yi ) } ni=1 drawn i.i.d . from some joint distribution P on Rd × R , where xi ∈ Rd is the input features and y ∈ R is the real-valued response ( label ) . Let F ( t|x ) : = P ( Y ≤ t|X = x ) denote the conditional CDF of Y |X . Our goal is to learn the α- ( conditional ) quantile of Y |X : q ? α ( x ) : = inf { t ∈ R : F ( t|x ) ≥ α } . For example , q ? 0.95 ( x ) is the ground truth 95 % quantile of the true conditional distribution Y |X , and can be seen as the “ ideal ” 95 % upper confidence bound for the label y given the features x . Throughout this paper we work with upper quantiles , that is , α ∈ ( 0.5 , 1 ) ( some typical choices are α ∈ { 0.8 , 0.9 , 0.95 } ) ; by symmetry our results hold for learning lower quantiles as well . Coverage For any learned quantile function f̂ : Rd → R , the marginal coverage ( henceforth “ coverage ” ) of f̂ is the probability of y ≤ f̂ ( x ) on a new test example ( x , y ) : Coverage ( f̂ ) : = P ( x , y ) ( y ≤ f̂ ( x ) ) = Ex [ P ( y ≤ f̂ ( x ) |x ) ] . ( 1 ) For learning the α-quantile ( α > 0.5 ) , we usually expect Coverage ( f̂ ) ≈ α , i.e . f̂ ( x ) covers the label y on approximately α proportion of the data , under the ground truth data distribution . We say that f̂ has under-coverage if Coverage ( f̂ ) < α and over-coverage if Coverage ( f̂ ) > α . Note that these two notions are not symmetric : Over-coverage means that the learned upper quantile f̂ ( x ) is overly conservative ( higher than enough ) , and is typically tolerable ; In contrast , under-coverage means that f̂ ( x ) fails to cover y with α probability , and is typically considered as a failure . We remark that while there exist more fine-grained notions of coverage such as conditional coverage [ 10 ] , the ( marginal ) coverage is still a basic requirement for any quantile learning algorithm . Quantile regression We consider quantile regression , a standard method for learning quantiles from data [ 35 ] . Quantile regression estimates the true quantile function qα ( · ) via the pinball loss [ 34 , 56 ] ` α ( t ) = − ( 1− α ) t1 { t ≤ 0 } + αt1 { t > 0 } . ( 2 ) Note that in the special case of α = 0.5 , we have ` 0.5 ( t ) = |t|/2 , and thus the pinball loss strictly generalizes the absolute loss ( for learning medians ) to learning any quantile . Given the training dataset and any function class { fθ : θ ∈ Θ } ( e.g . linear models or neural networks ) , quantile regression solves the ( unregularized ) empirical risk minimization ( ERM ) problem θ̂ = arg min θ∈Θ R̂n ( θ ) : = 1 n n∑ i=1 ` α ( yi − fθ ( xi ) ) . ( 3 ) ( We take θ̂ as any minimizer of R̂n when the minimizer is non-unique . ) Let R ( θ ) : = E [ R̂n ( θ ) ] denote the corresponding population risk . It is known that the population risk over all ( measurable ) functions is minimized at the true quantile q ? α = arg minf R ( f ) under minimal regularity conditions ( for completeness , we provide a proof in Appendix B.1 ) .

Summary: In this work, the authors consider linear quantile regression, paying particular attention to the "coverage" properties of the resulting predictor. The traditional goal of quantile regression is to learn a quantile of the conditional distribution of output $Y$, conditioned on the inputs $X$. Denoting the predictor returned by any quantile regression algorithm by $\hat{f}(\cdot)$, one can use this to construct a predictive interval for a freshly-drawn $Y$. The authors consider the one-sided interval $C(X) = (-\infty, \hat{f}(X)]$, and the event $\\{ Y \in C(X) \\}$. The authors call the probability of this event (over the draw of a pair $(X,Y)$) the coverage of $\hat{f}$. Their main results elucidate conditions under which the traditional pinball-loss based ERM quantile regression solution is biased below the desired quantile level, that is, the predictive interval tends to be "too small," due to $\hat{f}$ under-estimating the desired quantile.  In particular, they show that in the limit where the sample size and dimensionality are proportional, the coverage (the aforementioned probability, which depends on $\hat{f}$ and thus the training data) converges below the desired quantile, with this error depending directly on the ratio of dimension to sample size $d/n$. Furthermore, they show (under fixed $n$ and $d$) how for a certain family of data distributions and a linear quantile model, given any estimator of the weights, the downward bias term scales with the weight estimation error. Their empirical tests also effectively highlight the phenomena described formally.