# New Bandwidth Selection for Kernel Quantile Estimators

Authors: Ali Al-Kenani; Keming Yu
Journal: Journal of Probability and Statistics
 (2012)

---

## Abstract

We propose a cross-validation method suitable for smoothing of kernel quantile estimators. In particular, our proposed method selects the bandwidth parameter, which is known to play a crucial role in kernel smoothing, based on unbiased estimation of a mean integrated squared error curve of which the minimising value determines an optimal bandwidth. This method is shown to lead to asymptotically optimal bandwidth choice and we also provide some general theory on the performance of optimal, data-based methods of bandwidth choice. The numerical performances of the proposed methods are compared in simulations, and the new bandwidth selection is demonstrated to work very well.


---

## Body

## 1. Introduction

The estimation of population quantiles is of great interest when one is not prepared to assume a parametric form for the underlying distribution.  In addition, due to their robust nature, quantiles often arise as natural quantities to estimate when the underlying distribution is skewed [1].  Similarly, quantiles often arise in statistical inference as the limits of confidence interval of an unknown quantity.LetX1,X2,…,Xnbe independent and identically distributed random sample drawn from an absolutely continuous distribution function Fwith density f. Further, let X(1)≤X(2)⋯≤Xndenote the corresponding order statistics. For (0<p<1) a quantile function Q(p)is defined as follows:
(1.1)Q(p)=inf⁡{x:F(x)≥p}.
If Q̂(p) denotes pth sample quantile, then Q̂(p)=x([np]+1) where [np] denotes the integral part ofnp. Because of the variability of individual order statistics, the sample quantiles suffer from lack of efficiency. In order to reduce this variability, different approaches of estimating sample quantiles through weighted order statistics have been proposed. A popular class of these estimators is called kernel quantile estimators. Parzen [2] proposed a version of the kernel quantile estimator as below:
(1.2)Q̃K(p)=∑i=1n[∫i-1/ni/nKh(t-p)dt]X(i).
From (1.2) one can readily observe that Q̃K(p) puts most weight on the order statistics X(i), for which i/n is close top. In practice,  the following approximation to Q̃K(p) is often used:(1.3)Q̃AK(p)=∑i=1n[n-1Kh(i/n-p)]X(i).
Yang [3] proved that Q̃K(p) and Q̃AK(p)are asymptotically equivalent in terms of mean square errors. Similarly, Falk [4] demonstrates that, from a relative deficiency perspective, the asymptotic performance of Q̃AK(p) is better than that of the empirical sample quantile.In this paper, we propose a cross-validation method suitable for smoothing of kernel quantile estimators. In particular, our proposed method selects the bandwidth parameter, which is known to play a crucial role in kernel smoothing, based on unbiased estimation of a mean integrated squared error curve of which the minimising value determines an optimal bandwidth. This method is shown to lead to asymptotically optimal bandwidth choice and we also provide some general theory on the performance of optimal, data-based methods of bandwidth choice. The numerical performances of the proposed methods are compared in simulations, and the new bandwidth selection is demonstrated to work very well.


## 2. Data-Based Selection of the Bandwidth

Bandwidth plays a critical role in the implementation of practical estimation. Specifically, the choice of the smoothing parameter determines the tradeoff between the amount of smoothness obtained and closeness of the estimation to the true distribution [5].Several data-based methods can be made to find the asymptotically optimal bandwidthh in kernel quantile estimators forQ̃AK(p) given by (1.3). One of these methods use derivatives of the quantile density for Q̃AK(p).Building on Falk [4], Sheather and Marron [1] gave the MSE of Q̃AK(p) as follows. If f is not symmetric or f is symmetric but p≠0.5,
(2.1)AMSE(Q̃AK(p))=14μ2(k)2[Q′′(p)]2h4+p(1-p)[Q′(p)]2n-1-R(K)[Q′(p)]2n-1h,
where R(K)=2∫-∞∞uK(u)K-1(u)du, μ2(k)=∫-∞∞u2K(u)duand K-1 is the antiderivative of K.IfQ′>0 then
(2.2)hopt=α(K)⋅β(Q)⋅n-1/3,
where α(K)=[R(K)/μ2(k)2]1/3, β(Q)=[Q′(p)/Q′′(p)]2/3.There is no single optimal bandwidth minimizing theAMSE(Q̃AK(p)) when F is symmetric andp=0.5. Also, If q=0, we need higher terms and the AMSE(Q̃AK(p)) can be shown to be(2.3)AMSE(Q̃AK(p))=(14-1n)h4[Q′′(p)]2μ2(k)2+2n-1h2[Q′′(p)]2∫(q-ht)tK(t)j(t)dt,where j(t)=∫-∞txK(x)dx,see Cheng and Sun [6].In order to obtainhopt we need to estimate Q′=q and Q′′=q′. It follows from (1.3) that the estimator of Q′=q can be constructed as follows:(2.4)q̃AK(p)=Q̃′AK(p)=∑i=1nX(i)[Ka((i-1)n-p)-Ka(in-p)].
Jones [7] derived that the AMSE(q̃AK(p)) as
(2.5)AMSE(q̃AK(p))=a44μ2(k)2[q′′(p)]2+1na[q(p)]2∫K2(y)dy.
By minimizing (2.5), we obtain the asymptotically optimal bandwidth for Q̃′AK(p):
(2.6)aopt*=[[Q′(p)]2∫K2(y)dyn[Q′′′(p)]2μ2(k)2]1/5.
To estimate Q′′=q′ in (2.2), we employ the known result
(2.7)Q̃′′AK(p)=ddpQ̃′AK(p)=1a2∑i=1nX(i)[K′((i-1)/n-pa)-K′(i/n-pa)],
and it readily follows that
(2.8)aopt**=[3[Q′(p)]2∫K′2(x)dxn[Q′′′′(p)]2μ2(k)2]1/7
which represents the asymptotically optimal bandwidth for Q̃′′AK(p). By substituting a=aopt* in (2.4) and a=aopt** in (2.7) we can compute hopt.


## 3. Cross-Valdation Bandwidth Selection

When measuring the closeness of an estimated and true function the mean integrated squared (MISE) defined as(3.1)MISE(h)=E∫01{Q̃(p)-Q(p)}2dp
is commonly used as a global measure of performance.The value which minimisesMISE(h) is the optimal smoothing parameter, and it is unknown in practice. The following ASE(h) is the discrete form of error criterion approximatingMISE(h):(3.2)ASE(h)=1n∑i=1n{Q̃(in)-Q(in)}2.The unknownQ(p) is replaced by Q̂(p) and a function of cross-validatory procedure is created as:(3.3)1n∑i=1n{Q̃-i(in)-Q̂(in)}2,where Q̃-i(i/n) denotes the kernel estimator evaluated at observation xi, but constructed from the data with observation xi omitted.The general approach of crossvalidation is to compare each observation with a value predicted by the model based on the remainder of the data. A method for density estimation was proposed by Rudemo [8] and Bowman [9]. This method can be viewed as representing each observation by a Dirac delta function δ(x-xi), whose expectation is f(x), and contrasting this with a density estimate based on the remainder of the data. In the context of distribution functions, a natural characterisation of each observation is by the indicator function I(x-xi)whose expectation is F(x). This implies that the kernel method for density estimation can be expressed as(3.4)f̃(x)=1n∑i=1nKh(x-xi),whenh→0Kh(x-xi)→δ(x-xi).The kernel method for distribution function(3.5)F̃(x)=1n∑i=1nW(x-xih),whereW is a distribution function, h is the bandwidth controls the degree of smoothing. When h→0(3.6)W(x-xih)⟶I(x-xi),whereI(x-xi)is the indicator function(3.7)I(x-xi)={1,ifx-xi≥0,0,otherwise.Now, from (1.3) when h→0(3.8)Q̃AK(p)⟶δ(in-p)X(i),
and thus a cross-validation function can be written as
(3.9)CV(h)=1n∑i=1n∫01{δ(in-p)X(i)-Q̃-i(in)}2dp.
The smoothing parameter h is then chosen to minimise this function. By subtracting a term that characterise the performance of the true (p) we have(3.10)H(h)=CV(h)-1n∑i=1n∫01{δ(in-p)X(i)-Q(in)}2dp
which does not involve h. By expanding the braces and taking expectation, we obtain
(3.11)H(h)=1n∑i=1n∫01{Q̃2-i(in)-2δ(in-p)X(i)Q̃-i(in)+2δ(in-p)X(i)Q(in)-Q2(in)}dp.
When n→∞ the (np)th order statistic x(np) is asymptotically normally distributed
(3.12)x(np)~AN(Q(p),p(1-p)n[f(Q(p))]2),E{H(h)}=E[1n∑i=1n∫01{Q̃2-i(in)-2δ(in-p)X(i)Q̃-i(in)+2δ(in-p)X(i)Q(in)-Q2(in)}dp[1n∑i=1n∫01{Q̃2-i(in)-2δ(in-p)X(i)Q̃-i(in)+2δ(in-p)X(i)Q(in)],E{H(h)}=1n∑i=1n∫01[E{Q̃2-i(in)}-2δ(in-p)Q(in)E{Q̃-i(in)}+2δ(in-p)Q2(in)-Q2(in)]dp,E{H(h)}=E∫01{Q̃n-1(in)-Q(in)}2dp,
where the notation Q̃n-1(i/n) with positive subscript denotes a kernel estimator based on a sample size of n-1. The proceeding arguments demonstrate that CV(h)provides an asymptotic unbiased estimator of the true MISE(h) curve for a sample size n-1. The identity at (3.12) strongly suggests that crossvalidation should perform well.


## 4. Theoretical Properties

From (3.1), we can write MISE(h)=∫01bias2(Q̃K(p))dp+∫01var(Q̃K(p))dp.Sheather and Marron [1] have shown that
(4.1)bias(Q̃K(p))=12h2μ2(k)Q′′(p)+0(h2).
while Falk [4, page 263] proved that(4.2)var(Q̃K(p))=p(1-p)[Q′(p)]2n-1-R(K)[Q′(p)]2n-1h+0(n-1h).On combining the expressions for bias and variance we can express the mean integrated square error as(4.3)MISE(h)=14h4μ2(k)2∫01[Q′′(p)]2dp+p(1-p)∫01[Q′(p)]2dpn-1-R(K)∫01[Q′(p)]2dpn-1h+0(h4+n-1h),and forC1=p(1-p)∫01[Q′(p)]2dp,C2=R(K)∫01[Q′(p)]2dp and C3=μ2(k)2∫01[Q′′(p)]2dp the MISE can be expressed as
(4.4)MISE(h)=C1n-1-C2n-1h+14C3h4+0(h4+n-1h).
Therefore, the asymptotically optimal bandwidth is h0=Cn-1/3, whereC={C2/C3}1/3.We can see from (3.12) that H(h) may be a good approximation to MISE(h) or at least to that function evaluated for a sample of size n-1rather than n. Additionally, this is true if we adjusted H(h) by adding the quantity(4.5)Jn=∫01{(Q̂(p)-Q(p))2-E(Q̂(p)-Q(p))2}.
This quantity is demean and does not depend on h which makes it attractive for obtaining a particularly good approximation to MISE(h).Theorem 4.1.
Suppose thatQ(p) is bounded on [0,1] and right continuous at the point 0, and that K is a compactly supported density and symmetric about 0. Then, for each δ,ε,C>0,
(4.6)H(h)+J=MISE(h)+02{(n-3/2+n-1h3/2+n-1/2h3)nδ}
with probability1, uniformly in 0≤h≤Cnδ, as n→∞.(An outline proof of the above theorem is in the appendix).From the above theorem,  we can conclude that minimisation ofH(h) produces a bandwidth that is asymptotically equivalent to the bandwidth h0 that minimises MISE(h).Corollary 4.2.
Suppose that the conditions of previous theorem hold. Ifĥ denotes the bandwidth that minimises CV(h) in the range 0≤h≤Cnδ, for any C>0 and any 0≤ε≤1/3, then
(4.7)ĥh0⟶1
with probability 1 as n→∞.


## 5. A Simulation Study

A numerical study was conducted to compare the performances of the two bandwidth selection methods. Namely, the method presented by Sheather and Marron [1] and our proposed method.In order to account for different shapes for our simulation study we consider a standard normal, Exp(1), Log-normal(0,1) and double exponential distributions and we calculate 18 quantiles ranging from p=0.05 to p=0.95. Through the numerical study the Gaussian kernel was used as the kernel function. Sample sizes of 100, 200 and 500 were used, with 100 simulations in each case. The performance of the methods was assessed through the mean squared errors criterion (MSE). MSE(h)=E{Q̃(p)-Q(p)}2. And the relative efficiency (R.E)(5.1)R.E=[MISEMethod2(hMethod2,opt)MISEMethod1(hMethod1,opt)].Further, for comparison purposes we refer to our proposed method and that of Sheather and Marron [1] as method 1 and method 2 respectively.(a) Standard normal distribution (see Table1 and Figure 1).Table 1
Mean squared errors results for bandwidth selection methods for different sample sizes and for data from a normal distribution.
pn=100n=200n=5000.05method 10.348419560.3210738700.298936771method 20.296367580.1643647380.0905980820.10method 10.076459560.0654405750.054697205method 20.049477450.0228463550.0155669070.15method 10.022915010.0139206680.007384189method 20.029397080.0133862340.0050058490.20method 10.018919190.0092737460.003152866method 20.022288280.0100941720.0038122090.25method 10.015969480.0085813980.003000777method 20.018359120.0088806390.0035687720.30method 10.016149810.0080356670.003208531method 20.016391480.0082998380.0033754450.35method 10.014618800.0076775670.003534028method 20.015447900.0077636290.0030120450.40method 10.012794740.0073754280.002899081method 20.014945060.0072484970.0026612300.45method 10.012242680.0061288170.002183302method 20.014441530.0067904900.0022958300.55method 10.014140500.0063488930.001922013method 20.013732580.0067024300.0020994460.60method 10.013753730.0063927210.002007274method 20.013417630.0067627980.0022548690.65method 10.013447730.0060635020.002589679method 20.012905690.0068019010.0025072020.70method 10.013208320.0063941020.002456085method 20.012339480.0070010640.0026916780.75method 10.015032640.0070118670.002789939method 20.012198290.0072163260.0026796090.80method 10.016048470.0072466050.002715445method 20.013278360.0076023460.0027912400.85method 10.017571710.0092395890.004770755method 20.017409310.0095221810.0038484740.90method 10.031923790.0232929750.019942754method 20.037027740.0180539760.0122504130.95method 10.153238930.1477739630.150811561method 20.248251880.1468401770.092517440Left panel: plots of the quantile estimators for method 1 (solid line), method 2 (dotted line), and true quantile (dashed line) for different sample sizes and for data from a normal distribution. Right panel: box plots of mean squared errors for the quantile estimators for method 1 and method 2 for different sample sizes.
(a)(b)(c)(d)(e)(f)(b) Exponential distribution (see Table2 and Figure 2).Table 2
Mean squared errors results for bandwidth selection methods for different sample sizes and for data from an exponential distribution.
pn=100n=200n=5000.05method 10.0016870250.00146999900.0014107454method 20.00060232360.00024767458.122873e-050.10method 10.0013062110.00092293380.0007744410method 20.00082252540.00040758221.749150e-040.15method 10.0015896460.00089404860.0006237375method 20.00129635760.00069382873.186597e-040.20method 10.0021879900.00114770630.0006801504method 20.00191881720.00103582724.746909e-040.25method 10.0029164170.00158056780.0008156225method 20.00268386590.00140965236.303538e-040.30method 10.0038275110.00197242070.0010289166method 20.00365426880.00183589567.948940e-040.35method 10.0049196180.00255403230.0012720751method 20.00483016570.00233183589.724792e-040.40method 10.0058681130.00319323550.0016253398method 20.00600922430.00289987511.170038e-030.45method 10.0072677830.00399624260.0021094081method 20.00727856410.00353638161.417269e-030.55method 10.0117769760.00651482220.0039208447method 20.01105991560.00555485522.154130e-030.60method 10.0128645210.00703666990.0026965785method 20.01385853650.00703595612.626137e-030.65method 10.0181730970.00864763490.0031472559method 20.01697094130.00888322633.255114e-030.70method 10.0211255320.01116075010.0041235720method 20.02010497200.01147031804.201740e-030.75method 10.0240258360.01507852890.0057215181method 20.02297639520.01494902505.812526e-030.80method 10.0373673440.02046763680.0081595071method 20.04071068850.01816479768.020787e-030.85method 10.0577855390.03174048710.0098128398method 20.08386576810.03006561491.134861e-020.90method 10.0787973790.04264184100.0152139697method 20.18784568520.11178200162.156987e-020.95method 10.1212391020.08101354500.0284524316method 20.66683238360.49237326841.478679e-01Left panel: plots of the quantile estimators for method 1 (solid line), method 2 (dotted line) and true quantile (dashed line) for different sample sizes and for data from an exponential distribution. Right panel: box plots of mean squared errors for the quantile estimators for method 1 and method 2 for different sample sizes.
(a)(b)(c)(d)(e)(f)(c) Log-normal distribution (see Table3 and Figure 3).Table 3
Mean squared errors results for bandwidth selection methods for different sample sizes and for data from a Log-normal distribution.
pn=100n=200n=5000.05method 10.0016630320.00100985730.0006568989method 20.0023841360.00072704410.00036135410.10method 10.0018631410.00084383330.0002915013method 20.0026019940.00083614750.00029819380.15method 10.0026331530.00134928700.0004451506method 20.0026235520.00119431440.00037385080.20method 10.0037534580.00199223560.0006866399method 20.0031073510.00147245250.00056850220.25method 10.0049566350.00271408780.0009886053method 20.0045643820.00229520790.00085577560.30method 10.0064801950.00356031710.0015897314method 20.0064369670.00315742640.00119389240.35method 10.0088588500.00479723720.0023446072method 20.0084431290.00386261050.00154439700.40method 10.0100539690.00559891430.0022496198method 20.0108933980.00517357210.00175795790.45method 10.0129989400.00690583620.0030102466method 20.0136079310.00636067580.00197995510.55method 10.0196878500.01154314730.0051386226method 20.0205811100.01008288100.00295544660.60method 10.0238818830.01292279020.0046644050method 20.0258454190.01290811380.00403018440.65method 10.0321555370.01604761260.0056732073method 20.0357370080.01671474690.00565286580.70method 10.0450279650.02495768360.0077709058method 20.0426813150.02239363020.00776163460.75method 10.0607156760.03188911760.0121926243method 20.0592761980.03237387490.01041192170.80method 10.0876947540.04508149110.0165993582method 20.0907046300.05303747100.01681624260.85method 10.1405373740.08402903730.0311728395method 20.1938571960.11319499070.03502188550.90method 10.2899444170.16422360620.0679038026method 20.5520926890.27633018180.11124336330.95method 11.1197171370.47640266160.1984216218method 22.3066726681.31590086680.2217620895Left panel: plots of the quantile estimators for method 1 (solid line), method 2 (dotted line) and true quantile (dashed line) for different sample sizes and for data from a Log-normal distribution. Right panel: box plots of mean squared errors for the quantile estimators for method 1 and method 2 for different sample sizes.
(a)(b)(c)(d)(e)(f)(d) Double exponential distribution (see Table4 and Figure 4).Table 4
Mean squared errors results for bandwidth selection methods for different sample sizes and for data from a double exponential distribution.
pn=100n=200n=5000.05method 10.353724200.2882077420.251339747method 20.454588190.3157043200.0513853720.10method 10.071230720.0436841600.029307307method 20.148689520.0978710720.0236013680.15method 10.050817690.0253589460.009241326method 20.093772440.0352071510.0109102140.20method 10.024890790.0153602420.007647199method 20.049973480.0248643590.0080131590.25method 10.018638020.0122049040.004401402method 20.031179420.0191010330.0062472790.30method 10.018696110.0120311620.004145965method 20.025169320.0146803350.0048471910.35method 10.015622790.0095608730.003235724method 20.020174040.0113558080.0035133860.40method 10.014300680.0078607750.002493813method 20.016695050.0091652030.0026213450.45method 10.013863310.0075877050.002485022method 20.015296640.0082215010.0021042650.55method 10.015014580.0078010510.002013993method 20.012806130.0077964110.0022275690.60method 10.017122030.0090769220.002233672method 20.013944540.0094756050.0027912360.65method 10.019462410.0111298700.003521070method 20.018408940.0125589980.0036281690.70method 10.020983940.0119974050.003255335method 20.023330920.0157924660.0045340600.75method 10.027919430.0168854710.004419826method 20.029374570.0198521220.0054693590.80method 10.035328060.0213197140.005471649method 20.042946340.0247578040.0072701870.85method 10.054638900.0304899510.011338629method 20.084411440.0353064150.0120541820.90method 10.091886210.0585871640.030485192method 20.147554440.0838442320.0243994400.95method 10.281849450.2244323720.180645893method 20.514622090.3191474350.076406491Left panel: plots of the quantile estimators for method 1 (solid line), method 2 (dotted line) and true quantile (dashed line) for different sample sizes and for data from a double exponential distribution. Right panel: box plots of mean squared error for the quantile estimators for method 1 and method 2 for different sample sizes.
(a)(b)(c)(d)(e)(f)We can compute and summarize the relative efficiency ofhMethod1,opt for the all previous distributions in Table 5.Table 5
The relative efficiency (R.E) ofhMethod 1,opt.
nStandard normal dist.Exponential dist.Log normal dist.Double exponential dist.1001.0372762.6362501.8060821.5209032000.69863242.9528082.0963071.3086675000.44558282.3244231.1735470.4519134From Tables1, 2, 3, and 4,  for the all distributions, it can be observed that in 52.3% of cases our method produces lower mean squared errors, slightly wins Sheather-Marron method.Also, from Table5 which describes the relative efficiency for hMethod1,opt we can see hMethod1,opt more efficient from hMethod2,opt for all the cases except the standard normal distribution cases with n=200,500 and double exponential distribution cases with n=500.So, we may conclude that in terms of MISE our bandwidth selection method is more efficient than Sheather-Marron for skewed distributions but not for symmetric distributions.


## 6. Conclusion

In this paper we have a proposed a cross-validation-based-rule for the selection of bandwidth for quantile functions estimated by kernel procedure. The bandwidth selected by our proposed method is shown to be asymptotically unbiased and in order to assess the numerical performance, we conduct a simulation study and compare it with the bandwidth proposed by Sheather and Marron [1]. Based on the four distributions considered the proposed bandwidth selection appears to provide accurate estimates of quantiles and thus we believe that the new bandwidth selection method is a practically useful method to get bandwidth for the quantile estimator in the form (1.3).



---

*Source: 138450-2012-02-01.xml*