# Optimal Bandwidth Selection for Kernel Density Functionals Estimation

Authors: Su Chen
Journal: Journal of Probability and Statistics
 (2015)

---

## Abstract

The choice of bandwidth is crucial to the kernel density estimation (KDE) and kernel based regression. Various bandwidth selection methods for KDE and local least square regression have been developed in the past decade. It has been known that scale and location parameters are proportional to density functionals∫
γ
(
x
)
f
2
(
x
)
d
x with appropriate choice of γ
(
x
) and furthermore equality of scale and location tests can be transformed to comparisons of the density functionals among populations. ∫
γ
(
x
)
f
2
(
x
)
d
x can be estimated nonparametrically via kernel density functionals estimation (KDFE). However, the optimal bandwidth selection for KDFE of ∫
γ
(
x
)
f
2
(
x
)
d
x has not been examined. We propose a method to select the optimal bandwidth for the KDFE. The idea underlying this method is to search for the optimal bandwidth by minimizing the mean square error (MSE) of the KDFE. Two main practical bandwidth selection techniques for the KDFE of ∫
γ
(
x
)
f
2
(
x
)
d
x are provided: Normal scale bandwidth selection (namely, “Rule of Thumb”) and direct plug-in bandwidth selection. Simulation studies display that our proposed bandwidth selection methods are superior to existing density estimation bandwidth selection methods in estimating density functionals.


---

## Body

## 1. Introduction

Suppose that a random variableX with a probability density function (p.d.f.) f
(
x
) belongs to a location-scale family. Let μ and σ be the location and scale parameter of X, respectively. We have f
(
x
)
=
1
/
σ
f
0
(
x
-
μ
/
σ
) for some base function f
0
(
x
). If f
(
x
) is a symmetric function, then f
0
(
x
) is usually chosen to be the same class of distribution with mean zero. For instance, if f
(
x
) is the p.d.f. of Normal distribution with mean μ and standard deviation σ, then f
0
(
x
) is usually chosen to be the density of standard Normal distribution. In the nonparametric world, f
(
x
) is not assumed to have any prespecified distributional format. Therefore, μ and σ are unknown and can not be estimated by any distribution based method such as maximum likelihood estimate. Ahmad [1] proposed a nonparametric kernel estimation of location and scale parameters via density functionals estimation with known base functions. The location and scale functions are written in terms of density functionals as follows:(1)
σ
=
∫
f
0
2
x
d
x
∫
f
2
x
;
μ
,
σ
d
x
,
(2)
μ
=
∫
x
f
2
x
;
μ
,
σ
d
x
-
∫
x
f
0
2
x
d
x
∫
f
2
x
;
μ
,
σ
d
x
(3)
=
σ
∫
x
f
2
x
;
μ
,
σ
d
x
-
∫
x
f
0
2
x
d
x
∫
f
0
2
x
;
μ
,
σ
d
x
.Apparently, the location μ and scale σ only rely on two functionals of unknown density f
(
x
), namely, ∫
f
2
(
x
;
μ
,
σ
)
d
x and ∫
x
f
2
(
x
;
μ
,
σ
)
d
x, if f
0
(
x
) is known. Ahmad [1] showed that the new kernel location and scale estimates had better asymptotic property than MLE. Simulation results in Ahmad and Amezziane [2], a subsequent work of Ahmad [1], indicated that the kernel location and scale estimators have a comparable variability to that of the MLE and smaller than that of Huber’s M-estimator. However, it is usually difficult or impossible to know the base density especially in the nonparametric world. Moreover f
(
x
) can be derived in terms of μ and σ if the base density f
0
(
x
) is given. In this case, it becomes a parametric situation and MLE can be considered. From this point of view, Ahmad’s scale and location estimates are not very practical in real world application because the base density function f
0
(
x
) needs to be known first.Chen [3] proposed kernel-based nonparametric tests of equality of scale and location parameters among K populations based on the kernel scale and location estimators proposed by Ahmad [1]. To test H
0
:
σ
1
=
σ
2
=
⋯
=
σ
K is equivalent to test H
0
:
∫
f
1
2
(
x
)
d
x
=
∫
f
2
2
(
x
)
d
x
=
⋯
=
∫
f
K
2
(
x
)
d
x according to (1), where σ
i and f
i
(
x
) are the scale and density function of ith population, respectively, and i
=
1,2
,
…
,
K. Likewise, H
0
:
μ
1
=
μ
2
=
⋯
=
σ
K is equivalent to H
0
:
∫
x
f
1
2
(
x
)
d
x
=
∫
x
f
2
2
(
x
)
d
x
=
⋯
=
∫
x
f
K
2
(
x
)
d
x by (3) if homogeneous scale is assumed. This fact motivates Chen [3] to build test statistics for equality of scale and location on the density functionals estimation of ∫
f
i
2
(
x
)
d
x and ∫
x
f
i
2
(
x
)
d
x, respectively. Chen [3] brought a new life to the two kernel density functionals estimations, which were originally introduced to estimate location and scale parameter by Ahmad [1]. When comparing the scale (or location) parameters among K populations, the differences in scale (or location) can be completely determined by ∫
f
2
(
x
)
d
x (or ∫
x
f
2
(
x
)
d
x) and f
0
(
x
) becomes irrelevant if we assume  K populations are from same distributional family but differed only in locations and/or scales. Thus the assumption of having to know base density f
0
(
x
) as required in kernel scale and location estimation was successfully dropped. To find a good estimate of density functionals ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x becomes our next concern.Aubuchon and Hettmansperger [4] proposed a kernel estimation of ∫
f
2
(
x
)
d
x by a convolution of kernel density estimation function with the empirical CDF and showed its asymptotic equivalence to Lehmann’s estimator (see Lehmann [5] for details) based on the Wilcoxon confidence interval. Ahmad [1] provided two approaches to estimate ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x, one is similar to Aubuchon and Hettmansperger [4] and the other is to approximate the density function f
(
x
) with an orthogonal series expansion, and then estimate the functionals of density. Grübel [6] estimated the density functionals ∫
ϕ
(
f
(
x
)
)
d
x for known ϕ
(
·
) under certain conditions through the kernel density estimate of the unknown f
(
·
).Choice of bandwidth (window width or smoothing parameter) is crucial for every kernel based procedure, such as kernel density estimation and kernel regression. A vast amount of literatures has been devoted in choosing practical optimal bandwidth for techniques built on kernel estimation. Representative surveys of bandwidth selection techniques can be found in Bowman [10], Jones et al. [11], Loader [12], and Wand and Jones [13]. Jones et al. [11] grouped data-based bandwidth selection methods for density estimation into “first generation” method and “second generation” method. The first generation methods, including the least-square cross-validation (LSCV) in Bowman [8] and biased cross-validation in Scott and Terrell [14], suffer from a slow relative rate of convergence to h
MISE of order n
-
1
/
10. Härdle and Marron [15] applied the least squares cross-validation idea to bandwidth selection on Nadaraya-Watson estimator. The second generation methods are mainly based on plug-in techniques. The idea of “plug-in” is to replace f
′
′ with a consistent estimate f
^
′
′ first proposed by Nadaraya [16] and Woodroofe [17]; however, the practical choice of pilot bandwidth was not discussed. Sheather and Jones [9] proposed a refined plug-in method, so-called “solve-the-equation (STE)” plug-in h
^
STE, which has faster rate of convergence of order n
-
5
/
14 than cross-validation estimators. Smoothed cross-validation (SCV) is also a plug-in type method with pilot bandwidth of format g
=
C
n
p
h
m. It was developed by Hall et al. [18]. Müller [19] and Staniswalis [20] employed the idea in the kernel regression. Hall et al. [21] constructed root-n bandwidth selectors and achieved the optimal n
-
1
/
2 relative rate of convergence by appropriate choice of the parameters in pilot bandwidth g
=
C
n
p
h
m. Gasser et al. [22] and Ruppert et al. [23] borrowed the simple direct plug-in idea to local linear regression. Fan and Gijbels [24] applied “cross-validation technique, the Normal-reference method, and the plug-in approach” for the density estimation setting to their corresponding bandwidth selectors for local polynomial regression method.However, few literature studies the optimal bandwidth for the estimation of∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x, which are two important density functionals for estimating location and scale parameters as discussed in the prior paragraphs. Aubuchon and Hettmansperger [4] chose the bandwidth by removing the 1
/
n bias term. Grübel [6] suggested using the MISE-optimal choice of bandwidth in kernel density estimation. Chen [3] uses the least-square cross-validation bandwidth selection method for density estimation. In this paper, we will derive optimal bandwidth selection of kernel location and scale estimation by minimizing the MSE of the kernel functionals estimation for ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x. This paper will also propose two practical bandwidth selection methods and then compare them with various bandwidth selections for kernel density estimation such as Rule-of-Thumb, direct plug-in (DPI), least square cross-validation, and biased cross-validation (BCV).For simplicity of illustration, a unified format (i.e.,∫
γ
(
x
)
f
2
(
x
)
d
x) of the two density functionals mentioned above will be used throughout the paper. When γ
(
x
)
=
1 and x, it equals ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x, respectively. The paper is organized as follows. The optimal bandwidth for estimation ∫
γ
(
x
)
f
2
(
x
)
d
x in terms of AMSE criterion is derived in Section 2.1. Two practical bandwidth selection methods for ∫
γ
(
x
)
f
2
(
x
)
d
x are provided in Sections 2.2 and 2.3 when γ
(
x
)
=
1 and x. Asymptotic distribution of direct plug-in bandwidth for kernel functionals estimation of ∫
γ
(
x
)
f
2
(
x
)
d
x is given in Section 2.3 as well. Section 3 conducts three simulation studies to explore the properties of proposed bandwidth selection methods and evaluate their performance compared to several classical bandwidth selection methods for kernel density estimation.


## 2. Main Results

### 2.1. Optimal Bandwidth Selection

DefineU
1
=
∫
f
2
(
x
)
d
x and U
2
=
∫
x
f
2
(
x
)
d
x. Let us write U
1 and U
2 in a more general density functionals U
=
∫
γ
(
x
)
f
2
(
x
)
d
x. Note that U
1 and U
2 are special cases of U, where γ
(
x
) is 1 and x, respectively. Suppose X
1
,
X
2
,
…
,
X
n are n independent random variable from a distribution with density function f
(
x
), where f
(
x
) is unknown. Similar to Aubuchon and Hettmansperger [4] and Grübel [6], we obtain the kernel density functionals estimate of U by ∫
γ
(
x
)
f
^
n
(
x
)
d
F
n
(
x
), where f
^
n
(
x
) is the kernel density estimate of f and F
n
(
x
) is the empirical CDF. Thus, a kernel density functionals estimate of U is given by(4)
U
^
=
1
n
n
-
1
∑
i
=
1
n
∑
j
≠
i
n
γ
X
i
+
γ
X
j
2
K
h
X
i
-
X
j
,where K
h
(
·
)
=
(
1
/
h
)
K
(
·
/
h
) and K
(
·
) is the kernel function (details can be found in Wand’s book). The following theorem provides the mean and variance of U
^ in (4) for fixed h.Theorem 1.
ForU
^ in (4), the expected value and variance of U
^ are given by(5)
E
U
^
=
U
+
1
2
E
γ
y
f
′
′
y
μ
2
K
h
2
+
o
h
2
,
(6)
Var
⁡
U
^
=
4
n
-
1
∫
γ
2
y
f
3
y
d
y
-
∫
γ
y
f
2
y
d
y
2
+
2
n
-
2
h
-
1
R
K
∫
γ
2
y
f
2
y
d
y
+
o
n
-
1
+
o
n
-
2
h
-
1
,where R
(
g
)
=
∫
g
2
(
y
)
d
y and μ
2
(
g
)
=
∫
y
2
g
(
y
)
d
y.We prove this in AppendixA. The first term in (6) is nonnegative by Jensen’s inequality. Then the MSE of U
^ can be written as follows:(7)
MSE
U
^
=
Var
U
^
+
Bias
U
^
2
=
4
n
-
1
∫
γ
2
y
f
3
y
d
y
-
∫
γ
y
f
2
y
d
y
2
+
2
n
-
2
h
-
1
R
K
∫
γ
2
y
f
2
y
d
y
+
1
4
E
2
γ
y
f
′
′
y
μ
2
2
K
h
4
+
o
n
-
2
h
-
1
+
o
h
4
.Therefore, the optimal bandwidth selection for density functionals estimation of U
=
∫
γ
(
x
)
f
2
(
x
)
d
x is h
MSE
;
U, the minimizer of MSE
(
U
^
). To obtain a closed form of optimal bandwidth for kernel functionals estimation of U, the minimizer of the asymptotic mean square error (AMSE) of U
^ is studied instead. The optimal bandwidth for estimation of U with respect to AMSE criterion is given by(8)
h
AMSE
;
U
=
2
R
K
R
γ
x
f
x
∫
γ
x
f
′
′
x
f
x
d
x
2
μ
2
2
K
1
/
5
n
-
2
/
5
,where(9)
MSE
U
^
=
AMSE
U
^
+
o
n
-
1
+
o
h
4
.However, h
AMSE in (8) is not computable since R
(
γ
(
x
)
f
(
x
)
)
=
∫
γ
2
(
x
)
f
2
(
x
)
d
x and ∫
γ
(
x
)
f
′
′
(
x
)
f
(
x
)
d
x depend on unknown function f
(
x
). A quick and simple guess of AMSE-optimal bandwidth is “Normal scale” bandwidth. It gives reasonable answers whenever the data are close to Normal. In the next section, Normal scale bandwidth selection will be studied for γ
=
1 and x, respectively.


### 2.2. Normal Scale Bandwidth Selection

Whenγ
=
1, U
=
∫
γ
(
x
)
f
2
(
x
)
d
x reduces to U
1
=
∫
f
2
(
x
)
d
x. By (8) in Section 2.1, the bandwidth that minimizes MSE
(
U
^
1
) asymptotically is(10)
h
AMSE
;
U
1
=
2
R
K
R
f
E
2
f
′
′
y
μ
2
2
K
1
/
5
n
-
2
/
5
,where U
^
1 is the kernel density functionals estimation of U
1.Proposition 2.
Iff is Normal with mean 0 and variance σ
2 then the Normal scale AMSE-optimal bandwidth selector for U
^
1 is given by(11)
h
^
N
S
;
U
1
=
16
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where σ
^ is some estimate of σ.The proof of Proposition2 can be found in Appendix B. If Gaussian kernel is chosen, that is, K
(
·
) is the density of standard Normal distribution, then R
(
K
)
=
1
/
2
π and μ
2
(
K
)
=
1. Hence (11) is simplified to(12)
h
^
ROT
;
U
1
=
8
5
σ
^
n
-
2
/
5
≃
1.515717
σ
^
n
-
2
/
5
,which can be called “Rule-of-Thumb” (ROT) bandwidth selector for kernel scale estimation.Whenγ
=
x, U becomes U
2
=
∫
x
f
2
(
x
)
d
x. The bandwidth selector that minimizes AMSE
(
U
^
2
) is(13)
h
AMSE
;
U
2
=
2
R
K
R
x
f
x
E
2
x
f
′
′
x
μ
2
2
K
1
/
5
n
-
2
/
5
,followed by (8) in Section 2.1.Proposition 3.
Iff is Normal with mean μ and variance σ
2 then the Normal scale AMSE-optimal bandwidth selector for U
^
2 is given by(14)
h
^
N
S
;
U
2
=
2
+
σ
^
2
μ
^
2
8
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where σ
^ is an estimate of σ and μ
^ is an estimate of μ. If σ
^
=
s (sample standard deviation) and μ
^
=
x
¯ (sample mean) then (14) can be rewritten as(15)
h
^
N
S
;
U
2
=
2
+
c
^
v
2
8
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where c
^
v
=
s
/
x
¯ is the coefficient of variation (CV). Particularly when μ
=
0 for fixed σ, h
^
N
S
;
U
2 goes to infinity.The proof of Proposition3 is given in Appendix C. When kernel function K
(
·
) is the density of standard Normal distribution, then the “Rule-of-Thumb” bandwidth selector for kernel location estimation is(16)
h
^
ROT
;
U
2
=
4
2
+
c
^
v
2
5
σ
^
n
-
2
/
5
.Both (14) and (16) infer that the larger the location of f
(
x
) in absolute value is, the smaller the optimal bandwidth is needed. In another word, the optimal-AMSE bandwidth for N
(
0
,
σ
2
) goes to infinity. This fact also not merely applies to Normal with zero mean but also can be extended to any distribution with p.d.f. an even function (symmetric distribution around zero).Corollary 4.
For any distribution with even density functionf
(
x
), that is, f
(
-
x
)
=
f
(
x
), then the optimal-AMSE bandwidth selector is h
A
M
S
E
;
U
2
=
∞.Remarks. (1) The optimal bandwidth for estimation of U
1 is not effected by the location of f
(
x
), but the scale parameter. However, the optimal bandwidth for estimation of U
2 not only depends on scale but also varies along with the location. This fact will be illustrated by the simulation study in Section 3.1. Note that scale parameter is determined by U
1 and location parameter is determined by U
2.(2) The common choice of(
σ
^
) is sample standard deviation s as in Silverman [7]. However, Wand and Jones [13] recommended the smaller value between s and interquartile range σ
^
IQR. Janssen et al. [25] also studied other more sophisticated estimates of σ.


### 2.3. Direct Plug-In Bandwidth Selection

If the distribution ofX
i’s, that is, f
(
x
), departs far from Normal distribution, then Normal scale bandwidth selector will be problematic. Note that R
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y in (8) are unknown and need to be estimated to obtain a practical optimal bandwidth selector. A natural estimate of R
(
γ
(
y
)
f
(
y
)
) is(17)
R
^
=
∫
γ
2
y
f
^
y
d
F
n
y
=
1
n
n
-
1
∑
i
=
1
n
∑
j
≠
i
n
γ
2
X
i
+
γ
2
X
j
2
K
g
X
i
-
X
j
.Similarly, ϕ
=
∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y can be estimated by(18)
ϕ
^
=
∫
γ
y
f
^
′
′
y
d
F
n
y
=
1
n
n
-
1
g
3
·
∑
i
=
1
n
∑
j
≠
i
n
γ
X
i
+
γ
X
j
2
K
′
′
X
i
-
X
j
g
.Replacement of R
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y by R
^ and ϕ
^ leads to thedirect plug-in (DPI) bandwidth selector for U
=
∫
γ
(
x
)
f
2
(
x
)
d
x:(19)
h
^
DPI
;
U
=
2
R
K
R
^
μ
2
2
K
ϕ
^
2
1
/
5
n
-
2
/
5
.Obviously, the kernel density functionals estimates in (17) and (18) rely on the choice of pilot bandwidth g. Simple candidates for pilot bandwidth are to useNormal scale bandwidth selector proposed in Section 2.2 for ∫
γ
(
x
)
f
2
(
x
)
d
x or smoothing parameters for traditional density estimate (e.g., ROT, LSCV, BCV, and DPI surveyed in Wand and Jones [13]). The DPI bandwidth selection can be practically computed through the following procedures.Step 1.
Estimateg using the Normal scale bandwidth proposed in Section 2.2 (i.e., h
^
NS
;
U
1 for estimation of U
1 and h
^
NS
;
U
2 for estimation of U
2) or bandwidth selection for density estimation (such as ROT [7], LSCV [8], BVC [14], and DPI [9]).Step 2.
EstimateR
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y using R
^
(
g
) in (17) and ϕ
^
(
g
) in (18).Step 3.
The DPI bandwidth selection forU is obtained followed by (19).The performance of these pilot bandwidth selections is compared in terms of MSE ofU
^ through Monte Carlo simulation in Section 3.2 (Simulation Study  2). Next, we will study the asymptotic distribution of h
^
DPI
;
U. The limiting distribution of practical bandwidth selector is very important in that the rate of convergence is the chief concern.Proposition 5.
Ifγ
(
y
) and density function f
(
y
) are continuous and satisfy ∫
γ
2
(
y
)
f
2
(
y
)
d
y
<
∞ and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y
<
∞, then(20)
n
1
/
5
h
^
D
P
I
;
U
h
M
S
E
;
U
-
1
⟶
D
N
0
,
σ
D
P
I
;
U
2
.The proof of Proposition5 is provided in Appendix D. Thus the direct plug-in bandwidth selection for functional density estimation h
^
DPI
;
U has relative convergence rate of order n
-
1
/
5.Remarks. Particularly, when γ
(
y
)
=
1, the DPI bandwidth selector for estimation of U
1 is h
^
DPI
;
U
1
=
[
2
R
(
K
)
R
^
1
/
μ
2
2
(
K
)
ϕ
^
1
2
]
1
/
5
n
-
2
/
5, where R
^
1
=
1
/
n
(
n
-
1
)
∑
i
=
1
n
∑
j
≠
i
n
K
g
(
X
i
-
X
j
) and ϕ
^
1
=
1
/
n
(
n
-
1
)
g
3
∑
i
=
1
n
∑
j
≠
i
n
K
′
′
{
(
X
i
-
X
j
)
/
g
}. Likewise, when γ
(
y
)
=
y, the DPI bandwidth selector for estimation of U
2 is h
^
DPI
;
U
2
=
[
2
R
(
K
)
R
^
2
/
μ
2
2
(
K
)
ϕ
^
2
2
]
1
/
5
n
-
2
/
5, where R
^
2
=
1
/
n
(
n
-
1
)
∑
i
=
1
n
∑
j
≠
i
n
X
i
2
+
X
j
2
/
2
K
g
(
X
i
-
X
j
) and ϕ
^
2
=
1
/
n
(
n
-
1
)
g
3
∑
i
=
1
n
∑
j
≠
i
n
X
i
+
X
j
/
2
K
′
′
{
(
X
i
-
X
j
)
/
g
}.


## 2.1. Optimal Bandwidth Selection

DefineU
1
=
∫
f
2
(
x
)
d
x and U
2
=
∫
x
f
2
(
x
)
d
x. Let us write U
1 and U
2 in a more general density functionals U
=
∫
γ
(
x
)
f
2
(
x
)
d
x. Note that U
1 and U
2 are special cases of U, where γ
(
x
) is 1 and x, respectively. Suppose X
1
,
X
2
,
…
,
X
n are n independent random variable from a distribution with density function f
(
x
), where f
(
x
) is unknown. Similar to Aubuchon and Hettmansperger [4] and Grübel [6], we obtain the kernel density functionals estimate of U by ∫
γ
(
x
)
f
^
n
(
x
)
d
F
n
(
x
), where f
^
n
(
x
) is the kernel density estimate of f and F
n
(
x
) is the empirical CDF. Thus, a kernel density functionals estimate of U is given by(4)
U
^
=
1
n
n
-
1
∑
i
=
1
n
∑
j
≠
i
n
γ
X
i
+
γ
X
j
2
K
h
X
i
-
X
j
,where K
h
(
·
)
=
(
1
/
h
)
K
(
·
/
h
) and K
(
·
) is the kernel function (details can be found in Wand’s book). The following theorem provides the mean and variance of U
^ in (4) for fixed h.Theorem 1.
ForU
^ in (4), the expected value and variance of U
^ are given by(5)
E
U
^
=
U
+
1
2
E
γ
y
f
′
′
y
μ
2
K
h
2
+
o
h
2
,
(6)
Var
⁡
U
^
=
4
n
-
1
∫
γ
2
y
f
3
y
d
y
-
∫
γ
y
f
2
y
d
y
2
+
2
n
-
2
h
-
1
R
K
∫
γ
2
y
f
2
y
d
y
+
o
n
-
1
+
o
n
-
2
h
-
1
,where R
(
g
)
=
∫
g
2
(
y
)
d
y and μ
2
(
g
)
=
∫
y
2
g
(
y
)
d
y.We prove this in AppendixA. The first term in (6) is nonnegative by Jensen’s inequality. Then the MSE of U
^ can be written as follows:(7)
MSE
U
^
=
Var
U
^
+
Bias
U
^
2
=
4
n
-
1
∫
γ
2
y
f
3
y
d
y
-
∫
γ
y
f
2
y
d
y
2
+
2
n
-
2
h
-
1
R
K
∫
γ
2
y
f
2
y
d
y
+
1
4
E
2
γ
y
f
′
′
y
μ
2
2
K
h
4
+
o
n
-
2
h
-
1
+
o
h
4
.Therefore, the optimal bandwidth selection for density functionals estimation of U
=
∫
γ
(
x
)
f
2
(
x
)
d
x is h
MSE
;
U, the minimizer of MSE
(
U
^
). To obtain a closed form of optimal bandwidth for kernel functionals estimation of U, the minimizer of the asymptotic mean square error (AMSE) of U
^ is studied instead. The optimal bandwidth for estimation of U with respect to AMSE criterion is given by(8)
h
AMSE
;
U
=
2
R
K
R
γ
x
f
x
∫
γ
x
f
′
′
x
f
x
d
x
2
μ
2
2
K
1
/
5
n
-
2
/
5
,where(9)
MSE
U
^
=
AMSE
U
^
+
o
n
-
1
+
o
h
4
.However, h
AMSE in (8) is not computable since R
(
γ
(
x
)
f
(
x
)
)
=
∫
γ
2
(
x
)
f
2
(
x
)
d
x and ∫
γ
(
x
)
f
′
′
(
x
)
f
(
x
)
d
x depend on unknown function f
(
x
). A quick and simple guess of AMSE-optimal bandwidth is “Normal scale” bandwidth. It gives reasonable answers whenever the data are close to Normal. In the next section, Normal scale bandwidth selection will be studied for γ
=
1 and x, respectively.


## 2.2. Normal Scale Bandwidth Selection

Whenγ
=
1, U
=
∫
γ
(
x
)
f
2
(
x
)
d
x reduces to U
1
=
∫
f
2
(
x
)
d
x. By (8) in Section 2.1, the bandwidth that minimizes MSE
(
U
^
1
) asymptotically is(10)
h
AMSE
;
U
1
=
2
R
K
R
f
E
2
f
′
′
y
μ
2
2
K
1
/
5
n
-
2
/
5
,where U
^
1 is the kernel density functionals estimation of U
1.Proposition 2.
Iff is Normal with mean 0 and variance σ
2 then the Normal scale AMSE-optimal bandwidth selector for U
^
1 is given by(11)
h
^
N
S
;
U
1
=
16
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where σ
^ is some estimate of σ.The proof of Proposition2 can be found in Appendix B. If Gaussian kernel is chosen, that is, K
(
·
) is the density of standard Normal distribution, then R
(
K
)
=
1
/
2
π and μ
2
(
K
)
=
1. Hence (11) is simplified to(12)
h
^
ROT
;
U
1
=
8
5
σ
^
n
-
2
/
5
≃
1.515717
σ
^
n
-
2
/
5
,which can be called “Rule-of-Thumb” (ROT) bandwidth selector for kernel scale estimation.Whenγ
=
x, U becomes U
2
=
∫
x
f
2
(
x
)
d
x. The bandwidth selector that minimizes AMSE
(
U
^
2
) is(13)
h
AMSE
;
U
2
=
2
R
K
R
x
f
x
E
2
x
f
′
′
x
μ
2
2
K
1
/
5
n
-
2
/
5
,followed by (8) in Section 2.1.Proposition 3.
Iff is Normal with mean μ and variance σ
2 then the Normal scale AMSE-optimal bandwidth selector for U
^
2 is given by(14)
h
^
N
S
;
U
2
=
2
+
σ
^
2
μ
^
2
8
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where σ
^ is an estimate of σ and μ
^ is an estimate of μ. If σ
^
=
s (sample standard deviation) and μ
^
=
x
¯ (sample mean) then (14) can be rewritten as(15)
h
^
N
S
;
U
2
=
2
+
c
^
v
2
8
π
R
K
μ
2
2
K
1
/
5
σ
^
n
-
2
/
5
,where c
^
v
=
s
/
x
¯ is the coefficient of variation (CV). Particularly when μ
=
0 for fixed σ, h
^
N
S
;
U
2 goes to infinity.The proof of Proposition3 is given in Appendix C. When kernel function K
(
·
) is the density of standard Normal distribution, then the “Rule-of-Thumb” bandwidth selector for kernel location estimation is(16)
h
^
ROT
;
U
2
=
4
2
+
c
^
v
2
5
σ
^
n
-
2
/
5
.Both (14) and (16) infer that the larger the location of f
(
x
) in absolute value is, the smaller the optimal bandwidth is needed. In another word, the optimal-AMSE bandwidth for N
(
0
,
σ
2
) goes to infinity. This fact also not merely applies to Normal with zero mean but also can be extended to any distribution with p.d.f. an even function (symmetric distribution around zero).Corollary 4.
For any distribution with even density functionf
(
x
), that is, f
(
-
x
)
=
f
(
x
), then the optimal-AMSE bandwidth selector is h
A
M
S
E
;
U
2
=
∞.Remarks. (1) The optimal bandwidth for estimation of U
1 is not effected by the location of f
(
x
), but the scale parameter. However, the optimal bandwidth for estimation of U
2 not only depends on scale but also varies along with the location. This fact will be illustrated by the simulation study in Section 3.1. Note that scale parameter is determined by U
1 and location parameter is determined by U
2.(2) The common choice of(
σ
^
) is sample standard deviation s as in Silverman [7]. However, Wand and Jones [13] recommended the smaller value between s and interquartile range σ
^
IQR. Janssen et al. [25] also studied other more sophisticated estimates of σ.


## 2.3. Direct Plug-In Bandwidth Selection

If the distribution ofX
i’s, that is, f
(
x
), departs far from Normal distribution, then Normal scale bandwidth selector will be problematic. Note that R
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y in (8) are unknown and need to be estimated to obtain a practical optimal bandwidth selector. A natural estimate of R
(
γ
(
y
)
f
(
y
)
) is(17)
R
^
=
∫
γ
2
y
f
^
y
d
F
n
y
=
1
n
n
-
1
∑
i
=
1
n
∑
j
≠
i
n
γ
2
X
i
+
γ
2
X
j
2
K
g
X
i
-
X
j
.Similarly, ϕ
=
∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y can be estimated by(18)
ϕ
^
=
∫
γ
y
f
^
′
′
y
d
F
n
y
=
1
n
n
-
1
g
3
·
∑
i
=
1
n
∑
j
≠
i
n
γ
X
i
+
γ
X
j
2
K
′
′
X
i
-
X
j
g
.Replacement of R
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y by R
^ and ϕ
^ leads to thedirect plug-in (DPI) bandwidth selector for U
=
∫
γ
(
x
)
f
2
(
x
)
d
x:(19)
h
^
DPI
;
U
=
2
R
K
R
^
μ
2
2
K
ϕ
^
2
1
/
5
n
-
2
/
5
.Obviously, the kernel density functionals estimates in (17) and (18) rely on the choice of pilot bandwidth g. Simple candidates for pilot bandwidth are to useNormal scale bandwidth selector proposed in Section 2.2 for ∫
γ
(
x
)
f
2
(
x
)
d
x or smoothing parameters for traditional density estimate (e.g., ROT, LSCV, BCV, and DPI surveyed in Wand and Jones [13]). The DPI bandwidth selection can be practically computed through the following procedures.Step 1.
Estimateg using the Normal scale bandwidth proposed in Section 2.2 (i.e., h
^
NS
;
U
1 for estimation of U
1 and h
^
NS
;
U
2 for estimation of U
2) or bandwidth selection for density estimation (such as ROT [7], LSCV [8], BVC [14], and DPI [9]).Step 2.
EstimateR
(
γ
(
y
)
f
(
y
)
) and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y using R
^
(
g
) in (17) and ϕ
^
(
g
) in (18).Step 3.
The DPI bandwidth selection forU is obtained followed by (19).The performance of these pilot bandwidth selections is compared in terms of MSE ofU
^ through Monte Carlo simulation in Section 3.2 (Simulation Study  2). Next, we will study the asymptotic distribution of h
^
DPI
;
U. The limiting distribution of practical bandwidth selector is very important in that the rate of convergence is the chief concern.Proposition 5.
Ifγ
(
y
) and density function f
(
y
) are continuous and satisfy ∫
γ
2
(
y
)
f
2
(
y
)
d
y
<
∞ and ∫
γ
(
y
)
f
′
′
(
y
)
f
(
y
)
d
y
<
∞, then(20)
n
1
/
5
h
^
D
P
I
;
U
h
M
S
E
;
U
-
1
⟶
D
N
0
,
σ
D
P
I
;
U
2
.The proof of Proposition5 is provided in Appendix D. Thus the direct plug-in bandwidth selection for functional density estimation h
^
DPI
;
U has relative convergence rate of order n
-
1
/
5.Remarks. Particularly, when γ
(
y
)
=
1, the DPI bandwidth selector for estimation of U
1 is h
^
DPI
;
U
1
=
[
2
R
(
K
)
R
^
1
/
μ
2
2
(
K
)
ϕ
^
1
2
]
1
/
5
n
-
2
/
5, where R
^
1
=
1
/
n
(
n
-
1
)
∑
i
=
1
n
∑
j
≠
i
n
K
g
(
X
i
-
X
j
) and ϕ
^
1
=
1
/
n
(
n
-
1
)
g
3
∑
i
=
1
n
∑
j
≠
i
n
K
′
′
{
(
X
i
-
X
j
)
/
g
}. Likewise, when γ
(
y
)
=
y, the DPI bandwidth selector for estimation of U
2 is h
^
DPI
;
U
2
=
[
2
R
(
K
)
R
^
2
/
μ
2
2
(
K
)
ϕ
^
2
2
]
1
/
5
n
-
2
/
5, where R
^
2
=
1
/
n
(
n
-
1
)
∑
i
=
1
n
∑
j
≠
i
n
X
i
2
+
X
j
2
/
2
K
g
(
X
i
-
X
j
) and ϕ
^
2
=
1
/
n
(
n
-
1
)
g
3
∑
i
=
1
n
∑
j
≠
i
n
X
i
+
X
j
/
2
K
′
′
{
(
X
i
-
X
j
)
/
g
}.


## 3. Simulation Study

Three simulation studies are carried out to evaluate [Simulation Study  1] the accuracy of h
^
NS
;
U
1 and h
^
NS
;
U
2 (Normal scale bandwidth for U
1 and U
2) comparing to h
AMSE
;
U and h
MSE
;
U under normality assumption; [Simulation Study  2] the optimal choices of pilot bandwidth for h
^
DPI
;
U
1 and h
^
DPI
;
U
2 in terms of MSE of U
^
1 and U
^
2, respectively; [Simulation Study  3] the performance of proposed practical optimal bandwidth selection methods (ROT and DPI proposed in Sections 2.2 and 2.3) versus traditional (classical) bandwidth selection for kernel density estimate in terms of MSE of U
^. As to the choice of kernel function K
(
·
), it has been shown in literatures that the choice of bandwidth overrides the effect of choice of kernel function. So for simplicity, we just use the Gaussian kernel in all the three simulation studies.


### 3.1.Simulation Study  1

The purpose of this study is threefold (1) to evaluate the performance ofh
^
NS
;
U
1 and h
^
NS
;
U
2 when samples are from Normal distribution, (2) to study h
MSE
;
U
1 and h
MSE
;
U
2 (the optimal bandwidths that minimize the MSE of U
^
1 and U
^
2, resp.) in terms of the location parameter of Normal distribution, and (3) to illustrate numerically that optimal bandwidth that minimizes the MSE of U
2 goes to infinity when location parameter gets closer to zero.Figure1 plots the MSE of U
^
1 versus the choice of bandwidth when sample of sizes 20, 50, 100, and 200 is drawn from N
(
0
,
2
) and N
(
1
,
2
), respectively (the simulation result is not sensitive to the choice of scale). The blue curve in each subplot represents the MSE(U
^
1) as bandwidth ranges from 0 to 2. The minimum point of the blue curve indicates h
MSE
;
U
1. The red vertical line in the subplot represents   h
AMSE
;
U
1 and is computed from (10) by replacing f
(
y
) with the p.d.f. of N
(
μ
,
2
), where μ
=
0 in Figure 1(a) and μ
=
1 in Figure 1(b). h
^
NS
;
U
1 is an estimate of h
AMSE
;
U
1 (an asymptotic approximation of h
MSE
;
U
1) under normality assumption. Simulation results in Figure 1 show that h
^
NS
;
U
1 tends to have small variance and stabilized around the true h
MSE
;
U
1 for normality data. The optimal bandwidth does not change with location parameter μ as shown in Figure 1(a) (μ
=
0) and Figure 1(b) (μ
=
1) (more simulation results based on location parameters other than 0 and 1 are available upon request.).Figure 1
The MSE ofU
^
1 when underlying distribution is Normal with mean μ
=
0 (a), μ
=
1 (b) and standard deviation 2. The blue curve is the MSE(U
^
1) versus the bandwidth h. The boxplot of h
^
ROT
;
U
1 is based on the 100 sets of simulated samples of size n
=
20,50,100,200 from Normal distribution with mean μ and standard deviation 2. The red vertical line represents h
AMSE
;
U
1 if f is Normal with known mean 0 and variance σ
2.
(a)
μ
=
0
(b)
μ
=
1Figure2 plots the MSE of U
^
2 versus the choice of bandwidth when sample of sizes 20, 50, 100, and 200 is drawn from N
(
0
,
2
), N
(
0.5
,
2
), N
(
1
,
2
), and N
(
5
,
2
), respectively. Similar to Figure 1, the blue curve and red vertical line represent MSE(U
^
2) and h
AMSE
;
U
2. The boxplot of h
^
NS
;
U
2 is based on the 100 sets of simulated samples of size n
=
10,50,100,200 from Normal distribution with mean μ
=
0,0.5,1
,
5 and standard deviation 2. The red vertical line disappears in Figure 1(a) due to the fact that h
AMSE
;
U
2
→
∞ as μ
→
0. Also the MSE of U
^
2 (blue curve) strictly decreases as h rises in Figure 2(a). To conclude from Figure 1(a), the optimal bandwidth for kernel functional estimation of U
2 goes to infinity when mean of underlying distribution is zero, which is consistent to Proposition 3. However, the boxplot in Figure 2(a) infers that the distribution of h
^
NS
;
U
2 is right-skewed with median, 1st quantile, and 3rd quantile around one, which is far departure from the true value h
MSE
;
U
2 as well as h
AMSE
;
U
2. When μ slightly deviates from zero, just as the case in Figures 2(b) and 2(c), h
^
NS
;
U
2 tends to be less variate (and skewed), overlap with h
AMSE
;
U
2 (red vertical line), and get closer to h
MSE
;
U
2 (valley of blue curve), especially as the sample size grows. When μ increases up to 1 and above as shown in Figure 2(d), the median of h
^
NS
;
U
2, h
AMSE
;
U
2, and h
MSE
;
U
2 coincide when sample size is 50+. More simulation results can be found in Supplementary Material available online at http://dx.doi.org/10.1155/2015/242683.Figure 2
The bandwidth forU
^
2 when underlying distribution is Normal with mean μ
=
0 (a), μ
=
0.5 (b), μ
=
1 (c), and μ
=
5 (d) and standard deviation 2.
(a)
μ
=
0
(b)
μ
=
0.5
(c)
μ
=
1
(d)
μ
=
5


### 3.2.Simulation Study  2

Several candidate bandwidth selection methods are available to serve as a pilot bandwidthg, such as classical bandwidth selection methods for kernel density estimate (described in Section 2.3 Step 1), the optimal bandwidth for U (U
1 or U
2) on Normal scale references, namely, “h
^
NS
;
U
1” and “h
^
NS
;
U
2” in Section 2.2. This subsection aims to study the pilot bandwidth for R
^ and ϕ
^ required to estimate U in (19) (note that U is simplified to U
1 and U
2 when γ
(
x
)
=
1 and x, resp.). Five different choices of pilot bandwidth are studied in this subsection to estimate R
^ and ϕ
^ for h
^
DPI
;
U under five different underlying distributions: (i) Cauchy with location (μ was set to be 1 in simulation study for estimation of U
1 and 5 for estimation of U
2)  μ and scale 2/3, (ii) Generalized Pareto with location μ, scale 2/3, and shape 1, (iii) Normal with location μ and scale 2/3, (iv) Mix-Normal I, and (v) Mix-Normal II. Mix-Normal I and Mix-Normal II are weighted mixtures of two Normal distributions: Mix-Normal I is 1
/
2
N
(
-
2,2
/
3
)
+
1
/
2
N
(
0,2
/
3
) and Mix-Normal I is 3
/
4
N
(
0,1
)
+
1
/
2
N
(
1.5,1
/
3
). Cauchy distribution is a well-known fat-tailed symmetric distribution. Generalized Pareto with shape 1 is a extremely fat-tailed asymmetric (right-skewed) distribution. Density curves of the two Mix-Normal -distributions are given in Figure 3. It is obvious that both Mix-Normal I and Mix-Normal II are bimodal distributions in contrast to unimodal distributions such as Cauchy, Generalized Pareto, and Normal. Mix-Normal I is symmetric bimodal and Mix-Normal II is asymmetric (left-skewed) bimodal distribution. The motivation behind the choice of distributions is to see whether the optimal bandwidth is sensitive to the skewness, extreme outliers, and complex shape of the distributions in contrast to Normal.Figure 3
Density curve of (a) Mix-Normal I:1
/
2
N
(
-
2,2
/
3
)
+
1
/
2
N
(
0,2
/
3
); (b) Mix-Normal II: 3
/
4
N
(
0,1
)
+
1
/
2
N
(
1.5,1
/
3
).
(a)
(b)Figures4 and 5 compare five candidate pilot bandwidth selection methods in terms of the boxplots of MSE of U
^
1 and U
^
2 when sample size is 100. The five candidates for pilot bandwidth selection considered in this paper are (1) Rule-of-Thumb bandwidth for U
1 and U
2 proposed in Section 2.2 (“1-ROT(s)” means g
^
ROT
;
U
1; “1-ROT(L)” means g
^
ROT
;
U
2); (2) Rule-of-Thumb bandwidth for kernel density estimation (KDE) proposed in Scott [26] (“2-ROT(d)” means g
^
ROT
;
density
=
1.06
∗
σ
^
n
-
1
/
5, where σ
^ is the minimum of standard deviation and interquantile range); (3) least-square cross-validation bandwidth for density estimation proposed in Bowman [8] (“3-UCV(d)” means g
^
LSCV
;
density for density estimation); (4) biased cross-validation for density estimation proposed in Scott and Terrell [14] (“4-BCV(d)” means g
^
BCV
;
density for density estimation); (5) direct plug-in bandwidth for density estimation reported by Sheather [27] (“5-DPI(d)” means g
^
DPI
;
density for density estimation). As shown in Figures 4 and 5 that g
^
BCV
;
density is theworst candidate for pilot bandwidth in the density functionals estimation ∫
γ
(
x
)
f
(
x
)
d
x. The pilot bandwidth choice g
^
ROT
;
density gives the lowest MSE(U
^
1) for Cauchy and Normal samples, while g
^
LSCV
;
density results in the lowest MSE(U
^
1) for Generalized Pareto, Mix-Normal I, and Mix-Normal II samples. Similar conclusions can be found in the pilot bandwidth choice for estimation of U
2; however, g
^
ROT
;
U
2 leads to slightly smaller MSE(U
^
2) than g
^
LSCV
;
density for Mix-Normal I and Mix-Normal II samples.Simulation Study  1 illustrates that the Normal reference bandwidth (including h
^
NS
;
U
2 and h
^
ROT
;
U
2) is not a reliable estimate when the location parameter of underlying distribution is close to zero. Therefore, g
^
LSCV
;
density, rather than h
^
ROT
;
U
2, is recommended to serve as pilot bandwidth in estimation of h
^
DPI
;
U
2.Figure 4
MSE ofU
^
1 in terms of the choice of pilot bandwidth g for h
^
DPI
;
U
1; “1-ROT(s)” means g
^
ROT
;
U
1; “2-ROT(d)” means g
^
ROT
;
density; “3-UCV(d)” means g
^
LSCV
;
density; “4-BCV(d)” means g
^
BCV
;
density; “5-DPI(d)” means g
^
DPI
;
density.
(a)
Cauchy
(b)
Pareto
(c)
Normal
(d)
Mix-Normal I
(e)
Mix-Normal IIFigure 5
MSE ofU
^
2 in terms of the choice of pilot bandwidth g for h
^
DPI
;
U
2; “1-ROT(L)” means g
^
ROT
;
U
2; “2-ROT(d)” means g
^
ROT
;
density; “3-UCV(d)” means g
^
LSCV
;
density; “4-BCV(d)” means g
^
BCV
;
density; “5-DPI(d)” means g
^
DPI
;
density.
(a)
Cauchy
(b)
Pareto
(c)
Normal
(d)
Mix-Normal I
(e)
Mix-Normal II


### 3.3.Simulation Study  3

This section aims to evaluate the performance of our proposed bandwidthh
^
ROT
;
U
1 (or h
^
ROT
;
U
2) in Section 2.2 and h
^
DPI
;
U
1 (or h
^
DPI
;
U
2) in Section 2.3 and compare with the classical bandwidth selection methods for density estimation, that is, h
^
ROT
;
density, h
^
LSCV
;
density, and h
^
DPI
;
density in estimation of U
1 (or U
2).Simulation Study  2 recommends g
^
ROT
;
density and g
^
LSCV
;
density to be pilot bandwidth to estimate direct plug-in bandwidth for estimation of U
1 and U
2 among the 5 candidate pilot bandwidth methods and 5 different underlying distributions. Therefore, h
^
DPI
;
U
1 (or h
^
DPI
;
U
2) with these two pilot bandwidth selection methods, g
^
ROT
;
density and g
^
LSCV
;
density,  is considered separately in this subsection. Direct plug-in bandwidth for estimation of U
1 with pilot bandwidth g
^
ROT
;
density is denoted by “h
^
DPI
a
(
s
)” in Table 1 and “2a-DPI(s)” in Figure 1, and with pilot bandwidth g
^
LSCV
;
density is denoted by “h
^
DPI
b
(
s
)" in Table 1 and “2b-DPI(s)” in Figure 1.Table 1
Optimal bandwidth selection in estimation ofU
1 with comparison to classical bandwidth selection methods.
Bandwidth selection for estimation ofU
1
Proposed method
Classical bandwidth for KDE
h
^
R
O
T
(
s
)

1
h
^
D
P
I
a
(
s
)

2
h
^
D
P
I
b
(
s
)

3
h
^
R
O
T
(
d
)

4
h
^
U
C
V
(
d
)

5
h
^
D
P
I
(
d
)

6
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
Normal
Mean
.197
.154
.236
.171
.221
.163
.302
.270
.359
.298
.300
.268
1st quantile
.182
.144
.205
.156
.200
.153
.278
.254
.307
.258
.265
.243
Median
.197
.154
.233
.170
.227
.167
.301
.270
.376
.317
.300
.268
3rd quantile
.214
.163
.265
.185
.254
.183
.327
.287
.431
.355
.336
.293
Cauchy
Mean
.325
.240
.320
.222
.290
.205
.497
.422
.440
.331
.419
.338
1st quantile
.273
.214
.264
.198
.230
.174
.417
.377
.330
.265
.348
.310
Median
.316
.236
.312
.219
.295
.207
.484
.415
.455
.353
.416
.342
3rd quantile
.365
.265
.368
.245
.355
.238
.557
.465
.554
.415
.478
.387
Pareto
Mean
.442
.332
.263
.178
.145
.098
.676
.583
.207
.143
.374
.284
1st quantile
.329
.267
.210
.151
.103
.078
.502
.469
.140
.106
.298
.241
Median
.422
.320
.251
.174
.137
.091
.646
.563
.200
.131
.359
.278
3rd quantile
.519
.386
.300
.202
.174
.109
.794
.678
.260
.173
.428
.327
Mix-Normal I
Mean
.378
.289
.389
.279
.349
.253
.578
.507
.522
.413
.495
.417
1st Quantile
.360
.279
.342
.252
.282
.211
.551
.490
.389
.320
.438
.376
Median
.377
.288
.382
.273
.362
.259
.577
.506
.510
.403
.489
.407
3rd Quantile
.397
.298
.431
.309
.428
.304
.607
.524
.658
.510
.544
.460
Mix-Normal II
Mean
.342
.263
.364
.255
.322
.227
.522
.461
.492
.372
.467
.389
1st Quantile
.321
.251
.318
.229
.254
.188
.490
.441
.339
.279
.413
.347
Median
.342
.262
.361
.254
.345
.231
.523
.461
.509
.355
.464
.387
3rd Quantile
.364
.274
.403
.278
.397
.275
.556
.481
.642
.461
.519
.425
1ROT bandwidth for estimation of U
1 given by (12); 2DPI bandwidth for estimation of U
1 with pilot bandwidth g
^
R
O
T
;
d
e
n
s
i
t
y; 3DPI bandwidth for estimation of U
1 with pilot bandwidth g
^
L
S
C
V
;
d
e
n
s
i
t
y; 4ROT bandwidth for density estimation proposed in Silverman [7]; 5LSCV bandwidth for density estimation proposed in Bowman [8]; 6DPI bandwidth for density estimation proposed in Sheather and Jones [9].The summary statistics (mean, 1st quantile, median, and 3rd quantile) for the three proposed bandwidths and 3 classical bandwidths are provided in Tables1 and 2 to estimate U
1 and U
2, respectively. Samples of size 50 and 100 from five different underlying distribution (3 unimodal distributions and 2 bimodal distributions) as inSimulation Study  2 are considered in both simulation studies of U
1 and U
2 estimation.Table 2
Optimal bandwidth selection in estimation ofU
2 with comparison to classical bandwidth selection methods.
Bandwidth selection for estimation ofU
2
Proposed method
Classical bandwidth for KDE
h
^
R
O
T
(
l
)

1
h
^
D
P
I
a
(
l
)

2
h
^
D
P
I
b
(
l
)

3
h
^
R
O
T
(
d
)

4
h
^
U
C
V
(
d
)

5
h
^
D
P
I
(
d
)

6
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
Normal
Mean
.086
.067
.236
.173
.221
.163
.303
.271
.358
.298
.300
.270
1st quantile
.079
.063
.203
.156
.198
.150
.278
.256
.296
.256
.263
.244
Median
.086
.067
.236
.173
.229
.170
.304
.272
.380
.320
.302
.272
3rd quantile
.094
.071
.267
.190
.254
.185
.329
.287
.432
.361
.334
.296
Cauchy
Mean
.137
.104
.312
.221
.283
.200
.488
.422
.428
.320
.410
.337
1st quantile
.114
.092
.258
.194
.223
.164
.406
.373
.317
.238
.342
.298
Median
.136
.103
.304
.217
.285
.201
.483
.416
.431
.335
.405
.339
3rd quantile
.157
.115
.359
.244
.347
.236
.554
.468
.548
.414
.475
.387
Pareto
Mean
.191
.147
.276
.188
.145
.099
.673
.595
.201
.143
.372
.290
1st quantile
.142
.122
.216
.163
.107
.076
.501
.495
.140
.106
.296
.254
Median
.180
.142
.263
.185
.138
.093
.635
.574
.193
.128
.357
.285
3rd quantile
.219
.166
.318
.208
.173
.114
.775
.673
.245
.174
.430
.322
Mix-Normal I
Mean
.146
.112
.566
.369
.508
.342
.578
.507
.551
.417
.505
.420
1st quantile
.140
.109
.398
.289
.352
.254
.549
.490
.390
.332
.440
.378
Median
.147
.112
.481
.334
.454
.316
.577
.507
.549
.407
.500
.414
3rd quantile
.154
.116
.634
.391
.593
.382
.608
.523
.719
.500
.563
.458
Mix-Normal II
Mean
.103
.080
.442
.272
.442
.247
.521
.461
.489
.359
.465
.387
1st quantile
.094
.076
.310
.220
.258
.173
.491
.443
.341
.257
.412
.348
Median
.106
.081
.363
.246
.350
.226
.522
.458
.490
.353
.461
.382
3rd quantile
.114
.087
.468
.282
.470
.278
.555
.481
.637
.456
.514
.419
1ROT bandwidth for estimation of U
2 given by (16); 2DPI bandwidth for estimation of U
2 with pilot bandwidth g
^
R
O
T
;
d
e
n
s
i
t
y; 3DPI bandwidth for estimation of U
2 with pilot bandwidth g
^
L
S
C
V
;
d
e
n
s
i
t
y; 4ROT bandwidth for density estimation proposed in Silverman [7]; 5LSCV bandwidth for density estimation proposed in Bowman [8]; 6DPI bandwidth for density estimation proposed in Sheather and Jones [9].In general, the optimal bandwidth for kernel density functionals estimation (estimation ofU
1 and U
2 in this paper) is smaller than the one for kernel density estimation under same sample size and underlying distribution as shown in Tables 1 and 2, except for the least square cross-validation bandwidth h
^
LSCV
;
density for density estimation on Generalized Pareto samples. In another word, kernel density functionals estimation requires less smoothness in the estimation, which exaggerates some characteristics of the sample. For instance, the location and scale estimation will be more sensitive to the outliers than density estimation.To evaluate the performance of our proposed bandwidth selection methods in contrast to classical bandwidth selection methods for density estimation in estimation ofU
1 and U
2, the MSE of U
^
1 (and U
^
2) are computed and compared. Figures 6 and 7 demonstrate the boxplot of MSE of U
^
1 and U
^
2, respectively, in terms of 6 bandwidths shown in Tables 1 and 2 under five different distributions: Normal, Cauchy, Generalized Pareto, Mix-Normal I, and Mix-Normal II. Both figures illustrate that (a) MSE of U
^
1 and U
^
2 decreases as sample size increases from 50 to 100; (b) MSE of U
^
1 and U
^
2 is larger for samples from asymmetric distribution rather than symmetric distribution, from bimodal distribution rather than unimodal distribution. Figure 6 infers that (i) Normal scale bandwidths for both estimation of U
1 (h
^
ROT
;
U
1) and density estimation (h
^
ROT
;
density) lead to smaller MSE
(
U
^
1
) relative to other 4 types of bandwidth selection methods for Normal samples of size 50. When sample size goes up to 100, h
^
ROT
;
U
1 outperforms h
^
ROT
;
density with a smaller MSE; (ii) for Cauchy samples with location 1 and scale 2/3, h
^
ROT
;
density becomes the worst choice in kernel density functionals estimation of U
1, especially for relative large sample size. Our proposed bandwidth h
^
ROT
;
U
1 results in the smallest MSE in this case; (iii) for Generalized Pareto samples with shape 1, location 1, and scale 2/3, both h
^
ROT
;
U
1 and h
^
ROT
;
density perform very poorly. However, h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density gives the smallest MSE for Pareto samples, which can partly be explained by the fact that g
^
LSCV
;
density gives the second smallest MSE; (iv) for bimodal distributed samples (including Mix-Normal I and Mix-Normal II), the three proposed bandwidth selection methods for estimation of U
1 completely dominate the three classical density estimation bandwidth selection methods. Among the three proposed bandwidth, h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density gives minimum MSE
(
U
^
1
).Figure 6
The MSE ofU
^
1 in terms of bandwidth selection h; “1-ROT(s)” means h
^
ROT
;
U
1; “2a-DPI(s)” means h
^
DPI
;
U
1 with pilot bandwidth g
^
ROT
;
density; “2b-DPI(s)” means h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density; “3-ROT(d)” means h
^
ROT
;
density; “4-UCV(d)” means h
^
LSCV
;
density; “5-DPI(d)” means h
^
DPI
;
density.Figure 7
The MSE ofU
^
2 in terms of bandwidth selection h; “1-ROT(L)” means h
^
ROT
;
U
2; “2a-DPI(L)” means h
^
DPI
;
U
2 with pilot bandwidth g
^
ROT
;
density; “2b-DPI(L)” means h
^
DPI
;
U
2 with pilot bandwidth g
^
LSCV
;
density; “3-ROT(d)” means h
^
ROT
;
density; “4-UCV(d)” means h
^
LSCV
;
density; “5-DPI(d)” means h
^
DPI
;
density.Figure7 compares the performance of the three proposed practical bandwidth selection methods to three classical density estimation bandwidths in terms of kernel density functionals estimation of U
2. It is shown that h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density outperforms other 5 bandwidth methods in kernel density functionals estimation of U
2 for Normal and Cauchy samples. Similar to estimation of U
1, direct plug-in bandwidth designed for estimation of U
2, that is, h
^
DPI
;
U
2 with pilot bandwidth g
^
LSCV
;
density, beats other candidates for Generalized Pareto samples. The optimal bandwidth selection (with three practical estimates h
^
ROT
;
U
2, h
^
DPI
a
(
s
), and h
^
DPI
b
(
s
)) for estimation of U
2 proposed in this paper performs significantly better than the bandwidth selection for density estimation in density functionals estimation for Mix-Normal distributions. Among the three proposed bandwidth selection methods for estimation of U
2, h
^
ROT
;
U
2 works better than the other for a mixture of 2 Normal distributions as indicated in Figure 7.


## 3.1.Simulation Study  1

The purpose of this study is threefold (1) to evaluate the performance ofh
^
NS
;
U
1 and h
^
NS
;
U
2 when samples are from Normal distribution, (2) to study h
MSE
;
U
1 and h
MSE
;
U
2 (the optimal bandwidths that minimize the MSE of U
^
1 and U
^
2, resp.) in terms of the location parameter of Normal distribution, and (3) to illustrate numerically that optimal bandwidth that minimizes the MSE of U
2 goes to infinity when location parameter gets closer to zero.Figure1 plots the MSE of U
^
1 versus the choice of bandwidth when sample of sizes 20, 50, 100, and 200 is drawn from N
(
0
,
2
) and N
(
1
,
2
), respectively (the simulation result is not sensitive to the choice of scale). The blue curve in each subplot represents the MSE(U
^
1) as bandwidth ranges from 0 to 2. The minimum point of the blue curve indicates h
MSE
;
U
1. The red vertical line in the subplot represents   h
AMSE
;
U
1 and is computed from (10) by replacing f
(
y
) with the p.d.f. of N
(
μ
,
2
), where μ
=
0 in Figure 1(a) and μ
=
1 in Figure 1(b). h
^
NS
;
U
1 is an estimate of h
AMSE
;
U
1 (an asymptotic approximation of h
MSE
;
U
1) under normality assumption. Simulation results in Figure 1 show that h
^
NS
;
U
1 tends to have small variance and stabilized around the true h
MSE
;
U
1 for normality data. The optimal bandwidth does not change with location parameter μ as shown in Figure 1(a) (μ
=
0) and Figure 1(b) (μ
=
1) (more simulation results based on location parameters other than 0 and 1 are available upon request.).Figure 1
The MSE ofU
^
1 when underlying distribution is Normal with mean μ
=
0 (a), μ
=
1 (b) and standard deviation 2. The blue curve is the MSE(U
^
1) versus the bandwidth h. The boxplot of h
^
ROT
;
U
1 is based on the 100 sets of simulated samples of size n
=
20,50,100,200 from Normal distribution with mean μ and standard deviation 2. The red vertical line represents h
AMSE
;
U
1 if f is Normal with known mean 0 and variance σ
2.
(a)
μ
=
0
(b)
μ
=
1Figure2 plots the MSE of U
^
2 versus the choice of bandwidth when sample of sizes 20, 50, 100, and 200 is drawn from N
(
0
,
2
), N
(
0.5
,
2
), N
(
1
,
2
), and N
(
5
,
2
), respectively. Similar to Figure 1, the blue curve and red vertical line represent MSE(U
^
2) and h
AMSE
;
U
2. The boxplot of h
^
NS
;
U
2 is based on the 100 sets of simulated samples of size n
=
10,50,100,200 from Normal distribution with mean μ
=
0,0.5,1
,
5 and standard deviation 2. The red vertical line disappears in Figure 1(a) due to the fact that h
AMSE
;
U
2
→
∞ as μ
→
0. Also the MSE of U
^
2 (blue curve) strictly decreases as h rises in Figure 2(a). To conclude from Figure 1(a), the optimal bandwidth for kernel functional estimation of U
2 goes to infinity when mean of underlying distribution is zero, which is consistent to Proposition 3. However, the boxplot in Figure 2(a) infers that the distribution of h
^
NS
;
U
2 is right-skewed with median, 1st quantile, and 3rd quantile around one, which is far departure from the true value h
MSE
;
U
2 as well as h
AMSE
;
U
2. When μ slightly deviates from zero, just as the case in Figures 2(b) and 2(c), h
^
NS
;
U
2 tends to be less variate (and skewed), overlap with h
AMSE
;
U
2 (red vertical line), and get closer to h
MSE
;
U
2 (valley of blue curve), especially as the sample size grows. When μ increases up to 1 and above as shown in Figure 2(d), the median of h
^
NS
;
U
2, h
AMSE
;
U
2, and h
MSE
;
U
2 coincide when sample size is 50+. More simulation results can be found in Supplementary Material available online at http://dx.doi.org/10.1155/2015/242683.Figure 2
The bandwidth forU
^
2 when underlying distribution is Normal with mean μ
=
0 (a), μ
=
0.5 (b), μ
=
1 (c), and μ
=
5 (d) and standard deviation 2.
(a)
μ
=
0
(b)
μ
=
0.5
(c)
μ
=
1
(d)
μ
=
5


## 3.2.Simulation Study  2

Several candidate bandwidth selection methods are available to serve as a pilot bandwidthg, such as classical bandwidth selection methods for kernel density estimate (described in Section 2.3 Step 1), the optimal bandwidth for U (U
1 or U
2) on Normal scale references, namely, “h
^
NS
;
U
1” and “h
^
NS
;
U
2” in Section 2.2. This subsection aims to study the pilot bandwidth for R
^ and ϕ
^ required to estimate U in (19) (note that U is simplified to U
1 and U
2 when γ
(
x
)
=
1 and x, resp.). Five different choices of pilot bandwidth are studied in this subsection to estimate R
^ and ϕ
^ for h
^
DPI
;
U under five different underlying distributions: (i) Cauchy with location (μ was set to be 1 in simulation study for estimation of U
1 and 5 for estimation of U
2)  μ and scale 2/3, (ii) Generalized Pareto with location μ, scale 2/3, and shape 1, (iii) Normal with location μ and scale 2/3, (iv) Mix-Normal I, and (v) Mix-Normal II. Mix-Normal I and Mix-Normal II are weighted mixtures of two Normal distributions: Mix-Normal I is 1
/
2
N
(
-
2,2
/
3
)
+
1
/
2
N
(
0,2
/
3
) and Mix-Normal I is 3
/
4
N
(
0,1
)
+
1
/
2
N
(
1.5,1
/
3
). Cauchy distribution is a well-known fat-tailed symmetric distribution. Generalized Pareto with shape 1 is a extremely fat-tailed asymmetric (right-skewed) distribution. Density curves of the two Mix-Normal -distributions are given in Figure 3. It is obvious that both Mix-Normal I and Mix-Normal II are bimodal distributions in contrast to unimodal distributions such as Cauchy, Generalized Pareto, and Normal. Mix-Normal I is symmetric bimodal and Mix-Normal II is asymmetric (left-skewed) bimodal distribution. The motivation behind the choice of distributions is to see whether the optimal bandwidth is sensitive to the skewness, extreme outliers, and complex shape of the distributions in contrast to Normal.Figure 3
Density curve of (a) Mix-Normal I:1
/
2
N
(
-
2,2
/
3
)
+
1
/
2
N
(
0,2
/
3
); (b) Mix-Normal II: 3
/
4
N
(
0,1
)
+
1
/
2
N
(
1.5,1
/
3
).
(a)
(b)Figures4 and 5 compare five candidate pilot bandwidth selection methods in terms of the boxplots of MSE of U
^
1 and U
^
2 when sample size is 100. The five candidates for pilot bandwidth selection considered in this paper are (1) Rule-of-Thumb bandwidth for U
1 and U
2 proposed in Section 2.2 (“1-ROT(s)” means g
^
ROT
;
U
1; “1-ROT(L)” means g
^
ROT
;
U
2); (2) Rule-of-Thumb bandwidth for kernel density estimation (KDE) proposed in Scott [26] (“2-ROT(d)” means g
^
ROT
;
density
=
1.06
∗
σ
^
n
-
1
/
5, where σ
^ is the minimum of standard deviation and interquantile range); (3) least-square cross-validation bandwidth for density estimation proposed in Bowman [8] (“3-UCV(d)” means g
^
LSCV
;
density for density estimation); (4) biased cross-validation for density estimation proposed in Scott and Terrell [14] (“4-BCV(d)” means g
^
BCV
;
density for density estimation); (5) direct plug-in bandwidth for density estimation reported by Sheather [27] (“5-DPI(d)” means g
^
DPI
;
density for density estimation). As shown in Figures 4 and 5 that g
^
BCV
;
density is theworst candidate for pilot bandwidth in the density functionals estimation ∫
γ
(
x
)
f
(
x
)
d
x. The pilot bandwidth choice g
^
ROT
;
density gives the lowest MSE(U
^
1) for Cauchy and Normal samples, while g
^
LSCV
;
density results in the lowest MSE(U
^
1) for Generalized Pareto, Mix-Normal I, and Mix-Normal II samples. Similar conclusions can be found in the pilot bandwidth choice for estimation of U
2; however, g
^
ROT
;
U
2 leads to slightly smaller MSE(U
^
2) than g
^
LSCV
;
density for Mix-Normal I and Mix-Normal II samples.Simulation Study  1 illustrates that the Normal reference bandwidth (including h
^
NS
;
U
2 and h
^
ROT
;
U
2) is not a reliable estimate when the location parameter of underlying distribution is close to zero. Therefore, g
^
LSCV
;
density, rather than h
^
ROT
;
U
2, is recommended to serve as pilot bandwidth in estimation of h
^
DPI
;
U
2.Figure 4
MSE ofU
^
1 in terms of the choice of pilot bandwidth g for h
^
DPI
;
U
1; “1-ROT(s)” means g
^
ROT
;
U
1; “2-ROT(d)” means g
^
ROT
;
density; “3-UCV(d)” means g
^
LSCV
;
density; “4-BCV(d)” means g
^
BCV
;
density; “5-DPI(d)” means g
^
DPI
;
density.
(a)
Cauchy
(b)
Pareto
(c)
Normal
(d)
Mix-Normal I
(e)
Mix-Normal IIFigure 5
MSE ofU
^
2 in terms of the choice of pilot bandwidth g for h
^
DPI
;
U
2; “1-ROT(L)” means g
^
ROT
;
U
2; “2-ROT(d)” means g
^
ROT
;
density; “3-UCV(d)” means g
^
LSCV
;
density; “4-BCV(d)” means g
^
BCV
;
density; “5-DPI(d)” means g
^
DPI
;
density.
(a)
Cauchy
(b)
Pareto
(c)
Normal
(d)
Mix-Normal I
(e)
Mix-Normal II


## 3.3.Simulation Study  3

This section aims to evaluate the performance of our proposed bandwidthh
^
ROT
;
U
1 (or h
^
ROT
;
U
2) in Section 2.2 and h
^
DPI
;
U
1 (or h
^
DPI
;
U
2) in Section 2.3 and compare with the classical bandwidth selection methods for density estimation, that is, h
^
ROT
;
density, h
^
LSCV
;
density, and h
^
DPI
;
density in estimation of U
1 (or U
2).Simulation Study  2 recommends g
^
ROT
;
density and g
^
LSCV
;
density to be pilot bandwidth to estimate direct plug-in bandwidth for estimation of U
1 and U
2 among the 5 candidate pilot bandwidth methods and 5 different underlying distributions. Therefore, h
^
DPI
;
U
1 (or h
^
DPI
;
U
2) with these two pilot bandwidth selection methods, g
^
ROT
;
density and g
^
LSCV
;
density,  is considered separately in this subsection. Direct plug-in bandwidth for estimation of U
1 with pilot bandwidth g
^
ROT
;
density is denoted by “h
^
DPI
a
(
s
)” in Table 1 and “2a-DPI(s)” in Figure 1, and with pilot bandwidth g
^
LSCV
;
density is denoted by “h
^
DPI
b
(
s
)" in Table 1 and “2b-DPI(s)” in Figure 1.Table 1
Optimal bandwidth selection in estimation ofU
1 with comparison to classical bandwidth selection methods.
Bandwidth selection for estimation ofU
1
Proposed method
Classical bandwidth for KDE
h
^
R
O
T
(
s
)

1
h
^
D
P
I
a
(
s
)

2
h
^
D
P
I
b
(
s
)

3
h
^
R
O
T
(
d
)

4
h
^
U
C
V
(
d
)

5
h
^
D
P
I
(
d
)

6
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
Normal
Mean
.197
.154
.236
.171
.221
.163
.302
.270
.359
.298
.300
.268
1st quantile
.182
.144
.205
.156
.200
.153
.278
.254
.307
.258
.265
.243
Median
.197
.154
.233
.170
.227
.167
.301
.270
.376
.317
.300
.268
3rd quantile
.214
.163
.265
.185
.254
.183
.327
.287
.431
.355
.336
.293
Cauchy
Mean
.325
.240
.320
.222
.290
.205
.497
.422
.440
.331
.419
.338
1st quantile
.273
.214
.264
.198
.230
.174
.417
.377
.330
.265
.348
.310
Median
.316
.236
.312
.219
.295
.207
.484
.415
.455
.353
.416
.342
3rd quantile
.365
.265
.368
.245
.355
.238
.557
.465
.554
.415
.478
.387
Pareto
Mean
.442
.332
.263
.178
.145
.098
.676
.583
.207
.143
.374
.284
1st quantile
.329
.267
.210
.151
.103
.078
.502
.469
.140
.106
.298
.241
Median
.422
.320
.251
.174
.137
.091
.646
.563
.200
.131
.359
.278
3rd quantile
.519
.386
.300
.202
.174
.109
.794
.678
.260
.173
.428
.327
Mix-Normal I
Mean
.378
.289
.389
.279
.349
.253
.578
.507
.522
.413
.495
.417
1st Quantile
.360
.279
.342
.252
.282
.211
.551
.490
.389
.320
.438
.376
Median
.377
.288
.382
.273
.362
.259
.577
.506
.510
.403
.489
.407
3rd Quantile
.397
.298
.431
.309
.428
.304
.607
.524
.658
.510
.544
.460
Mix-Normal II
Mean
.342
.263
.364
.255
.322
.227
.522
.461
.492
.372
.467
.389
1st Quantile
.321
.251
.318
.229
.254
.188
.490
.441
.339
.279
.413
.347
Median
.342
.262
.361
.254
.345
.231
.523
.461
.509
.355
.464
.387
3rd Quantile
.364
.274
.403
.278
.397
.275
.556
.481
.642
.461
.519
.425
1ROT bandwidth for estimation of U
1 given by (12); 2DPI bandwidth for estimation of U
1 with pilot bandwidth g
^
R
O
T
;
d
e
n
s
i
t
y; 3DPI bandwidth for estimation of U
1 with pilot bandwidth g
^
L
S
C
V
;
d
e
n
s
i
t
y; 4ROT bandwidth for density estimation proposed in Silverman [7]; 5LSCV bandwidth for density estimation proposed in Bowman [8]; 6DPI bandwidth for density estimation proposed in Sheather and Jones [9].The summary statistics (mean, 1st quantile, median, and 3rd quantile) for the three proposed bandwidths and 3 classical bandwidths are provided in Tables1 and 2 to estimate U
1 and U
2, respectively. Samples of size 50 and 100 from five different underlying distribution (3 unimodal distributions and 2 bimodal distributions) as inSimulation Study  2 are considered in both simulation studies of U
1 and U
2 estimation.Table 2
Optimal bandwidth selection in estimation ofU
2 with comparison to classical bandwidth selection methods.
Bandwidth selection for estimation ofU
2
Proposed method
Classical bandwidth for KDE
h
^
R
O
T
(
l
)

1
h
^
D
P
I
a
(
l
)

2
h
^
D
P
I
b
(
l
)

3
h
^
R
O
T
(
d
)

4
h
^
U
C
V
(
d
)

5
h
^
D
P
I
(
d
)

6
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
n
=
50
100
Normal
Mean
.086
.067
.236
.173
.221
.163
.303
.271
.358
.298
.300
.270
1st quantile
.079
.063
.203
.156
.198
.150
.278
.256
.296
.256
.263
.244
Median
.086
.067
.236
.173
.229
.170
.304
.272
.380
.320
.302
.272
3rd quantile
.094
.071
.267
.190
.254
.185
.329
.287
.432
.361
.334
.296
Cauchy
Mean
.137
.104
.312
.221
.283
.200
.488
.422
.428
.320
.410
.337
1st quantile
.114
.092
.258
.194
.223
.164
.406
.373
.317
.238
.342
.298
Median
.136
.103
.304
.217
.285
.201
.483
.416
.431
.335
.405
.339
3rd quantile
.157
.115
.359
.244
.347
.236
.554
.468
.548
.414
.475
.387
Pareto
Mean
.191
.147
.276
.188
.145
.099
.673
.595
.201
.143
.372
.290
1st quantile
.142
.122
.216
.163
.107
.076
.501
.495
.140
.106
.296
.254
Median
.180
.142
.263
.185
.138
.093
.635
.574
.193
.128
.357
.285
3rd quantile
.219
.166
.318
.208
.173
.114
.775
.673
.245
.174
.430
.322
Mix-Normal I
Mean
.146
.112
.566
.369
.508
.342
.578
.507
.551
.417
.505
.420
1st quantile
.140
.109
.398
.289
.352
.254
.549
.490
.390
.332
.440
.378
Median
.147
.112
.481
.334
.454
.316
.577
.507
.549
.407
.500
.414
3rd quantile
.154
.116
.634
.391
.593
.382
.608
.523
.719
.500
.563
.458
Mix-Normal II
Mean
.103
.080
.442
.272
.442
.247
.521
.461
.489
.359
.465
.387
1st quantile
.094
.076
.310
.220
.258
.173
.491
.443
.341
.257
.412
.348
Median
.106
.081
.363
.246
.350
.226
.522
.458
.490
.353
.461
.382
3rd quantile
.114
.087
.468
.282
.470
.278
.555
.481
.637
.456
.514
.419
1ROT bandwidth for estimation of U
2 given by (16); 2DPI bandwidth for estimation of U
2 with pilot bandwidth g
^
R
O
T
;
d
e
n
s
i
t
y; 3DPI bandwidth for estimation of U
2 with pilot bandwidth g
^
L
S
C
V
;
d
e
n
s
i
t
y; 4ROT bandwidth for density estimation proposed in Silverman [7]; 5LSCV bandwidth for density estimation proposed in Bowman [8]; 6DPI bandwidth for density estimation proposed in Sheather and Jones [9].In general, the optimal bandwidth for kernel density functionals estimation (estimation ofU
1 and U
2 in this paper) is smaller than the one for kernel density estimation under same sample size and underlying distribution as shown in Tables 1 and 2, except for the least square cross-validation bandwidth h
^
LSCV
;
density for density estimation on Generalized Pareto samples. In another word, kernel density functionals estimation requires less smoothness in the estimation, which exaggerates some characteristics of the sample. For instance, the location and scale estimation will be more sensitive to the outliers than density estimation.To evaluate the performance of our proposed bandwidth selection methods in contrast to classical bandwidth selection methods for density estimation in estimation ofU
1 and U
2, the MSE of U
^
1 (and U
^
2) are computed and compared. Figures 6 and 7 demonstrate the boxplot of MSE of U
^
1 and U
^
2, respectively, in terms of 6 bandwidths shown in Tables 1 and 2 under five different distributions: Normal, Cauchy, Generalized Pareto, Mix-Normal I, and Mix-Normal II. Both figures illustrate that (a) MSE of U
^
1 and U
^
2 decreases as sample size increases from 50 to 100; (b) MSE of U
^
1 and U
^
2 is larger for samples from asymmetric distribution rather than symmetric distribution, from bimodal distribution rather than unimodal distribution. Figure 6 infers that (i) Normal scale bandwidths for both estimation of U
1 (h
^
ROT
;
U
1) and density estimation (h
^
ROT
;
density) lead to smaller MSE
(
U
^
1
) relative to other 4 types of bandwidth selection methods for Normal samples of size 50. When sample size goes up to 100, h
^
ROT
;
U
1 outperforms h
^
ROT
;
density with a smaller MSE; (ii) for Cauchy samples with location 1 and scale 2/3, h
^
ROT
;
density becomes the worst choice in kernel density functionals estimation of U
1, especially for relative large sample size. Our proposed bandwidth h
^
ROT
;
U
1 results in the smallest MSE in this case; (iii) for Generalized Pareto samples with shape 1, location 1, and scale 2/3, both h
^
ROT
;
U
1 and h
^
ROT
;
density perform very poorly. However, h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density gives the smallest MSE for Pareto samples, which can partly be explained by the fact that g
^
LSCV
;
density gives the second smallest MSE; (iv) for bimodal distributed samples (including Mix-Normal I and Mix-Normal II), the three proposed bandwidth selection methods for estimation of U
1 completely dominate the three classical density estimation bandwidth selection methods. Among the three proposed bandwidth, h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density gives minimum MSE
(
U
^
1
).Figure 6
The MSE ofU
^
1 in terms of bandwidth selection h; “1-ROT(s)” means h
^
ROT
;
U
1; “2a-DPI(s)” means h
^
DPI
;
U
1 with pilot bandwidth g
^
ROT
;
density; “2b-DPI(s)” means h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density; “3-ROT(d)” means h
^
ROT
;
density; “4-UCV(d)” means h
^
LSCV
;
density; “5-DPI(d)” means h
^
DPI
;
density.Figure 7
The MSE ofU
^
2 in terms of bandwidth selection h; “1-ROT(L)” means h
^
ROT
;
U
2; “2a-DPI(L)” means h
^
DPI
;
U
2 with pilot bandwidth g
^
ROT
;
density; “2b-DPI(L)” means h
^
DPI
;
U
2 with pilot bandwidth g
^
LSCV
;
density; “3-ROT(d)” means h
^
ROT
;
density; “4-UCV(d)” means h
^
LSCV
;
density; “5-DPI(d)” means h
^
DPI
;
density.Figure7 compares the performance of the three proposed practical bandwidth selection methods to three classical density estimation bandwidths in terms of kernel density functionals estimation of U
2. It is shown that h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density outperforms other 5 bandwidth methods in kernel density functionals estimation of U
2 for Normal and Cauchy samples. Similar to estimation of U
1, direct plug-in bandwidth designed for estimation of U
2, that is, h
^
DPI
;
U
2 with pilot bandwidth g
^
LSCV
;
density, beats other candidates for Generalized Pareto samples. The optimal bandwidth selection (with three practical estimates h
^
ROT
;
U
2, h
^
DPI
a
(
s
), and h
^
DPI
b
(
s
)) for estimation of U
2 proposed in this paper performs significantly better than the bandwidth selection for density estimation in density functionals estimation for Mix-Normal distributions. Among the three proposed bandwidth selection methods for estimation of U
2, h
^
ROT
;
U
2 works better than the other for a mixture of 2 Normal distributions as indicated in Figure 7.


## 4. Discussion

The optimal bandwidth, along with three practical bandwidth selection methods for kernel density functionals estimation of format∫
γ
(
x
)
f
2
(
x
)
d
x, is discussed in this paper. Necessity and urgency of this study are due to the fact that ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x are two core portions for scale and location, respectively. Chen [3] shed a light on a novel field of nonparametric analytic method for experimental design relying on kernel density functionals estimation of ∫
f
2
(
x
)
d
x and ∫
x
f
2
(
x
)
d
x. Simulation studies in Chen [3] found that kernel-based equality of scale and location tests built on estimation of ∫
γ
(
x
)
f
2
(
x
)
d
x outperform traditional Levene’s test of variance and ANOVA test, respectively, in particular to fat-tailed distribution, such as Cauchy. Our proposed choice of bandwidth selection methods can be directly applied to Chen’s kernel-based equality of location tests, a nonparametric analog of ANOVA test, namely, “kernel-based ANOVA test.” As we discussed in Section 1, Chen [3] uses the least-square cross-validation bandwidth which is designed for density estimation. However, the test statistics of the scale and location test are constructed on the kernel functionals estimation of ∫
γ
(
x
)
f
2
(
x
)
d
x. Our proposed bandwidth will provide the “kernel-based ANOVA” test a better estimate of kernel functionals estimation of ∫
γ
(
x
)
f
2
(
x
)
d
x, which in turn may improve the performance of the “kernel-based ANOVA” test. Kernel-based ANOVA, like other group comparison methods, has a broad application in various fields, such as biomedical sciences, education, and psychology, to compare the differences among 2 or more groups. A real-life example in education by Mimoto and Zitikis [28] is to compare the differences of quantitative abilities between science and nonscience majored students.To broaden the application of this kernel-based nonparametric experimental design and accelerate the methodology development, optimal bandwidth for kernel density functionals estimation is in need to increase the accuracy of scale or location estimation and its relevant hypothesis testing and confidence intervals.∫
γ
(
x
)
f
2
(
x
)
d
x is a function of density f
(
x
) and hence the classical bandwidth selection for kernel density estimation, such as ROT, UCV, BCV, and DPI, can serve as a choice of bandwidth h in its kernel density functionals estimation described in (4). However, our simulation study shows that the proposed optimal bandwidth for ∫
γ
(
x
)
f
2
(
x
)
d
x is superior to the classical bandwidth selection for KDE in estimations U
1 and U
2, two determinant integrations to scale and location parameters. It is also suggested by the simulation study that, in scale estimation, the proposed Rule-of-Thumb bandwidth h
^
ROT
;
U
1 is highly recommended for samples from symmetric unimodal distribution, whereas direct plug-in bandwidth h
^
DPI
;
U
1 with pilot bandwidth g
^
LSCV
;
density is recommended for samples from asymmetric unimodal or bimodal distribution. In location estimation, direct plug-in bandwidth h
^
DPI
;
U
2 with pilot bandwidth g
^
LSCV
;
density is still the best choice for asymmetric unimodal distribution, such as Generalized Pareto, whereas direct plug-in bandwidth h
^
DPI
;
U
2 with pilot bandwidth g
^
ROT
;
density is recommended for samples from symmetric and unimodal distribution, such as Normal and Cauchy. The proposed Rule-of-Thumb bandwidth h
^
ROT
;
U
2 is recommended for samples from bimodal distribution, such as a mixture of two Normal distributions in location estimation or location related testing.


## 5. Conclusion

Kernel smoothing method has been actively studied in density estimation and local kernel regression in the past decade. It is a modern data analytic method with its expertise in capturing the local humps and valley in the distributions or regression functions. Few literature applied kernel smoothing techniques in analysis of experimental design, even the one-way ANOVA model. Ahmad [1] proposed a purely nonparametric method to estimate the location and scale parameter of any unknown distribution using kernel methods. Chen [3] presented a nonparametric nonrank based version of ANOVA using the kernel-based estimate of location (namely, “ANDFE” test) and opened a door to kernel-based nonparametric techniques for experimental design analysis. Chen [3] revealed that smoothing parameter (i.e., bandwidth) substantially affects the size and power of the ANDFE test.This paper derived an optimal bandwidth for a nonparametric kernel density functionals estimation of location and scale of unknown densityf
(
x
) and thus for ANDFE test of equality of location and scale. Thereafter two practical bandwidth selection methods ROT and DPI for location and scale estimation are proposed. Compared to traditional bandwidth selection methods designed for kernel density estimation (e.g., Jones et al. [11]) or kernel regression (e.g., Fan and Gijbels [24]), our proposed bandwidth selection methods demonstrate more accuracy in the estimation of U
1 or U
2. Simulation studies showed that our proposed novel optimal bandwidth method designed for kernel density functionals estimation of scale and location parameters works better than classical bandwidth selection designed for kernel density estimation, in particular bimodal distributions.



---

*Source: 242683-2015-08-06.xml*