# Bernstein-Type Inequality for Widely Dependent Sequence and Its Application to Nonparametric Regression Models

Authors: Aiting Shen
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

We present the Bernstein-type inequality for widely dependent random variables. By using the Bernstein-type inequality and the truncated method, we further study the strong consistency of estimator of fixed design regression model under widely dependent random variables, which generalizes the corresponding one of independent random variables. As an application, the strong consistency for the nearest neighbor estimator is obtained.


---

## Body

## 1. Introduction

Let{Xn,n≥1} be a sequence of random variables defined on a fixed probability space (Ω,ℱ,P). It is well known that the Bernstein-type inequality for the partial sum ∑i=1n‍Xi plays an important role in probability limit theory and mathematical statistics. The main purpose of the paper is to present the Bernstein-type inequality, by which, we will further investigate the strong consistency for the estimator of nonparametric regression models based on widely dependent random variables.


### 1.1. Brief Review

Consider the following fixed design regression model:(1)Yni=g(xni)+εni,i=1,2,…,n,
where xni are known fixed design points from A, where A⊂ℝp is a given compact set for some p≥1, g(·) is an unknown regression function defined on A, and εni are random errors. Assume that for each n≥1, (εn1,εn2,…,εnn) have the same distribution as (ε1,ε2,…,εn). As an estimator of g(·), the following weighted regression estimator will be considered:
(2)gn(x)=∑i=1n‍Wni(x)Yni,x∈A⊂ℝp,
where Wni(x)=Wni(x;xn1,xn2,…,xnn), i=1,2,…,n are the weight functions.The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. For instance, when εni are assumed to be independent, consistency and asymptotic normality have been studied by Georgiev and Greblicki [2], Georgiev [3] and Müller [4] among others. Results for the case when εni are dependent have also been studied by various authors in recent years. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case where it forms an Lq-mixingale sequence for some 1≤q≤2. Roussas [6] discussed strong consistency and quadratic mean consistency for gn(x) under mixing conditions. Roussas et al. [7] established asymptotic normality of gn(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] discussed again asymptotic normality of gn(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] studied the asymptotic normality for double array sum of linear time series. Hu et al. [10] gave the mean consistency, complete consistency, and asymptotic normality of regression models with linear process errors. Liang and Jing [11] presented some asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. Yang et al. [12] generalized the results of Liang and Jing [11] for negatively associated sequences to the case of negatively orthant dependent sequences and obtained the strong consistency for the estimator of the nonparametric regression models based on negatively orthant dependent errors. Wang et al. [13] studied the complete consistency of the estimator of nonparametric regression models based on ρ~-mixing sequences, and so forth. The main purpose of this paper is to investigate the strong consistency for the estimator of the nonparametric regression models based on widely dependent random variables, which contains independent random variables, negatively associated random variables, negatively orthant dependent random variables, extended negatively orthant dependent random variables, and some positively dependent random variables as specials cases. For more details about the strong consistency for the estimator of g(·), Ren and Chen [14] obtained the strong consistency for the least squares estimator of β and the nonparametric estimator of g(·) based on negatively associated samples, Baek and Liang [15] studied the strong consistency for the weighted least squares estimator of β and nonparametric estimator of g(·) in a semi-parametric model under negatively associated samples, which extended the corresponding one on independent random error settings, Liang et al. [16] also studied the strong consistency in a in semiparametric model for a linear process with negatively associated innovations and established the convergence rate, they also pointed out that their results on nonparametric estimator of g(·) can attain the optimal convergence rate, and so forth.


### 1.2. Concepts of Wide Dependence

In this section, we will present some wide dependence structures introduced in Wang et al. [17].Definition 1.
For the random variables{εn,n≥1}, if there exists a finite real sequence {gU(n),n≥1} satisfying for each n≥1 and for all xi∈(-∞,∞), 1≤i≤n,
(3)P(ε1>x1,ε2>x2,…,εn>xn)≤gU(n)∏i=1n‍P(εi>xi),
then we say that the random variables {εn,n≥1} are widely upper orthant dependent (WUOD); if there exists a finite real sequence {gL(n),n≥1} satisfying for each n≥1 and for all xi∈(-∞,∞), 1≤i≤n,
(4)P(ε1≤x1,ε2≤x2,…,εn≤xn)≤gL(n)∏i=1n‍P(εi≤xi),
then we say that the {εn,n≥1} are widely lower orthant dependent (WLOD, in short); if they are both WUOD and WLOD, then we say that the {εn,n≥1} are widely orthant dependent (WOD).
WUOD, WLOD, and WOD random variables are called by a joint name wide dependent (WD) random variables, andgU(n), gL(n), n≥1, are called dominating coefficients.For examples of WD random variables with various dominating coefficients, we refer the reader to Wang et al. [17]. These examples show that WD random variables contain some common negatively dependent random variables, some positively dependent random variables, and some others. For details about WD random variables, one can refer to Wang et al. [17], Wang and Cheng [18], Wang et al. [19], Chen et al. [20], and so forth.In what follows, denoteg(n)=max{gU(n),gL(n)}. Recall that when gL(n)=gU(n)=1 for any n≥1 in (3) and (4), the random variables {εn,n≥1} are called negatively upper orthant dependent (NUOD) and negatively lower orthant dependent (NLOD), respectively. If they are both NUOD and NLOD, then we say that the random variables {εn,n≥1} are negatively orthant dependent (NOD) (see, e.g., Ebrahimi and Ghosh [21], Block et al. [22], Joag-Dev and Proschan [23], Wang et al. [24–26], Wu [27, 28], Wu and Jiang [29], or Wu and Chen [30]).If both (3) and (4) hold when gL(n)=gU(n)=M for some constant M, the random variables {Xn,n≥1} are called extended negatively upper orthant dependent (ENUOD) and extended negatively lower orthant dependent (ENLOD), respectively. If they are both ENUOD and ENLOD, then we say that the random variables {εn,n≥1} are extended negatively orthant dependent (ENOD) (see, e.g., Liu [31]). The concept of general extended negative dependence was proposed by Liu [31, 32] and further promoted by Chen et al. [33, 34], Shen [35], Wang and Chen [18], S. J. Wang and W. S. Wang [36] and Wang et al. [37], and so forth.Wang et al. [17] obtained the following properties for WD random variables, which will be used to prove the main results of the paper.Proposition 2.
(1) Let{εn,n≥1} be WLOD (WUOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1). If {fn(·),n≥1} are nondecreasing, then {fn(εn),n≥1} are still WLOD (WUOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1); if {fn(·),n≥1} are nonincreasing, then {fn(εn),n≥1} are WUOD (WLOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1).
(2) If{εn,n≥1} are nonnegative and WUOD with dominating coefficients gU(n), n≥1, then for each n≥1,
(5)E∏i=1n‍εi≤gU(n)∏i=1n‍Eεi.
In particular, if{εn,n≥1} are WUOD with dominating coefficients gU(n), n≥1, then for each n≥1 and any s>0,
(6)Eexp{s∑i=1n‍εi}≤gU(n)∏i=1n‍Eexp{sεi}.By Proposition2, we can get the following corollary immediately.Corollary 3.
(1) Let{εn,n≥1} be WD. If {fn(·),n≥1} are nondecreasing (or nonincreasing), then {fn(εn),n≥1} are still WD.
(2) If{Xn,n≥1} are WD, then for each n≥1 and any s∈ℝ,
(7)Eexp{s∑i=1n‍εi}≤g(n)∏i=1n‍Eexp{sεi}.In this paper, we will present the Bernstein-type inequality for WD random variables. By using the Bernstein-type inequality, we will further investigate the strong consistency for the estimator of nonparametric regression models based on WD errors.This work is organized as follows: the Bernstein-type inequality for WD random variables is provided in Section2 and strong consistency for the estimator of nonparametric regression models based on WD errors is investigated in Section 3.Throughout the paper,C denotes a positive constant not depending on n, which may be different in various places. an=O(bn) represents an≤Cbn for all n≥1. Let ⌈x⌉ denote the integer part of x and I(A) be the indicator function of the set A. Denote x+=xI(x≥0) and x-=-xI(x<0). Let {εn,n≥1} be a sequence of WD random variables. Denote Sn=∑i=1n‍εi. In the sequel, we will use the following different assumptions in different situations:
(8)limn→∞g(n)e-anc=0,(9)limn→∞g(n)e-dlog3/2n=0,
where a, c, and d are finite positive constants.


## 1.1. Brief Review

Consider the following fixed design regression model:(1)Yni=g(xni)+εni,i=1,2,…,n,
where xni are known fixed design points from A, where A⊂ℝp is a given compact set for some p≥1, g(·) is an unknown regression function defined on A, and εni are random errors. Assume that for each n≥1, (εn1,εn2,…,εnn) have the same distribution as (ε1,ε2,…,εn). As an estimator of g(·), the following weighted regression estimator will be considered:
(2)gn(x)=∑i=1n‍Wni(x)Yni,x∈A⊂ℝp,
where Wni(x)=Wni(x;xn1,xn2,…,xnn), i=1,2,…,n are the weight functions.The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. For instance, when εni are assumed to be independent, consistency and asymptotic normality have been studied by Georgiev and Greblicki [2], Georgiev [3] and Müller [4] among others. Results for the case when εni are dependent have also been studied by various authors in recent years. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case where it forms an Lq-mixingale sequence for some 1≤q≤2. Roussas [6] discussed strong consistency and quadratic mean consistency for gn(x) under mixing conditions. Roussas et al. [7] established asymptotic normality of gn(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] discussed again asymptotic normality of gn(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] studied the asymptotic normality for double array sum of linear time series. Hu et al. [10] gave the mean consistency, complete consistency, and asymptotic normality of regression models with linear process errors. Liang and Jing [11] presented some asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. Yang et al. [12] generalized the results of Liang and Jing [11] for negatively associated sequences to the case of negatively orthant dependent sequences and obtained the strong consistency for the estimator of the nonparametric regression models based on negatively orthant dependent errors. Wang et al. [13] studied the complete consistency of the estimator of nonparametric regression models based on ρ~-mixing sequences, and so forth. The main purpose of this paper is to investigate the strong consistency for the estimator of the nonparametric regression models based on widely dependent random variables, which contains independent random variables, negatively associated random variables, negatively orthant dependent random variables, extended negatively orthant dependent random variables, and some positively dependent random variables as specials cases. For more details about the strong consistency for the estimator of g(·), Ren and Chen [14] obtained the strong consistency for the least squares estimator of β and the nonparametric estimator of g(·) based on negatively associated samples, Baek and Liang [15] studied the strong consistency for the weighted least squares estimator of β and nonparametric estimator of g(·) in a semi-parametric model under negatively associated samples, which extended the corresponding one on independent random error settings, Liang et al. [16] also studied the strong consistency in a in semiparametric model for a linear process with negatively associated innovations and established the convergence rate, they also pointed out that their results on nonparametric estimator of g(·) can attain the optimal convergence rate, and so forth.


## 1.2. Concepts of Wide Dependence

In this section, we will present some wide dependence structures introduced in Wang et al. [17].Definition 1.
For the random variables{εn,n≥1}, if there exists a finite real sequence {gU(n),n≥1} satisfying for each n≥1 and for all xi∈(-∞,∞), 1≤i≤n,
(3)P(ε1>x1,ε2>x2,…,εn>xn)≤gU(n)∏i=1n‍P(εi>xi),
then we say that the random variables {εn,n≥1} are widely upper orthant dependent (WUOD); if there exists a finite real sequence {gL(n),n≥1} satisfying for each n≥1 and for all xi∈(-∞,∞), 1≤i≤n,
(4)P(ε1≤x1,ε2≤x2,…,εn≤xn)≤gL(n)∏i=1n‍P(εi≤xi),
then we say that the {εn,n≥1} are widely lower orthant dependent (WLOD, in short); if they are both WUOD and WLOD, then we say that the {εn,n≥1} are widely orthant dependent (WOD).
WUOD, WLOD, and WOD random variables are called by a joint name wide dependent (WD) random variables, andgU(n), gL(n), n≥1, are called dominating coefficients.For examples of WD random variables with various dominating coefficients, we refer the reader to Wang et al. [17]. These examples show that WD random variables contain some common negatively dependent random variables, some positively dependent random variables, and some others. For details about WD random variables, one can refer to Wang et al. [17], Wang and Cheng [18], Wang et al. [19], Chen et al. [20], and so forth.In what follows, denoteg(n)=max{gU(n),gL(n)}. Recall that when gL(n)=gU(n)=1 for any n≥1 in (3) and (4), the random variables {εn,n≥1} are called negatively upper orthant dependent (NUOD) and negatively lower orthant dependent (NLOD), respectively. If they are both NUOD and NLOD, then we say that the random variables {εn,n≥1} are negatively orthant dependent (NOD) (see, e.g., Ebrahimi and Ghosh [21], Block et al. [22], Joag-Dev and Proschan [23], Wang et al. [24–26], Wu [27, 28], Wu and Jiang [29], or Wu and Chen [30]).If both (3) and (4) hold when gL(n)=gU(n)=M for some constant M, the random variables {Xn,n≥1} are called extended negatively upper orthant dependent (ENUOD) and extended negatively lower orthant dependent (ENLOD), respectively. If they are both ENUOD and ENLOD, then we say that the random variables {εn,n≥1} are extended negatively orthant dependent (ENOD) (see, e.g., Liu [31]). The concept of general extended negative dependence was proposed by Liu [31, 32] and further promoted by Chen et al. [33, 34], Shen [35], Wang and Chen [18], S. J. Wang and W. S. Wang [36] and Wang et al. [37], and so forth.Wang et al. [17] obtained the following properties for WD random variables, which will be used to prove the main results of the paper.Proposition 2.
(1) Let{εn,n≥1} be WLOD (WUOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1). If {fn(·),n≥1} are nondecreasing, then {fn(εn),n≥1} are still WLOD (WUOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1); if {fn(·),n≥1} are nonincreasing, then {fn(εn),n≥1} are WUOD (WLOD) with dominating coefficients gL(n), n≥1(gU(n),n≥1).
(2) If{εn,n≥1} are nonnegative and WUOD with dominating coefficients gU(n), n≥1, then for each n≥1,
(5)E∏i=1n‍εi≤gU(n)∏i=1n‍Eεi.
In particular, if{εn,n≥1} are WUOD with dominating coefficients gU(n), n≥1, then for each n≥1 and any s>0,
(6)Eexp{s∑i=1n‍εi}≤gU(n)∏i=1n‍Eexp{sεi}.By Proposition2, we can get the following corollary immediately.Corollary 3.
(1) Let{εn,n≥1} be WD. If {fn(·),n≥1} are nondecreasing (or nonincreasing), then {fn(εn),n≥1} are still WD.
(2) If{Xn,n≥1} are WD, then for each n≥1 and any s∈ℝ,
(7)Eexp{s∑i=1n‍εi}≤g(n)∏i=1n‍Eexp{sεi}.In this paper, we will present the Bernstein-type inequality for WD random variables. By using the Bernstein-type inequality, we will further investigate the strong consistency for the estimator of nonparametric regression models based on WD errors.This work is organized as follows: the Bernstein-type inequality for WD random variables is provided in Section2 and strong consistency for the estimator of nonparametric regression models based on WD errors is investigated in Section 3.Throughout the paper,C denotes a positive constant not depending on n, which may be different in various places. an=O(bn) represents an≤Cbn for all n≥1. Let ⌈x⌉ denote the integer part of x and I(A) be the indicator function of the set A. Denote x+=xI(x≥0) and x-=-xI(x<0). Let {εn,n≥1} be a sequence of WD random variables. Denote Sn=∑i=1n‍εi. In the sequel, we will use the following different assumptions in different situations:
(8)limn→∞g(n)e-anc=0,(9)limn→∞g(n)e-dlog3/2n=0,
where a, c, and d are finite positive constants.


## 2. Bernstein-Type Inequality for WD Random Variables

In this section, we will present the Bernstein-type inequality for WD random variables, which will be used to prove the strong consistency for estimator of the nonparametric regression model based on WD random variables.Theorem 4.
Let{εn,n≥1} be a sequence of WD random variables with Eεi=0 and |εi|≤b for each i≥1, where b is a positive constant. Denote σi2=Eεi2 and Bn2=∑i=1n‍σi2 for each n≥1. Then for any ε>0,
(10)P(Sn≥ε)≤gU(n)exp{-ε22Bn2+(2/3)bε},(11)P(|Sn|≥ε)≤2g(n)exp{-ε22Bn2+(2/3)bε}.Proof.
For anyt>0, by Taylor’s expansion, EXi=0 and the inequality 1+x≤ex for x∈ℝ, we can get that for i=1,2,…,n,
(12)Eexp{tεi}=1+∑j=2∞‍E(tεi)jj!≤1+∑j=2∞‍tjE|εi|jj!=1+t2σi22∑j=2∞‍tj-2E|εi|j(1/2)σi2j!≐1+t2σi22Fi(t)≤exp{t2σi22Fi(t)},
where
(13)Fi(t)=∑j=2∞‍tj-2E|εi|j(1/2)σi2j!,i=1,2,…,n.
Denote C=b/3 and Mn=bε/3Bn2+1. Choosing t>0 such that tC<1 and
(14)tC≤Mn-1Mn=CεCε+Bn2.
It is easy to check that for i=1,2,…,n and j≥2,
(15)E|εi|j≤σi2bj-2≤12σi2Cj-2j!,
which implies that for i=1,2,…,n,
(16)Fi(t)=∑j=2∞‍tj-2E|εi|j(1/2)σi2j!≤∑j=2∞‍(tC)j-2=(1-tC)-1≤Mn.
By Markov’s inequality, Corollary 3, (12), and (16), we can get
(17)P(Sn≥ε)≤e-tεEexp{tSn}≤e-tεgU(n)∏i=1n‍Eexp{tεi}≤gU(n)exp{-tε+t2Bn22Mn}.
Taking t=ε/Bn2Mn=ε/(Cε+Bn2). It is easily seen that tC<1 and tC=Cε/(Cε+Bn2). Substituting t=ε/Bn2Mn into the right-hand side of (17), we can obtain (10) immediately. By (10), we have
(18)P(Sn≤-ε)=P(-Sn≥ε)≤gL(n)exp{-ε22Bn2+(2/3)bε},
since {-εn,n≥1} is still a sequence of WD random variables. The desired result (11) follows from (10) and (18) immediately.By Theorem4, we can get the following complete convergence for WD random variables immediately.Corollary 5.
Let{εn,n≥1} be a sequence of WD random variables with Eεi=0 and |εi|≤b for each i≥1, where b is a positive constant. Assume that ∑i=1∞‍Eεi2<∞. r>0. Let the dominating coefficients gU(n), gL(n), n≥1 satisfy (8) with any finite positive constant a and c=r. Then
(19)n-rSn⟶0,completely,asn⟶∞.Proof.
For anyε>0, it follows from (11) that
(20)∑n=1∞‍P(|Sn|≥nrε)≤2∑n=1∞‍g(n)exp{-n2rε22∑i=1∞EXi2+(2/3)bnrε}≤C1∑n=1∞‍[exp(-C)]nr<∞,
which implies (19). Here C and C1 are positive constants not depending on n.


## 3. The Strong Consistency for the Estimator of Nonparametric Regression Models Based on WD Errors

Unless otherwise specified, we assume throughout the paper thatgn(x) is defined by (2). For any function g(x), we use c(g) to denote all continuity points of the function g on A. The norm ∥x∥ is the Euclidean norm. For any fixed design point x∈A, the following assumptions on weight functions Wni(x) will be used:(A1)
|∑i=1n‍Wni(x)-1|=O(n-1/4);(A2)
∑i=1n‍|Wni(x)|≤C for all n≥1 and max1≤i≤n|Wni(x)|=O(n-1/2log-3/2n);(A3)
∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>σn-1/4)=O(n-1/4) for some σ>0.Theorem 6.
Let{εn,n≥1} be a sequence of mean zero WD random variables such that supn≥1Eεn2<∞. Suppose that the conditions (A1)–(A3) hold true and (9) holds for any positive constant d. Assume that g(x) satisfies a local Lipschitz condition around the point x. Then for any x∈A,
(21)gn(x)⟶g(x),asn⟶∞,a.s.Proof.
Forx∈A, we have by (1) and (2) that
(22)|Egn(x)-g(x)|≤∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥≤σn-1/4)=+∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>σn-1/4)=+|g(x)|·|∑i=1n‍Wni(x)-1|.
By (22), the conditions (A1)–(A3) and the assumption on g(x), we have
(23)|Egn(x)-g(x)|=O(n-1/4),x∈A.
Hence, to prove (21), we only need to show that
(24)gn(x)-Egn(x)⟶0,asn⟶∞,a.s.
For fixed design point x∈A, without loss of generality, we assume that Wni(x)>0 in what follows (otherwise, we use Wni+(x) and Wni-(x) instead of Wni(x), respectively, and note that Wni(x)=Wni+(x)-Wni-(x)). Let
(25)ε1,i(n)=-i1/2I(εni<-i1/2)+εniI(|εni|≤i1/2)+i1/2I(εni>i1/2),ε2,i(n)=(εni-i1/2)I(εni>i1/2),ε3,i(n)=(εni+i1/2)I(εni<-i1/2),ε1,i=-i1/2I(εi<-i1/2)+εiI(|εi|≤i1/2)+i1/2I(εi>i1/2),ε2,i=(εi-i1/2)I(εi>i1/2),ε3,i=(εi+i1/2)I(εi<-i1/2).
Since Eεni=Eεi=0 for each n, it is easy to see that
(26)gn(x)-Egn(x)=∑i=1n‍Wni(x)εni=∑i=1n‍Wni(x)[ε1,i(n)-Eε1,i(n)]=+∑i=1n‍Wni(x)[ε2,i(n)-Eε2,i(n)]=+∑i=1n‍Wni(x)[ε3,i(n)-Eε3,i(n)]=:Tn1+Tn2+Tn3.
By the condition (A2), we can see that
(27)max1≤i≤n|Wni(x)(ε1,i-Eε1,i)|≤2n1/2max1≤i≤n|Wni(x)|≤Clog-3/2n,∑i=1n‍Var[Wni(x)(ε1,i-Eε1,i)]≤∑i=1n‍Wni2(x)Eεi2≤Cmax1≤i≤n|Wn,i(x)|∑i=1n‍|Wni(x)|≤Cn-1/2log-3/2n.
For fixed x∈A and n, since (εn1,εn2,…,εnn) have the same distribution as (ε1,ε2,…,εn) and {Wni(x)(ε1,i-Eε1,i),1≤i≤n} are WD with mean zero, we have by applying Theorem 4 that for every ϵ>0,
(28)P(|Tn1|≥ϵ)=P(|∑i=1n‍Wni(x)[ε1,i(n)-Eε1,i(n)]|≥ϵ)=P(|∑i=1n‍Wni(x)[ε1,i-Eε1,i]|≥ϵ)≤2g(n)exp{-ϵ2Cn-1/2log-3/2n+Cϵlog-3/2n}≤2g(n)exp{-Clog3/2n}≤Cn-2,========iifornlargeenough,
which implies
(29)Tn1=∑i=1n‍Wni(x)[ε1,i(n)-Eε1,i(n)]⟶0,asn⟶∞,a.s.
by Borel-Cantelli lemma.
Next, we will estimateTn2 and Tn3. It can be checked by supn≥1Eεn2<∞ that
(30)∑i=1∞‍Eε2,i(n)i1/2log5/4(2i)=∑i=1∞‍Eε2,ii1/2log5/4(2i)≤∑i=1∞‍E[εiI(εi>i1/2)]i1/2log5/4(2i)≤∑i=1∞‍Eεi2ilog5/4(2i)<∞,
which implies
(31)∑i=1∞‍ε2,i(n)i1/2log5/4(2i)<∞,a.s.
Consequently, by Kronecker’s lemma, we have that
(32)1n1/2log5/4(2n)∑i=1n‍ε2,i(n)⟶0,a.s.
Thus, by the condition (A2), it is easy to see that
(33)|∑i=1n‍Wni(x)ε2,i(n)|≤max1≤i≤n|Wni(x)|∑i=1n‍ε2,i(n)≤Cn-1/2log-3/2n∑i=1n‍ε2,i(n)=o(log-1/4n),a.s.
By supn≥1Eεn2<∞ and (A2) again, we have
(34)|∑i=1n‍Wn,i(x)Eε2,i(n)|=|∑i=1n‍Wn,i(x)Eε2,i|≤max1≤i≤n|Wni(x)|∑i=1n‍E[|εi|I(|εi|≥i1/2)]≤Cn-1/2log-3/2n∑i=1n‍i-1/2E[εi2I(|εi|≥i1/2)]=O(log-3/2n).
Combining (33) and (34), it follows that
(35)|Tn2|=|∑i=1n‍Wni(x)[ε2,i(n)-Eε2,i(n)]|=o(log-1/4n),a.s.
Likewise, by supn≥1Eεn2<∞, we can see that
(36)∑i=1∞‍E|ε3,i(n)|i1/2log5/4(2i)=∑i=1∞‍E|ε3,i|i1/2log5/4(2i)≤∑i=1∞‍-E[εiI(εi<-i1/2)]i1/2log5/4(2i)≤∑i=1∞‍Eεi2ilog5/4(2i)<∞,
which implies
(37)∑i=1∞‍|ε3,i(n)|i1/2log5/4(2i)<∞,a.s.
Hence, by Kronecker’s lemma,
(38)1n1/2log5/4(2n)∑i=1n‍|ε3,i(n)|⟶0,a.s.
Consequently, we have by (A2) that
(39)|∑i=1n‍Wni(x)ε3,i(n)|≤max1≤i≤n|Wni(x)|∑i=1n‍|ε3,i(n)|=o(log-1/4n),a.s.
On the other hand, by (A2) and supn≥1Eεn2<∞ again, we can see that
(40)|∑i=1n‍Wni(x)Eε3,i(n)|=|∑i=1n‍Wni(x)Eε3,i|≤max1≤i≤n|Wni(x)|∑i=1n‍E[|εi|I(|εi|>i1/2)]≤Cn-1/2log-3/2n∑i=1n‍i-1/2E[εi2I(|εi|>i1/2)]=O(log-3/2n).
From the statements above, we have
(41)|Tn3|=|∑i=1n‍Wn,i(x)[ε3,i(n)-Eε3,i(n)]|=o(log-1/4n),a.s.
Therefore, (24) follows from (26), (29), (35), and (41) immediately. This completes the proof of the theorem.Theorem 7.
Let{εn,n≥1} be a sequence of mean zero WD random variables such that supn≥1Eεn4<∞. Suppose that the conditions (A1)–(A3) hold true and (9) holds for any positive constant d. Assume that g(x) satisfies a local Lipschitz condition around the point x. Then for any x∈A,
(42)gn(x)-g(x)=O(n-1/4),a.s.Proof.
According to (23), we can see that in order to prove (42), we only need to show that
(43)|gn(x)-Egn(x)|=O(n-1/4),a.s.
We still assume that Wni(x)>0 in what follows. The proof is similar to that of Theorem 6. We use the same notations εq,i(n), εq,i and Tnq for q=1,2,3 as those in Theorem 6, where i1/2 is replaced by i1/4. Obviously supn≥1Eεn4<∞ implies supn≥1Eεn2<∞. It follows by (A2) that
(44)max1≤i≤n|Wni(x)(ε1,i-Eε1,i)|≤2n1/4max1≤i≤n|Wn,i(x)|≤Cn-1/4log-3/2n,∑i=1n‍Var[Wni(x)(ε1,i-Eε1,i)]≤∑i=1n‍Wn,i2(x)Eεi2≤Cn-1/2log-3/2n.
By applying Theorem 4 and (9), we can see that for every ϵ>0,
(45)P(|Tn1|≥ϵn-1/4)=P(∑i=1n‍Wni(x)[ε1,i(n)-Eε1,i(n)]≥ϵn-1/4)=P(∑i=1n‍Wni(x)[ε1,i-Eε1,i]≥ϵn-1/4)≤2g(n)exp{-ϵ2n-1/2Cn-1/2log-3/2n+Cϵn-1/2log-3/2n}≤2g(n)exp{-Clog3/2n}≤Cn-2,fornlargeenough,
which implies by Borel-Cantelli lemma that
(46)n1/4Tn1⟶0,a.s.
Meanwhile, it can be checked by supn≥1Eεn4<∞ that
(47)∑i=1∞‍Eε2,i(n)i1/4log3/2(2i)=∑i=1∞‍Eε2,ii1/4log3/2(2i)≤∑i=1∞‍E[εiI(εi>i1/4)]i1/4log3/2(2i)≤∑i=1∞‍Eεi4ilog3/2(2i)<∞,
which implies
(48)∑i=1∞‍ε2,i(n)i1/4log3/2(2i)<∞,a.s.
Then, we have by Kronecker’s lemma that
(49)1n1/4log3/2(2n)∑i=1n‍ε2,i(n)⟶0,a.s.
Consequently, it follows by (A2) that
(50)|∑i=1n‍Wni(x)ε2,i(n)|≤max1≤i≤n|Wni(x)|∑i=1n‍ε2,i(n)=o(n-1/4),a.s.,(51)|∑i=1n‍Wni(x)Eε2,i(n)|=|∑i=1n‍Wni(x)Eε2,i|≤max1≤i≤n|Wni(x)|∑i=1n‍E[|εi|I(|εi|>i1/4)]≤Cn-1/2log-3/2n∑i=1n‍i-3/4E[εi4I(|εi|>i1/4)]=O(n-1/4log-3/2n).
On the other hand, it can be checked that
(52)∑i=1∞‍E|ε3,i(n)|i1/4log3/2(2i)=∑i=1∞‍E|ε3,i|i1/4log3/2(2i)≤∑i=1∞‍-E[εiI(εi<-i1/4)]i1/4log3/2(2i)≤∑i=1∞‍Eεi4ilog3/2(2i)<∞,
which implies
(53)∑i=1∞‍|ε3,i(n)|i1/4log3/2(2i)<∞,a.s.
So, by Kronecker’s lemma,
(54)1n1/4log3/2(2n)∑i=1n‍|ε3,i(n)|⟶0,a.s.
Consequently, we have by (A2) that
(55)|∑i=1n‍Wni(x)ε3,i(n)|≤max1≤i≤n|Wni(x)|∑i=1n‍|ε3,i(n)|=o(n-1/4),a.s.,(56)|∑i=1n‍Wni(x)Eε3,i(n)|=|∑i=1n‍Wni(x)Eε3,i|≤max1≤i≤n|Wni(x)|∑i=1n‍E[|εi|I(|εi|>i1/4)]≤Cn-1/2log-3/2n∑i=1n‍i-3/4E[εi4I(|εi|>i1/4)]=O(n-1/4log-3/2n).
Finally, similar to the proof of (21), we can get (43) immediately by (46)–(56). This completes the proof of the theorem.As an application of Theorems6 and 7, we give the strong consistency for the nearest neighbor estimator of g(x). Without loss of generality, put A=[0,1], taking xni=i/n, i=1,2,…,n. For any x∈A, we rewrite |xn1-x|,|xn2-x|,…,|xnn-x| as follows:
(57)|xR1(x)(n)-x|≤|xR2(x)(n)-x|≤⋯≤|xRn(x)(n)-x|,
if |xni-x|=|xnj-x|, then |xni-x| is permuted before |xnj-x| when xni<xnj.Let1≤kn≤n, the nearest neighbor weight function estimator of g(x) in model (1) is defined as follows:
(58)g~n(x)=∑i=1n‍W~ni(x)Yni,
where
(59)W~ni(x)={1kn,if|xni-x|≤|xRkn(x)(n)-x|,0,otherwise.Based on the notations above, we can get the following result by using Theorems6 and 7.Corollary 8.
Let{εn,n≥1} be a sequence of mean zero WD random variables and (9) holds for any positive constant d. Assume that g(x) satisfies a local Lipschitz condition around the point x. Denote kn=⌈n5/8⌉.(i)
Ifsupn≥1Eεn2<∞, then (21) holds for any x∈A.(ii)
Ifsupn≥1Eεn4<∞, then (42) holds for any x∈A.Proof.
It suffices to show that the conditions(A1)–(A3) are satisfied. For any x∈[0,1], it follows from the definitions of Ri(x) and W~ni(x) that
(60)∑i=1n‍W~ni(x)=∑i=1n‍W~nRi(x)(x)=∑i=1kn‍1kn=1,max1≤i≤nW~ni(x)=1kn≤Cn-5/8,W~ni(x)≥0,∑i=1n‍|W~ni(x)|·|g(xni)-g(x)|I(|xni-x|>σn-1/4)≤C∑i=1n‍(xni-x)2|W~ni(x)|σ2n-1/2=C∑i=1kn‍(xRi(x)(n)-x)2n1/2knσ2≤C∑i=1kn‍(i/n)2n1/2knσ2≤C(knnσ)2n1/2≤Cn-1/4,∀a>0.
Hence, conditions (A1)–(A3) are satisfied. By Theorems 6 and 7, we can get (i) and (ii) immediately. This completes the proof of the corollary.



---

*Source: 862602-2013-07-18.xml*