# Complete Consistency of the Estimator of Nonparametric Regression Models Based onρ~-Mixing Sequences

Authors: Xuejun Wang; Fengxi Xia; Meimei Ge; Shuhe Hu; Wenzhi Yang
Journal: Abstract and Applied Analysis
 (2012)

---

## Abstract

We study the complete consistency for estimator of nonparametric regression model based onρ~-mixing sequences by using the classical Rosenthal-type inequality and the truncated method. As an application, the complete consistency for the nearest neighbor estimator is obtained.


---

## Body

## 1. Introduction

Consider the following fixed design nonparametric regression model:(1.1)Yni=g(xni)+εni,i=1,2,…,n,
where xni are known fixed design points from A, where A⊂ℝp is a given compact set for some p≥1, g(·) is an unknown regression function defined on A, and εni are random errors. Assume that for each n≥1, (εn1,εn2,…,εnn) have the same distribution as (ε1,ε2,…,εn). As an estimator of g(·), the following weighted regression estimator will be considered:
(1.2)gn(x)=∑i=1n‍Wni(x)Yni,x∈A⊂ℝp,
where Wni(x)=Wni(x;xn1,xn2,…,xnn), i=1,2,…,n are the weight function.The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. For instance, when εni are assumed to be independent, consistency and asymptotic normality have been studied by Georgiev and Greblicki [2], Georgiev [3] and Müller [4] among others. Results for the case when εni are dependent have also been studied by various authors in recent years. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case that forms an Lq-mixingale sequence for some 1≤q≤2. Roussas [6] discussed strong consistency and quadratic mean consistency for gn(x) under mixing conditions. Roussas et al. [7] established asymptotic normality of gn(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] discussed again asymptotic normality of gn(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] studied the asymptotic normality for double array sum of linear time series. Hu et al. [10] gave the mean consistency, complete consistency, and asymptotic normality of regression models with linear process errors. Liang and Jing [11] presented some asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences, Yang et al. [12] generalized the results of Liang and Jing [11] for negatively associated sequences to the case of negatively orthant dependent sequences, and so forth. The main purpose of this section is to investigate the complete consistency for estimator of the nonparametric regression model based on ρ~-mixing random variables.In the following, we will give the definition of sequence ofρ~-mixing random variables.Let{Xn,n≥1} be a random variable sequence defined on a fixed probability space (Ω,ℱ,P). Write ℱ𝒮=σ(Xi,i∈S⊂ℕ). Given σ-algebras ℬ,ℛ in ℱ, let
(1.3)ρ(ℬ,ℛ)=supX∈L2(ℬ),Y∈L2(ℛ)|EXY-EXEY|(VarXVarY)1/2.
Define the ρ~-mixing coefficients by
(1.4)ρ~(k)=sup{ρ(ℱ𝒮,ℱ𝒯):finitesubsetsS,T⊂ℕ,suchthatdist(S,T)≥k},k≥0.
Obviously, 0≤ρ~(k+1)≤ρ~(k)≤1, and ρ~(0)=1.Definition 1.1.
A sequence of random variables{Xn,n≥1} is said to be a ρ~-mixing sequence if there exists k∈ℕ such that ρ~(k)<1.ρ
~-mixing random variables were introduced by Bradley [13], and many applications have been found. Many authors have studied this concept providing interesting results and applications. See for example, Bradley [13] for the central limit theorem, Bryc and Smoleński [14], Peligrad [15], and Utev and Peligrad [16] for moment inequalities, Gan [17], Kuczmaszewska [18], Wu and Jiang [19], and Wang et al. [20] for almost sure convergence, Peligrad and Gut [21], Gan [17], Cai [22], Kuczmaszewska [23], Zhu [24], An and Yuan [25], Sung [26] and Wang et al. [27] for complete convergence, and Peligrad [15] for invariance principle, Zhou et al. [28] and Sung [29] for strong law of large numbers, and so forth. When these are compared with the corresponding results of independent random variable sequences, there still remains much to be desired.This work is organized as follows: main result of the paper is provided in Section2. Some preliminary lemmas are presented in Section 3, and the proof of the main result is given in Section 4.Throughout the paper,C denotes a positive constant not depending on n, which may be different in various places. an=O(bn) represents an≤Cbn for all n≥1. Let ⌈x⌉ denote the integer part of x and let I(A) be the indicator function of the set A. Denote x+=xI(x≥0) and x-=-xI(x<0).


## 2. Main Result

Unless otherwise specified, we assume throughout the paper thatgn(x) is defined by (1.2). For any function g(x), we use c(g) to denote all continuity points of the function g on A. The norm ∥x∥ is the Euclidean norm. For any fixed design point x∈A, the following assumptions on weight function Wni(x) will be used:(A1)
∑i=1n‍Wni(x)→1 as n→∞;(A2)
∑i=1n‍|Wni(x)|≤C<∞ for all n;(A3)
∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a)→0 as n→∞ for all a>0.Based on the assumptions above, we can get the following complete consistency of the nonparametric regression estimatorgn(x).Theorem 2.1.
Let{εn,n≥1} be a sequence of ρ~-mixing random variables with mean zero, which is stochastically dominated by a random variable X. Assume that conditions (A1)–(A3) hold true. If there exists some s>0 such that E|X|1+1/s<∞ and
(2.1)max1≤i≤n|Wni(x)|=O(n-s),
then
(2.2)gn(x)→g(x)completely,x∈c(g).As an application of Theorem2.1, we give the complete consistency for the nearest neighbor estimator of g(x). Without loss of generality, put A=[0,1], taking xni=i/n, i=1,2,…,n. For any x∈A, we rewrite |xn1-x|,|xn2-x|,…,|xnn-x| as follows:
(2.3)|xR1(x)(n)-x|≤|xR2(x)(n)-x|≤⋯≤|xRn(x)(n)-x|,
if |xni-x|=|xnj-x|, then |xni-x| is permuted before |xnj-x| when xni<xnj.Let1≤kn≤n, the nearest neighbor weight function estimator of g(x) in model (1.1) is defined as follows:
(2.4)g~n(x)=∑i=1n‍W~ni(x)Yni,
where
(2.5)W~ni(x)={1kn,if|xni-x|≤|xRkn(x)(n)-x|.0,otherwise.Based on the notations above, we can get the following result by using Theorem2.1.Corollary 2.2.
Let{εn,n≥1} be a sequence of ρ~-mixing random variables with mean zero, which is stochastically dominated by a random variable X. Suppose that g is continuous on the compact set A. If there exists some 0<s<1 such that kn=⌈ns⌉ and E|X|1+1/s<∞, then
(2.6)g~n(x)→g(x)completely,x∈c(g).


## 3. Preliminaries

In this section, we will present some important lemmas which will be used to prove the main results of the paper. The first one is the Rosenthal-type inequality, which was proved by Utev and Peligrad [16].Lemma 3.1.
Let{Xn,n≥1} be a ρ~-mixing sequence of random variables, EXi=0, E|Xi|p<∞ for some p≥2 and for every i≥1. Then there exists a positive constant C depending only on p such that
(3.1)E(max1≤j≤n|∑i=1j‍Xi|p)≤C{∑i=1n‍E|Xi|p+(∑i=1n‍EXi2)p/2}.The concept of stochastic domination will be used in this work.Definition 3.2.
A sequence{Xn,n≥1} of random variables is said to be stochastically dominated by a random variable X if there exists a positive constant C such that
(3.2)P(|Xn|>x)≤CP(|X|>x)
for all x≥0 and n≥1.Lemma 3.3.
Let{Xn,n≥1} be a sequence of random variables which is stochastically dominated by a random variable X. For any α>0 and b>0, the following two statements hold:
(3.3)E|Xn|αI(|Xn|≤b)≤C1[E|X|αI(|X|≤b)+bαP(|X|>b)],E|Xn|αI(|Xn|>b)≤C2E|X|αI(|X|>b),
where C1 and C2 are positive constants.


## 4. Proofs of the Main Results

Proof of Theorem2.1.
Forx∈c(g) and a>0, we have by (1.1) and (1.2) that
(4.1)|Egn(x)-g(x)|≤∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥≤a)+∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a)+|g(x)|·|∑i=1n‍Wni(x)-1|.
Since x∈c(g), hence for any ε>0, there exists a δ>0 such that |g(x′)-g(x)|<ε when ∥x′-x∥<δ. Thus, by setting 0<a<δ in (4.1), we can get that
(4.2)|Egn(x)-g(x)|≤ε∑i=1n‍|Wni(x)|+|g(x)|·|∑i=1n‍Wni(x)-1|+∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a).
By conditions (A1)–(A3) and the arbitrariness of ε>0, we can get that
(4.3)limn→∞Egn(x)=g(x),x∈c(g).
For fixed design point x∈c(g), note that Wni(x)=Wni+(x)-Wni-(x), so without loss of generality, we assume that Wni(x)≥0 in what follows.
From the condition (2.1), we assume that
(4.4)max1≤i≤nWni(x)=n-s,n≥1.
By (4.3), we can see that in order to prove (2.2), we only need to show that
(4.5)gn(x)-Egn(x)=∑i=1n‍Wni(x)εni→0completelyasn→∞.
That is to say, it suffices to show that for all ε>0,
(4.6)∑n=1∞‍P(|∑i=1n‍Wni(x)εni|>ε)<∞.
For fixed n≥1, denote
(4.7)Xni=Wni(x)εniI(|Wni(x)εni|≤1),i=1,2,…,n.
It is easy to check that for any ε>0,
(4.8)(|∑i=1n‍Wni(x)εni|>ε)⊂(max1≤i≤n|Wni(x)εni|>1)⋃‍(|∑i=1n‍Xni|>ε),
which implies that
(4.9)∑n=1∞‍P(|∑i=1n‍Wni(x)εni|>ε)≤∑n=1∞‍∑i=1n‍P(|Wni(x)εni|>1)+∑n=1∞‍P(|∑i=1n‍Xni|>ε)≐I+J.
Hence, to prove (4.6), it suffices to show that I<∞ and J<∞.
By condition (A1) and E|X|1+1/s<∞, we can get that
(4.10)∑n=1∞‍∑i=1n‍P(|Wni(x)εni|>1)≤C∑n=1∞‍∑i=1n‍P(|Wni(x)X|>1)≤C∑n=1∞‍∑i=1n‍Wni(x)E|X|I(|Wni(x)X|>1)≤C∑n=1∞‍E|X|I(|X|>ns)≤C∑n=1∞‍∑k=n∞‍E|X|I(ks≤|X|<(k+1)s)=C∑k=1∞‍∑n=1k‍E|X|I(ks≤|X|<(k+1)s)=C∑k=1∞‍kE|X|I(ks≤|X|<(k+1)s)≤C∑k=1∞‍E|X|1+1/sI(ks≤|X|<(k+1)s)≤CE|X|1+1/s<∞,
which implies that I<∞.
Next, we will prove thatJ<∞. Firstly, we will show that
(4.11)|∑i=1n‍EXni|→0,asn→∞.
Actually, by the conditions Eεi=0, Lemma 3.3, (4.4) and E|X|1+1/s<∞, we can see that
(4.12)|∑i=1n‍EXni|=|∑i=1n‍EWni(x)εiI(|Wni(x)εi|>1)|≤C∑i=1n‍E|Wni(x)εi|1+1/sI(|Wni(x)εi|>1)≤C∑i=1n‍Wni1+1/s(x)E|X|1+1/sI(|Wni(x)X|>1)≤C(max1≤i≤nWni(x))1/s∑i=1n‍Wni(x)E|X|1+1/sI(|X|>ns)≤C(n-s)1/sE|X|1+1/sI(|X|>ns)=Cn-1E|X|1+1/sI(|X|>ns)→0,asn→∞,
which implies (4.11). Hence, to prove J<∞, we only need to show that for all ε>0,
(4.13)J*≐∑n=1∞‍P(|∑i=1n‍(Xni-EXni)|>ε2)<∞.
By Markov’s inequality, Lemma 3.1, Cr’s inequality, and Jensen’s inequality, we have for p≥2 that
(4.14)J*≤C∑n=1∞‍E(|∑i=1n‍(Xni-EXni)|p)≤C∑n=1∞‍(∑i=1n‍E|Xni|2)p/2+C∑n=1∞‍∑i=1n‍E|Xni|p≐J1+J2.
Take
(4.15)p>max{2,2s,1+1s},
which implies that -sp/2<-1 and -s(p-1)<-1. By Cr’s inequality and Lemma 3.3, we can get
(4.16)J1≤C∑n=1∞‍[∑i=1n‍P(|Wni(x)X|>1)+∑i=1n‍E|Wni(x)X|2I(|Wni(x)X|≤1)]p/2.
If s>1, then we have by Markov’s inequality, E|X|1+1/s<∞ and (4.4) that
(4.17)J1≤C∑n=1∞‍(∑i=1n‍Wni1+1/s(x)E|X|1+1/s)p/2≤C∑n=1∞‍[(max1≤i≤nWni(x))1/s∑i=1n‍Wni(x)]p/2≤C∑n=1∞‍n-p/2<∞.
If 0<s≤1, then we have by Markov’s inequality, E|X|1+1/s<∞ and (4.4) again that
(4.18)J1≤C∑n=1∞‍(∑i=1n‍Wni2(x)E|X|2)p/2≤C∑n=1∞‍[(max1≤i≤nWni(x))∑i=1n‍Wni(x)]p/2≤C∑n=1∞‍n-sp/2<∞.
From (4.14)–(4.18), we have proved that J1<∞.
ByCr’s inequality and Lemma 3.3, we can see that
(4.19)J2≤C∑n=1∞‍∑i=1n‍P(|Wni(x)X|>1)+C∑n=1∞‍∑i=1n‍E|Wni(x)X|pI(|Wni(x)X|≤1)≐J3+J4.J3<∞ has been proved by (4.10). In the following, we will show that J4<∞. Denote
(4.20)Inj={i:[n(j+1)]-s<Wni(x)≤(nj)-s},n≥1,j≥1.
It is easily seen that Ink⋂‍Inj=∅ for k≠j and ⋃j=1∞‍Inj={1,2,…,n} for all n≥1. Hence,
(4.21)J4≤C∑n=1∞‍∑j=1∞‍∑i∈Inj‍E|Wni(x)X|pI(|Wni(x)X|≤1)≤C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-spE|X|pI(|X|≤[n(j+1)]s)≤C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-sp∑k=0n(j+1)‍E|X|pI(k≤|X|1/s<k+1)=C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-sp∑k=02n‍E|X|pI(k≤|X|1/s<k+1)+C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-sp∑k=2n+1n(j+1)‍E|X|pI(k≤|X|1/s<k+1)≐J5+J6.
It is easily seen that for all m≥1,
(4.22)C≥∑i=1n‍Wni(x)=∑j=1∞‍∑i∈Inj‍Wni(x)≥∑j=1∞‍(♯Inj)[n(j+1)]-s≥∑j=m∞‍(♯Inj)[n(j+1)]-s≥∑j=m∞‍(♯Inj)[n(j+1)]-s[n(m+1)n(j+1)]s(p-1)=∑j=m∞‍(♯Inj)[n(j+1)]-sp[n(m+1)]s(p-1),
which implies that for all m≥1,
(4.23)∑j=m∞‍(♯Inj)(nj)-sp≤Cn-s(p-1)·m-s(p-1).
Therefore,
(4.24)J5≐C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-sp∑k=02n‍E|X|pI(k≤|X|1/s<k+1)≤C∑n=1∞‍n-s(p-1)∑k=02n‍E|X|pI(k≤|X|1/s<k+1)≤C∑k=02‍∑n=1∞‍n-s(p-1)E|X|pI(k≤|X|1/s<k+1)+C∑k=2∞‍∑n=⌈k/2⌉∞‍n-s(p-1)E|X|pI(k≤|X|1/s<k+1)≤C+C∑k=2∞‍k1-s(p-1)E|X|pI(k≤|X|1/s<k+1)≤C+C∑k=2∞‍E|X|p+1/s-(p-1)I(k≤|X|1/s<k+1)≤C+CE|X|1+1/s<∞,J6≐C∑n=1∞‍∑j=1∞‍(♯Inj)(nj)-sp∑k=2n+1n(j+1)‍E|X|pI(k≤|X|1/s<k+1)≤C∑n=1∞‍∑k=2n+1∞‍∑j≥k/n-1‍(♯Inj)(nj)-spE|X|pI(k≤|X|1/s<k+1)≤C∑n=1∞‍∑k=2n+1∞‍n-s(p-1)(kn)-s(p-1)E|X|pI(k≤|X|1/s<k+1)≤C∑k=2∞‍∑n=1⌈k/2⌉‍k-s(p-1)E|X|pI(k≤|X|1/s<k+1)≤C∑k=2∞‍k1-s(p-1)E|X|pI(k≤|X|1/s<k+1)≤C∑k=2∞‍E|X|p+1/s-(p-1)I(k≤|X|1/s<k+1)≤CE|X|1+1/s<∞.
Thus, the inequality (4.13) follows from (4.14)–(4.19), (4.21), and (4.24). This completes the proof of the theorem.Proof of Corollary2.2.
It suffices to show that the conditions of Theorem2.1 are satisfied. Since g is continuous on the compact set A, hence, g is uniformly continuous on the compact set A, which implies that {|g(xni)-g(x)|:1≤i≤n,n≥1} is bounded on set A.
For anyx∈[0,1], if follows from the definition of Ri(x) and W~ni(x) that
(4.25)∑i=1n‍W~ni(x)=∑i=1n‍W~nRi(x)(x)=∑i=1kn‍1kn=1,max1≤i≤nW~ni(x)=1kn,W~ni(x)≥0,∑i=1n‍|W~ni(x)|·|g(xni)-g(x)|I(|xni-x|>a)≤C∑i=1n‍(xni-x)2|W~ni(x)|a2=C∑i=1kn‍(xRi(x)(n)-x)2kna2≤C∑i=1kn‍(i/n)2kna2≤C(knna)2,∀a>0.
Hence, conditions (A1)–(A3) and (2.1) are satisfied. By Theorem 2.1, we can get (2.6) immediately. This completes the proof of the corollary.



---

*Source: 907286-2012-12-10.xml*