# The Strong Consistency of the Estimator of Fixed-Design Regression Model under Negatively Dependent Sequences

Authors: Xuejun Wang; Meimei Ge; Shuhe Hu; Xize Wang
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

We study the strong consistency of estimator of fixed design regression model under negatively dependent sequences by using the classical Rosenthal-type inequality and the truncated method. As an application, the strong consistency for the nearest neighbor estimator is obtained.


---

## Body

## 1. Introduction

Let{Xn,n≥1} be a sequence of random variables defined on a fixed probability space (Ω,ℱ,P). It is well known that the Rosenthal-type inequality for the partial sum ∑i=1n‍Xi plays an important role in probability limit theory and mathematical statistics. The main purpose of the paper is to investigate the strong consistency of the estimator of fixed design regression model under negatively dependent sequences, by using the Rosenthal-type inequality.Consider the following fixed design regression model:(1)Yni=g(xni)+ɛni,i=1,2,…,n,n≥1,
where xni are known fixed design points from A, where A⊂ℝp is a given compact set for some p≥1, g(·) is an unknown regression function defined on A, and ɛni are random errors. Assume that, for each n≥1, (ɛn1,ɛn2,…,ɛnn) have the same distribution as (ɛ1,ɛ2,…,ɛn). As an estimator of g(·), the following weighted regression estimator will be considered:
(2)gn(x)=∑i=1n‍Wni(x)Yni,x∈A⊂ℝp,
where Wni(x)=Wni(x;xn1,xn2,…,xnn), i=1,2,…,n, are the weight function.The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. For instance, when ɛni are assumed to be independent, consistency and asymptotic normality have been studied by Georgiev and Greblicki [2], Georgiev [3], and Müller [4] among others. Results for the case when ɛni are dependent have also been studied by various authors in recent years. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case which form an Lq-mixingale sequence for some 1≤q≤2. Roussas [6] discussed strong consistency and quadratic mean consistency for gn(x) under mixing conditions. Roussas et al. [7] established asymptotic normality of gn(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] discussed again asymptotic normality of gn(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] studied the asymptotic normality for double array sum of linear time series. Hu et al. [10] gave the mean consistency, complete consistency, and asymptotic normality of regression models with linear process errors. Liang and Jing [11] presented some asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences; Yang et al. [12] generalized the results of Liang and Jing [11] for negatively associated sequences to the case of negatively orthant dependent sequences. Shen [13] presented the Bernstein-type inequality for widely dependent random variables and gave its application to nonparametric regression models, and so forth. The main purpose of this section is to investigate the strong consistency of estimator of the fixed design regression model based on negatively dependent random variables.The concept of negatively dependent random variables was introduced by Lehmann [14] as follows.A finite collection of random variablesX1,X2,…,Xn is said to be negatively dependent (or negatively orthant dependent, ND in short) if
(3)P(X1>x1,X2>x2,…,Xn>xn)≤∏i=1n‍P(Xi>xi),P(X1≤x1,X2≤x2,…,Xn≤xn)≤∏i=1n‍P(Xi≤xi)
hold for all x1,x2,…,xn∈ℝ. An infinite sequence {Xn,n≥1} is said to be ND if every finite subcollection is ND.Obviously, independent random variables are ND. Joag-Dev and Proschan [15] pointed out that negatively associated (NA, in short) random variables are ND. They also presented an example in which X=(X1,X2,X3,X4) possesses ND but does not possess NA. The another example which is ND but is not NA was provided by Wu [16] as follows.Example 1.
LetXi be a binary random variable such that P(Xi=1)=P(Xi=0)=0.5 for i=1,2,3. Let (X1,X2,X3) take the values (0,0,1), (0,1,0), (1,0,0), and (1,1,1), each with probability 1/4. It can be verified that all the ND conditions hold. However,
(4)P(X1+X3≤1,X2≤0)=48≰P(X1+X3≤1)P(X2≤0)=38.
Hence, X1,X2, and X3 are not NA.So we can see that ND is weaker than NA. A number of well-known multivariate distributions have the ND properties, such as multinomial, convolution of unlike multinomials, multivariate hypergeometric, dirichlet, dirichlet compound multinomial, and multinomials having certain covariance matrices. Because of the wide applications of ND random variables, the limiting behaviors of ND random variables have received more and more attention recently. A number of useful results for ND random variables have been established by many authors. We refer to Volodin [17] for the Kolmogorov exponential inequality, Asadian et al. [18] for Rosenthal's type inequality, Kim [19] for Hájek-Rényi type inequality, Amini et al. [20, 21], Ko and Kim [22], Klesov et al. [23], and Wang et al. [24] for almost sure convergence, Amini and Bozorgnia [25], Kuczmaszewska [26], Taylor et al. [27], Zarei and Jabbari [28], Wu [16, 29], Sung [30], and Wang et al. [31] for complete convergence, Wang et al. [32] for exponential inequalities and inverse moment, Shen [33] for strong limit theorems for arrays of rowwise ND random variables, Shen [34] for strong convergence rate for weighted sums of arrays of rowwise ND random variables, and so on. When these are compared with the corresponding results of independent random variable sequences, there still remains much to be desired.This work is organized as follows: some preliminary lemmas are presented in Section2, and the main results and their proofs are provided in Section 3.Throughout the paper,C denotes a positive constant not depending on n, which may be different in various places. an=O(bn) represents an≤Cbn for all n≥1. Let [x] denote the integer part of x, and let I(A) be the indicator function of the set A. Denote x+=xI(x≥0) and x-=-xI(x<0).


## 2. Preliminaries

In this section, we will present some important lemmas which will be used to prove the main results of the paper.Lemma 2 (cf. Bozorgnia et al. [35]).
Let the random variablesX1,X2,…,Xn be ND, and let f1,f2,…,fn be all nondecreasing (or all nonincreasing) functions. Then random variables f1(X1),f2(X2),…,fn(Xn) are ND.Lemma 3 (cf. Asadian et al. [18]).
Letp≥2, and let {Xn,n≥1} be a sequence of ND random variables with EXn=0 and E|Xn|p<∞ for every n≥1. Then there exists a positive constant C depending only on p such that for every n≥1,
(5)E|∑i=1n‍Xi|p≤C{∑i=1n‍E|Xi|p+(∑i=1n‍EXi2)p/2}.The following concept of stochastic domination will be used in this work.Definition 4.
A sequence{Xn,n≥1} of random variables is said to be stochastically dominated by a random variable X if there exists a positive constant C such that
(6)P(|Xn|>x)≤CP(|X|>x)
for all x≥0 and n≥1.By the definition of stochastic domination and integration by parts, we can get the following property for stochastic domination. For the details of the proof, one can refer to Wu [36, 37] or Shen and Wu [38].Lemma 5.
Let{Xn,n≥1} be a sequence of random variables which is stochastically dominated by a random variable X. For any α>0 and b>0, the following two statements hold:
(7)E|Xn|αI(|Xn|≤b)≤C1[E|X|αI(|X|≤b)+bαP(|X|>b)],E|Xn|αI(|Xn|>b)≤C2E|X|αI(|X|>b),
where C1 and C2 are positive constants.


## 3. Main Results and Their Proofs

Unless otherwise specified, we assume throughout the paper thatgn(x) is defined by (2). For any function g(x), we use c(g) to denote all continuity points of the function g on A. The norm ∥x∥ is the Eucledean norm. For any fixed design point x∈A, the following assumptions on weight function Wni(x) will be used:(A1)
∑i=1n‍Wni(x)→1 as n→∞;(A2)
∑i=1n‍|Wni(x)|≤C<∞ for all n;(A3)
∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a)→0 as n→∞ for all a>0.Based on the assumptions above, we can get the following strong consistency of the fixed design regression estimatorgn(x).Theorem 6.
Let{ɛn,n≥1} be a sequence of mean zero ND random variables, which is stochastically dominated by a random variable X. Assume that conditions (A1)–(A3) hold true. If E|X|p<∞ for some p>1, and if there exist some s∈(1/p,1) such that
(8)max1≤i≤n|Wni(x)|=O(n-s),
then for any x∈c(g),
(9)gn(x)⟶g(x)a.s.,asn⟶∞.Proof.
Forx∈c(g) and a>0, we have by (1) and (2) that
(10)|Egn(x)-g(x)|≤∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥≤a)+∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a)+|g(x)|·|∑i=1n‍Wni(x)-1|.
Since x∈c(g), hence for any ɛ>0, there exist a δ>0 and ∥x*-x∥<δ such that |g(x*)-g(x)|<ɛ. If we take a∈(0,δ) in (10), we can get that
(11)|Egn(x)-g(x)|≤ɛ∑i=1n‍|Wni(x)|+|g(x)|·|∑i=1n‍Wni(x)-1|+∑i=1n‍|Wni(x)|·|g(xni)-g(x)|I(∥xni-x∥>a).
Therefore, we have by conditions (A1)–(A3) that
(12)limn→∞Egn(x)=g(x),x∈c(g).
For a fixed design point x∈c(g), without loss of generality, we assume that Wni(x)≥0 (otherwise, we use Wni+(x) and Wni-(x) instead of Wni(x), resp., and note that Wni(x)=Wni+(x)-Wni-(x)).
By (12), we can see that in order to prove (9), we only need to show that
(13)gn(x)-Egn(x)=∑i=1n‍Wni(x)ɛni≐∑i=1nRni⟶0a.s.,asn⟶∞,
where Rni=Wni(x)ɛni.
For fixedɛ>0, choose 1/p<δ<s and some positive integer N (to be specified later). Denote, for i=1,2,…,n,
(14)Xni(1)=-n-δI(Rni<-n-δ)+RniI(|Rni|≤n-δ)+n-δI(Rni>n-δ),Xni(2)=(Rni-n-δ)I(n-δ<Rni<ɛN),Xni(3)=(Rni+n-δ)I(-n-δ>Rni>-ɛN),Xni(4)=(Rni-n-δ)I(Rni≥ɛN)+(Rni+n-δ)I(Rni≤-ɛN).
It is easy to check that Xni(1)+Xni(2)+Xni(3)+Xni(4)=Rni. Hence, to prove (13), it suffices to show that, for any x∈c(g),
(15)∑i=1n‍Xni(j)⟶0a.s.,asn⟶∞,j=1,2,3,4.
ByEɛn=0 and E|X|p<∞, we can see that
(16)|∑i=1n‍EXni(1)|≤∑i=1n‍[n-δP(|Rni|>n-δ)+E|Rni|I(|Rni|>n-δ)]≤∑i=1n‍n-δE|Rni|pn-pδ≤Cnδ(p-1)∑i=1n‍|Wni(x)|pE|X|p≤Cnδ(p-1)(max1≤i≤n|Wni(x)|)p-1∑i=1n‍|Wni(x)|≤Cn-(p-1)(s-δ)⟶0,asn⟶∞.
Hence, to prove ∑i=1n‍Xni(1)→0a.s., it suffices to show that
(17)∑n=1∞‍P(|∑i=1n‍(Xni(1)-EXni(1))|>ɛ)<∞.
It is easily seen that, for fixed n≥1 and x∈c(g), {Xni(1),1≤i≤n} are still ND random variables by Lemma 2. Applying Lemma 3, we have for q≥2 that
(18)∑n=1∞‍P(|∑i=1n‍(Xni(1)-EXni(1))|>ɛ)≤C∑n=1∞‍∑i=1n‍E|Xni(1)|q+C∑n=1∞‍(∑i=1n‍E|Xni(1)|2)q/2≐CI11+CI12.
Taking q>max{p,(s+1+p(δ-s))/δ,2/((s-δ)(p-1)+δ),2/s}, we can get that
(19)(q-p)δ+s(p-1)>1,qs2>1,[(2-p)δ+s(p-1)]q2=[(s-δ)(p-1)+δ]q2>1.
ForI11, we have by Cr's inequality, Lemma 5, and condition (A2) that
(20)I11≤C∑n=1∞‍∑i=1n‍[n-qδP(|Rni|>n-δ)dd.dddd+E|Rni|qI(|Rni|≤n-δ)]≤C∑n=1∞‍∑i=1n‍[n-qδP(|Wni(x)X|>n-δ)dddddd+E|Wni(x)X|qI(|Wni(x)X|≤n-δ)]≤C∑n=1∞‍∑i=1n‍n-qδE|Wni(x)X|pn-pδ≤C∑n=1∞‍n-(q-p)δ(max1≤i≤n|Wni(x)|)p-1×∑i=1n‍|Wni(x)|≤C∑n=1∞‍n-[(q-p)δ+s(p-1)]<∞.
ForI12, if 1<p<2, we have by the proof of (20) that
(21)I12≤C∑n=1∞‍{∑i=1n‍[n-2δP(|Rni|>n-δ)ddddddd+E|Rni|2I(|Rni|≤n-δ)]∑i=1n‍}q/2≤C∑n=1∞‍n-[(2-p)δ+s(p-1)]q/2<∞;
if p≥2, we have by EX2<∞ that
(22)I12≤C∑n=1∞‍{∑i=1n‍[n-2δP(|Rni|>n-δ)ddddddddddd+E|Rni|2I(|Rni|≤n-δ)]∑i=1n‍}q/2≤C∑n=1∞‍{∑i=1n‍[n-2δP(|Wni(x)X|>n-δ)+Wni2(x)]}q/2≤C∑n=1∞‍n-[(2-p)δ+s(p-1)]q/2+C∑n=1∞‍n-qs/2<∞.
By (18)–(22), we can get (17), which together with (16) imply that ∑i=1n‍Xni(1)→0a.s.
Next, we will prove that∑i=1n‍Xni(2)→0a.s. Since 0≤Xni(2)<ɛ/N, |∑i=1n‍Xni(2)|=∑i=1n‍Xni(2)>ɛ implies that there are at least N  i’s such that Xni(2)≠0. Hence,
(23)P(|∑i=1n‍Xni(2)|>ɛ)≤P(thereareatleastNi’ssuchthatXni(2)≠0)≤∑1≤i1<i2<⋯<iN≤n‍P(Xni1(2)≠0,Xni2(2)≠0,ddddddddddddddd…,XniN(2)≠0)≤∑1≤i1<i2<⋯<iN≤n‍P(Rni1>n-δ,Rni2>n-δ,…,RniN>n-δ)≤∑1≤i1<i2<⋯<iN≤n‍P(Rni1>n-δ)P(Rni2>n-δ)ddddddddddddd⋯P(RniN>n-δ)≤[∑i=1n‍P(Rni>n-δ)]N≤[∑i=1n‍P(|Rni|>n-δ)]N≤C[∑i=1n‍E|Wni(x)X|pn-pδ]N≤C[npδ(max1≤i≤n|Wni(x)|)p-1∑i=1n‍|Wni(x)|]N≤Cn-[(s-δ)p+s]N,
which is summable if we choose large integer N such that [(s-δ)p+s]N>1. Hence, ∑i=1n‍Xni(2)→0a.s. by Borel-Cantelli lemma.
Since-ɛ/N<Xni(3)≤0, |∑i=1n‍Xni(3)|=-∑i=1n‍Xni(3)>ɛ implies that there are at least Ni’s such that Xni(3)≠0. Similarly, we have ∑i=1n‍Xni(3)→0a.s. as n→∞.
Finally, we will prove that∑i=1n‍Xni(4)→0a.s. as n→∞. It is easily seen that
(24)|∑i=1n‍Xni(4)|≤∑i=1n‍|Rni|I(|Rni|≥ɛN)+n-δ∑i=1n‍I(|Rni|≥ɛN)≤Cn-s∑i=1n‍|ɛni|I(|ɛni|≥Cns)+n-δ∑i=1n‍I(|ɛni|≥Cns)≤Cn-s∑i=1n‍|ɛni|I(|ɛni|≥Cis)+n-δ∑i=1n‍I(|ɛni|≥Cis)≐Hn1+Hn2.
To prove that ∑i=1n‍Xni(4)→0a.s. as n→∞, it suffices to show that Hn1→0a.s. and Hn2→0a.s. as n→∞. Firstly, we will show that
(25)∑i=1∞‍i-s|ɛni|I(|ɛni|≥Cis)<∞a.s.,(26)∑i=1∞‍i-δI(|ɛni|≥Cis)<∞a.s.
Denote that Tm=∑i=1m‍i-s|ɛni|I(|ɛni|≥Cis). For m≥k≥1, it follows by Lemma 5 and E|X|p<∞ that
(27)E|Tm-Tk|=∑i=k+1m‍i-sE|ɛni|I(|ɛni|≥Cis)≤C∑i=k+1m‍i-sE|X|I(|X|≥Cis)≤C∑i=k+1m‍i-sp≤Ck-sp+1⟶0ask⟶∞,
which implies that {Tm,m≥1} is a Cauchy sequence in L1, and hence there exists a random variable T such that E|T|<∞ and E|Tm-T|→0. It follows by Lemma 5 and E|X|p<∞ again that
(28)P(|T2k-T|>ɛ)≤CE|T2k-T|≤Climsupm→∞E|T2k-Tm|≤C∑i=2k+1∞‍i-sE|ɛni|I(|ɛni|≥Cis)≤C∑i=2k+1∞‍i-sE|X|I(|X|≥Cis)≤C∑i=2k+1∞‍i-sp≤C2-k(sp-1),P(max2k-1<m≤2k|Tm-T2k-1|>ɛ)≤CE(max2k-1<m≤2k|Tm-T2k-1|)≤C∑i=2k-1+12k‍i-sE|ɛni|I(|ɛni|≥Cis)≤C∑i=2k-1+12k‍i-sp≤C2-k(sp-1),
which together imply that Tm→Ta.s., and thus (25) holds. Similarly, we can get (26). Therefore, Hn1→0a.s. and Hn2→0a.s. follow by (25)-(26) and the Kronecker's lemma immediately. This completes the proof of the theorem.As an application of Theorem6, we give the strong consistency for the nearest neighbor estimator of g(x). Without loss of generality, putting A=[0,1], taking xni=i/n, i=1,2,…,n, and n≥1. For any x∈A, we rewrite |xn1-x|,|xn2-x|,…,|xnn-x| as follows:
(29)|xR1(x)(n)-x|≤|xR2(x)(n)-x|≤⋯≤|xRn(x)(n)-x|.
If |xni-x|=|xnj-x|, then |xni-x| is permuted before |xnj-x| when xni<xnj.Let1≤kn≤n, the nearest neighbor weight function estimator of g(x) in model (1) is defined as follows:
(30)g~n(x)=∑i=1n‍W~ni(x)Yni,
where
(31)W~ni(x)={1kn,if|xni-x|≤|xRkn(x)(n)-x|,0,otherwise.Based on the notations above, we can get the following result by using Theorem6.Corollary 7.
Let{ɛn,n≥1} be a sequence of mean zero ND random variables, which is stochastically dominated by a random variable X. Assume that g is continuous on the compact set A. If E|X|p<∞ for some p>1 and if there exists some s∈(1/p,1) such that kn=[ns], then for any x∈c(g),
(32)g~n(x)⟶g(x)a.s.,asn⟶∞.Proof.
It suffices to show that the conditions of Theorem6 are satisfied. Since g is continuous on the compact set A, hence g is uniformly continuous on the compact set A, which implies that {|g(xni)-g(x)|:1≤i≤n,n≥1} is bounded on the set A.
For anyx∈[0,1], if follows from the definition of Ri(x) and W~ni(x) that
(33)∑i=1n‍W~ni(x)=∑i=1n‍W~nRi(x)(x)=∑i=1kn‍1kn=1,max1≤i≤nW~ni(x)=1kn,W~ni(x)≥0,∑i=1n‍|W~ni(x)|·|g(xni)-g(x)|I(|xni-x|>a)≤C∑i=1n‍(xni-x)2|W~ni(x)|a2=C∑i=1kn‍(xRi(x)(n)-x)2kna2≤C∑i=1kn‍(i/n)2kna2≤C(knna)2,∀a>0.
Hence, conditions (A1)–(A3) and (8) are satisfied. By Theorem 6, we can get (32) immediately. This completes the proof of the corollary.



---

*Source: 521618-2013-11-26.xml*