# On the Stochastic Restrictedr-k Class Estimator and Stochastic Restricted r-d Class Estimator in Linear Regression Model

Authors: Jibo Wu
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

The stochastic restrictedr-k class estimator and stochastic restricted r-d class estimator are proposed for the vector of parameters in a multiple linear regression model with stochastic linear restrictions. The mean squared error matrix of the proposed estimators is derived and compared, and some properties of the proposed estimators are also discussed. Finally, a numerical example is given to show some of the theoretical results.


---

## Body

## 1. Introduction

The problem of multicollinearity or the ill-conditioned design matrix in linear regression model is very well known in statistics. In order to overcome this problem, different remedies have been introduced. One of the most important estimation methods is to consider biased estimators, such as the principal component regression (PCR) estimator [1], the ridge estimator (ORE) by Hoerl and Kennard [2], the r-k class estimator [3], the Liu estimator (LE) by Liu [4], the r-d class estimator [5], the r-k-d class estimator [6], and the principal component Liu-type estimator [7].An alternative method to deal with multicollinearity problem is to consider parameter estimation with some restrictions on the unknown parameters, which may be exact or stochastic restrictions [8]. When stochastic additional restrictions on the parameter vector are supposed to hold, Durbin [9], Theil and Goldberger [10], and Theil [11] proposed the ordinary mixed estimator (OME). By grafting the ordinary regression ridge estimator and LE into the mixed estimation, Li and Yang [12] and Hubert and Wijekoon [13] introduced a stochastic restricted ridge estimator (SRRE) and stochastic restricted Liu estimator (SRLE), respectively, and Liu et al. [14] proposed the weighted mixed almost unbiased ridge estimator in linear regression model.In this paper, in order to overcome multicollinearity, we introduce a stochastic restrictedr-k class estimator and a stochastic restricted r-d class estimator for the vector of parameters in a linear regression model when additional stochastic linear restrictions are assumed to hold. Performance of the proposed estimators with respect to the mean squared error matrix (MSEM) criterion is discussed.The rest of the paper is organized as follows. The model specifications and the new estimators are introduced in Section2. Then, the superiority of the proposed estimators is discussed in Section 3 and a numerical example is given to illustrate the behavior of the estimators in Section 4. Finally, some conclusion remarks are given in Section 5.


## 2. Model Specifications and the Estimators

Consider the linear regression model(1)y=Xβ+ε,
where y is an n×1 vector of observation, X is an n×p known design matrix of rank p, β is a p×1 vector of unknown parameters, and ε is an n×1 vector of disturbances with expectation E(ε)=0 and variance-covariance matrix Cov(ε)=σ2In.For the unrestricted model given by (1), the ORE proposed by Hoerl and Kennard [2] and the LE presented by Liu [4] are defined as follows:
(2)β^(k)=Sk-1X′y=Rkβ^OLS,β^(d)=(S+I)-1(X′y+dβ^OLS)=Fdβ^OLS,
where k>0, 0<d<1, S=X′X, Sk=S+kI, Rk=Sk-1S, Fd=(S+I)-1(S+dI), and β^OLS=S-1X′y is the ordinary least squares (OLS) estimator of β.Now let us consider the spectral decomposition of the matrix given as(3)X′X=(Tr,Tp-r)(Λr00Λp-r)(Tr′Tp-r′),
where Λr and Λp-r are diagonal matrices such that that the main diagonal elements of the r×r matrix Λr are the r largest eigenvalues of X′X, while Λp-r are the remaining p-r eigenvalues. The p×p matrix T=(Tr,Tp-r) is orthogonal with Tr=(t1,t2,…,tr) consisting of its first r columns and Tp-r=(tr+1,tr+2,…,tp) consisting of the remaining p-r columns of the matrix T. The PCR estimator for β can be written as
(4)β^r=Tr(Tr′X′XTr)-1Tr′X′y.Ther-k class estimator proposed by Baye and Parker [3] and the r-d class estimator proposed by Kaçıranlar and Sakallıoğlu [5] are defined as
(5)β^rk(r,k)=Tr(Tr′X′XTr+kIr)-1Tr′X′y,k>0,β^rd(r,d)=Tr(Tr′X′XTr+Ir)-1×(Tr′X′y+dTr′β^r),0<d<1.Followed by Xu and Yang [15], the r-k class estimator and the r-d class estimator could be rewritten as follows:
(6)β^rk(r,k)=Tr(Tr′X′XTr+kIr)-1Tr′X′y=TrTr′β^(k)=TrTr′Rkβ^OLS,(7)β^rd(r,d)=Tr(Tr′X′XTr+Ir)-1(Tr′X′y+dTr′β^r)=TrTr′β^(d)=TrTr′Fdβ^OLS.In addition to the model (1), let us be given some prior information about β in the form of a set of j independent stochastic linear restrictions as follows:
(8)r=Rβ+e,e~(0,σ2W),
where R is a j×p known matrix of rank j, e is a j×1 vector of disturbances with mean 0 and dispersion matrix σ2W, W is supposed to be known and positive definite, and the j×1 vector r can be interpreted as a random variable with expectation E(r)=Rβ. Therefore the restriction (9) does not hold exactly but in the mean, and we suppose r to be known, that is, to be the realized value of the random vector, so that all the expectations are conditional on r [8]. In the following discussions, we do not mention this separately. Furthermore, it is also supposed that the random vector ε is stochastically independent of e.For the restricted model specified by (1) and (8), the stochastic restricted ridge estimator (SRRE) proposed by Li and Yang [12] and the stochastic restricted Liu estimator (SRLE) proposed by Hubert and Wijekoon [13] are defined as
(9)β^SRRE(k)=Rkβ^OME=Rk(S+R′W-1R)-1(X′y+R′W-1r),(10)β^SRLE(d)=Fdβ^OME=Fd(S+R′W-1R)-1(X′y+R′W-1r),
where β^OME=(S+R′W-1R)-1(X′y+R′W-1r) is the well-known ordinary mixed estimator (OME) of β.We are now ready to propose a new stochastic restrictedr-k class estimator which is defined by combing the OME and r-k class estimator and a new stochastic restricted r-d class estimator which is defined by combing the OME and r-d class estimator as follows:
(11)β^SRrk(r,k)=TrTr′Rkβ^OME=TrTr′Rk(S+R′W-1R)-1(X′y+R′W-1r),β^SRrd(r,d)=TrTr′Fdβ^OME=TrTr′Fd(S+R′W-1R)-1(X′y+R′W-1r).From (11), we can see that(1)
whenR=0, we may conclude that β^SRrk(r,k)=β^rk(r,k) and β^SRrd(r,d)=β^rd(r,d);(2)
whenr=p, we may conclude that β^SRrk(r,k)=β^SRRE(k) and β^SRrd(r,d)=β^SRLE(d).At the end of this section, we will list some lemmas which are needed in the following proofs.Lemma 1 (see [8]).
Assume that square matricesA and C are not singular and B and D are matrices with proper orders; then (A+BCD)-1=A-1-A-1B(C-1+DA-1B)-1DA-1.Lemma 2 (see [16]).
LetA be a nonnegative definite matrix, namely, A≥0 and let α be some vector; then A-αα′≥0 if and only if α′A+α≤1, α∈ℜ(A).Lemma 3 (see [17]).
Letn×n matrices be A>0, B≥0, and A-B≥0⇔λ1(BA-1)≤1, where λ1(BA-1) is the largest eigenvalue of BA-1.In the next section, we will make comparison of the biased estimators.


## 3. Model Specifications and the Estimators

The mean squared error matrix (MSEM) of an estimatorβ^ is defined as
(12)M(β^)=E(β^-β)(β^-β)′=Cov(β^)+Bias(β^)Bias(β^)′,
where Cov(β^) is the dispersion matrix and Bias(β^) is the bias vector. For the two given estimators β^1, β^2, the estimator β^2 is said to be superior to the estimator β^1 in the matrix MSE criterion if and only if
(13)Δ(β^1,β^2)=M(β^1)-M(β^2)≥0.Note that the MSEM criterion is always superior to the scalar mean squared error criterion (MSE); we only consider the MSEM comparisons among the estimators.


### 3.1. MSEM Comparisons of ther-k Class Estimator and the Stochastic Restricted r-k (SRrk) Class Estimator

In this subsection, we consider the MSEM comparison between ther-k class estimator and the stochastic restricted r-k (SRrk) class estimator.Firstly, we can compute the bias vector and the variance of stochastic restrictedr-k (SRrk) class estimator as follows:
(14)Bias(β^SRrk(r,k))=E(β^SRrk(r,k))-β=TrTr′Rkβ-β=(TrTr′Rk-I)β,Cov(β^SRrk(r,k))=σ2TrTr′RkARkTrTr′,
where A=(S+R′W-1R)-1.Therefore, the MSEM of the stochastic restrictedr-k class estimator is given by
(15)MSEM(β^SRrk(r,k))=σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′.From (6), we can compute the bias vector and the variance of r-k class estimator as follows:
(16)Bias(β^rk(r,k))=E(β^rk(r,k))-β=TrTr′Rkβ-β=(TrTr′Rk-I)β,Cov(β^rk(r,k))=σ2TrTr′RkS-1RkTrTr′.Therefore, the MSEM of ther-k class estimator is given by
(17)MSEM(β^rk(r,k))=σ2TrTr′RkS-1RkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′.Now we consider the following difference of the MSEM:(18)Δ1=MSEM(β^rk(r,k))-MSEM(β^SRrk(r,k))=σ2TrTr′RkS-1RkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′-{(TrTr′Rk-I)′σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′}=σ2TrTr′Rk(S-1-A)RkTrTr′.Theorem 4.
The stochastic restrictedr-k class estimator β^SRrk(r,k) always dominates the r-k class estimator β^rk(r,k) in the MSEM criterion.Proof.
By Lemma1, we can obtain A=(S+R′W-1R)-1=S-1-S-1R′(W+RS-1R′)-1RS-1; then we obtain S-1-A=S-1R′(W+RS-1R′)-1RS-1≥0. Therefore, from (18), we may conclude that Δ1≥0; that is to say, the stochastic restricted r-k class estimator β^SRrk(r,k) always dominates the r-k class estimator β^rk(r,k) in the MSEM criterion.


### 3.2. MSEM Comparisons of the Stochastic Restricted Ridge Estimator (SRRE) and the Stochastic Restrictedr-k (SRrk) Class Estimator

In this subsection, we consider the MSEM comparison between the stochastic restricted ridge estimator (SRRE) and the stochastic restrictedr-k (SRrk) class estimator.Firstly, from (9), we can compute the bias vector and the variance of stochastic restricted ridge estimator (SRRE) as follows:
(19)Bias(β^SRRE(k))=E(β^SRRE(k))-β=(Rk-I)β,Cov(β^SRRE(k))=σ2RkARk.
Then, we can obtain the MSEM of the SRRE as follows:
(20)MSEM(β^SRRE(k))=σ2RkARk+(Rk-I)ββ′(Rk-I)′.
Now let us consider the following difference:
(21)Δ2=MSEM(β^SRRE(k))-MSEM(β^SRrk(r,k))=σ2RkARk+(Rk-I)ββ′(Rk-I)′-{(TrTr′Rk-I)′σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′}=σ2D+b1b1′-b2b2′,
where D=RkARk-TrTr′RkARkTrTr′, b1=(Rk-I)β and b2=(TrTr′Rk-I)β.Theorem 5.
Whenk>0 and λ1{(TrTr′RkARkTrTr′)(RkARk)-1}≤1, then the stochastic restricted r-k class estimator β^SRrk(r,k) is superior to the stochastic restricted ridge estimator β^SRRE(k) in the MSEM criterion if and only if b2′(σ2D+b1b1′)+b2≤1, b2∈ℜ  (σ2D+b1b1′).Proof.
Fork>0, it is easy to see that Rk=Sk-1S>0 and A=(S+R′W-1R)-1>0. So we can conclude that RkARk>0.
On the other hand,TrTr′RkARkTrTr′ can be rewritten as
(22)TrTr′RkARkTrTr′=T(Ir000)T′RkARk(Ir000)T′.
From (22), it is easy to see that TrTr′RkARkTrTr′≥0. Thus, by Lemma 2, we can obtain that D≥0 when λ1{(TrTr′RkARkTrTr′)(RkARk)-1}≤1. So from (21) and applying Lemma 3, we have Δ2 if and only if b2′(σ2D+b1b1′)+b2≤1, b2∈ℜ(σ2D+b1b1′). This theorem is proved.


### 3.3. MSEM Comparisons of ther-d Class Estimator and the Stochastic Restricted r-d (SRrd) Class Estimator

In this subsection, we consider the MSEM comparison between ther-d class estimator and the stochastic restricted r-d (SRrd) class estimator.Firstly, we can compute the bias vector and the variance of stochastic restrictedr-d (SRrd) class estimator as follows:
(23)Bias(β^SRrd(r,d))=E(β^SRrd(r,d))-β=TrTr′Fdβ-β=(TrTr′Fd-I)β,Cov(β^SRrd(r,d))=σ2TrTr′FdAFdTrTr′.Therefore, we can obtain the MSEM of the stochastic restrictedr-d class estimator as follows:
(24)MSEM(β^SRrd(r,d))=σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′.From (6), we can compute the bias vector and the variance of the r-d class estimator as follows:
(25)Bias(β^rd(r,d))=E(β^rd(r,d))-β=TrTr′Fdβ-β=(TrTr′Fd-I)β,Cov(β^rd(r,d))=σ2TrTr′FdS-1FdTrTr′.
Then, we can obtain the MSEM of the r-d class estimator as follows:
(26)MSEM(β^rd(r,d))=σ2TrTr′FdS-1FdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′.
Now we consider the following difference:
(27)Δ3=MSEM(β^rd(r,d))-MSEM(β^SRrd(r,d))=σ2TrTr′FdS-1FdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′-{(TrTr′Fd-I)′σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′}=σ2TrTr′Fd(S-1-A)FdTrTr′.Theorem 6.
The stochastic restrictedr-d class estimator β^SRrd(r,d) always dominates the r-d class estimator β^rd(r,d) in the MSEM criterion.Proof.
By Lemma1, we can obtain A=(S+R′W-1R)-1=S-1-S-1R′(W+RS-1R′)-1RS-1; then we obtain S-1-A=S-1R′(W+RS-1R′)-1RS-1≥0. Therefore, from (27), we may conclude that Δ3≥0; that is to say, the stochastic restricted r-d class estimator β^SRrd(r,d) always dominates the r-d class estimator β^rd(r,d) in the MSEM criterion.


### 3.4. MSEM Comparisons of the Stochastic Restricted Liu Estimator (SRLE) and the Stochastic Restrictedr-d (SRrd) Class Estimator

In this subsection, we consider the MSEM comparison between the stochastic restricted Liu estimator (SRLE) and the stochastic restrictedr-d (SRrd) class estimator.Firstly, from (10), we can compute the bias vector and the variance of stochastic restricted Liu estimator (SRLE) as follows:
(28)Bias(β^SRLE(d))=E(β^SRLE(d))-β=(Fd-I)β,Cov(β^SRLE(d))=σ2FdAFd.Therefore, we can obtain the MSEM of the stochastic restricted Liu estimator (SRLE) as follows:(29)MSEM(β^SRLE(d))=σ2FdAFd+(Fd-I)ββ′(Fd-I).Now we consider the following difference:(30)Δ4=MSEM(β^SRLE(d))-MSEM(β^SRrd(r,d))=σ2FdAFd+(Fd-I)ββ′(Fd-I)-{(TrTr′Fd-I)′σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′}=σ2H+b3b3′-b4b4′,
where H=FdAFd-TrTr′FdAFdTrTr′, b3=(Fd-I)β and b4=(TrTr′Fd-I)β.Theorem 7.
When0<d<1 and λ1{(TrTr′FdAFdTrTr′)(FdAFd)-1}≤1, then the stochastic restricted r-d class estimator β^SRrd(r,d) is superior to the stochastic restricted Liu estimator β^SRLE(d) in the MSEM criterion if and only if b4′(σ2H+b3b3′)+b4≤1, b4∈ℜ(σ2H+b3b3′).Proof.
For0<d<1, it is easy to see that Fd=(S+I)-1(S+dI)>0 and A=(S+R′W-1R)-1>0. So we can conclude that FdAFd>0.
On the other hand,TrTr′FdAFdTrTr′ can be rewritten as
(31)TrTr′FdAFdTrTr′=T(Ir000)T′FdAFd(Ir000)T′.
From (31), it is easy to see that TrTr′FdAFdTrTr′≥0. Thus, by Lemma 2, we can obtain that H≥0 when λ1{(TrTr′FdAFdTrTr′)(FdAFd)-1}≤1. So from (30) and applying Lemma 3, we have Δ4 if and only if b4′(σ2H+b3b3′)+b4≤1, b4∈ℜ(σ2H+b3b3′). This theorem is proved.Remark. For practical use, we may replace the unknown parameters in the theorems with their appropriate estimators.


## 3.1. MSEM Comparisons of ther-k Class Estimator and the Stochastic Restricted r-k (SRrk) Class Estimator

In this subsection, we consider the MSEM comparison between ther-k class estimator and the stochastic restricted r-k (SRrk) class estimator.Firstly, we can compute the bias vector and the variance of stochastic restrictedr-k (SRrk) class estimator as follows:
(14)Bias(β^SRrk(r,k))=E(β^SRrk(r,k))-β=TrTr′Rkβ-β=(TrTr′Rk-I)β,Cov(β^SRrk(r,k))=σ2TrTr′RkARkTrTr′,
where A=(S+R′W-1R)-1.Therefore, the MSEM of the stochastic restrictedr-k class estimator is given by
(15)MSEM(β^SRrk(r,k))=σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′.From (6), we can compute the bias vector and the variance of r-k class estimator as follows:
(16)Bias(β^rk(r,k))=E(β^rk(r,k))-β=TrTr′Rkβ-β=(TrTr′Rk-I)β,Cov(β^rk(r,k))=σ2TrTr′RkS-1RkTrTr′.Therefore, the MSEM of ther-k class estimator is given by
(17)MSEM(β^rk(r,k))=σ2TrTr′RkS-1RkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′.Now we consider the following difference of the MSEM:(18)Δ1=MSEM(β^rk(r,k))-MSEM(β^SRrk(r,k))=σ2TrTr′RkS-1RkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′-{(TrTr′Rk-I)′σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′}=σ2TrTr′Rk(S-1-A)RkTrTr′.Theorem 4.
The stochastic restrictedr-k class estimator β^SRrk(r,k) always dominates the r-k class estimator β^rk(r,k) in the MSEM criterion.Proof.
By Lemma1, we can obtain A=(S+R′W-1R)-1=S-1-S-1R′(W+RS-1R′)-1RS-1; then we obtain S-1-A=S-1R′(W+RS-1R′)-1RS-1≥0. Therefore, from (18), we may conclude that Δ1≥0; that is to say, the stochastic restricted r-k class estimator β^SRrk(r,k) always dominates the r-k class estimator β^rk(r,k) in the MSEM criterion.


## 3.2. MSEM Comparisons of the Stochastic Restricted Ridge Estimator (SRRE) and the Stochastic Restrictedr-k (SRrk) Class Estimator

In this subsection, we consider the MSEM comparison between the stochastic restricted ridge estimator (SRRE) and the stochastic restrictedr-k (SRrk) class estimator.Firstly, from (9), we can compute the bias vector and the variance of stochastic restricted ridge estimator (SRRE) as follows:
(19)Bias(β^SRRE(k))=E(β^SRRE(k))-β=(Rk-I)β,Cov(β^SRRE(k))=σ2RkARk.
Then, we can obtain the MSEM of the SRRE as follows:
(20)MSEM(β^SRRE(k))=σ2RkARk+(Rk-I)ββ′(Rk-I)′.
Now let us consider the following difference:
(21)Δ2=MSEM(β^SRRE(k))-MSEM(β^SRrk(r,k))=σ2RkARk+(Rk-I)ββ′(Rk-I)′-{(TrTr′Rk-I)′σ2TrTr′RkARkTrTr′+(TrTr′Rk-I)ββ′(TrTr′Rk-I)′}=σ2D+b1b1′-b2b2′,
where D=RkARk-TrTr′RkARkTrTr′, b1=(Rk-I)β and b2=(TrTr′Rk-I)β.Theorem 5.
Whenk>0 and λ1{(TrTr′RkARkTrTr′)(RkARk)-1}≤1, then the stochastic restricted r-k class estimator β^SRrk(r,k) is superior to the stochastic restricted ridge estimator β^SRRE(k) in the MSEM criterion if and only if b2′(σ2D+b1b1′)+b2≤1, b2∈ℜ  (σ2D+b1b1′).Proof.
Fork>0, it is easy to see that Rk=Sk-1S>0 and A=(S+R′W-1R)-1>0. So we can conclude that RkARk>0.
On the other hand,TrTr′RkARkTrTr′ can be rewritten as
(22)TrTr′RkARkTrTr′=T(Ir000)T′RkARk(Ir000)T′.
From (22), it is easy to see that TrTr′RkARkTrTr′≥0. Thus, by Lemma 2, we can obtain that D≥0 when λ1{(TrTr′RkARkTrTr′)(RkARk)-1}≤1. So from (21) and applying Lemma 3, we have Δ2 if and only if b2′(σ2D+b1b1′)+b2≤1, b2∈ℜ(σ2D+b1b1′). This theorem is proved.


## 3.3. MSEM Comparisons of ther-d Class Estimator and the Stochastic Restricted r-d (SRrd) Class Estimator

In this subsection, we consider the MSEM comparison between ther-d class estimator and the stochastic restricted r-d (SRrd) class estimator.Firstly, we can compute the bias vector and the variance of stochastic restrictedr-d (SRrd) class estimator as follows:
(23)Bias(β^SRrd(r,d))=E(β^SRrd(r,d))-β=TrTr′Fdβ-β=(TrTr′Fd-I)β,Cov(β^SRrd(r,d))=σ2TrTr′FdAFdTrTr′.Therefore, we can obtain the MSEM of the stochastic restrictedr-d class estimator as follows:
(24)MSEM(β^SRrd(r,d))=σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′.From (6), we can compute the bias vector and the variance of the r-d class estimator as follows:
(25)Bias(β^rd(r,d))=E(β^rd(r,d))-β=TrTr′Fdβ-β=(TrTr′Fd-I)β,Cov(β^rd(r,d))=σ2TrTr′FdS-1FdTrTr′.
Then, we can obtain the MSEM of the r-d class estimator as follows:
(26)MSEM(β^rd(r,d))=σ2TrTr′FdS-1FdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′.
Now we consider the following difference:
(27)Δ3=MSEM(β^rd(r,d))-MSEM(β^SRrd(r,d))=σ2TrTr′FdS-1FdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′-{(TrTr′Fd-I)′σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′}=σ2TrTr′Fd(S-1-A)FdTrTr′.Theorem 6.
The stochastic restrictedr-d class estimator β^SRrd(r,d) always dominates the r-d class estimator β^rd(r,d) in the MSEM criterion.Proof.
By Lemma1, we can obtain A=(S+R′W-1R)-1=S-1-S-1R′(W+RS-1R′)-1RS-1; then we obtain S-1-A=S-1R′(W+RS-1R′)-1RS-1≥0. Therefore, from (27), we may conclude that Δ3≥0; that is to say, the stochastic restricted r-d class estimator β^SRrd(r,d) always dominates the r-d class estimator β^rd(r,d) in the MSEM criterion.


## 3.4. MSEM Comparisons of the Stochastic Restricted Liu Estimator (SRLE) and the Stochastic Restrictedr-d (SRrd) Class Estimator

In this subsection, we consider the MSEM comparison between the stochastic restricted Liu estimator (SRLE) and the stochastic restrictedr-d (SRrd) class estimator.Firstly, from (10), we can compute the bias vector and the variance of stochastic restricted Liu estimator (SRLE) as follows:
(28)Bias(β^SRLE(d))=E(β^SRLE(d))-β=(Fd-I)β,Cov(β^SRLE(d))=σ2FdAFd.Therefore, we can obtain the MSEM of the stochastic restricted Liu estimator (SRLE) as follows:(29)MSEM(β^SRLE(d))=σ2FdAFd+(Fd-I)ββ′(Fd-I).Now we consider the following difference:(30)Δ4=MSEM(β^SRLE(d))-MSEM(β^SRrd(r,d))=σ2FdAFd+(Fd-I)ββ′(Fd-I)-{(TrTr′Fd-I)′σ2TrTr′FdAFdTrTr′+(TrTr′Fd-I)ββ′(TrTr′Fd-I)′}=σ2H+b3b3′-b4b4′,
where H=FdAFd-TrTr′FdAFdTrTr′, b3=(Fd-I)β and b4=(TrTr′Fd-I)β.Theorem 7.
When0<d<1 and λ1{(TrTr′FdAFdTrTr′)(FdAFd)-1}≤1, then the stochastic restricted r-d class estimator β^SRrd(r,d) is superior to the stochastic restricted Liu estimator β^SRLE(d) in the MSEM criterion if and only if b4′(σ2H+b3b3′)+b4≤1, b4∈ℜ(σ2H+b3b3′).Proof.
For0<d<1, it is easy to see that Fd=(S+I)-1(S+dI)>0 and A=(S+R′W-1R)-1>0. So we can conclude that FdAFd>0.
On the other hand,TrTr′FdAFdTrTr′ can be rewritten as
(31)TrTr′FdAFdTrTr′=T(Ir000)T′FdAFd(Ir000)T′.
From (31), it is easy to see that TrTr′FdAFdTrTr′≥0. Thus, by Lemma 2, we can obtain that H≥0 when λ1{(TrTr′FdAFdTrTr′)(FdAFd)-1}≤1. So from (30) and applying Lemma 3, we have Δ4 if and only if b4′(σ2H+b3b3′)+b4≤1, b4∈ℜ(σ2H+b3b3′). This theorem is proved.Remark. For practical use, we may replace the unknown parameters in the theorems with their appropriate estimators.


## 4. Numerical Example

In order to illustrate our theoretical results, we now consider in this section the data set on Total National Research and Development Expenditure as a Percent of Gross National Product originally due to Gruber [18] and later considered by Akdeniz and Erol [19]. In this paper, we use the same data, which is presented in Table 1.Table 1
Total Research and Development Expenditures as a percent of Gross National Product by Country: 1972–1986.
Year
Y
X
1
X
2
X
3
X
4
1972
2.3
1.9
2.2
1.9
3.7
1975
2.2
1.8
2.2
2.0
3.8
1979
2.2
1.8
2.4
2.1
3.6
1980
2.3
1.8
2.4
2.2
3.8
1981
2.4
2.0
2.5
2.3
3.8
1982
2.5
2.1
2.6
2.4
3.7
1983
2.6
2.1
2.6
2.6
3.8
1984
2.6
2.2
2.6
2.6
4.0
1985
2.7
2.3
2.8
2.8
3.7
1986
2.7
2.3
2.7
2.8
3.8Firstly, we obtain the ordinary least squares estimator ofβ:
(32)β^OLS=S-1X′y=(0.6455,0.0896,0.1436,0.1526)′
with MSE(β^OLS)=0.0808 and σ^OLS2=0.0015. Consider the following stochastic linear restrictions:
(33)r=Rβ+e,R=(1,-2,-2,-2),e~(0,σ^OLS2).For ther-k class estimator (rk), the stochastic restricted ridge estimator (SRRE), and the stochastic restricted r-k class estimator (SRrk), their estimated mean squared error values are given in Table 2. For the r-d class estimator (rd), the stochastic restricted Liu estimator (SRLE) and the stochastic restricted r-d class estimator (SRrd), their estimated mean squared error values are given in Table 3. Their estimated mean squared error values are got by replacing in the corresponding theoretical MSE expressions all unknown model parameters by their OLS estimator.Table 2
Estimated MSE values ofrk, SRRE, and SRrk.
k
=
0.000
k
=
0.001
k
=
0.003
k
=
0.005
k
=
0.007
k
=
0.1
k
=
0.2
SRRE
0.0445
0.0423
0.0391
0.0372
0.0363
0.1214
0.1653
r
k
0.0621
0.0607
0.0588
0.0587
0.0574
0.1335
0.1726
SRrk
0.0331
0.0330
0.0334
0.0344
0.0358
0.1308
0.1716Table 3
Estimated MSE values ofrd, SRLE, and SRrd.
d
=
0.60
d
=
0.70
d
=
0.80
d
=
0.85
d
=
0.9
d
=
0.95
d
=
1.0
SRLE
0.0540
0.0435
0.0384
0.0379
0.0388
0.0409
0.0445
r
d
0.0732
0.0633
0.0582
0.0574
0.0577
0.0593
0.0621
SRrd
0.0622
0.0486
0.0392
0.0362
0.0340
0.0331
0.0331From Table2, we can see that the stochastic restricted r-k class estimator is always better than the r-k class estimator. As fact, the stochastic restricted r-k class estimator has more information about the unknown parameter, so this estimator is better than the r-k class estimator. When k is small, then the stochastic restricted r-k class estimator is superior over the stochastic restricted ridge estimator. However, when k becomes big, then the stochastic restricted ridge estimator is superior over the stochastic restricted r-k class estimator.From Table3, we can find that stochastic restricted r-d class estimator is always better than the r-d class estimator. When d is big, then the stochastic restricted r-d class estimator is superior over the stochastic restricted Liu estimator. However, when d becomes smaller, then the stochastic restricted Liu estimator is superior over the stochastic restricted r-d class estimator.


## 5. Conclusion

In this paper, the stochastic restrictedr-k class estimator (SRrk) and stochastic restricted r-d class estimator (SRrd) are proposed and some properties of the two estimators have also been discussed. In particular, we show that the proposed SRrk and SRrd are proved to have smaller mean squared error than the rk, SRRE, rd, and SRLE, respectively.



---

*Source: 173836-2014-01-02.xml*