# On the Performance of Principal Component Liu-Type Estimator under the Mean Square Error Criterion

Authors: Jibo Wu
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

Wu (2013) proposed an estimator, principal component Liu-type estimator, to overcome multicollinearity. This estimator is a general estimator which includes ordinary least squares estimator, principal component regression estimator, ridge estimator, Liu estimator, Liu-type estimator,r-k class estimator, and r-d class estimator. In this paper, firstly we use a new method to propose the principal component Liu-type estimator; then we study the superior of the new estimator by using the scalar mean squares error criterion. Finally, we give a numerical example to show the theoretical results.


---

## Body

## 1. Introduction

Consider the multiple linear regression model(1)y=Xβ+ϵ,
where y is an n×1 vector of observation, X is an n×p known matrix of rank p, β is a p×1 vector of unknown parameters, and ϵ is an n×1 vector of disturbances with expectation E(ϵ)=0 and variance-covariance matrix Cov(ϵ)=σ2In.According to the Gauss-Markov theorem, the classical ordinary least squares estimator (OLSE) is obtained as follows:(2)β^=(X′X)-1X′y.
The OLSE has been regarded as the best estimator for a long time. However, when multicollinearity is present and the matrix X′X is ill-conditioned, the OLSE is no longer a good estimator. To improve OLSE, many ways have been proposed. One way is to consider biased estimator, such as, principal component regression estimator [1], ridge estimator [2], Liu estimator [3], Liu-type estimator [4], two-parameter ridge estimator [5], r-k class estimator [6], r-d class estimator [7], and modified r-k class estimator [8].An alternative method to overcome the multicollinearity is to consider the restrictions. Xu and Yang [9] introduced a stochastic restricted Liu estimator; Li and Yang [10] introduced a stochastic restricted ridge estimator.To overcome multicollinearity, Hoerl and Kennard [2] solve the following problem:
(3)minβ{(y-Xβ)′(y-Xβ)+k(ββ′-c)},
where k is a Lagrangian multiplier and c is a constant, and obtain the ridge estimator (RE):
(4)β^(k)=(X′X+kI)-1X′y,k>0.Liu [3] introduced the Liu estimator (LE):
(5)β^(d)=(X′X+I)-1(X′y+dβ^),0<d<1,
where β^ is OLSE. This estimator can be obtained by solving the following problem:
(6)minβ{(y-Xβ)′(y-Xβ)+(β-dβ^)(β-dβ^)′}.
This estimator can also be obtained by the following ways. Suppose that β satisfied dβ^=β+e. Then, we use the mixed method [11]; we can obtain the Liu estimator.Recently, Huang et al. [4] introduced a Liu-type estimator which includes the OLSE, RE, and LE, defined as follows:
(7)β^(k,d)=(X′X+kI)-1(X′y+dβ^),k>0,0<d<1,
where β^ is OLSE. This estimator can be obtained by solving the following problem:
(8)minβ{(k1/2β-dβ^k1/2)(k1/2β-dβ^k1/2)′(y-Xβ)′(y-Xβ)+(k1/2β-dβ^k1/2)(k1/2β-dβ^k1/2)′}.Let us consider the following transformation for the model (1):
(9)y=XTT′β+ϵ=Zα+ϵ,
where
(10)X′X=(Tr,Tp-r)(Λr00Λp-r)(Tr′Tp-r′).Λr and Λp-r are diagonal matrices such that that the main diagonal elements of the r×r matrix Λr are the r largest eigenvalues of X′X, while Λp-r are the remaining p-r eigenvalues. The p×p matrix T=(Tr,Tp-r) is orthogonal with Tr=(t1,t2,…,tr) consisting of its first r columns and Tp-r=(tr+1,tr+2,…,tp) consisting of the remaining p-r columns of the matrix T. The PCRE for β can be written as
(11)β^r=Tr(Tr′X′XTr)-1Tr′X′y.Baye and Parker [6] introduced the application of ridge approach to improve the PCR estimator, namely, r-k class estimator as
(12)β^r(k)=Tr(Tr′X′XTr+kIr)-1Tr′X′y,k>0.Alternatively, Kaçıranlar and Sakallıoğlu [7] introduced the r-d class estimator which is the combination of the LE and the PCRE, which is defined as follows:
(13)β^r(d)=Tr(Tr′X′XTr+Ir)-1(Tr′X′y+dTr′β^r),aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa0<d<1.Wu [12] proposed the principal component Liu-type estimator (PCTTE), which is defined as
(14)β^r(k,d)=Tr(Tr′X′XTr+kIr)-1(Tr′X′y+dTr′β^r),aaaaaaaaaaaaaaaaaaaaaaaaaaaaaak>0,0<d<1.In this paper, firstly we use a new method to propose the principal component Liu-type estimator. Then, we show that, under certain conditions, the PCTTE is superior to the related estimator in the mean square error criterion. Finally, we give a numerical example to illustrate the theoretical results.


## 2. The Principal Component Liu-Type Estimator

Using the symbols in (9) and (10), (1) can be written as follows:
(15)y=XTrTr′β+XTp-rTp-r′β+ϵ=Zrαr+Zp-rαp-r+ϵ.
The PCRE can be obtained by omitted Zp-rαp-r, and the model (15) reduced to:
(16)y=Zrαr+ϵ.
Then, solve the following problem:
(17)(y-Zrαr)′(y-Zrαr),
we obtain
(18)α^r=(Zr′Zr)-1Zr′y.
Then, transforming α^r to the original parameter space, we can get the PCRE of parameter β.Now, we give a method to obtain the principal component Liu-type estimator. Letc be a constant and k a Lagrangian multiplier, minimizing
(19)(y-Zrαr)′(y-Zrαr)+1k[(kαr-dα^r)(kαr-dα^r)′-c],
where α^r=(Zr′Zr)-1Zr′y. Then we get
(20)α^r(k,d)=(Zr′Zr+kIr)-1(Zr′y+dα^r).
After transforming back to original parameters pace, we obtain
(21)β^r(k,d)=Tr(Tr′X′XTr+kIr)-1×(Tr′X′y+dTr′β^r),k>0,0<d<1.  This estimator can also be got by minimizing the function
(22)(kαr-dα^r)(kαr-dα^r)′+k[(y-Zrαr)′(y-Zrαr)-c].
It is easy to see that the new estimator β^r(k,d) has the following properties:(1)
β^r(1,1)=β^r=Tr(Tr′X′XTr)-1Tr′X′y is the PCRE;(2)
β^p(1,1)=β^OLS=(X′X)-1X′y is the OLSE;(3)
β^r(1,d)=β^r(d)=β^r(d)=Tr(Tr′X′XTr+Ir)-1(Tr′X′y+dTr′β^r) is the r-d class estimator;(4)
β^p(1,d)=β^(d)=(X′X+I)-1(X′y+dβ^OLS) is the LE;(5)
β^r(k,0)=β^r(k)=Tr(Tr′X′XTr+kIr)-1Tr′X′y is the r-k class estimator;(6)
β^p(k,0)=β^(k)=(X′X+kI)-1X′y is the RE;(7)
β^p(k,d)=β^(k,d)=(X′X+kI)-1(X′y+kdβ^OLS) is the LTE.


## 3. Superiority of the Principal Component Liu-Type Estimator over Some Estimators in the Mean Square Error Criterion

The mean square error (MSE) of an estimatorβ^ is defined as
(23)M(β^)=E[(β^-β)′(β^-β)]=trV(β^)+[Bias(β^)]′[Bias(β^)],
where V(β^) is the dispersion matrix and Bias(β^) is the bias vector. For two given estimators β^1 and β^2, the estimator β^2 is said to be superior to β^1 in the MSE criterion, if and only if
(24)Δ(β^1,β^2)=M(β^1)-M(β^2)≥0.


### 3.1.β^r(k,d) versus β^r

Firstly, we compute that(25)V(β^r(k,d))=σ2TrSr(k)-1Sr(d)Λr-1Sr(d)Sr(k)-1Tr′,E(β^r(k,d))=TrSr(k)-1Sr(d)Tr′β,
where Sr(k)=Λr+kIr. Then, the mean square error (MSE) of β^r(k,d) is given as follows:
(26)M(β^r(k,d))=trV(β^r(k,d))+[Bias(β^r(k,d))]′[Bias(β^r(k,d))]=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2.Letk=d in (26); we obtain the MSE of β^r as follows:
(27)M(β^r)=∑i=1r‍σ2λi+∑i=r+1p‍αi2.Now we consider the following difference:(28)M(β^r(k,d))-M(β^r)=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-{∑i=1r‍σ2λi+∑i=r+1p‍αi2}=∑i=1r‍(d-k)(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2.
If σ2-αi2λi>0, then when 0<d<k,  M(β^r(k,d))-M(β^r)<0. If σ2-αi2λi<0, then when 0<d<k<min{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)} and max{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)}<k<d<1,  M(β^r(k,d))-M(β^r)<0. So we have the following theorem.Theorem 1.
The estimatorβ^r(k,d) is superior to the estimator for β^r under the mean square error criterion for:(a)
0<d<k if σ2-αi2λi>0 for all i=1,…,r,(b)
0<d<k<min{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)} and max{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)}<k<d<1, i=1,…,r if σ2-αi2λi<0 for all i=1,…,r.


### 3.2.β^r(k,d) versus β^r(k)

From the definition of theβ^r(k), we know that let d=0 in β^r(k,d), and we obtain the β^r(k).Theorem 2.
Letkαi2-σ2>0 for all i=1,…,r. Then, there exists a strictly positive d such that β^r(k,d) is superior to β^r(k) in the mean square error criterion for 0<d<(∑i=1r‍((kαi2-σ2)/(λi+k)2))/(∑i=1r‍((σ2+αi2λi)/(λi(λi+k)2)))<1.Proof.
We know thatM(β^r(k))=Mβ^r(k,0), so that by continuity it is sufficient to show that M(β^r(k,d)) decreasing in the neighborhood of d=0.
Performing the calculus for fixedk, we can see that
(29)∂M(β^r(k,d))∂d=2∑i=1r‍σ2(λi+d)-(k-d)αi2λiλi(λi+k)2.
So when 0<d<(∑i=1r‍((kαi2-σ2)/(λi+k)2))/(∑i=1r‍((σ2+αi2λi)/(λi(λi+k)2)))<1, (∂M(β^r(k,d)))/(∂d)<0; that is to say, M(β^r(k,d))<M(β^r(k,0))=M(β^r(k)). The proof of Theorem 2 is completed.


### 3.3.β^r(k,d) versus β^r(d)

From the definition of theβ^r(d), we know that let k=1 in β^r(k,d), and we obtain that β^r(d),
(30)M(β^r(d))=∑i=1r‍σ2(λi+d)2+(1-d)2αi2λiλi(λi+1)2+∑i=r+1p‍αi2.Now, we discuss the following difference:(31)Δ=M(β^r(k,d))-M(β^r(k))=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-{∑i=1r‍σ2(λi+d)2+(1-d)2αi2λiλi(λi+1)2+∑i=r+1p‍αi2}=∑i=1r‍σ2(1-k)(λi+d)(λi+d)(2λi+k+1)λi(λi+1)2(λi+k)2-∑i=1r‍(×(λi(λi+1)2(λi+k)2)-1(σ2(1-k)(λi+d)αi2λi×[(k+1-2d)λi+2k-d-dk]σ2(1-k)(λi+d)αi2λi)×(λi(λi+1)2(λi+k)2)-1(σ2(1-k)(λi+d)αi2λi).
Then by (31), if we want Δ<0, then
(32)d<d1=∑i=1r‍(2λi+k+1)σ2+2αi2λi2+(1+k)αi2λiλi(λi+1)2(λi+k)2×(∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2)-1
when k<1, it is easy to know that d1 is always less than 1, and d1 is to be bigger than 0, if
(33)∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2>0.
Then, we get
(34)k>(2λi+1)σ2-αi2λiαi2(λi+2)-σ2
for αi2(λi+2)-σ2>0 with all i=1,…,r.Thus, we obtain the following theorem.Theorem 3.
Ifαi2(λi+2)-σ2>0 and k>{max(((2λi+1)σ2-αi2λi)/(αi2(λi+2)-σ2)),0} for all i=1,…,r, then the β^r(k,d) is better than the β^r(d) in the mean square error sense for k<1 and
(35)d<∑i=1r‍(2λi+k+1)σ2+2αi2λi2+(1+k)αi2λiλi(λi+1)2(λi+k)2×(∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2)-1.


### 3.4.β^r(k,d) versus β^

In this subsection, we will give the comparison of theβ^r(k,d) and β^ under the mean square error criterion.Letk=d and r=p in (26); we obtain the MSE of β^r as follows:
(36)M(β^)=∑i=1p‍σ2λi.In order to compare theβ^r(k,d) and β^, now we consider the following difference:
(37)M(β^r(k,d))-M(β^)=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-∑i=1p‍σ2λi=∑i=1r‍(d-k)×(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2+∑i=r+1p‍αi2λi-σ2λi
when k>d>d2=(∑r=1r‍(((αi2λi-σ2)k-2σ2λi)/(λi(λi+k)2)))/(∑r=1r‍((αi2λi+σ2)/(λi(λi+k)2))), then
(38)∑i=1r‍(d-k)(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2<0.
Since the lower bound of d is less than 1, the lower bound of d may be less than 0. If the lower bound of of d is less than 0, then we can choose any d in [0,1]. Thus, we can get the following theorem.Theorem 4.
(1)
Ifαi2λi-σ2<0  for all i=1,2,…,p, then β^r(k,d) is superior to β^ in the mean square error sense for any k>0 and 0<d<1.
(2)
Ifαi2λi-σ2<0 for some i=1,2,…,r, then β^r(k,d) is superior to β^ in the mean square error sense for k>d>(∑r=1r‍(((αi2λi-σ2)k-2σ2λi)/(λi(λi+k)2)))/(∑r=1r‍((αi2λi+σ2)/(λi(λi+k)2))) when ∑i=r+1p‍((αi2λi-σ2)/λi)<0.


### 3.5.β^r(k,d) versus β^(k,d)

In this subsection, we will compareβ^r(k,d) with β^(k,d) in the mean square error sense.Defineυi=(λi+d)/(λi+k), then the mean square error of β^r(k,d) can be written as
(39)M(β^r(k,d))=∑1=1r‍[σ2υi2λi+(υi-1)2αi2]+∑i=r+1p‍αi2.
Firstly, we discuss the difference between M(β^r(k,d)) and M(β^r-1(k,d)):
(40)M(β^r(k,d))-M(β^r-1(k,d))=υr[υr(σ2λr+αr2)-2αr2]=υrλi+k[d(σ2λr+αr2)+σ2-αr2λr-2kαr2].
So when σ2<αr2λr+2kαr2, if 1>d>((αr2λr+2kαr2-σ2)/((σ2/λr)+αr2))>0, then M(β^r(k,d))-M(β^r-1(k,d))>0.If0<d<((αr2λr+2kαr2-σ2)/((σ2/λr)+αr2))<1, then M(β^r(k,d))-M(β^r-1(k,d))<0.Then we have the following theorem.Theorem 5.
(1)
Ifσ2<αi2λi+2kαi2 for some i∈Np, then there exists a nonnegative d3 such that M(β^(k,d))>M(β^r(k,d)) for 0<d3<d<1, where Np=r+1,…,p.
(2)
Ifσ2<αi2λi+2kαi2 for all i∈Np, then there exists a nonnegative d3 such that M(β^(k,d))<M(β^r(k,d)) for 0<d<d4<1, where Np=r+1,…,p, where
(41)d3=mini∈Np{1,maxi∈Npαi2λi+2kαi2-σ2(σ2/λi)+αi2},d4=mini∈Npαi2λi+2kαi2-σ2(σ2/λi)+αi2.


## 3.1.β^r(k,d) versus β^r

Firstly, we compute that(25)V(β^r(k,d))=σ2TrSr(k)-1Sr(d)Λr-1Sr(d)Sr(k)-1Tr′,E(β^r(k,d))=TrSr(k)-1Sr(d)Tr′β,
where Sr(k)=Λr+kIr. Then, the mean square error (MSE) of β^r(k,d) is given as follows:
(26)M(β^r(k,d))=trV(β^r(k,d))+[Bias(β^r(k,d))]′[Bias(β^r(k,d))]=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2.Letk=d in (26); we obtain the MSE of β^r as follows:
(27)M(β^r)=∑i=1r‍σ2λi+∑i=r+1p‍αi2.Now we consider the following difference:(28)M(β^r(k,d))-M(β^r)=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-{∑i=1r‍σ2λi+∑i=r+1p‍αi2}=∑i=1r‍(d-k)(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2.
If σ2-αi2λi>0, then when 0<d<k,  M(β^r(k,d))-M(β^r)<0. If σ2-αi2λi<0, then when 0<d<k<min{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)} and max{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)}<k<d<1,  M(β^r(k,d))-M(β^r)<0. So we have the following theorem.Theorem 1.
The estimatorβ^r(k,d) is superior to the estimator for β^r under the mean square error criterion for:(a)
0<d<k if σ2-αi2λi>0 for all i=1,…,r,(b)
0<d<k<min{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)} and max{((αi2λi+σ2)d+2σ2λi)/(αi2λi-σ2)}<k<d<1, i=1,…,r if σ2-αi2λi<0 for all i=1,…,r.


## 3.2.β^r(k,d) versus β^r(k)

From the definition of theβ^r(k), we know that let d=0 in β^r(k,d), and we obtain the β^r(k).Theorem 2.
Letkαi2-σ2>0 for all i=1,…,r. Then, there exists a strictly positive d such that β^r(k,d) is superior to β^r(k) in the mean square error criterion for 0<d<(∑i=1r‍((kαi2-σ2)/(λi+k)2))/(∑i=1r‍((σ2+αi2λi)/(λi(λi+k)2)))<1.Proof.
We know thatM(β^r(k))=Mβ^r(k,0), so that by continuity it is sufficient to show that M(β^r(k,d)) decreasing in the neighborhood of d=0.
Performing the calculus for fixedk, we can see that
(29)∂M(β^r(k,d))∂d=2∑i=1r‍σ2(λi+d)-(k-d)αi2λiλi(λi+k)2.
So when 0<d<(∑i=1r‍((kαi2-σ2)/(λi+k)2))/(∑i=1r‍((σ2+αi2λi)/(λi(λi+k)2)))<1, (∂M(β^r(k,d)))/(∂d)<0; that is to say, M(β^r(k,d))<M(β^r(k,0))=M(β^r(k)). The proof of Theorem 2 is completed.


## 3.3.β^r(k,d) versus β^r(d)

From the definition of theβ^r(d), we know that let k=1 in β^r(k,d), and we obtain that β^r(d),
(30)M(β^r(d))=∑i=1r‍σ2(λi+d)2+(1-d)2αi2λiλi(λi+1)2+∑i=r+1p‍αi2.Now, we discuss the following difference:(31)Δ=M(β^r(k,d))-M(β^r(k))=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-{∑i=1r‍σ2(λi+d)2+(1-d)2αi2λiλi(λi+1)2+∑i=r+1p‍αi2}=∑i=1r‍σ2(1-k)(λi+d)(λi+d)(2λi+k+1)λi(λi+1)2(λi+k)2-∑i=1r‍(×(λi(λi+1)2(λi+k)2)-1(σ2(1-k)(λi+d)αi2λi×[(k+1-2d)λi+2k-d-dk]σ2(1-k)(λi+d)αi2λi)×(λi(λi+1)2(λi+k)2)-1(σ2(1-k)(λi+d)αi2λi).
Then by (31), if we want Δ<0, then
(32)d<d1=∑i=1r‍(2λi+k+1)σ2+2αi2λi2+(1+k)αi2λiλi(λi+1)2(λi+k)2×(∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2)-1
when k<1, it is easy to know that d1 is always less than 1, and d1 is to be bigger than 0, if
(33)∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2>0.
Then, we get
(34)k>(2λi+1)σ2-αi2λiαi2(λi+2)-σ2
for αi2(λi+2)-σ2>0 with all i=1,…,r.Thus, we obtain the following theorem.Theorem 3.
Ifαi2(λi+2)-σ2>0 and k>{max(((2λi+1)σ2-αi2λi)/(αi2(λi+2)-σ2)),0} for all i=1,…,r, then the β^r(k,d) is better than the β^r(d) in the mean square error sense for k<1 and
(35)d<∑i=1r‍(2λi+k+1)σ2+2αi2λi2+(1+k)αi2λiλi(λi+1)2(λi+k)2×(∑i=1r‍λiσ2(2λi+k+1)-αi2λi2(k+1)-2kαi2λiλi(λi+1)2(λi+k)2)-1.


## 3.4.β^r(k,d) versus β^

In this subsection, we will give the comparison of theβ^r(k,d) and β^ under the mean square error criterion.Letk=d and r=p in (26); we obtain the MSE of β^r as follows:
(36)M(β^)=∑i=1p‍σ2λi.In order to compare theβ^r(k,d) and β^, now we consider the following difference:
(37)M(β^r(k,d))-M(β^)=∑i=1r‍σ2(λi+d)2+(k-d)2αi2λiλi(λi+k)2+∑i=r+1p‍αi2-∑i=1p‍σ2λi=∑i=1r‍(d-k)×(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2+∑i=r+1p‍αi2λi-σ2λi
when k>d>d2=(∑r=1r‍(((αi2λi-σ2)k-2σ2λi)/(λi(λi+k)2)))/(∑r=1r‍((αi2λi+σ2)/(λi(λi+k)2))), then
(38)∑i=1r‍(d-k)(αi2λi+σ2)d+2σ2λi+(σ2-αi2λi)kλi(λi+k)2<0.
Since the lower bound of d is less than 1, the lower bound of d may be less than 0. If the lower bound of of d is less than 0, then we can choose any d in [0,1]. Thus, we can get the following theorem.Theorem 4.
(1)
Ifαi2λi-σ2<0  for all i=1,2,…,p, then β^r(k,d) is superior to β^ in the mean square error sense for any k>0 and 0<d<1.
(2)
Ifαi2λi-σ2<0 for some i=1,2,…,r, then β^r(k,d) is superior to β^ in the mean square error sense for k>d>(∑r=1r‍(((αi2λi-σ2)k-2σ2λi)/(λi(λi+k)2)))/(∑r=1r‍((αi2λi+σ2)/(λi(λi+k)2))) when ∑i=r+1p‍((αi2λi-σ2)/λi)<0.


## 3.5.β^r(k,d) versus β^(k,d)

In this subsection, we will compareβ^r(k,d) with β^(k,d) in the mean square error sense.Defineυi=(λi+d)/(λi+k), then the mean square error of β^r(k,d) can be written as
(39)M(β^r(k,d))=∑1=1r‍[σ2υi2λi+(υi-1)2αi2]+∑i=r+1p‍αi2.
Firstly, we discuss the difference between M(β^r(k,d)) and M(β^r-1(k,d)):
(40)M(β^r(k,d))-M(β^r-1(k,d))=υr[υr(σ2λr+αr2)-2αr2]=υrλi+k[d(σ2λr+αr2)+σ2-αr2λr-2kαr2].
So when σ2<αr2λr+2kαr2, if 1>d>((αr2λr+2kαr2-σ2)/((σ2/λr)+αr2))>0, then M(β^r(k,d))-M(β^r-1(k,d))>0.If0<d<((αr2λr+2kαr2-σ2)/((σ2/λr)+αr2))<1, then M(β^r(k,d))-M(β^r-1(k,d))<0.Then we have the following theorem.Theorem 5.
(1)
Ifσ2<αi2λi+2kαi2 for some i∈Np, then there exists a nonnegative d3 such that M(β^(k,d))>M(β^r(k,d)) for 0<d3<d<1, where Np=r+1,…,p.
(2)
Ifσ2<αi2λi+2kαi2 for all i∈Np, then there exists a nonnegative d3 such that M(β^(k,d))<M(β^r(k,d)) for 0<d<d4<1, where Np=r+1,…,p, where
(41)d3=mini∈Np{1,maxi∈Npαi2λi+2kαi2-σ2(σ2/λi)+αi2},d4=mini∈Npαi2λi+2kαi2-σ2(σ2/λi)+αi2.


## 4. Numerical Example

To illustrate our theoretical results, firstly we use a numerical example to investigate the estimators studied in the dataset originally due to Gruber [13], and later considered by Akdeniz and Erol [14]. We assemble the data as follows:
(42)X=(1.92.21.93.71.82.22.03.81.82.42.13.61.82.42.23.82.02.52.33.82.12.62.43.72.12.62.63.82.22.62.64.02.32.82.83.72.32.72.83.8),y=(2.32.22.22.32.42.52.62.62.72.7).
Firstly, we can compute the OLSE of β as follows:
(43)β^=(0.6921,0.6258,-0.1154,0.2866,0.0256)′.
For the OLSE, PCRE, r-k class estimator, r-d class estimator, Liu-type estimator (LTE), and new estimator (PCTTE), their estimated mean square error (MSE) values are obtained by replacing all unknown model parameters by their, respectively, least squares estimators in corresponding expressions.Firstly, we see the comparison of the OLSE and the PCTTE. Whend=0.6 is fixed, then if the values of k is big then the new estimator has smaller MSE values than OLSE; that is to say, the new estimator is better than the OLSE. So we see that the new estimator improved the OLSE.From Figure3, we see that when d, fixed, if 0<d<k, then the new estimator is better than the PCRE. In Theorem 1, we see that if 0<d<k, the new estimator is better. For the numerical example, when 0<d<k, the new estimator is more efficient than the PCRE.By Figures1, 2, 3, 4, 5, 6, 7, and 8, we see that when 0<d<k, then new estimator is better than the LTE. So in practice, we can choose bigger k and smaller d.Figure 1
The MSE values of OLSE and PCTTE whend=0.6.Figure 2
The MSE values of OLSE and PCTTE whenk=0.56.Figure 3
The MSE values of PCRE and PCTTE whend=0.6.Figure 4
The MSE values of PCRE and PCTTE whenk=0.56.Figure 5
The MSE values ofr-k class estimator and PCTTE when d=0.1.Figure 6
The MSE values ofr-d class estimator and PCTTE when k=0.9.Figure 7
The MSE values of LTE and PCTTE whenk=0.8.Figure 8
The MSE values of LTE and PCTTE whend=0.3.


## 5. Conclusion

In this paper, we use a new method to propose the principal component Liu-type estimator. Then, we discuss the superiority of the new estimator with the OLSE, PCRE,r-k class estimator, r-d class estimator, and Liu-type estimator in the sense of mean square error. We also give a method to choose the biasing parameters. Finally, we give a numerical example to illustrate the performance the various estimators.



---

*Source: 858794-2013-12-04.xml*