# Improved Liu-Type Estimator of Parameters in

Authors: Jibo Wu
Journal: ISRN Applied Mathematics
 (2014)

---

## Abstract

We consider the parameter estimation in two seemingly unrelated regression systems. To overcome the multicollinearity, we propose a Liu-type estimator in seemingly unrelated regression systems. The superiority of the new estimator over the classic estimator in the mean square error is discussed and we also discuss the admissibility of the Liu-type estimator.


---

## Body

## 1. Introduction

Consider the following two seemingly unrelated regressions (SUR):(1)Y1=X1β1+ε1,Y2=X2β2+ε2,
where Yi(i=1,2) are n×1 vectors of observations, Xi(i=1,2) are n×p matrices with full column rank, βi(i=1,2) are p×1 vectors of unknown regression parameters, εi(i=1,2) are n×1 vectors of error variables, and
(2)E(εi)=0,Cov(εi,εj)=σijIn,i,j=1,2,
where V=(σij) is a 2×2 positive definite matrix. This system has been used in many fields, such as social biological sciences and econometrics. Zellner [1, 2] firstly defined this system and later it was discussed by many researchers, such as Wang et al. [3], Roozbeh et al. [4], and Singh et al. [5].For system (1), when σ12=0, we can get the estimator of β as follows:
(3)β^=(β^1β^2)=((X1′X1)-1X1′Y1(X2′X2)-1X2′Y2).
When the error vector is related, the estimator β^ may not fully use the information of the parameter. How to use the information to improve the β^ is a problem. Revankar [6] and Srivastava and Giles [7] have discussed this problem. Wang [8] uses the covariance-improve method to estimate the parameter in two seemingly unrelated regressions. With the prior information, Wang [8] proposes the covariance-improve estimator of β1 and β2:
(4)β~1=(X1′X1)-1X1′Y1-σ12σ22(X1′X1)-1X1′N2Y2,β~2=(X2′X2)-1X2′Y2-σ12σ11(X2′X2)-1X2′N1Y1,
where Nj=I-Pj, Pj=Xj(XjXj′)-1Xj′, j=1,2.When the design matrixXi is ill-conditioned, the estimator β~i, i=1,2 is no longer a good estimator. Many researchers have discussed this problem. One method to overcome this problem is to consider biased estimator, such as Liu and Wang [9] introduce the ridge estimator, Liu [10] proposes the principal regression component estimator, Qiu [11] proposes the classic c-k type estimator, and Roozbeh et al. [12] introduce the ridge estimator.In this paper, we propose a Liu-type estimator to overcome the multicollinearity in the two seemingly unrelated regressions. Then we discuss the superiority of the Liu-type estimator over the covariance-improve estimator in the mean square error criterion, and we also discuss the selection of the parameter in the proposed estimator. Since the estimator ofβ2 is similar to the estimator of β1, so in this paper, we only discuss the estimator of β1.The rest of the paper is organized as follows. The Liu-type estimator is given in Section2 and the properties of the new estimator under the mean square error criterion are discussed in Section 3. The admissibility of the proposed estimator is studied in Section 4 and some conclusion remarks are given in Section 5.


## 2. The Proposed Estimator

In this section, we use the prior information to obtain the Liu-type estimator. Firstly, we give a lemma.Lemma 1 (see [10]).
Letθ~ be an estimator of θ, θ^ the prior information with E(θ^)=0, and
(5)Cov(θ~θ^)=(V11V12V12V22).
Then in the estimatorA={θ*(X)=θ~+Xθ^}, the estimator θ*=θ~-V12V22-1θ^ has minimum mean square error and
(6)MSE(θ*)=MSE(θ~)-tr⁡(V12V22-1V21).Now we establish the Liu-type estimator in SUR model. SetZ2 satisfy X2′Z2=0 and X2 has the max column full rank; then E(Z2′Y2)=0 and(7)Cov(β^1(k,d)Z2′Y2)=(σ11(X1′X1+kI)-2(X1′X1+dI)2(X1′X1)-1σ12(X1′X1+kI)-1(X′X+dI)(X1′X1)-1X1′Z2σ12Z2′X1(X1′X1)-1(X1′X1+kI)-1(X′X+dI)σ22Z2′Z2).By Lemma1, when V is known, we establish a new estimator for β1:
(8)β~1(k,d)=β^1(k,d)-σ12σ22(X1′X1+kI)-1×(X1′X1+dI)(X1′X1)-1X1′Z2(Z2′Z2)-1Z2′Y2=(X1′X1+kI)-1(X1′X1+dI)(X1′X1)-1X1′Y1-σ12σ22(X1′X1+kI)-1(X1′X1+dI)×(X1′X1)-1X1′N2Y2,
where β^1(k,d)=(X1′X1+kI)-1(X′X+dI)(X1′X1)-1X1′Y1 is the estimator for the single model in (1), k>0, 0<d<1, d<k, N2=I-P2, and P2=X2(X2′X2)-1X2. We call this estimator as Liu-type estimator in SUR model. By the definition of the Liu-type estimator, it is easy to know that it is a general estimator which includes the covariance-improve estimator proposed by Wang [8] and the ridge estimator proposed by Liu [10] as special cases.(1) Whenk=d=0, then
(9)β~1(k,d)=β~1=(X1′X1)-1X1′Y1-σ12σ22(X1′X1)-1X1′N2Y2.(2) Whend=1, then
(10)β~1(k,d)=β~1(k)=(X1′X1+kI)-1X1′Y1-σ12σ22(X1′X1+kI)-1X1′N2Y2.In next section, we will give the properties of the new estimator.


## 3. MSE: The Superiority of the New Estimator over the Covariance-Improve Estimator

In this section, we discuss the statistical properties of the new estimatorβ~(k,d). Firstly, we give the definition of the mean square error (MSE).Letb* be an estimator of β; the MSE of b* is defined as
(11)MSE(b*,β)=E[(b*-β)′(b*-β)]=tr⁡(Cov(b*))+(bias(b*))′(bias(b*)),
where bias(b*)=E(b*)-β.In the following theorem, we give the superiority of the new estimator over the covariance-improve estimator in the mean square error.Theorem 2.
For the SUR model (1), when V is known,(a)
when0<d<1 fixed,
(1)
ifσ11(1/λi-ρ122(aii/λi2))-αi2≥0, then MSE(β~1)≥MSE(β~1(k,d));(2)
ifσ11(1/λi-ρ122(aii/λi2))-αi2<0, when 0<k<(σ11(2λi+d)(1/λi+ρ122(aii/λi2))+dαi2)/(αi2-σ11(1/λi-ρ122(aii/λi2))), then MSE(β~1)≥MSE(β~1(k,d));(b)
whenk>0 fixed,
(1)
ifσ11(2λi+k)(1/λi+ρ122(aii/λi2))≥kαi2, then MSE(β~1)≥MSE(β~1(k,d));(2)
ifσ11(2λi+k)(1/λi+ρ122(aii/λi2))<kαi2, when (kαi2-σ11(2λi+k)(1/λi+ρ122(aii/λi2)))/(σ11(1/λi-ρ122(aii/λi2))+αi2)≤d<1, then MSE(β~1)≥MSE(β~1(k,d)).Proof.
By the definition of the covariance-improve estimatorβ~1, we obtain
(12)E(β~1)=β1,Cov(β~1)=σ11(X1′X1)-1-σ122σ22(X1′X1)-1X1′N2X1(X1′X1)-1.
Then by the definition of the mean square error, we obtain
(13)MSE(β~1)=tr⁡(Cov(β~1))=σ11tr⁡((X1′X1)-1)-σ122σ22tr⁡((X1′X1)-1X1′N2X1(X1′X1)-1).
Since X1′X1>0, there exists an orthogonal matrix such that Q′X1′X1Q=Λ=diag⁡(λ1,…,λp1), where λ1≥⋯≥λp1 is the ordered eigenvalues of X1′X1; thus we obtain
(14)MSE(β~1)=σ11∑i=1p1(1λi-ρ122aiiλi2),
where A=Q′X1′N2X1Q=(aij). Since Cov(β~1)≥0 so we get that 1/λi-ρ122(aii/λi2)≥0, i=1,…,p1.
Now we compute the MSE of the Liu-type estimator (β~1(k,d)) for SUR model (1). Firstly, we compute that
(15)E(β~1(k,d))=(X1′X1+kI)-1(X1′X1+dI)β1,Cov(β~1(k,d))=(X1′X1+kI)-1(X1′X1+dI)×Cov(β~1)(X1′X1+dI)(X1′X1+kI)-1.
Thus we have
(16)MSE(β~1(k,d))=tr⁡(Cov(β~1(k,d)))+bias(β~1(k,d))′bias(β~1(k,d))′=σ11∑i=1p1(λi+d)2(λi+k)2(1λi-ρ122aiiλi2)+(k-d)2∑i=1p1αi2(λi+k)2,
where α=Q′β1.
In order to compareβ~1(k,d) and β~1, we consider the following difference:
(17)MSE(β~1)-MSE(β~1(k,d))=σ11∑i=1p1(1λi-ρ122aiiλi2)-σ11∑i=1p1(λi+d)2(λi+k)2(1λi-ρ122aiiλi2)-(k-d)2∑i=1p1αi2(λi+k)2=∑i=1p1(k-d)(λi+k)2[(2λi+k+d)(1λi-ρ122aiiλi2)-(k-d)αi2].
When 0<d<1 fixed, then we write (17) as follows:
(18)MSE(β~1)-MSE(β~1(k,d))=∑i=1p1(k-d)(λi+k)2{[σ11(2λi+d)(1λi+ρ122aiiλi2)+dαi2]hhhhhhhhhhhhh+k[σ11(1λi-ρ122aiiλi2)-αi2]}.
So(1)
Ifσ11(1/λi-ρ122(aii/λi2))-αi2≥0, then MSE(β~1)≥MSE(β~1(k,d));(2)
ifσ11(1/λi-ρ122(aii/λi2))-αi2<0, when 0<k<(σ11(2λi+d)(1/λi+ρ122(aii/λi2))+dαi2)/(αi2-σ11(1/λi-ρ122(aii/λi2))), then MSE(β~1)≥MSE(β~1(k,d)).
Whenk>0 fixed, then we write (17) as follows:
(19)MSE(β~1)-MSE(β~1(k,d))=∑i=1p1(k-d)(λi+k)2{[σ11(2λi+k)(1λi+ρ122aiiλi2)-kαi2]hhhhhhhhhhhh+d[σ11(1λi-ρ122aiiλi2)+αi2]}.
So if(1)
σ11(2λi+k)(1/λi+ρ122(aii/λi2))≥kαi2, then MSE(β~1)≥MSE(β~1(k,d));(2)
σ11(2λi+k)(1/λi+ρ122(aii/λi2))<kαi2, when (kαi2-σ11(2λi+k)(1/λi+ρ122(aii/λi2)))/(σ11(1/λi-ρ122(aii/λi2))+αi2)≤d<1, then MSE(β~1)≥MSE(β~1(k,d)).


## 4. The Admissible of the New Estimatorβ~1(k,d)

As we all know, the admissible of an estimator is an important problem in SUR model. In this section, we discuss the admissible of the new estimatorβ~1(k,d). Firstly, we give the definition of the admissible.Definition 3.
Letθ^1 and θ^2 be two estimators of θ, for arbitrarily risk loss function R(·,·), (1) R(θ^1,θ)≤R(θ^2,θ) for arbitrarily θ. (2) There at least exist a θ0 such that the inequality is satisfied, Then we see that θ^1 is uniformly better than θ^2 for risk loss function R(·,·). If, in some class estimators, there is not exist an estimator that is uniformly better than θ~, we can say that θ~ is admissible estimator of θ for risk loss function R(·,·) in that class estimators.Lemma 4 (see [13]).
For linear model(20)y=Xβ+e,E(e)=0,Cov(e)=σ2Σ,
where Σ>0 is a known matrix and X is full column rank. Let T=X′Σ-1X, β*(Σ)=T-1X′Σ-1y; then in the linear estimators of Sβ, Lβ*(Σ) is admissible estimator of Sβ if and only if
(21)LT-1L′≤LT-1S′.Now we discuss the admissibility of the new estimator. In this section we consider the quadratic loss (mean square error).Theorem 5.
For the SUR model (1), when V is know, if X1′X2=0, then β~1(k,d) is admissible estimator of β1 in the class of linear estimators Φ.Proof.
IfX1′X2=0, then
(22)β~1(k,d)=(X1′X1+kI)-1(X1′X1+dI)(X1′X1)-1X1′Y1-σ12σ22(X1′X1+kI)-1×(X1′X1+dI)(X1′X1)-1X1′Y2.
For SUR model (1), let Y=(Y1′,Y2′)′, X=diag⁡(X1,X2), β=(β1′,β2′), and ε=(ε1′,ε2′), Then (1) can be written as
(23)Y=Xβ+ε,E(ε)=0,Cov(ε)=V,
where
(24)V=(σ11Iσ12Iσ21Iσ22I),β*=(β1′*(V),β2′*(V))
and (24) satisfies the following equation:
(25)X′V-1Xβ*=X′V-1Y.
Note that
(26)V-1=(σ11σ22-σ122)-1(σ22I-σ21I-σ12Iσ11I).
Then (25) can be written as follows:
(27)(σ22X1′X1-σ21X1′X2-σ12X2′X1σ11X2′X2)(β1′*(V)β2′*(V))=(σ22X1′-σ21X1′-σ12X2′σ11X2′)(Y1Y2).
When X1′X2=0, (25) can be written as follows:
(28)(σ22X1′X100σ11X2′X2)(β1′*(V)β2′*(V))=(σ22X1′-σ21X1′-σ12X2′σ11X2′)(Y1Y2).
Then by (28), we may get
(29)X1′X1β1′*(V)=X1′Y1-σ12σ22X1′Y2
multiplication (X1′X1+kI)-1(X1′X1+dI)(X1′X1)-1 in (28); we have
(30)(X1′X1+kI)-1(X1′X1+dI)β1′*(V)=β~1(k,d).
Then
(31)β~1(k,d)=(X1′X1+kI)-1(X1′X1+dI)(I,0)β′*(V).
LetL=(X1′X1+kI)-1(X1′X1+dI)(I,0) and S=(I,0); by Lemma 4, if we want to prove that β~1(k,d) is admissible estimator of β1, we only need to prove LT-1L′≤LT-1S′. Since X1′X2=0, we may conclude that
(32)T=X′V-1X=(σ11σ22-σ122)-1(σ22X1′X100σ11X2′X2).
Then
(33)LT-1L′=(σ11σ22-σ122)(X1′X1+kI)-1×(X1′X1+dI)(I,0)(σ22X1′X100σ11X2′X2)-1×(I,0)(X1′X1+dI)(X1′X1+kI)-1=σ11σ22-σ122σ22(X1′X1+kI)-1(X1′X1+dI)×(X1′X1)-1(X1′X1+dI)(X1′X1+kI)-1,LT-1S′=(σ11σ22-σ122)(X1′X1+kI)-1(X1′X1+dI)(I,0)×(σ22X1′X100σ11X2′X2)-1(I,0)=σ11σ22-σ122σ22(X1′X1+kI)-1×(X1′X1+dI)(X1′X1)-1.
It is obvious that L′T-1L′≤L′T-1S′. The proof is completed.Theorem 6.
For the SUR model (1), when V is known, if P1P2=P2P1, then β~1(k,d) is admissible estimator of β1 in the class of linear estimators Φ.Proof.
For SUR model (1), when P1P2=P2P1, by Lin [14], we obtain β~1=β1*(V); thus we obtain
(34)β~1(k,d)=(X1′X1+kI)-1(X1′X1+dI)β~1=(X1′X1+kI)-1(X1′X1+dI)β1*(V)=(X1′X1+kI)-1(X1′X1+dI)(I,0)β*(V)=Lβ*(V).
Similar to Theorem 5, we only need to prove LT-1L′≤LT-1S′; L and S are defined as Theorem 5:
(35)T=X′V-1X=(σ11σ22-σ122)-1(σ22X1′X1-σ21X1′X2-σ12X2′X1σ11X2′X2).
Denote T-1=(T11T12T21T22); by Rao et al. [15], we have
(36)T11=σ11σ22-σ122σ22[I-ρ122(X1′X1)-1X1′P2X1]-1×(X1′X1)-1.
Then we have
(37)LT-1L′=(X1′X1+kI)-1(X1′X1+dI)T11×(X1′X1+dI)(X1′X1+kI)-1=σ11σ22-σ122σ22(X1′X1+kI)-1(X1′X1+dI)×[I-ρ122(X1′X1)-1X1′P2X1]-1(X1′X1)-1×(X1′X1+dI)(X1′X1+kI)-1,LT-1S′=(X1′X1+kI)-1(X1′X1+dI)T11=σ11σ22-σ122σ22(X1′X1+kI)-1(X1′X1+dI)×[I-ρ122(X1′X1)-1X1′P2X1]-1(X1′X1)-1.
Obviously L′T-1L′≤L′T-1S′. The proof is completed.


## 5. Conclusion Remarks

In this paper we consider the parameter estimation in seemingly unrelated regression system. A Liu-type estimator is proposed to overcome the multicollinearity in seemingly unrelated regression system. The superiority of the new estimator has been also discussed and the admissibility of the new estimator is also discussed.



---

*Source: 679835-2014-03-16.xml*