# On the Weighted Mixed Almost Unbiased Ridge Estimator in Stochastic Restricted Linear Regression

Authors: Chaolin Liu; Hu Yang; Jibo Wu
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

We introduce the weighted mixed almost unbiased ridge estimator (WMAURE) based on the weighted mixed estimator (WME) (Trenkler and Toutenburg 1990) and the almost unbiased ridge estimator (AURE) (Akdeniz and Erol 2003) in linear regression model. We discuss superiorities of the new estimator under the quadratic bias (QB) and the mean square error matrix (MSEM) criteria. Additionally, we give a method about how to obtain the optimal values of parametersk and w. Finally, theoretical results are illustrated by a real data example and a Monte Carlo study.


---

## Body

## 1. Introduction

Consider the linear regression model(1)y=Xβ+ε,
where y=(y1,y2,…,yn)T is an n-dimensional response vector, X=(X1,X2,…,Xn)T with Xi=(xi1,xi2,…,xip)T is a known n×p matrix of full column rank, β is a p×1 vector of unknown parameters, and ε is an n×1 vector of errors with expectation E(ε)=0 and covariance matrix Cov(ε)=σ2In,In is an identity matrix of order n×n.It is well known that the ordinary least squares estimator (LS) forβ is given by
(2)β^LS=(XTX)-1XTy,
which has been treated as the best estimator for a long time. However, many results have proved that LS is no longer a good estimator when the multicollinearity is present in model (1). To tackle this problem, some suitable biased estimators have been developed, such as principal component regression estimator (PCR) [1], ordinary ridge estimator (RE) [2], r-k class estimator [3], Liu estimator (LE) [4], and r-d class estimator [5]. Kadiyala [6] introduced a class of almost unbiased shrinkage estimator which can be not only almost unbiased but also more efficient than the LS. Singh et al. [7] introduced the almost unbiased generalized ridge estimator by the jackknife procedure, and Akdeniz and Kaçiranlar [8] studied the almost unbiased generalized Liu estimator. By studying bias corrected estimators of the RE and the LE, Akdeniz and Erol [9] discussed the almost unbiased ridge estimator (AURE) and the almost unbiased Liu estimator (AULE).An alternative technique to tackle the multicollinearity is to consider the parameter estimator in addition to the sample information, such as some exact or stochastic restrictions on unknown parameters. When additional stochastic linear restrictions on unknown parameters are assumed to be held, Durbin [10], Theil and Goldberger [11], and Theil [12] proposed the ordinary mixed estimator (OME). Hubert and Wijekoon [13] proposed the stochastic restricted Liu estimator, and Yang and Xu [14] obtained a new stochastic restricted Liu estimator. By grafting the RE into the mixed estimation procedure, Li and Yang [15] introduced the stochastic restricted ridge estimator. When the prior information and the sample information are not equally important, Schaffrin and Toutenburg [16] studied the weighted mixed regression and developed the weighted mixed estimator (WME). Li and Yang [17] grafted the RE into the weighted mixed estimation procedure and proposed the weighted mixed ridge estimator (WMRE).In this paper, by combining the WME and the AURE, we propose a weighted mixed almost unbiased ridge estimator (WMAURE) for unknown parameters in a linear regression model when additional stochastic linear restriction is supposed to be held. Furthermore, we discuss the performance of the new estimator over the LS, WME, AURE, and WMRE with respect to the quadratic bias (QB) and the mean square error matrix (MSEM) criteria.The rest of the paper is organized as follows. In Section2, we describe the statistical model and propose the weighted mixed almost unbiased ridge estimator. We compare the new estimator with the weighted mixed ridge estimator and the almost unbiased ridge estimator under the quadratic bias criterion in Section 3. In Section 4, superiorities of the proposed estimator over relative estimators are considered under the mean square error matrix criterion. In Section 5, the selection of parameters k and w is discussed. Finally, to justify the superiority of the new estimator, we perform a real data example and a Monte Carlo simulation study in Section 6. We give some conclusions in Section 7.


## 2. The Proposed Estimator

The ordinary ridge estimator proposed by Hoerl and Kennard [2] is defined as
(3)β^RE(k)=(S+kI)-1XTy=Sk-1XTy,
where Sk=S+kI, k>0. Let Tk=(I+kS-1)-1=Sk-1S=SSk-1; we may rewrite β^RE(k) as
(4)β^RE(k)=Tkβ^LS=TkS-1XTy=S-1TkXTy.The almost unbiased ridge estimator obtained by Akdeniz and Erol [9] is denoted as
(5)β^AURE(k)=(I-k2Sk-2)β^LS=(I+kSk-1)Tkβ^LS.In addition to model (1), let us give some prior information about β in the form of a set of J which is independent stochastic linear restriction
(6)r=Rβ+e,e~(0,σ2W),
where R is a J×p known matrix of rank J, e is a J×1 vector of disturbances with expectation 0 and covariance matrix σ2W, W is supposed to be known and positive definite, and the J×1 vector r can be interpreted as a random variable with expectation E(r)=Rβ. Then, we can derive that (6) does not hold exactly but in the mean. We assume r to be a realized value of the random vector, so that all expectations are conditional on r [18]. We will not separately mention this in the following discussions. Furthermore, it is also supposed that ε is stochastically independent of e.For the restricted model specified by (1) and (6), the OME introduced by Durbin [10], Theil and Goldberger [11], and Theil [12] is defined as
(7)β^OME=(S+RTW-1R)-1(XTy+RTW-1r).When the prior information and the sample information are not equally important, Schaffrin and Toutenburg [16] considered the WME which is denoted as
(8)β^WME(w)=(S+wRTW-1R)-1(XTy+wRTW-1r),
where w(0≤w≤1) is a nonstochastic and nonnegative scalar weight.Note that(9)(S+wRTW-1R)-1=S-1-wS-1RT(W+wRS-1RT)-1RS-1,(10)(S-1-wS-1RT(W+wRS-1RT)-1RS-1)wRTW-1r=wS-1RT(W+wRS-1RT)-1r.Then, the WME (8) can be rewritten as
(11)β^WME(w)=β^LS+wS-1RT(W+wRS-1RT)-1(r-Rβ^LS).Additionally, by combining the WME and RE, Li and Yang [17] obtained the WMRE which is defined as
(12)β^WMRE(w,k)=β^RE(k)+wS-1RT(W+wRS-1RT)-1(r-Rβ^RE(k)).The WMRE also can be rewritten as(13)β^WMRE(w,k)=(S+wRTW-1R)-1(TkXTy+wRTW-1r).Now, based on the WME [16] and the AURE [9], we can define the following weighted mixed almost unbiased ridge estimator:
(14)β^WMAURE(w,k)=β^AURE(k)+wS-1RT(W+wRS-1RT)-1×(r-Rβ^AURE(k))=(I-k2Sk-2)β^LS+wS-1RT(W+wRS-1RT)-1×(r-R(I-k2Sk-2)β^LS)=(S-1-wS-1RT(W+wRS-1RT)-1RS-1)×(I-k2Sk-2)XTy+wS-1RT(W+wRS-1RT)-1r,
which is according to the way in [17].Using (10), (14) can be rewritten as
(15)β^WMAURE(w,k)=(S-1-wS-1RT(W+wRS-1RT)-1RS-1)×((I-k2Sk-2)XTy+wRTW-1r)=(S+wRTW-1R)-1((I-k2Sk-2)XTy+wRTW-1r).From the definition ofβ^WMAURE(w,k), it can be seen that β^WMAURE(w,k) is a general estimator, and as special cases of it, the WME, LS, and AURE can be described as
(16)β^WMAURE(w,0)=β^WME(w),β^WMAURE(0,0)=β^LS,
and if R=0,
(17)β^WMAURE(w,k)=β^AURE(k).It is easy to compute expectation values and covariance matrices of the LS, WME, WMRE, AURE, and WMAURE as(18)E(β^LS)=β,COV(β^LS)=σ2S-1,E(β^WME(w))=β,COV(β^WME(w))=σ2A(S+w2RTW-1R)A,E(β^WMRE(w,k))=β+A(Tk-I)Sβ,COV(β^WMRE(w,k))=σ2A(TkSTk+w2RTW-1R)A,E(β^AURE(k))=(I-k2Sk-2)β=(I+kSk-1)Tkβ,COV(β^AURE(k))=σ2(I-k2Sk-2)S-1(I-k2Sk-2)=σ2(I+kSk-1)TkS-1Tk(I+kSk-1),E(β^WMAURE(w,k))=β-k2ASk-2Sβ,COV(β^WMAURE(w,k))=σ2A((I-k2Sk-2)S(I-k2Sk-2)+w2RTW-1R)A,
where A=(S+wRTW-1R)-1.In the rest of the sections, we intend to study the performance of the new estimator over relative estimators under the quadratic bias and the mean square error matrix criteria.


## 3. Quadratic Bias Comparison of Estimators

In this section, quadratic bias comparisons are performed among the AURE, WMRE, and WMAURE. Letβ^ be the estimator of β, then the quadratic bias of β^ is defined as QB(β^)=Bias(β^)TBias(β^), where Bias(β^)=E(β^)-β. Based on the definition of quadratic bias, we can easily get quadratic biases of AURE, WMRE, and WMAURE:
(19)QB(β^AURE(k))=Bias(β^AURE(k))TBias(β^AURE(k))QB(β^AURE(k))=k4βTSk-4β,QB(β^WMRE(w,k))=Bias(β^WMRE(w,k))TBias(β^WMRE(w,k))=k2βTSSk-1A2Sk-1Sβ,QB(β^WMAURE(w,k))=Bias(β^WMAURE(w,k))TBias(β^WMAURE(w,k))=k4βTSSk-2A2Sk-2Sβ.


### 3.1. Quadratic Bias Comparison between the AURE and WMAURE

Here, we focus on the quadratic bias comparison between the AURE and WMAURE. The difference of quadratic biases can be derived as(20)Δ1=QB(β^AURE(k))-QB(β^WMAURE(w,k))=k4βTSk-4β-k4βTSSk-2A2Sk-2Sβ=k4βTSk-4β-k4βTSk-2SA2SSk-2β=k4βTSk-2(I-SA2S)Sk-2β.Firstly, we just considerI-SA2S. Note that (I-SA2S)T=I-SA2S and
(21)I-SA2S=SS-2S-SA2S=S(S-2-A2)S.It can be seen thatA-2-S2=(S+wRTW-1R)2-S2>0, namely, A-2>S2. Then we can get
(22)S-2-A2>0.Thus, we haveI-SA2S>0 by (21) and (22). Therefore, we can derive that Δ1=QB(β^AURE(k))-QB(β^WMAURE(w,k))>0 and the β^WMAURE(w,k) outperforms the β^AURE(k) according to the quadratic bias criterion.Based on the above analysis, we can derive the following theorem.Theorem 1.
According to the quadratic bias criterion, the WMAURE performs better than the AURE.


### 3.2. Quadratic Bias Comparison between the WMRE and WMAURE

Similarly, the quadratic bias comparison between the WMRE and WMAURE will be discussed. The difference of quadratic biases of both estimators can be obtained by(23)Δ2=QB(β^WMRE(w,k))-QB(β^WMAURE(w,k))=k2βTSSk-1A2Sk-1Sβ-k4βTSSk-2A2Sk-2Sβ=k2βTSSk-1(A2-k2Sk-1A2Sk-1)Sk-1Sβ.We considerA2-k2Sk-1A2Sk-1. Note that (A2-k2Sk-1A2Sk-1)T=A2-k2Sk-1A2Sk-1 and
(24)A2-k2Sk-1A2Sk-1=A2-kSk-1A2+kSk-1A2-k2Sk-1A2Sk-1=(I-kSk-1)A2+kSk-1A2(I-kSk-1).ForS=XTX>0, there exists an orthogonal matrix P such that S=PΛPT, where Λ=diag(λ1,…,λp) and λ1≥⋯≥λp denote the ordered eigenvalues of S. Therefore, we can easily compute that I-kSk-1=Pdiag(τ1,…,τp)PT, where τi=1-(k/(λi+k))=λi/(λi+k), i=1,…,p. For k>0, λi>0, we obtain τi>0 which means that I-kSk-1>0. Thus, we have (I-kSk-1)1/2A2(I-kSk-1)1/2>0 and (I-kSk-1)1/2kSk-1A2(I-kSk-1)1/2>0. Note that (I-kSk-1)A2, kSk-1A2(I-kSk-1) and (I-kSk-1)1/2A2(I-kSk-1)1/2, (I-kSk-1)1/2kSk-1A2(I-kSk-1)1/2 have same nonzero eigenvalues, respectively, which means that A2-k2Sk-1A2Sk-1>0.Therefore, we can derive thatΔ2=QB(β^WMRE(w,k))-QB(β^WMAURE(w,k))>0 and the β^WMAURE(w,k) performs better than the β^WMRE(w,k) under the quadratic bias criterion.We can get the following theorem.Theorem 2.
According to the quadratic bias criterion, the WMAURE outperforms the WMRE.


## 3.1. Quadratic Bias Comparison between the AURE and WMAURE

Here, we focus on the quadratic bias comparison between the AURE and WMAURE. The difference of quadratic biases can be derived as(20)Δ1=QB(β^AURE(k))-QB(β^WMAURE(w,k))=k4βTSk-4β-k4βTSSk-2A2Sk-2Sβ=k4βTSk-4β-k4βTSk-2SA2SSk-2β=k4βTSk-2(I-SA2S)Sk-2β.Firstly, we just considerI-SA2S. Note that (I-SA2S)T=I-SA2S and
(21)I-SA2S=SS-2S-SA2S=S(S-2-A2)S.It can be seen thatA-2-S2=(S+wRTW-1R)2-S2>0, namely, A-2>S2. Then we can get
(22)S-2-A2>0.Thus, we haveI-SA2S>0 by (21) and (22). Therefore, we can derive that Δ1=QB(β^AURE(k))-QB(β^WMAURE(w,k))>0 and the β^WMAURE(w,k) outperforms the β^AURE(k) according to the quadratic bias criterion.Based on the above analysis, we can derive the following theorem.Theorem 1.
According to the quadratic bias criterion, the WMAURE performs better than the AURE.


## 3.2. Quadratic Bias Comparison between the WMRE and WMAURE

Similarly, the quadratic bias comparison between the WMRE and WMAURE will be discussed. The difference of quadratic biases of both estimators can be obtained by(23)Δ2=QB(β^WMRE(w,k))-QB(β^WMAURE(w,k))=k2βTSSk-1A2Sk-1Sβ-k4βTSSk-2A2Sk-2Sβ=k2βTSSk-1(A2-k2Sk-1A2Sk-1)Sk-1Sβ.We considerA2-k2Sk-1A2Sk-1. Note that (A2-k2Sk-1A2Sk-1)T=A2-k2Sk-1A2Sk-1 and
(24)A2-k2Sk-1A2Sk-1=A2-kSk-1A2+kSk-1A2-k2Sk-1A2Sk-1=(I-kSk-1)A2+kSk-1A2(I-kSk-1).ForS=XTX>0, there exists an orthogonal matrix P such that S=PΛPT, where Λ=diag(λ1,…,λp) and λ1≥⋯≥λp denote the ordered eigenvalues of S. Therefore, we can easily compute that I-kSk-1=Pdiag(τ1,…,τp)PT, where τi=1-(k/(λi+k))=λi/(λi+k), i=1,…,p. For k>0, λi>0, we obtain τi>0 which means that I-kSk-1>0. Thus, we have (I-kSk-1)1/2A2(I-kSk-1)1/2>0 and (I-kSk-1)1/2kSk-1A2(I-kSk-1)1/2>0. Note that (I-kSk-1)A2, kSk-1A2(I-kSk-1) and (I-kSk-1)1/2A2(I-kSk-1)1/2, (I-kSk-1)1/2kSk-1A2(I-kSk-1)1/2 have same nonzero eigenvalues, respectively, which means that A2-k2Sk-1A2Sk-1>0.Therefore, we can derive thatΔ2=QB(β^WMRE(w,k))-QB(β^WMAURE(w,k))>0 and the β^WMAURE(w,k) performs better than the β^WMRE(w,k) under the quadratic bias criterion.We can get the following theorem.Theorem 2.
According to the quadratic bias criterion, the WMAURE outperforms the WMRE.


## 4. Mean Square Error Matrix Comparisons of Estimators

In this section, We will compare the proposed estimator with relative estimators under the mean square error matrix (MSEM) criterion.For the sake of convenience, we list some lemmas needed in the following discussions.Lemma 3.
LetM be a positive definite matrix, namely, M>0; let α be a vector, then M-ααT≥0 if and only if αTM-1α≤1.Proof.
See [19].Lemma 4.
Letβ^i=Aiy, i=1,2 be two competing homogeneous linear estimators of β. Suppose that D=COV(β^1)-COV(β^2)>0, then MSEM(β^1)-MSEM(β^2)≥0 if and only if d2T(D+d1d1T)-1d2≤1, where MSEM(β^i), di denote the MSEM and bias vector of β^i, respectively.Proof.
See [20].Lemma 5.
Let twon×n matrices M>0, N≥0, then M>N⇔λ1(NM-1)<1.Proof.
See [18].Firstly, the MSEM of an estimatorβ^ is defined as
(25)MSEM(β^)=E((β^-β)(β^-β)T)=COV(β^)+Bias(β^)Bias(β^)T.For two given estimatorsβ^1 and β^2, the estimator β^2 is said to be superior to β^1 under the MSEM criterion if and only if
(26)M(β^1,β^2)=MSEM(β^1)-MSEM(β^2)≥0.The scalar mean square error (MSE) is defined asMSE(β^)=tr(MSEM(β^)). It is well known that the MSEM criterion is superior over the MSE criterion; we just compare the MSEM of the WMAURE with other relative estimators.Now, from (18), we can easily obtain the MSEM of the WME, WMRE, AURE, and WMAURE as follows:
(27)MSEM(β^LS)=σ2S-1,(28)MSEM(β^WME(w))=σ2A(S+w2RTW-1R)A,(29)MSEM(β^WMRE(w,k))=σ2A(TkSTk+w2RTW-1R)A+b1b1T,(30)MSEM(β^AURE(k))=σ2(I+kSk-1)TkS-1Tk(I+kSk-1)+b2b2T,(31)MSEM(β^WMAURE(w,k))=σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)×A+b3b3T,
where b1=A(Tk-I)Sβ, b2=-k2Sk-2β and b3=-k2ASk-2Sβ.


### 4.1. MSEM Comparison of the WME and WMAURE

To compare MSEM values between the WMAURE and WME. Firstly, from (28) and (31), the difference of MSEM values between the WME and WMAURE can be gained by
(32)M(β^WME(w),β^WMAURE(w,k))=MSEM(β^WME(w))-MSEM(β^WMAURE(w,k))=σ2A(S+w2RTW-1R)A-σ2A×((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A-b3b3T=σ2A(S-(I+kSk-1)TkSTk(I+kSk-1))A-b3b3T=σ2D1-b3b3T,
where D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A.Theorem 6.
The WMAUREβ^WMAURE(w,k) is superior to the WME β^WME(w) under MSEM criterion, namely, M(β^WME(w),β^WMAURE(w,k))≥0 if and only if b3TD1-1b3≤σ2.Proof.
Note thatS=PΛPT. We can easily compute that S-(I+kSk-1)TkSTk(I+kSk-1)=PΓ(1)PT=Pdiag(τ1(1),…,τp(1))PT, where Γ(1)=Λ-(I+k(Λ+kI)-1)(I+kΛ-1)-1Λ(I+kΛ-1)-1(I+k(Λ+kI)-1) and
(33)τi(1)=λi-(1+k(λi+k)-1)(1+kλi-1)-1×λi(1+kλi-1)-1(1+k(λi+k)-1)=λi-λi+2kλi+kλiλi+kλiλiλi+kλi+2kλi+k=(k4+4k3λi+2k2λi2)λi(λi+k)4>0,i=1,…,p,
which means S-(I+kSk-1)TkSTk(I+kSk-1)>0. Observing that A=(S+wRTW-1R)-1>0, we have D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A>0.
Applying Lemma3, we can get that M(β^WME(w),β^WMAURE(w,k))≥0 if and only if b3TD1-1b3≤σ2.
This completes the proof.


### 4.2. MSEM Comparison of the WMRE and WMAURE

Similarly, we compare MSEM values between the WMAURE and WMRE. Firstly, from (29) and (31), the difference of MSEM values between the WMRE and the WMAURE can be computed by
(34)M(β^WMAURE(w,k),β^WMRE(w,k))=MSEM(β^WMAURE(w,k))-MSEM(β^WMRE(w,k))=σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A+b3b3T-σ2A(TkSTk+w2RTW-1R)A-b1b1T=σ2A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A-b1b1T=σ2D2+b3b3T-b1b1T,
where D2=A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A.Theorem 7.
The WMAUREβ^WMAURE(w,k)(k>0) is superior to the WMRE β^WMRE(w,k) under MSEM criterion, namely, M(β^WMAURE(w,k),β^WMRE(w,k))<0 if and only if b1T(σ2D2+b3b3T)-1b1>1.Proof.
Firstly, we prove(I+kSk-1)TkSTk(I+kSk-1)-TkSTk>0.
Note thatS=PΛPT, we can compute (I+kSk-1)TkSTk(I+kSk-1)-TkSTk=PΓ(2)PT=Pdiag(τ1(2),…,τp(2))PT, where
(35)Γ(2)=(I+k(Λ+kI)-1)(1+kΛ-1)-1Λ(1+kΛ-1)-1×(I+k(Λ+kI)-1)-(1+kΛ-1)-1Λ(1+kΛ-1)-1,τi(2)=(1+k(λi+k)-1)(1+kλi)-1λi(1+kλi)-1×(1+k(λi+k)-1)-(1+kλi)-1λi(1+kλi)-1=(1+kλi)-1λi(1+kλi)-1(λi+2kλi+k)2>0,i=1,…,p.
Thus, we can get(I+kSk-1)TkSTk(I+kSk-1)-TkSTk>0. Observing that A>0, we have D2=A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A>0.
Applying Lemma4, we can get M(β^WMAURE(w,k),β^WMRE(w,k))<0 if and only if b1T(σ2D2+b3b3T)-1b1>1.
This completes the proof.


### 4.3. MSEM Comparison of the AURE and WMAURE

Now, we compare MSEM values between the WMAURE and AURE. Firstly, from (30) and (31), the difference of MSEM values between the AURE and WMAURE can be obtained by
(36)M(β^AURE(k),β^WMAURE(w,k))=MSEM(β^AURE(k))-MSEM(β^WMAURE(w,k))=σ2(I+kSk-1)TkSTk(I+kSk-1)+b2b2T-b3b3T-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A=σ2D3+b2b2T-b3b3T,
where D3=(I+kSk-1)TkSTk(I+kSk-1)-A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A.Theorem 8.
Whenλ1(A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A((I+kSk-1)TkSTk(I+kSk-1))-1)<1, the WMAURE is superior to the AURE in the MSEM sense, namely, M(β^AURE(k),β^WMAURE(w,k))≥0 if and only if b3T(σ2D3+b2b2T)-1b3≤1.Proof.
Note that(I+kSk-1)TkSTk(I+kSk-1)>0 and A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A>0. When λ1(A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A((I+kSk-1)TkSTk(I+kSk-1))-1)<1, we can get that D3>0 by applying Lemma 5. Thus, from (36) and applying Lemma 4, we have M(β^AURE(k),β^WMAURE(w,k))≥0 if and only if b3T(σ2D3+b2b2T)-1b3≤1.
The proof is completed.


### 4.4. MSEM Comparison of the LS and WMAURE

Finally, we compare MSEM values between the LS and WMAURE. From (27) and (31), the difference of MSEM values between the LS and WMAURE can be computed by
(37)M(β^LS,β^WMAURE(w,k))=MSEM(β^LS)-MSEM(β^WMAURE(w,k))=σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)σ2S-1-σ2A+w2RTW-1R)A-b3b3T=σ2S-1-σ2A+σ2A-MSEM(β^WME(w))+MSEM(β^WME(w))-MSEM(β^WMAURE(w,k))=(σ2S-1-σ2A)+(σ2A-σ2A(S+w2RTW-1R)A)+σ2D1-b3b3T,
where A=(S+wRTW-1R)-1 and D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A>0 according to Section 4.1.Firstly, using (9), we can compute that
(38)σ2S-1-σ2A=σ2wS-1RT(W+wRS-1RT)-1RS-1>0.Moreover, it can be computed that(39)σ2A-σ2A(S+w2RTW-1R)A=σ2AA-1A-σ2A(S+w2RTW-1R)A=σ2A(S+wRTW-1R-S-w2RTW-1R)A=w(1-w)σ2A(RTW-1R)A>0.Therefore,σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A>0. Applying Lemma 3, we can get that M(β^LS,β^WMAURE(w,k))≥0 if and only if b3T(σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A)b3≤0.Based on the above analysis, we can state the following theorem.Theorem 9.
The WMAURE is superior to the LS according to the MSEM criterion, namely,M(β^LS,β^WMAURE(w,k))≥0 if and only if b3T(σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A)b3≤0.


## 4.1. MSEM Comparison of the WME and WMAURE

To compare MSEM values between the WMAURE and WME. Firstly, from (28) and (31), the difference of MSEM values between the WME and WMAURE can be gained by
(32)M(β^WME(w),β^WMAURE(w,k))=MSEM(β^WME(w))-MSEM(β^WMAURE(w,k))=σ2A(S+w2RTW-1R)A-σ2A×((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A-b3b3T=σ2A(S-(I+kSk-1)TkSTk(I+kSk-1))A-b3b3T=σ2D1-b3b3T,
where D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A.Theorem 6.
The WMAUREβ^WMAURE(w,k) is superior to the WME β^WME(w) under MSEM criterion, namely, M(β^WME(w),β^WMAURE(w,k))≥0 if and only if b3TD1-1b3≤σ2.Proof.
Note thatS=PΛPT. We can easily compute that S-(I+kSk-1)TkSTk(I+kSk-1)=PΓ(1)PT=Pdiag(τ1(1),…,τp(1))PT, where Γ(1)=Λ-(I+k(Λ+kI)-1)(I+kΛ-1)-1Λ(I+kΛ-1)-1(I+k(Λ+kI)-1) and
(33)τi(1)=λi-(1+k(λi+k)-1)(1+kλi-1)-1×λi(1+kλi-1)-1(1+k(λi+k)-1)=λi-λi+2kλi+kλiλi+kλiλiλi+kλi+2kλi+k=(k4+4k3λi+2k2λi2)λi(λi+k)4>0,i=1,…,p,
which means S-(I+kSk-1)TkSTk(I+kSk-1)>0. Observing that A=(S+wRTW-1R)-1>0, we have D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A>0.
Applying Lemma3, we can get that M(β^WME(w),β^WMAURE(w,k))≥0 if and only if b3TD1-1b3≤σ2.
This completes the proof.


## 4.2. MSEM Comparison of the WMRE and WMAURE

Similarly, we compare MSEM values between the WMAURE and WMRE. Firstly, from (29) and (31), the difference of MSEM values between the WMRE and the WMAURE can be computed by
(34)M(β^WMAURE(w,k),β^WMRE(w,k))=MSEM(β^WMAURE(w,k))-MSEM(β^WMRE(w,k))=σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A+b3b3T-σ2A(TkSTk+w2RTW-1R)A-b1b1T=σ2A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A-b1b1T=σ2D2+b3b3T-b1b1T,
where D2=A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A.Theorem 7.
The WMAUREβ^WMAURE(w,k)(k>0) is superior to the WMRE β^WMRE(w,k) under MSEM criterion, namely, M(β^WMAURE(w,k),β^WMRE(w,k))<0 if and only if b1T(σ2D2+b3b3T)-1b1>1.Proof.
Firstly, we prove(I+kSk-1)TkSTk(I+kSk-1)-TkSTk>0.
Note thatS=PΛPT, we can compute (I+kSk-1)TkSTk(I+kSk-1)-TkSTk=PΓ(2)PT=Pdiag(τ1(2),…,τp(2))PT, where
(35)Γ(2)=(I+k(Λ+kI)-1)(1+kΛ-1)-1Λ(1+kΛ-1)-1×(I+k(Λ+kI)-1)-(1+kΛ-1)-1Λ(1+kΛ-1)-1,τi(2)=(1+k(λi+k)-1)(1+kλi)-1λi(1+kλi)-1×(1+k(λi+k)-1)-(1+kλi)-1λi(1+kλi)-1=(1+kλi)-1λi(1+kλi)-1(λi+2kλi+k)2>0,i=1,…,p.
Thus, we can get(I+kSk-1)TkSTk(I+kSk-1)-TkSTk>0. Observing that A>0, we have D2=A((I+kSk-1)TkSTk(I+kSk-1)-TkSTk)A>0.
Applying Lemma4, we can get M(β^WMAURE(w,k),β^WMRE(w,k))<0 if and only if b1T(σ2D2+b3b3T)-1b1>1.
This completes the proof.


## 4.3. MSEM Comparison of the AURE and WMAURE

Now, we compare MSEM values between the WMAURE and AURE. Firstly, from (30) and (31), the difference of MSEM values between the AURE and WMAURE can be obtained by
(36)M(β^AURE(k),β^WMAURE(w,k))=MSEM(β^AURE(k))-MSEM(β^WMAURE(w,k))=σ2(I+kSk-1)TkSTk(I+kSk-1)+b2b2T-b3b3T-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A=σ2D3+b2b2T-b3b3T,
where D3=(I+kSk-1)TkSTk(I+kSk-1)-A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A.Theorem 8.
Whenλ1(A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A((I+kSk-1)TkSTk(I+kSk-1))-1)<1, the WMAURE is superior to the AURE in the MSEM sense, namely, M(β^AURE(k),β^WMAURE(w,k))≥0 if and only if b3T(σ2D3+b2b2T)-1b3≤1.Proof.
Note that(I+kSk-1)TkSTk(I+kSk-1)>0 and A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A>0. When λ1(A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A((I+kSk-1)TkSTk(I+kSk-1))-1)<1, we can get that D3>0 by applying Lemma 5. Thus, from (36) and applying Lemma 4, we have M(β^AURE(k),β^WMAURE(w,k))≥0 if and only if b3T(σ2D3+b2b2T)-1b3≤1.
The proof is completed.


## 4.4. MSEM Comparison of the LS and WMAURE

Finally, we compare MSEM values between the LS and WMAURE. From (27) and (31), the difference of MSEM values between the LS and WMAURE can be computed by
(37)M(β^LS,β^WMAURE(w,k))=MSEM(β^LS)-MSEM(β^WMAURE(w,k))=σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)σ2S-1-σ2A+w2RTW-1R)A-b3b3T=σ2S-1-σ2A+σ2A-MSEM(β^WME(w))+MSEM(β^WME(w))-MSEM(β^WMAURE(w,k))=(σ2S-1-σ2A)+(σ2A-σ2A(S+w2RTW-1R)A)+σ2D1-b3b3T,
where A=(S+wRTW-1R)-1 and D1=A(S-(I+kSk-1)TkSTk(I+kSk-1))A>0 according to Section 4.1.Firstly, using (9), we can compute that
(38)σ2S-1-σ2A=σ2wS-1RT(W+wRS-1RT)-1RS-1>0.Moreover, it can be computed that(39)σ2A-σ2A(S+w2RTW-1R)A=σ2AA-1A-σ2A(S+w2RTW-1R)A=σ2A(S+wRTW-1R-S-w2RTW-1R)A=w(1-w)σ2A(RTW-1R)A>0.Therefore,σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A>0. Applying Lemma 3, we can get that M(β^LS,β^WMAURE(w,k))≥0 if and only if b3T(σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A)b3≤0.Based on the above analysis, we can state the following theorem.Theorem 9.
The WMAURE is superior to the LS according to the MSEM criterion, namely,M(β^LS,β^WMAURE(w,k))≥0 if and only if b3T(σ2S-1-σ2A((I+kSk-1)TkSTk(I+kSk-1)+w2RTW-1R)A)b3≤0.


## 5. Selection of Parametersk and w

In this section, we give a method about how to choose parametersk and w. Firstly, a linear regression model can be transformed to a canonical form by the orthogonal transformation. Let Q be an orthogonal matrix such that QTXTXQ=Λ=diag(λ1,…,λp), where λi is the eigenvalue of XTX, and X~=XQ, β~=QTβ=(β~1,…,β~p)T. Then, we get a canonical form of model (1) as
(40)y=X~β~+ε.Note thatβ~^WMAURE(w,k)=QTβ^WMAURE(w,k) and MSEM(β~^WMAURE(w,k))=QTMSEM(β^WMAURE(w,k))Q. It is supposed that S and RTW-1R are commutative, then we have
(41)MSEM(β~^WMAURE(w,k))=σ2(Λ+wΨ)-1[(I+k(Λ+kI)-1)(Λ+kI)-1σ2(Λ+wΨ)-1×Λ3(Λ+kI)-1(I+k(Λ+kI)-1)+w2Ψ]×(Λ+wΨ)-1+k4(Λ+wΨ)-1(Λ+kI)-2×Λβ~β~TΛ(Λ+kI)-2(Λ+wΨ)-1,
where QTRTW-1RQ=Ψ=diag(ξ1,…,ξp).Optimal values fork and w can be derived by minimizing
(42)g(w,k)=tr(MSEM(β~^WMAURE(w,k)))=∑i=1p‍σ2[λi3(λi+2k)2+wξi2(λi+k)4]+k4λi2β~i2(λi+wξi)2(λi+k)4.For a fixed value ofw, differentiating g(w,k) with respect to k leads to
(43)∂g(w,k)∂k=∑i=1p‍((+k4λi2β~i2]1β~i2[σ2[4(λi+2k)λi3+4wξi2(λi+k)3]+4k3λi2β~i2]=∑i=1p‍×(λi+k)-4[β~i2σ2[(λi+2k)2λi3+wξi2(λi+k)4]+k4λi2β~i2])((λi+wξi)2(λi+k)5)-1λi2β~i2)=∑i=1p‍4kλi3(k2β~i2-2σ2k-σ2λi)(λi+wξi)2(λi+k)5,
and equating it to zero. Note that k>0 and after unknown parameters σ2 and β~ are replaced by their unbiased estimators, we obtain the optimal estimator of k for a fixed w value as
(44)k^=σ^2∑i=1pλi3+σ^4(∑i=1pλi3)2+σ^2(∑i=1pλi4)(∑i=1pλi3β~^i2)∑i=1pλi3β~^i2.Thew value which minimizes the function g(w,k) can be found by differentiating g(w,k) with respect to w when k is fixed
(45)∂g(w,k)∂w=∑i=1p‍({((λi+wξi)3(λi+k)4)-1))-1σ2ξi2(λi+k)4(λi+wξi)-2w[σ2λi3(λi+2k)2+wσ2ξi2(λi+k)4+k4λi2β~i2]((λi+wξi)3(λi+k)4)-1))-1}×((λi+wξi)3(λi+k)4)-1)=∑i=1p‍(×((λi+wξi)3(λi+k)4)-1{λi2β~i2-2w2σ2ξi2(λi+k)4+σ2λiξi2(λi+k)4+w[σ2ξi3(λi+k)4-2σ2λi3(λi+2k)2-2k4λi2β~i2]}×((λi+wξi)3(λi+k)4)-1)
and equating it to zero. After unknown parameters σ2 and β~ are replaced by their unbiased estimators, we get the optimal estimator of w for a fixed k value as
(46)w^=∑i=1ph^2i2+8(∑i=1ph^1i)2-∑i=1ph^2i∑i=1p4h^1i,
where h^1i=σ^2ξi2(λi+k)4 and h^2i=2σ^2λi3(λi+2k)2+2k4λi2β~^i2-σ^2ξi3(λi+k)4.


## 6. Numerical Example and Monte Carlo Simulation

In order to verify our theoretical results, we firstly conduct an experiment based on a real data set originally due to Woods et al. [21]. In this experiment, we replace the unknown parameters β and σ2 by their unbiased estimators, which is according to the way in [17]. The result here and below is performed with R 2.14.1.We can easily obtain that the condition number is about3.66×107. This information indicates a serious multicollinearity among the regression vector. The ordinary least squares estimator of β is
(47)β^LS=S-1XTy=(62.4054,1.5511,0.5102,0.1019,-0.1441)T
with σ^LS2=5.983.Consider the following stochastic linear restrictions used in [17]:
(48)r=Rβ+e,R=(1111101311),r=(63.94982.5648),e~(0,σ^LS2I2).For the WMRE, AURE, and WMAURE, their quadratic bias values are given in Table1 and their estimated MSE values are obtained in Table 2 by replacing all unknown parameters in the corresponding theoretical MSE expressions by their least squares estimators.Table 1
Estimated QB values of the WMRE, AURE, and WMAURE.
w
=
0.1
w
=
0.4
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
WMRE
16.567
20.217
20.336
1.160
1.413
1.421
AURE
2456.43
3662.01
3706.28
2456.43
3662.01
3706.28
WMAURE
13.166
19.627
19.864
0.922
1.375
1.391
w
=
0.8
w
=
1
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
WMRE
0.296
0.360
0.362
0.190
0.231
0.232
AURE
2456.43
3662.01
3706.28
2456.43
3662.01
3706.28
WMAURE
0.235
0.351
0.355
0.151
0.225
0.228Table 2
Estimated MSE values of the WME, WMRE, AURE, and WMAURE.
w
=
0.1
w
=
0.4
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
4912.13
4912.13
4912.13
4912.13
4912.13
4912.13
WME
59.771
59.771
59.771
39.278
39.278
39.278
WMRE
50.321
53.666
53.783
38.616
38.848
38.855
AURE
2663.66
3666.43
3709.16
2663.66
3666.43
3709.16
WMAURE
47.718
53.094
53.323
38.434
38.810
38.826
w
=
0.8
w
=
1
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
4912.13
4912.13
4912.13
4912.13
4912.13
4912.13
WME
38.639
38.639
38.639
38.620
38.620
38.620
WMRE
38.470
38.528
38.530
38.620
38.549
38.550
AURE
2663.66
3666.43
3709.16
2663.66
3666.43
3709.16
WMAURE
38.423
38.519
38.524
38.482
38.543
38.546It can be seen from Table1 that the WMAURE has smaller quadratic bias values than the WMRE and AURE for every case, which agrees with our theoretical finding in Section 3. From Table 2, we can get that MSE values of our proposed estimator are the smallest among the LS, WME, WMRE, AURE, and WMAURE when w is fixed, which agrees with our theoretical finding in Theorems 6–9.To further illustrate the behavior of our proposed estimator, we are to perform a Monte Carlo simulation study under different levels of multicollinearity. Following the way in [22, 23], we can get explanatory variables by the following equations:
(49)xij=(1-ρ2)1/2zij+ρzi,p+1,i=1,2,…,n,j=1,2,…,p,
where zij is an independent standard normal pseudorandom number, and ρ is specified so that the theoretical correlation between any two explanatory variables is given by ρ2. A dependent variable is generated by
(50)yi=β1xi1+β2xi2+β3xi3+β4xi4+εi,i=1,2,…,n,
where εi is a normal pseudo-random number with mean zero and variance σi2. In this study, we choose (β1,β2,β3,β4)T=(40,1,2,3)T, n=60, p=4, σi2=1, and the stochastic restriction r=Rβ+e, R=(40-312120), e~N(0,0.1I2). Furthermore, we discuss three cases when ρ=0.8,0.9,0.99.For three different levels of multicollinearity, MSE values of LS, WME, AURE, WMRE, and WMAURE are obtained in Tables3, 4, and 5, respectively. From Tables 3–5, we can derive the following results.(1)
With the increase of multicollinearity, MSE values of the LS, WME, WMRE, AURE, and WMAURE are increasing. And for all cases, the WMAURE has smaller estimated MSE values than the LS, AURE, and WME.(2)
The value ofw is the level of the weight to the sample information and the prior information; we can see from three tables that estimated MSE values of the WME, WMRE, and WMAURE become more and more smaller when the value of w increases. It can be concluded that we get more exact estimator of the parameter with more depended prior information.Table 3
Estimated MSE values of the WME, WMRE, AURE, and WMAURE whenρ=0.8.
w
=
0.1
w
=
0.4
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.0898720
0.0898720
0.0898720
0.0898720
0.0898720
0.0898720
WME
0.0889010
0.0889010
0.0889010
0.0867887
0.0867887
0.0867887
WMRE
0.0889032
0.0902973
0.0911375
0.0867896
0.088059
0.0888259
AURE
0.0898720
0.0898718
0.0898717
0.0898720
0.0898718
0.0898717
WMAURE
0.0889010
0.0889008
0.0889007
0.0867887
0.0867885
0.0867884
w
=
0.8
w
=
1
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.0898720
0.0898720
0.0898720
0.0898720
0.0898720
0.0898720
WME
0.0854822
0.0854822
0.0854822
0.0853378
0.0853378
0.0853378
WMRE
0.0854817
0.0866088
0.0872931
0.0853367
0.0864015
0.0870495
AURE
0.0898720
0.0898718
0.0898717
0.0898720
0.0898718
0.0898717
WMAURE
0.0854822
0.0854821
0.0854820
0.0853378
0.0853376
0.0853375Table 4
Estimated MSE values of the WME, WMRE, AURE, and WMAURE whenρ=0.9.
w
=
0.1
w
=
0.4
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.0987050
0.0987050
0.0987050
0.0987050
0.0987050
0.0987050
WME
0.0975363
0.0975363
0.0975363
0.0950143
0.0950143
0.0950143
WMRE
0.0975389
0.0992148
0.1002248
0.0950152
0.0965273
0.0974417
AURE
0.0987050
0.0987047
0.0987046
0.0987050
0.0987047
0.0987398
WMAURE
0.0975363
0.0975361
0.0975359
0.0950143
0.095014
0.0950139
w
=
0.8
w
=
1
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.0987050
0.0987050
0.0987050
0.0987050
0.0987050
0.0987050
WME
0.0934738
0.0934738
0.0934738
0.0933053
0.0933053
0.0933053
WMRE
0.0934729
0.0948017
0.0956092
0.0933036
0.0945530
0.0953142
AURE
0.0987050
0.0987047
0.0987398
0.0987050
0.0987047
0.0987398
WMAURE
0.0934738
0.0934736
0.0934735
0.0933053
0.0933050
0.0933049Table 5
Estimated MSE values of the WME, WMRE, AURE, and WMAURE whenρ=0.99.
w
=
0.1
w
=
0.4
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.1376903
0.1376903
0.1376903
0.1376903
0.1376903
0.1376903
WME
0.1354379
0.1354379
0.1354379
0.1307427
0.1307427
0.1307427
WMRE
0.1354425
0.138655
0.1405906
0.130743
0.1335329
0.1352221
AURE
0.1376903
0.1376896
0.1376892
0.1376903
0.1376896
0.1376892
WMAURE
0.1354379
0.1354372
0.1354368
0.1307426
0.130742
0.1307416
w
=
0.8
w
=
1
k
=
0.01
k
=
0.08
k
=
0.1
k
=
0.01
k
=
0.08
k
=
0.1
LS
0.1376903
0.1376903
0.1376903
0.1376903
0.1376903
0.1376903
WME
0.1280238
0.1280238
0.1280238
0.127739
0.127739
0.127739
WMRE
0.1280196
0.1303649
0.1317949
0.127733
0.1298949
0.1312179
AURE
0.1376903
0.1376896
0.1376892
0.1376903
0.1376896
0.1376892
WMAURE
0.1280238
0.1280231
0.1280228
0.127739
0.1277384
0.127738


## 7. Conclusions

In this paper, we propose the WMAURE based on the WME [16] and the AURE [9] and discuss some properties of the new estimator over the relative estimators. In particular, we prove that the WMAURE has smaller quadratic bias than the AURE and WMRE and derive that the proposed estimator is superior to the LS, WME, WMRE, and AURE in the mean squared error matrix sense under certain conditions. The optimal values of parameters k and w are obtained. Furthermore, we perform a real data example and a Monte Carlo study to support the finding of our theoretical results.



---

*Source: 902715-2013-04-30.xml*