# Risk Comparison of Improved Estimators in a Linear Regression Model with Multivariatet Errors under Balanced Loss Function

Authors: Guikai Hu; Qingguo Li; Shenghua Yu
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

Under a balanced loss function, we derive the explicit formulae of the risk of the Stein-rule (SR) estimator, the
positive-part Stein-rule (PSR) estimator, the feasible minimum mean squared error (FMMSE) estimator, and the adjusted feasible
minimum mean squared error (AFMMSE) estimator in a linear regression model with multivariatet errors. The results show
that the PSR estimator dominates the SR estimator under the balanced loss and multivariate t errors. Also, our numerical results
show that these estimators dominate the ordinary least squares (OLS) estimator when the weight of precision of estimation is
larger than about half, and vice versa. Furthermore, the AFMMSE estimator dominates the PSR estimator in certain occasions.


---

## Body

## 1. Introduction

In the literature, many statisticians have studied the risk comparisons of various estimators in the linear model with normal errors and have generated substantial results. However, the assumption of normality restricts the range of possible applications. The multivariatet distributions are more realistic and accurate than multivariate normal distributions in modeling real-word data due to their heavy tails. Moreover, multivariate t distribution plays an important role in robust statistical inference. Therefore, various inference problems based on these distributions have been studied. The sampling performance of estimators is an important aspect among them.Let us now consider a linear regression model(1)y=Xβ+u,
where y is an n×1 vector of observations on a dependent variable. X is an n×k full rank matrix of observations. β is a k×1 vector of coefficients. We assume that u has a multivariate t distribution with the probability density function given by
(2)P(u∣α,σ)=g(α)(σ2)n/21(α+u′u/σ2)(n+α)/2,
where g(α)=αα/2Γ((α+n)/2)/πn/2Γ(α/2). It is well known that its mean vector and covariance matrix are given by
(3)E(u)=0,forα>1,E(uu′)=ασ2(α-2)In,forα>2.
As is shown in Zellner [1], the multivariate t distribution can be viewed as a mixture of multivariate normal and inverted gamma distributions:
(4)P(u∣α,σ)=∫0∞‍PN(u∣τ)·PIG(τ∣α,σ)dτ,
where
(5)PN(u∣τ)=(2πτ2)-n/2exp⁡(-uu′2τ2),PIG(τ∣α,σ)=2(ασ2/2)α/2Γ(α/2)·τ-(α+1)exp⁡(-α·σ22τ2).The ordinary least squares (OLS) estimator ofβ is b=S-1X′y, where S=X′X. Also, the Stein-rule (SR) estimator is
(6)bSR=(1-ae′eb′Sb)b,
where e=y-Xb, v=n-k, and a is a constant such that 0≤a≤2(k-2)/(v+2). Under the mean squared error of prediction, Stein [2] and James and Stein [3] proved that the SR estimator dominates the OLS estimator when the numbers of explanatory variables are more than two and the MSE of the SR estimator is minimized if a=(k-2)/(v+2). Thus, we use this value of a hereafter. From then on, lots of improved estimators have been proposed. For example, Baranchik [4] proposed the positive-part Stein-rule (PSR) estimator defined as
(7)bPSR=max⁡[0,1-ae′eb′Sb]b.
Farebrother [5] proposed the feasible minimum mean squared error (FMMSE) estimator which is
(8)bFMMSE=(b′Sbb′Sb+e′e/v)b.
Further, Ohtani [6] extended the FMMSE estimator to the adjusted feasible minimum mean squared error (AFMMSE) estimator by adjusting the degrees of the freedom of the component of the FMMSE estimator. The AFMMSE estimator is
(9)bAFMMSE=(b′Sb/kb′Sb/k+e′e/v)b.Some results related to the comparisons of these estimators have been established. For example, Giles [7] considered the pretest estimator for linear restrictions. Namba [8] studied the PMSE performance of the biased estimators in a regression model when relevant regressors are omitted. Namba and Ohtani [9] gave the risk comparison of the Stein-rule estimator under the Pitman nearness criterion. There is a common characteristic in their studies. That is, the used loss functions were the quadratic function and its variants. However, in regression analysis, we are often interested in using an estimator which has high precision of estimation and high goodness of fit of model. In this situation, Zellner [10] proposed a balanced loss function which takes account of both precision of estimation and goodness of fit. Balanced loss function is a more comprehensive and reasonable standard than quadratic loss and residual sum of squares. Much work has been done about the balanced loss risk comparisons of improved estimators in the normal linear model. Some examples are Giles et al. [11], Ohtani et al. [12], Ohtani [13], and so on. Their results show that SR estimator is not admissible and is dominated by PSR estimator. However, do the conclusions still hold under multivariate t errors and balanced loss function? And, do these estimators still dominate the OLS estimator? It is interesting to discuss them under multivariate t distributions and balanced loss function. Thus, we will give the explicit formulae for the balanced loss risk of these estimators and compare their sampling performance by theoretical and numerical analysis. In the next section, the explicit formulae of balanced loss risk of these estimators are derived. In Section 3, we compare the risk performance by numerical evaluations. The proofs of main results are given in Section 4.


## 2. Balanced Loss Function and Risk

In order to discuss the performance of considered estimators, we consider the balanced loss function as(10)L(β~,β)=θ(y-Xβ~)′(y-Xβ~)+(1-θ)×(Xβ~-Xβ)′(Xβ~-Xβ),
where θ is a scalar such that 0≤θ≤1, and β~ is any estimator of β. The corresponding risk function is R(β~)=E[L(β~,β)]. Since u has a multivariate t distribution which can be viewed as the mixture of multivariate normal and inverted gamma distribution, we have
(11)R(β~)=E[L(β~,β)]=EτE[L(β~,β)∣τ].If the null hypothesis isH0:β=0 and the alternative is H1:β≠0, then the test statistic for H0 is F=(b′Sb/k)/(e′e/v). In the same way as that of Namba [8], we consider the general pretest estimator as
(12)β^=I(F≥c)(1+γe′eb′Sb)ωb,
where I(A) is an indicator function such that I(A)=1 if an event A occurs and I(A)=0 otherwise. c is the critical value of the pretest, and ω is an arbitrary integer. The term β^ reduces to the SR estimator when c=0,γ=-a, and ω=1, and it reduces to the PSR estimator when c=av/k,γ=-a, and ω=1. Furthermore, β^ reduces to the FMMSE estimator when c=0, γ=1/v, and ω=-1, and it reduces to the AFMMSE estimator when c=0, γ=k/v, and ω=-1, respectively.To derive the formulae ofR(β^), we first compute E[L(β^,β)∣τ], assuming that τ is given. If we denote u1=b′Sb/τ2, u2=e′e/τ2, then u1~χk2(λ1), and u2~χn-k2 for given τ, where λ1=β′Sβ/τ2, χf2(λ) is the noncentral chi-square distribution with f degrees of freedom and noncentrality parameter λ. Thus, using u1 and u2, we define the functions H(p,q,γ,c) and J(p,q,γ,c) as
(13)H(p,q,γ,c)=E[I(vk·u1u2≥c)(u1+γu2u1)pu1q∣τ],J(p,q,γ,c)=E[I(vk·u1u2≥c)(u1+γu2u1)pu1qβ′Sbτ2∣τ],
where p,q are arbitrary integers. By direct computation, we have
(14)E[L(β^,β)∣τ]=θ[β′Sβ+nτ2-2τ2H(ω,1,γ,c)+τ2H(2ω,1,γ,c)]+(1-θ)[τ2H(2ω,1,γ,c)-2τ2J(ω,0,γ,c)+β′Sβ].
In the following, we first give one lemma in order to obtain the explicit formulae of risk.Lemma 1.
The explicit formulae ofH(p,q,γ,c) and J(p,q,γ,c) are
(15)H(p,q,γ,c)=∑i=0∞‍wi(λ1)Gi(p,q,γ,c),J(p,q,γ,c)=λ1∑i=0∞‍wi(λ1)Gi+1(p,q,γ,c),
where Gi(p,q,γ,c)=(2qΓ(n/2+i+q)/Γ(k/2+i)Γ(v/2))∫c*1‍tk/2+i+q-p-1(1-t)v/2-1[γ+(1-γ)t]pdt, wi(λ)=exp⁡(-λ/2)(λ/2)i/i!, and c*=kc/(kc+v).By this lemma and (11) and (14), we have the following theorem.Theorem 2.
Under model (1) and loss function (10), the risk of the general pretest estimator β^ is
(16)R(β^)=β′Sβ+nθσ2αα-2+σ22αα/2Γ(α/2)×∑i=0∞‍Gi(2ω,1,γ,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1-θσ2αα/2Γ(α/2)∑i=0∞‍Gi(ω,1,γ,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1-2(1-θ)σ2αα/2Γ(α/2)×∑i=0∞‍Gi+1(ω,0,γ,c)i!Γ(α/2+i)θ1i+1(θ1+α)α/2+i.According to this theorem, we can obtain the risk of SR, PSR, FMMSE, and AFMMSE estimators, respectively, and discuss their dominance properties. Firstly, we analyze the dominance properties between SR and PSR estimators.Theorem 3.
The PSR estimator dominates the SR estimator in terms of balanced loss risk when the error term of the model obeys a multivariatet distribution.When the error term obeyed a multivariate normal distribution, Baranchik [4] proved that the PSR estimator dominated uniformly the SR estimator under the quadratic loss, and Ohtani [13] also proved that the SRSV estimator dominated uniformly the SR estimator under a balanced loss function. This theorem shows that when the loss function is extended to a balanced loss function, the dominance of the PSR estimator over the SR estimator still holds even if the error term obeys a multivariate t distribution. This implies that the SR estimator is not admissible under a balanced loss function and multivariate t errors.Since further theoretical analysis of the risk of the SR, PSR, FMMSE, and AFMMSE estimators is difficult, we will compare them by numerical analysis in the next section.


## 3. Numerical Analysis

In order to compare the balanced loss risk of the estimators, we evaluated the values of relative risk defined by(R(β^)/σ2)/(R(b)/σ2). Thus, the estimator β^ has smaller risk than the OLS estimator when the value of relative risk is smaller than unity. By Theorem 2, we can obtain the risks of the OLS, SR, PSR, FMMSE, and AFMMSE estimators, respectively. In the following, taking θ1=β′Sβ/σ2, the parameter values used in the numerical evaluations are θ1 = various values, θ=0,0.25,0.5,0.75,1, n=20,30,40, k=3,5,8, and α=3,5,7,10,20. The numerical evaluations are executed on a personal computer using Version 7.9 (R2009b) MATLAB Software. In order to evaluate the integral in the risk expressions of these estimators, we use Trapezoidal method with 1000 equal subdivisions. Following the method used by Namba [8], the infinite series in these risk expressions is judged to converge when the increment of the infinite series becomes smaller than 10-12. Now, we give the relative risk of the SR, PSR, FMMSE, and AFMMSE estimators for the case of n=20, k=5, θ=0,0.25,0.5,0.75,1, θ1=0,1,2,4,6,8,10,15,20, and α=3,20 in Tables 1 and 2, respectively. According to Tables 1 and 2, it is sufficient to illustrate the result of Theorem 3. That is, the PSR estimator dominates the SR estimator under a balanced loss even if the error term obeys a multivariate t distribution. We also find that when precision of estimation is more important (i.e., θ<0.5), the SR and PSR estimators dominate the OLS estimator under the balanced loss function, and vice versa. This shows that the dominance of the SR and PSR estimators over the OLS estimator is not robust about the loss function. From Table 1, the FMMSE and AFMMSE estimators dominate the OLS estimator when the weight of precision of estimation is larger than about half, and vice versa. This indicates that the dominance results of the FMMSE and AFMMSE estimators over the OLS estimator do not hold necessarily under the balanced loss function. It is easy to see that the risk of the AFMMSE estimator is much smaller than the risks of the SR and PSR estimators if θ<0.5. However, the AFMMSE estimator does not dominate the FMMSE estimator under the balanced loss function when θ≥0.75.Table 1
Relative risk of the SR, PSR, FMMSE, and AFMMSE estimators forn=20, k=5 and α=3.
θ
1
θ
0
0.25
0.5
0.75
1
SR
0
0.4705
0.8234
0.9999
1.1058
1.1764
1
0.5015
0.8338
0.9999
1.0996
1.1661
2
0.5262
0.8420
1.0000
1.0947
1.1579
4
0.5640
0.8546
1.0000
1.0872
1.1453
6
0.5923
0.8641
1.0000
1.0815
1.1358
8
0.6148
0.8716
1.0000
1.0770
1.1284
10
0.6333
0.8778
1.0000
1.0733
1.1222
15
0.6686
0.8895
1.0000
1.0663
1.1104
20
0.6942
0.8980
1.0000
1.0611
1.1019
PSR
0
0.3533
0.7255
0.9116
1.0233
1.0977
1
0.3918
0.7437
0.9196
1.0252
1.0955
2
0.4231
0.7582
0.9258
1.0263
1.0934
4
0.4715
0.7804
0.9349
1.0275
1.0893
6
0.5079
0.7968
0.9413
1.0279
1.0857
8
0.5367
0.8096
0.9461
1.0280
1.0826
10
0.5603
0.8200
0.9499
1.0278
1.0798
15
0.6050
0.8395
0.9567
1.0271
1.0740
20
0.6371
0.8533
0.9614
1.0262
1.0695
FMMSE
0
0.7254
0.8680
0.9394
0.9821
1.0107
1
0.7391
0.8747
0.9425
0.9832
1.0104
2
0.7503
0.8802
0.9451
0.9841
1.0101
4
0.7680
0.8888
0.9492
0.9855
1.0096
6
0.7816
0.8954
0.9523
0.9865
1.0092
8
0.7926
0.9008
0.9548
0.9873
1.0089
10
0.8019
0.9052
0.9569
0.9879
1.0086
15
0.8197
0.9139
0.9609
0.9892
1.0080
20
0.8329
0.9202
0.9639
0.9901
1.0076
AFMMSE
0
0.3357
0.7042
0.8885
0.9991
1.0728
1
0.3641
0.7186
0.8959
1.0022
1.0731
2
0.3885
0.7308
0.9020
1.0047
1.0732
4
0.4285
0.7508
0.9119
1.0086
1.0730
6
0.4605
0.7666
0.9196
1.0114
1.0726
8
0.4871
0.7795
0.9258
1.0135
1.0720
10
0.5096
0.7905
0.9309
1.0152
1.0713
15
0.5541
0.8119
0.9407
1.0180
1.0696
20
0.5875
0.8277
0.9478
1.0198
1.0679Table 2
Relative risk of the SR, PSR, FMMSE, and AFMMSE estimators forn=20, k=5 and α=20.
θ
1
θ
0
0.25
0.5
0.75
1
SR
0
0.4705
0.8234
0.9999
1.1058
1.1764
1
0.5537
0.8512
1.0000
1.0892
1.1487
2
0.6176
0.8725
1.0000
1.0765
1.1274
4
0.7074
0.9025
1.0000
1.0585
1.0975
6
0.7662
0.9221
1.0000
1.0468
1.0779
8
0.8068
0.9356
1.0000
1.0386
1.0644
10
0.8362
0.9454
1.0000
1.0328
1.0546
15
0.8823
0.9608
1.0000
1.0235
1.0392
20
0.9087
0.9696
1.0000
1.0183
1.0304
PSR
0
0.3533
0.7255
0.9116
1.0233
1.0977
1
0.4568
0.7743
0.9331
1.0283
1.0918
2
0.5386
0.8122
0.9490
1.0311
1.0858
4
0.6560
0.8653
0.9699
1.0327
1.0745
6
0.7329
0.8988
0.9818
1.0316
1.0648
8
0.7852
0.9209
0.9888
1.0295
1.0566
10
0.8220
0.9359
0.9929
1.0271
1.0499
15
0.8771
0.9574
0.9976
1.0216
1.0377
20
0.9066
0.9682
0.9991
1.0176
1.0299
FMMSE
0
0.7254
0.8680
0.9394
0.9821
1.0107
1
0.7621
0.8860
0.9479
0.9851
1.0099
2
0.7914
0.9003
0.9547
0.9874
1.0091
4
0.8348
0.9213
0.9646
0.9906
1.0079
6
0.8648
0.9359
0.9714
0.9927
1.0069
8
0.8864
0.9463
0.9762
0.9942
1.0062
10
0.9025
0.9540
0.9798
0.9952
1.0055
15
0.9288
0.9666
0.9855
0.9968
1.0044
20
0.9444
0.9740
0.9888
0.9977
1.0036
AFMMSE
0
0.3357
0.7042
0.8885
0.9991
1.0728
1
0.4122
0.7429
0.9083
1.0075
1.0736
2
0.4767
0.7752
0.9245
1.0140
1.0737
4
0.5783
0.8255
0.9492
1.0234
1.0728
6
0.6535
0.8622
0.9666
1.0292
1.0710
8
0.7105
0.8896
0.9791
1.0328
1.0686
10
0.7548
0.9105
0.9883
1.0350
1.0661
15
0.8297
0.9447
1.0023
1.0368
1.0598
20
0.8748
0.9645
1.0093
1.0361
1.0541In sum, our results show that when the loss function and error terms are extended from the usual quadratic loss function and normal distribution to balanced loss function and multivariatet distribution, the dominance of the PSR estimator over the SR estimator is robust. However, the dominance of these estimators over the OLS estimator is not robust.


## 4. Proof of Main Results

Proof of Lemma1.
For givenτ, u1~χk2(λ1) and u2~χn-k2; meanwhile, u1 and u2 are mutually independent. Therefore, we have
(17)H(p,q,γ,c)=E[I(vku1u2≥c)(u1+γu2u1)pu1q∣τ]=∑i=0∞‍wi(λ1)(1/2)n/2+iΓ(k/2+i)Γ(v/2)×∬R‍(u1+γu2)pu1k/2+i+q-p-1u2v/2-1×exp⁡(-u1+u22)du1du2,
where R is the region such that (v/k)(u1/u2)≥c.
Making use of the change of variables,v1=u1/u2, v2=u2, the integral in (17) reduces to
(18)∫kc/v∞‍∫0∞‍v1k/2+i+q-p-1v2n/2+i+q-1(v1+γ)p×exp⁡(-v2(1+v1)2)dv2dv1.
Again, making use of the change of variables, z=v2(1+v1)/2, v1=v1, the integral in (18) becomes
(19)2n/2+i+qΓ(n2+i+q)×∫kc/v∞‍v1k/2+i+q-p-1(v1+γ)p(1+v1)-(n/2+i+q)dv1.
Further, making use of the change of a variable, t=v1/(1+v1), the integral in (19) reduces to
(20)∫kc/(kc+v)1‍tk/2+i+q-p-1[γ+(1-γ)t]p(1-t)v/2-1dt.
By (17)–(20), we have
(21)H(p,q,γ,c)=∑i=0∞‍wi(λ1)Gi(p,q,γ,c).
Next, we derive the formula forJ(p,q,γ,c). Noting that ∂λ1/∂β=2Sβ/τ2 and differentiating H(p,q,γ,c) with respect to β, we have
(22)∂H(p,q,γ,c)∂β=∑i=0∞‍∂wi(λ1)∂βGi(p,q,γ,c)=-Sβτ2H(p,q,γ,c)+Sβτ2∑i=0∞‍wi(λ1)Gi+1(p,q,γ,c).
Sinceu1=b′Sb/τ2 and b~N(β,τ2(X′X)-1), H(p,q,γ,c) can be expressed as
(23)H(p,q,γ,c)=∬R‍(u1+γu2u1)pu1qf1(b)f2(u2)du2db,
where f2(u2) is the density function of u2 and
(24)f1(b)=1(2π)k/2|τ2(X′X)-1|1/2×exp⁡[-(b-β)′X′X(b-β)2τ2].
Differentiating (23) with respect to β, we have
(25)∂H(p,q,γ,c)∂β=∬R‍(u1+γu2u1)pu1qf1(b)f2(u2)Sb-Sβτ2du2db=∬R‍(u1+γu2u1)pu1qf1(b)f2(u2)Sbτ2du2db-Sβτ2H(p,q,γ,c),
which together with (22) yields that
(26)∬R‍(u1+γu2u1)pu1qf1(b)f2(u2)Sbτ2du2db=Sβτ2∑i=0∞‍wi(λ1)Gi+1(p,q,γ,c).
Multiplying β′ from the left of the above, we have
(27)J(p,q,γ,c)=λ1∑i=0∞‍wi(λ1)Gi+1(p,q,γ,c).
This completes the proof of this lemma.Proof of Theorem2.
By Lemma1, we have
(28)Eτ[τ2H(p,q,γ,c)]=∑i=0∞‍Gi(p,q,γ,c)∫0∞‍τ2wi(λ1)PIG(τ∣α,σ)dτ=∑i=0∞‍Gi(p,q,γ,c)i!αα/2η1iΓ(α/2)21-α/2-iσα×∫0∞‍τ-(α-1+2i)exp⁡(-η1+ασ22τ2)dτ,
where η1=β′Sβ. Making use of the change of a variable, t1=(η1+ασ2)/2τ2, (28) becomes
(29)σ22∑i=0∞‍Gi(p,q,γ,c)i!αα/2Γ(α/2)×Γ(α2+i-1)(η1σ2)i(η1σ2+α)-α/2-i+1.
Taking θ1=η1/σ2, (29) becomes
(30)σ22αα/2Γ(α/2)∑i=0∞‍Gi(p,q,γ,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1,
which together with (28) and (29) yields
(31)Eτ[τ2H(p,q,γ,c)]=σ22αα/2Γ(α/2)×∑i=0∞‍Gi(p,q,γ,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1.
In a similar way, we have(32)Eτ[τ2J(p,q,γ,c)]=σ2αα/2Γ(α/2)∑i=0∞‍Gi+1(p,q,γ,c)i!Γ(α/2+i)θ1i+1(θ1+α)α/2+i.
Obviously, we have Eτ(τ2)=σ2(α/(α-2)). This together with (11), (14), (31), and (32) yields the expression of R(β^). The proof of this theorem is completed.Proof of Theorem3.
By Theorem2, let γ=-a,ω=1; we have
(33)R(β^)=β′Sβ+nθσ2αα-2+σ22αα/2Γ(α/2)×∑i=0∞‍Gi(2,1,-a,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1-θσ2αα/2Γ(α/2)∑i=0∞‍Gi(1,1,-a,c)i!Γ(α/2+i-1)θ1i(θ1+α)α/2+i-1-2(1-θ)σ2αα/2Γ(α/2)×∑i=0∞‍Gi+1(1,0,-a,c)i!Γ(α/2+i)θ1i+1(θ1+α)α/2+i.
Since(34)∂∫c*1‍tk/2+i+q-p-1(1-t)v/2-1[γ+(1-γ)t]pdt∂c=-(kckc+v)k/2+i+q-p-1(1-kckc+v)v/2-1×[γ+(1-γ)kckc+v]pkv(kc+v)2,
hence, differentiating (33) with respect to c and performing some manipulations, we have
(35)∂R(β^)∂c=σ2αα/2Γ(α/2)×∑i=0∞‍Γ(n/2+i+1)Γ(α/2+i-1)θ1iΓ(k/2+i)Γ(v/2)(θ1+α)α/2+i-1i!×(kc)k/2+i-1vv/2-1(kc+v)n/2+i-1×[-a+(1+a)kckc+v]×{(α/2+i-1)θ1(k/2+i)(θ1+α)-[-a+(1+a)kckc+v]+2θkckc+v+2(1-θ)kckc+v(α/2+i-1)θ1(k/2+i)(θ1+α)}.
From (35), when γ=-a,ω=1, a condition for R(β^) to be monotonically decreasing is
(36)-a+(1+a)kckc+v≤0.
Thus, R(β^) is monotonically decreasing on c∈[0,av/k] if γ=-a,ω=1. Since β^ becomes the SR estimator when γ=-a,ω=1, and c=0 and it reduces to the PSR estimator when γ=-a, ω=1, and c=av/k, the PSR estimator dominates the SR estimator. This completes the proof.



---

*Source: 129205-2014-05-06.xml*