# New Restricted Liu Estimator in a Partially Linear Model

Authors: Jibo Wu; Yong Li
Journal: Discrete Dynamics in Nature and Society
 (2020)

---

## Abstract

In this paper, we introduce a new restricted Liu estimator in a partially linear model when addition linear constraints are assumed to hold. We also consider the asymptotic normality of the new estimator. Finally, a numerical example and a simulation study are listed to illustrate the performance of the new estimator.


---

## Body

## 1. Introduction

Consider the following partially linear model:(1)yi=xi′β+fti+εi,i=1,…,n,where yi shows a scalar dependent variable, xi=xi1,…,xip′, ti denotes explanatory variables, and β=β1,…,βp′ denotes a vector of unknown parameters, and in this paper, we suppose that f· be an unknown smooth function, ti′s show values of an extra univariate variable such as the time at which the observation is made, and εi be an independent random error with Eεi=0 and Varεi=σ2<∞.The statisticians have mainly discussed how to estimate the parametric component ofβ, and many methods have been proposed to estimate β such as methods by Akdeniz and Duran [1], Akdeniz et al. [2], Duran and Akdeniz [3], Akdeniz et al. [4], Akdeniz et al. [5], Heckman [6], Liang [7], Liu et al. [8], Speckman [9], Wu [10, 11], Wu and Asar [12], Yatchew [13], Yang et al. [14], and Yang and Li [15].As we all know, in regression analysis, the presence of multicollinearity among regressor variables can lead to highly unstable least squares estimates of the regression parameters. To deal with this problem, some biased estimating methods have been proposed, such as Hoerl and Kennard [16], Liu [17], and Xu and Yang [18, 19].The theory and approach on biased estimation were mainly with the linear regression model. How to obtain biased estimator in partially linear models is also important and interesting. In literature studies, some biased estimators have been proposed to estimateβ in partially linear models. Hu [20] introduced a ridge estimator by the parametric component β. Liu et al. [8] introduced a PCR estimator in partially linear models. For more references, one can refer to Roozbeh [21], Roozbeh et al. [22], Akdeniz and Roozbeh [23], Roozbeh et al. [24], Roozbeh and Hanzah [25], and Wei and Wang [26].In practice, we may find that the unknown parameterβ may have some restrictions, such as linear restrictions, stochastic linear restrictions, and other restrictions. Many authors have studied these linear restrictions; namely, Akdeniz and Tabakan [27] introduced the restricted ridge estimator for the parametric component β and Akdeniz and Duran [1] proposed a restricted Liu estimator in a partially linear model.As we all know, there are few papers discussing the asymptotic normality of the estimator ofβ. In this paper, we continue this work in this aspect. First, we will introduce a new restricted Liu estimator for the parametric component β when some addition linear restrictions are assumed to hold, and this estimator is different form the Liu-type estimator which Akdeniz and Duran [1] proposed. Second, we will study the asymptotic normality of the new estimator.The rest of the paper is organized as follows. In Section2, we propose a new restricted Liu estimator for the linear parametric component to deal with multicollinearity, and we will discuss the asymptotic normality of the new estimator in Section 3. A numerical example is given to show the performance of the proposed estimator in Section 4. A simulation study is given to show the performance of the proposed estimator in Section 5, and some conclusion remarks are listed in Section 6.


## 2. New Restricted Liu Estimator

In this section, based on the profile least squares estimator, we will present a new restricted Liu estimator for the unknown parameterβ.To present the new estimator, we first give some assumptions (Gao [28]).Assumption 1.
There exist bounded functionshj· over 0,1, j=1,…,p, so that(2)xij=hjti+uij,i=1,…,n,j=1,…,p,where ui=ui1,…,uij denotes real vectors which fulfill(3)limn⟶∞∑i=1nuiui′n=Σ.Assumption 2.
Let the functionsf· and hs· have the Lipschitz condition of order 1 on 0,1, s=1,…,p. The Lipschitz condition is that with any function fx, there exists a constant k, with any x,y in 0,1, so that fx−fy≤kx−y.Assumption 3.
The positive weight functionWni. fulfills the following:(i)
max1≤i≤n∑j=1nWni=O1(ii)
max1≤i,j≤nWnitj=On−2/3(iii)
max1≤i≤n∑j=1nWnitjIti−tj>cn=OdnHere, I denotes the indicator function, dn appeases limsupn⟶∞ndn3<∞, and cn appeases limsupn⟶∞ncn3<∞.


### 2.1. Profile Least Squares Estimator

In this paper, we letxi′,ti,yi,i=1,…,n satisfy partially linear model (1). When we obtain an estimator of β, then we can obtain an estimator of f·, which is given as follows:(4)f^t,β=∑i=1nWnityi−xi′β.And we also suppose that the positive weight functionsWni· satisfy Assumption 3.By (1) and (4), we obtain the following:(5)y˜=X˜β+ε,where y˜=y1˜,…,yn˜′, X˜=x1˜,…,xn˜′, yi˜=yi−∑jnWnjtiyi, and xi˜=xi−∑jnWnjtixi with i=1,…,n.To estimateβ, one can usually use the profile least squares method to get the profile least squares (PLS) estimator:(6)β^=X′˜,X˜−1X′,˜y˜.


### 2.2. New Restricted Liu Estimator

In order to deal with multicollinearity, Akdeniz and Duran [1] proposed the Liu estimator (LE), which is denoted as follows:(7)β^d=X′˜X˜+Ip−1X′˜y˜+dβ^,0<d<1.Now, we consider the following linear restrictions:(8)Rβ=r,where R denotes a known m×p matrix with rank m<p and r shows a known m×1 vector. For models (1) and (8), restricted profile least squares (RPLS) estimator [1] is denoted by(9)βr^=β^−X′˜X˜−1R′RX′˜X˜−1R′−1Rβ^−r.We are now ready to introduce a new restricted Liu estimator, which is obtained by combining the Liu estimator proposed by Akdeniz and Duran [1] and the restricted profile least squares (RPLS) estimator as follows:(10)βr^d=β^d−X′˜X˜−1R′RX′˜X˜−1R′−1Rβ^d−r,where β^d=X′˜X˜+Ip−1X′˜y˜+dβ^ is the Liu estimator. By (10), we can see that the new restricted Liu estimator is different from the restricted Liu estimator β^dr=β^d−X′˜X˜+Ip−1R′RX′˜X˜+Ip−1R′−1Rβ^d−r proposed by Akdeniz and Duran [1], and so this Liu estimator is a new restricted estimator.From the definition ofβr^d, it is easy to see that βr^1=βr^.In this paper, we mainly discuss how to estimateβ. For the estimator of f, we will discuss it in the next study.


## 2.1. Profile Least Squares Estimator

In this paper, we letxi′,ti,yi,i=1,…,n satisfy partially linear model (1). When we obtain an estimator of β, then we can obtain an estimator of f·, which is given as follows:(4)f^t,β=∑i=1nWnityi−xi′β.And we also suppose that the positive weight functionsWni· satisfy Assumption 3.By (1) and (4), we obtain the following:(5)y˜=X˜β+ε,where y˜=y1˜,…,yn˜′, X˜=x1˜,…,xn˜′, yi˜=yi−∑jnWnjtiyi, and xi˜=xi−∑jnWnjtixi with i=1,…,n.To estimateβ, one can usually use the profile least squares method to get the profile least squares (PLS) estimator:(6)β^=X′˜,X˜−1X′,˜y˜.


## 2.2. New Restricted Liu Estimator

In order to deal with multicollinearity, Akdeniz and Duran [1] proposed the Liu estimator (LE), which is denoted as follows:(7)β^d=X′˜X˜+Ip−1X′˜y˜+dβ^,0<d<1.Now, we consider the following linear restrictions:(8)Rβ=r,where R denotes a known m×p matrix with rank m<p and r shows a known m×1 vector. For models (1) and (8), restricted profile least squares (RPLS) estimator [1] is denoted by(9)βr^=β^−X′˜X˜−1R′RX′˜X˜−1R′−1Rβ^−r.We are now ready to introduce a new restricted Liu estimator, which is obtained by combining the Liu estimator proposed by Akdeniz and Duran [1] and the restricted profile least squares (RPLS) estimator as follows:(10)βr^d=β^d−X′˜X˜−1R′RX′˜X˜−1R′−1Rβ^d−r,where β^d=X′˜X˜+Ip−1X′˜y˜+dβ^ is the Liu estimator. By (10), we can see that the new restricted Liu estimator is different from the restricted Liu estimator β^dr=β^d−X′˜X˜+Ip−1R′RX′˜X˜+Ip−1R′−1Rβ^d−r proposed by Akdeniz and Duran [1], and so this Liu estimator is a new restricted estimator.From the definition ofβr^d, it is easy to see that βr^1=βr^.In this paper, we mainly discuss how to estimateβ. For the estimator of f, we will discuss it in the next study.


## 3. Properties of the New Estimator

To obtain the properties of the new estimator, we first list some lemmas.Lemma 1.
If Assumptions1–3 hold, we have(11)limn⟶∞n−1X˜′X˜=Σ.Proof.
See Gao et al.’s study [28].Lemma 2.
Under Assumptions1–3, PLS estimator β^ is an asymptotic normality of β, i.e.,(12)nβ^−β⟶LN0,σ2Σ−1,where ⟶L denotes convergence in distribution.Proof.
See Gao et al.’s study [28].Theorem 1.
Under Assumptions1–3, Liu estimator β^d is an asymptotic normality of β, i.e.,(13)nβ^d−β⟶LN0,σ2Σ−1Proof.
By (7), we have(14)β^dβ=X˜′X˜+Ip−1X˜′Y˜+dβ^−β=X˜′X˜+Ip−1X˜′X˜+dIpβ^−β+d−1X˜′X˜+Ip−1β.
Then,(15)nβ^d−β=nX˜′X˜+Ip−1X˜′Y˜+dβ^−β=X˜′X˜+Ip−1X˜′X˜+dIpnβ^−β+d−1nX˜′X˜+Ip−1β.
By Lemma1, it is easy to derive that(16)1nX˜′X˜+Ip⟶pΣ,(17)1nX˜′X˜+dIp⟶pΣ,(18)1−dnX˜′X˜+Ipβ=Opn−1/2,where ⟶p shows convergence in probability and Op· denotes the infinitesimal of higher order. Thus, by Lemma 2 and (15)–(18), we obtain(19)X˜′X˜+Ip−1X˜′X˜+dIpΣ−1X˜′X˜+dIpX˜′X˜+Ip−1=Σ−1.
Then, by the Slutsky theorem and (15) and (19), we obtain(20)nβ^d−β⟶LN0,σ2Σ−1.
Now, we present the asymptotic normality of the new restricted Liu estimator.Theorem 2.
Under Assumptions1–3, the new restricted Liu estimator β^rd is an asymptotic normality of β, i.e.,(21)nβ^rd−β⟶LN0,σ2Γ,where Γ=Σ−1−Σ−1R′RΣ−1R′−1RΣ−1.Proof.
By (10), we obtain(22)β^rd−β=β^d−X′˜X˜−1R′RX′˜X˜−1R′−1Rβ^d−r−β=β^d−β−X˜′X˜−1R′RX˜′X˜−1R′−1Rβ^d−Rβ+Rβ−r=β^d−β−X˜′X˜−1R′RX˜′X˜−1R′−1Rβ^d−β−X˜′X˜−1R′RX˜′X˜−1R′−1Rβ−r=β^d−β−X˜′X˜−1R′RX˜′X˜−1R′−1Rβ^d−β=Ip−X˜′X˜−1R′RX˜X˜−1R′−1Rβ^d−β.
By Lemma1 and (22), we obtain(23)X˜′X˜−1R′RX˜′X˜−1R′−1R⟶pΣ−1R′RΣ−1R′−1R.
Thus by (23), we have(24)I−Σ−1R′RΣ−1R′−1RΣ−1I−Σ−1R′RΣ−1R′−1R′=Γ,where Γ=Σ−1−Σ−1R′RΣ−1R′−1RΣ−1. Thus, by (22)–(24) and the Slutsky theorem, we derive(25)nβ^rd−β⟶LN0,σ2Γ.Remark 1.
By the asymptotic covariance matrices ofβ^rd and β^d, we note that Σ−1−Γ=Σ−1R′RΣ−1R′−1RΣ−1 is a positive definite matrix. That is, to mean if the linear restrictions (8) are assumed to hold, the new restricted Liu estimator β^rd is more efficient than the Liu estimator β^d, and this agrees with the practice; thus, the new restricted Liu estimator contains more information for β than the Liu estimator.


## 4. Numerical Example

In this section, we will use a numerical example to illustrate the performance of the new estimator. We consider the hedonic prices of housing attributes. The partially linear model was estimated by Ho [29] using semiparametric least squares. The data consist of 92 detached homes sold during 1987 in the Ottowa area [30].The specification of the partially linear model is(26)yi=β0+β1frplci+β2grgei+β3luxi+β4avginci+β5dhwyi+β6nrbedi+β7usespci+lotareai+εi.The matrixX′X has eigenvalues 3.79,4.58,15.76,18.59,20.99,90.73,and236956.33 and the condition number of X is 250.069, which is large. The Gaussian kernel function(27)Kti−tjh=1h1πexp−ti−tj22h2is used when obtaining the matrices Wti and h is taken to be 0.5. We use the method presented by Wu and Asar [12] to estimate Wti. In these data, we choose Rβ=0, where(28)R=0.50.50−0.5−0.5−0.50.And we use Liu [17] to estimate d. Then, we obtain(29)MSEβ^=423.5498,MSEβ^d=392.1701,MSEβ^r=415.6479,MSEβ^rd=355.226.By the numerical example, we can see that the new estimator has smaller MSE than other estimators.


## 5. Monte Carlo Simulation Study

### 5.1. Design of the Simulation

In this section, we use a Monte Carlo simulation to discuss the performances of the estimators PLS, LE, and RPLS and the new estimator in the sense of MSE criterion. Although the purpose of this paper is to compare the estimators when there is multicollinearity, we use the following equation to obtain the explanatory variables having different degrees of multicollinearity:(30)xij=1−γ21/2zij+γzip,i=1,…,n,j=1,…,p,where zij show independent standard normal pseudo-random numbers. We form the data matrix as X=xij. We discuss three different values of the degree of correlation γ corresponding to 0.9,0.99, and 0.999.In order to obtain then observations of the dependent variable, we first use the following Doppler function:(31)fti=ti1−tisin2.1πti+0.05,where ti=i−0.5/n, i=1,2,…,n, and the sample size n varies between 30, 50, 100, 200, and 400 in this simulation study. After estimating the function f, we use equation (1) to obtain the dependent variables. Finally, we estimate the function f again using equation (4).The Gaussian kernel function(32)Kti−tjh=1h1πexp−ti−tj22h2is used when obtaining the matrices Wti and h is taken to be 0.05. In this paper, we study p=4 and consider the following restriction:(33)R=15−3−1−1−2−10.Moreover, we also consider σ2=5.


### 5.2. Results of the Simulation

The results of the simulation are presented in Tables1–5. From Tables 1–5, we see that an increase in the degree of correlation makes an increase in the simulated MSE values of the estimators in all cases.Table 1
The estimated MSE of the estimators whenn=30.
γβ^β^dβ^rβ^rdγ=0.90.118420.058370.118130.05814γ=0.990.165190.081430.164740.08098γ=0.9990.193990.095630.193470.09501Table 2
The estimated MSE of the estimators whenn=50.
γβ^β^dβ^rβ^dγ=0.90.164200.083960.164000.08346γ=0.990.229110.117120.229100.11615γ=0.9990.260960.137540.269500.13622Table 3
The estimated MSE of the estimators whenn=100.
γβ^β^dβ^rβ^rdγ=0.90.069980.041280.069880.04210γ=0.990.097620.058830.097450.05871γ=0.9990.114600.069090.114420.06891Table 4
The estimated MSE of the estimators whenn=200.
γβ^β^dβ^rβ^rdγ=0.90.037370.022930.037330.02291γ=0.990.052130.031980.052060.03194γ=0.9990.061220.037560.061130.03750Table 5
The estimated MSE of the estimators whenn=400.
γβ^βdβ^rβ^rdγ=0.90.020540.012710.020490.01270γ=0.990.028590.017720.028580.01771γ=0.9990.033580.020810.033560.02080Moreover, the asymptotic behaviours of the estimators are satisfied in all of the cases, namely, an increase in the sample size makes a decrease in the MSE values. And the new estimator performs best compared to other estimators in all cases.


## 5.1. Design of the Simulation

In this section, we use a Monte Carlo simulation to discuss the performances of the estimators PLS, LE, and RPLS and the new estimator in the sense of MSE criterion. Although the purpose of this paper is to compare the estimators when there is multicollinearity, we use the following equation to obtain the explanatory variables having different degrees of multicollinearity:(30)xij=1−γ21/2zij+γzip,i=1,…,n,j=1,…,p,where zij show independent standard normal pseudo-random numbers. We form the data matrix as X=xij. We discuss three different values of the degree of correlation γ corresponding to 0.9,0.99, and 0.999.In order to obtain then observations of the dependent variable, we first use the following Doppler function:(31)fti=ti1−tisin2.1πti+0.05,where ti=i−0.5/n, i=1,2,…,n, and the sample size n varies between 30, 50, 100, 200, and 400 in this simulation study. After estimating the function f, we use equation (1) to obtain the dependent variables. Finally, we estimate the function f again using equation (4).The Gaussian kernel function(32)Kti−tjh=1h1πexp−ti−tj22h2is used when obtaining the matrices Wti and h is taken to be 0.05. In this paper, we study p=4 and consider the following restriction:(33)R=15−3−1−1−2−10.Moreover, we also consider σ2=5.


## 5.2. Results of the Simulation

The results of the simulation are presented in Tables1–5. From Tables 1–5, we see that an increase in the degree of correlation makes an increase in the simulated MSE values of the estimators in all cases.Table 1
The estimated MSE of the estimators whenn=30.
γβ^β^dβ^rβ^rdγ=0.90.118420.058370.118130.05814γ=0.990.165190.081430.164740.08098γ=0.9990.193990.095630.193470.09501Table 2
The estimated MSE of the estimators whenn=50.
γβ^β^dβ^rβ^dγ=0.90.164200.083960.164000.08346γ=0.990.229110.117120.229100.11615γ=0.9990.260960.137540.269500.13622Table 3
The estimated MSE of the estimators whenn=100.
γβ^β^dβ^rβ^rdγ=0.90.069980.041280.069880.04210γ=0.990.097620.058830.097450.05871γ=0.9990.114600.069090.114420.06891Table 4
The estimated MSE of the estimators whenn=200.
γβ^β^dβ^rβ^rdγ=0.90.037370.022930.037330.02291γ=0.990.052130.031980.052060.03194γ=0.9990.061220.037560.061130.03750Table 5
The estimated MSE of the estimators whenn=400.
γβ^βdβ^rβ^rdγ=0.90.020540.012710.020490.01270γ=0.990.028590.017720.028580.01771γ=0.9990.033580.020810.033560.02080Moreover, the asymptotic behaviours of the estimators are satisfied in all of the cases, namely, an increase in the sample size makes a decrease in the MSE values. And the new estimator performs best compared to other estimators in all cases.


## 6. Conclusions

In this paper, we propose a new restricted Liu estimator when some additional linear restrictions are assumed to hold on the linear parametric component. And we study the properties of the proposed estimator. Finally, a data example and a simulation study are given to show the performance of these estimators.



---

*Source: 4690562-2020-08-18.xml*